WO2022222096A1 - 手绘图形识别方法、装置和系统，以及计算机可读存储介质 - Google Patents

手绘图形识别方法、装置和系统，以及计算机可读存储介质 Download PDF

Info

Publication number
WO2022222096A1
WO2022222096A1 PCT/CN2021/088951 CN2021088951W WO2022222096A1 WO 2022222096 A1 WO2022222096 A1 WO 2022222096A1 CN 2021088951 W CN2021088951 W CN 2021088951W WO 2022222096 A1 WO2022222096 A1 WO 2022222096A1
Authority
WO
WIPO (PCT)
Prior art keywords
stroke
hand
drawn
strokes
closed
Prior art date
Application number
PCT/CN2021/088951
Other languages
English (en)
French (fr)
Inventor
郑瑞
Original Assignee
京东方科技集团股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 京东方科技集团股份有限公司 filed Critical 京东方科技集团股份有限公司
Priority to PCT/CN2021/088951 priority Critical patent/WO2022222096A1/zh
Priority to US17/636,629 priority patent/US20230351787A1/en
Priority to CN202180000861.8A priority patent/CN115589786A/zh
Publication of WO2022222096A1 publication Critical patent/WO2022222096A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/32—Digital ink
    • G06V30/333—Preprocessing; Feature extraction
    • G06V30/347—Sampling; Contour coding; Stroke extraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/20—Drawing from basic elements, e.g. lines or circles
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/32—Digital ink
    • G06V30/36—Matching; Classification

Definitions

  • the present disclosure relates to the field of computer technology, and in particular, to a method, device and system for recognizing hand-drawn graphics, and a computer-readable storage medium.
  • hand-drawn input is widely used due to its advantages of simplicity and efficiency.
  • Hand-drawn recognition facilitates natural human-computer interaction.
  • touch screen provides users with a better hand-drawn input mode, allowing users to input a variety of graphics online.
  • the related hand-drawn pattern recognition technology is mainly based on image recognition, and outputs standard figures corresponding to hand-drawn figures, such as circles, squares, rectangles, isosceles triangles and other symmetrical figures, which cannot effectively meet the needs of more complex scenes.
  • a method for recognizing hand-drawn graphics including:
  • the input includes a set of strokes, the set of strokes includes at least one stroke, each stroke includes an information sequence of multiple track points, and the information of each track point includes the coordinates and writing time of the track point;
  • the type of the closed figure is identified based on the coordinate data of the strokes.
  • the stroke set includes a plurality of strokes, each stroke includes two endpoints, and the information of each track point further includes a writing state value of the track point, wherein the writing state value is used to represent the value of the track point.
  • Writing state, the writing state of the trajectory point includes pen up and pen down, and according to the coordinate data of the input stroke, determining whether the stroke set constitutes a closed figure includes:
  • a connection relationship is formed between the endpoint of one stroke and the other stroke, according to the endpoint of the current stroke and the connection relationship between the previous strokes to determine whether the stroke set constitutes a closed figure includes:
  • the previous strokes constitute a connection relationship but do not constitute a closed figure.
  • any endpoint of the current stroke does not form a connection relationship with the previous stroke, it is determined that the current stroke does not form a connection relationship with the previous stroke and the stroke set does not form a closure, and no graphic recognition is performed .
  • one endpoint of the current stroke forms a connection relationship with one of the previous strokes, but the other endpoint of the current stroke does not constitute a connection relationship with other strokes, it is determined that the current stroke and the previous stroke are in a connection relationship. A connection relationship is formed between them, but the stroke set does not form a closed graph, and it is determined whether the j+1th stroke is received within the range of the acquisition time threshold;
  • the j+1st stroke is taken as the current stroke of the stroke set, and according to the connection relationship between the endpoint of the current stroke and the previous stroke, it is determined whether the stroke set is form a closed figure;
  • the hand-drawn pattern recognition method further includes storing stroke endpoint values in a storage area, wherein:
  • Two endpoint values are stored in the storage area, and the two endpoint values include the first endpoint value and the second endpoint value;
  • the two endpoint values of the first stroke are respectively the first endpoint value and the second endpoint value
  • the jth stroke and the previous stroke form a connection relationship but do not form a closed figure
  • the endpoint value of the connection In the case that the jth stroke and the previous stroke form a connection relationship but do not form a closed figure, use the endpoint value in the jth stroke that is not connected to the previous stroke in the storage area to replace the jth stroke in the previous stroke with the jth stroke.
  • the stroke set includes a plurality of consecutive strokes, each stroke includes two endpoints, and in the plurality of consecutive strokes, the interval between the pen down time of each stroke and the pen up time of the previous stroke is less than
  • the collection time threshold, according to the coordinate data of the input strokes, to determine whether the stroke set constitutes a closed figure includes:
  • the strokes in the multiple continuous strokes are regarded as the strokes to be processed, and according to the connection relationship between the endpoints of the strokes to be processed and the previous strokes, it is determined whether the set of strokes constitutes a closed figure, and the strokes to be processed are a set of strokes
  • the jth stroke in , j is a positive integer greater than 1.
  • connection relationship is formed between the end point of one stroke and the other stroke, according to the end point of the stroke to be processed
  • the stroke set constitutes a closed graph, m, n are positive integers and m is greater than or equal to n;
  • any endpoint of the stroke to be processed does not form a connection relationship with the previous stroke, it is determined that the stroke to be processed does not form a connection relationship with the previous stroke, and the set of strokes does not form a closed graph;
  • the previous strokes constitute a connection relationship but do not constitute a closed figure.
  • j+1 strokes are used as the strokes to be processed in the stroke set, according to the strokes to be processed.
  • the method for recognizing hand-drawn graphics further includes: for the first stroke in the stroke set, identifying the category of the graphics formed by the strokes, and when the identified category is a specified graphics, executing a preset for the specified graphics handwriting processing operations.
  • the identified category is not a specified graphic, it is determined whether the stroke constitutes a closed graphic; when it constitutes a closed graphic, the category of the closed graphic is identified; when it does not constitute a closed graphic, it is determined to be within the collection time threshold range Whether the second stroke is received; if the second stroke is received within the collection time threshold range, the second stroke is taken as the current stroke, and it is judged whether the second stroke and the first stroke constitute a closed figure.
  • the handwriting processing operation includes: erasing, undoing, marking, or highlighting.
  • the specified graphic includes an inverted N shape, a polyline, or a "Z" shape.
  • the erasing operation is performed if the designated pattern is a "zigzag" shape.
  • the hand-drawn figure recognition method further includes: outputting a corrected drawing figure corresponding to the hand-drawn figure according to the category of the closed figure.
  • the hand-drawn pattern recognition method further includes:
  • the key points are corner points.
  • the hand-drawn pattern recognition method further includes:
  • identifying the class of the closed figure includes using a deep learning algorithm to identify the class of the closed figure.
  • the deep learning algorithm includes a recurrent neural network.
  • the categories of graphics include at least one of polygons, circles, and ellipses.
  • the hand-drawn pattern recognition method further includes:
  • the set of trajectory points is sparsely sampled according to the number of trajectory points required for class identification of the closed graph.
  • the terminal side determines whether the set of strokes constitutes a closed figure according to the coordinate data of the input strokes, and if the set of strokes constitutes a closed figure, sends the coordinate data of the corresponding strokes to the cloud; the cloud side based on the strokes The coordinate data of , identify the category of closed graphics.
  • a method for recognizing hand-drawn graphics including:
  • the input includes a set of strokes, the set of strokes includes at least one stroke, each stroke includes an information sequence of multiple track points, and the information of each track point includes the coordinates and writing time of the track point;
  • locating key points of hand-drawn graphics includes:
  • locating the key points of the hand-drawn graphics includes:
  • the coordinate offset includes the abscissa offset and the ordinate offset
  • the coordinate offset is divided into multiple intervals
  • the key points for locating the hand-drawn graphics include:
  • locating the key points of the hand-drawn graphics further includes:
  • the key points of the hand-drawn graphics are located based on the trajectory points P i+n corresponding to the adjacent coordinate offsets On and On-1 .
  • locating the key points of the hand-drawn graphics further includes:
  • the first included angle is not greater than the first angle threshold, or when the adjacent coordinate offsets On and On -1 belong to the same interval, if the difference between the offsets Onk and On -1 is the same The offset between them is the same as the interval to which On-1 belongs, but the interval to which the offset Onk and On-1 belong is different, and the trajectory point P i+n-k+ corresponding to the offset On-k+1 is determined.
  • the key points of the hand-drawn graphics are located based on the trajectory points P i+n corresponding to the adjacent coordinate offsets On and On-1 .
  • locating the key points of the hand-drawn graphics further includes:
  • the key points of the hand-drawn graphics are located based on the trajectory points P i+n corresponding to the adjacent coordinate offsets On and On-1 .
  • locating the key points of the hand-drawn graphics further includes:
  • the relationship between the intervals to which the coordinate offsets belong is determined.
  • the key points are corner points.
  • the hand-drawn pattern recognition method further includes:
  • the set of track points is sparsely sampled according to the number of track points required for locating key points of the hand-drawn graphics.
  • a hand-drawn pattern recognition device comprising:
  • a processor coupled to the memory, the processor configured to perform one or more steps of the hand-drawn pattern recognition method of any of the preceding embodiments based on instructions stored in the memory device.
  • a hand-drawn pattern recognition system comprising:
  • a touch screen configured to receive input of hand-drawn graphics, and generate an information sequence including a plurality of trajectory points when a user draws a graphic on the touch screen;
  • the hand-drawn pattern recognition device includes:
  • a first processor located at the terminal, configured to determine whether the set of strokes constitutes a closed figure according to the coordinate data of the input strokes;
  • the second processor located in the cloud, is configured to identify the category of the closed graphics based on the coordinate data of the strokes, or to locate key points of the hand-drawn graphics when the set of strokes constitutes a closed graphics.
  • a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements one or more of the hand-drawn pattern recognition methods described in any of the foregoing embodiments step.
  • FIG. 1 is a flowchart illustrating a method for recognizing hand-drawn graphics according to an embodiment of the present disclosure
  • FIG. 1A is a schematic diagram illustrating a hand-drawn graphic according to one embodiment of the present disclosure
  • FIG. 2 is a flowchart illustrating a method for recognizing hand-drawn graphics according to another embodiment of the present disclosure
  • FIG. 2A is a flowchart illustrating locating key points of hand-drawn graphics according to one embodiment of the present disclosure
  • 2B is a schematic diagram illustrating interval division according to an embodiment of the present disclosure
  • FIG. 3 is a flowchart illustrating a method for recognizing hand-drawn graphics according to yet another embodiment of the present disclosure
  • FIG. 4A is a flowchart illustrating a method for recognizing hand-drawn graphics according to still another embodiment of the present disclosure
  • 4B is a block diagram illustrating a hand-drawn pattern recognition device according to an embodiment of the present disclosure
  • FIG. 5 is a block diagram illustrating a hand-drawn pattern recognition system according to one embodiment of the present disclosure
  • FIG. 6 is a block diagram illustrating a computer system for implementing some embodiments of the present disclosure.
  • the present disclosure proposes a new hand-drawn graphics recognition scheme.
  • the present disclosure proposes to obtain the coordinate information of the trajectory points in real time through the hand-drawn trajectory, for example, during the user's drawing process, to determine whether the user has completed the drawing of the graphics and what target graphics the user is drawing.
  • it is possible to further determine the specific category of the target graphic whether the graphic is a regular regular polygon, a circle, or an inclined (eg, scalene) polygon.
  • key points (such as inflection points or corner points) of the target graphic may also be located based on the track point information. Using this information, the geometric figures that meet the user's needs can be drawn to achieve better human-computer interaction effects.
  • FIG. 1 is a flowchart illustrating a method for recognizing hand-drawn graphics according to an embodiment of the present disclosure.
  • the hand-drawn graphic recognition method includes: step S1, receiving the input of the hand-drawn graphic; step S3, according to the coordinate data of the input strokes, determine whether the stroke set constitutes a closed graphic; step S5, in the stroke set constitutes a closed graphic In this case, the class of the closed figure is identified based on the coordinate data of the stroke.
  • a stroke set is input, the stroke set includes at least one stroke, and each stroke includes an information sequence of a plurality of trajectory points.
  • the information of each track point includes the coordinates and writing time of the track point.
  • the information of each track point also includes the writing state value of the track point.
  • the writing state value is used to represent the writing state of the track point, and the writing state of the track point includes the pen up and the pen down.
  • the writing state value may be a first value representing a pen up of a stroke, or a second value representing a pen down.
  • the writing state value of the end track point of a stroke is a first value (eg, 1) representing the uplift
  • the writing state value of the starting track point of a stroke is the second value (eg, 0) representing the pen down.
  • the writing state value may also include a third value representing writing or running of the pen. That is, the track points can be divided into corresponding strokes according to the writing state value of each track point.
  • an information sequence including a plurality of trajectory points is generated when a graphic is hand-drawn on the touch screen. For example, when writing on a touch screen such as an electromagnetic screen or a capacitive screen, the sequence of hand-drawn track point information stored in chronological order is collected.
  • terminals such as all-in-one conference machines, mobile phones, and smart ink tablets use software to obtain the sequence information of hand-drawn track points generated when drawing on the touch screen, and the hand-drawn track points can be assigned according to the actual needs of the business.
  • the sequence information is transmitted to the cloud server or embedded terminal.
  • the object to be recognized must be located first. For example, it is determined which strokes constitute a closed figure, and then each closed figure is identified based on these strokes. In step S3, based on the coordinates of each track point included in the stroke, it is determined whether the set of strokes constitutes a closed figure.
  • the stroke set may include one stroke or multiple strokes. Each stroke includes two endpoints. It should be understood that a closed figure is not necessarily strictly closed. A detailed description will be given below with reference to FIG. 1A .
  • FIG. 1A is a schematic diagram illustrating a hand-drawn graphic according to one embodiment of the present disclosure.
  • determining whether the stroke forms a closed figure includes: determining whether the distance between the down and up positions of the stroke is less than a distance threshold. That is to say, in the case where the distance between the pen-down position where a stroke starts handwriting and the pen-up position where the stroke ends handwriting is smaller than the distance threshold, as shown in (a) in FIG. 1A , it can be considered that the stroke constitutes a closed graph . However, as shown in (b) of FIG. 1A , the distance between the pen-down position and the pen-up position of a stroke is relatively large, and it is considered that this stroke does not constitute a closed figure.
  • the handwriting processing is preset for the specified graphic.
  • the handwriting processing includes: erasing, undoing, marking, or highlighting and the like. For example, in the case where the designated pattern is a "Z" shape, an erasing operation is performed.
  • Using specified graphics to perform preset handwriting processing can provide a faster operating experience.
  • the identified category is not a specified graphic
  • determine whether the stroke constitutes a closed graphic when it constitutes a closed graphic, identify the category of the closed graphic; when it does not constitute a closed graphic, determine whether the second stroke is received within the collection time threshold range. Stroke; when the second stroke is received within the collection time threshold range, the second stroke is taken as the current stroke, and it is judged whether the second stroke and the first stroke constitute a closed figure. Subsequently, further processing may be performed according to a situation in which the stroke set includes multiple strokes.
  • the distance threshold is a value dynamically determined according to the coordinates of the track points, so it can be adaptively adjusted according to the characteristics of the hand-drawn graphics.
  • the distance threshold can also be set as a fixed value according to actual needs.
  • determining whether the stroke set constitutes a closed graph includes: determining the time between when a stroke is down and when a previous stroke is up. Whether the interval is less than the collection time threshold; if it is smaller than the collection time interval, it is determined whether the endpoint of a stroke forms a connection relationship with the previous stroke. If the distance between the end point of one stroke and any end point of another stroke is less than the distance threshold, the end point of one stroke and the other stroke form a connection relationship.
  • Strokes that do not form a closed figure are output as is.
  • the length of the stroke when the length of the stroke is abnormal, for example, the length of the stroke exceeds the preset length range, that is, the length of the stroke is very long or very short, it can be regarded as abnormal data, and no graphing is performed.
  • Recognition for example, the strokes can be output as is, or nothing is displayed. That is, the method for recognizing hand-drawn graphics further includes: determining whether the length of the stroke is within a preset length range according to the coordinate data of the input stroke. When the length of the stroke is within the preset length range, the stroke is used to identify the closed figure; when the length of the stroke exceeds the preset length range, the graphic recognition is not performed. Determine whether the stroke length is abnormal, either before or after determining whether the continuous strokes constitute a closed figure. Through such setting, the efficiency of pattern recognition can be improved.
  • the above-mentioned step of determining whether a set of strokes constitutes a closed figure can be based on the relationship between the processing speed of the processor and the input speed of the hand-drawn figure, and can choose to judge in real time for each received current stroke, or perform a centralized process for a plurality of continuous strokes. judge.
  • the two situations will be described in detail below with reference to different embodiments.
  • the judgment is made in real time for each received current stroke, that is, the connection relationship between the endpoint of the current stroke and the previous stroke is judged, and then it is determined whether the stroke set constitutes a closed graph.
  • determining whether the stroke set constitutes a closed graph includes: judging whether the received interval between the pen-down of the current stroke and the pen-lift time of the previous stroke is less than the collection time threshold; in the case of less than the collection time threshold , take the current stroke as the jth stroke of the stroke set, and determine whether the stroke set constitutes a closed figure according to the connection relationship between the endpoint of the current stroke and the previous stroke, where j is a positive integer greater than 1.
  • the previous strokes form a connection relationship but do not form a closed graph, for example, determine whether the interval between the stroke of the jth stroke and the stroke time of the j-1th stroke in the stroke set is less than the collection time threshold; If it is less than the collection time threshold, the jth stroke is added to the stroke set, and according to the connection relationship between the jth stroke's endpoint and the 1st to j-1th strokes, it is determined whether the 1st to jth strokes constitute a closed figure.
  • the previous strokes form a connection relationship but do not form a closed figure, that is, the j-1th stroke and its previous stroke form a connection relationship but do not form a closed figure.
  • any endpoint of the current stroke does not form a connection relationship with the previous stroke, it is determined that the current stroke and the previous stroke do not form a connection relationship and the stroke set does not form a closed figure.
  • one endpoint of the current stroke forms a connection relationship with one of the previous strokes, but the other endpoint of the current stroke does not constitute a connection relationship with other strokes, it is determined that the stroke set constitutes a connection relationship but does not constitute a closed graph, and does not Perform pattern recognition.
  • the stroke set constitutes a connection relationship but does not constitute a closed graph
  • determine whether the j+1th stroke is received within the collection threshold time range if the j+1th stroke is received within the collection time threshold range, Take j+1 strokes as the current strokes of the stroke set, and determine whether the stroke set constitutes a closed graph according to the connection relationship between the endpoints of the current stroke and the previous strokes; the j+1th stroke is not received within the collection time threshold range
  • pattern recognition is not performed, for example, a set of strokes is output as it is.
  • any endpoint of the third stroke does not form a connection relationship with the first and second strokes, it is determined that the third stroke and the previous stroke do not form a connection relationship nor a closed figure. In this case, the 1st to 3rd strokes are output as they are, and the stroke count can be cleared to zero.
  • the fourth stroke is received within the collection threshold time range.
  • the fourth stroke is taken as the current stroke, and the aforementioned judgment is performed again, that is, according to the connection relationship between the endpoint of the current stroke and the previous stroke, it is determined whether the stroke set constitutes a closed figure.
  • the 1st to 3rd strokes are output as they are. In the case of the original output, the stroke count can be cleared to facilitate the judgment of the relationship between the subsequent strokes.
  • the hand-drawn pattern recognition method further includes storing stroke endpoint values in a storage area. Two endpoint values are stored in the storage area, and the two endpoint values include a first endpoint value and a second endpoint value.
  • the two endpoint values of the first stroke are the first endpoint value and the second endpoint value, respectively.
  • the second stroke according to the relationship between the endpoint value of the second stroke and the first endpoint value and the second endpoint value in the storage area, determine the connection relationship between the second stroke and the first stroke, In the case where the second stroke and the first stroke form a connection relationship but do not form a closed figure, replace the end point value in the second stroke that is not connected with the first stroke from the storage area to replace the connection between the first stroke and the first stroke from the storage area.
  • the endpoint value of the connection of the 2 strokes are the first endpoint value and the second endpoint value, respectively.
  • the jth stroke according to the relationship between the endpoint value of the jth stroke and the first endpoint value and the second endpoint value in the storage area, determine the connection relationship between the jth stroke and the previous stroke ; In the case where the jth stroke and the previous stroke form a connection relationship but do not form a closed figure, replace the jth stroke with the jth stroke in the storage area with the endpoint value that is not connected to the previous stroke in the storage area. The endpoint value of the stroke connection.
  • the judgment is performed on a set of multiple continuous strokes, that is, the connection relationship between the endpoint of a certain stroke (such as the jth stroke) in the current multiple continuous strokes and the previous stroke is judged, and then the stroke is determined.
  • the collection forms a closed graph. That is, according to the time sequence, the strokes in a plurality of continuous strokes are sequentially regarded as the strokes to be processed, and according to the connection relationship between the endpoints of the strokes to be processed and the previous strokes, it is determined whether the set of strokes constitutes a closed figure, and the strokes to be processed are: The jth stroke in the stroke set.
  • determining whether the set of strokes constitutes a closed graph includes: forming a connection relationship with the j-mth stroke at one end point of the stroke to be processed, and forming a connection relationship with the j-nth stroke at the other end point of the stroke to be processed Under the circumstance, it is determined that the stroke set constitutes a closed figure, m and n are positive integers and m is greater than or equal to n; in the case that any endpoint of the stroke to be processed does not form a connection relationship with the previous stroke, it is determined that the stroke to be processed and the previous stroke are not connected.
  • the strokes do not form a connection relationship and the set of strokes does not form a closed figure; an end point of the stroke to be processed forms a connection relationship with one of the previous strokes, but the other end point of the stroke to be processed does not form a stroke with other strokes.
  • a connection relationship it is determined that a connection relationship is formed between the to-be-processed stroke and the previous stroke, but the stroke set does not constitute a closed figure.
  • the previous strokes constitute a connection relationship but do not constitute a closed figure.
  • j+1 strokes are taken as the strokes to be processed in the stroke set, according to the relationship between the endpoints of the strokes to be processed and the previous strokes. If it is determined that the stroke set to be processed does not form a connection relationship with the previous stroke and the stroke set does not form a closed figure, no graphic recognition is performed, for example, the stroke set is output as it is. In the case of the original output, the stroke count can still be cleared to facilitate the judgment of the relationship between the subsequent strokes.
  • any endpoint of the fourth stroke does not constitute a connection relationship with the first to third strokes, it is determined that the fourth stroke and the previous stroke do not constitute a connection relationship nor a closed figure. In this case, pattern recognition is not performed, and, for example, the first to fourth strokes may be output as they are. In addition, the stroke count can be cleared to zero, and the fifth stroke can be used as the first stroke to continue to perform corresponding processing.
  • the fourth stroke In the case where one end point of the fourth stroke forms a connection relationship with the second stroke, but the other end point does not form a connection relationship with the first stroke and the third stroke, it is determined that the fourth stroke is connected with the previous stroke. Connecting relationships but not forming a closed graph.
  • the fifth stroke is further regarded as the to-be-processed stroke, and the aforementioned judgment is performed again, that is, according to the connection relationship between the endpoints of the to-be-processed stroke and the previous stroke, it is determined whether the stroke set constitutes a closed figure.
  • a deep learning algorithm may be used to identify the category of the closed figure according to the coordinate data of the trajectory points included in the stroke.
  • the categories of closed figures may include various standard figures or non-standard figures, or may be called regular figures or irregular figures, or may also be called symmetrical figures or asymmetric figures.
  • the categories of graphics may include at least one of polygons, circles, and ellipses.
  • Polygons may include, for example, symmetrical figures such as rectangles, isosceles triangles, isosceles trapezoids, and parallelograms, and may also include asymmetric figures such as non-isosceles triangles and non-isosceles trapezoids. It should be understood that all closed graphs satisfy the basic characteristics of graphs. For example, for the strokes forming a circle, the coordinates of each track point must meet certain conditions, and there are at least four offsets between track points in four directions.
  • a corrected drawing figure corresponding to the hand-drawn figure may be output according to the type of the closed figure.
  • the above-mentioned correction drawing graphic is a graphic after necessary correction is made to the hand-drawn graphic.
  • the hand-drawn line is corrected as a straight line (for example, corresponding to a polygon) or a curve (such as corresponding to a circle or an ellipse).
  • classes of closed figures are identified using a Recurrent Neural Network (RNN).
  • RNN Recurrent Neural Network
  • a bidirectional long short-term memory (BiLSTM) network can be used, combined with a linear layer (Linear) to perform graphic category recognition, and a comprehensive and accurate recognition result can be obtained.
  • BiLSTM bidirectional long short-term memory
  • Linear linear layer
  • the number of layers of the LSTM network can be determined according to the difficulty and time-consuming of the task, and one or more layers can be used.
  • the role of the Linear layer is to linearly transform the output to the specified dimension. More complex RNN networks can also be used depending on the accuracy requirements.
  • FIG. 2 is a flowchart illustrating a method for recognizing hand-drawn graphics according to another embodiment of the present disclosure.
  • the difference between FIG. 2 and FIG. 1 is that the hand-drawn pattern recognition method in FIG. 2 further includes steps S7 and S9. Only the differences between FIG. 2 and FIG. 1 will be described below, and the similarities will not be repeated.
  • step S7 the key points of the hand-drawn graphics are located.
  • the key points of the hand-drawn graphics are located based on the coordinate offsets between the trajectory points.
  • a keypoint may also be referred to as an inflection or corner of the graph, indicating that a significant change in slope occurs at the point of the trajectory.
  • the intersections of the edges are inflection points or corners.
  • the process of locating the key points of the hand-drawn graphics includes steps S71-S73.
  • step S71 coordinate offsets between track points are calculated.
  • the coordinate offset includes the abscissa offset and the ordinate offset.
  • the coordinate offset between the trajectory points P i+n+1 and P i+n be O n
  • the coordinate offset between the track points P i+n and P i+n-1 is O n -1
  • the coordinate offsets On and On-1 are adjacent coordinate offsets.
  • step S72 the coordinate offset is divided into a plurality of sections according to the magnitude of the abscissa offset and the ordinate offset between the track points.
  • the abscissa offset is recorded as dx
  • the ordinate offset is recorded as dy. That is , On can be expressed as (dx n , dy n ).
  • the coordinate offset is divided into multiple intervals. According to the recognition difficulty of hand-drawn graphics, determine the number of divided sections.
  • the abscissa offset and ordinate offset between the track points according to the interval to which the coordinate offset belongs; and determine the coordinates according to the reassignment.
  • the relationship between the intervals to which the offset belongs. Reassignment is convenient to identify the interval to which the coordinate offset belongs, and can directly reflect the trend of the input trajectory point. Locating the key points of the graph based on the coordinate offset is simpler and less computationally expensive than locating the key points of the graph based on the slope.
  • Table 1 shows the relationship between the interval division result, the reassignment result and the coordinate offset.
  • Table 1 The relationship between interval division results and coordinate offsets, and the results of reassignment
  • the coordinate offset of 0 ⁇ slope dy/dx ⁇ 1 is divided into the first interval, and -1 ⁇ slope dy/dx ⁇ 0
  • the coordinate offset is divided into the third interval, and the coordinate offset with dy/dx greater than 1 or less than -1 is divided into the second interval; if dy ⁇ 0, the coordinate offset with -1 ⁇ slope dy/dx ⁇ 0 is divided into the second interval.
  • the offset is divided into the first interval, the coordinate offset with 0 ⁇ slope dy/dx ⁇ 1 is divided into the third interval, and the coordinate offset with slope dy/dx greater than 1 or less than -1 is divided into the fourth interval .
  • the result of the reassignment is that dy is equal to 0 and dx is equal to the original value; for the second interval and the fourth interval, that is, the slope of the The absolute value of
  • FIG. 2B shows a schematic diagram of interval division according to some embodiments of the present disclosure.
  • the 360-degree interval can be divided into 4 intervals, and the absolute value of the slope of the segmentation is 1, which intuitively shows the result of interval division according to Table 1. It should be understood that if the hand-drawn graphic is complicated, more intervals, for example, 8 intervals or more, may be divided for finer calculation.
  • step S73 the key points of the hand-drawn graphics are located according to the relationship between the intervals to which the coordinate offsets belong.
  • Table 2 shows the relationship between the intervals to which adjacent coordinate offsets belong, and the corresponding key point positioning results.
  • Table 2 The relationship between the intervals to which the adjacent coordinate offsets belong, and the corresponding key point positioning results
  • the trajectory point P i+n locates the key point of the hand-drawn graphic, where i is a positive integer and n is a positive integer greater than 1.
  • the trajectory point P i+n corresponding to the offsets On and On-1 can be used as the key point of the hand-drawn graphic.
  • FIG. 2B there is an interval between the first interval and the third interval, and there is no interval between the first interval and the second interval or between the fourth interval.
  • the adjacent coordinate offsets On and On -1 belong to adjacent intervals, it is determined that the line connecting the trajectory points P i+n and P i+n - 1 corresponding to On corresponds to On-1 Whether the first included angle between the line connecting the trajectory points P i+n+1 and P i+n is greater than the first angle threshold; if the first included angle is greater than the first angle threshold, based on the adjacent coordinates
  • the trajectory point P i+n corresponding to the offsets On and On-1 together locates the key points of the hand-drawn graphics.
  • the trajectory point P i+n corresponding to the offsets On and On-1 can be used as the key point of the hand-drawn graphic.
  • the first interval is between the second interval or adjacent to the fourth interval.
  • the first angle threshold is, for example, 30 degrees.
  • the first included angle is not greater than the first angle threshold, or when the adjacent coordinate offsets On and On -1 belong to the same interval, if the difference between the offsets Onk and On -1 is the same The offset between them is the same as the interval to which On-1 belongs, but the interval to which the offset Onk and On-1 belong is different, and the trajectory point P i+n-k+ corresponding to the offset On-k+1 is determined.
  • k is a positive integer greater than 2
  • n is a positive integer greater than 3; or if the offset between O n+k and O n is the same as the interval to which O n belongs, but the offset amount O n +k is different from the interval to which On belongs, determine the trajectory point P i+n+ k-1 corresponding to the offset O n +k-1 or the trajectory point P i + corresponding to P i+n+k and On-1 Whether the second included angle between the connecting line of n and the connecting line of the trajectory points P i+ n -1 and P i+n corresponding to On-1 is greater than the second angle threshold.
  • the key points of the hand-drawn graphics are located based on the trajectory points P i+n corresponding to the offsets On and On-1 in common.
  • the trajectory point P i+n corresponding to the offsets On and On-1 can be used as the key point of the hand-drawn graphic.
  • the second angle threshold may be the same as the first angle threshold, or may be slightly different.
  • the embodiments of the present disclosure fully consider the complex scene of hand-drawn graphics.
  • the angle between the polylines hand-drawn by the user is an arc angle.
  • the method for recognizing hand-drawn graphics further includes: performing sparse sampling on the input trajectory points based on coordinates. After sparse sampling, the category recognition and key point location of the graph can be performed, which can reduce unnecessary computation and improve the speed of graph recognition. In some embodiments, before identifying the category of the closed graphic, or before locating the key points of the hand-drawn graphic, sparse sampling is performed on the trajectory points according to a predetermined distance threshold to obtain the trajectory point set. For example, in the case where the distance between two input trajectory points is smaller than a predetermined threshold, remove any one of the trajectory points to obtain a trajectory point set.
  • a predetermined threshold for example, 20 pixels
  • one of them can be deleted, so that part of the dense track points can be removed.
  • the change threshold for example, 0.1
  • the change threshold for example, 0.1
  • the set of trajectory points may be sparsely sampled according to the number of trajectory points required for the category identification of the closed graph.
  • the set of track points may also be sparsely sampled according to the number of track points required for key point positioning of the hand-drawn graphic.
  • step S9 according to the categories and key points of the closed graphics, output the corrected drawing graphics corresponding to the hand-drawn graphics.
  • the output correction drawing graphics may be regular graphics or irregular graphics. In some embodiments, it may be first determined whether the closed graphics is a regular graphics according to the category of the closed graphics identified in step S3 and combined with the coordinates of the trajectory points.
  • a closed figure is identified as a polygon
  • the length of each side of the polygon and the included angle between adjacent sides can be calculated according to the coordinates of the trajectory points, and then it can be determined whether the closed figure is a regular figure. If the judgment result is a regular graphic, parameters such as center point, width, and height can be determined according to the coordinates of the track point, and a regular graphic corresponding to the hand-drawn graphic can be output. If the judgment result is an irregular figure, the irregular figure corresponding to the hand-drawn figure can be output according to the calculated parameters such as side length and included angle. For example, in the case that the hand-drawn graphic is identified as an irregular triangle, corner points 1, 2, and 3 can be obtained by using key point positioning, and the irregular triangle corresponding to the hand-drawn graphic can be output by connecting the three corner points.
  • FIG. 3 is a flowchart illustrating a method for recognizing hand-drawn graphics according to yet another embodiment of the present disclosure. The difference between FIG. 3 and FIG. 2 is that step S2 is further included. Only the differences between FIG. 3 and FIG. 2 will be described below, and the similarities will not be repeated.
  • step S2 it is determined whether the length of the stroke is within a preset length range according to the input coordinate data of the stroke.
  • the stroke is used to identify the closed figure.
  • the length of the stroke exceeds the preset length range, that is, the length of the stroke is very long or very short, it can be regarded as abnormal data, and the stroke is output as it is without performing pattern recognition. This can improve the efficiency of pattern recognition.
  • the terminal side may determine whether the stroke set constitutes a closed figure according to the coordinate data of the input stroke, and if the stroke set constitutes a closed figure, send the coordinate data of the corresponding stroke to the cloud; the cloud side Based on the coordinate data of the stroke, the class of the closed figure is identified.
  • FIG. 4A is a flowchart illustrating a method for recognizing hand-drawn graphics according to still another embodiment of the present disclosure.
  • the hand-drawn graphic recognition method includes: step S1, receiving the input of the hand-drawn graphic; step S5', identifying the type of the hand-drawn graphic according to the coordinate data of the input stroke; step S7, locating the key points of the hand-drawn graphic; step S9. According to the categories and key points of the hand-drawn graphics, output the correction drawing graphics corresponding to the hand-drawn graphics.
  • step S3 is not included, and step S5 ′ is different from step S5 . Since it is not necessary to perform the judgment of the closed figure in FIG. 2, that is, step S3 may not be performed, so different from identifying the type of closed figure in step S5, the type of hand-drawn figure is identified in step S5'.
  • the key points of the hand-drawn graphics can be located in the cloud.
  • Embodiments of the present disclosure also provide a hand-drawn graphic recognition device.
  • FIG. 4B is a block diagram illustrating a hand-drawn pattern recognition apparatus according to an embodiment of the present disclosure.
  • the hand-drawn pattern recognition device 40 includes: a memory 401 and a processor 402 coupled to the memory 401 .
  • the memory 401 is configured to store instructions for executing the corresponding embodiments of the hand-drawn pattern recognition method.
  • the processor 402 is configured to perform, based on the instructions stored in the memory 401, one or more steps of the hand-drawn pattern recognition method in any of some embodiments of the present disclosure.
  • one or more steps in the foregoing hand-drawn pattern recognition method may be implemented by a processor, and may be implemented in any manner of software, hardware, firmware or a combination thereof.
  • embodiments of the present disclosure may also take the form of a computer program product embodied on one or more non-volatile storage media containing computer program instructions. Therefore, an embodiment of the present disclosure further provides a computer-readable storage medium on which computer instructions are stored, and when the instructions are executed by a processor, implement one or more steps in the method for recognizing hand-drawn graphics in any of the foregoing embodiments.
  • FIG. 5 is a block diagram illustrating a hand-drawn pattern recognition system according to one embodiment of the present disclosure.
  • the hand-drawn pattern recognition system 5 includes: a touch screen 50 configured to receive input of hand-drawn patterns, and generate an information sequence including a plurality of trajectory points when a user draws a pattern on the touch-screen; and The aforementioned hand-drawn pattern recognition device 40 .
  • the hand-drawn pattern recognition device 40 includes: a first processor 41, located in the terminal, configured to determine whether a set of strokes constitutes a closed figure according to the coordinate data of the input strokes; a second processor 42, located in the cloud, When the set of strokes constitutes a closed figure, it is configured to identify the category of the closed figure based on the coordinate data of the strokes, or to locate the key points of the hand-drawn figure.
  • the first processor 41 and the second processor 42 of the hand-drawn pattern recognition device 40 can also be located on the cloud server, perform hand-drawn pattern recognition on the received hand-drawn track point information sequence, and transmit the pattern recognition result to the designated Receiver, such as a terminal.
  • FIG. 6 is a block diagram illustrating a computer system for implementing some embodiments of the present disclosure.
  • the computer system can be expressed in the form of a general-purpose computing device, and the computer system can be used to implement the hand-drawn pattern recognition apparatus of the above-mentioned embodiment.
  • the computer system includes a memory 610, a processor 620, and a bus 600 that connects various system components.
  • Memory 610 may include, for example, system memory, non-volatile storage media, and the like.
  • the system memory stores, for example, an operating system, an application program, a boot loader (Boot Loader), and other programs.
  • System memory may include volatile storage media such as random access memory (RAM) and/or cache memory.
  • RAM random access memory
  • the non-volatile storage medium stores, for example, instructions for executing corresponding embodiments of the display method.
  • Non-volatile storage media include, but are not limited to, magnetic disk memory, optical memory, flash memory, and the like.
  • Processor 620 may be implemented as a general purpose processor, digital signal processor (DSP), application specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete hardware components such as discrete gates or transistors.
  • DSP digital signal processor
  • ASIC application specific integrated circuit
  • FPGA field programmable gate array
  • each device such as the judging device and the determining device can be implemented by a central processing unit (CPU) running instructions in a memory for executing the corresponding steps, or can be implemented by a dedicated circuit for executing the corresponding steps.
  • CPU central processing unit
  • bus 600 may use any of a variety of bus structures.
  • bus structures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Peripheral Component Interconnect (PCI) bus.
  • ISA Industry Standard Architecture
  • MCA Micro Channel Architecture
  • PCI Peripheral Component Interconnect
  • the computer system may also include an input-output interface 630, a network interface 640, a storage interface 650, and the like.
  • the interfaces 630 , 640 , 650 and the memory 610 and the processor 620 can be connected through the bus 600 .
  • the input and output interface 630 may provide a connection interface for input and output devices such as a monitor, a mouse, and a keyboard.
  • Network interface 640 provides a connection interface for various networked devices.
  • the storage interface 640 provides a connection interface for external storage devices such as a floppy disk, a U disk, and an SD card.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • User Interface Of Digital Computer (AREA)
  • Character Discrimination (AREA)

Abstract

本公开涉及一种手绘图形识别方法、装置和系统，以及计算机可读存储介质。手绘图形识别方法包括：接收手绘图形的输入，输入包括笔划集合，笔划集合包括至少一个笔划，每个笔划包括多个轨迹点的信息序列，每个轨迹点的信息包括该轨迹点的坐标、书写时间；根据输入的笔划的坐标数据，确定笔划集合是否构成闭合图形；在笔划集合构成闭合图形的情况下，基于笔划的坐标数据，识别闭合图形的类别。

Description

手绘图形识别方法、装置和系统，以及计算机可读存储介质 技术领域
本公开涉及计算机技术领域，特别涉及一种手绘图形识别方法、装置和系统，以及计算机可读存储介质。
背景技术
相比于传统的键盘输入，手绘输入由于其简便而高效等优点，被广泛应用。手绘识别为自然的人机交互提供了便利。随着技术的发展，触控屏为用户提供了更好的手绘输入模式，允许用户在线输入多种图形。
相关的手绘图形识别技术，主要基于图像识别，输出与手绘图形对应的标准图形，例如圆、正方形、长方形、等腰三角形等对称图形，无法有效适应更多复杂场景的需求。
发明内容
根据本公开的一些实施例，提供了一种手绘图形识别方法，包括：
接收手绘图形的输入，输入包括笔划集合，笔划集合包括至少一个笔划，每个笔划包括多个轨迹点的信息序列，每个轨迹点的信息包括该轨迹点的坐标、书写时间；
根据输入的笔划的坐标数据，确定笔划集合是否构成闭合图形；
在笔划集合构成闭合图形的情况下，基于笔划的坐标数据，识别闭合图形的类别。
在一些实施例中，笔划集合包括多个笔划，每个笔划包括两个端点，每个轨迹点的信息还包括该轨迹点的书写状态值，其中，所述书写状态值用于表示轨迹点的书写状态，所述轨迹点的书写状态包括抬笔和落笔，根据输入的笔划的坐标数据，确定笔划集合是否构成闭合图形包括：
判断接收的当前笔划的落笔与上一个笔划的抬笔时间之间的间隔是否小于采集时间阈值；
在小于采集时间阈值的情况下，将当前笔划作为笔划集合的第j个笔划，根据当前笔划的端点与之前的笔划之间的连接关系，确定笔划集合是否构成闭合图形，j为大于1的正整数。
在一些实施例中，在一个笔划的端点与另一个笔划的任一端点之间的距离小于距 离阈值的情况下，一个笔划的端点与另一个笔划之间构成连接关系，根据当前笔划的端点与之前的笔划之间的连接关系，确定笔划集合是否构成闭合图形包括：
在当前笔划的一个端点与第j-m个笔划构成连接关系，且当前笔划的另一个端点与第j-n个笔划构成连接关系的情况下，确定笔划集合构成闭合图形，m、n为正整数且m大于或等于n，
其中，之前的笔划之间构成连接关系但不构成闭合图形。
在一些实施例中，在当前笔划的任一端点与之前的笔划都不构成连接关系的情况下，确定当前笔划与之前的笔划之间不构成连接关系且笔划集合不构成闭合，不进行图形识别。
在一些实施例中，在当前笔划的一个端点与之前的笔划中的一个笔划构成连接关系，但当前笔划的另一个端点与其他笔划都不构成连接关系的情况下，确定当前笔划与之前的笔划之间构成连接关系但笔划集合不构成闭合图形，并确定采集时间阈值范围内是否接收到第j+1个笔划；
在采集时间阈值范围内接收到第j+1个笔划的情况下，将j+1个笔划作为笔画集合的当前笔划，根据当前笔划的端点与之前的笔划之间的连接关系，确定笔划集合是否构成闭合图形；
在采集时间阈值范围内没有接收到第j+1个笔划的情况下，不进行图形识别。
在一些实施例中，所述手绘图形识别方法还包括在存储区中存储笔划的端点值，其中：
存储区中存储两个端点值，所述两个端点值包括第一端点值和第二端点值；
对于第1个笔划，第1个笔划的两个端点值分别为第一端点值和第二端点值；
对于第j个笔划，根据第j个笔划的端点值与存储区中的第一端点值和第二端点值之间的关系，确定第j个笔划与之前的笔划之间的连接关系，
在第j个笔划与之前的笔划构成连接关系但不构成闭合图形的情况下，在存储区中用第j个笔划中与之前的笔划不连接的端点值替代之前的笔划中与第j个笔划连接的端点值。
在一些实施例中，笔划集合包括多个连续的笔划，每个笔划包括两个端点，在多个连续的笔划中，每一个笔划的落笔与上一个笔划的抬笔时间之间的间隔都小于采集时间阈值，根据输入的笔划的坐标数据，确定笔划集合是否构成闭合图形包括：
根据时间顺序依次将多个连续的笔划中的笔划作为待处理笔划，根据待处理笔划 的端点与之前的笔划之间的连接关系，确定笔划集合是否构成闭合图形，所述待处理笔划为笔划集合中的第j个笔划，j为大于1的正整数。
在一些实施例中，在一个笔划的端点与另一个笔划的任一端点之间的距离小于距离阈值的情况下，一个笔划的端点与另一个笔划之间构成连接关系，根据待处理笔划的端点与之前的笔划之间的连接关系，确定笔划集合是否构成闭合图形包括：
在待处理笔划的一个端点与第j-m个笔划构成连接关系，且待处理笔划的另一个端点与第j-n个笔划构成连接关系的情况下，确定笔划集合构成闭合图形，m、n为正整数且m大于或等于n；
在待处理笔划的任一端点与之前的笔划都不构成连接关系的情况下，确定待处理笔划与之前的笔划之间不构成连接关系且笔划集合也不构成闭合图形；
在待处理笔划的一个端点与之前的笔划中的一个笔划构成连接关系，但待处理笔划的另一个端点与其他笔划都不构成连接关系的情况下，确定待处理笔划与之前的笔划之间构成连接关系但笔划集合不构成闭合图形，
其中，之前的笔划之间构成连接关系但不构成闭合图形。
在一些实施例中，在确定待处理笔划与之前的笔划之间构成连接关系但笔划集合不构成闭合图形的情况下，将j+1个笔划作为笔画集合的待处理笔划，根据待处理笔划的端点与之前的笔划之间的连接关系，确定笔划集合是否构成闭合图形；
在确定待处理笔划与之前的笔划之间不构成连接关系且笔划集合不构成闭合图形的情况下，不进行图形识别。
在一些实施例中，所述手绘图形识别方法还包括：对于笔划集合中的第1个笔划，识别该笔划构成的图形的类别，当识别出的类别为指定图形时，执行针对指定图形预设的手写笔迹处理操作。
在一些实施例中，当识别出的类别不是指定图形时，确定所述笔划是否构成闭合图形；当构成闭合图形时，识别闭合图形的类别；当不构成闭合图形时，确定采集时间阈值范围内是否接收到第2个笔划；在采集时间阈值范围内接收到第2个笔划的情况下，将第2个笔划作为当前笔划，判断第2个笔划与第1个笔划是否构成闭合图形。
在一些实施例中，所述手写笔迹处理操作包括：擦除、撤销、标记、或着重显示。
在一些实施例中，所述指定图形包括：倒N形、多折线或“Z”字形。
在一些实施例中，在所述指定图形为“Z”字形的情况下，执行擦除操作。
在一些实施例中，所述手绘图形识别方法还包括：根据闭合图形的类别，输出与手绘图形对应的校正绘制图形。
在一些实施例中，所述手绘图形识别方法还包括：
基于轨迹点之间的坐标偏移量，定位手绘图形的关键点；
根据闭合图形的类别和关键点，输出与手绘图形对应的校正绘制图形。
在一些实施例中，所述关键点为角点。
在一些实施例中，所述手绘图形识别方法还包括：
根据输入的笔划的坐标数据，确定笔划的长度是否在预设长度范围内；
在笔划的长度在预设长度范围内的情况下，利用该笔划进行闭合图形的识别；
在笔划的长度超出预设长度范围内的情况下，不进行图形识别。
在一些实施例中，识别闭合图形的类别包括：利用深度学习算法，识别闭合图形的类别。
在一些实施例中，深度学习算法包括循环神经网络。
在一些实施例中，图形的类别包括多边形、圆形、椭圆形中的至少一种。
在一些实施例中，所述手绘图形识别方法还包括：
在识别闭合图形的类别之前，根据预定距离阈值对轨迹点进行稀疏采样，得到轨迹点集合；和/或
根据闭合图形的类别识别所需的轨迹点数目，对所述轨迹点集合进行稀疏采样。
在一些实施例中，终端侧根据输入的笔划的坐标数据，确定笔划集合是否构成闭合图形，并在笔划集合构成闭合图形的情况下，将相应的笔划的坐标数据发送给云端；云端侧基于笔划的坐标数据，识别闭合图形的类别。
根据本公开的另一些实施例，提供了一种手绘图形识别方法，包括：
接收手绘图形的输入，输入包括笔划集合，笔划集合包括至少一个笔划，每个笔划包括多个轨迹点的信息序列，每个轨迹点的信息包括该轨迹点的坐标、书写时间；
根据输入的笔划的坐标数据，识别手绘图形的类别；
定位手绘图形的关键点；
根据手绘图形的类别和关键点，输出与手绘图形对应的校正绘制图形。
在一些实施例中，定位手绘图形的关键点包括：
基于轨迹点之间的坐标偏移量，定位手绘图形的关键点。
在一些实施例中，基于轨迹点之间的坐标偏移量，定位手绘图形的关键点包括：
计算轨迹点之间的坐标偏移量，坐标偏移量包括横坐标偏移量和纵坐标偏移量；
根据轨迹点之间的横坐标偏移量和纵坐标偏移量的大小，将坐标偏移量划分到多个区间；
根据坐标偏移量所属区间之间的关系，定位手绘图形的关键点。
在一些实施例中，对于包括轨迹点P i、P i+1、P i+2……P i+n的信息序列，设轨迹点P i+n+1和P i+n之间的坐标偏移量为O n，轨迹点P i+n和P i+n-1之间的坐标偏移量为O n-1，坐标偏移量O n与O n-1为相邻的坐标偏移量，根据坐标偏移量所属区间之间的关系，定位手绘图形的关键点包括：
在相邻的坐标偏移量O n与O n-1所属区间之间有区间间隔的情况下，基于相邻的坐标偏移量O n与O n-1共同对应的轨迹点P i+n，定位手绘图形的关键点，其中i为正整数，n为大于1的正整数。
在一些实施例中，根据坐标偏移量所属区间之间的关系，定位手绘图形的关键点还包括：
在相邻的坐标偏移量O n与O n-1所属区间相邻的情况下，确定O n对应的轨迹点P i+n和P i+n+1的连线与O n-1对应的轨迹点P i+n-1和P i+n的连线之间的第一夹角是否大于第一角度阈值；
在第一夹角大于第一角度阈值的情况下，基于相邻的坐标偏移量O n与O n-1共同对应的轨迹点P i+n，定位手绘图形的关键点。
在一些实施例中，根据坐标偏移量所属区间之间的关系，定位手绘图形的关键点还包括：
在第一夹角不大于第一角度阈值的情况下，或者在相邻的坐标偏移量O n与O n-1所属区间相同的情况下，若偏移量O n-k与O n-1之间的偏移量与O n-1所属区间相同、但偏移量O n-k与O n-1所属区间不同，确定偏移量O n-k+1对应的轨迹点P i+n-k+1或P i+n-k+2和O n对应的轨迹点P i+n的连线与O n对应的轨迹点P i+n和P i+n+1的连线之间的第二夹角是否大于第二角度阈值，k为大于2的正整数，n为大于3的正整数；
在第二夹角大于第二角度阈值的情况下，基于相邻的坐标偏移量O n与O n-1共同对应的轨迹点P i+n，定位手绘图形的关键点。
在一些实施例中，根据坐标偏移量所属区间之间的关系，定位手绘图形的关键点还包括：
在第一夹角不大于第一角度阈值的情况下，或者在相邻的坐标偏移量O n与O n-1 所属区间相同的情况下，若偏移量O n+k与O n之间的偏移量与O n所属区间相同、但偏移量O n+k与O n所属区间不同，确定偏移量O n+k-1对应的轨迹点P i+n+k-1或P i+n+k和O n-1对应的轨迹点P i+n的连线与O n-1对应的轨迹点P i+n-1和P i+n的连线之间的第二夹角是否大于第二角度阈值，k为大于2的正整数，n为大于3的正整数；
在第二夹角大于第二角度阈值的情况下，基于相邻的坐标偏移量O n与O n-1共同对应的轨迹点P i+n，定位手绘图形的关键点。
在一些实施例中，根据坐标偏移量所属区间之间的关系，定位手绘图形的关键点还包括：
根据坐标偏移量所属区间，对轨迹点之间的横坐标偏移量和纵坐标偏移量分别进行重赋值；
根据重赋值，确定坐标偏移量所属区间之间的关系。
在一些实施例中，所述关键点为角点。
在一些实施例中，所述手绘图形识别方法还包括：
在定位手绘图形的关键点之前，根据预定距离阈值对轨迹点进行稀疏采样，得到轨迹点集合；和/或
根据手绘图形的关键点定位所需的轨迹点数目，对所述轨迹点集合进行稀疏采样。
根据本公开的又一些实施例，提供了一种手绘图形识别装置，包括：
存储器；和
耦接至所述存储器的处理器，所述处理器被配置为基于存储在所述存储器装置中的指令，执行前述任一实施例所述的手绘图形识别方法的一个或多个步骤。
根据本公开的再一些实施例，提供了一种手绘图形识别系统，包括：
触控屏，被配置为接收手绘图形的输入，在用户于所述触控屏上手绘图形时产生包括多个轨迹点的信息序列；
前述实施例所述的手绘图形识别装置。
在一些实施例中，所述手绘图形识别装置包括：
第一处理器，位于终端，被配置为根据输入的笔划的坐标数据，确定笔划集合是否构成闭合图形；
第二处理器，位于云端，被配置为在笔划集合构成闭合图形的情况下，基于笔划的坐标数据，识别闭合图形的类别，或者被配置为定位手绘图形的关键点。
根据本公开的其他实施例，提供了一种计算机可读存储介质，其上存储有计算机 程序，该程序被处理器执行时实现前述任一实施例所述的手绘图形识别方法的一个或多个步骤。
通过以下参照附图对本公开的示例性实施例的详细描述，本公开的其它特征及其优点将会变得清楚。
附图说明
构成说明书的一部分的附图描述了本公开的实施例，并且连同说明书一起用于解释本公开的原理。
参照附图，根据下面的详细描述，可以更加清楚地理解本公开，其中：
图1是示出根据本公开一个实施例的手绘图形识别方法的流程图；
图1A是示出根据本公开一个实施例的手绘图形的示意图；
图2是示出根据本公开另一个实施例的手绘图形识别方法的流程图；
图2A是示出根据本公开一个实施例的定位手绘图形的关键点的流程图；
图2B是示出根据本公开一个实施例的区间划分的示意图；
图3是示出根据本公开又一个实施例的手绘图形识别方法的流程图；
图4A是示出根据本公开再一个实施例的手绘图形识别方法的流程图；
图4B是示出根据本公开一个实施例的手绘图形识别装置的框图；
图5是示出根据本公开一个实施例的手绘图形识别系统的框图；
图6是示出用于实现本公开一些实施例的计算机系统的框图。
应当明白，附图中所示出的各个部分的尺寸并不是按照实际的比例关系绘制的。此外，相同或类似的参考标号表示相同或类似的构件。
具体实施方式
现在将参照附图来详细描述本公开的各种示例性实施例。对示例性实施例的描述仅仅是说明性的，决不作为对本公开及其应用或使用的任何限制。本公开可以以许多不同的形式实现，不限于这里所述的实施例。提供这些实施例是为了使本公开透彻且完整，并且向本领域技术人员充分表达本公开的范围。应注意到：除非另外具体说明，否则在这些实施例中阐述的部件和步骤的相对布置应被解释为仅仅是示例性的，而不是作为限制。
本公开使用的所有术语(包括技术术语或者科学术语)与本公开所属领域的普通 技术人员理解的含义相同，除非另外特别定义。还应当理解，在诸如通用字典中定义的术语应当被解释为具有与它们在相关技术的上下文中的含义相一致的含义，而不应用理想化或极度形式化的意义来解释，除非这里明确地这样定义。
对于相关领域普通技术人员已知的技术、方法和设备可能不作详细讨论，但在适当情况下，所述技术、方法和设备应当被视为说明书的一部分。
用户在白板等设备上手绘图形时，可能会画出各种不规则的图形，例如非对称的图形，依靠已知的图形识别技术，无法有效识别。为了正确有效地识别种类繁多且复杂的手绘图形，本公开提出了新的手绘图形识别方案。
本公开提出了通过手绘轨迹，例如在用户画图过程中实时获取轨迹点的坐标信息，判断用户是否已经完成了图形的绘画、用户画的是什么目标图形。在本公开的一些实施例中，还可以进一步判断目标图形的具体类别，图形是否是常规的正多边形或圆或倾斜(如不等边)的多边形。在另一些实施例中，还可以基于轨迹点信息，定位目标图形的关键点(如拐点或角点)。利用这些信息，可以绘制出符合用户需要的几何图形，实现更好的人机交互效果。
图1是示出根据本公开一个实施例的手绘图形识别方法的流程图。如图1所示，手绘图形识别方法包括：步骤S1，接收手绘图形的输入；步骤S3，根据输入的笔划的坐标数据，确定笔划集合是否构成闭合图形；步骤S5，在笔划集合构成闭合图形的情况下，基于笔划的坐标数据，识别闭合图形的类别。
在步骤S1，输入笔划集合，笔划集合包括至少一个笔划，每个笔划包括多个轨迹点的信息序列。每个轨迹点的信息包括该轨迹点的坐标、书写时间。每个轨迹点的信息还包括该轨迹点的书写状态值。书写状态值用于表示轨迹点的书写状态，所述轨迹点的书写状态包括抬笔和落笔。例如书写状态值可以为表示笔划的抬笔的第一值、或表示落笔的第二值。例如，一个笔划的终点轨迹点的书写状态值为表示抬笔的第一值(例如1)，一个笔划的起点轨迹点的书写状态值为表示落笔的第二值(例如0)。当然，书写状态值还可以包括表示压笔书写或运笔的第三值。即，可以根据每个轨迹点的书写状态值，将轨迹点划分到对应的笔划。
在一些实施例中，在触控屏上手绘图形时产生包括多个轨迹点的信息序列。例如，在电磁屏、电容屏等触控屏上书写时，采集按时间顺序存储的手绘轨迹点信息序列。在另一些实施例中，诸如会议一体机、手机、智能墨水平板的终端，利用软件获取在触控屏上画图时产生的手绘轨迹点序列信息，并且可以根据业务的实际需求，将手绘 轨迹点序列信息传送给云服务端或嵌入式终端。
对用户输入的手绘图形进行识别，首先要定位出待识别的对象。例如，判断哪些笔划构成了闭合图形，然后基于这些笔划来分别识别各个闭合图形。在步骤S3，基于笔划包括的各轨迹点的坐标，确定笔划集合是否构成闭合图形。这里，笔划集合可以包括一个笔划，也可以包括多个笔划。每个笔划包括两个端点。应当理解，闭合图形不一定是严格的闭合。下面结合图1A来具体描述。
图1A是示出根据本公开一个实施例的手绘图形的示意图。
在输入为一个笔划的情况下，如图1A中(a)-(b)所示，确定笔划是否构成闭合图形包括：确定笔划的落笔与抬笔位置之间的距离是否小于距离阈值。也就是说，在一个笔划开始手写的落笔位置与这个笔划结束手写的抬笔位置之间的距离小于距离阈值的情况下，如图1A中(a)所示，可以认为这个笔划构成了闭合图形。而对于图1A中(b)所示，一个笔划的落笔与抬笔位置之间的距离较大，认为这个笔划不构成闭合图形。
在一些实施例中，在对于笔划集合中的第一个笔划(可根据笔划计数确定)，可以先不判断该笔划是否构成闭合图形，而先直接进行图形类别的识别。在识别出的类别为指定图形的情况下，例如倒“N”形、多折线或“Z”字形，则针对指定图形预设的手写笔迹处理。所述手写笔迹处理包括：擦除、撤销、标记、或着重显示等等。例如，在所述指定图形为“Z”字形的情况下，执行擦除操作。利用指定图形来执行预设的手写笔迹处理，能够提供更快捷的操作体验。
当识别出的类别不是指定图形时，确定所述笔划是否构成闭合图形；当构成闭合图形时，识别闭合图形的类别；当不构成闭合图形时，确定采集时间阈值范围内是否接收到第2个笔划；在采集时间阈值范围内接收到第2个笔划的情况下，将第2个笔划作为当前笔划，判断第2个笔划与第1个笔划是否构成闭合图形。后续，可以根据笔划集合包括多个笔划的情形进行进一步的处理。
在一些实施例中，距离阈值是根据轨迹点的坐标动态确定的值，因此可以根据手绘图形的特征自适应调整。当然，根据实际需要也可以将距离阈值设为固定值。
在笔划集合包括多个笔划的情况下，如图1A中(c)-(e)所示，确定笔划集合是否构成闭合图形则包括：确定一个笔划的落笔与上一个笔划的抬笔时间之间的间隔是否小于采集时间阈值；在小于采集时间间隔的情况下，确定一个笔划的端点与之前的笔划是否构成连接关系。若在一个笔划的端点与另一个笔划的任一端点之间的距离 小于距离阈值的情况下，则一个笔划的端点与另一个笔划之间构成连接关系。也就是说，在连续的笔划之间的时间间隔小于采集时间阈值(例如1秒、3秒或其他时间)的情况下，认为这些连续的笔划属于同一个图形，时间间隔超出采集时间阈值的笔划，认为不属于同一个图形。并且，根据当前笔划的端点与之前的笔划之间的连接关系，确定笔划集合是否构成闭合图形。在连续的笔划之间都构成连接关系的情况下，如图1A中(c)和(d)所示，可以认为这些连续的笔划构成了闭合图形。而对于图1A中(e)所示，有两个笔划不构成连接关系，即一个笔划中存在一个端点与其他笔划的端点之间的距离较大的情况，认为这些笔划不构成闭合图形。
对于不构成闭合图形的笔划，按原样输出。在一些实施例中，考虑到实际场景，在笔划的长度出现异常时，例如笔划的长度超过预设长度范围，即笔划的长度很长或很短，都可以认为是异常数据，则不进行图形识别，例如可将笔划按原样输出，或什么也不显示。即，手绘图形识别方法还包括：根据输入的笔划的坐标数据，确定笔划的长度是否在预设长度范围内。在笔划的长度在预设长度范围内的情况下，利用该笔划进行闭合图形的识别；在笔划的长度超出预设长度范围内的情况下，不进行图形识别。确定笔划的长度是否异常，可以在确定连续的笔划是否构成闭合图形前，也可以在其后。通过这样的设置，可以提高图形识别的效率。
上述确定笔划集合是否构成闭合图形的步骤，可以根据处理器的处理速度与手绘图形的输入速度之间的关系，选择针对每一个接收的当前笔划实时进行判断，或者针对多个连续的笔划集中进行判断。下面结合不同的实施例分别对这两种情形进行详细描述。
在一些实施例中，针对每一个接收的当前笔划实时进行判断，即判断当前笔划的端点与之前的笔划之间的连接关系，进而确定笔划集合是否构成闭合图形。
根据输入的笔划的坐标数据，确定笔划集合是否构成闭合图形包括：判断接收的当前笔划的落笔与上一个笔划的抬笔时间之间的间隔是否小于采集时间阈值；在小于采集时间阈值的情况下，将当前笔划作为笔划集合的第j个笔划，根据当前笔划的端点与之前的笔划之间的连接关系，确定笔划集合是否构成闭合图形，j为大于1的正整数。其中，之前的笔划之间构成连接关系但不构成闭合图形，例如，判断第j个笔划的落笔与笔划集合中第j-1个笔划的抬笔时间之间的间隔是否小于采集时间阈值；在小于采集时间阈值的情况下，将第j笔划加入笔划集合，根据第j个笔划的端点与第1至j-1个笔划之间的连接关系，确定第1至j个笔划是否构成闭合图形。这里， 之前的笔划之间构成连接关系但不构成闭合图形，即，第j-1个笔划与其之前的笔划构成连接关系但不构成闭合图形。
在当前笔划的一个端点与第j-m个笔划构成连接关系，且当前笔划的另一个端点与第j-n个笔划构成连接关系的情况下，确定笔划集合构成闭合图形，m、n为正整数且m大于或等于n。
在当前笔划的任一端点与之前的笔划都不构成连接关系的情况下，确定当前笔划与之前的笔划之间不构成连接关系且笔划集合不构成闭合图形。
在当前笔划的一个端点与之前的笔划中的一个笔划构成连接关系，但当前笔划的另一个端点与其他笔划都不构成连接关系的情况下，确定笔划集合构成连接关系但不构成闭合图形，不进行图形识别。
在确定笔划集合构成连接关系但不构成闭合图形的情况下，确定采集阈值时间范围内是否接收到第j+1个笔划；在采集时间阈值范围内接收到第j+1个笔划的情况下，将j+1个笔划作为笔画集合的当前笔划，根据当前笔划的端点与之前的笔划之间的连接关系，确定笔划集合是否构成闭合图形；在采集时间阈值范围内没有接收到第j+1个笔划的情况下，不进行图形识别，例如原样输出笔划集合。
下面以j＝3、m＝2、n＝1，第1至2个笔划构成连接关系但不构成闭合图形为例，描述上述各种情形。
在第3个笔划的一个端点与第1个笔划构成连接关系，且另一个端点与第2个笔划构成连接关系的情况下，确定第1至3个笔划构成闭合图形。
在第3个笔划的任一端点与第1个和第2个笔划都不构成连接关系的情况下，确定第3个笔划与之前的笔划既不构成连接关系也不构成闭合图形。这种情况下，原样输出第1至3个笔划，且可以将笔划计数清零。
在第3个笔划的一个端点与第1个笔划构成连接关系，但第3个笔划的另一个端点与第2个笔划不构成连接关系的情况下，确定第3个笔划与之前的笔划构成连接关系但不构成闭合图形。
在确定第3个笔划与之前的笔划构成连接关系但不构成闭合图形的情况下，进一步判断采集阈值时间范围内是否接收到第4个笔划。在接收到第4个笔划的情况下，将第4个笔划作为当前笔划，再次执行前述判断，即根据当前笔划的端点与之前的笔划之间的连接关系，确定笔划集合是否构成闭合图形。在采集阈值时间范围内没有接收到第4个笔划的情况下，原样输出第1至3个笔划。在原样输出的情况下，可以将 笔划计数清零，以便于后续的笔划之间关系的判断。
在一些实施例中，手绘图形识别方法还包括在存储区中存储笔划的端点值。存储区中存储两个端点值，两个端点值包括第一端点值和第二端点值。
对于第1个笔划，第1个笔划的两个端点值分别为第一端点值和第二端点值。对于第2个笔划，根据第2个笔划的端点值与存储区中的第一端点值和第二端点值之间的关系，确定第2个笔划与第1个笔划之间的连接关系，在第2个笔划与第1个笔划构成连接关系但不构成闭合图形的情况下，从存储区中用第2个笔划中与第1个笔划不连接的端点值替代第1个笔划中与第2个笔划连接的端点值。
即，对于第j个笔划，根据第j个笔划的端点值与存储区中的第一端点值和第二端点值之间的关系，确定第j个笔划与之前的笔划之间的连接关系；在第j个笔划与之前的笔划构成连接关系但不构成闭合图形的情况下，在存储区中用第j个笔划中与之前的笔划不连接的端点值替代之前的笔划中与第j个笔划连接的端点值。
在另一些实施例中，针对多个连续的笔划集中进行判断，即判断当前多个连续笔划中某一个笔划(如第j个笔划)的端点与之前的笔划之间的连接关系，进而确定笔划集合是否构成闭合图形。即，根据时间顺序依次将多个连续的笔划中的笔划作为待处理笔划，根据待处理笔划的端点与之前的笔划之间的连接关系，确定笔划集合是否构成闭合图形，所述待处理笔划为笔划集合中的第j个笔划。
根据输入的笔划的坐标数据，确定笔划集合是否构成闭合图形包括：在待处理笔划的一个端点与第j-m个笔划构成连接关系，且待处理笔划的另一个端点与第j-n个笔划构成连接关系的情况下，确定笔划集合构成闭合图形，m、n为正整数且m大于或等于n；在待处理笔划的任一端点与之前的笔划都不构成连接关系的情况下，确定待处理笔划与之前的笔划之间不构成连接关系且笔划集合也不构成闭合图形；在待处理笔划的一个端点与之前的笔划中的一个笔划构成连接关系，但待处理笔划的另一个端点与其他笔划都不构成连接关系的情况下，确定待处理笔划与之前的笔划之间构成连接关系但笔划集合不构成闭合图形。其中，之前的笔划之间构成连接关系但不构成闭合图形。
在确定待处理笔划与之前的笔划之间构成连接关系但笔划集合不构成闭合图形的情况下，将j+1个笔划作为笔画集合的待处理笔划，根据待处理笔划的端点与之前的笔划之间的连接关系，确定笔划集合是否构成闭合图形；在确定待处理笔划与之前的笔划之间不构成连接关系且笔划集合不构成闭合图形的情况下，不进行图形识别， 例如原样输出笔划集合。在原样输出的情况下，仍可以将笔划计数清零，以便于后续的笔划之间关系的判断。
下面以笔划集合包括8个连续的笔划，j＝4、m＝2、n＝1，第1至3个笔划构成连接关系但不构成闭合图形为例，描述上述各种情形。
在第4个笔划的一个端点与第2个笔划构成连接关系，且另一个端点与第3个笔划构成连接关系的情况下，确定第2至4个笔划构成闭合图形。
在第4个笔划的任一端点与第1至3个笔划都不构成连接关系的情况下，确定第4个笔划与之前的笔划既不构成连接关系也不构成闭合图形。这种情况下，不进行图形识别，例如可以原样输出第1至4个笔划。且可以将笔划计数清零，继续以第5个笔划为第一笔划进行相应处理。
在第4个笔划的一个端点与第2个笔划构成连接关系，但另一个端点与第1个笔划和第3个笔划都不构成连接关系的情况下，确定第4个笔划与之前的笔划构成连接关系但不构成闭合图形。这种情况下，进一步将第5个笔划作为待处理笔划，再次执行前述判断，即根据待处理笔划的端点与之前的笔划之间的连接关系，确定笔划集合是否构成闭合图形。
而对于构成闭合图形的笔划，则在步骤S5，可以根据笔划包括的轨迹点的坐标数据，利用深度学习算法，对闭合图形进行类别的识别。
闭合图形的类别可以包括各种标准图形或非标准图形，或者称为规则图形或不规则图形，或者也可以称为对称图形或非对称图形。具体地，图形的类别可以包括多边形、圆形、椭圆形中的至少一种。多边形例如可以包括矩形、等腰三角形、等腰梯形、平行四边形等对称图形，也可以包括非等腰三角形、非等腰梯形等非对称图形。应当理解，闭合图形都满足图形的基本特征，例如，对于构成圆的笔划，各个轨迹点的坐标需满足一定的条件，至少在轨迹点之间有上下左右四个方向的偏移量。
在步骤S5识别闭合图形的类别之后，可以根据闭合图形的类别，输出与手绘图形对应的校正绘制图形。
上述校正绘制图形是对手绘图形进行了必要的校正后的图形，例如，根据手绘的线所包括的各轨迹点之间的关系，将手绘的线校正为直线(如与多边形对应)或曲线(如与圆或椭圆对应)。
在一些实施例中，利用循环神经网络(RNN)，识别闭合图形的类别。例如，可以采用双向长短期记忆(BiLSTM)网络，结合线性层(Linear)进行图形的类别识 别，得到全面、精确的识别结果。
LSTM网络的层数可以根据任务的难易程度以及耗时情况确定，可以采用1层或者更多层。Linear层的作用是线性变换输出指定维度。根据准确率需要也可使用更加复杂的RNN网络。
图2是示出根据本公开另一个实施例的手绘图形识别方法的流程图。图2与图1的不同之处在于，图2中的手绘图形识别方法还包括步骤S7和S9。下面将仅描述图2与图1的不同之处，相同之处不再赘述。
在步骤S7，定位手绘图形的关键点。在一些实施例中，基于轨迹点之间的坐标偏移量，定位手绘图形的关键点。关键点也可称作图形的拐点或角点，表示在该轨迹点处发生了斜率的显著变化。例如对于多边形而言，各条边的交点为拐点或角点。应当理解，步骤S7和识别图形的类别的步骤S5的执行顺序不受限制，可以先执行步骤S7再执行步骤S5，也可以先执行步骤S5再执行S7。当然，先识别图形的类别，然后再基于图形的类别定位图形的关键点，计算效率更高。
下面结合图2A描述根据本公开一些实施例的定位手绘图形的关键点的流程。如图2A所示，定位手绘图形的关键点的流程包括步骤S71-S73。
在步骤S71，计算轨迹点之间的坐标偏移量。坐标偏移量包括横坐标偏移量和纵坐标偏移量。对于包括轨迹点P i、P i+1、P i+2……P i+n的信息序列，设轨迹点P i+n+1和P i+n之间的坐标偏移量为O n，轨迹点P i+n和P i+n-1之间的坐标偏移量为O n-1，坐标偏移量O n与O n-1为相邻的坐标偏移量。
在步骤S72，根据轨迹点之间的横坐标偏移量和纵坐标偏移量的大小，将坐标偏移量划分到多个区间。
对于计算得到的坐标偏移量，横坐标偏移量记为dx，纵坐标偏移量记为dy。即，O n可表示为(dx n,dy n)。通过比较dx和dy的大小，将坐标偏移量划分到多个区间。根据手绘图形的识别难度，确定划分区间的个数。
在一些实施例中，为了减小计算量，还可以根据坐标偏移量所属区间，对轨迹点之间的横坐标偏移量和纵坐标偏移量分别进行重赋值；根据重赋值，确定坐标偏移量所属区间之间的关系。重赋值便于标识坐标偏移量所属的区间，可以直观反映出输入的轨迹点的走势。基于坐标偏移量定位图形的关键点，相比基于斜率来定位图形的关键点，计算更简单，计算量更小。
下面以划分到4个区间为例，结合表1说明区间划分和重赋值。表1示出区间划 分结果、重赋值结果与坐标偏移量的关系。
表1：区间划分结果与坐标偏移量的关系、重赋值的结果
Figure PCTCN2021088951-appb-000001
如表1所示，对于两个轨迹点，若dy＞0，则将0＜斜率dy/dx≤1的坐标偏移量划分到第一区间，而将-1≤斜率dy/dx＜0的坐标偏移量划分到第三区间，将dy/dx大于1或小于-1的坐标偏移量划分到第二区间；若dy＜0，则将-1≤斜率dy/dx＜0的坐标偏移量划分到第一区间，将0＜斜率dy/dx≤1的坐标偏移量划分到第三区间，而将斜率dy/dx大于1或小于-1的坐标偏移量划分到第四区间。
对于第一区间和第三区间，即斜率的绝对值|dy/dx|小于或等于1，重赋值的结果为dy等于0、dx等于原值；对于第二区间和第四区间，即斜率的绝对值|dy/dx|大于1。
对于dy＝0的情形，可以与dy＞0或dy＜0的情形进行类似处理。
图2B示出根据本公开一些实施例的区间划分的示意图。如图2B所示，可以将360度区间划分为4个区间，分割的斜率绝对值为1，直观地示出依据表1进行区间划分的结果。应当理解，如果手绘图形复杂，为了更精细的计算，可以划分更多区间，例如8个区间或更多。
接下来，在步骤S73，根据坐标偏移量所属区间之间的关系，定位手绘图形的关键点。下面结合表2说明如何定位手绘图形的关键点。表2示出相邻坐标偏移量所属区间之间的关系，以及对应的关键点定位结果。
表2：相邻坐标偏移量所属区间之间的关系，以及对应的关键点定位结果
Figure PCTCN2021088951-appb-000002
Figure PCTCN2021088951-appb-000003
如表2所示，在相邻的坐标偏移量O n与O n-1所属区间之间有区间间隔的情况下， 基于相邻的坐标偏移量O n与O n-1共同对应的轨迹点P i+n定位手绘图形的关键点，其中i为正整数，n为大于1的正整数。例如，可以将偏移量O n与O n-1共同对应的轨迹点P i+n作为手绘图形的关键点。如图2B所示，第一区间与第三区间之间有区间间隔，第一区间与第二区间之间或与第四区间之间没有区间间隔。
在相邻的坐标偏移量O n与O n-1所属区间相邻的情况下，确定O n对应的轨迹点P i+n和P i+n-1的连线与O n-1对应的轨迹点P i+n+1和P i+n的连线之间的第一夹角是否大于第一角度阈值；在第一夹角大于第一角度阈值的情况下，基于相邻的坐标偏移量O n与O n-1共同对应的轨迹点P i+n定位手绘图形的关键点。例如，可以将偏移量O n与O n-1共同对应的轨迹点P i+n作为手绘图形的关键点。如图2B所示，第一区间与第二区间之间或与第四区间相邻。第一角度阈值例如为30度。
在第一夹角不大于第一角度阈值的情况下，或者在相邻的坐标偏移量O n与O n-1所属区间相同的情况下，若偏移量O n-k与O n-1之间的偏移量与O n-1所属区间相同、但偏移量O n-k与O n-1所属区间不同，确定偏移量O n-k+1对应的轨迹点P i+n-k+1或P i+n-k+2和O n对应的轨迹点P i+n的连线与O n对应的轨迹点P i+n和P i+n+1的连线之间的第二夹角，k为大于2的正整数，n为大于3的正整数；或者若偏移量O n+k与O n之间的偏移量与O n所属区间相同、但偏移量O n+k与O n所属区间不同，确定偏移量O n+k-1对应的轨迹点P i+n+k-1或P i+n+k和O n-1对应的轨迹点P i+n的连线与O n-1对应的轨迹点P i+n-1和P i+n的连线之间的第二夹角是否大于第二角度阈值。
在第二夹角大于第二角度阈值的情况下，基于偏移量O n与O n-1共同对应的轨迹点P i+n定位手绘图形的关键点。例如，可以将偏移量O n与O n-1共同对应的轨迹点P i+n作为手绘图形的关键点。第二角度阈值可以与第一角度阈值相同，也可以略有不同。
本公开实施例充分考虑了手绘图形的复杂场景，例如用户手绘的折线夹角是带有圆弧的角，通过综合分析附近轨迹点的信息，能够更准确地定位手绘图形的关键点。
在一些实施例中，手绘图形识别方法还包括：对输入的轨迹点，基于坐标进行稀疏采样。在稀疏采样后再进行图形的类别识别和关键点定位，可以减少不必要的计算量，提高图形识别的速度。在一些实施例中在识别闭合图形的类别之前，或者在定位手绘图形的关键点之前，根据预定距离阈值对轨迹点进行稀疏采样，得到轨迹点集合。例如，对于输入的两个轨迹点之间的距离小于预定阈值的情况，去除其中任一个轨迹点，得到轨迹点集合。例如，对于两个轨迹点之间的距离小于预定阈值(例如20像素)的，可以删除其中一个，这样可以去除部分密集的轨迹点。又例如，对于一个轨 迹点，在该轨迹点与其前一个轨迹点、后一个轨迹点的连线的斜率之间的差值小于或等于变化阈值(例如0.1)的情况下，则可以认为这三个轨迹点在一条直线上，因此可以删除该轨迹点，以减少待识别的数据量。对于手绘图形较简单的情况下，还可以根据情况取固定的轨迹点数量(例如40)，然后按固定数量取轨迹点。例如，对于四边形，每条边可以取8个轨迹点。
进一步地，还可以根据闭合图形的类别识别所需的轨迹点数目，对所述轨迹点集合进行稀疏采样。也可以根据手绘图形的关键点定位所需的轨迹点数目，对所述轨迹点集合进行稀疏采样。
在步骤S9，根据闭合图形的类别和关键点，输出与手绘图形对应的校正绘制图形。
与闭合图形的类别对应，输出的校正绘制图形可能是规则图形，也可以是不规则图形。在一些实施例中，可以先根据在步骤S3中识别的闭合图形的类别，结合轨迹点的坐标，判断闭合图形是否为规则图形。
例如，若识别出闭合图形为多边形，根据轨迹点的坐标，可以计算出多边形各条边的边长以及相邻边的夹角，进而判断闭合图形是否为规则图形。如果判断结果为规则图形，则可以根据轨迹点的坐标确定中心点、宽度、高度等参数，输出与手绘图形对应的规则图形。如果判断结果为非规则图形，则可以根据计算出的边长、夹角等参数输出与手绘图形对应的非规则图形。例如，在识别出手绘图形为非规则三角形的情况下，利用关键点定位可以得到角点1、2、3，将3个角点连线即可输出与手绘图形对应的非规则三角形。
图3是示出根据本公开又一个实施例的手绘图形识别方法的流程图。图3与图2的区别在于还包括步骤S2。下面将仅描述图3与图2的不同之处，相同之处不再赘述。
如图3所示，在步骤S2，根据输入的笔划的坐标数据，确定笔划的长度是否在预设长度范围内。在笔划的长度在预设长度范围内的情况下，利用该笔划进行闭合图形的识别。在笔划的长度超过预设长度范围的情况下，即笔划的长度很长或很短，都可以认为是异常数据，将笔划按原样输出，而不进行图形识别。这样可以提高图形识别的效率。
如图3所示，对于一个笔划的情形，如构成闭合图形，则识别图形的类别，并相应输出校正绘制图形；如不构成闭合图形，但在步骤S31中判断构成指定图形(如“Z”字形)，则进入步骤S32，执行擦除操作；如既不构成闭合图形也不构成指定图形，则原样输出笔划。
当然，如前所述，对于一个笔划的情形，也可以先不判断是否构成闭合图形，而直接进行图形类别的识别，如构成指定图形则执行擦除操作；如不构成指定图形但被识别为某一类别的图形，则绘制相应类别的图形；否则原样输出笔划。
在前述实施例中，可以由终端侧根据输入的笔划的坐标数据，确定笔划集合是否构成闭合图形，并在笔划集合构成闭合图形的情况下，将相应的笔划的坐标数据发送给云端；云端侧基于笔划的坐标数据，识别闭合图形的类别。
图4A是示出根据本公开再一个实施例的手绘图形识别方法的流程图。
如图4A所示，手绘图形识别方法包括：步骤S1、接收手绘图形的输入；步骤S5'、根据输入的笔划的坐标数据，识别手绘图形的类别；步骤S7、定位手绘图形的关键点；步骤S9、根据手绘图形的类别和关键点，输出与手绘图形对应的校正绘制图形。
可以看到，图4A与图2的区别在于不包括步骤S3，且步骤S5'与步骤S5不同。由于不需要执行图2中闭合图形的判断，即可以不执行步骤S3，所以不同于步骤S5中识别闭合图形的类别，在步骤S5'中识别手绘图形的类别。
在上述实施例中，定位手绘图形的关键点，可以在云端进行。
至此，已经详细描述了本公开各种方法实施例，下面描述相应的产品实施例。本公开实施例还提供了一种手绘图形识别装置。
图4B是示出根据本公开一个实施例的手绘图形识别装置的框图。
如图4B所示，手绘图形识别装置40包括：存储器401以及耦接至该存储器401的处理器402。存储器401配置成存储执行手绘图形识别方法对应实施例的指令。处理器402被配置为基于存储在存储器401中的指令，执行本公开中任意一些实施例中的手绘图形识别方法中的一个或多个步骤。
应当理解，前述手绘图形识别方法中的一个或多个步骤可以通过处理器来实现，并且可以软件、硬件、固件或其结合的任一种方式实现。
除了手绘图形识别方法和装置之外，本公开实施例还可采用在一个或多个包含有计算机程序指令的非易失性存储介质上实施的计算机程序产品的形式。因此，本公开实施例还提供一种计算机可读存储介质，其上存储有计算机指令，该指令被处理器执行时实现前述任意实施例中的手绘图形识别方法中的一个或多个步骤。
图5是示出根据本公开一个实施例的手绘图形识别系统的框图。
如图5所示，手绘图形识别系统5包括：触控屏50，被配置为接收手绘图形的输入，在用户于所述触控屏上手绘图形时产生包括多个轨迹点的信息序列；和前述的手 绘图形识别装置40。
在一些实施例中，手绘图形识别装置40包括：第一处理器41，位于终端，被配置为根据输入的笔划的坐标数据，确定笔划集合是否构成闭合图形；第二处理器42，位于云端，被配置为在笔划集合构成闭合图形的情况下，基于笔划的坐标数据，识别闭合图形的类别，或者被配置为定位手绘图形的关键点。
当然，手绘图形识别装置40的第一处理器41和第二处理器42也可以都位于云服务端，对接收到的手绘轨迹点信息序列进行手绘图形识别，并将图形识别结果传给指定的接收方，例如终端。
图6是示出用于实现本公开一些实施例的计算机系统的框图。
如图6所示，计算机系统可以通用计算设备的形式表现，该计算机系统可以用来实现上述实施例的手绘图形识别装置。计算机系统包括存储器610、处理器620和连接不同系统组件的总线600。
存储器610例如可以包括系统存储器、非易失性存储介质等。系统存储器例如存储有操作系统、应用程序、引导装载程序(Boot Loader)以及其他程序等。系统存储器可以包括易失性存储介质，例如随机存取存储器(RAM)和/或高速缓存存储器。非易失性存储介质例如存储有执行显示方法的对应实施例的指令。非易失性存储介质包括但不限于磁盘存储器、光学存储器、闪存等。
处理器620可以用通用处理器、数字信号处理器(DSP)、应用专用集成电路(ASIC)、现场可编程门阵列(FPGA)或其它可编程逻辑设备、分立门或晶体管等分立硬件组件方式来实现。相应地，诸如判断设备和确定设备的每个设备，可以通过中央处理器(CPU)运行存储器中执行相应步骤的指令来实现，也可以通过执行相应步骤的专用电路来实现。
总线600可以使用多种总线结构中的任意总线结构。例如，总线结构包括但不限于工业标准体系结构(ISA)总线、微通道体系结构(MCA)总线、外围组件互连(PCI)总线。
计算机系统还可以包括输入输出接口630、网络接口640、存储接口650等。这些接口630、640、650以及存储器610和处理器620之间可以通过总线600连接。输入输出接口630可以为显示器、鼠标、键盘等输入输出设备提供连接接口。网络接口640为各种联网设备提供连接接口。存储接口640为软盘、U盘、SD卡等外部存储设备提供连接接口。
至此，已经详细描述了本公开的各种实施例。为了避免遮蔽本公开的构思，没有描述本领域所公知的一些细节。本领域技术人员根据上面的描述，完全可以明白如何实施这里公开的技术方案。
虽然已经通过示例对本公开的一些特定实施例进行了详细说明，但是本领域的技术人员应该理解，以上示例仅是为了进行说明，而不是为了限制本公开的范围。本领域的技术人员应该理解，可在不脱离本公开的范围和精神的情况下，对以上实施例进行修改或者对部分技术特征进行等同替换。本公开的范围由所附权利要求来限定。

Claims (37)

  1. 一种手绘图形识别方法，包括：
    接收手绘图形的输入，输入包括笔划集合，笔划集合包括至少一个笔划，每个笔划包括多个轨迹点的信息序列，每个轨迹点的信息包括该轨迹点的坐标和书写时间；
    根据输入的笔划的坐标数据，确定笔划集合是否构成闭合图形；
    在笔划集合构成闭合图形的情况下，基于笔划的坐标数据，识别闭合图形的类别。
  2. 根据权利要求1所述的手绘图形识别方法，其中，笔划集合包括多个笔划，每个笔划包括两个端点，每个轨迹点的信息还包括该轨迹点的书写状态值，其中，所述书写状态值用于表示轨迹点的书写状态，所述轨迹点的书写状态包括抬笔和落笔，根据输入的笔划的坐标数据，确定笔划集合是否构成闭合图形包括：
    判断接收的当前笔划的落笔与上一个笔划的抬笔时间之间的间隔是否小于采集时间阈值；
    在小于采集时间阈值的情况下，将当前笔划作为笔划集合的第j个笔划，根据当前笔划的端点与之前的笔划之间的连接关系，确定笔划集合是否构成闭合图形，j为大于1的正整数。
  3. 根据权利要求2所述的手绘图形识别方法，其中，在一个笔划的端点与另一个笔划的任一端点之间的距离小于距离阈值的情况下，一个笔划的端点与另一个笔划之间构成连接关系，根据当前笔划的端点与之前的笔划之间的连接关系，确定笔划集合是否构成闭合图形包括：
    在当前笔划的一个端点与第j-m个笔划构成连接关系，且当前笔划的另一个端点与第j-n个笔划构成连接关系的情况下，确定笔划集合构成闭合图形，m、n为正整数且m大于或等于n，
    其中，之前的笔划之间构成连接关系但不构成闭合图形。
  4. 根据权利要求3所述的手绘图形识别方法，其中：
    在当前笔划的任一端点与之前的笔划都不构成连接关系的情况下，确定当前笔划与之前的笔划之间不构成连接关系且笔划集合不构成闭合，不进行图形识别。
  5. 根据权利要求3所述的手绘图形识别方法，其中：
    在当前笔划的一个端点与之前的笔划中的一个笔划构成连接关系，但当前笔划的另一个端点与其他笔划都不构成连接关系的情况下，确定当前笔划与之前的笔划之间构成连接关系但笔划集合不构成闭合图形，并确定采集时间阈值范围内是否接收到第j+1个笔划；
    在采集时间阈值范围内接收到第j+1个笔划的情况下，将j+1个笔划作为笔画集合的当前笔划，根据当前笔划的端点与之前的笔划之间的连接关系，确定笔划集合是否构成闭合图形；
    在采集时间阈值范围内没有接收到第j+1个笔划的情况下，不进行图形识别。
  6. 根据权利要求3所述的手绘图形识别方法，还包括在存储区中存储笔划的端点值，其中：
    存储区中存储两个端点值，所述两个端点值包括第一端点值和第二端点值；
    对于第1个笔划，第1个笔划的两个端点值分别为第一端点值和第二端点值；
    对于第j个笔划，根据第j个笔划的端点值与存储区中的第一端点值和第二端点值之间的关系，确定第j个笔划与之前的笔划之间的连接关系，
    在第j个笔划与之前的笔划构成连接关系但不构成闭合图形的情况下，在存储区中用第j个笔划中与之前的笔划不连接的端点值替代之前的笔划中与第j个笔划连接的端点值。
  7. 根据权利要求1所述的手绘图形识别方法，其中，笔划集合包括多个连续的笔划，每个笔划包括两个端点，在多个连续的笔划中，每一个笔划的落笔与上一个笔划的抬笔时间之间的间隔都小于采集时间阈值，根据输入的笔划的坐标数据，确定笔划集合是否构成闭合图形包括：
    根据时间顺序依次将多个连续的笔划中的笔划作为待处理笔划，根据待处理笔划的端点与之前的笔划之间的连接关系，确定笔划集合是否构成闭合图形，所述待处理笔划为笔划集合中的第j个笔划，j为大于1的正整数。
  8. 根据权利要求7所述的手绘图形识别方法，其中，在一个笔划的端点与另一个 笔划的任一端点之间的距离小于距离阈值的情况下，一个笔划的端点与另一个笔划之间构成连接关系，根据待处理笔划的端点与之前的笔划之间的连接关系，确定笔划集合是否构成闭合图形包括：
    在待处理笔划的一个端点与第j-m个笔划构成连接关系，且待处理笔划的另一个端点与第j-n个笔划构成连接关系的情况下，确定笔划集合构成闭合图形，m、n为正整数且m大于或等于n；
    在待处理笔划的任一端点与之前的笔划都不构成连接关系的情况下，确定待处理笔划与之前的笔划之间不构成连接关系且笔划集合也不构成闭合图形；
    在待处理笔划的一个端点与之前的笔划中的一个笔划构成连接关系，但待处理笔划的另一个端点与其他笔划都不构成连接关系的情况下，确定待处理笔划与之前的笔划之间构成连接关系但笔划集合不构成闭合图形，
    其中，之前的笔划之间构成连接关系但不构成闭合图形。
  9. 根据权利要求8所述的手绘图形识别方法，其中：
    在确定待处理笔划与之前的笔划之间构成连接关系但笔划集合不构成闭合图形的情况下，将j+1个笔划作为笔画集合的待处理笔划，根据待处理笔划的端点与之前的笔划之间的连接关系，确定笔划集合是否构成闭合图形；
    在确定待处理笔划与之前的笔划之间不构成连接关系且笔划集合不构成闭合图形的情况下，不进行图形识别。
  10. 根据权利要求1至6任一项所述的手绘图形识别方法，还包括：
    对于笔划集合中的第1个笔划，识别该笔划构成的图形的类别，当识别出的类别为指定图形时，执行针对指定图形预设的手写笔迹处理操作。
  11. 根据权利要求10所述的手绘图形识别方法，其中：
    当识别出的类别不是指定图形时，确定所述笔划是否构成闭合图形；
    当构成闭合图形时，识别闭合图形的类别；
    当不构成闭合图形时，确定采集时间阈值范围内是否接收到第2个笔划；
    在采集时间阈值范围内接收到第2个笔划的情况下，将第2个笔划作为当前笔划，判断第2个笔划与第1个笔划是否构成闭合图形。
  12. 根据权利要求10所述的手绘图形识别方法，其中，所述手写笔迹处理操作包括：擦除、撤销、标记、或着重显示。
  13. 根据权利要求10所述的手绘图形识别方法，其中，所述指定图形包括：倒“N”形、多折线或“Z”字形。
  14. 根据权利要求13所述的手绘图形识别方法，其中，在所述指定图形为“Z”字形的情况下，执行擦除操作。
  15. 根据权利要求1至14中任一项所述的手绘图形识别方法，还包括：
    根据闭合图形的类别，输出与手绘图形对应的校正绘制图形。
  16. 根据权利要求1至14中任一项所述的手绘图形识别方法，还包括：
    定位手绘图形的关键点；
    根据闭合图形的类别和关键点，输出与手绘图形对应的校正绘制图形。
  17. 根据权利要求16所述的手绘图形识别方法，其中，所述关键点为角点。
  18. 根据权利要求1至14中任一项所述的手绘图形识别方法，还包括：
    根据输入的笔划的坐标数据，确定笔划的长度是否在预设长度范围内；
    在笔划的长度在预设长度范围内的情况下，利用该笔划进行闭合图形的识别；
    在笔划的长度超出预设长度范围内的情况下，不进行图形识别。
  19. 根据权利要求1至14任一项所述的手绘图形识别方法，其中，识别闭合图形的类别包括：
    利用深度学习算法，识别闭合图形的类别。
  20. 根据权利要求19所述的手绘图形识别方法，其中，深度学习算法包括循环神经网络。
  21. 根据权利要求1至19任一项所述的手绘图形识别方法，其中，图形的类别包括多边形、圆形、椭圆形中的至少一种。
  22. 根据权利要求1至21任一项所述的手绘图形识别方法，还包括：
    在识别闭合图形的类别之前，根据预定距离阈值对轨迹点进行稀疏采样，得到轨迹点集合；和/或
    根据闭合图形的类别识别所需的轨迹点数目，对所述轨迹点集合进行稀疏采样。
  23. 根据权利要求1至22任一项所述的手绘图形识别方法，其中：
    终端侧根据输入的笔划的坐标数据，确定笔划集合是否构成闭合图形，并在笔划集合构成闭合图形的情况下，将相应的笔划的坐标数据发送给云端；
    云端侧基于笔划的坐标数据，识别闭合图形的类别。
  24. 一种手绘图形识别方法，包括：
    接收手绘图形的输入，输入包括笔划集合，笔划集合包括至少一个笔划，每个笔划包括多个轨迹点的信息序列，每个轨迹点的信息包括该轨迹点的坐标、书写时间；
    根据输入的笔划的坐标数据，识别手绘图形的类别；
    定位手绘图形的关键点；
    根据手绘图形的类别和关键点，输出与手绘图形对应的校正绘制图形。
  25. 根据权利要求24所述的手绘图形识别方法，其中，定位手绘图形的关键点包括：
    基于轨迹点之间的坐标偏移量，定位手绘图形的关键点。
  26. 根据权利要求25所述的手绘图形识别方法，其中，基于轨迹点之间的坐标偏移量，定位手绘图形的关键点包括：
    计算轨迹点之间的坐标偏移量，坐标偏移量包括横坐标偏移量和纵坐标偏移量；
    根据轨迹点之间的横坐标偏移量和纵坐标偏移量的大小，将坐标偏移量划分到多个区间；
    根据坐标偏移量所属区间之间的关系，定位手绘图形的关键点。
  27. 根据权利要求26所述的手绘图形识别方法，其中，对于包括轨迹点P i、P i+1、P i+2……P i+n的信息序列，设轨迹点P i+n+1和P i+n之间的坐标偏移量为O n，轨迹点P i+n和P i+n-1之间的坐标偏移量为O n-1，坐标偏移量O n与O n-1为相邻的坐标偏移量，根据坐标偏移量所属区间之间的关系，定位手绘图形的关键点包括：
    在相邻的坐标偏移量O n与O n-1所属区间之间有区间间隔的情况下，基于相邻的坐标偏移量O n与O n-1共同对应的轨迹点P i+n，定位手绘图形的关键点，其中i为正整数，n为大于1的正整数。
  28. 根据权利要求27所述的手绘图形识别方法，其中，根据坐标偏移量所属区间之间的关系，定位手绘图形的关键点还包括：
    在相邻的坐标偏移量O n与O n-1所属区间相邻的情况下，确定O n对应的轨迹点P i+n和P i+n+1的连线与O n-1对应的轨迹点P i+n-1和P i+n的连线之间的第一夹角是否大于第一角度阈值；
    在第一夹角大于第一角度阈值的情况下，基于相邻的坐标偏移量O n与O n-1共同对应的轨迹点P i+n，定位手绘图形的关键点。
  29. 根据权利要求28所述的手绘图形识别方法，其中，根据坐标偏移量所属区间之间的关系，定位手绘图形的关键点还包括：
    在第一夹角不大于第一角度阈值的情况下，或者在相邻的坐标偏移量O n与O n-1所属区间相同的情况下，若偏移量O n-k与O n-1之间的偏移量与O n-1所属区间相同、但偏移量O n-k与O n-1所属区间不同，确定偏移量O n-k+1对应的轨迹点P i+n-k+1或P i+n-k+2和O n对应的轨迹点P i+n的连线与O n对应的轨迹点P i+n和P i+n+1的连线之间的第二夹角是否大于第二角度阈值，k为大于2的正整数，n为大于3的正整数；
    在第二夹角大于第二角度阈值的情况下，基于相邻的坐标偏移量O n与O n-1共同对应的轨迹点P i+n，定位手绘图形的关键点。
  30. 根据权利要求28所述的手绘图形识别方法，其中，根据坐标偏移量所属区间之间的关系，定位手绘图形的关键点还包括：
    在第一夹角不大于第一角度阈值的情况下，或者在相邻的坐标偏移量O n与O n-1所属区间相同的情况下，若偏移量O n+k与O n之间的偏移量与O n所属区间相同、但偏 移量O n+k与O n所属区间不同，确定偏移量O n+k-1对应的轨迹点P i+n+k-1或P  i+n+k和O n-1对应的轨迹点P i+n的连线与O n-1对应的轨迹点P i+n-1和P i+n的连线之间的第二夹角是否大于第二角度阈值，k为大于2的正整数，n为大于3的正整数；
    在第二夹角大于第二角度阈值的情况下，基于相邻的坐标偏移量O n与O n-1共同对应的轨迹点P i+n，定位手绘图形的关键点。
  31. 根据权利要求25至30任一项所述的手绘图形识别方法，其中，根据坐标偏移量所属区间之间的关系，定位手绘图形的关键点还包括：
    根据坐标偏移量所属区间，对轨迹点之间的横坐标偏移量和纵坐标偏移量分别进行重赋值；
    根据重赋值，确定坐标偏移量所属区间之间的关系。
  32. 根据权利要求24至30任一项所述的手绘图形识别方法，其中，所述关键点为角点。
  33. 根据权利要求24至32任一项所述的手绘图形识别方法，还包括：
    在定位手绘图形的关键点之前，根据预定距离阈值对轨迹点进行稀疏采样，得到轨迹点集合；和/或
    根据手绘图形的关键点定位所需的轨迹点数目，对所述轨迹点集合进行稀疏采样。
  34. 一种手绘图形识别装置，包括：
    存储器；和
    耦接至所述存储器的处理器，所述处理器被配置为基于存储在所述存储器装置中的指令，执行权利要求1至33任一项所述的手绘图形识别方法的一个或多个步骤。
  35. 一种手绘图形识别系统，包括：
    触控屏，被配置为接收手绘图形的输入，在用户于所述触控屏上手绘图形时产生包括多个轨迹点的信息序列；
    如权利要求34所述的手绘图形识别装置。
  36. 根据权利要求35所述的手绘图形识别系统，其中，所述手绘图形识别装置包括：
    第一处理器，位于终端，被配置为根据输入的笔划的坐标数据，确定笔划集合是否构成闭合图形；
    第二处理器，位于云端，被配置为在笔划集合构成闭合图形的情况下，基于笔划的坐标数据，识别闭合图形的类别，或者被配置为定位手绘图形的关键点。
  37. 一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现如权利要求1至33任一项所述的手绘图形识别方法的一个或多个步骤。
PCT/CN2021/088951 2021-04-22 2021-04-22 手绘图形识别方法、装置和系统，以及计算机可读存储介质 WO2022222096A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
PCT/CN2021/088951 WO2022222096A1 (zh) 2021-04-22 2021-04-22 手绘图形识别方法、装置和系统，以及计算机可读存储介质
US17/636,629 US20230351787A1 (en) 2021-04-22 2021-04-22 Hand-Drawn Graphic Recognition Method, Apparatus and System, and Computer-Readable Storage Medium
CN202180000861.8A CN115589786A (zh) 2021-04-22 2021-04-22 手绘图形识别方法、装置和系统，以及计算机可读存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/088951 WO2022222096A1 (zh) 2021-04-22 2021-04-22 手绘图形识别方法、装置和系统，以及计算机可读存储介质

Publications (1)

Publication Number Publication Date
WO2022222096A1 true WO2022222096A1 (zh) 2022-10-27

Family

ID=83723545

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/088951 WO2022222096A1 (zh) 2021-04-22 2021-04-22 手绘图形识别方法、装置和系统，以及计算机可读存储介质

Country Status (3)

Country Link
US (1) US20230351787A1 (zh)
CN (1) CN115589786A (zh)
WO (1) WO2022222096A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116074639A (zh) * 2023-01-17 2023-05-05 北京达佳互联信息技术有限公司 图像生成方法、装置、电子设备及存储介质
CN116630994B (zh) * 2023-07-19 2023-10-20 腾讯科技（深圳）有限公司 输入法中的笔势识别方法、相关装置和介质
CN117315665A (zh) * 2023-11-30 2023-12-29 上海又寸科技有限公司 一种基于原笔迹识别的题目自动批阅方法及系统

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101944175A (zh) * 2009-07-07 2011-01-12 华中师范大学 物理运动学实验的动态几何虚拟方法
CN105117054A (zh) * 2015-08-12 2015-12-02 珠海优特电力科技股份有限公司 一种手写输入的识别方法及系统
CN105719328A (zh) * 2016-01-19 2016-06-29 华中师范大学 一种手绘几何图形规范化方法及系统
CN111626238A (zh) * 2020-05-29 2020-09-04 京东方科技集团股份有限公司 文本识别方法、电子设备及存储介质
CN111898593A (zh) * 2020-09-29 2020-11-06 湖南新云网科技有限公司 一种几何图形的形状识别方法、装置、设备及存储介质
CN112181207A (zh) * 2020-10-30 2021-01-05 海信视像科技股份有限公司 一种显示设备及几何图形识别方法

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3567597A1 (en) * 2018-05-11 2019-11-13 Anoto Korea Corp. Method and apparatus of diagnostic test

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101944175A (zh) * 2009-07-07 2011-01-12 华中师范大学 物理运动学实验的动态几何虚拟方法
CN105117054A (zh) * 2015-08-12 2015-12-02 珠海优特电力科技股份有限公司 一种手写输入的识别方法及系统
CN105719328A (zh) * 2016-01-19 2016-06-29 华中师范大学 一种手绘几何图形规范化方法及系统
CN111626238A (zh) * 2020-05-29 2020-09-04 京东方科技集团股份有限公司 文本识别方法、电子设备及存储介质
CN111898593A (zh) * 2020-09-29 2020-11-06 湖南新云网科技有限公司 一种几何图形的形状识别方法、装置、设备及存储介质
CN112181207A (zh) * 2020-10-30 2021-01-05 海信视像科技股份有限公司 一种显示设备及几何图形识别方法

Also Published As

Publication number Publication date
CN115589786A (zh) 2023-01-10
US20230351787A1 (en) 2023-11-02

Similar Documents

Publication Publication Date Title
WO2022222096A1 (zh) 手绘图形识别方法、装置和系统，以及计算机可读存储介质
US10679146B2 (en) Touch classification
WO2019128646A1 (zh) 人脸检测方法、卷积神经网络参数的训练方法、装置及介质
WO2020215565A1 (zh) 手部图像的分割方法、装置及计算机设备
US9384398B2 (en) Method and apparatus for roof type classification and reconstruction based on two dimensional aerial images
CN106650648B (zh) 擦除笔迹的识别方法和系统
US11823474B2 (en) Handwritten text recognition method, apparatus and system, handwritten text search method and system, and computer-readable storage medium
CN109697414B (zh) 一种文本定位方法及装置
JP2011204141A (ja) 手書き入力装置、手書き入力方法及び手書き入力プログラム
KR101559502B1 (ko) 실시간 손 포즈 인식을 통한 비접촉식 입력 인터페이스 방법 및 기록 매체
TWI637325B (zh) 使用多顏色通道無記號偵測之便箋辨識與管理
CN111492407B (zh) 用于绘图美化的系统和方法
WO2021196013A1 (zh) 单词识别方法、设备及存储介质
CN114494751A (zh) 证照信息识别方法、装置、设备及介质
CN113888546A (zh) 手绘图形的规整方法、电子设备及存储介质
US7295707B2 (en) Method for aligning gesture features of image
US20150199033A1 (en) Method for simulating a graphics tablet based on pen shadow cues
CN110737364B (zh) 一种安卓系统下触摸书写加速的控制方法
TWI507919B (zh) 追蹤與記錄指尖軌跡的影像處理方法
Shankar et al. Sketching in three dimensions: A beautification scheme
WO2020093329A1 (zh) 一种终端设备的数据输入方法、终端设备及存储介质
CN111104886A (zh) 手势识别方法、装置、设备和存储介质
US20150142784A1 (en) Retrieval device and method and computer program product
CN115359499A (zh) 数字图像处理方法及装置
JP6696868B2 (ja) 情報処理装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21937335

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 19.02.2024)

122 Ep: pct application non-entry in european phase

Ref document number: 21937335

Country of ref document: EP

Kind code of ref document: A1