CN113869308A

CN113869308A - Pattern recognition method and device, storage medium and electronic equipment

Info

Publication number: CN113869308A
Application number: CN202111124598.1A
Authority: CN
Inventors: 林文松
Original assignee: Beijing Kingsoft Office Software Inc; Zhuhai Kingsoft Office Software Co Ltd; Wuhan Kingsoft Office Software Co Ltd
Current assignee: Beijing Kingsoft Office Software Inc; Zhuhai Kingsoft Office Software Co Ltd; Wuhan Kingsoft Office Software Co Ltd
Priority date: 2021-09-24
Filing date: 2021-09-24
Publication date: 2021-12-31

Abstract

The invention discloses a pattern recognition method and device, a storage medium and electronic equipment. Wherein, the method comprises the following steps: acquiring a screenshot of a candidate graph currently drawn on an operation interface and a drawing track point set of the candidate graph; determining a first-level graph category to which the candidate graph belongs based on the screenshot; identifying a second level graph category from the subcategories of the first level graph category based on the set of drawn trace points; and displaying the target graph fitted according to the second-level graph category in the operation interface. The invention solves the technical problem of lower identification accuracy caused by single identification dimension.

Description

Pattern recognition method and device, storage medium and electronic equipment

Technical Field

The invention relates to the field of computers, in particular to a method and a device for recognizing a graph, a storage medium and electronic equipment.

Background

Nowadays, many designers begin to directly use drawing software in a tablet personal computer to complete drawing, and the drawing task can be completed by the designers at any time and any place. However, currently available drawing software provides too few graphics to support in the canvas function. Unlike the operation habits of most users, many graphics support only a single-dimensional recognition setting.

That is to say, the pattern recognition method provided by the prior art has the problem that the recognition dimension is relatively single, so that a part of the pattern drawn by the user cannot be accurately recognized.

In view of the above problems, no effective solution has been proposed.

Disclosure of Invention

The embodiment of the invention provides a pattern recognition method and device, a storage medium and electronic equipment, which at least solve the technical problem of low recognition accuracy caused by single recognition dimension.

According to an aspect of an embodiment of the present invention, there is provided a pattern recognition method including: acquiring a screenshot of a candidate graph currently drawn on an operation interface and a drawing track point set of the candidate graph; determining the first-level graph category to which the candidate graph belongs based on the screenshot; identifying a second level graphics category from the sub-categories of the first level graphics category based on the set of drawing trace points; and displaying the target graph fitted according to the second-level graph type in the operation interface.

According to another aspect of the embodiments of the present invention, there is also provided a pattern recognition apparatus, including: the acquisition unit is used for acquiring a screenshot of a candidate graph currently drawn on an operation interface and a drawing track point set of the candidate graph; the determining unit is used for determining the first-level graph type to which the candidate graph belongs based on the screenshot; the recognition unit is used for recognizing a second-level graph category from the subcategories of the first-level graph categories based on the drawing track point set; and the display unit is used for displaying the target graph fitted according to the second-level graph type in the operation interface.

According to a further aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned pattern recognition method when running.

According to still another aspect of the embodiments of the present invention, there is also provided an electronic device, including a memory and a processor, where the memory stores a computer program, and the processor is configured to execute the pattern recognition method through the computer program.

In the embodiment of the invention, the screenshot of the candidate graph drawn on the operation interface and the drawing track point set of the candidate graph are obtained, the first-level graph category to which the candidate graph belongs is determined based on the screenshot, then the first-level graph category is analyzed by combining the drawing track point set, and the second-level graph category corresponding to the candidate graph is further determined, so that the target graph to be displayed is fitted based on the second-level graph category. That is to say, different fine categories are made based on the drawing track point set on the operation interface, so that the drawn graph can be more accurately identified, the effect of improving the accuracy of the identification result is achieved, and the problem of lower accuracy of the graph identification result in the related technology is solved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:

FIG. 1 is a flow diagram of an alternative pattern recognition method according to an embodiment of the present invention;

FIG. 2 is a diagram illustrating an effect of an alternative pattern recognition method according to an embodiment of the present invention;

FIG. 3 is a diagram illustrating the effects of an alternative pattern recognition method according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of an alternative pattern recognition apparatus according to an embodiment of the present invention;

fig. 5 is a schematic structural diagram of an alternative electronic device according to an embodiment of the present invention.

Detailed Description

In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

According to an aspect of an embodiment of the present invention, there is provided a pattern recognition method, as shown in fig. 1, the method including:

s102, acquiring a screenshot of a candidate graph drawn on an operation interface at present and a drawing track point set of the candidate graph;

s104, determining the first-level graph category to which the candidate graph belongs based on the screenshot;

s106, identifying a second-level graph category from the subcategories of the first-level graph category based on the drawing track point set;

and S108, displaying the target graph fitted according to the second-level graph type in the operation interface.

Optionally, in this embodiment, the above-mentioned graph recognition method may be applied, but not limited, to a graph drawing tool software application, and when a user draws a graph on a canvas provided by the tool software, an image of the currently drawn graph (which may be referred to as a screenshot) may be intercepted, and a set of drawing trace points generated in the drawing process may be retained. And identifying and classifying the category of the graph in the screenshot by adopting a classification model, further analyzing by combining a track point set after the category is confirmed, determining a specific graph parameter corresponding to the graph, and displaying the generated graph on a canvas according to the specific parameter.

It should be noted that the first-level graphics category may be, but is not limited to, a large category of graphics categories, such as: straight lines, broken lines, rectangles, polygons, circles, ellipses, arrowheads, and the like. The second-level graphics category may be, but is not limited to, a subclass of graphics categories, and for example, a rectangle category may include: parallelogram, square, rectangle, rhombus, etc. Here, for example, the second level graphics categories may also include, but are not limited to: heart shape, pentagram, right angle unilateral arrow, right angle bilateral arrow, straight line bilateral arrow, pitch arc bilateral arrow etc. do not do any restriction in this embodiment.

Optionally, in this embodiment, the classification model may be, but is not limited to, a lightweight neural network classification algorithm, which is mainly applied to mobile terminals and embedded devices, so as to facilitate direct invocation, for example, MobileNet _ V2.

By the embodiment provided by the application, when the graph is drawn on the operation interface, the screenshot of the drawn candidate graph and the drawing track point set thereof can be obtained, the classification model is adopted to identify and classify the category of the candidate graph in the screenshot, the first-level graph category is confirmed and then the drawing track point set is combined for analysis, the second-level graph category corresponding to the candidate graph is further determined, and therefore the target graph to be displayed is fitted based on the second-level graph category. That is to say, the drawing of the graph is not limited to the large-class recognition and classification result, but the drawing of the graph is further classified in detail by combining with the drawing track point set, so that the drawn graph can be recognized more accurately, the effect of improving the accuracy of the recognition result is achieved, and the problem of low accuracy of the graph recognition result in the related technology is solved.

As an alternative, determining the first-level graph category to which the candidate graph belongs based on the screenshot includes:

s1, classifying the screenshot by adopting a classification model to obtain a classification result, wherein the classification model is a neural network model for identifying the type of the graph displayed in the screenshot;

and S2, determining the first-level graph category according to the classification result.

Optionally, in this embodiment, the training preparation process of the classification model may be, but is not limited to, by including the following: marking the sample data in advance to obtain a sample graph with a classification label; the sample pattern is then preprocessed, such as data washing, expansion (scaling fill, rotation, etc.). Furthermore, in order to prevent the distribution of the sample pattern types from being uneven, the quantity statistics can be carried out on each type, and the quantity statistics can be respectively expanded to one hundred thousand in each type to establish a training data set, a testing data set and a verification data set (8: 1: 1).

Training the initial classification model by using the training data set obtained based on the process until a preset training convergence condition is reached, and determining the function in the classification model to be applied by using the current model function reaching the convergence condition. The screenshots are then classified and identified based on the classification model to determine the large categories in which the screenshots are located, wherein the total number of the large categories can be 11 categories, such as rectangles, circles, straight lines and the like.

Through the embodiment that this application provided, adopt classification model to carry out the primary classification to the screenshot to carry out the primary screening to the recognition result of candidate figure, so that follow-up can determine meticulous classification result more fast high-efficiently, thereby reach the purpose that improves the efficiency to figure recognition.

As an optional scheme, classifying the screenshot by using a classification model, and obtaining a classification result includes:

s1, adjusting the input size of the screenshot to a target size, wherein the size information of the target size comprises: the display length of the screenshot, the display width of the screenshot and the display color value of the screenshot;

and S2, inputting the screenshots under the target size into the classification model to obtain a classification result, wherein the application format of the model framework used in the classification model is a format which is allowed to be called by the mobile device or the embedded device.

Optionally, in this embodiment, the classification algorithm used in the classification model may be, but is not limited to, MobileNet _ V2, MobileNet _ V2 is a lightweight neural network classification algorithm, which is a network constructed based on deep-level separable convolution, and splits the standard convolution into two operations: depth convolution (Depthwise convolution) is different from standard convolution, in which the convolution kernel is used on all input channels (input channels), and point-by-point convolution (position convolution), which uses a different convolution kernel for each input channel, that is, one convolution kernel corresponds to one input channel, and is thus an operation at the level of depth. Instead, the poitwise convolution is simply a normal convolution, but it uses a convolution kernel of 1 × 1.

In addition, the mobileNetV2 is mainly applied to a mobile terminal, a trained model is converted into a tflite model by using a tenserflow frame, the model in the tflite format can be directly called for mobile and embedded devices, the final test accuracy can reach more than 99%, the size of the model is 607k, and the time for processing a single image is about 200 ms.

Optionally, in this embodiment, before the screenshot is input into the classification model for classification, the size of the screenshot may be adjusted, but is not limited to be adjusted, so that the screenshot is suitable for a data processing process inside the current classification model. The target size may be, but is not limited to, set to 224 x 3, i.e., 224 pixels each in length and width, color value RGB.

According to the embodiment provided by the application, the screenshot is subjected to size adjustment in advance so that the screenshot is suitable for a processing mode of a model frame in a classification model, and the problem of inaccurate identification caused by non-uniform sizes is avoided. Thereby improving the accuracy of the classification result of the classification model.

As an alternative, identifying a second level graphics category from the subcategories of the first level graphics category based on the set of drawing trace points comprises:

s1, performing feature recognition on the drawing track point set to obtain a graph fitting parameter;

and S2, determining the second-level graph category from the subclasses of the first-level graph category according to the graph fitting parameters.

Optionally, in this embodiment, the graph fitting parameter may be, but is not limited to, a parameter for fitting a drawn graph. For example, when four corner points of a quadrangle are known, the quadrangle can be drawn, and coordinates of the four corner points can be used as a graph fitting parameter (also referred to as a key parameter) of the quadrangle. For another example, the figure fitting parameters of the circle may include the center of the circle, the radius, and the like. The figure fitting parameters of the ellipse may include a major and minor semi-axis, a central point, and a rotation angle. Here, this is an example, and this is not limited in this embodiment. After the second-level graph type is determined based on the graph fitting parameters, the corresponding target graph can be drawn based on the graph fitting parameters, and the drawing result can be shown in fig. 2 to fig. 3.

For example, in the case that the first-level graph category to which the candidate graph belongs is identified to be an ellipse based on the screenshot in the classification model, a least square method is used for fitting a drawing track point set of the ellipse to obtain information of a major semi-axis, a minor semi-axis, a central point and a rotation angle of the ellipse after fitting. Judging the proportion of the long half shaft and the short half shaft, and judging the long shaft: if the short axis is less than 1.2, judging that the graph in the screenshot is circular, and returning circular parameters such as the circle center (central point) and the radius (the sum of the major and minor semi-axes is divided by 2); long axis: and if the minor axis is more than or equal to 1.2, judging that the graph in the screenshot is an ellipse, and returning ellipse parameters such as major and minor semiaxes, a central point and a rotation angle.

The description is made with specific reference to the following examples:

the identification process of the rectangle can be as follows:

searching a Convex Hull (Convex Hull) is defined as that a subset S (corresponding graph) of a plane is called as Convex, wherein the Convex Hull track refers to that if and only if any two points p, S belongs to S, a line segment ps completely belongs to S in the track point set, so that the angular points of the Convex Hull are obtained, and if the number of the angular points is not equal to 4, an error is reported;

obtaining a graphic area according to the angular point; obtaining a minimum circumscribed rectangle (rectangle center, length and width, rotation angle) of the graph according to the point set, and calculating the area minAreaRect _ area of the rectangle; calculating the value rate of area/minAreaRect _ area;

calculating the number num of four corners of the graph close to 90 degrees (80-100 degrees) according to the four corner points;

if rate is greater than 0.7 and num is greater than 2, judging the rectangle (rectangle/square), and simultaneously calculating the width-height ratio of the circumscribed rectangle, if the width-length ratio belongs to (0.8,1.2), the circumscribed rectangle is square, otherwise, the circumscribed rectangle is rectangular; if rate is less than 0.9 and num is 0, judging the parallelogram; otherwise, reporting an error.

If the graph is determined to be a square graph based on the graph fitting parameters, the side length is (length + width)/2, the central point is the central point of the external rectangle, and the rotation angle is the rotation angle of the external rectangle; a square is drawn on the operator interface.

If the graph is determined to be a rectangular graph based on the graph fitting parameters, the length, the width, the central point and the rotating angle are consistent with the minimum circumscribed rectangle; a rectangle is drawn on the operation interface.

If the graph is determined to be a parallelogram type graph based on the graph fitting parameters, determining the accurate position of a fourth corner point according to the first three corner points, calculating the rotation angle of the parallelogram, and converting to obtain coordinates of the four corner points after rotation; and connecting lines in pairs according to coordinates of the four corner points to draw a parallelogram.

Secondly, the process of identifying the arc can be as follows:

the key parameters for identifying the type of graph comprise: two points at the head and the tail, and arc control points.

Obtaining a head point and a tail point according to the drawing track point set, calculating a point which is farthest from a connecting line of the head point and the tail point in the point set, namely a farthest point of an arc line, and converting the farthest point according to a Bezier curve formula to obtain a control point; and identifying the graph fitting parameters of the arc graphs based on the position relation of the points. An arc is drawn based on the position coordinates of the respective points.

Thirdly, the process of identifying the straight line can be as follows:

the key parameters for identifying the type of graph comprise: two intersections of the straight line with the canvas.

Fitting a vector representation (cosk, sink) of the slope k of the straight line to a point coordinate (x, y) on the straight line according to the cv2.fitline () function and the trajectory drawing point set;

and calculating two intersection points of the straight line and the canvas boundary according to the slope k, the coordinate (x, y) of one point and the size of the input graph, and drawing the straight line according to the intersection points, wherein the size of the input graph limits the size of the canvas to prevent the straight line from being drawn out of the border.

The identification process of the circle can be as follows:

and fitting the ellipse by using a least square method according to the drawn track point set to obtain the central point, the major axis, the minor axis and the rotation angle of the ellipse, recording the ratio of the major axis to the minor axis, and recording the ellipse as a circle if the major axis and the minor axis belong to the range of (0.8 and 1.2), otherwise, recording the ellipse as an ellipse.

Key parameters for segment identification include: center point, major and minor axis distances, and angle of rotation.

If the figure is determined to be an ellipse based on the figure fitting parameters, drawing an ellipse according to the central point, the major axis, the minor axis and the rotation angle;

if the graph is determined to be a circular graph based on the graph fitting parameters, the radius is determined to be (long axis + short axis)/4, and a circle is drawn according to the central point and the radius.

Five, triangle

The key parameters for identifying the type of graph comprise: three vertices.

And obtaining the optimal outer triangle of the graph by a cv2. minEnclosetrigle () function, returning the coordinates of three vertexes, and drawing the triangle after connecting two lines.

Six, straight line single side arrow

The key parameters for identifying the type of graph comprise: two points at the head and the tail of the straight line, the vertex position of the arrow and the direction of the single arrow.

Calculating the included angle of a vector formed between two points in the drawing trace point set; and traversing the whole point set, wherein the point with the largest included angle is the vertex of the arrow. And (3) storing a point set between the starting point and the vertex of the arrow as a straight line trunk, wherein the straight line fitting process can be referred to in the straight line fitting method.

Seven, arc single side arrow

The key parameters for identifying the type of graph comprise: two points at the head and the tail of the arc, arc control points, arrow vertex positions and single arrow directions.

And calculating an included angle of a vector formed between two points in the drawing trace point set, traversing the whole point set, and taking the point with the largest included angle as an arrow vertex. The point set between the starting point and the arrow vertex is saved as the main trunk of the arc, and the method for fitting the arc can refer to the process for fitting the arc.

Through the embodiment provided by the application, the candidate graphs in the screenshot are subjected to feature recognition by combining with the drawing track point set so as to determine the specific parameters for fitting the graphs, so that the fitted target graphs are accurately drawn based on the specific parameters, and the accuracy of the graph drawing result is further ensured.

As an optional scheme, performing feature recognition on the set of plotted track points to obtain a graph fitting parameter includes:

s1, acquiring the graph corner points of the convex hull track under the condition of identifying the convex hull track from the drawing track point set;

and S2, determining the position coordinates of the graph corner points as the graph fitting parameters for fitting and generating the N-shaped class graph under the condition that the total number of the graph corner points reaches N.

Optionally, in this embodiment, the value of N may be 4 or 5. When N is 4, the N-polygon may be a quadrangle, and it is further determined that the N-polygon belongs to a square or a parallelogram according to information such as a side length or an included angle of the N-polygon. When N is 5, the N-polygon may be a pentagon, or the like.

For example, taking a drawing rectangle as an example, searching a convex hull according to a drawing track point set, wherein the convex hull track refers to the track point set, and only if for any two points p, S ∈ S, the line segment ps all belongs to S, so as to obtain the corner points of the convex hull, and if the number of the corner points is not equal to 4, an error is reported; if the number of corner points is equal to 4, it is further determined which type of quadrangle it is.

For another example, taking the drawing of a five-pointed star as an example, the key parameters for identifying the type of graphics include: five vertices. Searching and identifying convex hull tracks (which can be called as convex hulls for short) in the drawing track point set to obtain graph angular points, and reporting errors if the total number of the obtained graph angular points is not five angular points; and if the number of the five corner points is five, determining the position coordinates of the corner points as the figure fitting parameters of the five-pointed star, and connecting lines in pairs on the basis of the position coordinates to draw the five-pointed star.

As an optional scheme, performing feature extraction on the set of plotted trajectory points to obtain a graph fitting parameter includes:

s1, determining a straight line fitted according to the drawing track point set as a central line;

s2, fitting the minimum circumscribed triangle of the candidate graph according to the drawing track point set;

s3, determining intersection points between the three side line segments corresponding to the minimum circumscribed triangle and the central line;

s4, sorting distances between the respective midpoint positions of the three edge lines and the intersection points;

s5, determining the edge section corresponding to the minimum distance as the edge section positioned at the top end of the heart shape;

and S6, determining the heart shape direction parameter and the heart shape size parameter which are calculated based on the side line segment at the top end of the heart shape as the graph fitting parameters for fitting and generating the heart shape class graph.

It should be noted that the key parameters for identifying such a pattern include: two points of the heart-shaped groove and the tail part which determine the direction.

Fitting a straight line according to the drawing track point set, namely fitting the center line of the heart shape; fitting the minimum circumscribed triangle of the graph according to the point set to obtain three vertex coordinates of the triangle; and obtaining three edge line segments from the three vertexes, calculating the middle points of the three edge lines, calculating the intersection points of the three edge lines and the central line, sequencing the distances between the three middle points and the intersection points, wherein the distance is the minimum item, and the corresponding edge line is the edge line at the top end of the heart shape. The minimum circumscribed triangle is a closed graph obtained by fitting a connecting line enclosing the track point set.

According to the middle point of the edge line of the top end of the heart shape and the vertex opposite to the edge line (the bottom of the heart shape), the connecting line of the two points can determine the direction and the size of the heart shape. Determining the heart shape direction parameter and the heart shape size parameter as the figure fitting parameters of the heart shape type figure, and drawing the heart shape.

s1, calculating included angles between vectors formed by any two points in the drawing trace point set to obtain a plurality of included angles;

s2, traversing the drawing track point set, and determining the point corresponding to the maximum included angle in the included angles as the vertex of the arrow;

s3, determining a point set between the head position of the initial end of the track and the vertex of the arrow as a right-angle trunk;

and S4, determining the position coordinates of each point on the right-angle trunk and the position coordinates of the arrow top point as the graph fitting parameters for fitting and generating the right-angle unilateral arrow graph.

It should be noted that the key parameters for identifying such a pattern include: two points at the head and the tail of the right angle, turning points, the positions of the top points of the arrows and the direction of the single arrow.

Calculating the included angle of a vector formed between two points in the drawing trace point set; and traversing the whole point set, wherein the point with the largest included angle is the vertex of the arrow. And (3) storing a point set between the starting point and the vertex of the arrow as a right-angle trunk, wherein the right-angle fitting process can be referred to in the right-angle fitting method. Determining the position coordinates of the starting point and the arrow top point as the figure fitting parameters of the right-angle single-sided arrow, and drawing the figure.

The head and tail positions may be, but are not limited to, two end positions in the trace point set, where the head position and the tail position are opposite, and the positions between the head position and the tail position may be interchanged, which is not limited.

s1, acquiring a first position coordinate of a head position located at the initial end of the track and a second position coordinate of a tail position located at the tail end of the track according to the drawn track point set;

s2, determining the position coordinates of the point with the farthest vertical distance relative to the connecting line from the drawing track point set as the position coordinates of the farthest point of the arc line, wherein the connecting line is the connecting line between the first position coordinate and the second position coordinate, and determining the position coordinates of the point with the farthest vertical distance relative to the connecting line between the first position coordinate and the second position coordinate from the drawing track point set as the position coordinates of the turning point;

s3, converting the position coordinate of the farthest point into the position coordinate of the control point;

and S4, determining the first position coordinate, the second position coordinate and the position coordinate of the control point as graph fitting parameters for fitting and generating the polygonal line graph.

Alternatively, the shape of the bezier curve is determined by the starting point, the end point and the control point. When the positions of the starting point and the end point are determined, adjusting the positions of the control points will result in bezier curves of different shapes. For example, the first-order Bezier curve only has a starting point and an end point and has no control point. Higher order bezier curves may produce different curves where the control points will be used to determine the radian of the curve.

It should be noted that the key parameters for identifying the polyline-type graph include: two points at the head and the tail and arc control points.

Obtaining the coordinates of the head point and the tail point (namely the first position coordinate corresponding to the head position and the second position coordinate corresponding to the tail position) according to the drawing track point set, connecting the head point and the tail point, and calculating the point which is farthest from the vertical distance of the connecting line in the point set, namely the position coordinate of the farthest point of the arc line. The position coordinates of the farthest point are converted into the position coordinates of the control point by a bezier curve formula.

And then, fitting a broken line according to the first position coordinates, the second position marks and the position coordinates of the control points.

s1, determining candidate angles according to the first position coordinates, the second position coordinates and the position coordinates of the control points;

s2, performing right-angle correction on the candidate angle to obtain a corrected point position coordinate;

and S3, determining the corrected point position coordinates as graph fitting parameters for fitting and generating a right-angle graph.

It should be noted that, in this embodiment, the right angle correction may be, but is not limited to, correcting an included angle between two lines to obtain a right angle of 90 degrees. For example, assuming that the candidate angle is an angle close to 90 degrees, such as 89 degrees or 92 degrees, the right angle correction can be performed by the above-described right angle correction. For example, a side between the first position coordinates and the position coordinates of the control point is taken as a reference, or a side between the second position coordinates and the position coordinates of the control point is taken as a reference, the corresponding foot and vertical line are found, and the rectangular correction of 90 degrees is performed using a parallel line parallel to the ground and a vertical line perpendicular to the ground.

As an optional scheme, before determining the candidate angle according to the first position coordinate, the second position coordinate, and the position coordinate of the control point, the method further includes:

s1, carrying out straight line detection on the drawn track point set to obtain a detection result;

s2, merging a plurality of line segments in the detection result, and determining two line segments with the maximum length after merging as right-angle edges;

s3, determining a turning point based on the drop foot of the right-angle side;

s4, the first position coordinates and the second position coordinates are determined in the direction in which the square edge extends in the direction opposite to the foot.

It should be noted that the right-angle side is two sides perpendicular to each other, and the drop foot is a turning point of a right angle. Thus, after extending in opposite directions of the foot (i.e., opposite to the direction toward the foot), the coordinates of the leading and trailing positions will be obtained

As an optional scheme, after performing a right-angle correction on the candidate angle to obtain a corrected point position coordinate, the method further includes:

s1, counting a first point number around the first position coordinate and a second point number around the second position coordinate;

s2, determining the top point of the arrow according to the first point and the second point;

and S3, determining the corrected point position coordinates and the position coordinates of the arrow top points as the graph fitting parameters for fitting and generating the right-angle double-sided arrow type graph.

It should be noted that the key parameters for identifying such a pattern include: right angle turning point, right angle head and tail points and arrow top point.

In this embodiment, hough transform is performed on a graph to detect straight line segments, and the line segments are merged; after merging, selecting two edges with the largest length as right-angle edges; obtaining a turning point by solving the drop foot according to the two right-angle edges; the intersection points of the opposite directions of the extending legs of the right-angle side line segments and the frame are two points from head to tail; calculating the angles of two right-angle sides according to the coordinates of the head and the tail points and the three-point coordinates of the turning points, and performing right-angle correction according to the angles to obtain regular coordinates of the turning points; counting the number of black points around the head and the tail points, wherein the large number of the black points is the vertex of an arrow; drawing straight line segments according to connecting lines of three points in pairs, and processing out-of-range points of the canvas; arrows are drawn according to the arrow positions.

The number of points around the head and tail positions may be, but not limited to, a number representing a density of points plotted in a region having a radius of a predetermined length with the head position or the tail position as a center point, thereby determining whether to have an arrow vertex.

s1, in the case that the drawing track point set comprises two subsets, calculating the lengths of two line segments based on the two subsets, and determining the line segment with the maximum length in the two line segments as an arc main body;

s2, determining the top point of the arrow according to the number of points around the two end points of the arc main body;

and S3, determining the position coordinates of the two end points of the arc main body and the position coordinates of the arrow top point as the graph fitting parameters for fitting and generating the arc bilateral arrow type graph.

It should be noted that the key parameters for identifying such a pattern include: three control points with directions, the last control point is an arrow vertex.

In this embodiment, the above-mentioned graph can be defined as, but not limited to, two stroke graphs, one pen is the main arc line, and one pen is the arrow. The trace points of the two strokes are respectively stored in txt and other files, and the length of the line segment of the two point sets (namely two subsets in the trace point set) is calculated, so that the curve with longer length is taken as an arc main body. Then, calculating three control points by the arc main body according to the algorithm flow in the arc; and calculating the arithmetic mean point of the arrow part, wherein the distance between the head point and the tail point of the arc and the arithmetic mean point of the arrow is the vertex of the arrow. The number of surrounding points may be, but is not limited to, a circle area drawn by using points at the head and tail positions as the center and a certain radius as the distance, and is determined as the number of periodic points.

s1, performing straight line fitting on the drawn track point set by using a first preset function to obtain the slope of a fitted candidate straight line and the position coordinates of a target point on the candidate straight line;

s2, calculating the intersection point position coordinates of two intersection points of the straight line and the canvas boundary according to the slope of the candidate straight line, the position coordinates of the target point and the size of the graph for drawing the track point;

and S3, determining the coordinates of the intersection point position as graph fitting parameters for fitting and generating the straight line graph.

It should be noted that the key parameters for identifying such a pattern include: two intersections of the straight line with the canvas.

In the present embodiment, a vector representation (cosk, sink) of the slope k of a straight line is fitted to a point set according to the cv2.fitline () function and a point coordinate (x, y) on the straight line; and calculating two intersection points of the straight line and the canvas boundary according to the slope, the coordinates of one point and the size of the input graph, and drawing the straight line according to the intersection points, wherein the size of the input graph limits the size of the canvas, so that the straight line drawing is prevented from crossing the boundary.

s1, obtaining the distance from each trace point in the trace point set to the candidate straight line;

s2, determining a plurality of track points with the distance larger than a threshold value as reference points;

s3, calculating the average value of the distances corresponding to the reference points;

s4, determining reference distances from two end points of the candidate straight line to a target position respectively, wherein the target position is a position corresponding to the average value of the distances;

s5, determining an endpoint corresponding to the minimum value of the reference distance as an arrow vertex;

and S6, determining the position coordinates of the end points and the position coordinates of the arrow top points as graph fitting parameters for fitting and generating the straight line double-sided arrow type graph.

It should be noted that the key parameters for identifying such a pattern include: two intersections of the straight line with the canvas, the arrow vertices.

In the present embodiment, a vector representation (cosk, sink) of the slope k of a straight line is fitted to a point set according to the cv2.fitline () function and a point coordinate (x, y) on the straight line; calculating two intersection points of the straight line and the canvas boundary according to the slope, the coordinates of one point and the size of the input graph, and drawing the straight line according to the intersection points, wherein the size of the input graph limits the size of the canvas to prevent the straight line from being drawn out of bounds; then, five points of the point set which are farthest from the straight line are obtained, the five points are averaged to obtain a farthest average point, the distance between two end points of the straight line and the farthest average point is judged, and the smaller point is the vertex of the arrow; drawing a straight-line bilateral arrow according to the vertex of the arrow.

The above-mentioned farthest reference distance may be, but not limited to, an endpoint corresponding to each of the first N distance values selected from the distances sorted from large to small.

s1, fitting the drawing track point set by a least square method to obtain a fitting ellipse;

s2, acquiring the position coordinates, major and minor axes and rotation angles of the central point of the fitting ellipse;

s3, calculating the ratio of the major axis to the minor axis, wherein, under the condition that the ratio is in the target value interval, the fitting ellipse is determined to be a circle; determining the fitting ellipse as an ellipse under the condition that the ratio exceeds the target numerical value interval;

and S4, determining the ratio as a graph fitting parameter for fitting to generate a graph of a circular class.

And fitting the ellipse by using a least square method according to the point set to obtain the central point, the major axis, the minor axis and the rotation angle of the ellipse, recording the ratio of the major axis to the minor axis, and recording the ellipse as a circle if the major axis and the minor axis belong to the range of (0.8,1.2), otherwise, recording the ellipse as an ellipse.

It should be noted that the key parameters for identifying such a pattern include: a center point, a long axis distance and a short axis distance, and a rotation angle; if the ellipse is obtained, drawing the ellipse according to the central point, the major axis, the minor axis and the rotation angle; if the circle is formed, the radius is (major axis + minor axis)/4, and the circle is drawn according to the center point and the radius.

s1, calculating vertex coordinates of three vertexes of an outsourcing triangle for drawing the locus point set by using a second preset function;

and S2, determining the vertex coordinates of the three vertexes as graph fitting parameters for fitting and generating the triangle-like graph.

It should be noted that the key parameters for identifying such a pattern include: three vertices.

In this embodiment, the optimal circumscribed triangle of the graph is obtained by the cv2.minenclosingtriangle () function, the coordinates of three vertices are returned, and two-by-two connection lines are drawn to draw the triangle.

It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.

According to another aspect of the embodiment of the invention, a pattern recognition device for implementing the pattern recognition method is also provided. As shown in fig. 4, the apparatus includes:

the acquiring unit 402 acquires a screenshot of a candidate graph currently drawn on the operation interface and a drawing track point set of the candidate graph; is/are as follows

The determining unit 404 determines the first-level graph category to which the candidate graph belongs based on the screenshot;

an identifying unit 406, configured to identify a second level graphics category from the sub-categories of the first level graphics category based on the set of drawing trace points;

and the display unit 408 is used for displaying the target graph fitted according to the second-level graph category in the operation interface.

In this embodiment, reference may be made to the above method embodiment for an embodiment to be implemented by each unit module in the pattern recognition apparatus, which is not described herein again.

According to yet another aspect of the embodiments of the present invention, there is also provided an electronic device for implementing the pattern recognition method, as shown in fig. 5, the electronic device includes a memory 502 and a processor 504, the memory 502 stores a computer program therein, and the processor 504 is configured to execute the steps in any one of the method embodiments through the computer program.

Optionally, in this embodiment, the electronic device may be located in at least one network device of a plurality of network devices of a computer network.

Optionally, in this embodiment, the processor may be configured to execute the following steps by a computer program:

s1, acquiring a screenshot of a candidate graph drawn on the operation interface at present and a drawing track point set of the candidate graph;

s2, determining the first-level graph category to which the candidate graph belongs based on the screenshot;

s3, identifying a second-level graph category from the subcategories of the first-level graph category based on the drawing track point set;

and S4, displaying the target graph fitted according to the second-level graph type in the operation interface.

Alternatively, it can be understood by those skilled in the art that the structure shown in fig. 5 is only an illustration, and the electronic device may also be a terminal device such as a smart phone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a palmtop computer, a Mobile Internet Device (MID), a PAD, and the like. Fig. 5 is a diagram illustrating a structure of the electronic device. For example, the electronics may also include more or fewer components (e.g., network interfaces, etc.) than shown in FIG. 5, or have a different configuration than shown in FIG. 5.

The memory 502 may be used to store software programs and modules, such as program instructions/modules corresponding to the pattern recognition method and apparatus in the embodiments of the present invention, and the processor 504 executes various functional applications and data processing by running the software programs and modules stored in the memory 502, so as to implement the pattern recognition method described above. The memory 502 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 502 may further include memory located remotely from the processor 504, which may be connected to the terminal over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof. The memory 502 may be, but not limited to, specifically configured to store information such as a graph fitting parameter of each graph. As an example, as shown in fig. 5, the memory 502 may include, but is not limited to, the obtaining unit 402, the determining unit 404, the identifying unit 406, and the displaying unit 408 of the pattern recognition apparatus. In addition, the image recognition device may further include, but is not limited to, other module units in the image recognition device, which is not described in detail in this example.

Optionally, the transmission device 506 is used for receiving or sending data via a network. Examples of the network may include a wired network and a wireless network. In one example, the transmission device 506 includes a Network adapter (NIC) that can be connected to a router via a Network cable and other Network devices to communicate with the internet or a local area Network. In one example, the transmission device 506 is a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.

In addition, the electronic device further includes: a display 508 for displaying the drawn candidate figure and the finally corrected target figure; and a connection bus 510 for connecting the respective module parts in the above-described electronic apparatus.

In other embodiments, the terminal device or the server may be a node in a distributed system, where the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting a plurality of nodes through a network communication. Nodes can form a Peer-To-Peer (P2P, Peer To Peer) network, and any type of computing device, such as a server, a terminal, and other electronic devices, can become a node in the blockchain system by joining the Peer-To-Peer network.

According to an aspect of the application, a computer program product or computer program is provided, comprising computer instructions, the computer instructions being stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the pattern recognition method described above. Wherein the computer program is arranged to perform the steps of any of the above method embodiments when executed.

Alternatively, in the present embodiment, the above-mentioned computer-readable storage medium may be configured to store a computer program for executing the steps of:

Alternatively, in this embodiment, a person skilled in the art may understand that all or part of the steps in the methods of the foregoing embodiments may be implemented by a program instructing hardware associated with the terminal device, where the program may be stored in a computer-readable storage medium, and the storage medium may include: flash disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.

The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.

The integrated unit in the above embodiments, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in the above computer-readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing one or more computer devices (which may be personal computers, servers, network devices, etc.) to execute all or part of the steps of the method according to the embodiments of the present invention.

In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the several embodiments provided in the present application, it should be understood that the disclosed client may be implemented in other manners. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims

1. A method of pattern recognition, comprising:

acquiring a screenshot of a candidate graph currently drawn on an operation interface and a drawing track point set of the candidate graph;

determining a first-level graph category to which the candidate graph belongs based on the screenshot;

identifying a second level graphics category from a sub-category of the first level graphics category based on the set of drawn trajectory points;

and displaying the target graph fitted according to the second-level graph type in the operation interface.

2. The method of claim 1, wherein determining the first level graphics class to which the candidate graphics belongs based on the screenshot comprises:

classifying the screenshot by adopting a classification model to obtain a classification result, wherein the classification model is a neural network model used for identifying the category of the graph displayed in the screenshot;

and determining the first-level graph category according to the classification result.

3. The method of claim 2, wherein classifying the screenshot by using a classification model to obtain a classification result comprises:

adjusting the input size of the screenshot to a target size, wherein the size information of the target size comprises: the display length of the screenshot, the display width of the screenshot and the display color value of the screenshot;

and inputting the screenshots under the target size into the classification model to obtain the classification result, wherein the application format of the model framework used in the classification model is a format which is allowed to be called by a mobile device or an embedded device.

4. The method of claim 1, wherein identifying a second level graphics category from the subcategories of the first level graphics category based on the set of drawn trajectory points comprises:

carrying out feature recognition on the drawing track point set to obtain a graph fitting parameter;

and determining the second-level graph category from the subcategories of the first-level graph categories according to the graph fitting parameters.

5. The method of claim 4, wherein performing feature recognition on the set of plotted trajectory points to obtain a graph fitting parameter comprises:

under the condition that the convex hull track is identified from the drawing track point set, acquiring a graph corner point of the convex hull track;

and under the condition that the total number of the graph corner points reaches N, determining the position coordinates of the graph corner points as the graph fitting parameters for fitting and generating the N-edge type graph.

6. The method of claim 4, wherein the extracting features from the set of plotted trajectory points to obtain the graph fitting parameters comprises:

determining a straight line fitted according to the drawing track point set as a central line;

fitting a minimum circumscribed triangle of the candidate graph according to the drawing track point set;

determining intersection points between the three side line segments corresponding to the minimum external triangle and the central line;

sequencing the distances between the midpoint positions of the three edge lines and the intersection points;

determining the side line segment corresponding to the minimum distance as the side line segment positioned at the top end of the heart shape;

and determining the heart shape direction parameter and the heart shape size parameter which are calculated based on the side line segment positioned at the top end of the heart shape as the graph fitting parameters for fitting and generating the heart shape class graph.

7. The method of claim 4, wherein the extracting features from the set of plotted trajectory points to obtain the graph fitting parameters comprises:

calculating included angles between vectors formed by any two points in the drawing trace point set to obtain a plurality of included angles;

traversing the point set of the drawn locus, and determining a point corresponding to the maximum included angle in the plurality of included angles as an arrow vertex;

determining a point set between the head position of the initial end of the track and the vertex of the arrow as a right-angle trunk;

and determining the position coordinates of each point on the right-angle trunk and the position coordinates of the arrow vertex as the graph fitting parameters for fitting and generating the right-angle unilateral arrow graph.

8. The method of claim 4, wherein the extracting features from the set of plotted trajectory points to obtain the graph fitting parameters comprises:

acquiring a first position coordinate of a head position located at the initial end of the track and a second position coordinate of a tail position located at the tail end of the track according to the drawn track point set;

determining the position coordinate of a point with the farthest vertical distance relative to a connecting line from the drawing track point set as the position coordinate of the farthest point of the arc line, wherein the connecting line is the connecting line between the first position coordinate and the second position coordinate;

converting the position coordinate of the farthest point into the position coordinate of the control point;

and determining the first position coordinate, the second position coordinate and the position coordinate of the control point as the graph fitting parameters for fitting and generating the polygonal line graph.

9. The method of claim 8, wherein extracting features from the set of plotted trajectory points to obtain a graph fitting parameter comprises:

determining candidate angles according to the first position coordinates, the second position coordinates and the position coordinates of the control points;

performing right-angle correction on the candidate angle to obtain a corrected point position coordinate;

and determining the corrected point position coordinates as the graph fitting parameters for fitting and generating the right-angle graphs.

10. The method of claim 9, further comprising, prior to determining a candidate angle based on the first location coordinate, the second location coordinate, and the location coordinate of the control point:

performing linear detection on the drawing track point set to obtain a detection result;

combining a plurality of line segments in the detection result, and determining two line segments with the maximum length after combination as right-angle edges;

determining a turning point based on the drop foot of the right-angle side;

and determining the first position coordinate and the second position coordinate in the direction after the right-angle edge extends along the opposite direction of the foot.

11. The method of claim 10, after performing a right angle correction on the candidate angle to obtain corrected coordinates of the point location, further comprising:

counting a first point number around the first position coordinate and a second point number around the second position coordinate;

determining the vertex of the arrow according to the first point number and the second point number;

and determining the corrected point position coordinates and the position coordinates of the arrow top points as the graph fitting parameters for fitting and generating the right-angle bilateral arrow graph.

12. The method of claim 4, wherein the extracting features from the set of plotted trajectory points to obtain the graph fitting parameters comprises:

in a case where the set of plotted trace points includes two subsets, calculating lengths of two line segments based on the two subsets, and determining a line segment having a largest length of the two line segments as an arc subject;

determining the vertex of the arrow according to the number of points around the two endpoints of the arc main body;

and determining the position coordinates of the two end points of the arc main body and the position coordinates of the arrow vertex as the graph fitting parameters for fitting and generating the arc bilateral arrow type graph.

13. The method of claim 4, wherein the extracting features from the set of plotted trajectory points to obtain the graph fitting parameters comprises:

performing straight line fitting on the drawing track point set by using a first preset function to obtain the slope of a fitted candidate straight line and the position coordinates of a target point on the candidate straight line;

calculating the intersection point position coordinates of two intersection points of the straight line and the canvas boundary according to the slope of the candidate straight line, the position coordinates of the target point and the size of the graph of the drawing track point;

and determining the intersection point position coordinates as the graph fitting parameters for fitting and generating the straight line type graph.

14. The method of claim 13, wherein extracting features from the set of plotted trajectory points to obtain a graph fitting parameter comprises:

obtaining the distance from each trace point in the drawing trace point set to the candidate straight line;

determining a plurality of track points with the distance larger than a threshold value as reference points;

calculating the average value of the distances corresponding to the reference points;

determining a reference distance between each of two end points of the candidate straight line and a target position, wherein the target position is a position corresponding to an average value of the distances;

determining an endpoint corresponding to the minimum value of the reference distance as an arrow vertex;

and determining the position coordinates of the end points and the position coordinates of the arrow vertexes as the graph fitting parameters for fitting and generating the straight line bilateral arrow type graph.

15. The method of claim 4, wherein the extracting features from the set of plotted trajectory points to obtain the graph fitting parameters comprises:

fitting the drawn track point set by adopting a least square method to obtain a fitting ellipse;

acquiring the position coordinate, the major axis and the minor axis of the central point of the fitting ellipse and the rotation angle;

calculating a ratio between the major axis and the minor axis, wherein the fitted ellipse is determined to be a circle if the ratio is in a target value interval; determining the fitted ellipse as an ellipse if the ratio exceeds the target value interval;

and determining the ratio as the graph fitting parameter for fitting to generate a circular graph.

16. The method of claim 4, wherein the extracting features from the set of plotted trajectory points to obtain the graph fitting parameters comprises:

calculating the vertex coordinates of three vertexes of the outsourcing triangle of the drawing track point set by using a second preset function;

and determining the vertex coordinates of the three vertexes as the graph fitting parameters for fitting and generating the triangle-like graph.

17. A pattern recognition apparatus, comprising:

the acquisition unit is used for acquiring a screenshot of a candidate graph currently drawn on an operation interface and a drawing track point set of the candidate graph;

the determining unit is used for determining the first-level graph category to which the candidate graph belongs based on the screenshot;

an identifying unit configured to identify a second level graphics category from the sub-categories of the first level graphics category based on the set of drawing trace points;

and the display unit is used for displaying the target graph fitted according to the second-level graph type in the operation interface.

18. A computer-readable storage medium, comprising a stored program, wherein the program when executed performs the method of any one of claims 1 to 16.

19. An electronic device comprising a memory and a processor, characterized in that the memory has stored therein a computer program, the processor being arranged to execute the method of any of claims 1 to 16 by means of the computer program.