CN111666882B

CN111666882B - Method for extracting answers of handwriting test questions

Info

Publication number: CN111666882B
Application number: CN202010510103.8A
Authority: CN
Inventors: 王家奎; 王旺
Original assignee: Wuhan Veilytech Co ltd
Current assignee: Wuhan Veilytech Co ltd
Priority date: 2020-06-08
Filing date: 2020-06-08
Publication date: 2023-04-25
Anticipated expiration: 2040-06-08
Also published as: CN111666882A

Abstract

The invention discloses a method for extracting answers of handwriting test questions, which comprises the steps that one picture is a source test question picture A, the other picture is an answer picture B to be extracted, and photographing and uploading are carried out; the method also comprises the following steps: s1: performing corresponding preprocessing, such as size scaling, on the graphs A and B, and extracting feature points; s2: performing primary matching on the characteristic points of the graphs A and B by using a coarse matching method, calculating a homography matrix from the graph B to the graph A, and correspondingly mapping the graph B into a graph C, wherein the sizes of the graph C and the graph A are consistent, and the angles are close; the invention has scientific and reasonable structure, safe and convenient use, improves the on-line examination question review function, reduces the problems of lower angle and flatness of the uploaded pictures of students, facilitates the extraction of answers, reduces the inconvenience brought by the extremely high precision requirement in the past, and is suitable for popularization and use.

Description

Method for extracting answers of handwriting test questions

Technical Field

The invention relates to the technical field of picture processing, in particular to a method for extracting answers of handwriting test questions.

Background

The examination is a strict knowledge level identification method, the learning ability and other abilities of students can be checked through the examination, in order to ensure the fairness and fairness of the results, the examination room is required to have strong discipline constraint, and is specially provided with a main examination, an invigilation and other examination supervising processes, any cheating actions are absolutely forbidden, otherwise legal and criminal responsibilities are about to be born, the examination is that a group of people with different educational resources can complete the same answer sheet within a certain time, however, the meaning of the examination is not limited to the same, and the examination can also be carried out in an omnibearing way on a person for one target, so that the examination is really that people with different social positions can have the opportunity to change themselves, and a test sheet can be realized, the examination questions of the test sheet are compared, and correct answers are adjusted;

with the continuous development of online education, the acceptance degree of online teaching is gradually increased, and with the increase of the number of students, the workload of teachers is increased, the online examination question review function can effectively lighten the work of teachers, more energy is put in teaching work and knowledge grasping conditions of study students, and the current online examination question review function is often insufficient in accuracy for acquiring examination questions uploaded by the students and extracting answer parts, or has higher requirements on the angle and flatness of uploading pictures by the students, so that the online examination question review function is inconvenient to use.

Disclosure of Invention

The invention provides a handwriting test question answer extraction method, which can effectively solve the problems that the acceptance degree of network teaching is gradually increased along with the continuous development of online education in the background technology, the workload of teachers is increased along with the increase of the number of students, the work of teachers can be effectively lightened by an online test question review function, more energy is put on teaching work and the knowledge grasping condition of study students, the accuracy of the part for acquiring test questions uploaded by the students and extracting answers is often insufficient, or the requirements on the angle and the flatness of the pictures uploaded by the students are higher, and the use is inconvenient.

In order to achieve the above purpose, the present invention provides the following technical solutions: a method for extracting answers of handwriting test questions comprises taking one picture as a source test question picture A and the other picture as an answer picture B to be extracted, and taking a picture for uploading;

the method also comprises the following steps:

s1: performing corresponding preprocessing, such as size scaling, on the graphs A and B, and extracting feature points;

s2: performing primary matching on the characteristic points of the graphs A and B by using a coarse matching method, calculating a homography matrix from the graph B to the graph A, and correspondingly mapping the graph B into a graph C, wherein the sizes of the graph C and the graph A are consistent, and the angles are close;

s3: extracting characteristic points from the graphs A and C by using a characteristic point matching method based on a graph neural network, and performing characteristic point matching to obtain characteristic point matching pairs corresponding to the graphs A and C;

s4: the feature point matching pairs calculate the positions of the labeling frames in the graph A in the graph C by using a corresponding algorithm;

s5: and extracting answer information in the graph C.

According to the technical scheme, the source test question picture A is marked in advance by marking the answer position to be extracted.

According to the above technical solution, in step S1, the image preprocessing portion, the size scaling is beneficial to the overall running speed, but the accuracy is reduced, and the sift technique is used for feature point extraction.

According to the above technical solution, in step S2, the matching technique may use KNN technique or FLANN technique, and the calculation of the homography matrix may be performed through a corresponding interface provided by OpenCV;

the purpose of this step is to rotate the graph B to an angle close to the graph a through this matching, and can remove the possible redundant content of the graph B in the shooting process, and reduce the error in the next matching, but due to the possible distortion, deformation and other conditions of the graph B, the homography matrix cannot be used to make the graph B perfectly coincide with the graph a, so that the position coordinates marked by the graph a cannot be directly adopted.

According to the above technical solution, in the step S3, the neural network model of the graph may perform feature extraction on the input image by using a CNN network, perform matching through the GNN network, and finally return to the matched feature point pair; the higher the accuracy of the feature point matching technique used in this step, the better.

According to the above technical solution, in the step S4, since the coordinates of the marked frame in the graph a are not necessarily feature points, the approximate position of the marked frame in the graph C is calculated by using an algorithm;

algorithm design: the label box of FIG. A has four vertices, respectively (x 1, y 1), (x 2, y 2), (x 3, y 3), and (x 4, y 4), followed by two steps;

the first step: inputting (x 1, y 1), calculating Euclidean distance between the point and all the characteristic points, taking a plurality of points closest to the point, calculating an average position (x, y) by using the closest points, and then calculating the relative distance between the point and the input point (w=x1-x, h=y1-y);

and a second step of: in the graph C, finding the matching point of the nearest characteristic point of the vertex of the graph A, similarly calculating the average position (x ', y'), and calculating the approximate position (w+x ', h+y') of the input point in the graph C by using the relative position calculated in the first step;

and sequentially inputting the remaining three points to obtain the region mapped by the labeling frame of the graph A in the graph C.

According to the above technical solution, in step S5, a corresponding area is obtained in step S4, and the content of the area is extracted, i.e. the answer portion in the test question.

Compared with the prior art, the invention has the beneficial effects that: the invention has scientific and reasonable structure, safe and convenient use, improves the on-line examination question review function, reduces the problems of lower angle and flatness of the uploaded pictures of students, facilitates the extraction of answers, reduces the inconvenience caused by the extremely high precision requirement in the past, can also achieve the rapid and accurate extraction of the handwriting examination papers in the online examination papers, reduces the degree of the past examination papers, and is not suitable for the application, but the extraction of the method greatly facilitates the answer extraction of the examination papers, is convenient, quick and reliable, and is suitable for popularization and use.

Drawings

The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate the invention and together with the embodiments of the invention, serve to explain the invention.

In the drawings:

FIG. 1 is a schematic flow diagram of an example of the present invention;

FIG. 2 is a schematic diagram of a blank test question (panel A);

FIG. 3 is a schematic diagram of a blank test question labeling frame;

FIG. 4 is a schematic diagram of a handwriting test question (panel B);

FIG. 5 is a graph (panel C) of a schematic diagram of a handwriting after homography matrix transformation;

FIG. 6 is a block diagram of the neural network module of FIG. 6;

FIG. 7 is a block diagram of a portion of the GNN module of the neural network of FIG. 7;

FIG. 8 is a feature point matching pair wiring diagram;

FIG. 9 is a schematic diagram of the labeling frame of FIG. A drawn directly on FIG. C;

fig. 10 is a schematic diagram of the final answer area drawn on the graph C.

Detailed Description

The preferred embodiments of the present invention will be described below with reference to the accompanying drawings, it being understood that the preferred embodiments described herein are for illustration and explanation of the present invention only, and are not intended to limit the present invention.

Examples: the invention provides a technical scheme, a method for extracting answers of handwriting test questions, which comprises the following steps:

s1, respectively extracting features of two pictures by using a sift method provided by OpenCV;

s2, performing feature matching on feature points of the two pictures by using a KNN method, calculating a homography matrix H by matching the output feature points with a findhomography method of input OpenCV, and obtaining a picture C on a picture B according to the matrix H;

s3, inputting the graph A and the graph C into a SuperGLUE graph neural network model to obtain feature point matching pairs;

s4, calculating the position of the marked frame in the graph A by utilizing the feature point matching pair;

s5, extracting the handwriting answer in the graph C.

According to the above technical solution, in the step S1, the two pictures are respectively a prepared blank test question picture (picture a) and are marked;

referring to fig. 2 and 3, a same set of test question pictures (picture B) with answers filled in;

see fig. 4; the feature point extraction method used is the sift method provided by OpenCV.

According to the above technical solution, in step S2, a KNN matching method is used to obtain a feature point matching pair of two pictures, and then a homography method provided by OpenCV is used to calculate a homography matrix H for affine transformation from the graph B to the graph a, and the graph B is transformed correspondingly to obtain the graph C, see fig. 5.

According to the above technical solution, step S3 of the above step, the neural network model structure of the graph is shown in fig. 6 and 7, which includes two parts, respectively as follows:

extracting features of the input picture through a CNN network, wherein the specific structure is shown in fig. 6;

the extracted features are matched through the GNN network, and the specific structure is shown in fig. 6;

in the GNN network, the Attention module is executed for 9 times, and the model structure is shown in fig. 7;

the matching results are connected with each other as shown in fig. 8.

According to the technical scheme, the step S4 is divided into the following steps after more accurate characteristic point pairs are obtained in the previous step:

circularly acquiring a labeling frame in the graph A;

acquiring four vertexes of each labeling frame, and sequentially inputting each vertex into an algorithm;

inputting points (Px, py), traversing Euclidean distance between each characteristic point of the graph A and the point, and finally obtaining 5 points closest to the point through screening;

acquiring 5 corresponding characteristic points of the 5 points on the graph C;

average positions (xA, yA) and (xC, yC) of 5 points on the map a and the map C were calculated, respectively, that is:

xA=(x1+x2+x3+x4+x5)/5；

yA=(y1+y2+y3+y4+y5)/5；

xC=(x’1+x’2+x’3+x’4+x’5)/5；

yC=(y’1+y’2+y’3+y’4+y’5)/5；

calculating the relative distance (w, h) of the input point (Px, py) from (xA, yA) on the graph a, namely:

w=Px-xA；

h=Py-yA；

the relative distance is obtained according to the previous step, and the approximate position (Px ', py') of the map C is calculated by using the relative distance and the calculated average position (xC, yC) on the map C, namely:

Px’=w+xC；

Py’=h+yC；

sequentially inputting, and finally obtaining the approximate positions of the four vertexes of each labeling frame to form a quadrangle, and framing the position of the area where the answer on the graph C is located;

finally, the positions of all the label frame mappings are obtained, see fig. 10.

According to the technical scheme, in the step 5, the mapping positions obtained in the previous step are summarized, and the answer of the handwriting test question can be obtained by screenshot on the graph C.

Finally, it should be noted that: the foregoing is merely a preferred example of the present invention, and the present invention is not limited thereto, but it is to be understood that modifications and equivalents of some of the technical features described in the foregoing embodiments may be made by those skilled in the art, although the present invention has been described in detail with reference to the foregoing embodiments. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for extracting answers of handwriting test questions is characterized in that: the method comprises the steps that one picture is a source test question picture A, the other picture is an answer picture B to be extracted, and photographing and uploading are carried out;

the method also comprises the following steps:

s1: performing corresponding preprocessing, size scaling and feature point extraction on the graphs A and B;

s5: extracting answer information in the graph C;

in the step S4, since the coordinates of the marked frame in the graph a are not necessarily feature points, the approximate position of the marked frame in the graph C is calculated by an algorithm;

2. The method for extracting answers to test questions of handwriting according to claim 1, wherein the source test question picture a is labeled in advance by labeling processing, and the answer position to be extracted is labeled.

3. The method according to claim 1, wherein in the step S1, the image preprocessing portion is scaled to a size that is favorable for the overall operation speed, but the accuracy is reduced, and the feature point extraction is performed by using a sift technique.

4. The method according to claim 1, wherein in step S2, the matching technique uses KNN technique or FLANN technique, and the calculation of the homography matrix is performed through a corresponding interface provided by OpenCV.

5. The method according to claim 1, wherein the neural network model of the graph in step S3 performs feature extraction on the input image by using a CNN network, performs matching by using the GNN network, and finally returns the matched feature point pairs.

6. The method according to claim 1, wherein in step S5, a corresponding area is obtained in step S4, and the content of the area is extracted to obtain an answer portion in the test question.