CN110378310B - Automatic generation method of handwriting sample set based on answer library - Google Patents
Automatic generation method of handwriting sample set based on answer library Download PDFInfo
- Publication number
- CN110378310B CN110378310B CN201910678950.2A CN201910678950A CN110378310B CN 110378310 B CN110378310 B CN 110378310B CN 201910678950 A CN201910678950 A CN 201910678950A CN 110378310 B CN110378310 B CN 110378310B
- Authority
- CN
- China
- Prior art keywords
- handwriting
- text
- answer
- picture
- template
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
- G06V10/225—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition based on a marking or identifier characterising the area
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/40—Document-oriented image-based pattern recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/30—Writer recognition; Reading and verifying signatures
- G06V40/33—Writer recognition; Reading and verifying signatures based only on signature image, e.g. static signature recognition
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention discloses an automatic generation method of a handwriting sample set based on an answer library, which specifically comprises the following steps: (1) separation of fingerprints: separating the print and handwritten characters in the picture of the work or test paper containing the handwritten text; (2) character positioning: positioning the characters in the character pictures to obtain positioning coordinates; (3) answer acquisition: determining answer coordinates of the handwriting; (4) answer alignment: performing actual sorting and alignment operation on answers to the questions to finish detection of the corresponding relation between the answers and answer library data; (5) answer classification: judging character types according to the data information of the answer library, cutting out corresponding handwriting answers, and storing the cut answer pictures as basic samples into a folder; (6) sample synthesis: and (3) automatically synthesizing a training sample set by randomly reading the folder names and the files in the folders according to the data set of the basic sample obtained in the step (5).
Description
Technical Field
The invention belongs to the technical field of deep learning sample preparation, in particular to a method for automatically generating a handwriting sample data set by using various image processing technologies.
Background
With the development of artificial intelligence, computers gradually replace heavy human effort and begin to possess intelligence. For example, in the field of text recognition, an automatic reading system for student test paper, an intelligent bill recognition and input system and the like are created; in the field of target detection, a face recognition system, a target tracking system and the like are born; in the field of voice recognition, a man-machine dialogue system, a voice intelligent control system and the like are born; in these different application fields, the level of intelligence possessed by the computer depends on the level of learning resources, and the computer obtains higher intelligence through uninterrupted learning. In particular, in the field of text recognition, a large amount of sample data is required as important resources for learning in the recognition of handwriting, so that a large amount of work is done on handwriting sample acquisition and data labeling, and the recognition becomes a primary problem of artificial intelligence research in the field.
Therefore, there is a need to develop an automatic generation method of a handwriting sample set based on an answer library, which can automatically generate a large number of training sample sets according to a basic sample without manually classifying the basic sample in the case of providing the answer library.
Disclosure of Invention
The invention aims to solve the technical problem of providing an automatic generation method of a handwriting sample set based on an answer library, which can automatically generate a large number of training sample sets according to the basic samples without manually classifying the basic samples under the condition of providing the answer library.
In order to solve the technical problems, the invention adopts the following technical scheme: the automatic generation method of the handwriting sample set based on the answer library specifically comprises the following steps:
(1) Fingerprint separation: separating the print and handwritten characters in the picture of the work or test paper containing the handwritten text;
(2) Character positioning: positioning the characters in the character pictures of the separated handwriting;
(3) Answer acquisition: firstly, determining answer areas of the questions, and then determining answer coordinates of handwriting;
(4) Answer alignment: combining the number of different questions and the number information of answers stored in the answer database, performing actual sorting and alignment operation on the answers of the questions, and finishing the detection of the corresponding relation between the answers and the answer database data;
(5) Answer classification: directly judging character types according to the data information of the answer library, cutting out corresponding handwriting answers, and storing the cut answer pictures as basic samples into corresponding folders;
(6) Sample synthesis: according to the data set of the basic sample obtained in the step (5), the obtained answer pictures are uniformly scaled to the size of 32 pixels in an equal ratio by randomly reading the file names and files in the file folders, and then the digitized pictures are automatically synthesized in a matrix row splicing mode, so that a training sample set is finally obtained.
By adopting the technical scheme, as the images simultaneously contain the printing body and the handwriting characters, in order to ensure that the answers of the handwriting characters are completely extracted, and character pictures of the answers of the handwriting characters are correctly classified and stored according to the information of an answer library to serve as basic samples for automatically synthesizing a final sample data set; firstly, processing and analyzing text pictures of an operation or test paper containing a handwriting text, and adopting handwriting printing separation based on template matching to realize complete extraction of handwriting characters in the two pictures; extracting a complete handwriting answer by utilizing the coordinate information of the answer area and the positioning result of the separated handwriting characters; then, carrying out alignment comparison on the handwritten answers and the answer library information, classifying the answers, and storing the answers into corresponding folders; and finally, automatically synthesizing training sample set data according to the corresponding basic sample information and labeling the training sample set data so as to achieve the aim of meeting the training requirements.
In the step (1), since the student test paper pictures contain the printed body and the handwritten character at the same time, in order to completely obtain the handwritten answer, the complete separation of the handwritten character and the printed body character is completed in a template matching mode; in the step (4), considering the phenomenon that the answer area does not answer in the actual answer sheet, the actual sorting and alignment operation is performed on the answers by combining the number of small questions in different question types and the number information of the answers stored in the answer database, and the corresponding relation between the answers and the answer library data is detected.
As a preferred embodiment of the present invention, the step (1) of separating the fingerprints includes the specific steps of separating the printed matter and the handwritten character in the picture of the job or test paper containing the handwritten text by means of template matching:
s11, correcting: the blank template and the operation or test paper picture containing the handwriting text are subjected to binarization processing, contour extraction and Hough transformation to detect the inclination of the corrected picture;
s12, handwriting separation: and separating the handwriting from the printing in the operation or test paper picture containing the handwriting text to obtain the pure handwriting picture.
As a preferred technical solution of the present invention, the specific step of positioning the character in the step (2) by using the independent edge pixel detection method includes:
s21 text positioning: positioning the handwriting text in the obtained pure handwriting picture;
s22, straight line detection: detecting whether a straight line exists in the handwriting text positioned in the step S21, if so, detecting the specific position of the straight line by adopting a horizontal and vertical kernel convolution technology and outputting coordinate values of the specific position; s23, interference screening: and screening the interference characters of the handwritten text, and removing the interfered handwritten characters to obtain the coordinates of the required handwritten characters.
By adopting the technical scheme, the operation or the test paper picture containing the handwriting text is subjected to image processing analysis by taking a blank template which is not filled with the handwriting after the original printing as a reference, and the positioning and the separation of the handwriting and the printing body are realized by adopting a plurality of algorithms, and meanwhile, the handwriting characters in the printing body and the handwriting picture are accurately positioned; the accuracy of the text recognition system is improved. In step S11, since a certain inclination is unavoidable when actually capturing the handwriting text-containing operation or the test paper picture, the handwriting text-containing operation or the test paper picture needs to be corrected so that the handwriting text content can be completely separated; the pure handwriting picture separated from the step S12 actually contains the contents of all handwriting, so that the text of the pure handwriting picture is positioned, and a positioning effect picture can be obtained; when the blank template has a straight line underline, and the underline belongs to a non-text and cannot be detected, in order to avoid the risk that the handwritten text (target information) in the upper area cannot be matched with the finally removed text caused by screening the coordinates in the fifth step, horizontal and vertical kernel convolution and other technologies are adopted to detect the specific position of the underline and output the coordinate value of the specific position.
As a preferred technical solution of the present invention, the step S12 specifically includes the following steps:
s121 template matching: searching a matching point of the blank template in the step S11 by utilizing sliding of the blank template through at least two matching algorithms, and performing rough matching on the operation or test paper picture containing the handwriting text and the blank template;
s122, cutting a template: cutting the blank template which is roughly matched with the operation or the test paper picture containing the handwriting text in the step S121, so that the size of the blank template is the same as that of the operation or the test paper picture containing the handwriting text;
s123 feature registration: searching key feature points by adopting a Scale Invariant Feature Transform (SIFT) algorithm, enabling the blank template to coincide with text content in the operation or test paper picture containing the handwriting text, and enabling the blank template to be registered with the operation or test paper picture containing the handwriting text;
s124, difference detection: comparing the registered operation or test paper picture containing the handwriting text in the step S123 with the blank template picture by adopting different area detection algorithms, finding out different areas in the blank template and the operation or test paper picture containing the handwriting text to obtain a detection result picture, and marking the detection result picture as a picture A;
s125, template subtraction: directly subtracting the values obtained after template gray processing is carried out on the blank template and the operation or test paper picture containing the handwritten text after registration in the step S123, so as to obtain a subtraction result graph, and marking the subtraction result graph as a graph B;
s126 same detection: removing the content of the handwriting part in the picture by an exclusive OR algorithm from the picture A obtained in the step S124 and the picture B obtained in the step S25 to obtain a picture C;
s127 handwriting extraction: performing OR operation on the graph A and the graph B again to obtain all overlapped text contents, and recording a graph D; and then subtracting the graph D from the graph C, and performing corrosion treatment and Gaussian denoising to obtain the final pure handwriting picture.
By adopting the technical scheme, the operation or test paper picture containing the handwriting text is subjected to image processing analysis by taking the blank template which is not filled with the handwriting after the original printing of the test paper picture as a reference, but the incomplete matching of the image and the similarity of handwriting and printing gray values are higher, so that the best matching point is found out by adopting the blank template matching and combining characteristic values to avoid difficult distinction, the closest coincidence of the contents of two pictures is realized, and the blank template subtraction principle and the exclusive or elimination operation of the same value and the contrast enhancement and image denoising algorithm are utilized, so that the aim of completely separating can be ensured while the difference between the handwriting and the printing body is increased; the method has the advantages that under the condition of providing a blank template, the method can be completely separated without considering the specific position of handwriting and the complexity of handwriting; the matching process mainly uses the sliding of a blank template to find the best matching point, and the result can not reach an ideal matching state, but can find the proper matching position of the template as far as possible, and cut out a picture with the same size as the template; the actual electronic file scanned pictures have different sizes, and when the templates are matched, the size of the blank template picture is smaller than that of the operation or test paper picture containing the handwriting text, so that the template picture needs to be cut; in order to completely overlap the blank template and the text part in the operation or test paper picture containing the handwriting text, the optimal difference state after template subtraction is achieved, so that the picture needs to be aligned; and a SIFT algorithm of scale-invariant feature transformation is adopted to find key feature points so as to achieve approximate alignment of a blank template and a work or test paper picture containing handwriting text, thereby registering the two pictures.
As a preferable technical solution of the present invention, in the step S11, the blank template and the printed text containing the operation or the test paper picture of the handwritten text have the same content; the binarization processing in the step S11 specifically comprises the following steps: setting a global threshold 128, setting pixel group pixel values greater than 128 to white, and setting pixel group pixel values less than 128 to black; the contour extraction adopts a digital binary image topology analysis algorithm based on boundary tracking, and the digital binary image topology analysis algorithm based on boundary tracking determines the surrounding relation of the binary image boundary so as to locate the image boundary; the Hough transformation detection is to calculate all possible straight lines on each point according to step length for pixel points in an input binary image, record the number of points passing by each straight line, screen the image meeting the condition according to a threshold value, thereby achieving the purpose of image detection, and can perform straight line detection through Hough transformation detection and calculate the coordinate value of a text area at the upper part of the image according to the initial and final coordinate values of the obtained straight lines. Binarization processing in the belonging step S11: global threshold 128 is set, pixel group pixel values greater than 128 are set to white, and pixel group pixel values less than 128 are set to black. Contour extraction: the digitized binary image topology analysis algorithm based on boundary tracking is adopted, the algorithm determines the surrounding relation of the binary image boundary, namely, the outer boundary, the hole boundary and the hierarchical relation thereof, and since the boundaries and the areas of the original image have one-to-one correspondence (the outer boundary corresponds to a connected area with the pixel value of 1, and the hole boundary corresponds to an area with the pixel value of 0), the image boundary can be positioned. Hough transform: the hough transform is mainly to calculate all possible straight lines at each point according to step sizes for pixel points in an input binary image. The number of points passing through each straight line is recorded, the images meeting the conditions are screened according to the threshold value, the purpose of image detection (such as straight lines, circles, rectangles and the like) is achieved, frames of scanned pictures are generally straight line frames, straight line detection can be carried out through Hough transformation, and straight line coordinates are output.
As a preferable technical scheme of the invention, the answer area coordinates are obtained by adopting a template matching zone bit method in the step (3) and the answer area position of each question is determined, and then the answer coordinates of all handwriting are finally determined according to the positioning coordinates of the handwriting characters obtained in the step (2) and the answer area coordinates obtained in the step (3).
As a preferred technical scheme of the present invention, in the step S21, the text is positioned by adopting an EAST deep learning algorithm, where the EAST deep learning algorithm is to position the text in the text picture by eliminating the middle redundancy process and reducing the detection time through the full convolution network FCN and the non-maximum suppression NMS; the EAST deep learning algorithm is based on a PVANet network, extracts and combines features under convolution kernels with different sizes, restores the combined features to the original size after pooling and combining treatment, sequentially sends the combined features into convolution kernels with the numbers of 128, 64 and 32 for convolution operation, and finally obtains the score of each text and the shape of the text in the picture respectively, thereby achieving the purpose of text detection.
As a preferred technical solution of the present invention, the filtering of the interfering characters in the handwritten text in step S23 is implemented by filtering coordinates of the characters, the coordinate filtering is performed by two coordinate lists, one is a text coordinate of the blank template, and the other is a separated handwritten text coordinate, and a difference range of the two coordinates is compared by a set threshold value to retain a target item and reject non-options.
As a preferred technical solution of the present invention, the method of template matching in step S121 is a square difference matching method, and a specific matching process is as follows:
s1211, reading a picture, and sliding the image of the blank template on the image to be matched;
s1212, sliding each fixed grid to obtain a sub-graph coordinate matrix, and normalizing the sub-graph coordinate matrix;
s1213, calculating a correlation coefficient, and finding out the coordinate of the maximum value of the correlation coefficient;
and S1214, obtaining a matching point according to the maximum value coordinates of the correlation coefficient.
As a preferred technical solution of the present invention, in the step S123, a SIFT algorithm of scale invariant feature transform is used to find key feature points to complete the detection and registration of feature points in the blank template and the two images of the operation or the test paper picture containing the handwriting text, which comprises the following specific processes:
s1231, detecting extreme points of images of one picture on all scale spaces of the picture through a Gaussian differential function; in order to find the extreme point on the scale space, the extreme value sampling point is screened by adopting a mode of comparing adjacent points, the sizes of the adjacent points on the image domain and the scale domain are seen, and stable sampling points are selected as characteristic points; s1232, determining the position and the scale of the key point by fitting a three-dimensional quadratic function, and simultaneously removing the key point with low contrast and an unstable edge response point to enhance the matching stability and improve the noise resistance;
s1233 adopts Euclidean distance of the feature vector of the key point as similarity judgment measure of the key point in the two images of the blank template and the operation or test paper picture containing the handwritten text, thereby achieving the aim of registering the two pictures.
As a preferred technical solution of the present invention, the specific process of the different area detection algorithm adopted in step S124 is as follows: performing binarization processing on the blank template and the operation or test paper picture containing the handwriting text after cutting in the step S122, performing pixel-by-pixel comparison at the same position after processing, reserving a white pixel difference region, reversing a black pixel region to change a pixel value 0 into 255, and reserving the pixel value 0 on the picture of the difference region; this allows for the complete output of different regions of the blank template than the job or coupon picture containing the handwritten text.
As a preferred technical solution of the present invention, the exclusive-or algorithm in step S126 specifically includes: and performing exclusive or operation on the images A and B, namely performing gray value exclusive or operation on the pixel points at the same position, setting the gray value to 0 under the condition of the same gray value and setting the gray value to 1 under the condition of different gray values, and outputting the image C.
As a preferred technical scheme of the present invention, in step S127, subtraction is performed on the graph D and the graph C, and the specific process of corrosion treatment and gaussian denoising is as follows:
s1271, performing matrix subtraction operation on the graph D and the graph C according to the corresponding pixel matrixes;
s1272, setting the corroded structural element as a rectangular structural element, and setting the size as a 3*3 matrix;
s1273, performing convolution operation, namely corrosion, on the subtracted picture by using the set structural element;
s1274 sets Gaussian low-pass filter parameters;
s1275 performs weighted averaging on the image matrix using a gaussian low pass filter, removing noise.
As a preferable technical scheme of the invention, the method for matching the marker bit by using the template is to match the corresponding marker character by a square difference matching method according to a marker template stored in advance, and obtain the coordinates of the answer area to determine the position of the answer area. The mark template comprises a left bracket, a right bracket, a circle, a box and the like.
As a preferable technical scheme of the invention, the specific process of comparing the positioning coordinates of the handwriting characters obtained in the step (2) with the coordinates of the answer region obtained in the step (3) is to compare the x-axis and y-axis coordinates of the left upper corner and the right lower corner of the rectangular frame for positioning the handwriting characters with the x-axis and y-axis coordinates of the left upper corner and the right lower corner of the answer region, determine the area of the overlapped area, and consider that the result of positioning in the step (2) belongs to the answer in the answer region positioned in the step (3) when the area of the overlapped area is more than 50%.
Compared with the prior art, the technical scheme has the beneficial effects that: the method comprises the steps of performing image processing analysis on an operation or test paper picture containing handwriting text by taking a blank template without handwriting after original printing as a reference, combining a plurality of algorithms to realize the positioning and separation of the handwriting and the printing body, and simultaneously, accurately positioning the handwriting characters in the printing body and the handwriting picture; extracting complete handwriting answers by combining the coordinate information of the answer area with the positioning coordinates of the separated handwriting characters, performing alignment comparison on the handwriting answers and the data of an answer library, classifying the answers and storing the answers into a folder, and finally automatically synthesizing training sample set data according to corresponding basic sample information and labeling the training sample set data so as to achieve the aim of meeting training requirements; therefore, under the condition that an answer library is provided, a large number of training sample sets can be automatically generated according to the basic samples without manually classifying the basic samples.
Drawings
The following is a further detailed description of embodiments of the invention with reference to the accompanying drawings:
FIG. 1 is a flow chart of the automated generation method of answer library-based handwriting sample sets of the present invention;
FIG. 2 is a detailed flow chart of steps (1) and (2) of the inventive answer library-based automatic generation method of a handwriting sample set;
FIG. 3 is an effect diagram of obtaining a pure handwriting text after separating a handwriting from a print in step S12 of the automatic generation method of a handwriting sample set based on an answer library of the present invention;
FIG. 4 is a diagram showing the effect of positioning a pure handwriting text containing a handwriting disturbance item in step S21 of the automatic answer library-based handwriting sample set generation method of the present invention;
FIG. 5 is an effect diagram of print character positioning of a blank template in the automatic answer library-based handwriting sample set generation method of the present invention;
FIG. 6 is a template underline detection effect diagram of step S22 in the automatic answer library-based handwritten sample set generation method of the present invention;
FIG. 7 is a diagram showing the effect of the answer positioning results of step (3) in the automatic answer library-based handwriting sample set generation method of the present invention;
FIG. 8 is a flowchart showing the step S121 in the handwriting positioning and separating method of the present invention;
FIG. 9 is a flowchart showing the step S127 in the handwriting positioning and separating method of the present invention;
FIG. 10 is a diagram A in step S124 of the automatic answer library-based handwriting sample set generation method of the present invention;
FIG. 11 is a diagram B in step S125 of the automatic answer library-based handwriting sample set generation method of the present invention;
FIG. 12 is a diagram C of step A126 in the automated answer library-based handwriting sample set generation method of the invention;
FIG. 13 is a diagram D of step A127 in the automated answer library-based handwriting sample set generation method of the present invention;
FIG. 14 is a diagram showing the results of the answer classification of step (5) in the automatic answer library-based handwriting sample set generation method of the present invention;
fig. 15 is a training sample set for automatic synthesis of step (6) in the automatic answer library-based handwriting sample set generation method of the present invention.
Detailed Description
Examples: as shown in fig. 1 to 15, the automatic generation method of the handwriting sample set based on the answer library specifically comprises the following steps:
(1) Fingerprint separation: separating the print and handwritten characters in the picture of the work or test paper containing the handwritten text;
(2) Character positioning: positioning the characters in the character pictures of the separated handwriting;
(3) Answer acquisition: firstly, determining answer areas of the questions, and then determining answer coordinates of handwriting;
(4) Answer alignment: combining the number of different questions and the number information of answers stored in the answer database, performing actual sorting and alignment operation on the answers of the questions, and finishing the detection of the corresponding relation between the answers and the answer database data;
(5) Answer classification: directly judging character types according to the data information of the answer library, cutting out corresponding handwriting answers, and storing the cut answer pictures as basic samples into corresponding folders;
(6) Sample synthesis: according to the data set of the basic sample obtained in the step (5), the obtained answer pictures are uniformly scaled to the size of 32 pixels in an equal ratio by randomly reading the file names and files in the file folders, and then the digitized pictures are automatically synthesized in a matrix row splicing mode, so that a training sample set is finally obtained.
By adopting the technical scheme, as the images simultaneously contain the printing body and the handwriting characters, in order to ensure that the answers of the handwriting characters are completely extracted, and character pictures of the answers of the handwriting characters are correctly classified and stored according to the information of an answer library to serve as basic samples for automatically synthesizing a final sample data set; firstly, processing and analyzing text pictures of an operation or test paper containing a handwriting text, and adopting handwriting printing separation based on template matching to realize complete extraction of handwriting characters in the two pictures; extracting a complete handwriting answer by utilizing the coordinate information of the answer area and the positioning result of the separated handwriting characters; then, carrying out alignment comparison on the handwritten answers and the answer library information, classifying the answers, and storing the answers into corresponding folders; and finally, automatically synthesizing training sample set data according to the corresponding basic sample information and labeling the training sample set data so as to achieve the aim of meeting the training requirements.
The specific steps of separating the fingerprints in the step (1) to separate the printed matter and the written character in the picture of the operation or test paper containing the handwritten text by a template matching mode include:
s11, correcting: the blank template and the operation or test paper picture containing the handwriting text are subjected to binarization processing, contour extraction and Hough transformation to detect the inclination of the corrected picture;
s12, handwriting separation: separating handwriting from printing in the operation or test paper picture containing the handwriting text to obtain a pure handwriting picture;
the specific step of positioning the character by the independent edge pixel detection method in the step (2) comprises the following steps:
s21 text positioning: positioning the handwriting text in the obtained pure handwriting picture;
s22, straight line detection: detecting whether a straight line exists in the handwriting text positioned in the step S21, if so, detecting the specific position of the straight line by adopting a horizontal and vertical kernel convolution technology and outputting coordinate values of the specific position;
s23, interference screening: and screening the interference characters of the handwritten text, and removing the interfered handwritten characters to obtain the coordinates of the required handwritten characters.
The blank template in the step S11 has the same content as the printed text of the operation or test paper picture containing the handwritten text; the binarization processing in the step S11 specifically comprises the following steps: setting a global threshold 128, setting pixel group pixel values greater than 128 to white, and setting pixel group pixel values less than 128 to black; the contour extraction adopts a digital binary image topology analysis algorithm based on boundary tracking, and the digital binary image topology analysis algorithm based on boundary tracking determines the surrounding relation of the binary image boundary so as to locate the image boundary; the Hough transformation detection is to calculate all possible straight lines on each point according to step length for pixel points in an input binary image, record the number of points passing through each straight line, screen images meeting the conditions according to a threshold value, thereby achieving the purpose of image detection, and can perform straight line detection through Hough transformation detection and calculate coordinate values of a text area at the upper part of the detected straight lines according to the obtained initial and final coordinate values of the straight lines; binarization processing in the belonging step S11: global threshold 128 is set, pixel group pixel values greater than 128 are set to white, and pixel group pixel values less than 128 are set to black. Contour extraction: the digitized binary image topology analysis algorithm based on boundary tracking is adopted, the algorithm determines the surrounding relation of the binary image boundary, namely, the outer boundary, the hole boundary and the hierarchical relation thereof, and since the boundaries and the areas of the original image have one-to-one correspondence (the outer boundary corresponds to a connected area with the pixel value of 1, and the hole boundary corresponds to an area with the pixel value of 0), the image boundary can be positioned. Hough transform: the hough transform is mainly to calculate all possible straight lines at each point according to step sizes for pixel points in an input binary image. Recording the number of points passing through each straight line, screening the images meeting the conditions according to a threshold value, achieving the aim of image detection (such as straight lines, circles, rectangles and the like), scanning frames of pictures to be straight line frames, carrying out straight line detection through Hough transformation, and outputting straight line coordinates;
the step S12 specifically includes the following steps:
s121 template matching: searching a matching point of the blank template in the step S11 by utilizing sliding of the blank template through at least two matching algorithms, and performing rough matching on the operation or test paper picture containing the handwriting text and the blank template;
s122, cutting a template: cutting the blank template which is roughly matched with the operation or the test paper picture containing the handwriting text in the step S121, so that the size of the blank template is the same as that of the operation or the test paper picture containing the handwriting text;
s123 feature registration: searching key feature points by adopting a Scale Invariant Feature Transform (SIFT) algorithm, enabling the blank template to coincide with text content in the operation or test paper picture containing the handwriting text, and enabling the blank template to be registered with the operation or test paper picture containing the handwriting text;
s124, difference detection: comparing the registered operation or test paper picture containing the handwriting text in the step S123 with the blank template picture by adopting different area detection algorithms, finding out different areas in the blank template and the operation or test paper picture containing the handwriting text to obtain a detection result picture, and marking the detection result picture as a picture A;
s125, template subtraction: directly subtracting the values obtained after template gray processing is carried out on the blank template and the operation or test paper picture containing the handwritten text after registration in the step S123, so as to obtain a subtraction result graph, and marking the subtraction result graph as a graph B;
s126 same detection: removing the content of the handwriting part in the picture by an exclusive OR algorithm from the picture A obtained in the step S124 and the picture B obtained in the step S25 to obtain a picture C;
s127 handwriting extraction: performing OR operation on the graph A and the graph B again to obtain all overlapped text contents, and recording a graph D; then subtracting the graph D from the graph C, and performing corrosion treatment and Gaussian denoising to obtain the final pure handwriting picture; s21 text positioning: positioning the handwriting text in the obtained pure handwriting picture; the operation or test paper picture containing the handwriting text is subjected to image processing analysis by taking a blank template which is not filled with handwriting after the original printing of the blank template as a reference, but because of incomplete matching of images and higher similarity of handwriting and printing gray values, in order to avoid difficult distinction, a best matching point is found by adopting a blank template matching combination characteristic value, so that the closest coincidence of the contents of two pictures is realized, and then the blank template subtraction principle and the exclusive or elimination operation of the same value and the contrast enhancement and image denoising algorithm are utilized, so that the aim of completely separating can be achieved while the difference between the handwriting and the printing is increased; the method has the advantages that under the condition of providing a blank template, the method can be completely separated without considering the specific position of handwriting and the complexity of handwriting; the matching process mainly uses the sliding of a blank template to find the best matching point, and the result can not reach an ideal matching state, but can find the proper matching position of the template as far as possible, and cut out a picture with the same size as the template; the actual electronic file scanned pictures have different sizes, and when the templates are matched, the size of the blank template picture is smaller than that of the operation or test paper picture containing the handwriting text, so that the template picture needs to be cut; in order to completely overlap the blank template and the text part in the operation or test paper picture containing the handwriting text, the optimal difference state after template subtraction is achieved, so that the picture needs to be aligned; a SIFT algorithm of scale-invariant feature transformation is adopted to find key feature points so as to achieve approximate alignment of a blank template and a work or test paper picture containing handwriting text, thereby registering the two pictures;
as shown in fig. 9 to 13, the template matching method in the step S121 is a square difference matching method, and a specific matching process is as follows:
s1211, reading a picture, and sliding the image of the blank template on the image to be matched;
s1212, sliding each fixed grid to obtain a sub-graph coordinate matrix, and normalizing the sub-graph coordinate matrix;
s1213, calculating a correlation coefficient, and finding out the coordinate of the maximum value of the correlation coefficient;
s1214, obtaining a matching point according to the maximum value coordinate of the correlation coefficient; in the step S123, a SIFT algorithm of scale invariant feature transform is used to find key feature points to complete detection and registration of feature points in the blank template and the two images of the operation or the test paper picture containing the handwritten text, which comprises the following specific processes:
s1231, detecting extreme points of images of one picture on all scale spaces of the picture through a Gaussian differential function; in order to find the extreme point on the scale space, the extreme value sampling point is screened by adopting a mode of comparing adjacent points, the sizes of the adjacent points on the image domain and the scale domain are seen, and stable sampling points are selected as characteristic points;
s1232, determining the position and the scale of the key point by fitting a three-dimensional quadratic function, and simultaneously removing the key point with low contrast and an unstable edge response point to enhance the matching stability and improve the noise resistance;
s1233, adopting the Euclidean distance of the feature vector of the key point as the similarity judgment measure of the key point in the two images of the blank template and the operation or test paper picture containing the handwriting text, thereby achieving the aim of registering the two pictures; the specific process of the different area detection algorithm adopted in step S124 is as follows: performing binarization processing on the blank template and the operation or test paper picture containing the handwriting text after cutting in the step S122, performing pixel-by-pixel comparison at the same position after processing, reserving a white pixel difference region, reversing a black pixel region to change a pixel value 0 into 255, and reserving the pixel value 0 on the picture of the difference region; thus, the blank template and different areas of the operation or test paper picture containing the handwriting text can be completely output; the exclusive-or algorithm in step S126 specifically includes: performing exclusive or operation on the images A and B, namely performing gray value exclusive or operation on pixel points at the same position, setting the gray value to 0 under the condition of the same gray value and setting the gray value to 1 under the condition of different gray values, so as to output an image C; in the step S127, the subtraction operation is performed on the graph D and the graph C, and the specific process of corrosion treatment and gaussian denoising is as follows:
s1271, performing matrix subtraction operation on the graph D and the graph C according to the corresponding pixel matrixes;
s1272, setting the corroded structural element as a rectangular structural element, and setting the size as a 3*3 matrix;
s1273, performing convolution operation, namely corrosion, on the subtracted picture by using the set structural element;
s1274 sets Gaussian low-pass filter parameters;
s1275, carrying out weighted average on the image matrix by using a Gaussian low-pass filter, and removing noise points;
the step (3) is to adopt a template matching zone bit method to obtain answer zone coordinates and determine answer zone positions of each question, and the template matching zone bit method is to match corresponding sign characters through a square difference matching method according to a pre-stored sign template and obtain the answer zone coordinates to determine the answer zone positions; comparing the positioning coordinates of the handwriting characters obtained in the step (2) with the coordinates of the answer area obtained in the step (3), wherein the specific process is to compare the x-axis and y-axis coordinates of the left upper corner and the right lower corner of the rectangular frame for positioning the handwriting characters with the x-axis and y-axis coordinates of the left upper corner and the right lower corner of the answer area, and determining the area of the overlapped area, wherein when the area of the overlapped area is greater than 50%, the positioning result of the step (2) is considered to belong to the answer in the answer area positioned in the step (3); thereby finally determining the answer coordinates of all handwriting; the result effect diagram is shown in fig. 7;
in the step S21, the text positioning is performed by adopting an EAST deep learning algorithm, wherein the EAST deep learning algorithm is to position the text in the text picture by eliminating the middle redundancy process and reducing the detection time through the full convolution network FCN and the non-maximum suppression NMS; the EAST deep learning algorithm is based on a PVANet network, extracts and combines features under convolution kernels with different sizes, restores the combined features to the original size after pooling and combining treatment, sequentially sends the combined features into convolution kernels with the numbers of 128, 64 and 32 for convolution operation, and finally obtains the score of each text and the shape of the text in the picture respectively, thereby achieving the aim of text detection; the interference character screening of the pure handwriting text in the step S23 is realized by screening the coordinates of the characters, the coordinate screening is performed by two coordinate lists, one is the text coordinates of the blank template, the other is the separated pure handwriting text coordinates, and the difference range of the two coordinates is compared by a set threshold value to keep the target item and reject non-options.
The embodiments of the present invention have been described in detail with reference to the drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge of one of ordinary skill in the art without departing from the spirit of the present invention.
Claims (7)
1. An automatic generation method of a handwriting sample set based on an answer library is characterized by comprising the following steps:
(1) Fingerprint separation: separating the print and handwritten characters in the picture of the work or test paper containing the handwritten text;
(2) Character positioning: positioning the character picture of the separated handwriting, and positioning the character in the character picture to obtain the positioning coordinate of the handwriting character;
(3) Answer acquisition: firstly, determining answer areas of the questions, and then determining answer coordinates of handwriting;
(4) Answer alignment: combining the number of different questions and the number information of answers stored in the answer database, performing actual sorting and alignment operation on the answers of the questions, and finishing the detection of the corresponding relation between the answers and the answer database data;
(5) Answer classification: directly judging character types according to the data information of the answer library, cutting out corresponding handwriting answers, and storing the cut answer pictures as basic samples into corresponding folders;
(6) Sample synthesis: according to the data set of the basic sample obtained in the step (5), uniformly scaling the obtained answer pictures to the size of 32 pixels in an equal ratio by randomly reading the file names and files in the file folders, automatically synthesizing the digitized pictures in a matrix row splicing mode, and finally obtaining a training sample set;
the specific steps of separating the fingerprints in the step (1) to separate the printed matter and the written character in the picture of the operation or test paper containing the handwritten text by a template matching mode include:
s11, correcting: the blank template and the operation or test paper picture containing the handwriting text are subjected to binarization processing, contour extraction and Hough transformation to detect the inclination of the corrected picture;
s12, handwriting separation: separating handwriting from printing in the operation or test paper picture containing the handwriting text to obtain a pure handwriting picture;
the specific step of locating the character by the independent edge pixel detection method in the step (2) comprises the following steps:
s21 text positioning: positioning the handwriting text in the obtained pure handwriting picture;
s22, straight line detection: detecting whether a straight line exists in the handwriting text positioned in the step S21, if so, detecting the specific position of the straight line by adopting a horizontal and vertical kernel convolution technology and outputting coordinate values of the specific position; s23, interference screening: screening the interference characters of the handwriting text, and removing the interference handwriting characters to obtain the coordinates of the required handwriting characters;
the step S12 specifically includes the following steps:
s121 template matching: searching a matching point of the blank template in the step S11 by utilizing sliding of the blank template through at least two matching algorithms, and performing rough matching on the operation or test paper picture containing the handwriting text and the blank template;
s122, cutting a template: cutting the blank template which is roughly matched with the operation or the test paper picture containing the handwriting text in the step S121, so that the size of the blank template is the same as that of the operation or the test paper picture containing the handwriting text;
s123 feature registration: searching key feature points by adopting a Scale Invariant Feature Transform (SIFT) algorithm, enabling the blank template to coincide with text content in the operation or test paper picture containing the handwriting text, and enabling the blank template to be registered with the operation or test paper picture containing the handwriting text;
s124, difference detection: comparing the registered operation or test paper picture containing the handwriting text in the step S123 with the blank template picture by adopting different area detection algorithms, finding out different areas in the blank template and the operation or test paper picture containing the handwriting text to obtain a detection result picture, and marking the detection result picture as a picture A;
s125, template subtraction: directly subtracting the values obtained after template gray processing is carried out on the blank template and the operation or test paper picture containing the handwritten text after registration in the step S123, so as to obtain a subtraction result graph, and marking the subtraction result graph as a graph B;
s126 same detection: removing the content of the handwriting part in the picture by an exclusive OR algorithm from the picture A obtained in the step S124 and the picture B obtained in the step S25 to obtain a picture C;
s127 handwriting extraction: performing OR operation on the graph A and the graph B again to obtain all overlapped text contents, and recording a graph D; and then subtracting the graph D from the graph C, and performing corrosion treatment and Gaussian denoising to obtain the final pure handwriting picture.
2. The automatic answer library-based handwriting sample set generation method according to claim 1, wherein the blank template in the step S11 and the printed text content of the job or test paper picture containing the handwriting text are the same; the binarization processing in the step S11 specifically comprises the following steps: setting a global threshold 128, setting pixel group pixel values greater than 128 to white, and setting pixel group pixel values less than 128 to black; the contour extraction adopts a digital binary image topology analysis algorithm based on boundary tracking, and the digital binary image topology analysis algorithm based on boundary tracking determines the surrounding relation of the binary image boundary so as to locate the image boundary; the Hough transformation detection is to calculate all possible straight lines on each point according to step length for pixel points in an input binary image, record the number of points passing by each straight line, screen the image meeting the condition according to a threshold value, thereby achieving the purpose of image detection, and can perform straight line detection through Hough transformation detection and calculate the coordinate value of a text area at the upper part of the image according to the initial and final coordinate values of the obtained straight lines.
3. The automatic generation method of the handwriting sample set based on the answer library according to claim 1, wherein in the step (3), an answer area coordinate is obtained by adopting a template matching flag bit method, the answer area position of each question is determined, and then the answer coordinates of all handwriting are finally determined according to the positioning coordinates of the handwriting characters obtained in the step (2) and the answer area coordinate obtained in the step (3).
4. The automatic answer library-based handwriting sample set generation method according to claim 1, wherein the text positioning in step S21 adopts an EAST deep learning algorithm for positioning the text in the text picture, the EAST deep learning algorithm is to suppress NMS through a full convolution network FCN and a non-maximum value, eliminate intermediate redundancy process, and reduce detection time; the EAST deep learning algorithm is based on a PVANet network, extracts and combines features under convolution kernels with different sizes, restores the combined features to the original size after pooling and combining treatment, sequentially sends the combined features into convolution kernels with the numbers of 128, 64 and 32 for convolution operation, and finally obtains the score of each text and the shape of the text in the picture respectively, thereby achieving the purpose of text detection.
5. The automatic answer library-based handwriting sample set generation method according to claim 1, wherein in the step S23, the interference character screening of the handwriting text is implemented by screening coordinates of characters, the coordinate screening is performed by two coordinate lists, one is a text coordinate of the blank template, the other is a separated handwriting text coordinate, and a difference range of the two coordinates is compared by a set threshold value to retain a target item and reject non-options.
6. The automatic generation method of handwriting sample set based on answer library according to claim 3, wherein the method of matching the flag bit by using the template is to match the corresponding flag character by a square difference matching method according to the pre-stored flag template, and obtain the coordinates of the answer area to determine the position of the answer area.
7. The automatic answer library-based handwriting sample set generation method according to claim 3, wherein the specific process of comparing the positioning coordinates of the handwriting characters obtained in the step (2) with the coordinates of the answer area obtained in the step (3) is to determine the overlapping area by comparing the x-axis and y-axis coordinates of the left upper corner and right lower corner of the rectangular frame for positioning the handwriting characters with the x-axis and y-axis coordinates of the left upper corner and right lower corner of the answer area, and when the overlapping area is greater than 50%, the result of positioning in the step (2) is considered to belong to the answer in the answer area positioned in the step (3).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910678950.2A CN110378310B (en) | 2019-07-25 | 2019-07-25 | Automatic generation method of handwriting sample set based on answer library |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910678950.2A CN110378310B (en) | 2019-07-25 | 2019-07-25 | Automatic generation method of handwriting sample set based on answer library |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110378310A CN110378310A (en) | 2019-10-25 |
| CN110378310B true CN110378310B (en) | 2023-10-10 |
Family
ID=68256112
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910678950.2A Active CN110378310B (en) | 2019-07-25 | 2019-07-25 | Automatic generation method of handwriting sample set based on answer library |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110378310B (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110895696A (en) * | 2019-11-05 | 2020-03-20 | 泰康保险集团股份有限公司 | Image information extraction method and device |
| CN110866501B (en) * | 2019-11-19 | 2022-04-29 | 中国建设银行股份有限公司 | Training data generation method, data identification method and computer storage medium |
| CN110991357A (en) * | 2019-12-06 | 2020-04-10 | 北京一起教育信息咨询有限责任公司 | Answer matching method and device and electronic equipment |
| CN110956167B (en) * | 2019-12-09 | 2023-04-28 | 南京红松信息技术有限公司 | Classification, discrimination, strengthening and separation method based on positioning characters |
| CN111401351B (en) * | 2020-03-06 | 2023-07-11 | 南京红松信息技术有限公司 | Segmentation method based on vertical character positioning expansion |
| CN111401354B (en) * | 2020-03-24 | 2023-07-11 | 南京红松信息技术有限公司 | End-to-end self-adaption based vertical adhesion character recognition method |
| JP2022092119A (en) * | 2020-12-10 | 2022-06-22 | キヤノン株式会社 | Image processing apparatus, image processing method, and program |
| CN112686143B (en) * | 2020-12-29 | 2023-12-01 | 科大讯飞股份有限公司 | Objective question filling identification method, electronic equipment and storage medium |
| CN113011299A (en) * | 2021-03-09 | 2021-06-22 | 天津职业技术师范大学(中国职业培训指导教师进修中心) | Method for adding special negative cases in text detection training set |
| CN112991410A (en) * | 2021-04-29 | 2021-06-18 | 北京世纪好未来教育科技有限公司 | Text image registration method, electronic equipment and storage medium thereof |
| CN113723252A (en) * | 2021-08-23 | 2021-11-30 | 上海财联社金融科技有限公司 | Identification method and system for table type text picture |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109634961A (en) * | 2018-12-05 | 2019-04-16 | 杭州大拿科技股份有限公司 | A kind of paper sample generating method, device, electronic equipment and storage medium |
| CN109740473A (en) * | 2018-12-25 | 2019-05-10 | 东莞市七宝树教育科技有限公司 | A kind of image content automark method and system based on marking system |
| CN109800746A (en) * | 2018-12-05 | 2019-05-24 | 天津大学 | A kind of hand-written English document recognition methods based on CNN |
| CN110020692A (en) * | 2019-04-13 | 2019-07-16 | 南京红松信息技术有限公司 | A kind of handwritten form separation and localization method based on block letter template |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101482920B (en) * | 2008-12-30 | 2010-12-22 | 广东国笔科技股份有限公司 | Hand-written character recognition method and system |
-
2019
- 2019-07-25 CN CN201910678950.2A patent/CN110378310B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109634961A (en) * | 2018-12-05 | 2019-04-16 | 杭州大拿科技股份有限公司 | A kind of paper sample generating method, device, electronic equipment and storage medium |
| CN109800746A (en) * | 2018-12-05 | 2019-05-24 | 天津大学 | A kind of hand-written English document recognition methods based on CNN |
| CN109740473A (en) * | 2018-12-25 | 2019-05-10 | 东莞市七宝树教育科技有限公司 | A kind of image content automark method and system based on marking system |
| CN110020692A (en) * | 2019-04-13 | 2019-07-16 | 南京红松信息技术有限公司 | A kind of handwritten form separation and localization method based on block letter template |
Non-Patent Citations (2)
| Title |
|---|
| Handwriting Recognition through Novel Combinational Segmentation Technique and Iterative Blocks Features;Madhuri Maheshwari et al.;《 2018 3rd IEEE International Conference on Recent Trends in Electronics, Information & Communication Technology (RTEICT)》;全文 * |
| 手写字符样本自动采集研究;张凯兵等;《西华大学学报(自然科学版)》;20040203;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110378310A (en) | 2019-10-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110378310B (en) | Automatic generation method of handwriting sample set based on answer library | |
| CN110210413B (en) | Multidisciplinary test paper content detection and identification system and method based on deep learning | |
| CN110020692B (en) | Handwriting separation and positioning method based on print template | |
| US8644616B2 (en) | Character recognition | |
| US8494273B2 (en) | Adaptive optical character recognition on a document with distorted characters | |
| CN110119741B (en) | Card image information identification method with background | |
| CN110033471B (en) | Frame line detection method based on connected domain analysis and morphological operation | |
| Gebhardt et al. | Document authentication using printing technique features and unsupervised anomaly detection | |
| CN112183038A (en) | Form identification and typing method, computer equipment and computer readable storage medium | |
| Tamilselvi et al. | A Novel Text Recognition Scheme using Classification Assisted Digital Image Processing Strategy | |
| US20110007366A1 (en) | System and method for classifying connected groups of foreground pixels in scanned document images according to the type of marking | |
| CN103310211B (en) | A kind ofly fill in mark recognition method based on image procossing | |
| CN109947273B (en) | Point reading positioning method and device | |
| Tardón et al. | Optical music recognition for scores written in white mensural notation | |
| CN114299528A (en) | Information extraction and structuring method for scanned document | |
| CN111626292B (en) | Text recognition method of building indication mark based on deep learning technology | |
| Yadav et al. | Text extraction in document images: highlight on using corner points | |
| CN111915635A (en) | Test question analysis information generation method and system supporting self-examination paper marking | |
| Keong et al. | Malaysian automatic number plate recognition system using Pearson correlation | |
| CN108921006B (en) | Method for establishing handwritten signature image authenticity identification model and authenticity identification method | |
| CN107609482B (en) | Chinese text image inversion discrimination method based on Chinese character stroke characteristics | |
| CN110766001B (en) | Bank card number positioning and end-to-end identification method based on CNN and RNN | |
| Lin et al. | Locating license plate based on edge features of intensity and saturation subimages | |
| CN104462111A (en) | Image retrieval database establishing method | |
| CN114241486A (en) | Method for improving accuracy rate of identifying student information of test paper |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |