CN114359917A - Handwritten Chinese character detection and recognition and font evaluation method - Google Patents
Handwritten Chinese character detection and recognition and font evaluation method Download PDFInfo
- Publication number
- CN114359917A CN114359917A CN202111520803.6A CN202111520803A CN114359917A CN 114359917 A CN114359917 A CN 114359917A CN 202111520803 A CN202111520803 A CN 202111520803A CN 114359917 A CN114359917 A CN 114359917A
- Authority
- CN
- China
- Prior art keywords
- chinese character
- chinese characters
- handwritten
- handwritten chinese
- recognition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Character Discrimination (AREA)
Abstract
The invention discloses a handwritten Chinese character detection and recognition and font evaluation method. The method comprises a Chinese character detection stage, a Chinese character recognition stage and a Chinese character evaluation stage. The flow of the Chinese character detection stage is as follows: and carrying out binarization on the identified image, extracting a handwriting part and forming a binary image. Inputting the binary image into a yolo neural network-based feature extraction module, and extracting features through convolutional layers. And on the feature map, predicting a boundary frame of each pixel point. And screening the boundary frame to obtain a predicted boundary frame. The handwritten Chinese characters are detected through a yolov5s model, the cut Chinese characters are identified through a CNN model, then a corresponding regular script template is generated, the contact ratio is calculated, scoring of the handwritten Chinese characters is realized through the contact ratio, the difference between the font of the handwritten Chinese characters and the regular script template can be evaluated more visually, and the standard degree of the handwritten Chinese characters is evaluated.
Description
Technical Field
The invention belongs to the technical field of pattern recognition, and particularly relates to a handwritten Chinese character detection and recognition algorithm and a character pattern evaluation method.
Background
With the rapid development of computer technology, especially smart terminals such as mobile phones play more and more important roles in people's life, handwritten Chinese character detection and recognition has become an important recognition mode for smart terminals. Specifically, handwritten character recognition refers to a process of machine scanning and analyzing original handwritten characters of a user, scoring a writing track to obtain an image of the written characters or symbols, and recognizing the characters through a recognition algorithm by a computer. With the wide development of deep learning, the application of deep learning in the field of handwritten Chinese character recognition has achieved great success.
Since Yann LcCun et al designed and applied to the task of handwritten digit recognition, convolutional neural networks have been developed rapidly and have become an important means for target recognition. Compared with a full connection layer in a common network, the convolution layer in the convolution network can better extract image characteristics by taking local relevance in an image into consideration, and further helps to improve the accuracy of the model. In addition, in a full-connection mode, N pairs of weights w and offsets b are needed for outputting vectors of N elements, and each output of a feature map in a convolution network shares the same convolution kernel, so that the parameter quantity is greatly reduced, and the reasoning speed of the model is improved.
YOLO (Young Look one) is a target detection model developed by Ultralytics and open at Github. After thousands of hours of training and improvement on an anonymous client data set, the minimum YOLOv5s model can reach an inference speed of about 5 milliseconds per picture in a GPU environment.
The present invention uses yolov5s model as Chinese character detector and a convolutional neural network comprising three convolutional layers and two fully connected layers as classifier. By means of the web technology, the recognition result can be visually displayed to the user, and the recognition result comprises the original handwriting of the user, the generated result of the superposition of the regular script template and the score.
Disclosure of Invention
The invention aims to provide a handwritten Chinese character detection, identification and font evaluation method, which realizes identification of handwritten Chinese character samples and font quality evaluation.
The handwritten Chinese character detection and identification and font evaluation method based on the neural network comprises the following specific steps:
step one, Chinese character detection stage, the concrete flow is:
1-1, carrying out binarization on the identified image, extracting a handwriting part and forming a binary image.
And 1-2, inputting the binary image into a yolo neural network-based feature extraction module, and extracting features through convolutional layers.
And 1-3, on the characteristic graph obtained in the step 1-2, predicting a boundary frame of each pixel point.
And 1-4, screening the boundary box to obtain a prediction boundary box.
Step two, Chinese character recognition stage, the concrete flow is: and C, cutting the Chinese characters according to the prediction boundary boxes obtained in the step I, zooming the Chinese characters into single character pictures with preset sizes, and inputting the single character pictures into a classifier to obtain Chinese character category prediction values corresponding to the single character pictures.
Step three, Chinese character evaluation stage, the concrete flow is:
and 3-1, respectively selecting corresponding regular script templates according to the predicted value of each Chinese character, wherein the size of the regular script templates is the same as that of the single character picture.
3-2, calculating the intersection area S of the regular script template and the single character picture∩And surfaceProduct S∪。
3-3. calculating the normative score
The higher the score, the more normative the handwritten Chinese character.
Preferably, in step 1-2, the yolo neural network comprises 23 convolution modules and 1 detection layer.
Preferably, in steps 1-4, a non-maximum suppression algorithm is used for the bounding box screening.
Preferably, the predetermined size in step two is 128 pixels long and wide.
Preferably, the classifier described in the second step adopts a CNN model, which includes three convolutional layers and 2 fully-connected layers and can output a probability of 3755 chinese characters. After the CNN model extracts the characteristics of the single character pictures, the obtained characteristic graph is expanded into a one-dimensional vector and sent into a full connection layer for classification, a softmax layer in the full connection layer obtains the probability of 3755 categories, and the category corresponding to the maximum probability value is used as a predicted value in the single character picture.
The invention has the beneficial effects that:
the handwritten Chinese characters are detected through the yolov5s model, the cut Chinese characters are identified through the CNN model, then the corresponding regular script template is generated, the contact ratio is calculated, the scoring of the handwritten Chinese characters is realized through the contact ratio, the difference between the font of the handwritten Chinese characters and the regular script template can be evaluated more visually, and the standard degree of the handwritten Chinese characters is evaluated.
Drawings
FIG. 1 is an overall architecture diagram of the present invention;
FIG. 2a is a schematic diagram of the network structure of yolov5s model used in the present invention;
FIG. 2b is a schematic diagram of a network structure of a CNN model used in the present invention;
FIG. 3a is a schematic diagram of the detection result of a sample with background according to the present invention;
FIG. 3b is a diagram illustrating the detection result of the sample without background according to the present invention;
FIG. 4 is a diagram illustrating classification results of a CNN model according to the present invention;
FIG. 5 is a schematic diagram of scoring a whole handwritten sample according to the present invention.
Detailed Description
In order to make the objects, contents, and advantages of the present invention clearer, the following detailed description of the embodiments of the present invention will be made in conjunction with the accompanying drawings and examples.
The invention provides a handwritten Chinese character detection and recognition and font evaluation method based on a neural network. The method realizes the evaluation of the character pattern of the handwritten Chinese character sample, and completes the automation of the character pattern scoring of the handwritten Chinese character to a certain extent. The invention is oriented to common handwritten Chinese character recognition tasks and refers to handwritten Chinese characters based on white background and black characters.
The method comprises two stages of handwritten Chinese character recognition and character key stroke analysis. The handwritten Chinese character recognition stage adopts a recurrent neural network algorithm with a self-attention mechanism, the algorithm generates a corresponding weight value for the hidden layer state vector of the top layer of the neural network at each moment, then the weight values are used for solving the weighted sum of the state vectors of the top layer at all the moments as the output of the neural network, and the recognition precision of the handwritten Chinese character is improved by using the self-attention mechanism. Secondly, after the neural network training is finished, giving an input character sample, extracting the weight of each corresponding track point, binarizing the weight vector by using a specific threshold value, combining the binarized weight vector with the track of the input character sample, and performing visual evaluation on the importance degree of each track point in the track of the input character.
As shown in fig. 1, the method for detecting, identifying and evaluating handwritten Chinese characters based on neural network comprises the following specific steps:
step one, Chinese character detection stage, the concrete flow is:
1-1, the data preprocessing module receives the recognized handwritten Chinese character picture, and extracts black handwriting parts from the picture to form a binary image.
And 1-2, receiving the binary image obtained in the step 1-1 by a yolo neural network-based feature extraction module, and extracting features through a convolutional layer to obtain feature images of three scales. As shown in fig. 2a, the yolo neural network employs the yolov5s model, which contains 23 convolution modules and 1 detection layer.
And 1-3, predicting a boundary frame of each pixel point on the feature map of the three scales.
And 1-4, screening the boundary box by using a non-maximum suppression algorithm to obtain the most reasonable prediction boundary box.
Step two, Chinese character recognition stage, the concrete flow is:
and 2-1, cutting Chinese characters on the original image according to the prediction boundary frames obtained in the step one, cutting the Chinese characters in the prediction boundary frames independently, and uniformly zooming the Chinese characters into single character pictures with the length and the width of 128 pixels. Respectively inputting each single character picture into a classifier; as shown in fig. 2b, the classifier uses a CNN model, which includes three convolutional layers and 2 full-connected layers, and is capable of outputting a probability of 3755 chinese characters.
And 2-2, respectively extracting the characteristics of each single character picture through a Convolutional Neural Network (CNN).
And 2-3, unfolding the characteristic diagram obtained in the step 2-2 into a one-dimensional vector, and sending the one-dimensional vector into a full-connection layer for classification.
And 2-4, obtaining the probability of 3755 categories by using the softmax layer in the full connection layer, wherein the category corresponding to the maximum probability value is used as a predicted value (namely a Chinese character recognition result) in the single character picture.
Step three, Chinese character evaluation stage, the concrete flow is:
and 3-1, respectively generating corresponding regular script templates according to the predicted value of each Chinese character, wherein the size of each regular script template is 128 pixels in length and width.
And 3-2, covering the generated regular script template on the binary image obtained in the step 1-1 to obtain an intersection and a union set.
3-3. calculating the normative score
Wherein S is∩The intersection area (i.e., the number of pixels) in step 3-2; s∪Is the union area in step 3-2. Higher scores indicate more normative handwritten Chinese characters.
The invention identifies the regular script sample with the background and the regular script sample without the background, and the results are respectively shown in fig. 3a and 3b, so that the invention can accurately identify the position and the category of the Chinese character and provides higher and stable scores.
The invention identifies single handwritten Chinese character and a plurality of handwritten Chinese characters in a whole page, and the results are respectively shown in figures 4 and 5.
Claims (5)
1. A handwritten Chinese character detection and identification and font evaluation method is characterized in that: step one, Chinese character detection stage, the concrete flow is:
1-1, carrying out binarization on the identified image, extracting a handwriting part and forming a binary image;
1-2, inputting the binary image into a yolo neural network-based feature extraction module, and extracting features through convolutional layers;
1-3, on the characteristic graph obtained in the step 1-2, predicting a boundary frame of each pixel point;
1-4, screening the boundary box to obtain a prediction boundary box;
step two, Chinese character recognition stage, the concrete flow is: performing Chinese character cutting according to each prediction boundary box obtained in the step one, zooming the Chinese characters into a single character picture with a preset size, and inputting the single character picture into a classifier to obtain a Chinese character category prediction value corresponding to the single character picture;
step three, Chinese character evaluation stage, the concrete flow is:
3-1, respectively selecting a corresponding regular script template according to the predicted value of each Chinese character, wherein the size of the regular script template is the same as that of the single character picture;
3-2, calculating the intersection area S of the regular script template and the single character picture∩Sum and gather area S∪；
3-3. calculating the normative score
The higher the score, the more K the handwritten Chinese character isAnd (5) standardizing.
2. The method for detecting, recognizing and evaluating handwritten Chinese characters according to claim 1, wherein: in step 1-2, the yolo neural network comprises 23 convolution modules and 1 detection layer.
3. The method for detecting, recognizing and evaluating handwritten Chinese characters according to claim 1, wherein: in steps 1-4, a non-maximum suppression algorithm is used for the screening of the bounding box.
4. The method for detecting, recognizing and evaluating handwritten Chinese characters according to claim 1, wherein: the predetermined size in step two is 128 pixels long and wide.
5. The method for detecting, recognizing and evaluating handwritten Chinese characters according to claim 1, wherein: the classifier in the second step adopts a CNN model which comprises three convolutional layers and 2 full-connection layers and can output the probability of 3755 Chinese characters; after the CNN model extracts the characteristics of the single character pictures, the obtained characteristic graph is expanded into a one-dimensional vector and sent into a full connection layer for classification, a softmax layer in the full connection layer obtains the probability of 3755 categories, and the category corresponding to the maximum probability value is used as a predicted value in the single character picture.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111520803.6A CN114359917A (en) | 2021-12-13 | 2021-12-13 | Handwritten Chinese character detection and recognition and font evaluation method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111520803.6A CN114359917A (en) | 2021-12-13 | 2021-12-13 | Handwritten Chinese character detection and recognition and font evaluation method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114359917A true CN114359917A (en) | 2022-04-15 |
Family
ID=81099465
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111520803.6A Pending CN114359917A (en) | 2021-12-13 | 2021-12-13 | Handwritten Chinese character detection and recognition and font evaluation method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114359917A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116012860A (en) * | 2022-12-29 | 2023-04-25 | 华南师范大学 | Teacher blackboard writing design level diagnosis method and device based on image recognition |
| CN117472257A (en) * | 2023-12-28 | 2024-01-30 | 广东德远科技股份有限公司 | Automatic regular script turning method and system based on AI algorithm |
-
2021
- 2021-12-13 CN CN202111520803.6A patent/CN114359917A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116012860A (en) * | 2022-12-29 | 2023-04-25 | 华南师范大学 | Teacher blackboard writing design level diagnosis method and device based on image recognition |
| CN116012860B (en) * | 2022-12-29 | 2024-01-16 | 华南师范大学 | Teacher blackboard writing design level diagnosis method and device based on image recognition |
| CN117472257A (en) * | 2023-12-28 | 2024-01-30 | 广东德远科技股份有限公司 | Automatic regular script turning method and system based on AI algorithm |
| CN117472257B (en) * | 2023-12-28 | 2024-04-26 | 广东德远科技股份有限公司 | Automatic regular script turning method and system based on AI algorithm |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110348319B (en) | Face anti-counterfeiting method based on face depth information and edge image fusion | |
| CN111488826B (en) | Text recognition method and device, electronic equipment and storage medium | |
| Nair et al. | Malayalam handwritten character recognition using convolutional neural network | |
| CN105608454B (en) | Character detecting method and system based on text structure component detection neural network | |
| CN111191695A (en) | Website picture tampering detection method based on deep learning | |
| CN114359917A (en) | Handwritten Chinese character detection and recognition and font evaluation method | |
| CN109299303B (en) | Hand-drawn sketch retrieval method based on deformable convolution and depth network | |
| CN112069900A (en) | Bill character recognition method and system based on convolutional neural network | |
| CN112580507A (en) | Deep learning text character detection method based on image moment correction | |
| CN114360038B (en) | Weak supervision RPA element identification method and system based on deep learning | |
| Karunarathne et al. | Recognizing ancient sinhala inscription characters using neural network technologies | |
| CN112307919A (en) | Improved YOLOv 3-based digital information area identification method in document image | |
| Cerman et al. | A mobile recognition system for analog energy meter scanning | |
| CN115116074A (en) | Handwritten character recognition and model training method and device | |
| CN110414505A (en) | Processing method, processing system and the computer readable storage medium of image | |
| Akinbade et al. | An adaptive thresholding algorithm-based optical character recognition system for information extraction in complex images | |
| CN114187595A (en) | Document layout recognition method and system based on fusion of visual features and semantic features | |
| Sethia et al. | Gesture recognition for American sign language using Pytorch and Convolutional Neural Network | |
| CN117437647B (en) | Oracle character detection method based on deep learning and computer vision | |
| Nandhini et al. | Sign language recognition using convolutional neural network | |
| Oktaviani et al. | Optical character recognition for hangul character using artificial neural network | |
| Varkentin et al. | Development of an application for car license plates recognition using neural network technologies | |
| Singh et al. | A comprehensive survey on Bangla handwritten numeral recognition | |
| Chen et al. | Scene text recognition based on deep learning: a brief survey | |
| CN112926569B (en) | Method for detecting natural scene image text in social network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |