CN114550179B - Method, system and equipment for guiding handwriting Chinese character blackboard writing - Google Patents
Method, system and equipment for guiding handwriting Chinese character blackboard writing Download PDFInfo
- Publication number
- CN114550179B CN114550179B CN202210024764.9A CN202210024764A CN114550179B CN 114550179 B CN114550179 B CN 114550179B CN 202210024764 A CN202210024764 A CN 202210024764A CN 114550179 B CN114550179 B CN 114550179B
- Authority
- CN
- China
- Prior art keywords
- blackboard
- writing
- layer
- guiding
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000012549 training Methods 0.000 claims abstract description 26
- 238000007781 pre-processing Methods 0.000 claims abstract description 12
- 238000013135 deep learning Methods 0.000 claims abstract description 11
- 239000002775 capsule Substances 0.000 claims description 31
- 230000006870 function Effects 0.000 claims description 17
- 238000004422 calculation algorithm Methods 0.000 claims description 12
- 230000004913 activation Effects 0.000 claims description 8
- 238000011176 pooling Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 5
- 239000011159 matrix material Substances 0.000 claims description 5
- 238000012545 processing Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 6
- 230000009466 transformation Effects 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 4
- 230000006835 compression Effects 0.000 description 3
- 238000007906 compression Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000035945 sensitivity Effects 0.000 description 3
- 235000009854 Cucurbita moschata Nutrition 0.000 description 2
- 240000001980 Cucurbita pepo Species 0.000 description 2
- 235000009852 Cucurbita pepo Nutrition 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 235000020354 squash Nutrition 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000003796 beauty Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/20—Education
- G06Q50/205—Education administration or guidance
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Business, Economics & Management (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Tourism & Hospitality (AREA)
- Educational Administration (AREA)
- Educational Technology (AREA)
- Strategic Management (AREA)
- Probability & Statistics with Applications (AREA)
- Economics (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- General Business, Economics & Management (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Drawing Aids And Blackboards (AREA)
- Character Input (AREA)
Abstract
A method for guiding handwriting blackboard Chinese characters on the basis of deep learning comprises the following steps: s100: collecting images of blackboard writing of blackboard Chinese characters for guidance as a training image set; s200: preprocessing the training image set; s300: constructing a blackboard writing overall layout data set and a single Chinese character aesthetic feeling data set; s400: constructing and training a blackboard-writing overall layout instruction model by utilizing the blackboard-writing overall layout data set; s500: constructing and training a single Chinese character aesthetic feeling degree guiding model by utilizing the single Chinese character aesthetic feeling degree data set; s600: and guiding the handwritten blackboard writing by using the trained blackboard writing overall layout guiding model and the single Chinese character aesthetic feeling guiding model. The method overcomes the problems of low efficiency, more human interference factors and the like when the traditional method needs to conduct the writing guidance by means of people, and provides assistance and support for the writing guidance.
Description
Technical Field
The present disclosure relates to computer image processing technology, and is especially one method, system and apparatus for guiding hand writing Chinese character blackboard writing.
Background
The blackboard writing is an important component of classroom teaching, is an effective means for teachers to guide students to master knowledge and form knowledge capability, and has excellent blackboard writing design and writing capability which are one of essential basic teaching skills for teachers. Therefore, the learning, design and writing capability culture of the blackboard writing theory are important contents of teaching capability culture and practice of students in the family stage of the universities and the universities, and the guidance is an indispensable part in the blackboard writing capability culture process, so that the reasonable, efficient and accurate guidance is beneficial to improving and promoting the thinking of students and further improving the blackboard writing quality of the students.
However, in the field of blackboard writing guidance, the currently adopted mode is a mode of manual guidance, namely, a review group is composed of a plurality of expertise experts, and the blackboard writing condition of a writer is guided. This has problems such as large workload and low efficiency.
Disclosure of Invention
In order to solve the problems, the present disclosure provides a method for guiding a handwriting blackboard writing of Chinese characters based on deep learning, comprising the following steps:
s100: collecting images of blackboard writing of blackboard Chinese characters for guidance as a training image set;
S200: preprocessing the training image set;
s300: constructing a blackboard writing overall layout data set and a single Chinese character aesthetic feeling data set;
s400: constructing and training a blackboard-writing overall layout instruction model by utilizing the blackboard-writing overall layout data set;
S500: constructing and training a single Chinese character aesthetic feeling degree guiding model by utilizing the single Chinese character aesthetic feeling degree data set;
S600: and guiding the handwritten blackboard writing by using the trained blackboard writing overall layout guiding model and the single Chinese character aesthetic feeling guiding model.
The disclosure also provides a system for guiding the writing of the handwriting blackboard Chinese characters based on deep learning, which comprises a client and a server, wherein,
The client comprises a user information registration unit, a login unit, a user writing information acquisition unit and a guiding result presentation unit;
The server comprises a preliminary data preprocessing unit, a subsequent data preprocessing unit, a blackboard writing overall layout guiding unit and a single Chinese character aesthetic feeling guiding unit.
The present disclosure also provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the method for guiding writing blackboard chinese characters based on deep learning when executing the computer program.
According to the technical scheme, aiming at the problems that the existing blackboard-writing guidance analysis is performed by relying on manual experience, is high in subjectivity and low in efficiency, and the like, the image is subjected to preliminary pretreatment by an existing image processing method by acquiring the blackboard-writing image uploaded by a user, and then the pretreatment of the image is completed by combining the existing image recognition and image segmentation algorithm. And detecting the area where the writing content of the blackboard writing is located in the blackboard and the relation of each character in each row and each line, and simultaneously acquiring the picture of each Chinese character. And guiding the board book from the two aspects of the whole layout of the board book and the aesthetic feeling degree of the single Chinese character through the trained Chinese character whole layout aesthetic feeling degree guiding model and the single Chinese character aesthetic feeling guiding model. The method can assist in guiding the writing ability of a writer, has high accuracy and is suitable for wide application.
Drawings
FIG. 1 is a flow chart of a method for guiding a handwritten blackboard Chinese character blackboard writing based on deep learning provided in one embodiment of the present disclosure;
FIG. 2 is a binarized operation chart of an image in one embodiment of the present disclosure;
FIG. 3 is a diagram of binarized image processing in one embodiment of the present disclosure;
FIG. 4 is a perspective transformation of an image in one embodiment of the present disclosure;
FIG. 5 is an exemplary diagram of a device in one embodiment of the present disclosure;
fig. 6 is a schematic view of a photographing angle in one embodiment of the present disclosure.
Detailed Description
In one embodiment, as shown in fig. 1, the present disclosure provides a method for guiding a handwriting blackboard writing of kanji based on deep learning, comprising the steps of:
s100: collecting images of blackboard writing of blackboard Chinese characters for guidance as a training image set;
S200: preprocessing the training image set;
s300: constructing a blackboard writing overall layout data set and a single Chinese character aesthetic feeling data set;
s400: constructing and training a blackboard-writing overall layout instruction model by utilizing the blackboard-writing overall layout data set;
S500: constructing and training a single Chinese character aesthetic feeling degree guiding model by utilizing the single Chinese character aesthetic feeling degree data set;
S600: and guiding the handwritten blackboard writing by using the trained blackboard writing overall layout guiding model and the single Chinese character aesthetic feeling guiding model.
For the embodiment, the method acquires the complete blackboard-writing image by acquiring the blackboard-writing image to be guided and correcting the image by means of image processing technologies such as Opencv and the like; recognizing the position information of the characters in the blackboard-writing from the level angles of the sections, the rows and the characters by means of the existing Chinese character recognition algorithm; further calculating the position proportion of the paragraph in the blackboard, the inclination degree of the lines, the interval relation between the lines, the size of the words, the interval relation between the words of the same line and the words, and the like; meanwhile, by means of an image cutting method, image information of a single Chinese character is obtained; a deep neural network is adopted to guide the layout of blackboard books in a blackboard and grade the aesthetic feeling of single Chinese characters; the problems of low efficiency, many human interference factors and the like when the traditional method needs to conduct the writing guidance by means of people are solved, and assistance and support are provided for the writing guidance.
In the process of collecting training images, a plurality of images of blackboard books used for guidance are collected as a training image set T blackboard-writing-old, the obtained data set is formed by writing one to four 6803 people larger than a certain affiliated university on a blackboard, the writing content is a Chinese famous warning sentence or an ancient poem, and each student writes about three crosses, and the regular script is used as a writing font.
In another embodiment, the step S200 further includes the steps of:
s201: extracting blackboard areas in the training image set;
s202: and identifying text content in the blackboard writing, and storing and arranging the identified information.
For this embodiment, in addition to the blackboard information, the single image information in the training image set T blackboard-writing-old often contains interference factors other than the blackboard. Therefore, it is necessary to process the image and extract the blackboard area.
A) According to the formulaAnd carrying out gray scale image processing on the image. Where i_r (x, y), i_g (x, y), i_b (x, y) represents the values of the RGB channels of the image, the gray values I (x, y) are obtained using an averaging method.
B) By the formulaA gray histogram is calculated, where γ k is the gray level of the pixel, n k is the number of pixels with gray level γ k, MN is the total number of pixels of the image, where M, N is the length and width of the image.
C) Considering comprehensively that for the blackboard writing image information, the pixel area occupied by the blackboard area is the largest, and the blackboard area is obviously different from the surrounding environment, and the image is converted into a binary image by means of the gray level histogram calculated in the last step, as shown in fig. 2.
D) The binarized image is input, and the operations of removing the cavity and the non-communication area are performed on the binarized image by means of opecv, so that an image P is obtained, as shown in fig. 3.
E) By detecting the outline of the blackboard in the image P, the coordinates of the four corners of the blackboard are determined.
F) The coordinate points corresponding to four angles of the blackboard obtained in the last step are based on the formulaProjective transformation is performed by projecting the picture into a new view plane, where (x, y) is the original image pixel coordinates, corresponding to the transformed image pixel coordinates (x ', y'). Wherein the matrix is transformedCan be split into four parts, namely, the four parts,Representing a linear transformation, [ m 6 m7 ] for translation, [ m 2 m5]T ] producing a perspective transformation, resulting in a blackboard image, the graphical transformation results being shown in figure 4.
G) And scaling the blackboard image to prepare for later work. The image is scaled to around 700 x 450.
And according to the existing Chinese character detection and recognition algorithm, recognizing text content in the blackboard writing, namely recognizing information such as each paragraph, each row and the position of each Chinese character in the blackboard writing. And then storing and sorting the identified information, and defining the position proportion of each section in the blackboard, the inclination angle of the rows, the interval relation between the rows, the size proportion between the words, the interval relation between each word and the like, so as to form the data information capable of describing the overall structural layout of the blackboard writing content. And dividing according to the position information of each Chinese character to cut each individual Chinese character, so as to prepare for aesthetic guidance of single Chinese character.
And recognizing the text information in the board book image by means of algorithms such as hundred-degree intelligent cloud high-precision handwritten Chinese character recognition and the like. In the embodiment, a hundred-degree intelligent cloud high-precision handwritten Chinese character recognition algorithm is used, so that text paragraphs, lines and characters in an image can be accurately recognized, and information such as horizontal position coordinates and vertical position coordinates of the paragraphs, lines and characters, width and height of an external rectangle can be fed back and output.
In another embodiment, the step S300 further includes the steps of:
S301: for the blackboard writing overall layout data set, the element information comprises the coordinate information of text paragraphs, lines and words in the blackboard, external matrix information and the overall pixels of the blackboard;
S302: for a single Chinese character aesthetic degree data set, cutting Chinese characters into pictures according to the circumscribed rectangle of each Chinese character, filling the images into N, wherein N is the resolution of the pictures, N is the value range of [64,224] under the condition that the text information in the images is not changed based on an image filling algorithm, and the image marking adopts a manual marking mode of an expert.
For this embodiment, the guidance on the sensitivity of the blackboard-writing needs to consider both aspects of the overall layout of the blackboard-writing and the sensitivity of the single chinese character, so two data sets, i.e., the overall layout data set T layout and the sensitivity data set T pic-aesthetic of the single chinese character, need to be constructed.
For the blackboard writing overall layout data set T layout, the element information is determined by the coordinate information of text paragraphs, lines and words in the blackboard, the external matrix information and the overall pixels of the blackboard. The labels are rated by expert according to self experience, and the dimension information is described from three levels of paragraphs, lines and words.
A) The text paragraph is located in the blackboard with the position description information of (X d,yd,Wd,hd), the blackboard pixel of (X p,Yp).(Xd,yd) representing the coordinates of the upper left corner of the rectangle circumscribed by the paragraph, W d representing the width of the rectangle circumscribed, and h d being the height of the rectangle circumscribed. By X d/Xp、yd/Yp、(Wd*hd)/(Xp*Yp) determines the three data elements of the vertical coordinate scale, the horizontal coordinate scale, and the outside rectangular pixel scale of the text paragraph.
B) The row-to-row spacing relationship, the degree of row tilt, is a major consideration in the overall layout problem.
The row-to-row spacing relationship is calculated as follows: (y h+1-yh-dh)/Yp. Wherein y h represents the y-axis coordinate point of the upper left corner of the circumscribed rectangle of the current line, d h represents the height of the circumscribed rectangle of the current line, y h+1 represents the y-axis coordinate point of the upper left corner of the circumscribed rectangle of the next line, yp is the width of the picture.
The row tilt is calculated as follows: arctan ((x 1, y 1), (x 2, y 2)), where (x 1, y 1) represents the coordinates of the top left corner of the circumscribed rectangle of the first word of the current line and (x 2, y 2) represents the coordinates of the top left corner of the circumscribed rectangle of the last word of the current line, while converting the calculated angle into radians.
For the interval relation between rows and the inclination degree of the rows, in order to ensure the accuracy and the effectiveness of data, the maximum value, the minimum value, the average value and the standard deviation of the row spacing are used as characteristic dimensions. Meanwhile, in order to ensure the effectiveness of the data, all feature dimension data are normalized, and the formula is as follows: where x i represents the current data value, min x represents the minimum value of the data in the current dimension, max x represents the maximum value of the data in the current dimension, and b and a represent the corresponding maximum and minimum values after the data is compressed.
C) The size of individual words, word-to-word spacing, are also important considerations in overall layout issues.
The size of a single word is calculated as follows: Wherein w and h are the width and height of a single Chinese character circumscribed rectangle, and X P,Yp is the width and height of a blackboard image.
The word-to-word spacing relationship is calculated as follows: (X h+1-Xh-W)/Xp where X h represents the horizontal axis coordinate of the upper left corner of the current word bounding rectangle, w is the width of the current word bounding rectangle, X h+1 represents the horizontal axis coordinate of the upper left corner of the left word adjacent to the current word, and X p is the width of the blackboard image.
For the size of a single word and the interval relation between words, in order to ensure the accuracy and effectiveness of data, the maximum value, the minimum value, the average value, the standard deviation of the word, the maximum value, the minimum value, the average value and the standard deviation of the word interval are used as characteristic dimensions. And simultaneously carrying out normalization processing on all the feature dimension data.
In this embodiment, the T layout dataset contains 21 feature dimensions altogether, the tag is calibrated by an expert, the score is determined by a percentile, and for 6803 data originally collected, T layout contains 6377 usable data altogether through screening and culling of the data.
For a single Chinese character aesthetic degree data set T pic-aesthetic, cutting Chinese characters into pictures according to the circumscribed rectangle of each Chinese character, filling the images into 64 x 3 under the condition of ensuring that the text information in the images is not changed based on an image filling algorithm, and marking the images in a manual marking mode of an expert. The multiple experts conduct guidance of the beauty degree according to own experience, and the guidance grade is classified into excellent grade, good grade, poor grade and the like.
In another embodiment, the blackboard writing overall layout guidance model in step S400 includes an input layer, a hidden layer and an output layer, where the hidden layer is a three-layer fully-connected layer, and the output layer activation function is a sigmoid activation function.
For this embodiment, the data set used in the model is T layout, and a neural network model is constructed, where the model includes an input layer, a hidden layer, and an output layer, and the hidden layer is a three-layer fully-connected layer, i.e., a Linear layer. The activation function is a ReLU activation functionThe output layer activation function is a sigmoid activation function
In another embodiment, the loss function used in the overall layout guidance model of the blackboard writing in step S400 is root mean square error, and the gradient descent algorithm is used for training, so that the loss is reduced until convergence, and the trained overall layout guidance model of the blackboard writing is obtained.
For this embodiment, the loss function used by the model is the mean square errorN is the training number, and in this embodiment, n takes a value of 50. Where y i is the original label and,To predict tags. Training by using a gradient descent algorithm, reducing loss until convergence, and further obtaining a required blackboard writing overall layout guidance model.
In another embodiment, the single kanji aesthetic guide model in step S500 includes a 1-VGG model for extracting image element information and a 2-CapsNet model for guiding the positional relationship of elements.
For this embodiment, the data set used for the model training is T pic-aesthetic. The core elements of Chinese characters comprise strokes and structural relations among the strokes. The traditional convolutional neural network for processing the image can only identify the elements in the image, lacks the capability of processing the relationship among the elements, and takes the vector as a basic unit to consider the position relationship among the elements. Therefore, the method combines the advantages of the two, and provides a network model of VGG-CapsNet: a 1-VGG for extracting image element information and a 2-CapsNet model for guiding the positional relationship of elements are included.
The method for calculating the aesthetic feeling of the single Chinese character comprises the following steps:
step1: for the 1-VGG model, the image input size is 64x64x3, the image size is 16x16x512 after feature extraction, and the output result is input into a conv5 convolution layer with the value of 2-CapsNet for convolution operation.
Step2: inputting the operation result of conv5 into a main capsule layer of 2-CapsNet, and reshaping the result into 16 8D capsules to obtain an output vector v of the capsules.
Step3: compression function by means of squashAnd compressing the length of the output vector v in the last step to 0-1. V is the input vector and V is the output vector in squash compression functions.
Step4: the main capsule layer is connected with the digital capsule layer, and a dynamic routing algorithm is operated on the digital capsule layer. The high-level capsules of the digital capsule layer receive the output vector v of the low-level capsules of the main capsule layer and multiply the output vector v with the corresponding weight matrix to obtain a prediction vector for predicting the position where the high-level features should exist.
Step5: calculating the coupling coefficient of the low-layer capsule and the high-layer capsuleWhich represents the weight that the low-level capsule output vector v can transfer to the high-level capsules, the number of which is equal to the number of high-level capsules. Where c ij is the coupling coefficient determined by the dynamic routing iteration process, and b ij identifies the logarithmic prior probability of the coupling of the bottom layer capsule to the higher layer capsule. After the first iteration, all values of C ij are equal, after which C ij is updated by the dynamic routing process, and the initial value of b ij is 0.
Step6: the predicted vector v is weighted and summed to obtain the total input vector s.
Step7: the vector S is compressed by means of squash compression functions, the length of the output vector is ensured to be 0-1, and the direction of the output vector is not changed.
Step8: calculating the dot product of the output vector of the capsule and the prediction vector, adding the old weight to obtain the new weight, and if the dot product value is a positive number, indicating that the predicted output of the capsule has consistency with the actual output of the capsule, so that the weight coefficient is increased, otherwise, the weight coefficient is decreased, and thus, the routing iteration operation is completed.
Step9: and obtaining a final output vector after 3 routing iterations, and calculating the length of the output vector, namely the probability of the aesthetic class of the current Chinese character.
The loss function used by the model is ：Lk＝Tkmax(0,m+-||vk||)2+λ(1-Tk)max(0,||vk||-m-)2,, where k is the aesthetic class of Chinese characters, T k indicates whether the predicted class exists, T k is 1 when the predicted class exists, and T k is 0 when the predicted class does not exist. m+ is a criterion for false positives, indicating that when the detected object is present and the actual output value of the network is smaller than m+, the total loss is increased. And when the predicted result is greater than m+, the total loss is unchanged. m-represents a criterion for false negatives, indicating that the total loss increases when the detected object is absent and the actual output value of the network is greater than m-. And when the existence probability of the predicted result category is smaller than m-, the total loss is unchanged. For the case where k is not present, the λ weight causes the initial learning process to no longer shrink the length of all capsules. Specifically, when k is present, the latter half of the formula is 0, and the longer the length of the capsule, the smaller the loss function; when k is absent, the first half of the formula is 0 and the shorter the length of the capsule, the smaller the loss function.
In another embodiment, the 1-VGG model comprises 4 two-dimensional convolution layers, each layer being connected in sequence: convolutional layer Conv1, reLU layer Conv1-ReLU, pooling layer Conv2, reLU layer Conv2-ReLU, pooling layer MaxPool d, convolutional layer Conv3, reLU layer Conv3-ReLU, pooling layer MaxPool d, convolutional layer Conv4; wherein the convolution kernel used by the 1-VGG model is 3*3.
For this embodiment, the convolution kernel used by the 1-VGG model is 3*3, and the use of a small convolution can improve the depth of the network, improve the effect of the neural network to a certain extent, and reduce the parameters at the same time under the condition of ensuring the same perceived field.
In another embodiment, the 2-CapsNet model comprises a convolution layer, a main capsule layer, a digital capsule layer and three full-connection layers, wherein the three full-connection layers are sequentially connected as follows: convolutional layer conv5, main capsule layer PRIMARYCAPS, digital capsule layer DigCaps, three full link layers link.
In another embodiment, the present disclosure discloses a system for guiding handwritten blackboard writing of blackboard characters based on deep learning, comprising a client and a server, wherein,
The client comprises a user information registration unit, a login unit, a user writing information acquisition unit and a guiding result presentation unit;
The server comprises a preliminary data preprocessing unit, a subsequent data preprocessing unit, a blackboard writing overall layout guiding unit and a single Chinese character aesthetic feeling guiding unit.
For the embodiment, the preliminary preprocessing of the data is to correct the blackboard writing image uploaded by the client, and only blackboard information is left after external interference factors are removed; the subsequent preprocessing of the data detects the text information in the blackboard information, completes the detection of the position and the calculation of the proportion from the three angles of the section, the line and the word, and divides and stores each text into images by means of the image dividing technology; the overall layout guidance of the blackboard writing is to score the layout of the blackboard writing by means of a deep neural network; the single Chinese character aesthetic feeling degree guidance is to grade the aesthetic feeling degree of the single Chinese character by means of VGG-CapsNet neural network.
The basic business flow is as follows: firstly, whether a user is registered or not is detected, if the user is not registered, the user is required to register, if the user is registered, the user logs in, then the user selects an image of an blackboard to be guided, after clicking on the image to start guiding, the image of the blackboard is automatically uploaded to a server, the server is responsible for implementing the guiding function, after finishing, the result is returned to the client, the user can check the overall layout score of the blackboard of the uploaded image of the blackboard and the aesthetic degree grade of Chinese characters, then the information of the blackboard guiding is saved to a data set, and meanwhile, the user can select to request the server again to guide or stop guiding in the middle of guiding.
In another embodiment, a computer device includes a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the deep learning-based method of guiding handwritten blackboard writing.
In another embodiment, an apparatus is provided that includes a means for writing instruction information entry and retrieval, a memory, a processor, and a system program stored on the memory for processing by the processor.
Aiming at the writing instruction information input and acquisition device, include: the system comprises an image acquisition module, an information input module, a control terminal module and a support moving module, which are shown in fig. 5.
The image acquisition module comprises a camera and a connecting rod, wherein the camera is used for acquiring information input by the information input module, and the connecting rod plays a role in supporting and connecting. For the camera, the maximum visual angle a of the lens of the camera is larger than the included angle b formed by the camera and the edge of the information input module, as shown in fig. 6.
The information input module comprises a blackboard and a fixed support, the blackboard is used for inputting blackboard writing test information by a tester, and the fixed support plays a role in fixing and supporting the blackboard.
The control terminal module is used for the operations of logging in a user account, acquiring writing information and the like, and is internally provided with a memory and a processor for running a client program, network connection equipment for connecting a network and other hardware devices.
The supporting and moving module consists of a fixed support, four wheels at the bottom of the support and fixed buckles, and a user can freely move the blackboard writing guidance information input and acquisition device when the user needs to move, and the wheels can be firmly fixed on the ground through the fixed buckles when the user moves to the corresponding position, so that the blackboard is not rocked when writing.
A memory for storing a system program; the program is stored in the memory and may be called for processing by the processor.
And a processor for executing the system program stored in the memory. The memory is coupled to the processor by a bus.
Although the embodiments of the present invention have been described above with reference to the accompanying drawings, the present invention is not limited to the above-described specific embodiments and application fields, and the above-described specific embodiments are merely illustrative, and not restrictive. Those skilled in the art, having the benefit of this disclosure, may effect numerous forms of the invention without departing from the scope of the invention as claimed.
Claims (5)
1. A method for guiding handwriting blackboard Chinese characters on the basis of deep learning comprises the following steps:
s100: collecting images of blackboard writing of blackboard Chinese characters for guidance as a training image set;
S200: preprocessing the training image set;
s300: constructing a blackboard writing overall layout data set and a single Chinese character aesthetic feeling data set;
s400: constructing and training a blackboard-writing overall layout instruction model by utilizing the blackboard-writing overall layout data set;
S500: constructing and training a single Chinese character aesthetic feeling degree guiding model by utilizing the single Chinese character aesthetic feeling degree data set;
S600: guiding the handwritten blackboard writing by using the trained blackboard writing overall layout guiding model and the single Chinese character aesthetic feeling guiding model;
Wherein,
The blackboard writing overall layout guidance model in the step S400 comprises an input layer, a hidden layer and an output layer, wherein the hidden layer is a three-layer full-connection layer, and the output layer activation function is a sigmoid activation function;
The loss function used by the blackboard-writing overall layout guidance model in the step S400 is root mean square error, a gradient descent algorithm is used for training, the loss is reduced until convergence, and then the trained blackboard-writing overall layout guidance model is obtained;
The single Chinese character aesthetic feeling degree guiding model in the step S500 comprises a 1-VGG model for extracting image element information and a 2-CapsNet model for guiding the position relation of elements;
The 1-VGG model comprises 4 two-dimensional convolution layers, and the layers are sequentially connected as follows: convolutional layer Conv1, reLU layer Conv1-ReLU, pooling layer Conv2, reLU layer Conv2-ReLU, pooling layer MaxPool d, convolutional layer Conv3, reLU layer Conv3-ReLU, pooling layer MaxPool d, convolutional layer Conv4; wherein, the convolution kernel used by the 1-VGG model is 3*3;
the 2-CapsNet model comprises a convolution layer, a main capsule layer, a digital capsule layer and three full-connection layers, wherein the three full-connection layers are sequentially connected: convolutional layer conv5, main capsule layer PRIMARYCAPS, digital capsule layer DigCaps, three full link layers link.
2. The method according to claim 1, said step S200 further comprising the steps of:
s201: extracting blackboard areas in the training image set;
s202: and identifying text content in the blackboard writing, and storing and arranging the identified information.
3. The method of claim 1, said step S300 further comprising the steps of:
S301: for the blackboard writing overall layout data set, the element information comprises the coordinate information of text paragraphs, lines and words in the blackboard, external matrix information and the overall pixels of the blackboard;
S302: for a single Chinese character aesthetic degree data set, cutting Chinese characters into pictures according to the circumscribed rectangle of each Chinese character, filling the images into N, wherein N is the resolution of the graph, N is the value range of [64,224] under the condition that the text information in the images is not changed based on an image filling algorithm, and the image marking adopts a manual marking mode of an expert.
4. A system for implementing the method for guiding the writing of blackboard Chinese characters based on deep learning according to any one of claims 1 to 3, comprising a client and a server, wherein,
The client comprises a user information registration unit, a login unit, a user writing information acquisition unit and a guiding result presentation unit;
The server comprises a preliminary data preprocessing unit, a subsequent data preprocessing unit, a blackboard writing overall layout guiding unit and a single Chinese character aesthetic feeling guiding unit.
5. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the deep learning based method of directing handwritten blackboard writing of chinese characters as claimed in any one of claims 1 to 3 when the computer program is executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210024764.9A CN114550179B (en) | 2022-01-11 | 2022-01-11 | Method, system and equipment for guiding handwriting Chinese character blackboard writing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210024764.9A CN114550179B (en) | 2022-01-11 | 2022-01-11 | Method, system and equipment for guiding handwriting Chinese character blackboard writing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114550179A CN114550179A (en) | 2022-05-27 |
| CN114550179B true CN114550179B (en) | 2024-08-09 |
Family
ID=81669740
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210024764.9A Active CN114550179B (en) | 2022-01-11 | 2022-01-11 | Method, system and equipment for guiding handwriting Chinese character blackboard writing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114550179B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114936787A (en) * | 2022-06-08 | 2022-08-23 | 武汉行已学教育咨询有限公司 | Online student teaching intelligent analysis management cloud platform based on artificial intelligence |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101393693A (en) * | 2008-09-12 | 2009-03-25 | 浙江大学 | Computer educating method for Chinese character writing |
| CN107844740A (en) * | 2017-09-05 | 2018-03-27 | 中国地质调查局西安地质调查中心 | A kind of offline handwriting, printing Chinese character recognition methods and system |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108710866B (en) * | 2018-06-04 | 2024-02-20 | 平安科技（深圳）有限公司 | Chinese character model training method, chinese character recognition method, device, equipment and medium |
| CN108985297A (en) * | 2018-06-04 | 2018-12-11 | 平安科技（深圳）有限公司 | Handwriting model training, hand-written image recognition methods, device, equipment and medium |
| CN109635883B (en) * | 2018-11-19 | 2023-04-18 | 北京大学 | Chinese character library generation method based on structural information guidance of deep stack network |
| CN109871851B (en) * | 2019-03-06 | 2023-05-12 | 长春理工大学 | Chinese character writing normalization judging method based on convolutional neural network algorithm |
| CN110298343A (en) * | 2019-07-02 | 2019-10-01 | 哈尔滨理工大学 | A kind of hand-written blackboard writing on the blackboard recognition methods |
| CN112434699A (en) * | 2020-11-25 | 2021-03-02 | 杭州六品文化创意有限公司 | Automatic extraction and intelligent scoring system for handwritten Chinese characters or components and strokes |
| CN113191309A (en) * | 2021-05-19 | 2021-07-30 | 杭州点望科技有限公司 | Method and system for recognizing, scoring and correcting handwritten Chinese characters |
-
2022
- 2022-01-11 CN CN202210024764.9A patent/CN114550179B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101393693A (en) * | 2008-09-12 | 2009-03-25 | 浙江大学 | Computer educating method for Chinese character writing |
| CN107844740A (en) * | 2017-09-05 | 2018-03-27 | 中国地质调查局西安地质调查中心 | A kind of offline handwriting, printing Chinese character recognition methods and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114550179A (en) | 2022-05-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110390269B (en) | PDF document table extraction method, device, equipment and computer readable storage medium | |
| US10789504B2 (en) | Method and device for extracting information in histogram | |
| CN112528963A (en) | Intelligent arithmetic question reading system based on MixNet-YOLOv3 and convolutional recurrent neural network CRNN | |
| US20240273932A1 (en) | Method for recognizing text, and apparatus | |
| Lin et al. | STAN: A sequential transformation attention-based network for scene text recognition | |
| CN111275082A (en) | Indoor object target detection method based on improved end-to-end neural network | |
| CN111488826A (en) | Text recognition method and device, electronic equipment and storage medium | |
| CN108399386A (en) | Information extracting method in pie chart and device | |
| BR112020018915A2 (en) | METHOD FOR IDENTIFYING AN OBJECT IN AN IMAGE AND MOBILE DEVICE FOR IMPLEMENTING THE METHOD | |
| CN112580507B (en) | Deep learning text character detection method based on image moment correction | |
| CN113223025B (en) | Image processing method and device, and neural network training method and device | |
| CN111626297A (en) | Character writing quality evaluation method and device, electronic equipment and recording medium | |
| CN109635687B (en) | Chinese character text line writing quality automatic evaluation method and system based on time sequence point set calculation | |
| CN111126459A (en) | Method and device for identifying fine granularity of vehicle | |
| CN116229056A (en) | Semantic segmentation method, device and equipment based on double-branch feature fusion | |
| CN111553423A (en) | Handwriting recognition method based on deep convolutional neural network image processing technology | |
| CN112800955A (en) | Remote sensing image rotating target detection method and system based on weighted bidirectional feature pyramid | |
| CN111666937A (en) | Method and system for recognizing text in image | |
| He et al. | Context-aware mathematical expression recognition: An end-to-end framework and a benchmark | |
| CN114550179B (en) | Method, system and equipment for guiding handwriting Chinese character blackboard writing | |
| CN115018999A (en) | Multi-robot-cooperation dense point cloud map construction method and device | |
| Zhong et al. | Least-squares method and deep learning in the identification and analysis of name-plates of power equipment | |
| CN114330247A (en) | Automatic insurance clause analysis method based on image recognition | |
| CN111611883B (en) | Table layout analysis method, system and equipment based on minimum cell clustering | |
| CN114694133B (en) | Text recognition method based on combination of image processing and deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |