CN114550179A - Method, system and equipment for guiding handwriting Chinese character blackboard writing - Google Patents

Method, system and equipment for guiding handwriting Chinese character blackboard writing Download PDF

Info

Publication number
CN114550179A
CN114550179A CN202210024764.9A CN202210024764A CN114550179A CN 114550179 A CN114550179 A CN 114550179A CN 202210024764 A CN202210024764 A CN 202210024764A CN 114550179 A CN114550179 A CN 114550179A
Authority
CN
China
Prior art keywords
blackboard
writing
chinese character
layer
guidance
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210024764.9A
Other languages
Chinese (zh)
Inventor
韩广欣
何聚厚
房蓓
李骏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shaanxi Normal University
Original Assignee
Shaanxi Normal University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shaanxi Normal University filed Critical Shaanxi Normal University
Priority to CN202210024764.9A priority Critical patent/CN114550179A/en
Publication of CN114550179A publication Critical patent/CN114550179A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services
    • G06Q50/20—Education
    • G06Q50/205—Education administration or guidance

Abstract

A method for guiding Chinese character writing on a handwriting blackboard based on deep learning comprises the following steps: s100: collecting images of blackboard Chinese character writing for guidance as a training image set; s200: preprocessing the training image set; s300: constructing a blackboard writing overall layout data set and a single Chinese character aesthetic degree data set; s400: constructing and training a blackboard writing overall layout guidance model by using the blackboard writing overall layout data set; s500: constructing and training a single Chinese character aesthetic feeling guidance model by using the single Chinese character aesthetic feeling data set; s600: and guiding the handwriting blackboard writing with the trained blackboard writing overall layout guidance model and the single Chinese character aesthetic feeling guidance model. The method solves the problems of low efficiency, multiple human interference factors and the like when people are needed to guide blackboard writing in the prior art, and provides assistance and support for blackboard writing guidance.

Description

Method, system and equipment for guiding handwriting Chinese character blackboard writing
Technical Field
The present disclosure belongs to the technical field of computer image processing, and particularly relates to a method, a system and a device for guiding writing of Chinese characters on a blackboard.
Background
The blackboard writing is an important component of classroom teaching, is an effective means for guiding students to master knowledge and form knowledge ability by teachers, and the excellent blackboard writing design and writing ability are one of the essential basic teaching skills of teachers. Therefore, the learning, designing and writing ability cultivation of the blackboard writing theory is an important content of teaching ability cultivation and practice of the teacher university students in the subject stage, and guidance is an indispensable part in the blackboard writing ability cultivation process, so that reasonable, efficient and accurate guidance is beneficial to improving and promoting the thinking of the students and further improving the blackboard writing quality of the students.
However, in the field of blackboard writing guidance, the currently adopted mode is a manual guidance mode, namely, a review group is composed of experts with abundant experience to guide the blackboard writing condition of a writer. This has problems such as a large workload and low efficiency.
Disclosure of Invention
In order to solve the above problems, the present disclosure provides a method for guiding a writing blackboard Chinese character blackboard writing based on deep learning, which includes the following steps:
s100: collecting images of blackboard Chinese character writing for guidance as a training image set;
s200: preprocessing the training image set;
s300: constructing a blackboard writing overall layout data set and a single Chinese character aesthetic degree data set;
s400: constructing and training a blackboard writing overall layout guidance model by using the blackboard writing overall layout data set;
s500: constructing and training a single Chinese character aesthetic feeling guidance model by using the single Chinese character aesthetic feeling data set;
s600: and guiding the handwriting blackboard writing with the trained blackboard writing overall layout guidance model and the single Chinese character aesthetic feeling guidance model.
The present disclosure also provides a system for guiding Chinese character writing on a handwriting blackboard based on deep learning, comprising a client and a server, wherein,
the client comprises a user information registration and login unit, a user writing information acquisition unit and a guidance result presentation unit;
the server comprises a preliminary data preprocessing unit, a subsequent data preprocessing unit, a blackboard writing overall layout guiding unit and a single Chinese character aesthetic degree guiding unit.
The disclosure also provides a computer device, which includes a memory, a processor and a computer program stored in the memory and capable of running on the processor, and is characterized in that the processor implements the method for guiding the writing blackboard Chinese character writing based on deep learning when executing the computer program.
By the technical scheme, aiming at the problems that the existing blackboard-writing guidance analysis is carried out by depending on manual experience, the subjectivity is strong, the efficiency is low and the like, the blackboard-writing image uploaded by a user is obtained, the image is subjected to preliminary preprocessing by the existing image processing method, and then the preprocessing of the image is completed by combining the existing image recognition and image segmentation algorithms. The relation between the area where the writing content of the blackboard writing is located in the blackboard and each character in each line and row is detected, and the picture of each Chinese character is obtained at the same time. And then the blackboard writing is guided in the aspects of the whole layout of the blackboard writing and the aesthetic feeling degree of a single Chinese character through the trained Chinese character whole layout aesthetic feeling guiding model and the single Chinese character aesthetic feeling guiding model. The method can assist in guiding the writing ability of a writer, has high accuracy and is suitable for wide application.
Drawings
FIG. 1 is a flowchart of a method for guiding Chinese character writing on a handwriting blackboard based on deep learning according to an embodiment of the present disclosure;
FIG. 2 is a diagram of a binarization operation of an image in one embodiment of the disclosure;
FIG. 3 is a binarized image processing diagram in one embodiment of the present disclosure;
FIG. 4 is a perspective transformation view of an image in one embodiment of the present disclosure;
FIG. 5 is an exemplary diagram of an apparatus in one embodiment of the present disclosure;
fig. 6 is a schematic view of a shooting angle in one embodiment of the present disclosure.
Detailed Description
In one embodiment, as shown in fig. 1, the present disclosure provides a method for guiding a writing blackboard Chinese character writing based on deep learning, which includes the following steps:
s100: collecting images of blackboard Chinese character writing for guidance as a training image set;
s200: preprocessing the training image set;
s300: constructing a blackboard writing overall layout data set and a single Chinese character aesthetic degree data set;
s400: constructing and training a blackboard writing overall layout guidance model by using the blackboard writing overall layout data set;
s500: constructing and training a single Chinese character aesthetic degree guide model by using the single Chinese character aesthetic degree data set;
s600: and guiding the handwriting blackboard writing with the trained blackboard writing overall layout guidance model and the single Chinese character aesthetic feeling guidance model.
For the embodiment, the method obtains the blackboard writing image to be guided, and corrects the image by means of image processing technologies such as Opencv and the like to obtain a complete blackboard writing image; recognizing the position information of characters in the blackboard-writing from the level angles of segments, lines and characters by means of the existing Chinese character recognition algorithm; further calculating information such as position proportion of the paragraph in the blackboard, inclination degree of the line, interval relation between lines, size of characters, interval relation between characters in the same line and the like; simultaneously, acquiring image information of a single Chinese character by means of an image cutting method; guiding the layout of the blackboard writing in the blackboard and grading the aesthetic feeling of a single Chinese character by adopting a deep neural network; the problems that the efficiency is low and the number of human interference factors is large when people are needed to guide blackboard writing in the prior art are solved, and assistance and support are provided for blackboard writing guidance.
Collecting a plurality of images of blackboard writing for guidance in the collection of training images as a training image set Tblackboard-writing-oldThe acquired data set is formed by writing one to four 6803 people from a college university to a college university on a blackboard, the writing contents are Chinese famous-word police sentences or ancient poems, each classmate writes about thirty characters, and the regular characters are used as writing characters.
In another embodiment, the step S200 further includes the steps of:
s201: extracting a blackboard area in the training image set;
s202: and recognizing the text content in the blackboard-writing, and storing and sorting the recognized information.
For this embodiment, T is the set of training imagesblackboard-writing-oldThe single image information in (1) often contains interference factors other than the blackboard in addition to the blackboard information. Therefore, it is necessary to process the image to extract the blackboard area.
a) According to the formula
Figure BDA0003464443260000041
And carrying out gray-scale image processing on the image. Wherein, I _ R (x, y), I _ G (x, y), I _ B (x, y) represent the values of the image RGB channels, and the gray value I (x, y) is obtained by adopting an averaging method.
b) By the formula
Figure BDA0003464443260000051
Calculating a gray level histogram wherein gammakIs the gray level of the pixel, nkIs of gray scale gammakMN is the total number of pixels of the image, wherein M, N is the length and width of the image.
c) Comprehensively considering that, for the blackboard-writing image information, the pixel area occupied by the blackboard area is the largest and has a significant difference from the surrounding environment, and the image is converted into a binary image by means of the gray histogram calculated in the previous step, as shown in fig. 2.
d) Inputting a binary image, and performing operations of removing a cavity and removing a non-connected region on the binary image by means of opecv to obtain an image P, as shown in fig. 3.
e) By detecting the outline of the blackboard in the image P, the coordinates of the four corners of the blackboard are determined.
f) Obtaining coordinate points of four corners corresponding to the blackboard by the last step based on a formula
Figure BDA0003464443260000052
And performing projection transformation, wherein the projection transformation is to project the picture to a new view plane, wherein (x, y) is the pixel coordinate of the original image, and the transformed image pixel coordinate (x ', y') is correspondingly obtained. Wherein the transformation matrix
Figure BDA0003464443260000053
Can be split into four parts, and the split part can be split into four parts,
Figure BDA0003464443260000054
represents a linear transformation, [ m ]6 m7]For translation, [ m ]2 m5]TPerspective transformation is generated to obtain a blackboard image, and the graph transformation result is shown in fig. 4.
g) And scaling the blackboard image according to a scale to prepare for later work. The image is scaled to around 700 x 450.
According to the existing Chinese character detection and Chinese character recognition algorithm, the text content in the blackboard-writing is recognized, namely, information such as each paragraph, each line, the position of each Chinese character and the like in the blackboard-writing is recognized. And then storing and sorting the recognized information, and making clear the position proportion of each section in the blackboard, the inclination angle of the line, the interval relation between the lines, the size proportion between characters, the interval relation between each character and the like to form data information capable of describing the overall structural layout of the blackboard-writing content. And the Chinese characters are divided according to the position information of each Chinese character, and each single Chinese character is cut out to prepare for aesthetic feeling guidance of the single Chinese character.
And recognizing the character information in the blackboard-writing image by using algorithms such as Baidu intelligent cloud high-precision handwritten Chinese character recognition and the like. In the embodiment, a Baidu intelligent cloud high-precision handwritten Chinese character recognition algorithm is used, so that the paragraphs, lines and characters in the text can be recognized accurately, and the horizontal position coordinates and the vertical position coordinates of the paragraphs, lines and characters, the width and the height of the circumscribed rectangle and other information can be fed back and output.
In another embodiment, the step S300 further includes the steps of:
s301: for the blackboard writing overall layout data set, the element information comprises the coordinate information of text paragraphs, lines and characters in the blackboard, the external matrix information and the overall pixels of the blackboard;
s302: for a single Chinese character aesthetic degree data set, cutting Chinese characters into pictures according to the circumscribed rectangle of each Chinese character, filling the images into N x N based on an image filling algorithm under the condition of ensuring that the character information in the images is not changed, wherein N is the resolution of the images, the value range of N is [64,224], and the image marking adopts an expert manual marking mode.
For the embodiment, the guidance of the blackboard writing aesthetic feeling needs to comprehensively consider two aspects of the whole layout of the blackboard writing and the single Chinese character aesthetic feeling, so that two data sets need to be constructed, namely the data set T of the whole layout of the blackboard writinglayoutHarmony Chinese character aesthetic feeling data set Tpic-aesthetic。
For blackboard writing overall layout data set TlayoutThe element information of which is composed of the text in the blackboardParagraph, line, word coordinate information, circumscribed matrix information, and overall pixel of the blackboard. The labels are subjected to score evaluation by experts according to self experience, and dimension information of the labels is described from three levels of paragraphs, lines and words.
a) The position of the text passage in the blackboard describes information as (X)d，yd，Wd，hd) The blackboard pixel is (X)p，Yp)。(Xd，yd) Denotes the coordinate of the top left corner of the rectangle circumscribing the paragraph, WdIndicates the width, h, of the circumscribed rectangledIs the height of the circumscribed rectangle. From Xd/Xp、yd/Yp、(Wd*hd)/(Xp*Yp) And determining three data elements of a vertical coordinate proportion, a horizontal coordinate proportion and an external rectangular pixel proportion of the text paragraph.
b) The spacing relationship between rows and the inclination of rows are important considerations in the overall layout.
The row-to-row spacing relationship is calculated as follows: (y)h+1-yh-dh)/Yp. Wherein y ishY-axis coordinate point, d, representing the top left corner of the rectangle circumscribed by the current rowhIndicating the height, y, of the bounding rectangle of the current lineh+1And a y-axis coordinate point representing the upper left corner of the next line of circumscribed rectangles, and Yp is the width of the picture.
The line tilt is calculated as follows: arctan ((x1, y1), (x2, y2)) where (x1, y1) represents the coordinates of the top left corner of the rectangle circumscribed by the first word of the current line and (x2, y2) represents the coordinates of the top left corner of the rectangle circumscribed by the last word of the current line, while converting the calculated angle to radians.
For the interval relation between rows and the inclination degree of the rows, in order to ensure the accuracy and effectiveness of data, the maximum value, the minimum value, the average value and the standard deviation of the row spacing and the maximum value, the minimum value, the average value and the standard deviation of the inclination angle of the rows are used as characteristic dimensions. Meanwhile, in order to ensure the validity of the data, all feature dimension data are normalized, and the formula is as follows:
Figure BDA0003464443260000081
wherein xiIndicates the current data value, minxRepresents the minimum value of the data in the current dimension, maxxRepresents the maximum value of the data in the current dimension, and b and a represent the corresponding maximum and minimum values after the data is compressed.
c) The size of a single word and the spacing relationship between words are also important considerations in the overall layout problem.
The size of a single word is calculated as follows:
Figure BDA0003464443260000082
wherein w, h are the width and height of the single character circumscribed rectangle, XP，YpThe width and height of the blackboard image.
The word-to-word spacing relationship is calculated as follows: (X)h+1-Xh-W)/Xp. Wherein XhThe coordinate of the horizontal axis representing the upper left corner of the circumscribed rectangle of the current character, w is the width of the circumscribed rectangle of the current character, Xh+1Representing the coordinate of the upper left-hand abscissa, X, of the left-hand character adjacent to the current characterpThe width of the blackboard image.
For the size of a single character and the interval relation between characters, in order to ensure the accuracy and effectiveness of data, the maximum value, the minimum value, the average value, the standard deviation of the characters and the maximum value, the minimum value, the average value and the standard deviation of the character interval are used as characteristic dimensions. And simultaneously carrying out normalization processing on all feature dimension data.
In this embodiment, TlayoutThe data set comprises 21 characteristic dimensions, the labels are subjected to score calibration by experts, the score is determined by a percentile system, and 6803 data which are originally collected are subjected to data screening and elimination, TlayoutAltogether containing 6377 available data.
For single Chinese character aesthetic degree data set Tpic-aestheticThe method comprises the steps of cutting Chinese characters into pictures according to the circumscribed rectangles of the Chinese characters, filling the pictures into 64x3 on the basis of an image filling algorithm under the condition that the character information in the pictures is not changed, and marking the pictures in a mode of manual marking by experts. ByAnd multiple experts guide the aesthetic feeling according to own experiences, and the guide grades are divided into three grades, namely excellent grade, good grade, poor grade and the like.
In another embodiment, the blackboard writing overall layout guidance model in step S400 includes an input layer, a hidden layer, and an output layer, where the hidden layer is a three-layer fully-connected layer, and the output layer activation function is a sigmoid activation function.
For this embodiment, the data set used for the model is TlayoutAnd constructing a neural network model, wherein the model comprises an input layer, a hidden layer and an output layer, and the hidden layer is a three-layer full-connection layer, namely a Linear layer. The activation function used is a ReLU activation function
Figure BDA0003464443260000091
The output layer activation function is a sigmoid activation function
Figure BDA0003464443260000092
In another embodiment, the loss function used by the blackboard writing overall layout guidance model in step S400 is a root mean square error, and a gradient descent algorithm is used for training to reduce the loss until convergence, so as to obtain the trained blackboard writing overall layout guidance model.
For this embodiment, the loss function used by the model is the mean square error
Figure BDA0003464443260000093
n is the number of training times, and in this embodiment, the value of n is 50. Wherein y isiIn order to be the original label, the label is,
Figure BDA0003464443260000094
is a predictive tag. And training by using a gradient descent algorithm, reducing loss until convergence, and further obtaining the required blackboard writing overall layout guidance model.
In another embodiment, the single Chinese character aesthetic feeling guidance model in step S500 includes a 1-VGG model for extracting information of image elements and a 2-CapsNet model for guiding the position relationship of the elements.
For this embodiment, the data set used for model training is Tpic-aesthetic. The core elements of a Chinese character include strokes and structural relationships between the strokes. The conventional convolutional neural network for processing the image can only identify elements in the image, the capability of processing the relationship between the elements is lacked, and the capsule network takes the vector as a basic unit and considers the position relationship between the elements. Therefore, the method integrates the advantages of the VGG and the CapsNet, and provides a network model of the VGG-CapsNet: the 1-VGG for extracting image element information and the 2-CapsNet model for guiding the position relation of elements are included.
The method for calculating the aesthetic feeling of the single Chinese character comprises the following steps:
step 1: for the 1-VGG model, the image input size is 64x64x3, after feature extraction, the output image size is 16x16x512, and the output result is input into the conv5 convolutional layer of the value 2-CapsNet for convolution operation.
Step 2: and inputting the operation result of the conv5 into a main capsule layer of the 2-CapsNet, and reshaping the result into 16 8D capsules to obtain an output vector v of the capsule.
Step 3: by means of square compression functions
Figure BDA0003464443260000101
And compressing the length of the output vector v in the previous step to 0-1. In the square compression function, V is the input vector and V is the output vector.
Step 4: the main capsule layer is connected with the digital capsule layer, and a dynamic routing algorithm is operated on the digital capsule layer. And the high-layer capsule of the digital capsule layer receives the output vector v of the low-layer capsule of the main capsule layer, and multiplies the output vector v by a corresponding weight matrix to obtain a prediction vector for predicting the position where the high-layer characteristic should exist.
Step 5: calculating the coupling coefficient of the low-layer capsule and the high-layer capsule
Figure BDA0003464443260000111
Which represents the weight that the lower capsule output vector v can pass to the higher capsules, by an amount equal to the number of higher capsules. Wherein, cijTo moveCoupling coefficient determined by a state routing iterative process, bijLog prior probabilities of coupling of the bottom layer capsules to the higher layer capsules are identified. After the first iteration, all CijAre all equal, then CijUpdated by dynamic routing procedures, bijIs 0.
Step 6: the predicted vectors v are summed in a weighted manner to obtain a total input vector s.
Step 7: and compressing the vector S by using a square compression function to ensure that the length of the output vector is 0-1 and the direction of the output vector is not changed.
Step 8: and calculating the dot product of the output vector of the capsule and the prediction vector, and adding the old weight to obtain a new weight, wherein if the dot product value is a positive number, the predicted output of the capsule is consistent with the actual output of the capsule, so that the weight coefficient is increased, otherwise, the weight coefficient is decreased, and the routing iteration operation is finished.
Step 9: and (4) obtaining a final output vector through 3 times of routing iteration, and calculating the length of the output vector, namely the probability of the aesthetic feeling grade of the current Chinese character.
The loss function used for the model is: l isk＝Tkmax(0，m+-||vk||)2+λ(1-Tk)max(0，||vk||-m-)2Wherein k is the Chinese character aesthetic category, TkIndicating whether the predicted category exists, when the predicted category exists, TkIs 1, when the prediction class does not exist, TkIs 0. m + is a criterion for false positives, indicating that when the detected object is present and the actual output value of the network is less than m +, the total loss is increased. And when the prediction result is larger than m +, the total loss is unchanged. m-represents a criterion for false negatives, and represents that when the detected object does not exist and the actual output value of the network is greater than m-, the total loss is increased. And when the prediction result category existence probability is less than m-, the total loss is unchanged. For the case where k is not present, the λ weight causes the initial learning process to no longer shrink the length of all capsules. Specifically, when k is present, the second half of the formula is 0, and the longer the length of the capsule, the smaller the loss function; when k is absent, the first half of the formula is0 and the shorter the length of the capsule, the smaller the loss function.
In another embodiment, the 1-VGG model comprises 4 two-dimensional convolution layers, each layer being connected in sequence as follows: convolutional layer Conv1, ReLU layer Conv1-ReLU, pooled layer Conv2, ReLU layer Conv2-ReLU, pooled layer MaxPool2d, convolutional layer Conv3, ReLU layer Conv3-ReLU, pooled layer MaxPool2d, convolutional layer Conv 4; wherein, the convolution kernel used by the 1-VGG model is 3 x 3.
For the embodiment, the convolution kernel used by the 1-VGG model is 3 x3, and the depth of the network can be improved by using small convolution under the condition of ensuring the same perception field, so that the effect of the neural network is improved to a certain extent, and parameters are reduced.
In another embodiment, the 2-CapsNet model comprises a convolution layer, a main capsule layer, a digital capsule layer and three full-connection layers, wherein the three full-connection layers are sequentially connected as follows: convolutional layer conv5, main capsule layer PrimaryCaps, digital capsule layer DigCaps, three full-connection layer Liner.
In another embodiment, the present disclosure discloses a system for guiding Chinese character writing on a handwriting blackboard based on deep learning, comprising a client and a server, wherein,
the client comprises a user information registration and login unit, a user writing information acquisition unit and a guidance result presentation unit;
the server comprises a preliminary data preprocessing unit, a subsequent data preprocessing unit, a blackboard writing overall layout guiding unit and a single Chinese character aesthetic feeling guiding unit.
For the embodiment, the primary preprocessing of the data is to correct the blackboard-writing image transmitted from the client, remove external interference factors and only leave blackboard information; subsequent preprocessing of data, detecting character information in blackboard information, completing position detection and proportion calculation from three angles of segment, line and character, and segmenting and storing each character as an image by means of an image segmentation technology; the overall layout guidance of the blackboard writing is to grade the layout of the blackboard writing by means of a deep neural network; the single Chinese character aesthetic feeling guidance is to grade the aesthetic feeling of a single Chinese character by means of a VGG-CapsNet neural network.
The basic business process is as follows: firstly, detecting whether a user is registered, if the user is not registered, the user is required to be registered, if the user is registered, the user logs in, then the user selects a blackboard-writing image to be guided, after the user clicks to start guiding, the blackboard-writing image can be automatically uploaded to a server side, the server is responsible for implementing a guiding function, after the guiding function is completed, a result is returned to a client side, the user can check the overall layout score of the blackboard-writing image which is uploaded and the aesthetic feeling level of Chinese characters, then the blackboard-writing guiding information can be stored in a data set, and meanwhile, the user can select to request the server again to guide or stop guiding in the process of guiding.
In another embodiment, a computer device comprises a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method for guiding a handwritten blackboard Chinese writing based on deep learning when executing the computer program.
In another embodiment, an apparatus is provided that includes a means for blackboard writing guidance information entry and retrieval, a memory, a processor, and a system program stored on the memory for processing by the processor.
To blackboard writing guide information input and acquisition device, include: the system comprises an image acquisition module, an information input module, a control terminal module and a supporting and moving module, and is shown in figure 5.
The image acquisition module comprises a camera and a connecting rod, the camera is used for acquiring information input by the information input module, and the connecting rod plays a role in supporting and connecting. For the camera, the maximum visual angle a of the camera lens is larger than the included angle b formed by the camera and the edge of the information input module, as shown in fig. 6.
The information input module consists of a blackboard and a fixed support, the blackboard is used for inputting blackboard writing test information by a tester, and the fixed support plays a role in fixedly supporting the blackboard.
The control terminal module is used for operations such as login of a user account, acquisition of writing information and the like, and is internally provided with a memory and a processor for running a client program, a network connection device for connecting a network and other hardware devices.
The support moving module comprises four wheels and fixed buckles at the bottom of the fixed support and the support, a user can freely move the blackboard-writing guide information input and acquisition device when the blackboard-writing guide information input and acquisition device needs to move, and the wheels are firmly fixed on the ground through the fixed buckles when the blackboard-writing guide information input and acquisition device moves to the corresponding positions, so that the blackboard is not prone to shaking during writing.
A memory for storing a system program; the program is stored in the memory and may be called for processing by the processor.
A processor for executing the system program stored in the memory. The memory and the processor are connected by bus coupling.
Although the embodiments of the present invention have been described above with reference to the accompanying drawings, the present invention is not limited to the above-described embodiments and application fields, and the above-described embodiments are illustrative, instructive, and not restrictive. Those skilled in the art, having the benefit of this disclosure, may effect numerous modifications thereto without departing from the scope of the invention as defined by the appended claims.

Claims (10)

1. A method for guiding Chinese character writing on a handwriting blackboard based on deep learning comprises the following steps:
s100: collecting images of blackboard Chinese character writing for guidance as a training image set;
s200: preprocessing the training image set;
s300: constructing a blackboard writing overall layout data set and a single Chinese character aesthetic degree data set;
s400: constructing and training a blackboard writing overall layout guidance model by using the blackboard writing overall layout data set;
s500: constructing and training a single Chinese character aesthetic feeling guidance model by using the single Chinese character aesthetic feeling data set;
s600: and guiding the handwriting blackboard writing with the trained blackboard writing overall layout guidance model and the single Chinese character aesthetic feeling guidance model.
2. The method according to claim 1, preferably, the step S200 further comprises the steps of:
s201: extracting a blackboard area in the training image set;
s202: and recognizing the text content in the blackboard-writing, and storing and sorting the recognized information.
3. The method of claim 1, said step S300 further comprising the steps of:
s301: for the blackboard writing overall layout data set, the element information comprises the coordinate information of text paragraphs, lines and characters in the blackboard, the external matrix information and the overall pixels of the blackboard;
s302: for a single Chinese character aesthetic degree data set, cutting Chinese characters into pictures according to the circumscribed rectangle of each Chinese character, filling the images into N x N based on an image filling algorithm under the condition of ensuring that the character information in the images is not changed, wherein N is the resolution of the images, the value range of N is [64,224], and the image marking adopts an expert manual marking mode.
4. The method according to claim 1, wherein the blackboard writing overall layout guidance model in step S400 comprises an input layer, a hidden layer and an output layer, wherein the hidden layer is a fully connected three-layer, and the output layer activation function is a sigmoid activation function.
5. The method according to claim 1, wherein the loss function used by the blackboard writing overall layout guidance model in step S400 is root mean square error, and the gradient descent algorithm is used for training to reduce the loss until convergence, thereby obtaining the trained blackboard writing overall layout guidance model.
6. The method of claim 1, wherein the single chinese character aesthetic perception guidance model in step S500 includes a 1-VGG model for extracting information of image elements and a 2-CapsNet model for guiding a position relationship of the elements.
7. The method of claim 6, wherein the 1-VGG model comprises 4 two-dimensional convolutional layers, each layer being connected in sequence as: convolutional layer Conv1, ReLU layer Conv1-ReLU, pooled layer Conv2, ReLU layer Conv2-ReLU, pooled layer MaxPool2d, convolutional layer Conv3, ReLU layer Conv3-ReLU, pooled layer MaxPool2d, convolutional layer Conv 4; wherein, the convolution kernel used by the 1-VGG model is 3 x 3.
8. The method of claim 6, wherein the 2-CapsNet model comprises a convolutional layer, a main capsule layer, a digital capsule layer, and three fully-connected layers, wherein the layers are sequentially connected as follows: convolutional layer conv5, main capsule layer PrimaryCaps, digital capsule layer DigCaps, three full-connection layer Liner.
9. A system for guiding Chinese character writing on a handwriting blackboard based on deep learning comprises a client and a server, wherein,
the client comprises a user information registration and login unit, a user writing information acquisition unit and a guidance result presentation unit;
the server comprises a preliminary data preprocessing unit, a subsequent data preprocessing unit, a blackboard writing overall layout guiding unit and a single Chinese character aesthetic feeling guiding unit.
10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method for guiding a handwritten blackboard chinese character writing based on deep learning according to any one of claims 1 to 8 when executing the computer program.
CN202210024764.9A 2022-01-11 2022-01-11 Method, system and equipment for guiding handwriting Chinese character blackboard writing Pending CN114550179A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210024764.9A CN114550179A (en) 2022-01-11 2022-01-11 Method, system and equipment for guiding handwriting Chinese character blackboard writing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210024764.9A CN114550179A (en) 2022-01-11 2022-01-11 Method, system and equipment for guiding handwriting Chinese character blackboard writing

Publications (1)

Publication Number Publication Date
CN114550179A true CN114550179A (en) 2022-05-27

Family

ID=81669740

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210024764.9A Pending CN114550179A (en) 2022-01-11 2022-01-11 Method, system and equipment for guiding handwriting Chinese character blackboard writing

Country Status (1)

Country Link
CN (1) CN114550179A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114936787A (en) * 2022-06-08 2022-08-23 武汉行已学教育咨询有限公司 Online student teaching intelligent analysis management cloud platform based on artificial intelligence

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114936787A (en) * 2022-06-08 2022-08-23 武汉行已学教育咨询有限公司 Online student teaching intelligent analysis management cloud platform based on artificial intelligence

Similar Documents

Publication Publication Date Title
WO2022148192A1 (en) Image processing method, image processing apparatus, and non-transitory storage medium
CN108416377B (en) Information extraction method and device in histogram
CN111275082A (en) Indoor object target detection method based on improved end-to-end neural network
CN111444821A (en) Automatic identification method for urban road signs
CN110807422A (en) Natural scene text detection method based on deep learning
CN111553200A (en) Image detection and identification method and device
CN109740515B (en) Evaluation method and device
CN111626297A (en) Character writing quality evaluation method and device, electronic equipment and recording medium
CN111553423A (en) Handwriting recognition method based on deep convolutional neural network image processing technology
CN112232371A (en) American license plate recognition method based on YOLOv3 and text recognition
CN112001394A (en) Dictation interaction method, system and device based on AI vision
CN113673338A (en) Natural scene text image character pixel weak supervision automatic labeling method, system and medium
CN111507353B (en) Chinese field detection method and system based on character recognition
CN111640116B (en) Aerial photography graph building segmentation method and device based on deep convolutional residual error network
CN114550179A (en) Method, system and equipment for guiding handwriting Chinese character blackboard writing
CN114898460A (en) Teacher non-verbal behavior detection method based on graph convolution neural network
CN111738264A (en) Intelligent acquisition method for data of display panel of machine room equipment
CN106909936B (en) Vehicle detection method based on double-vehicle deformable component model
CN115170471A (en) Part identification method and device based on image identification model
CN114494678A (en) Character recognition method and electronic equipment
CN111881732B (en) SVM (support vector machine) -based face quality evaluation method
CN114330247A (en) Automatic insurance clause analysis method based on image recognition
CN114550176A (en) Examination paper correcting method based on deep learning
CN111738088B (en) Pedestrian distance prediction method based on monocular camera
CN117437647B (en) Oracle character detection method based on deep learning and computer vision

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination