KR102344145B1 - Early childhood learning method using by handwriting recognition, and recording medium for recording the early childhood learning method - Google Patents

Abstract

Disclosed are a learning method for children using handwriting recognition, which provides descriptive questions on a screen of computer terminals or portable terminals used by a child and uses input handwriting image data to automatically perform scoring and provide a score by using handwriting recognition and natural language processing technology when the child directly writes an answer to the question, so that the child can perform learning anytime, and a recording medium storing the same. According to the present invention, the learning method for children comprises the following steps: (a) receiving a user's handwriting through a touch input; (b) processing an input of the user's handwriting through an optical character reader (OCR); (c) post-processing an OCR processing result to recognize the user's handwriting; (d) pre-processing the recognized user's handwriting; (e) automatically scoring the preprocessed handwriting recognition result through natural language processing; and (f) outputting a scoring result by post-processing the automatic scoring result.

Description

Early childhood learning method using by handwriting recognition, and recording medium for recording the early childhood learning method

The present invention relates to a method for learning children through handwriting recognition and a recording medium recording the same, and more particularly, to provide a narrative problem on the screen of a computer terminal or portable terminal used by an infant user, and to provide an answer to the problem by an infant user Infant learning method through handwriting recognition and recording medium that allows infant users to learn at any time 24 hours a day by providing automatic scoring using handwriting recognition and natural language processing technology using the input cursive image data when inputting by handwriting is about

The legacy scoring system previously provided by the applicant in the field of early childhood learning is a system in which the teacher directly checks the correct answers submitted by children, and determines and scores incorrect answers.

Due to the aftermath of COVID-19, the number of users (number of students) of the 'Milk T' learning service provided by the applicant has increased. For this reason, the automation of the legacy scoring system was required, and since a limited number of administrative teachers manage several students, the time (scoring) required to determine incorrect answers is not performed in real time.

Labor costs are high in the part where people directly judge, and it is necessary to improve the speed of the grading service and the unity of the grading result.

The need for handwriting recognition and automatic scoring services for complete automation (digitization) of simple and narrative types with fixed answers has emerged.

As a related prior patent document, there is Republic of Korea Patent Publication No. 10-2013919 (registration date: August 19, 2019), which includes a method for learning and managing handwriting with immediate feedback using multisensory stimuli and handwriting learning for the same A computer-readable recording medium in which a management program is recorded is described.

An object of the present invention is to provide a narrative problem on the screen of a computer terminal or portable terminal used by an infant user, and when the infant user directly writes and inputs the answer to the problem, handwriting recognition and natural language using the input cursive image data It is to provide an infant learning method through handwriting recognition that allows infant users to learn at any time 24 hours a day by providing automatic scoring with processing technology.

In accordance with an embodiment of the present invention for achieving the above object, there is provided a learning method for infants, comprising the steps of: (a) receiving a user's handwriting through a touch input; (b) OCR processing the input of the user's handwriting; (c) post-processing the OCR processing result to recognize the user's handwriting; (d) pre-processing the recognized user's handwriting; (e) automatically scoring the pre-processed handwriting recognition result through natural language processing; and (f) post-processing the automatic scoring to output a scoring result.

In step (c), the handwriting recognition unit may refine data through a binarization process, a labeling process, and a manual labeling merging process with respect to the handwriting image according to the handwriting input from the user.

In step (c), with respect to the binarization process, the handwriting recognition unit inputs the handwriting image according to the handwriting as a single-channel image of gray scale, and changes it to a maximum value when it exceeds the threshold value, and to 0 when it exceeds the threshold value can be converted

In step (c), the handwriting recognition unit scans an image from left to right and top to bottom according to a Connected Component Labeling technique with respect to the labeling process, and for pixels whose pixel values are not 0 Check the top and the left, if both are 0, assign a new label (number) to the current pixel, if only one of the two is non-zero, assign that label; if both are non-zero, assign one of the two labels, The two labels are set as equivalence, and the process of checking the top and the left is repeated for every pixel to reset the label for the equivalence label.

In the step (e), the natural language processing expresses the recognized word for the user's handwriting as an embedding vector in the form of a dense vector and word embedding, and the embedding vector is By using it, similarity can be derived with respect to two other words and sentences.

In step (e), the automatic scoring compares the handwriting recognition result with the content model answer, derives the scoring result by rule-based and edit distance calculation in the case of short answer type, and input answer characteristic and model answer characteristic in the case of narrative type It is possible to derive the scoring result by calculating the similarity by executing feature extraction for .

In the case of the narrative type, the feature extraction may be performed through spacing correction, morphological analysis of nouns and verbs, chunking of noun phrases and verb phrases, and negative expression recognition using indefinite tags.

According to the present invention, it is possible to more accurately recognize children's cursive handwriting by collecting, refining, and additionally learning the OCR model of the infant's cursive image data.

In addition, the present invention can provide correction for typos.

In addition, in the case of the short answer type, it is determined whether or not the correct answer is the correct answer by perfect matching, and the keyword-oriented correct answer can be processed after morpheme analysis.

In addition, in the case of the narrative type, different from the short answer type, multiple data clusters for the extended model answer are generated through typo correction, and the similarity close to the correct answer is generated and recognized by comparing the similarity between the confirmed model answer and the actual answer. have.

As a result, the present invention provides descriptive scoring in real time, so that the learner can check which keyword is the most important in the subject he or she is learning. By providing the elements necessary for the correct answer and elements that were not needed for the correct answer after grading, it is possible to provide the part that the learner missed in the subject, thereby causing the learner's understanding. And, the learner can be provided with important keywords needed for the actual answer.

1 is a configuration diagram schematically showing the overall configuration of an infant learning system through handwriting recognition according to an embodiment of the present invention.
2 is a configuration diagram schematically showing the internal configuration of an early childhood learning system according to an embodiment of the present invention.
3 is a diagram illustrating a process of refining infant handwriting image data by the handwriting recognition unit in the infant learning server according to an embodiment of the present invention.
4 is a diagram illustrating an example of labeling connection elements of a handwriting recognition unit according to an embodiment of the present invention.
5 is a diagram illustrating an automatic scoring process of an automatic scoring unit according to an embodiment of the present invention.
6 is a diagram illustrating short-answer-type processing and narrative-type processing by the automatic scoring unit according to an embodiment of the present invention.
7 is a flowchart illustrating a method for early childhood learning according to an embodiment of the present invention.
8 is a view showing sample data for children's cursive answers according to an embodiment of the present invention.
9A is a diagram illustrating a good result of OCR processing for a user's handwriting according to an embodiment of the present invention.
9B is a diagram illustrating a bad result of OCR processing for a user's handwriting according to an embodiment of the present invention.
10 is a diagram exemplifying the form of infant cursive refinement data of the handwriting recognition unit according to an embodiment of the present invention.
11 is a diagram illustrating a labeling process in the cursive image data refinement process of the handwriting recognition unit according to an embodiment of the present invention.
12 is a diagram illustrating a form of infant cursive learning data stored in an early childhood learning DB according to an embodiment of the present invention.
13 is a diagram illustrating a kernel of a Deep CNN and a feature map of image data passing through the kernel according to an embodiment of the present invention.
14 is a diagram illustrating a learned image feature of a Deep CNN according to an embodiment of the present invention.
15 is a diagram illustrating an example of comparing an answer input by a learner of the automatic scoring unit with an extended model answer according to an embodiment of the present invention.

Advantages and features of the present invention, and methods for achieving them, will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in a variety of different forms, only this embodiment allows the disclosure of the present invention to be complete, and common knowledge in the technical field to which the present invention belongs It is provided to fully inform the possessor of the scope of the invention, and the present invention is only defined by the scope of the claims. Accordingly, in some embodiments, well-known process steps, well-known device structures, and well-known techniques have not been specifically described in order to avoid obscuring the present invention. Like reference numerals refer to like elements throughout.

In order to clearly express various layers and regions in the drawings, the thicknesses are enlarged. Throughout the specification, like reference numerals are assigned to similar parts. When a part, such as a layer, film, region, plate, etc., is “on” another part, it includes not only cases where it is “directly on” another part, but also cases where there is another part in between. Conversely, when we say that a part is "just above" another part, we mean that there is no other part in the middle. Also, when a part of a layer, film, region, plate, etc. is said to be "under" another part, it includes not only the case where it is "directly under" another part, but also the case where there is another part in the middle. Conversely, when a part is said to be "just below" another part, it means that there is no other part in the middle.

Spatially relative terms "below", "beneath", "lower", "above", "upper", etc. It can be used to easily describe the correlation between an element or components and other elements or components. The spatially relative terms should be understood as terms including different orientations of the device during use or operation in addition to the orientation shown in the drawings. For example, when an element shown in the figures is turned over, an element described as "beneath" or "beneath" another element may be placed "above" the other element. Accordingly, the exemplary term “below” may include both directions below and above. The device may also be oriented in other orientations, and thus spatially relative terms may be interpreted according to orientation.

In the present specification, when a part is said to be connected to another part, this includes not only a case in which it is directly connected, but also a case in which it is electrically connected with another element interposed therebetween. In addition, when it is said that a part includes a certain component, this means that other components may be further included, rather than excluding other components, unless otherwise stated.

In this specification, terms such as first, second, third, etc. may be used to describe various components, but these components are not limited by the terms. The above terms are used for the purpose of distinguishing one component from other components. For example, without departing from the scope of the present invention, a first component may be referred to as a second or third component, and similarly, the second or third component may also be alternately named.

Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used with the meaning commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless clearly defined in particular.

Hereinafter, a system and method for early childhood learning through handwriting recognition according to a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a configuration diagram schematically showing the overall configuration of an infant learning system through handwriting recognition according to an embodiment of the present invention.

Referring to FIG. 1 , an early childhood learning system 100 according to an embodiment of the present invention includes a student terminal 110 , an early childhood learning server 120 , an early childhood learning database (DB) 130 , and a teacher terminal 140 . may include

The student terminal 110 and the teacher terminal 140 may be one of a smart phone or a tablet PC carried by a student or teacher, or a computer terminal connected by wire.

The student terminal 110 receives learning problems, such as multiple choice, short answer type, and narrative type, from the early childhood learning server 120 and displays it on the screen, and inputs the answer to the problem by the user's handwriting with a touch input from the student user.

The early childhood learning server 120 automatically scores by receiving the user's handwriting as an answer to the problem from the student terminal 110 as a touch input, and provides the scoring result to the student terminal 110 .

The early childhood learning DB 130 stores image data related to handwriting input from the student terminal 110, or stores handwriting input data according to short answer type and narrative type as image data.

In addition, the early childhood learning DB 130 stores the data on the learning dataset, the content model answer, including the infant cursive data.

The teacher terminal 140 transmits the teacher's manual scoring for the infant cursive data, handwritten answer data, etc. input from the student terminal 110 as image data to the early childhood learning server 130 .

2 is a configuration diagram schematically showing the internal configuration of an early childhood learning system according to an embodiment of the present invention.

Referring to FIG. 2 , the early childhood learning server 120 according to an embodiment of the present invention largely includes a handwriting recognition unit 210 and an automatic scoring unit 220 .

The handwriting recognition unit 210 receives and pre-processes the user's handwriting through a touch input, performs optical character reader (OCR) processing, and recognizes the user's handwriting through a post-processing process. In addition, the handwriting recognition unit 210 extracts unique features through learning of a kernel parameter from the input image data of the user's handwriting, and recognizes the characters through a deep convolutional neural network (CNN). can recognize

The automatic scoring unit 220 may pre-process the recognized user's handwriting, automatically score it through natural language processing, and output a scoring result through a post-processing process.

The handwriting recognition unit 210 may include a preprocessor 212 , an OCR engine unit 214 , and a postprocessor 216 .

The pre-processing unit 212 pre-processes the input user's handwriting.

The OCR engine unit 214 processes the pre-processed user's handwriting in an OCR method.

The post-processing unit 216 post-processes the OCR-processed user's handwriting.

The automatic scoring unit may include a pre-processing unit 222 , a natural language processing (NLP) scoring engine unit 224 , and a post-processing unit 226 .

The preprocessor 222 preprocesses the handwriting recognition data post-processed by the handwriting recognition unit 210 .

The NLP scoring engine unit 224 scores the pre-processed handwriting recognition data through a natural language processing (NLP) process.

The post-processing unit 226 post-processes a scoring result of the handwriting recognition data.

3 is a diagram illustrating a process of refining infant handwriting image data by the handwriting recognition unit in the infant learning server according to an embodiment of the present invention.

Referring to FIG. 3 , the handwriting recognition unit 210 according to an embodiment of the present invention performs a binarization process, a labeling process, and a manual labeling merging process for a handwritten image according to a handwriting input from a user. data can be refined.

In the binarization process, a handwriting image according to handwriting is input as a single-channel image of gray scale, and when it exceeds a threshold value, it is changed to a maximum value, and when it is less than a threshold value, it can be converted to 0.

The labeling process may be performed as follows according to the Connected Component Labeling technique shown in FIG. 4 . 4 is a diagram illustrating an example of labeling connection elements of a handwriting recognition unit according to an embodiment of the present invention.

1. Scan the image from left to right and top to bottom.

2. Check top and left for non-zero pixel values.

1) If all are 0, a new label (number) is assigned to the current pixel.

2) If only one of the two is not 0, the label is assigned.

3) If both are non-zero, one of the two labels is assigned, and these two labels are set as equivalence.

3. Repeat step 2 process to check top and left for all pixels.

4. Reset labels for equivalence labels.

In the labeling manual merging process, a labeling input may be received and merged by a user who directly judges the handwriting image by looking at the handwriting image.

On the other hand, natural language processing or natural language processing is one of the main fields of artificial intelligence that studies and implements human language phenomena to be described using machines such as computers. Natural language processing is naturally closely related to linguistics, which studies language itself, and language cognitive science, which explores the internal mechanisms of language phenomena, since the subject of study is language. It is a representative field that uses a lot of mathematical and statistical tools for implementation, especially machine learning tools. Various applications such as information retrieval, QA system, automatic document classification, newspaper article clustering, and interactive agent are being made. Although research on natural language has been going on for a long time, even in 2018, computers still cannot understand natural language like humans. Instead, the technology for processing a large amount of information using superficial probability and statistics without a deep understanding of the language has been greatly developed.

In the natural language processing in the automatic scoring unit 220 according to the embodiment of the present invention, the recognized word for the user's handwriting is expressed as an embedding vector in the form of a dense vector to embed a word. embedding), and using the embedding vector, similarity can be derived for the other two words and sentences.

5 is a diagram illustrating an automatic scoring process of an automatic scoring unit according to an embodiment of the present invention.

Referring to FIG. 5 , the automatic scoring in the automatic scoring unit 220 according to an embodiment of the present invention is a technique of comparing a handwriting recognition result with a content model answer.

The automatic scoring unit 220 divides the entire learner answer into learning data sampling and test data, extracts characteristics after manual scoring for the learning data sampling, learns the extracted characteristics, and then inputs the extracted characteristics to the automatic scoring model to perform automatic scoring run

Also, the automatic scoring unit 220 may derive a scoring result after automatically scoring the test data.

The automatic scoring is divided into short answer type and narrative type.

As shown in Figure 6, in the case of the short answer type, the scoring result is derived by rule-based and editing distance calculation, and in the case of the narrative type, the characteristic extraction for the input answer characteristic and the model answer characteristic is executed, and the similarity is calculated to obtain the scoring result. can be derived 6 is a diagram illustrating short-answer-type processing and narrative-type processing by the automatic scoring unit according to an embodiment of the present invention.

In the case of the narrative type, feature extraction can be performed through a spacing correction process, a morpheme analysis process for nouns and verbs, a chunking process for noun phrases and verb phrases, and a negative expression recognition process using indefinite tags.

7 is a flowchart illustrating a method for early childhood learning according to an embodiment of the present invention.

Referring to FIG. 7 , the early childhood learning server 120 according to the embodiment of the present invention first provides a short answer type or narrative type problem to the student terminal 110, so that the answer is answered by a touch input from the student terminal 110 A user's handwriting is inputted (S710).

That is, as shown in FIG. 8 , the early childhood learning server 120 receives the user's handwriting for the answer by a touch input from the student terminal 110 . 8 is a view showing sample data for children's cursive answers according to an embodiment of the present invention.

Next, the early childhood learning server 120 performs OCR processing on the input of the user's handwriting (S720).

That is, the early childhood learning server 120 inputs the child's handwriting received from the student terminal 110 into the OCR cursive engine, and as shown in FIGS. 9A and 9B , OCR processing to recognize the meaning of the handwriting. let it be 9A is a diagram illustrating a good result of OCR processing for a user's handwriting according to an embodiment of the present invention, and FIG. 9B is a diagram illustrating a bad result of OCR processing for a user's handwriting according to an embodiment of the present invention to be.

Computer Vision (CV) refers to a field of artificial intelligence that aims to give computers the thinking ability to recognize images or images through human vision. In the field of CV, the task of recognizing handwriting, cursive, and printed text is called Optical Character Recognition (OCR). OCR has been commercialized in various industries such as credit card and resident registration card recognition, license plate recognition, and print reading. Existing OCR models have been studied a lot as a probabilistic statistical method. However, since these methods use preprocessing and probability-based models designed by humans, they cannot be replaced for specific situations, and in some cases, the structure of the model is complicated and the operation speed may be slow. Recently, OCR using Deep Convolutional Neural Network (CNN) has emerged due to technological developments in hardware and artificial intelligence, resulting in higher recognition rate and performance than conventional OCR.

Since infant cursive data is written by infants who lack comprehension skills, the differences from adult cursive are 1) the free size of the leading, middle, and final consonants, and 2) the angle of the data due to the spelling.

Next, the early childhood learning server 120 post-processes the OCR processing result to recognize the user's handwriting (S730).

At this time, as shown in FIG. 10 , the handwriting recognition unit 210 in the infant learning server 120 performs a binarization process, a labeling process, Data can be refined through the labeling manual merging process. 10 is a diagram exemplifying the form of infant cursive refinement data of the handwriting recognition unit according to an embodiment of the present invention.

In the binarization process, the handwriting recognition unit 210 may input a handwriting image according to a handwriting as a single-channel image of gray scale, change it to a maximum value if it exceeds a threshold value, and may convert it to 0 if it exceeds the threshold value.

In addition, the handwriting recognition unit 210 scans the image from left to right and top to bottom according to the Connected Component Labeling technique for the labeling process as shown in FIG. 11 , and the pixel value is 0 We can assign a new label (number) to the current pixel by checking the top and left for any non-zero pixel. 11 is a diagram illustrating a labeling process in the cursive image data refinement process of the handwriting recognition unit according to an embodiment of the present invention. The handwriting recognition unit 210 assigns one of the two labels to the label if it is not 0, assigns one of the two labels if both are not 0, sets the two labels as equivalence, and assigns the two labels to all pixels. You can reset the label for the equivalence label by repeating the check up and left.

Also, as shown in FIG. 12 , the handwriting recognition unit 210 may recognize the infant cursive data by deep learning the infant cursive data stored in the infant learning DB 130 . 12 is a diagram illustrating a form of infant cursive learning data stored in an early childhood learning DB according to an embodiment of the present invention.

Deep learning is like the proposed network by referring to the shape of neurons, which are nerve cells. In general, it consists of an input layer (input layer), a hidden layer (hidden layer), and an output layer (output layer). When the network model becomes complex, it can have multiple hidden layers. The information is transmitted from left to right through this combination in the form of the output of the input information being input to the next layer. In deep learning, learning proceeds by updating the weights in a direction that can minimize the error between the learning data and the learning result. Learning continues to the point where the loss value is minimized. By using the learned model, it can be used in various areas such as regression analysis, classification, and pattern recognition.

CNN (Convolution Neural Network; CNN) is a neural network with the motif of animals recognizing or classifying objects. When animals distinguish between different objects, the idea of distinguishing objects by sensitively accepting a specific part of the object has emerged. A neural network called CNN was announced by getting a hint from these brain activities, and it is currently being widely used in the image field and video field.

The main function of CNN is to extract features from images. Feature extraction refers to finding unique features in image data, and CNN basically extracts image features through learning kernel parameters. And when these kernels are visualized, it is shown in FIG. 13 . 13 is a diagram illustrating a kernel of a Deep CNN according to an embodiment of the present invention and an image data feature map that has passed through the kernel, and FIG. 14 is a diagram illustrating a learned image feature of a Deep CNN according to an embodiment of the present invention. to be.

In Fig. 14, the left side is the input image, the center is the learned main area (image features) of the deep neural network, and the right side shows the overlapped image of the input data and the hitmap.

To learn a robust model from various data in deep learning, the recognition rate is improved by additionally learning the target dataset. Accordingly, as shown in Table 1 below, the applicant is collecting a total of 2.6 million cases of handwriting image data for infants every year, and about 1.5 million cases of data are accumulated in Korean/English subjects except math.

Accordingly, by refining the handwriting data of infants owned by the applicant, a learning dataset was constructed and additionally learned to improve the recognition rate for infant cursive.

As a result of learning by constructing a special image dataset called infant cursive, the applicant achieved an average recognition rate of 88% in Korean and 95% in English for their handwritten image data.

Next, the early childhood learning server 120 pre-processes the user's handwriting recognized by the handwriting recognition unit 210 by inputting it into the automatic scoring unit 220 (S740).

Next, in the early childhood learning server 120 , the automatic scoring unit 220 automatically scores the preprocessed handwriting recognition result through natural language processing ( S750 ).

Natural language processing or natural language processing is one of the main fields of artificial intelligence that studies and implements human language phenomena to be described using machines such as computers. Natural language processing is naturally closely related to linguistics, which studies language itself, and language cognitive science, which explores the internal mechanisms of language phenomena, since the subject of study is language. It is a representative field that uses a lot of mathematical and statistical tools for implementation, especially machine learning tools. Various applications such as information retrieval, QA system, automatic document classification, newspaper article clustering, and interactive agent are being made. Although research on natural language has been going on for a long time, even in 2018, computers still cannot understand natural language like humans. Instead, the technology for processing a large amount of information using superficial probability and statistics without a deep understanding of language has been greatly developed.

Natural language processing according to an embodiment of the present invention expresses a word for a recognized user's handwriting as an embedding vector in the form of a dense vector, performs word embedding, and utilizes the embedding vector. Thus, similarity can be derived with respect to the other two words and sentences.

Automatic scoring compares the handwriting recognition result with the content model answer, derives the scoring result by rule-based and edit distance calculation in the case of a short answer type, and extracts the characteristics of the input answer characteristic and the model answer characteristic in the case of the narrative type. A scoring result can be derived by calculating the similarity.

In the case of the narrative type, feature extraction can be performed through spacing correction, morphological analysis of nouns and verbs, chunking of noun phrases and verb phrases, and negative expression recognition using indefinite tags.

Next, the early childhood learning server 120 post-processes the automatic scoring and outputs the scoring result (S760).

An important factor in automatic scoring is language processing and classification of scoring units in the automatic scoring module is required. For example, short answer type and narrative type.

The short answer type only needs to go through a simple morpheme analysis and a decision on whether it is a synonym by chunking.

The narrative type requires feature extraction and calculation of similarity.

The descriptive feature extraction goes through a morpheme analysis process, and if it is not registered in advance in this process (the morpheme analysis is not performed), entity name recognition and keyword extraction may be required.

By adding a user dictionary using the mecab morpheme analyzer and education-related dictionary data, education-specific morpheme analysis is possible compared to other morpheme analyzers.

a) Correct typos (necessary to get extended model answers)

- As it is an early childhood learning system, it can correct typos. In the case of lower grades, the result is that there are especially many typos.

- In the case of lower grades, phonological similarity + syllable unit language model (probability of a certain syllable appearing before/after a specific syllable) is used.

- In the case of high school students, difficult dictation correction rules (dictionary) and spacing are corrected

- Space correction can be corrected by automatically generating learning data or learning the seq2seq model.

B) Short answer type

- Repeat processing/rule processing: It is determined whether the answer is correct or not by perfect matching.

- Correct answer expansion: After morphological analysis, keyword-oriented correct answers are processed. For example, Answer: Strawberry, Input Answer: Strawberry. Also, exact match False, repeated correct answer: True, etc.

- Editing distance (although the area overlaps with the correction of typos): Calculation of editing distance -> Return score

(ex. Correct answer: Strawberry, Input answer: Add --> Similarity score: 4/5 = 80%)

- It can be processed through chunking. A chunk is a chunk, part, group, or bundle. For example, the chunks are divided into 'soccer clubs/basketball clubs'. This process of dividing into chunks is called chunking.

C) Comparison of similarity in descriptive sentence units (2m/m --> future advancement)

- Unlike the short-answer type, multiple data clusters for the extended model answer are created by correcting typos, and as shown in FIG. 15, the degree of similarity close to the correct answer is generated by comparing the similarity between the confirmed model answer and the actual answer. For this similarity calculation, a methodology suitable for each subject is used. 15 is a diagram illustrating an example of comparing an answer input by a learner of the automatic scoring unit with an extended model answer according to an embodiment of the present invention.

- Scoring is based on descriptive questions ending in one sentence. Due to the characteristics of the Korean language, a word of an expression has several words as synonyms, and sets of these synonyms are constructed as an educational dictionary. In the dictionary work, words similar to one word are clustered in a vector cluster and used. For example, the word white will have the same vector cluster as several words such as hazy, blurry, opaque, white, white, etc. Based on this, a user word dictionary is built and the remaining unnecessary data is removed, the similarity of the sentences is calculated using the empty set, and the result is derived as a percentage of how similar to the correct answer is.

- By providing descriptive grading in real time, learners can check which keywords are most important in the subject they are learning.

- By providing the elements necessary for the correct answer and the elements that were not needed for the correct answer after grading, the learner can provide the part that the learner missed in the subject, thereby causing the learner's understanding.

- Learners can be provided with important keywords needed for actual answers.

D) Essay type

- Steps are necessary for essay-type grading.

a) Step 1: You can make step 1 essay data by collecting descriptive scoring data and attaching a linking ending. For example, a means something. And it has a ~~ feature.

b) Step 2: Expansion of the data set in Step 1 enables discussion on two questions. For example, a means ~~, which has the ~~ feature. Also, b means ~~, and has the ~~ feature.

c) Step 3: It is not a descriptive answer to the question, but the part where the individual's judgment about the two questions is entered in the question. For example, a means ~~, which has the ~~ feature. Also, b means ~~, and has the ~~ feature. Therefore, a and b are ~~~, so in this problem, one method has to be used.

c) Step 4 : Step 3 is included, and additionally, it is necessary to suggest a better method. For example, a means ~~, which has the ~~ feature. Also, b means ~~, and has the ~~ feature. Therefore, a and b are ~~~, so it is better to use the ~~ method in this problem. But a better way is to do ~~ through ~~.

- In order to do this type of grading, it is necessary to collect a lot of data for narrative grading. Since the applicant can collect a lot of education-related content, data is being collected from the first stage through narrative data. This will be used to collect data step by step.

- What can be provided to the learner through this is to suggest a flow of thought or a way of thinking. By providing a solution to the existing problem, learners can develop their creativity through the narrative type of learning that has been completed in the conventional narrative type.

Meanwhile, the early childhood learning method according to an embodiment of the present invention may be stored in a recording medium as a program executable in a computer device.

That is, the method for early childhood learning according to an embodiment of the present invention includes the steps of: (a) receiving a user's handwriting as a touch input; (b) OCR processing the input of the user's handwriting; (c) post-processing the OCR processing result to recognize the user's handwriting; (d) pre-processing the recognized user's handwriting; (e) automatically scoring the pre-processed handwriting recognition result through natural language processing; and (f) post-processing the automatic scoring and outputting a scoring result, which may be stored in a computer-readable recording medium.

The recording medium on which the early childhood learning method according to an embodiment of the present invention is recorded may be a magnetic memory device including a magnetic tunnel junction (MTJ) device, and a plurality of memory cells (C) arranged in a matrix form It may be implemented as a memory cell array comprising a.

Each of the plurality of memory cells C may include an access transistor T and a memory M. Also, the memory cell array may include a plurality of word lines (WL), a plurality of source lines (SL), and a plurality of bit lines (BL). Each of the plurality of memory cells C may be electrically connected to a corresponding word line WL, a source line SL, and a bit line BL.

The plurality of word lines WL may be disposed parallel to each other in the first direction (x-direction) and extend in the second direction (y-direction), respectively. The plurality of source lines SL may be disposed parallel to each other in the second direction while extending in the first direction. The plurality of bit lines BL may be disposed parallel to each other in the second direction while extending in the same first direction as the source line SL. However, the plurality of source lines SL and the plurality of bit lines BL may be alternately disposed along the second direction.

In such a memory cell array structure, the source line SL extending in the same direction as the bit line BL crosses the adjacent bit line BL up and down to share the memory cell C. As shown in FIG. In particular, when any one word line WL is selected, since there are no other memory cells C around the selected memory cell C, a problem of duplicate selection may not occur. In other words, in the memory cell array structure according to the embodiment of the present invention, there is no memory cell C immediately adjacent to any one memory cell C along one word line WL, and there is no memory cell immediately adjacent to the memory cell C. (C) may be disposed on another word line WL adjacent to one word line WL.

More specifically, the connection relationship between the memory cell C, the word line WL, the source line SL, and the bit line BL will be described as follows.

The plurality of word lines WL are connected to gates of the access transistors T of the plurality of memory cells C, and each of the two adjacent word lines WL includes access transistors disposed at different positions in the second direction. It can be connected to the gate of (T).

For example, the second word line WL1 is formed between the first bit line BL0 and the second source line SL1 , between the second bit line BL1 and the third source line SL2 , and the third bit line It is connected to the gates of the access transistors T disposed between BL2 and the fourth source line SL3 , and the third word line WL2 is connected between the first source line SL0 and the first bit line BL0 . , may be connected to the gates of the access transistors T disposed between the second source line SL1 and the second bit line BL1 and between the third source line SL2 and the third bit line BL2 . .

The plurality of source lines SL are connected to the sources or drains of the access transistors T of the plurality of memory cells C, and each of the plurality of source lines SL alternately along the first direction to another bit line BL ) and may be connected to the source or drain of the access transistors T. For example, the second source line SL1 is a source or drain of the access transistor T connected to the first bit line BL0 and the access transistor T connected to the second bit line BL1 along the first direction. can be connected alternately.

The plurality of bit lines BL are connected to drains or sources of the access transistors T of the plurality of memory cells C, and each of the plurality of bit lines BL is alternately connected to another source line SL in the first direction. ) and may be connected to the drain or source of the access transistors T. For example, the first bit line BL0 serves as the drain or source of the access transistor T connected to the first source line SL0 and the access transistor T connected to the second source line SL1 along the first direction. can be connected alternately. Here, it can be seen that the bit line BL is connected to the drain or source of the corresponding access transistor T through the corresponding memory M.

Based on this connection relationship, when any one of the plurality of word lines BL, and any one of the plurality of source lines SL or the plurality of bit lines BL is selected, only one memory cell C is generated. can be chosen. For example, when the third word line WL2 and the third source line SL2 or the third bit line BL2 are selected, between the second word line WL1 and the third word line WL2 and the third source A memory cell Cs disposed between the line SL2 and the third bit line BL2 may be selected.

For reference, due to the characteristics of the magnetic memory device, when any one word line and any one bit line are selected, the corresponding source line may be automatically determined. Conversely, when any one word line and any one source line are selected, the corresponding bit line may be automatically determined. Meanwhile, when the memory cell array is divided into blocks according to a column source line (CSL), 8 word lines, 8 source lines, and 8 bit lines may be disposed in one block.

The access transistor T constituting the memory cell C is turned on or turned off according to the voltage of the word line WL to control the supply of current to the memory M. can do. For example, the access transistor T may be a MOS transistor or a bipolar transistor. In addition, the memory M constituting the memory cell C may include a magnetic material. For example, the memory M may include a magnetic tunnel junction (MTJ) device. The memory M may perform a memory function by using a spin transfer torque (STT) phenomenon in which a magnetization direction of a magnetic material is changed by an input current.

For reference, a brief description of a magnetic memory device, for example, a magnetic random access memory (MRAM), flows through the MTJ device to store “0” and “1” states in the MTJ device, which is a memory device included in the magnetic memory device. The current must be bidirectional. That is, the direction of the current flowing through the MTJ element when writing data "0" and writing data "1" must be opposite to each other.

In order to form a structure that allows current to flow in the opposite direction, a source line exists in addition to the bit line in the magnetic memory device. The bit line and the source line change the potential difference between the MTJ element and the access transistor (or cell transistor) of the memory cell, so that the direction of the current flowing through the MTJ element of each memory cell can be selected.

The magnetic memory device according to the inventive concept may have a memory cell array structure capable of minimizing the size of a unit memory cell while applying an individual source line method. In other words, the memory cell array may be designed in such a way that one source line is disposed on one bit line. Accordingly, as in the case of the conventional individual source line method, the operating voltage can be lowered by changing the voltages of the bit line and the source line to each other.

Also, the memory cells are arranged in such a way that two source lines share one bit line, so that the memory cells are arranged in a zigzag manner along the word line WL and along the source line SL or the bit line BL. Therefore, the size of the unit memory cell may be minimized.

For example, the magnetic memory device according to the inventive concept may have a size of a unit memory cell Cu. Specifically, when the pitch between the word lines WL is 2F and the pitch between the source lines SL or the bit lines BL is 4F, the size of the unit memory cell Cu may be ^{8F 2 .} Here, F may mean a minimum lithographic feature size.

Meanwhile, the memory cells are arranged in a zigzag pattern along the word line WL and the source line SL or the bit line BL and are connected to the word line WL, the source line SL, and the bit line BL. , only one memory cell can be selected by selection of any one word line and any one bit line (or any one source line).

Although the memory cell is exemplified as the recording medium according to the embodiment of the present invention, the present invention is not limited thereto, and may be equally applied to CD, LP, CD-RW, DVD, Blu-ray Disc (BD), and the like.

As described above, according to the present invention, a narrative problem is provided on the screen of a computer terminal or a portable terminal used by an infant user, and when the infant user directly writes and inputs an answer to the problem, the input cursive image data is used Thus, it is possible to provide a method for learning children through handwriting recognition and a recording medium recording the same by automatically scoring and providing them with handwriting recognition and natural language processing technology so that infant users can learn at any time 24 hours a day.

In the above, the embodiments of the present invention have been mainly described, but various changes or modifications can be made at the level of those skilled in the art to which the present invention pertains. Such changes and modifications can be said to belong to the present invention without departing from the scope of the technical spirit provided by the present invention. Accordingly, the scope of the present invention should be judged by the claims described below.

100: early childhood learning system 110: student terminal
120: early childhood learning server 130: early childhood learning DB
140: teacher terminal 210: handwriting recognition unit
212: pre-processing unit 214: OCR engine unit
216: post-processing unit 220: automatic scoring unit
222: pre-processing unit 224: NPL scoring engine unit
226: post-processing unit

Claims (5)

(a) receiving a user's handwriting through a touch input;
(b) OCR processing the input of the user's handwriting;
(c) recognizing the user's handwriting by post-processing the OCR processing result by a handwriting recognition unit;
(d) pre-processing the recognized user's handwriting;
(e) automatically scoring the pre-processed handwriting recognition result through natural language processing; and
(f) outputting a scoring result by post-processing the automatic scoring;
including,
The automatic scoring step uses entity name recognition and keyword extraction for descriptive feature extraction, obtains extended model answers through typographical correction, and phonological similarity and syllable unit language model for lower grades for typographical correction. In the case of high school students, using dictation correction rules and spacing Create narrative scoring data by comparing the extended model answer with the learner input answer based on the narrative question ending in the sentence and scoring as an empty set while removing the remaining unnecessary data,
In the automatic scoring step, the descriptive scoring data is collected and a linking ending is added to make the first-stage essay data, and the data set for the first-stage essay data is expanded by “meaning a, has b features” and Create two-stage essay data that includes a question essay “It means a thing and has b features” In this case, create 3 stage essay data including the part where the judgment that d method should be used", and the above 3 stage essay data are included and additionally "a means a thing and has b characteristics. Therefore, a and b are c, so , it is better to use method d in this problem, but a better method is to do f through e” How to learn early childhood.
The method of claim 1,
In step (c), the handwriting recognition unit refines data through a binarization process, a labeling process, and a manual labeling merging process for a handwritten image according to the handwriting input from the user,
In the binarization process, the handwriting image according to the handwriting is input as a single-channel image of gray scale, and when it exceeds a threshold value, it is changed to a maximum value, and when it is less than the threshold value, it is converted to 0;
In the labeling process, according to the Connected Component Labeling technique, the image is scanned from left to right and from top to bottom, and the top and left sides are checked for pixels whose pixel values are not 0. Allocate a new label (number), if only one of the two is non-zero, assign it to that label, if not all zero, assign one of the two labels, set these two labels to equivalence, and Repeating the process of checking the top and left on the pixel to reset the label for the equivalent label.
The method of claim 1,
In the step (e), the natural language processing expresses the recognized word for the user's handwriting as an embedding vector in the form of a dense vector, word embedding, and the embedding vector Deriving the similarity with respect to the other two words and sentences using
In step (e), the automatic scoring compares the handwriting recognition result with the content model answer, derives the scoring result by rule-based and edit distance calculation in the case of short answer type, and input answer characteristic and model answer characteristic in the case of narrative type A method for early childhood learning that derives a scoring result by calculating the similarity by executing feature extraction for
4. The method of claim 3,
In the case of the narrative type, the feature extraction is performed through spacing correction, morpheme analysis for nouns and verbs, chunking for noun phrases and verb phrases, and negative expression recognition using indefinite tags.
(a) receiving a user's handwriting through a touch input;
(b) OCR processing the input of the user's handwriting;
(c) post-processing the OCR processing result to recognize the user's handwriting;
(d) pre-processing the recognized user's handwriting;
(e) automatically scoring the pre-processed handwriting recognition result through natural language processing; and
(f) outputting a scoring result by post-processing the automatic scoring;
including,
The automatic scoring step uses entity name recognition and keyword extraction for descriptive feature extraction, obtains extended model answers through typographical correction, and phonological similarity and syllable unit language model for lower grades for typographical correction. In the case of high school students, using dictation correction rules and spacing Create narrative scoring data by comparing the extended model answer with the learner input answer based on the narrative question ending in the sentence and scoring as an empty set while removing the remaining unnecessary data,
In the automatic scoring step, the descriptive scoring data is collected and a linking ending is added to make the first-stage essay data, and the data set for the first-stage essay data is expanded by “meaning a, has b features” and Create two-stage essay data that includes a question essay “It means a thing and has b features” In this case, create 3 stage essay data including the part where the judgment that d method should be used", and the above 3 stage essay data are included and additionally "a means a thing and has b characteristics. Therefore, a and b are c, so , in this problem, it is better to use method d, but a better method is to do f through e.” A computer readable program records the early childhood learning method that executes essay-type scoring by creating four-step narrative data. media.

Images