CN111078874B - Foreign Chinese difficulty assessment method based on decision tree classification of random subspace - Google Patents

Abstract

The invention discloses a decision tree classification method for assessing the difficulty of Chinese as a foreign language based on random subspace feature selection of svm and bert models. According to the length and readability of articles, 86 statistical features are generated and classified by svm to obtain Confidence level 1. The encoding feature is classified with svm to obtain a confidence level of 2. The two confidence levels obtained are fused as new features, and the decision tree is used for classification. For the encoding feature data: the output information result of the encoding-1 layer extracted by the BERT model, and then average->max pooling processing is performed to obtain a total of 768-dimensional features, which do not need to be normalized. The invention avoids the problems of inefficiency and underfitting of traditional algorithms, and uses all information most reasonably, so that the effect of increasing classification basis is remarkable. This method has achieved an accuracy rate of 85.6% in evaluating the difficulty of Chinese as a foreign language.

Description

Evaluation Method of Difficulty of Chinese as a Foreign Language Based on Random Subspace Decision Tree Classification

technical field

The invention belongs to the field of educational informatization, and in particular relates to a method for evaluating difficulty of Chinese as a foreign language based on decision tree classification of random subspace feature selection of SVM and BERT models.

Background technique

We all know that reading should be done step by step, from easy to difficult. Too difficult can easily lead to frustration of students' self-confidence and loss of interest in reading. Too simple and low-level repetition is not conducive to the continuous improvement of reading ability, and cannot meet the academic requirements for reading complex texts and conducting related research after entering university. In short, only the difficulty is suitable for the best. With the development of China, China is playing an increasingly important role on the international stage, which makes more people have the demand to learn Chinese. Learning Chinese texts is one of the most effective ways, but learning Chinese texts of a certain difficulty requires Chinese learners to have a certain level of cultural literacy. Crack down on the interests and hobbies of Chinese learners. And when cultivating the writing ability of Chinese learners, we should provide various styles for reference, and be able to judge and score based on the style of composition written by Chinese learners. Therefore, the classification of Chinese texts is the key technology to assist the Chinese learning system.

The difficulty level of graded Chinese as a foreign language refers to whether the readings of this level are suitable for Chinese learners who have reached this level of Chinese language, and whether the readings are too difficult or too easy.

Text classification is to use computers to automatically classify and mark text sets according to certain classification systems or standards. According to whether deep learning technology is used or not, it is divided into two categories. The first category is based on traditional machine learning text classification, and the second category is based on Deep Learning Text Classification. Of course, some cases in the text classification technology in the second category will use the combination of deep learning method and traditional machine learning method.

In the late 1990s, traditional machine learning developed rapidly, forming a set of inherent models for text classification problems, feature engineering + classifier model. The feature engineering here is to extract the information in the text, so that the computer can easily recognize and read the information in the text. Usually, the feature engineering is divided into three steps, the first step is text preprocessing, the second step is feature extraction, and the third step is text representation. . The more famous classifier models include Naive Bayesian classification algorithm, KNN, SVM, maximum entropy and so on.

In the NLP method based on deep neural networks, the words/words in the text are usually represented by one-dimensional vectors (generally called "word vectors"); on this basis, the neural network will convert each word or word in the text The one-dimensional word vector of is used as input, and after a series of complex transformations, a one-dimensional word vector is output as the semantic representation of the text. In particular, it is usually expected that the words/words with similar semantics are relatively close in the feature vector space, so that the text vector converted from the word/word vector can also contain more accurate semantic information. Therefore, the main input of the BERT model is the original word vector of each word/word in the text, which can be initialized randomly, or pre-trained using algorithms such as Word2Vector as the initial value; the output is the fusion of each word/word in the text Vector representation after full-text semantic information.

At present, most of the Chinese text classification is for simple and short text collections such as Weibo and news, but the existing methods for Chinese text classification for Chinese learners are not ideal.

Contents of the invention

For at least one of the above deficiencies or improvement needs of the prior art, especially due to the complexity of the text classification problem of Chinese learners, when facing the different needs of Chinese learners, the classification standards will change accordingly, Aiming at this task, the present invention proposes a method for assessing the difficulty of Chinese as a foreign language based on the fusion of Bert model, svm and decision tree features. According to the length of the article, readability and other characteristics, 86 statistical features are generated, and svm is used to classify, and the confidence level is 1. The encoding features are classified with svm to obtain a confidence level of 2. The two confidence levels obtained are fused as new features, and the decision tree is used for classification.

In order to achieve the above object, according to one aspect of the present invention, a method for evaluating the difficulty of Chinese as a foreign language based on the decision tree classification of random subspace feature selection of svm and bert model is provided, comprising the following steps:

S1. Preprocessing the articles in Chinese as a foreign language;

S2, for the article in Chinese as a foreign language after the preprocessing of step S1, according to the length of the article in Chinese as a foreign language, the legibility of the article, and the amount of new words in the article generate multiple features;

S3, then use the svm based on the random subspace to classify the articles containing all the above features, and obtain a confidence level of 1;

S4. For the Chinese-language articles after the preprocessing in step S1, the output information result of the -1 layer of encoding extracted by the BERT model is processed by average->max pooling to obtain the multi-dimensional encoding features of the article;

S5, classify the encoding features using the svm based on the random subspace, and obtain a confidence level of 2;

S6. Combine the obtained two confidence levels as new features, and use a decision tree to classify.

Preferably, in step S1, preprocessing the foreign language article includes saving it in txt format.

Preferably, in step S1, preprocessing the article in Chinese as a foreign language includes deleting blank lines in the article.

Preferably, in step S1, preprocessing the article in Chinese as a foreign language includes dividing the article into sentences.

Preferably, in step S1, sentence segmentation is to use python to cut each article in sentence units, store it in a list structure, and remove punctuation marks.

Preferably, the multiple features generated in step S2 include the total number of words, the total number of strokes, the number of paragraphs, the total number of sentences, and the number of new words.

Preferably, in step S6, the weighted average of the confidence level 1 and the confidence level 2 is used as the comprehensive output of this article. The above preferred technical features can be combined with each other as long as they do not conflict with each other.

Generally speaking, compared with the prior art, the above technical scheme conceived by the present invention has the following beneficial effects: the method for evaluating the difficulty of Chinese as a foreign language based on the random subspace feature selection of svm and bert model, using the Bert model Strong text feature extraction ability, obtaining the representation of foreign Chinese articles containing rich semantic information, combined with the statistical characteristics of traditional article words, which can make full use of various features of the article. The invention avoids the problems of inefficiency and underfitting of the traditional algorithm, uses all the information most reasonably, and makes the effect of increasing the basis of classification remarkable. This method has achieved an accuracy rate of 85.6% in evaluating the difficulty of Chinese as a foreign language.

Description of drawings

Fig. 1 is the overall schematic diagram of the foreign Chinese difficulty assessment method of the decision tree classification based on the random subspace feature selection of svm and bert model of the present invention;

Fig. 2 is a structural diagram of the encoding feature extracted from an article based on the Bert model used in the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not constitute a conflict with each other. The present invention will be further described in detail below in combination with specific embodiments.

As shown in Figure 1, the present invention provides a kind of method for assessing the difficulty of Chinese as a foreign language based on the decision tree classification of random subspace feature selection of svm and bert model, comprising the following steps:

S1. Preprocessing the article in Chinese as a foreign language, including saving it in txt format, deleting blank lines in the article, and dividing the article into sentences. Sentence segmentation is to use python to divide each article into sentences, store them in the list structure, and remove punctuation marks;

S2, for the article in Chinese as a foreign language after the step S1 preprocessing, according to the length of the article in Chinese as a foreign language, the legibility of the article, the number of new words in the article generates a plurality of features, such as 86, including the total number of words, the total number of strokes, the number of paragraphs, The total number of sentences, the number of new words;

S3, then use the svm based on the random subspace to classify the articles containing all the above features, and obtain a confidence level of 1;

S4. For the Chinese-language articles after the preprocessing in step S1, the output information result of the encoding layer -1 extracted by the BERT model is processed by average->max pooling to obtain the multi-dimensional encoding features of the article, as shown in Figure 2;

S5, classify the encoding features using the svm based on the random subspace, and obtain a confidence level of 2;

S6. Combine the obtained two confidence levels as new features, and use a decision tree to classify. Preferably, in step S6, the weighted average of the confidence level 1 and the confidence level 2 is used as the comprehensive output of this article. The above preferred technical features can be combined with each other as long as they do not conflict with each other.

Illustrate below with detailed example, the present invention provides a kind of method for assessing the difficulty of learning Chinese as a foreign language based on the decision tree classification of random subspace feature selection of svm and bert model, comprising the following steps:

(1) Use crawler technology to crawl the compositions on Composition Online according to grades (first grade in elementary school to third grade in high school), and correctly divide the data set based on grades, and write the grade information in front of the file name, and store It is in txt format.

(2) For the articles of each grade, one of the most representative articles should be selected as a benchmark article and taken out separately as the standard representative of each type.

(3) Use python to cut each article into sentences, store them in a list structure, and remove punctuation marks.

(4), for the above pre-processed Chinese as a foreign language article, according to the length of the Chinese as a foreign language article, the readability of the article, and the amount of new words in the article, multiple features are generated, such as 86, including the total number of words, the total number of strokes, and the number of paragraphs , the total number of sentences, the number of new words; the degree of difficulty of graded Chinese as a foreign language, the present invention investigates the degree of difficulty of graded Chinese as a foreign language from three angles, the first is the length of the reading, i.e. the number of Chinese words contained in the reading, the second is The readability of the reading material refers to the average sentence length and the average number of sentences per 100 words of the reading material. The third is the number of new words in the reading material, which means the number of new words appearing in the reading material.

(5) Then use the random subspace-based svm combination to classify the articles containing all the above features, and get a confidence level of 1.

(6) For the pre-processed Chinese language articles, the -1 layer output information of the encoding extracted by the BERT model is then processed by average->max pooling to obtain the multi-dimensional encoding features of the article, as shown in Figure 2. For The Bert structure will encode the input of each sentence, so the label attention weighting mechanism and the weight value of words will be changed. Multi-core will make the boundary of label embedding more detailed and better fit the data.

(7) Classify the encoding features using svm based on random subspaces to obtain a confidence level of 2.

(8) Combine the obtained two confidence levels as a new feature, and use a decision tree to classify. Among them, each article is cut into a combination of multiple sentences during training, so the sentence is the basic unit of input, and after classifying each sentence of an article, the weighted average is used as the Synthesized output of this post.

<Experiment description and results>

In this example, a total of 51,356 essays were crawled from 13 essays on the Internet, and the essays were classified according to 12 grades from elementary school to high school. 4,000 essays of various types were screened out, with a total of 48,000 essays, and the essays were stored in txt format. The ratio of training set, test set and verification set is 7:2:1, and then use the training set to implement training according to the specific implementation method, and observe the accuracy of the verification set to choose the time point to terminate the training.

Every time a model with a fixed core is trained, the sequence of all samples will be disrupted to retake the training set, test set, and verification set, and then train and verify again. A total of 10 rounds of operations have been performed. The results in the table below are the average of the 10 experimental results.

The specific experimental results are shown in Table 1.

Table 1. Experimental results

Model svm core F1-score mean SVM+Bert+DT(Decision Tree) linear kernel function 82.32% SVM+Bert+DT polynomial kernel function 82.47% SVM+Bert+DT RBF kernel function (Gaussian kernel function) 85.6%

In summary, aiming at the text classification problem of evaluating the difficulty of articles in Chinese as a foreign language, the present invention proposes a decision tree classification based on random subspace feature selection of svm and bert model and an automatic classification method for evaluating the difficulty of Chinese as a foreign language and using the Bert model Strong text feature extraction ability, obtaining the representation of foreign Chinese articles containing rich semantic information, combined with the statistical characteristics of traditional article words, which can make full use of various features of the article. The invention avoids the problems of inefficiency and underfitting of the traditional algorithm, uses all the information most reasonably, and makes the effect of increasing the basis of classification remarkable. This method has achieved an accuracy rate of 85.6% in evaluating the difficulty of Chinese as a foreign language.

It is easy for those skilled in the art to understand that the above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention, All should be included within the protection scope of the present invention.

Claims (7)

1. A foreign Chinese difficulty assessment method of decision tree classification based on random subspace feature selection of svm and bert models is characterized by comprising the following steps:

s1, preprocessing a foreign-Chinese language chapter;

s2, generating a plurality of characteristics of the foreign Chinese article preprocessed in the step S1 according to the length of the foreign Chinese article, the readability of the article and the word quantity of the article;

s3, classifying the articles containing all the characteristics by using svm combination based on a random subspace to obtain a confidence 1;

s4, extracting an encoding-1 layer output information result of the Chinese article preprocessed in the step S1 through a BERT model, and then performing average- > max firing processing to obtain a multi-dimensional encoding characteristic of the article;

s5, classifying the encoding characteristics by using svm based on a random subspace to obtain a confidence coefficient 2;

and S6, fusing the obtained 2 confidence coefficients to serve as new features, and classifying by using a decision tree.

2. The method for assessing difficulty of chinese language outbound through decision tree classification based on stochastic subspace feature selection of svm and bert models as claimed in claim 1, wherein:

in step S1, preprocessing the foreign Chinese language chapter includes storing the foreign Chinese language chapter in a txt format.

3. The method for assessing difficulty of chinese language outbound of decision tree classification based on stochastic subspace feature selection of svm and bert models of claim 2, wherein:

in step S1, preprocessing the foreign-chinese language chapter includes deleting empty lines in the article.

4. The method for assessing difficulty of chinese language outbound of decision tree classification based on stochastic subspace feature selection of svm and bert models of claim 3, wherein:

in step S1, preprocessing the foreign-chinese language seals includes sentence-splitting the articles.

5. The method for assessing difficulty of chinese language outbound through decision tree classification based on stochastic subspace feature selection of svm and bert models as claimed in claim 4, wherein:

in step S1, the sentence segmentation is to cut each article by using python in sentence units, store the article in a list structure, and remove punctuation marks.

6. The method for foreign chinese difficulty assessment by decision tree classification based on stochastic subspace feature selection of the svm and bert models according to claim 1, wherein:

the plurality of features generated in step S2 include the total number of words, the total number of strokes, the number of paragraphs, the number of sentences, and the number of generated words.

7. The method for foreign chinese difficulty assessment by decision tree classification based on stochastic subspace feature selection of the svm and bert models according to claim 1, wherein:

in step S6, the weighted average of confidence 1 and confidence 2 is used as the overall output of the article.

Images