CN115630140A - English reading material difficulty judgment method based on text feature fusion - Google Patents

Abstract

The invention relates to a method for judging difficulty of English reading materials based on text feature fusion, and belongs to the field related to natural language processing. Firstly, encoding an input English text aiming at an English reading material data set, inputting an encoded result into a pre-training language model, and calculating to obtain a feature vector containing semantic information. And then, carrying out part-of-speech tagging on the English text, inputting the obtained part-of-speech sequence into the LSTM, and calculating to obtain a feature vector containing grammar information. And counting relevant factors and extracting features of the relevant factors according to the factors influencing the difficulty degree of English reading materials. All the obtained feature vectors are spliced and input into a full-connection layer, and finally a numerical value from 0 to 1 is output through sigmoid to represent difficulty. The method can effectively judge the difficulty of English reading materials and better assist various self-adaptive learning services in English teaching.

Description

A method for judging the difficulty of English reading materials based on text feature fusion

technical field

The invention relates to a method for judging the difficulty of English reading materials based on text feature fusion, and belongs to the technical field of natural language processing.

Background technique

English is widely learned as a second language, and reading is an important part of English learning. How to accurately judge the difficulty of English reading materials so that people with different English levels can receive education suitable for their own English level, It is particularly important to further promote personalized learning.

In the early 20th century, studies on measuring the difficulty level of English reading materials appeared. Until now, the research on the difficulty judgment of English reading materials has been the core issue concerned by relevant researchers at home and abroad. Therefore, many researchers have conducted a lot of research on the factors that affect the difficulty of English reading materials, summed up many influencing factors, and produced many formulas for calculating the difficulty of English reading materials. English text of . However, with the continuous development of informatization, the generated texts become more and more complex, and the method of formulating rules is usually relatively simple and does not have good generalization ability, so good results cannot be achieved.

With the continuous development of language models, Google proposed the BERT (Bidirectional Encoder Representation from Transformers) model in October 2018, which brought the development of natural language processing into a new stage. BERT is a pre-trained language model. Unlike traditional language models, it only uses a one-way language model or two one-way language models for shallow splicing. Instead, it uses MLM (masked language model ) to conduct and train bidirectional Transformers, generate deep bidirectional language representations, and perform well in 11 different natural language processing (Natural Language Processing, NLP) tests. Many scholars have achieved good results in combining BERT with other tasks in the field of natural language processing. This method of migrating the trained model to a new model for training is called transfer learning. Considering that most of the tasks are related to a certain extent, passing the learned parameters to the new model in some way can greatly speed up the efficiency of the model. Fine-tuning is one of the methods of transfer learning. By freezing the convolutional layer in the pre-trained model and training other convolutional layers and fully connected layers, the learning time of the model can be further improved and the cost of model training can be reduced.

Contents of the invention

The technical problem to be solved by the present invention is to provide a method for judging the difficulty of English reading materials based on text feature fusion, which is used to improve the accuracy and efficiency of judging the difficulty of English reading materials.

The present invention proposes a method for judging the difficulty of English reading materials based on text feature fusion by summarizing the views of linguists on factors affecting the difficulty of English reading materials, and taking into account the advantages of pre-trained language models in natural language processing tasks. Integrate multiple text features and use deep learning technology to judge the difficulty of English reading materials.

The technical solution of the present invention is: a method for judging the difficulty of English reading materials based on the fusion of text features. First, the input English text is encoded for the English reading material data set, and the encoded information is input into the pre-trained In training the language model, the feature vector containing semantic information is obtained; then the input text is part-of-speech tagged, and the obtained part-of-speech sequence is input into LSTM to obtain the feature vector containing grammatical information; the factors affecting the difficulty of English reading materials are counted and analyzed It performs embedding representation, splicing all the features into the fully connected layer, and finally outputs a 0 to 1 value to indicate the difficulty through the sigmoid layer.

The specific steps of judging the difficulty of English reading are as follows:

Step1: Use the pre-trained language model to extract the semantic features of the text.

Firstly, for the English reading material data set (using Newsela data set and self-collected data set for experiments), the input English text is encoded, and the encoded information is input into the pre-trained language model that has been trained, and the semantic content is obtained. The feature vector of information.

The specific process is to first extract information such as word, sentence position and word position in the sentence to perform One-hot encoding, input the pre-training language model, and obtain the semantic feature vector. The pre-training model of the present invention selects the Bert model.

Step2: Grammatical information feature extraction.

The text is part-of-speech tagged, and the obtained part-of-speech sequence is input to LSTM to obtain a feature vector containing grammatical information.

Step3: Statistical information feature extraction.

The factors that affect the difficulty of English reading materials are counted and embedded to represent them, all the features are spliced and input into the fully connected layer, and finally a 0 to 1 value is output through the sigmoid layer to indicate the difficulty.

Step4: Difficulty prediction.

After the sigmoid layer output, a value from 0 to 1 is obtained to indicate the difficulty.

The Step1 is specifically:

Step1.1: Assume that the current input English text is S _t , and S _t contains n words, S _t = {w ₁ ,w ₂ ,…, _wi ,…,w _n }, where w _i represents the i-th word word.

The Bert model usually adds [CLS] at the beginning of a sentence to indicate the beginning of a paragraph, and [SEP] in the middle of two sentences to separate sentences.

The transformed sentence is S _BERT ={[CLS],w ₁ ,w ₂ ,...,[SEP],..., _wn-2 , _wn-1 , _wn ,[SEP]}.

Step1.2: Set the maximum length of S _BERT to M. If the length of S _t is less than M, then add [PAD] to S _BERT to make up. The S _BERT after the padding operation is:

S _BERT ＝{[CLS],w ₁ ,w ₂ ,…,[SEP],…,w _n-2 ,w _n-1 ,w _n ,[SEP],…,[PAD]}

If the length of S _t is greater than M, then truncate and discard the subsequent content, and the S _BERT after truncation is:

S _BERT ＝{[CLS],w ₁ ,w ₂ ,…,[SEP],…,w _M-2 ,w _M-1 ,w _M ,[SEP]}

Step1.3: Perform embedding encoding for each content in S _BERT , namely:

D_BERT表示预训练语言模型设定的嵌入维度。in,

D _BERT represents the embedding dimension set by the pre-trained language model.

Step1.4: Encode the sentence position of the content in S _BERT , namely:

S _{segmentembedding} ＝{E _A ,E _A ,E _A ,E _B ,E _B ,E _B ,E _B ,…,E _i ,E _i }

后续句子以此类推，E_i表示第i句话。Among them, E _A represents the first sentence, E _B represents the second sentence,

Subsequent sentences are deduced by analogy, and E _i represents the i-th sentence.

Step1.5: Encode the word position of the content in S _BERT , namely:

S _{position embedding} ＝{E ₁ ,E ₂ ,E ₃ ,…,E _i ,…,E _n-2 ,E _n-1 ,E _n ,…,E _M }

Among them, E _i represents the position code of the i-th word,

Step1.6: Input S _embedding , S _{segmrntembedding} , and S _{positionembedding} into the pre-trained language model (BERT is used by default), and obtain the feature vector O _BERT output by the last layer, namely:

Step1.7: There are many options for selecting the sentence vector, such as: 1) Take X _[CLS] as the sentence vector. 2) Perform average pooling on O _BERT and take the result. 3) Perform maximum pooling on O _BERT and take the result. 4) Use the result of _OBERT to further extract features using CNN. 5) Input the result of _OBERT into LSTM to extract features. In the task of the present invention, choose X _[CLS] as sentence vector.

Described Step2 specifically is:

Step2.1: For the input text S _t = {w ₁ ,w ₂ ,w ₃ ,…,w _n }, add [CLS] at the beginning of the sentence to indicate the beginning of a sentence, and add [SEP] in the middle of the two sentences Used to separate sentences, the converted sentences are:

S _sen ＝{[CLS],w ₁ ,w ₂ ,…,[SEP],…,w _n-2 ,w _n-1 ,w _n ,[SEP],…,[PAD]}

Step2.2: Perform part-of-speech tagging on S _sen to get:

S _POS ＝{[SPACE],[PRP],[VBP],[NNP],…,[RB],[JJ],[SPACE],…,[PAD]}

Among them, [SPACE] represents [CLS] and [SEP], [PRP] represents a pronoun, [VBP] represents a verb, [NNP] represents a noun, [RB] represents an adverb of degree, and [JJ] represents an adjective.

Step2.3: Embed S _POS to get E _POS , namely:

Among them, D _POS represents the embedding dimension of the part-of-speech token.

Step2.4: Input E _pos into LSTM, and take the output result O _pos of the last layer as the feature vector (ie, grammatical feature) of the sentence part-of-speech sequence, where

In Step2, the grammatical features of sentences are mainly calculated. Grammar and vocabulary are the key to distinguish the difficulty of English texts, so the complexity of grammar needs to be considered. The present invention takes the part-of-speech sequence of the sentence as input, and uses LSTM to learn the features of the sequence, thereby realizing the vectorized representation of the grammar and inputting it into the neural network for calculation in subsequent steps. In the existing methods, grammatical information is mainly obtained by counting the number of keywords and counting the co-occurrence of keywords. This method cannot fully represent the sequence information, so the use of LSTM in this invention can better learn grammatical features.

The Step3 is specifically:

In addition to semantics and grammar, the factors that affect the difficulty of English reading materials also need to consider sentence length, number of prepositions, average word length, etc. After adding the above information, the convergence of the model training is faster, and the robustness of the model is further improved.

Specific steps are as follows:

Step3.1: Make statistics on the sentence length and perform embedding operation: for the sentence S _t = {w ₁ ,w ₂ ,…,w _n }, the sentence length embedding is

Among them, L indicates that the vector is the embedding of the sentence length, n represents the number of words, and D represents the embedding dimension.

Step3.2: Count the number of prepositions and perform embedding operations: For the sentence S _t = {w ₁ ,w ₂ ,…,w _n }, the number of prepositions is embedded

Among them, P represents that the vector is the embedding of the number of prepositions, * represents the specific number, and D represents the embedding dimension.

Step3.3: Make statistics on the average word length and perform embedding operation: For the sentence S _t ={w ₁ ,w ₂ ,…,w ₁ }, the number of prepositions is embedded

Among them, A represents the embedding of the vector as the average word length, * represents the specific number, and D represents the embedding dimension.

进行拼接作为句子的统计信息：Step3.4: will

Concatenate as sentence statistics:

in,

Described Step4 is specifically:

Step4.1: Concatenate the semantic feature X _[CLS] , grammatical feature O _POS , and statistical information feature O _STA into the fully connected layer, then input the prediction result of the sigmoid layer and output:

Step4.2: Calculate the loss:

Among them, y _ic represents the true category of sample i, if it is equal to c, it takes 1, if it is not equal to 0, p _ic represents the predicted probability that observed sample i belongs to category c;

Step4.3: Use Adam to optimize the loss, the purpose is to minimize the loss, when the loss reaches the minimum, the model reaches the best effect.

In this part, the above three features are spliced and input into the neural network, and the sigmoid function is used to limit the output between [0,1], so as to realize the judgment of difficulty.

The beneficial effects of the present invention are: the present invention comprehensively considers the semantic information, grammatical information, statistical information and other features of the text when judging the difficulty of the English text. Compared with the traditional method, the present invention considers the importance of the English text semantic information. It uses LSTM to learn the grammatical information of the text, and at the same time, the traditional statistical information is also input into the neural network for calculation. Thus, a difficult-easiness judgment model with better effect and stronger robustness than traditional methods is obtained.

Description of drawings

Fig. 1 is a flow chart of steps of the present invention.

Detailed ways

The present invention will be further described below in combination with the accompanying drawings and specific embodiments.

Embodiment 1: As shown in Figure 1, a method for judging the difficulty of English reading materials based on text feature fusion, firstly, for the English reading material data set, the input English text is encoded, and the encoded information is input into the trained In a good pre-trained language model, the feature vector containing semantic information is obtained; then the English text is part-of-speech tagged, and the obtained part-of-speech sequence is input into the LSTM to obtain the feature vector containing grammatical information; the factors that affect the difficulty of English reading materials are counted And embedding it, all the features are spliced and input into the fully connected layer, and finally a 0 to 1 value is output through the sigmoid layer to indicate the difficulty.

Suppose there is a collection A of English reading materials, and there are N pieces of data of English reading materials in the collection, then A={S ₁ , S ₂ , S ₃ ,...,S _N }, where S _i represents the number of English reading materials in the collection Article i English reading material text. The specific steps of judging the difficulty of English reading are as follows:

Step1: The pre-training model of the present invention selects the Bert model. The pre-training language model part is mainly used to learn the semantic information of the text. Three features are required to input the pre-training language model, which are the features of each word, the sentence position feature and the word position Features, extract three features.

Step2: Grammatical feature extraction.

Step3: Statistical information feature extraction.

Step4: Difficulty prediction.

The Step1 is specifically:

Step1.1: Assume that the current input English text is S _t , and S _t contains n words, S _t = {w ₁ ,w ₂ ,…, _wi ,…,w _n }, where w _i represents the i-th word word.

The Bert model usually adds [CLS] at the beginning of a sentence to indicate the beginning of a paragraph, and [SEP] in the middle of two sentences to separate sentences.

The transformed sentence is S _BERT ={[CLS],w ₁ ,w ₂ ,...,[SEP],..., _wn-2 , _wn-1 , _wn ,[SEP]}.

Step1.2: Set the maximum length of S _BERT to M, if the length of S _t is less than M, then add [PAD] to S _BERT to complete, and S _BERT after the padding operation = {[CLS],w ₁ , w ₂ ,…,[SEP],…,w _n-2 ,w _n-1 ,w _n ,[SEP],…,[PAD]}.

If the length of S _t is greater than M, truncate and discard the subsequent content, S _BERT after the truncation operation = {[CLS],w ₁ ,w ₂ ,…,[SEP],…,w _M-2 ,w _{M- 1} ,w _M ,[SEP]}.

Step1.3: Perform embedding encoding for each content in S _BERT , namely:

D_BERT表示预训练语言模型设定的嵌入维度。in

D _BERT represents the embedding dimension set by the pre-trained language model.

Step1.4: Encode the sentence position of the content in S _BERT , namely:

S _{segmentembedding} ＝{E _A ,E _A ,E _A ,E _B ,E _B ,E _B ,E _B ,…,E _i ,E _i }

后续句子以此类推，其中E_i表示第i句话。Where E _A represents the first sentence, E _B represents the second sentence,

Subsequent sentences are deduced by analogy, where E _i represents the i-th sentence.

Step1.5: Encode the word position of the content in S _BERT , namely:

S _{position embedding} ＝{E ₁ ,E ₂ ,E ₃ ,…,E _i ,…,E _n-2 ,E _n-1 ,E _n ,…,E _M }

where E _i represents the position code of the i-th word,

Step1.6: Input S _embedding , S _{segmrntembedding} , and S _{positionembedding} into the pre-trained language model (BERT is used by default), and obtain the feature vector O _BERT output by the last layer, namely:

Step1.7: There are many options for selecting the sentence vector, such as: 1) Take X _[CLS] as the sentence vector. 2) Perform average pooling on O _BERT and take the result. 3) Perform maximum pooling on O _BERT and take the result. 4) Use the result of _OBERT to further extract features using CNN. 5) Input the result of _OBERT into LSTM to extract features. In the task of the present invention, choose X _[CLS] as sentence vector.

Described Step2 specifically is:

Step2.1: For the input text S _t = {w ₁ ,w ₂ ,w ₃ ,…,w _m }, add [CLS] at the beginning of the sentence to indicate the beginning of a sentence, and add [SEP] in the middle of the two sentences Used to separate sentences, the converted sentences are:

S _sen ＝{[CLS],w ₁ ,w ₂ ,…,[SEP],…,w _n-2 ,w _n-1 ,w _n ,[SEP],…,[PAD]}

Step2.2: Perform part-of-speech tagging on S _sen to get:

S _POS ＝{[SPACE],[PRP],[VBP],[NNP],…,[RB],[JJ],[SPACE],…,[PAD]}

Among them, [SPACE] represents [CLS] and [SEP], [PRP] represents a pronoun, [VBP] represents a verb, [NNP] represents a noun, [RB] represents an adverb of degree, and [JJ] represents an adjective.

Step2.3: Embed S _POS to get E _POS , namely:

Among them, D _POS represents the embedding dimension of the part-of-speech token.

Step2.4: Input E _pos into LSTM, and take the output result O _pos of the last layer as the feature vector of sentence grammatical information, where

The Step3 is specifically:

Since the factors that affect the difficulty of English reading materials are not only semantics and grammar, but also sentence length, number of prepositions, and average word length, etc. should be considered as influencing factors, these factors are counted and coded and then input into the model. The specific steps are as follows:

Step3.1: Make statistics on the sentence length and perform embedding operation: for the sentence S _t = {w ₁ ,w ₂ ,…,w _n }, the sentence length embedding is

Among them, L indicates that the vector is the embedding of the sentence length, n represents the number of words, and D represents the embedding dimension.

Step3.2: Count the number of prepositions and perform embedding operations: For the sentence S _t = {w ₁ ,w ₂ ,…,w _n }, the number of prepositions is embedded

Among them, P represents that the vector is the embedding of the number of prepositions, * represents the specific number, and D represents the embedding dimension.

Step3.3: Make statistics on the average word length and perform embedding operation: For the sentence S _t = {w ₁ ,w ₂ ,…,w _n }, the number of prepositions is embedded

Among them, A represents the embedding of the vector as the average word length, * represents the specific number, and D represents the embedding dimension.

进行拼接作为句子的统计信息：Step3.4: will

Concatenate as sentence statistics:

in,

Described Step4 is specifically:

Step4.1: Concatenate the semantic feature X _[CLS] , grammatical feature O _POS , and statistical information feature O _STA into the fully connected layer, then input the prediction result of the sigmoid layer and output:

Step4.2: Calculate the loss:

Among them, y _ic represents the true category of sample i, which is 1 if it is equal to c, and 0 if it is not equal, and p _ic represents the predicted probability that observed sample i belongs to category c.

Step4.3: Use Adam to optimize the loss, the purpose is to minimize the loss, when the loss reaches the minimum, the model reaches the best effect.

In this embodiment, two English reading material data sets CEFR and Newsela with difficulty marks are selected, and a data set CEED manually constructed by the present invention is selected. Among them, CEFR and CEED are public graded English reading text datasets, and the Newsela dataset is a non-public graded English reading text dataset (you can apply for it on the Newsela website). The basic data statistics are carried out on the three data sets, and the statistical results are shown in Table 1. Among them, Num represents the number of texts contained in the data set, and Class represents the number of grade categories.

Table 1 Basic information of the dataset

(1) CEFR consists of 1493 English texts. These texts are labeled according to the level of difficulty of the Common European Framework of Reference (CEFR) A1, A2, B1, B2, C1, and C2, and the difficulty increases from A1 to C2. The English text in the dataset is taken from free online sources, including the British Council, ESLFast, and CNN Daily Mail datasets. The content of the English text includes dialogues, descriptions, short stories, newspaper stories and other articles.

(2) CEED is collected from 469 real reading questions in English exams such as senior high school entrance examination, college entrance examination, CET-4, CET-6, CET-4 and CET-8. Level 6 difficulty is marked as S, level 6 difficulty is marked as L, special level 4 difficulty is marked as E, and level 8 difficulty is marked as B. Difficulty increases sequentially from the senior high school entrance examination to the eighth junior high school entrance examination.

(3) Newsela is composed of 10,722 English texts. Difficulty is divided according to the American K12 education standard. The difficulty of each English text is marked with numbers from 2 to 12, and the difficulty increases from 2 to 12.

The present invention arranges the English texts in the data set, and the arrangement process is as follows: the first step is to read each English text by paragraph; the second step is to mark the difficulty level corresponding to each paragraph; the third step is to give each The paragraphs are marked with difficulty tags. The fourth step is to calculate the number of words, prepositions, and average word levels contained in each paragraph, and finally organize them into a csv file. The number of paragraphs contained in the sorted data sets is as follows: CEFR contains 12096 paragraphs, Newsela contains 227971 paragraphs, and CEED contains 3381 paragraphs.

In order to better obtain the difficulty coefficient in subsequent experiments, add corresponding difficulty labels to the extracted paragraphs. In the CEFR dataset, the difficulty labels of A1, A2, B1, and B2 are set to 0, and the difficulty labels of C1 and C2 are set to 1. In the Newsela dataset, the difficulty label with a level greater than or equal to 6 is set to 1, and the difficulty label with a level less than 6 is set to 0. In the CEED data set, because of the similarity in classification, the present invention divides the data set into three subsets, the data of the high school entrance examination and the college entrance examination are divided into a subset, referred to as CEED-EE; the data of the fourth and sixth levels are divided into one Subset, referred to as CEED-CET; the data of Specialty 4 and Specialty 8 are divided into a subset, referred to as CEED-TEM. Among them, the difficulty labels of the senior high school entrance examination, level 4, and junior high school are set to 0, and the difficulty labels of the college entrance examination, level 6, and junior high school are set to 1. The number of positive and negative samples contained in each data set after sorting is shown in Table 2.

Table 2: Number of positive and negative samples

The present invention selects the classic pre-training language models for Fill-mask tasks in recent years, such as Bert, Bart, xlnet, roberta, and xlm-roberta, for testing, and compares them with CNN, LSTM, and BiLSTM. In parameter setting, use pytorch 1.10 version, use NVIDIA GeForce RTX 2080Ti GPU. All pre-trained models are obtained from Huggingface. The selection of hyperparameters is as follows: Batchsize is {16, 32, 64}, learning rate is {1e-3, 1e-4, 1e-5}, and word embedding dimension is 768. Different models were tested on different data sets, and the experimental results are as follows:

Table 3: Experimental results of different models in CEFR and Newsela

It can be seen from Table 3 that in the two data sets, the method of the present invention (when using BERT as the pre-trained language model) is the best in the three indicators AUC, ACC, RMSE and the two data sets. In the CEFR data set, in terms of AUC, ACC and RMS, the method of the present invention is higher than the second place, with AUC increased by 5.81%, ACC increased by 7.02%, and RMSE decreased by 5.14%. In the Newsela data set, on the AUC, ACC, and RMS indicators, the method of the present invention is also higher than the second place, and the AUC is increased by 1.63%, the ACC is increased by 1.04%, and the RMSE is reduced by 1.15%. When the data set is small (CEFR data set), the pre-trained language model can perform better with less data.

Table 4: Results of different pretrained language models in CEFR and Newsela

As shown in Table 4, the present invention compares the effects of different pre-trained language models, which respectively improve and enhance BERT for different tasks. From the results, the BERT model can achieve the best results in the CEFR data set. In terms of AUC, ACC, and RMS indicators, the BERT model is higher than the second place, with an increase of 0.35% in AUC and 0.24% in ACC. RMSE decreased by 0.92%. The XLNet model can achieve the best results on the Newsela dataset. Compared with BERT, it improves AUC by 0.37%, ACC by 0.58%, and RMSE by 0.60%. However, the overall gap of these pre-trained models is not large, but the results are all better than CNN and LSTM.

Table 5: Experimental results of different models in CEED

It can be seen from Table 5 that in the two data sets, the method of the present invention (when using BERT as the pre-trained language model) is the best in the three indicators AUC, ACC, RMSE and three data sets. In the CEED-EE data set, the method of the present invention is higher than the second place in terms of AUC, ACC, and RMS indicators, with an increase of 8.20% in AUC, an increase of 4.71% in ACC, and a decrease of 7.05% in RMSE. In the CEED-CET data set, in terms of AUC, ACC, and RMS, the method of the present invention is also higher than the second place, with AUC increased by 5.32%, ACC increased by 3.77%, and RMSE decreased by 1.95%. On the CEED-TEM dataset, the AUC and ACC indicators have increased by 9.09% and 12.5% respectively compared with the second place, and the REMSE indicators have decreased by 8.51%.

Table 6: Results of different pretrained language models in CEED

As shown in Table 6, the present invention also compares the effects of different pre-trained language models in the CEED dataset. Overall, RoBERTa can achieve better results in the three subsets of CEED. Compared with BERT in the CEED-EE dataset, it improves AUC by 5.06%, ACC by 8.49%, and RMSE by 13.1%. Compared with BERT in the CEED-CET dataset, it improves AUC by 3.99%, ACC by 11.32%, and RMSE by 11.20%. Compared with BERT in the CEED-TEM dataset, it improves AUC by 3.17%, ACC by 3.12%, and RMSE by 4.35%. On the whole, these pre-trained language models are better than CNN and LSTM in three indicators.

The specific embodiments of the present invention have been described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above embodiments. Variations.

Claims (5)

1. A method for judging difficulty of English reading materials based on text feature fusion is characterized by comprising the following steps:

step1: firstly, encoding an input English text aiming at an English reading material data set, and inputting encoded information into a trained pre-training language model to obtain a feature vector containing semantic information;

step2: performing part-of-speech tagging on the text, and inputting the obtained part-of-speech sequence into an LSTM to obtain a feature vector containing grammatical information;

step3: extracting statistical information features; counting the factors influencing the difficulty of English reading materials, carrying out embedding expression on the factors, splicing all the characteristics, and inputting the spliced characteristics into a full-connection layer;

step4: and finally, obtaining a numerical value representing difficulty from 0 to 1 through sigmoid layer output, and finishing difficulty judgment.

2. The method for determining difficulty of reading English material based on text feature fusion as claimed in claim 1, wherein Step1 is specifically:

step1.1: assume that the currently input English text is S _t ，S _t In which n words, S _t ＝{w ₁ ，w ₂ ，...，w _i ，...，w _n In which w _i Represents the ith word;

the converted sentence is S _BERT ＝{[CLS]，w ₁ ，w ₂ ，...，[SEP]，...，w _n-2 ，w _n-1 ，w _n ，[SEP]}；

Step1.2: will S _BERT Is set to M, if S _t If the length of (D) is less than M, the length of S is measured _BERT Addition of [ PAD]Performing filling, S after filling operation _BERT Comprises the following steps:

S _BERT ＝{[CLS]，w ₁ ，w ₂ ，…，[SEP]，…，w _n-2 ，w _n-1 ，w _n ，[SEP]，…，[PAD]}

if S _t Is greater than M, truncating and discarding the subsequent content, truncating the operated S _BERT Comprises the following steps:

S _BERT ＝{[CLS]，w ₁ ，w ₂ ，…，[SEP]，…，w _M-2 ，w _M-1 ，w _M ，[SEP]}

step1.3: to S _BERT Is embedding coded, namely:

wherein,

D _BERT representing the embedding dimension set by the pre-training language model;

step1.4: to S _BERT The content in (1) is sentence position coded, namely:

S _{segmentembedding} ＝{E _A ，E _A ，E _A ，E _B ，E _B ，E _B ，E _B ，...，E _i ，E _i }

wherein, E _A Denotes a first sentence, E _B The second sentence is represented by the first sentence,

subsequent sentences analogized in the same way, E _i Represents the ith sentence;

step1.5: to S _BERT The content in (1) is subjected to word position coding, namely:

S _{position embedding} ＝{E ₁ ，E ₂ ，E ₃ ，…，E _i ，…，E _n-2 ，E _n-1 ，E _n ，…，E _M }

wherein E is _i A position code representing the ith word,

step1.6: will S _embedding 、S _{segmrntembedding} 、S _{positionembedding} Inputting the result into a pre-training language model to obtain a feature vector O output by the last layer _BERT Namely:

step1.7: selecting X _[CLS] As a sentence vector.

3. The method for determining difficulty of reading English material based on text feature fusion as claimed in claim 1, wherein Step2 is specifically:

step2.1: for the input text S _t ＝{w ₁ ，w ₂ ，w ₃ ，...，w _n Add [ CLS ] at the beginning of sentence]Indicating the beginning of a sentence, adding [ SEP ] in the middle of two sentences]For separating sentences, the converted sentences are:

S _sen ＝{[CLS]，w ₁ ，w ₂ ，…，[SEP]，…，w _n-2 ，w _n-1 ，w _n ，[SEP]，…，[PAD]}

step2.2: to S _sen And performing part-of-speech tagging to obtain:

S _POS ＝{[SPACE]，[PRP]，[VBP]，[NNP]，…，[RB]，[JJ]，[SPACE]，…，[PAD]}

wherein [ SPACE ] represents [ CLS ] and [ SEP ], [ PRP ] represents pronouns, [ VBP ] represents verbs, [ NNP ] represents nouns, [ RB ] represents degree adverbs, [ JJ ] represents adjectives;

step2.3: to S _POS Performing embedded representation to obtain E _POS Namely:

wherein D is _POS An embedding dimension representing a part of speech token;

step2.4: will E _pos Inputting LSTM, and taking the last layer of output result O _pos As feature vectors of sentence grammar information, wherein

4. The method for determining difficulty of reading English material based on text feature fusion as claimed in claim 1, wherein Step3 is specifically:

step3.1: and (3) counting the sentence length and carrying out embedding operation: for sentence S _t ＝{w ₁ ，w ₂ ，...，w _n H, then the sentence length is embedded as

Wherein, L represents the embedding of the vector as the sentence length, n represents the number of words, and D represents the embedding dimension;

step3.2: counting the preposition number and carrying out embedding operation: for sentence S _t ＝{w ₁ ，w ₂ ，...，w _n }, number of prepositions embedded

Wherein, P represents the embedding of the vector as preposition number, which represents specific number, D represents the embedding dimension;

step3.3: and (3) counting the average word length and carrying out embedding operation: for sentence S _t ＝{w ₁ ，w ₂ ，...，w _n }, number of prepositions embedded

Wherein, A represents the embedding of the vector with the average word length, which represents the specific number, and D represents the embedding dimension;

step3.4: will be provided with

Splicing is performed as statistical information of sentences:

wherein,

5. the method for determining difficulty of reading English material based on text feature fusion as claimed in claim 1, wherein Step4 is specifically:

step4.1: applying semantic features X _[CLS] Grammatical feature O _POS Statistical information characteristic O _STA Inputting the spliced data into a full-connection layer, inputting a sigmoid layer prediction result and outputting:

step4.2: calculating the loss:

wherein, y _iC Representing the true class of sample i, if c is equal to 1, and if not equal to 0,p _iC Representing the predicted probability that the observation sample i belongs to the class c;

step4.3: adam is used to optimize the loss in order to minimize the loss, and when the loss is minimized, the model achieves the best results.

Images