CN111782811A - An E-government Sensitive Text Detection Method Based on Convolutional Neural Network and Support Vector Machine - Google Patents

Abstract

The invention relates to an electronic government affair sensitive text detection method based on a convolutional neural network and a support vector machine. The invention mainly comprises (1) an electronic government affair sensitive text detection model based on a convolutional neural network and a support vector machine; (2) a sensitive domain text classification model based on TFIDF and a support vector machine; (3) a policy document recognition model based on word vectors and a convolutional neural network is provided.

Description

An E-government Sensitive Text Retrieval Based on Convolutional Neural Network and Support Vector Machine test method

technical field

The invention relates to the technical field of machine learning, and a method for detecting sensitive text of electronic government affairs based on a convolutional neural network and a support vector machine.

Background technique

With the rapid development of the Internet and computer technology, the application of network information technology in social life is becoming more and more extensive. Due to the characteristics of openness and sharing of the network, the Internet and computer technology not only improve the efficiency of government work, but also bring some content security issues, posing certain threats to the information security of government departments, and the sensitivity of government departments. Information and documents may be leaked to the Internet through e-government platforms. In particular, policy documents in sensitive fields such as religion, military, and politics mostly contain sensitive information. Once they are leaked and spread, it will cause huge losses to national security. Therefore, how to accurately and quickly detect the sensitive information of e-government leaked into the network, reduce the rate of false positives and false negatives, and keep national secrets becomes a big challenge.

To protect sensitive text from being leaked, it is first necessary to determine whether the text content contains sensitive information. At present, a large part of sensitive text detection work is based on human-made rules. However, with the increasing number and complexity of sensitive electronic text documents, the existing sensitive detection methods are difficult to achieve high efficiency. , convenient requirements. In order to timely and comprehensively discover the sensitive information leaked into the Internet portal, how to develop a more efficient sensitive detection technology solution is a problem that cannot be ignored. At present, there are mainly two detection technologies: one is a detection method based on keyword matching, and sensitive word matching is the key core of this method, which is generally implemented by a string matching algorithm. The detection method based on keyword matching ignores the correlation between the modified word and the original word, and the accuracy is low. With the development of machine learning technology, another detection technology is to use text classification in machine learning for sensitive text detection. The sensitive content detection method based on traditional machine learning has less sensitive texts that can be used for training, and the accuracy rate is low. lower.

Therefore, in order to solve the above problems, the present invention proposes an e-government sensitive text detection method based on convolutional neural network and support vector machine in combination with the characteristics of e-government sensitive texts (involving policies and guidelines in sensitive fields).

SUMMARY OF THE INVENTION

The present invention proposes an e-government sensitive text detection method based on convolutional neural network and support vector machine, which mainly includes three major contents:

1. Propose a text classification model for sensitive fields based on TFIDF and support vector machine;

2. Propose a policy document recognition model based on word vector and convolutional neural network;

3. Propose an e-government sensitive text detection model based on sensitive field text classification and policy document recognition.

The details are as follows:

1. Propose a text classification model for sensitive fields based on TFIDF and support vector machines

The present invention mainly adopts the TFIDF weighting technology to construct the text vector, adopts the support vector machine algorithm, and constructs the sensitive field text classification model through continuous machine learning training, and the model is used for judging whether the text belongs to the sensitive field.

(1) Using TFIDF weighting technique to convert the domain text dataset into text vector. For each text in the dataset, a vector is used to represent the semantics of the text, each dimension of the vector corresponds to a word, and its value is the TFIDF value of the word in the text. TFIDF is used to assess the importance of a word to one of the texts in a document set.

The process of calculating the weight using the TFIDF weighting technique is as follows

The calculation formula consists of two parts: word frequency (TF) and inverse document frequency (IDF). The formula is as follows:

w _ij =tf _ij *idf _ij (3)

where n _ij represents the number of occurrences of the i-th feature word in the j-th text; N _j represents the total number of words in the j-th text; N _i is the number of texts containing the i-th feature word, and N is the total number of texts ; w _ij is the TFIDF value of the i-th feature word.

Since training TFIDF on a large-scale corpus will get a lot of words, for the consideration of time and space efficiency, the present invention limits the selection of 500 feature words, and preferentially selects words with high word frequency, and after constructing the vector, X={x ₁ , x ₂ ,...,x _i }, where x ₁ -x _i represents the vector corresponding to the ith text in the text training set D.

(2) Using the support vector machine algorithm to train the text data set to obtain the text classification model in the sensitive field. The process is as follows:

Modeling: Given training samples T = {(v ₁ , y ₁ ), (v ₁ , y ₂ ), ..., (v _n , y _n )}, where v ₁ -v _n are n text vectors, y ₁ -y _m is the label value of the sensitive field corresponding to the training text, the value of the text label belonging to the sensitive field is 1, and the value of the text label belonging to the non-sensitive field is -1. We need to find a hyperplane to classify the instances in each training set into different categories. The hyperplane is wx+b=0, and the classification decision model is f(x)=sign(wx+b), where sign represents the sign function, w is the weight of the model and b is the bias. In order to obtain the maximum interval hyperplane that can completely separate the sample points in the training sample set, the following optimization constraints need to be solved:

st y _i (w*x _i +b)-1≥0,i=1,2,...,n (5)

Find the optimal w, b, and finally get the sensitive field classification decision model f(x).

Detection: After the modeling is completed, input the text vector to be detected, and the obtained output value is the classification label value of the text. +1 represents the positive class, indicating that the text belongs to the sensitive field, and -1 represents the negative class, indicating that the text does not belong to the sensitive field. field.

2. Propose a policy document recognition model based on word vector and convolutional neural network

The present invention adopts the Word2vec technology to perform word vector training on the word sequence after the text segmentation to obtain the word vector corresponding to each word, which is used as the input data of the convolutional neural network to construct a policy document recognition model based on the convolutional neural network. The model is used for To determine whether the text is a policy document, the model mainly includes input layer, convolution layer, pooling layer, fully connected layer, etc.

(1) The first layer is the input layer. The input layer is an n*m matrix, denoted by the letter A. Wherein n is the number of words in a text word sequence, the present invention adopts padding technology to keep the length of all text word sequences consistent. m is the dimension of the word vector corresponding to each word. The present invention uses the Word2vec technology to perform word vector training, and maps each word into an m-dimensional word vector.

(2) The second layer is a convolutional layer. The convolution operation is performed by using convolution kernels of different sizes. The width of the convolution kernel is equal to the dimension m of the word vector, and the height is h. Suppose a convolution kernel is a matrix t of h*m. The convolution kernel slides down with a step size of 1, and the convolution operation is performed every time it passes through a window of size h*m to generate a new feature value c _i , a convolution kernel is processed to obtain a feature map c, a total of n -h+1 features, the calculation formula is as follows:

c _i =f(t*A[i:i+h-1]+b),i=1,2,...,n+h-1 (6)

where b is the bias term and f is the activation function.

(3) The third layer is the pooling layer. Since the size of feature maps obtained by convolution kernels of different sizes is different, the present invention uses the pooling function 1-max-pooling to extract the features of each feature map to keep their dimensions consistent. The principle of 1-max-pooling is to take a maximum value from multiple values.

(4) The fourth layer is a fully connected layer. The fully connected layer is used for classification, and the features extracted by the convolution and pooling layers are input into the softmax function for classification and training, and the policy document recognition model is obtained.

3. Propose an e-government sensitive text detection model based on sensitive field text classification and policy document recognition

Since the policy documents in the sensitive fields of e-government mostly involve sensitive content, in order to judge whether a text is a sensitive text, it is necessary to detect whether the text is a sensitive field and whether the text is a policy document. The sensitive field text classification model is used to detect whether the text belongs to the sensitive field, and then the policy document recognition model is used for the text belonging to the sensitive field to judge whether the text belongs to the policy document.

(1) Text classification in sensitive fields. Firstly, the text vector of the text to be detected is constructed, and then the support vector machine algorithm is used to establish a classification model of sensitive fields to calculate the classification results of the text. The input of the model is the text to be detected, the output is the classification result of the sensitive field of the text, and it is judged whether the text belongs to the sensitive field.

(2) Identification of policy documents. Firstly, the word vector is constructed based on the word2vec technology, and then the convolutional neural network is used to establish a model to calculate the recognition result of the policy document of the text. The input of the model is the text to be detected, the output is the recognition result of the policy document of the text, and it is judged whether the text belongs to the policy document.

Finally, the models in the above steps are integrated to obtain an e-government sensitive text detection model based on sensitive field text classification and policy document recognition.

Description of drawings

Fig. 1 is the working flow chart of the present invention.

Detailed ways

The invention is an electronic government affairs sensitive text detection method based on a convolutional neural network and a support vector machine. It mainly includes the following steps:

Step 1: Preprocess the text data. Firstly, the data set prepared by the present invention is cleaned to remove useless parts in the text, and then the Chinese word segmentation technology is used to segment the text to obtain the segmented text word sequence.

Step 2: Establish a text classification model for sensitive fields. Use TFIDF technology to calculate the weight of the words in the text, and convert the text word sequence into the corresponding text vector; use the support vector machine algorithm to establish a domain classification model, and the input content is the text vector and the corresponding classification label in the sensitive and non-sensitive areas. Continuous training makes the model get better classification results.

Step 3: Establish a policy document recognition model. First, the Word2vec model is used to train the word vector for the text, each word is represented by a word vector, and each text is converted into a corresponding matrix as the input data of the convolutional neural network; then, the convolution check of different sizes is used. The input matrix is convolved to obtain multiple feature maps; then, the pooling function 1-max-pooling is used to extract the features of each feature map, and the maximum feature value is output. Finally, the extracted features are input into the softmax function for classification, and the result of the policy document recognition of the text is obtained.

Step 4: Detection. First, the text to be detected is converted into a text vector, and the sensitive field text classification model established in step 1 is used to detect whether the text belongs to a sensitive field; then, for the text belonging to the sensitive field, the policy document recognition model established in step 2 is used to determine Whether the text is a policy document. Finally, it is detected that most of the policy documents in the sensitive fields of e-government are sensitive texts.

Claims (4)

1. A method for detecting E-government affair sensitive texts based on a convolutional neural network and a support vector machine is characterized by comprising the following steps:

(1) providing a sensitive field text classification model based on a TFIDF and a support vector machine;

(2) providing a policy document identification model based on word vectors and a convolutional neural network;

(3) an electronic government affair sensitive text detection model based on sensitive field text classification and policy document identification is provided.

2. The TFIDF and support vector machine based domain text classification model of claim 1, wherein: and calculating the weights of words in the sensitive field texts and the non-sensitive field texts by adopting a TFIDF technology, and constructing two types of text vectors. And adopting a support vector machine algorithm, taking the two types of text vectors and the classification labels thereof as input and output, and performing iterative training to obtain a final convergent sensitive field text classification model.

3. The word vector and convolutional neural network based policy document identification model of claim 1, wherein: vectorizing and expressing each keyword in the text of the policy official document and the text of the non-policy official document by adopting a word vector algorithm, and obtaining a word vector matrix of the text according to a word sequence to be used as the input of a convolutional neural network; carrying out convolution calculation on the input word vector matrixes by adopting convolution kernels with different sizes to obtain a plurality of characteristic graphs; and (3) reducing the dimension of the features of each feature map by using a pooling function, and finally inputting the features into a softmax classifier layer for classification training to obtain a policy document identification model.

4. The sensitive text detection model based on religious domain text classification and policy document identification according to claim 1, characterized in that: since policy documents in sensitive fields of e-government (e.g., sensitive fields such as religion, military, politics, etc.) mostly contain sensitive contents, in order to determine whether a text belongs to a sensitive text, it is necessary to detect whether the text is a sensitive field and whether the text is a policy document. And detecting whether the text belongs to the sensitive field by adopting the sensitive field text classification model, and judging whether the text belongs to the policy document by adopting a policy document identification model for the text belonging to the sensitive field.

Images