CN113553844A

CN113553844A - Domain identification method based on prefix tree features and convolutional neural network

Info

Publication number: CN113553844A
Application number: CN202110917561.8A
Authority: CN
Inventors: 刘光毅
Original assignee: Sichuan Changhong Electric Co Ltd
Current assignee: Sichuan Changhong Electric Co Ltd
Priority date: 2021-08-11
Filing date: 2021-08-11
Publication date: 2021-10-26
Anticipated expiration: 2041-08-11
Also published as: CN113553844B

Abstract

The invention relates to the field of natural language processing, and discloses a field identification method based on prefix tree features and a convolutional neural network, which is used for solving the problems that the accuracy rate of feature acquisition of a current model is not high, and the identification accuracy rate in the technical field is not high. The method comprises the steps of obtaining nouns of obvious domain characteristics in each category from user requests, training data and test data, storing the nouns as characteristic texts by taking the categories as names, generating prefix trees by taking the characteristic texts, inputting the request texts, then calculating to obtain weight matrixes of each domain as prefix tree characteristics, combining the prefix tree acquisition characteristics with a convolution characteristic diagram, inputting the prefix tree acquisition characteristics into a convolution neural network for further calculation, making up the situation that the convolution neural network possibly omits the characteristics or pays attention to wrong characteristics, enabling the judgment of the model on the input characteristics to be more accurate, and improving the accuracy of model prediction. The invention is suitable for field recognition.

Description

Domain identification method based on prefix tree features and convolutional neural network

Technical Field

The invention relates to the field of natural language processing, in particular to a field identification method based on prefix tree features and a convolutional neural network.

Background

The intention recognition is a direction in natural language processing, and common methods are: dictionary template based rule classification, past log matching (applicable to search engines), and classification model based intent recognition. These three methods are basically the mainstream methods at present. The dictionary-based template rule matching has limited universality, and when the request text changes (the language of the user request cannot be predicted), the situation of recognition errors is easy to occur. The log matching based approach is not applicable to voice interactive systems on television. The method difficulty of the classification model is mainly two points, one point is the lack of data sources, and because the method is relatively fixed and basically has supervised learning, a lot of marking data are needed. The second point is that it is difficult to identify classes despite their classification efforts, and the accuracy and scalability required is not comparable to previous classes.

At present, a text convolutional neural network (textCNN) model specially aiming at text classification is available, the classification performance on general Chinese texts is good, but the problems of low accuracy of feature acquisition and low recognition accuracy in the technical field still exist. Because the convolutional neural network model needs a training set with the data amount in each domain balanced as much as possible in the training process, however, firstly, the actual real user data needs to be divided into 48 domains, wherein the data in the television common domains such as VIDEO, TV, MUSIC, etc. is more, and each domain has a part of similar characteristics (such as "one-key viewing mode" of TV and "viewing mode" of smart home "); secondly, when data is constructed, a certain work may be used for constructing the data for multiple times (such as 'western grand tale'), the name of the work may be judged as a characteristic by a model, and a prediction result is influenced (for example, when 'western grand tale' generally exists in VIDEO data and 'song in western grand tale' is requested, VIDEO classification is easily predicted, and actual expectation is MUSIC); third, the convolutional neural network may miss part of the key feature information when acquiring features.

Disclosure of Invention

The technical problem to be solved by the invention is as follows: a field identification method based on prefix tree features and a convolutional neural network is provided to solve the problems that the accuracy rate of feature acquisition of a current model is not high, and the identification accuracy rate of the technical field is not high.

In order to solve the problems, the invention adopts the technical scheme that: performing characteristic analysis on the request text by using a prefix tree constructed in advance to obtain corresponding prefix tree characteristics; when the convolutional neural network carries out the field prediction on the request text, the convolutional features and the prefix tree features are spliced, and then the spliced features are input into a full-connection layer of the convolutional neural network to obtain a corresponding prediction result.

Further, the steps of constructing the prefix tree of the present invention include:

according to the tf-idf statistical method, high-frequency keywords in each field are obtained from the existing data, and after the high-frequency keywords are screened, the screened vocabulary is used as basic data of the component prefix tree; and taking each word of the high-frequency domain keywords in the basic data as a node of the tree, and generating a prefix tree of a father node, a child node and a grandchild node according to a normal reading sequence.

Further, the existing data includes user log data, training data, and test data.

Further, the present invention utilizes the prefix tree to perform feature analysis on the request text, and the specific steps of obtaining the prefix tree features include:

after the request text is completely participled, query matching is carried out in the prefix tree, the ratio of the character length of the matched keyword to the character length of the text is used as the weight of the field to which the request text belongs, the keyword is normalized to be between 0 and 1, and then the normalized keyword is converted into a matrix which is used as a prefix tree feature matrix.

The invention has the following beneficial effects: the invention utilizes the prefix tree model to correct the characteristic value obtained after the convolution of the convolutional neural network (textCNN), makes up the situation that the convolutional neural network may miss the characteristics or pay attention to the wrong characteristics, makes the judgment of the model on the input characteristics more accurate, and improves the accuracy of the model prediction.

Drawings

FIG. 1 is a flow chart of domain identification based on prefix tree features and convolutional neural networks.

Detailed Description

The invention provides a field recognition method based on prefix tree features and a convolutional neural network (textCNN), aiming at the problems that the accuracy rate of feature acquisition of a current model is not high and the recognition accuracy rate of the prior art is not high, the method comprises the steps of acquiring nouns of obvious field features in each classification from a user request, training data and test data, storing the nouns as feature texts by taking the classes as names, generating a prefix tree by taking the feature texts, inputting a weight matrix of each field calculated after the request texts are input as the prefix tree features, combining the prefix tree acquisition features with a convolutional feature map, inputting the convolutional neural network for further calculation, making up the situation that the convolutional neural network possibly omits the features or pays attention to wrong features, enabling the judgment of the model on the input features to be more accurate, and improving the accuracy of model prediction.

In order to illustrate the principles of the present invention in detail, the following sub-steps illustrate the inventive arrangements. The method comprises the following implementation steps:

1. and analyzing the requests of each field, and extracting keywords of each field through a tf-idf method to serve as feature texts.

The method mainly obtains and analyzes the request text of each field from the user request, the training data and the test data, obtains the key words of each field by a tf-idf method, extracts the nouns of the obvious field characteristics in each classification by manual screening, stores the classes as the characteristic text, and generates the data of the prefix tree.

2. And constructing a prefix tree based on the feature text.

In the prefix tree (the prefix tree is also called as a Trie tree or a Trie, a variant of a Hash tree) constructed in the step, a father node of the tree is a characteristic word of each field, a child node is a field where the characteristic word is located, such as 'I' think-listen-MUSIC ', the' I 'think-listen' is a keyword in the field of 'MUSIC', the keyword is split into single words, the 'I' is used as a father node, the 'thought' is a child node of the child node, and the final child node is sequentially deduced to be the 'MUSIC' in the field to which the keyword belongs.

3. And analyzing the characteristics of the request text and outputting a characteristic vector.

And performing full segmentation processing on the request text to obtain a full segmentation text, inputting the full segmentation text into a prefix tree, if no field exists, setting the feature vector as a 0 vector (without influencing the original feature), if the field exists, outputting, normalizing the keyword text to a matrix after normalizing the keyword text to be the 0-1 according to the ratio of the character length of the matched keyword to the character length of the text as the weight of the field to which the keyword belongs, and taking the normalized keyword text as the prefix tree feature matrix.

4. And splicing and fusing the prefix tree feature vector and the feature vector after convolution.

Fusing the characteristic matrix obtained in the step 2 with the characteristic matrix (the characteristic diagram obtained after the convolution of the original convolution neural network) after the convolution pooling layer to obtain a new characteristic matrix, wherein the matrix comprises the characteristics obtained by the prefix tree and can be used for correcting the characteristic value obtained by the model to enable the model to pay more attention to certain important parts

5. And inputting the fused feature vector into a full-connection layer, and obtaining a prediction result through softmax.

Examples

Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings and examples. As shown in fig. 1, the flow of the prefix tree algorithm and the overall algorithm in the embodiment is as follows:

s1, generating prefix tree data:

according to the tf-idf statistical method, the domain high-frequency keywords are obtained from the existing user log data, training data and test data, and the vocabulary is used as the data of the prefix tree after manual screening.

S2, constructing a prefix tree, and acquiring text characteristics:

after the characteristic words (namely the domain high-frequency keywords) are fully segmented, father nodes, child nodes, grandchild nodes and the like of the tree are sequentially formed according to the reading sequence, and the final child node is the domain to which the characteristic words belong. Meanwhile, the request text is fully segmented into words, such as "I want to see the movie", and is divided into [ "I", "want", "see", "electric", "movie"]And sequentially searching in the Trie to obtain the weight scores of all the fields: w ═ ω₁,ω₂,ω₃…ω_nN is the number of fields,

is a score of a certain domain weight, l represents a request messageLength of the character,/, of_iIndicating the length of the character matched to the keyword.

After the weight score is obtained, the weight score is normalized to represent the probability ratio of the request in a certain domain.

Converting the probability ratio into 1 × 48 vector, repeating for 15 times to obtain a 15 × 48 probability matrix, and then normalizing the probability matrix to obtain a prefix tree feature matrix:

s3, convolution feature fusion:

the request text is converted into vector representation of 15 × 271, the convolutional neural network performs convolution operation on each possible window of a sentence word to obtain a feature map, two feature vectors obtained through one-dimensional convolution and two-dimensional convolution respectively pass through a pooling layer to obtain two feature vectors, the two feature vectors are spliced to obtain a new feature vector, and meanwhile the Trie feature vector obtained in the last step is subjected to dimensionality processing and then spliced to obtain a final text feature vector.

S4, predicting a result:

and (5) inputting the feature vector obtained in the step (S3) into a full connection layer of the convolutional neural network, and obtaining a final prediction result through operations such as softmax and the like.

The embodiment proves that under the condition of the same training data, the characteristics are corrected through the prefix tree, the similar characteristics among different fields can be distinguished, and under the condition of not increasing the training data, the distinguishing of the fields with the similar characteristics is improved. The specific experimental results are as follows (experiments under the same training set and validation set):

1) the optimized convolutional neural network algorithm has lower loss, the loss before the feature fusion is 0.232, and the loss after the feature fusion is 0.161 when the training is finished;

2) the optimized convolutional neural network algorithm trains the finished model, and the whole performance (including accuracy rate, recall rate and F1 value) on the same verification set is better.

Claims

1. A field identification method based on prefix tree characteristics and a convolutional neural network is characterized in that a prefix tree constructed in advance is used for carrying out characteristic analysis on a request text to obtain corresponding prefix tree characteristics; when the convolutional neural network carries out the field prediction on the request text, after the convolutional characteristic and the prefix tree characteristic are spliced, the spliced characteristic is input to a full-connection layer of the convolutional neural network to obtain a corresponding prediction result.

2. The field identification method based on the prefix tree feature and the convolutional neural network as claimed in claim 1, wherein the step of constructing the prefix tree comprises:

according to the tf-idf statistical method, obtaining high-frequency domain keywords from the existing data, and after screening, using the screened vocabulary as the data of the prefix tree; and taking each word of the domain high-frequency keyword as a node of the tree, and generating a prefix tree of a father node, a child node and a grandchild node according to a normal reading sequence.

3. The field recognition method based on prefix tree features and convolutional neural network of claim 2, wherein the existing data comprises user log data, training data, and test data.

4. The field identification method based on the prefix tree feature and the convolutional neural network as claimed in claim 2, wherein the step of performing feature analysis on the request text using the prefix tree to obtain the prefix tree feature comprises: