WO2021174922A1 - Statement sentiment classification method and related device - Google Patents

Abstract

The present application provides a statement sentiment classification method and a related device. Said method comprises: using a feature extraction model to convert words before a missing word in a first statement sample into a first coding sequence according to a word order, converting words after the missing word in the first statement sample into a second coding sequence according to a reverse word order, and converting the missing word in the first statement sample into a tag vector of the first statement sample; using the feature extraction model to calculate a missing word vector of the first statement sample according to the first coding sequence and the second coding sequence; training the feature extraction model according to the missing word vector of the first statement sample and the tag vector of the first statement sample; and using an attribute classification model formed by the feature extraction model to identify attribute words of statements to be processed, and using a sentiment classification model formed by the feature extraction model to classify said statements which connect the attribute words. The present application enhances the accuracy and scene adaptability of sentiment classification. The present application also relates to a blockchain.

Description

Sentence emotion classification method and related equipment

This application claims to be submitted to the Chinese Patent Office on March 2, 2020. The application number is 202010137265.1. The application titled "Sentence Emotion Classification Method and Related Equipment" is the priority of the Chinese patent application, the entire content of which is incorporated into this application by reference.

Technical field

This application relates to the field of natural language processing, and specifically relates to a sentence emotion classification method, device, computer equipment, and computer storage medium.

Background technique

Generally, in the field of natural language processing of artificial intelligence, sentiment classification models (such as convolutional neural networks) are trained using sentences with sentiment labels in the designated domain, and sentences in the designated domain are classified with the trained sentiment classification model. The inventor realizes that the existing text sentiment classification method is only suitable for sentence sentiment classification tasks in a fixed field, and a larger training set is required to improve the accuracy of sentiment classification.

How to improve the scene adaptability of text emotion classification and the accuracy of emotion classification has become a problem to be solved at present.

Summary of the invention

In view of the above, it is necessary to propose a sentence emotion classification method, device, computer equipment, and computer storage medium, which can classify the emotion of the sentence and enhance the accuracy and scene adaptability of the emotion classification.

The first aspect of the present application provides a sentence sentiment classification method, and the sentence sentiment classification method includes:

Obtain a first sentence sample set. Each first sentence sample in the first sentence sample set contains a missing word; for each first sentence sample, the feature extraction model is used to calculate the first sentence sample before the missing word Words are converted into a first word vector sequence according to word order, words after the missing word in the first sentence sample are converted into a second word vector sequence according to the reverse word order, and the first sentence is converted according to a preset vocabulary coding table The missing words in the sample are transformed into the label vector of the first sentence sample; the feature extraction model is used to encode the first word vector sequence into a first encoding sequence, and the second word vector sequence is encoded as Second coding sequence; using the feature extraction model to calculate the missing word vector of the first sentence sample according to the first coding sequence and the second coding sequence; according to the missing word vector of the first sentence sample and the missing word vector The label vector of the first sentence sample trains the feature extraction model to obtain a first feature extraction model, and a new second feature extraction model is created so that the neural network structure of the second feature extraction model is the same as that of the first feature extraction model. The neural network structure is consistent, the weight of the first feature extraction model is used to update the weight of the second feature extraction model; the second sentence sample with attribute tags is used to train the first feature extraction model and the full connection An attribute classification model composed of layers; the attribute classification model is used to identify the attribute words of a plurality of sentences to be recognized, and each sentence to be recognized is connected with the attribute words of each sentence to be recognized to obtain the connection attribute words A plurality of sentences to be recognized; using the plurality of sentences to be recognized connected with attribute words with emotion labels to train an emotion classification model composed of the second feature extraction model and a deep learning model; and using the attribute classification model to recognize Processing the attribute words of the sentence, the emotion classification model classifies the sentence to be processed connecting the attribute words, and outputs the attribute word of the sentence to be processed and the emotion type of the sentence to be processed.

A second aspect of the present application provides a sentence emotion classification device, the device includes:

The obtaining module is used to obtain a first sentence sample set, and each first sentence sample in the first sentence sample set contains a missing word; the conversion module is used to obtain a feature extraction model for each first sentence sample. The words before the missing word in the first sentence sample are converted into the first word vector sequence in the word order, and the words after the missing word in the first sentence sample are converted into the second word vector sequence according to the reverse word order. Suppose that a vocabulary encoding table converts the missing words in the first sentence sample into a label vector of the first sentence sample; an encoding module is used to encode the first word vector sequence into The first coding sequence is used to code the second word vector sequence into a second coding sequence; the calculation module is used to use the feature extraction model to calculate the first coding sequence according to the first coding sequence and the second coding sequence. The missing word vector of the sentence sample; the first training module is used to train the feature extraction model according to the missing word vector of the first sentence sample and the label vector of the first sentence sample to obtain the first feature extraction model, and create a new The second feature extraction model makes the neural network structure of the second feature extraction model consistent with the neural network structure of the first feature extraction model, and updates the second feature extraction with the weight of the first feature extraction model The weight of the model; the second training module is used to train the attribute classification model composed of the first feature extraction model and the fully connected layer with second sentence samples with attribute labels; the connection module is used to use the attributes The classification model recognizes the attribute words of a plurality of sentences to be recognized, and connects each sentence to be recognized with the attribute words of each sentence to be recognized recognized to obtain the plurality of sentences to be recognized that connect the attribute words;

The third training module is used to train the emotion classification model composed of the second feature extraction model and the deep learning model with the plurality of sentences to be recognized connected with attribute words with emotion labels; the classification module is used to use all The attribute classification model identifies the attribute words of the sentence to be processed, and the emotion classification model classifies the sentence to be processed connecting the attribute words, and outputs the attribute words of the sentence to be processed and the emotion type of the sentence to be processed.

A third aspect of the present application provides a computer device, the computer device includes a processor, and the processor is configured to execute the following steps when executing computer-readable instructions stored in a memory:

Obtain a first sentence sample set. Each first sentence sample in the first sentence sample set contains a missing word; for each first sentence sample, the feature extraction model is used to calculate the first sentence sample before the missing word Words are converted into a first word vector sequence according to word order, words after the missing word in the first sentence sample are converted into a second word vector sequence according to the reverse word order, and the first sentence is converted according to a preset vocabulary coding table The missing words in the sample are transformed into the label vector of the first sentence sample; the feature extraction model is used to encode the first word vector sequence into a first encoding sequence, and the second word vector sequence is encoded as Second coding sequence; using the feature extraction model to calculate the missing word vector of the first sentence sample according to the first coding sequence and the second coding sequence; according to the missing word vector of the first sentence sample and the missing word vector The label vector of the first sentence sample trains the feature extraction model to obtain a first feature extraction model, and a new second feature extraction model is created so that the neural network structure of the second feature extraction model is the same as that of the first feature extraction model. The neural network structure is consistent, the weight of the first feature extraction model is used to update the weight of the second feature extraction model; the second sentence sample with attribute tags is used to train the first feature extraction model and the full connection An attribute classification model composed of layers; the attribute classification model is used to identify the attribute words of a plurality of sentences to be recognized, and each sentence to be recognized is connected with the attribute words of each sentence to be recognized to obtain the connection attribute words A plurality of sentences to be recognized; using the plurality of sentences to be recognized connected with attribute words with emotion labels to train an emotion classification model composed of the second feature extraction model and a deep learning model; and using the attribute classification model to recognize Processing the attribute words of the sentence, the emotion classification model classifies the sentence to be processed connecting the attribute words, and outputs the attribute word of the sentence to be processed and the emotion type of the sentence to be processed.

The fourth aspect of the present application provides a computer storage medium having computer-readable instructions stored thereon, and the computer-readable instructions implement the following steps when executed by a processor:

Obtain a first sentence sample set. Each first sentence sample in the first sentence sample set contains a missing word; for each first sentence sample, the feature extraction model is used to calculate the first sentence sample before the missing word Words are converted into a first word vector sequence according to word order, words after the missing word in the first sentence sample are converted into a second word vector sequence according to the reverse word order, and the first sentence is converted according to a preset vocabulary coding table The missing words in the sample are transformed into the label vector of the first sentence sample; the feature extraction model is used to encode the first word vector sequence into a first encoding sequence, and the second word vector sequence is encoded as Second coding sequence; using the feature extraction model to calculate the missing word vector of the first sentence sample according to the first coding sequence and the second coding sequence; according to the missing word vector of the first sentence sample and the missing word vector The label vector of the first sentence sample trains the feature extraction model to obtain a first feature extraction model, and a new second feature extraction model is created so that the neural network structure of the second feature extraction model is the same as that of the first feature extraction model. The neural network structure is consistent, the weight of the first feature extraction model is used to update the weight of the second feature extraction model; the second sentence sample with attribute tags is used to train the first feature extraction model and the full connection An attribute classification model composed of layers; the attribute classification model is used to identify the attribute words of a plurality of sentences to be recognized, and each sentence to be recognized is connected with the attribute words of each sentence to be recognized to obtain the connection attribute words A plurality of sentences to be recognized; using the plurality of sentences to be recognized connected with attribute words with emotion labels to train an emotion classification model composed of the second feature extraction model and a deep learning model; and using the attribute classification model to recognize Processing the attribute words of the sentence, the emotion classification model classifies the sentence to be processed connecting the attribute words, and outputs the attribute word of the sentence to be processed and the emotion type of the sentence to be processed.

This application performs emotion classification on sentences to enhance the accuracy and scene adaptability of emotion classification.

Description of the drawings

Fig. 1 is a flowchart of a sentence sentiment classification method provided by an embodiment of the present application.

Fig. 2 is a structural diagram of a sentence sentiment classification device provided by an embodiment of the present application.

Fig. 3 is a schematic diagram of a computer device provided by an embodiment of the present application.

Detailed ways

In order to be able to understand the above objectives, features and advantages of the application more clearly, the application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments of the application and the features in the embodiments can be combined with each other if there is no conflict.

Preferably, the sentence emotion classification method of the present application is applied to one or more computer devices. The computer device is a device that can automatically perform numerical calculation and/or information processing in accordance with pre-set or stored instructions. Its hardware includes, but is not limited to, a microprocessor and an application specific integrated circuit (ASIC) , Programmable Gate Array (Field-Programmable Gate Array, FPGA), Digital Processor (Digital Signal Processor, DSP), embedded equipment, etc.

The computer device may be a computing device such as a desktop computer, a notebook, a palmtop computer, and a cloud server. The computer device can interact with the user through a keyboard, a mouse, a remote control, a touch panel, or a voice control device.

Example one

Fig. 1 is a flowchart of a sentence sentiment classification method provided in Embodiment 1 of the present application. The sentence emotion classification method is applied to computer equipment.

The sentence sentiment classification method of the present application can perform sentiment classification on the sentence.

As shown in Figure 1, the sentence emotion classification method includes:

101. Obtain a first sentence sample set, where each first sentence sample in the first sentence sample set contains a missing word.

Multiple texts in different fields can be obtained, each text includes multiple sentences, each text is occluded multiple times, part of the words in the text is occluded each time, and a missing part is extracted from the text of each occluded part of the words The sentence of words is used as the first sentence sample.

Multiple texts in various fields such as tourism, electronic products, and patent services can be obtained. Each field includes multiple texts, and each text in each field can include multiple sentences. This embodiment does not limit the size of the field, such as the field of electronic products and the field of notebook computers, and the field of electronic products may include the field of notebook computers.

Each text in multiple texts in each field can be occluded multiple times, and a preset proportion of words in each text can be randomly occluded each time to obtain the first sentence with missing words in multiple texts in each field sample.

102. For each first sentence sample, use a feature extraction model to convert words before the missing word in the first sentence sample into a first word vector sequence in order of words, and convert the words after the missing word in the first sentence sample The words of are converted into a second word vector sequence according to the reverse word order, and the missing words in the first sentence sample are converted into the label vector of the first sentence sample according to a preset vocabulary coding table.

The feature extraction model includes the input layer, the forward hidden layer, the backward hidden layer, and the output layer.

In a specific embodiment, the use of the feature extraction model converts the words before the missing word in the first sentence sample into a first word vector sequence in order of words, and converts the words after the missing word in the first sentence sample The conversion of words into the second word vector sequence according to the reverse word order includes:

Convert the words before the missing word in the first sentence sample into a first coding vector sequence in word order, and convert the words after the missing word in the first sentence sample into a second coding vector in word order Sequence; convert the position number of the word before the missing word in the first sentence sample into a first position vector sequence, and convert the position number of the word after the missing word in the first sentence sample into A second position vector sequence; converting the first coding vector sequence and the first position vector sequence into a first word vector sequence, and converting the second coding vector sequence and the second position vector sequence into a second Sequence of word vectors.

For example, a first sentence sample is "<S>from <mask>Language Processing<E>", where "<S>" means the head word of the first sentence sample, and "<E>" means the first sentence sample The tail word, according to the preset vocabulary coding table, convert the word "<S>自" before the missing word "ran" into the first coding vector sequence {(0,0,0,0,1,0,0 in word order) , 0), (0, 0, 0, 0, 0, 0, 0, 1)}, the word "language processing <E>" after the missing word "ran" is transformed into the second coding vector sequence {( 0, 0, 0, 0, 0, 0, 1, 0), (0, 0, 0, 0, 0, 1, 0, 0), (1, 0, 0, 0, 0, 0, 0, 0), (0, 0, 1, 0, 0, 0, 0, 0), (0, 1, 0, 0, 0, 0, 0, 0)}, the preset vocabulary encoding table may adopt one -Encoding methods such as hot and word2vec. Convert the position number of the word before the missing word "ran" in the first sentence sample into the first position vector sequence {(1, 0, 0, 0, 0, 0, 0, 0), (0, 1, 0) ,0,0,0,0,0)}, convert the position number of the word after the missing word in the first sentence sample into a second position vector sequence {(0,0,0,1,0,0,0 , 0), (0, 0, 0, 0, 1, 0, 0, 0), (0, 0, 0, 0, 0, 1, 0, 0), (0, 0, 0, 0, 0 , 0, 1, 0), (0, 0, 0, 0, 0, 0, 0, 1)}. Add the first code vector in the first code vector sequence corresponding to each word before the missing word "ran" and the first position vector in the first position vector sequence to obtain the first word vector sequence {(1, 0 , 0, 0, 1, 0, 0, 0), (0, 1, 0, 0, 0, 0, 0, 1)}. Add the second code vector in the second code vector sequence corresponding to each word after the missing word "ran" and the second position vector in the second position vector sequence to obtain the second word vector sequence {(0, 0 , 0, 1, 0, 0, 1, 0), (0, 0, 0, 0, 1, 1, 0, 0), (1, 0, 0, 0, 0, 1, 0, 0), (0, 0, 1, 0, 0, 0, 1, 0), (0, 1, 0, 0, 0, 0, 0, 1)}.

According to the preset vocabulary encoding table, the missing word <mask> in the first sentence sample is converted into the label vector (0, 0, 0, 1, 0, 0, 0, 0) of the first sentence sample, namely "Ran" one-hot encoding.

103. Use the feature extraction model to encode the first word vector sequence as a first encoding sequence, and encode the second word vector sequence as a second encoding sequence.

In this embodiment, the forward hidden layer of the feature extraction model encodes the first word vector sequence into the first coding sequence, and the backward hidden layer of the feature extraction model encodes the second word vector sequence The sequence code is the second coding sequence. The forward hidden layer and the backward hidden layer respectively include N forward hidden sublayers and N backward hidden sublayers, each forward hidden sublayer includes U encoding modules, and each backward hidden sublayer The layer includes W encoding modules; wherein the u-th encoding module of the n-th forward hidden sublayer receives the vector Z _{n-1 output by the u-1th encoding module of the n-1th forward hidden sublayer, u-1} and the vector Z of the n 1-layer of the front output to hidden sublayer u-th encoding module _{n-1, u,} n-th layer before Z _n to the hidden sublayer u-th encoding module output _{vector, u} To the u-th coding module of the forward hidden sublayer of the n+1th layer and the u+1th coding module of the forward hidden sublayer of the n+1th layer, 2≤n≤N, 2≤u≤U. The u-th encoding module of the first layer forward hidden sublayer receives the u-1th word vector of the first word vector sequence and the u-th word vector of the first word vector sequence, and the Nth layer forward The hidden sub-layer is the first coding sequence. The first encoding module of the nth forward hidden sublayer receives the vector Z _n-1,1 output by the first encoding module of the n-1th forward hidden sublayer. The first encoding module of the nth forward hidden sublayer The first encoding module outputs the vector Z _n,1 to the first encoding module of the forward hidden sub-layer of the n+1th layer. The w-th encoding module of the nth layer of backward hidden sublayer receives the vector R _{n-1, w-1} and n-1 of the output of the w-1th encoding module of the n-1th layer of backward hidden sublayer _{The vector R n-1,w} output by the w-th encoding module of the backward hidden sub-layer, the output vector Z _n,w of the w-th encoding module of the n-th backward hidden sub-layer to the n+1-th layer is backward hidden The w-th coding module of the sub-layer and the w+1-th coding module of the n+1-th backward hidden sub-layer, 2≤w≤W. The w-th encoding module of the first layer of backward hidden sublayer receives the w-1th word vector of the second word vector sequence and the wth word vector of the second word vector sequence, and the Nth layer is backward The hidden sublayer is the second coding sequence. The first coding module of the nth layer of backward hidden sublayer receives the vector R _n-1,1 output by the first coding module of the n-1th layer of backward hidden sublayer, and the vector of the nth layer of backward hidden sublayer The first encoding module outputs the vector R _n,1 to the first encoding module of the hidden sub-layer at the n+1th layer.

In a specific embodiment, the encoding of the first word vector sequence into the first encoding sequence by the feature extraction model includes:

(a) The first encoding module of the forward hidden sublayer of the first layer encodes the first word vector of the first word vector sequence according to the first weight matrix subset in the initialized weight matrix set. _{The first vector Z 1,1} of the first intermediate vector sequence of the first coding sequence, the initialized weight matrix set includes N weight matrix subsets, the intermediate vector sequence of the first coding sequence and The intermediate vector sequence of the second coding sequence corresponds one-to-one in order. The forward hidden sublayer of the nth layer and the backward hidden sublayer of the nth layer share the nth weight matrix subset, and each weight matrix The subset includes multiple sets of weight matrixes and a fourth weight matrix, and each set of weight matrixes includes a V weight matrix, a Q weight matrix, and a K weight matrix.

Wherein, the V weight matrix, the Q weight matrix, and the K weight matrix in the multiple sets of weight matrix are used to calculate the first coding sequence and the second coding sequence based on multi-head attention. That is, the first coding sequence represents the above semantic information of the missing word in the first sentence sample, and the second coding sequence represents the following semantic information of the missing word in the first sentence sample information.

The first encoding module of the forward hidden sublayer of the first layer encodes the first word vector of the first word vector sequence into the first weight matrix subset in the initialized weight matrix set. _{The first vector Z 1,1} of the first intermediate vector sequence of the first coding sequence includes:

The first encoding module of the forward hidden sublayer of the first layer multiplies the first word vector of the first word vector sequence by the V in the multiple sets of weight matrixes in the first weight matrix subset. A weight matrix to obtain multiple V weight vectors of the first word vector of the first word vector sequence; connect the multiple V weight vectors of the first word vector of the first word vector sequence, Obtain the combined vector of the first word vector of the first word vector sequence; multiply the combined vector of the first word vector of the first word vector sequence by the fourth weight matrix to obtain the first _{The first vector Z 1,1} of the first intermediate vector sequence of the coding sequence.

The connection between the first coding module of two adjacent layers is similar to an ordinary neuron connection, and no attention mechanism is used.

(b) Starting from the second coding module of the forward hidden sub-layer of the first layer, the u-th coding module of the forward hidden sub-layer of the first layer will one by one according to the first weight matrix subset. The u-1th word vector of the first word vector sequence and the u-th word vector of the first word vector sequence are coded as the u-th vector Z _{1 of the first intermediate vector sequence of the first coding sequence ,u} , obtain the first intermediate vector sequence Z _{1 of} the first coding sequence Z 1 ={Z _1,1 ,...,Z _1,u ,...,Z _1,U }, wherein The u-th vector of the first intermediate vector sequence corresponds to the u-th word vector of the first word vector sequence in a one-to-one correspondence.

The u-th encoding module of the forward hidden sub-layer of the first layer calculates the u-1th word vector of the first word vector sequence and the first word vector sequence one by one according to the first weight matrix subset The u-th word vector encoding is the u-th vector Z _1,u of the first intermediate vector sequence of the first encoding sequence, and the first intermediate vector sequence Z _{1 of} the first encoding sequence Z 1 ={Z _1,1 ,…,Z _1,u ,…,Z _1,U } include:

(1) The second encoding module of the forward hidden sublayer of the first layer multiplies the second word vector of the first word vector sequence by the first group of weight matrix in the first weight matrix subset The V weight matrix in, obtain the V weight vector of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence.

(2) The second encoding module of the forward hidden sublayer of the first layer multiplies the second word vector of the first word vector sequence by the first group of weight matrix in the first weight matrix subset The Q weight matrix in, obtains the Q weight vector of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence.

(3) The second encoding module of the forward hidden sublayer of the first layer multiplies the second word vector of the first word vector sequence by the first group of weight matrix in the first weight matrix subset The K weight matrix in, obtains the K weight vector of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence.

(4) The second encoding module of the forward hidden sublayer of the first layer multiplies the first word vector of the first word vector sequence by the first group of weight matrix in the first weight matrix subset The V weight matrix in, obtains the V'weight vector of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence.

(5) The second encoding module of the forward hidden sublayer of the first layer multiplies the first word vector of the first word vector sequence by the first group of weight matrix in the first weight matrix subset The third K weight matrix in, obtain the K'weight vector of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence.

(6) The second coding module of the forward hidden sublayer of the first layer is based on the Q weight vector _{of the second vector Z 1,2 of the first intermediate vector sequence of the first coding sequence, the} a first intermediate vector of the two vectors encoding a first sequence of the sequence Z K weight vectors _{1 and 2,} the first a first intermediate vector of the two vectors coding sequence of the sequence Z K _1,2 'The weight vector determines the _{attention value of the V weight vector of the second vector Z 1,2} of the first intermediate vector sequence of the first coding sequence and the first intermediate vector sequence of the first coding sequence The attention value of the V'weight vector of the second vector Z _1,2.

(7) The second coding module of the forward hidden sublayer of the first layer is based on the V weight vector _{of the second vector Z 1,2 of the first intermediate vector sequence of the first coding sequence, the} The V'weight vector of the second vector of the first intermediate vector sequence Z _1,2 of the first coding sequence, the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence The attention value of the V weight vector and the attention value of the V'weight vector of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence determine the first coding sequence The first score of the second vector Z _{1,2 of an intermediate vector sequence.}

(8) (1)-(7) is the second encoding module of the forward hidden sublayer of the first layer to obtain the first encoding sequence according to the first set of weight matrices in the first weight matrix subset In the same way, the first score of the second vector Z _{1,2 of the first intermediate vector sequence of} Multiple scores of the second vector Z _{1,2 of an intermediate vector sequence.}

Multiple scores _{of the second vector Z 1,2} of the first intermediate vector sequence of the first coding sequence can be obtained from multiple sets of weight matrixes in the first weight matrix subset at the same time.

(9) The second coding module of the forward hidden sublayer of the first layer connects multiple scores _{of the second vector Z 1,2 of the first intermediate vector sequence of the first coding sequence,} The combined vector _{of the second vector Z 1,2} of the first intermediate vector sequence of the first coding sequence is obtained.

(10) The second coding module of the forward hidden sublayer of the first layer multiplies the combination vector of the _{second vector Z 1,2 of the first intermediate vector sequence of the first coding sequence by the first} A four-weight matrix is used to obtain an intermediate vector of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence.

(11) The feedforward network in the second coding module of the forward hidden sublayer of the first layer performs residual and normalization processing on the second intermediate vector sequence of the first coding sequence. The intermediate vectors of the two vectors Z _1,2 are coded, and the normalization process is performed again to obtain the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence.

(12)(1)-(11) is the second encoding module of the forward hidden sublayer of the first layer, according to the first weight matrix subset, the second word vector of the first word vector sequence And the first word vector of the first word vector sequence is coded as the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence. Similarly, the first layer of forward hiding The u-th coding module of the sub-layer can code to obtain the u-th vector Z _1,u of the first intermediate vector sequence of the first coding sequence, and obtain Z ₁ ={Z _1,1 ,...,Z _1,u ,…,Z _1,U }.

(c) Starting from the forward hidden sub-layer of the second layer, the n-th forward hidden sub-layer is used to calculate the n-1th intermediate vector sequence Z of the first coding sequence according to the n-th weight matrix subset. _{The n-1} code is the n-th intermediate vector sequence Z _n of the first code sequence.

In the same way that the feature extraction model encodes the first word vector sequence as a first encoding sequence, the feature extraction model encodes the second word vector sequence as a second encoding sequence R _n .

The u-th encoding module of the forward hidden sublayer of the first layer can encode the u-th vector Z _1,u of the first intermediate vector sequence of the first encoding sequence.

Each code in the forward hidden sublayer and the backward hidden sublayer of the same layer can run concurrently.

104. Use the feature extraction model to calculate the missing word vector of the first sentence sample according to the first coding sequence and the second coding sequence.

In this embodiment, the output layer of the feature extraction model is used to calculate the missing word vector of the first sentence sample according to the first coding sequence and the second coding sequence.

The vectors in the first coding sequence and the second coding sequence are summed in dimensions, and a sum vector obtained by the summation is multiplied by the output weight matrix and normalized to obtain the first The missing word vector of the sentence sample.

105. Train the feature extraction model according to the missing word vector of the first sentence sample and the label vector of the first sentence sample to obtain a first feature extraction model, create a second feature extraction model, and make the second feature The neural network structure of the extraction model is consistent with the neural network structure of the first feature extraction model, and the weight of the second feature extraction model is updated with the weight of the first feature extraction model.

The loss value of the missing word vector and the label vector of the first sentence sample may be calculated according to the cross-entropy loss function, and the weight matrix of the feature extraction model may be optimized according to the loss value.

A new intermediate feature extraction model can be created according to the neural network structure of the first feature extraction model. The neural network structure can include the number of neurons, the number of neuron layers, and the way of connection between neurons. The weight of the first feature extraction model can be copied. After the first feature extraction model is trained, the weight of the first feature extraction model enables the first feature extraction model to have strong feature extraction capabilities , Initializing the weight of the intermediate feature extraction model with the weight of the first feature extraction model to obtain the second feature extraction model that is the same as the first feature extraction model.

106. Use a second sentence sample with an attribute label to train an attribute classification model composed of the first feature extraction model and a fully connected layer.

For example, for the second sentence sample of a laptop computer, the attribute label may include resolution, processor, sound effects, etc. One sentence of the second sentence sample is "This computer responds quickly", and the attribute label is "Processor". Indicates that this second sample sentence includes the semantics of the processor.

The second sentence sample may be a sentence in a given field with attribute tags. A small number of the second sentence samples can be used to train the attribute classification model. Because the feature extraction model has been trained and can extract semantic information well, it is only necessary to fine-tune and adjust the weight matrix of the feature extraction model. The weight matrix in the fully connected layer is optimized. Wherein, the output of the first feature extraction model is the input of the fully connected layer.

In a specific embodiment, each sentence in the second sentence sample is equally divided into two parts according to the number of words, and the words in the first part of each sentence in the second sentence sample are similar to the first part. The words before the missing word of each sentence in the sentence sample, and the words in the latter part of each sentence of the second sentence sample are similar to the words after the missing word of each sentence in the first sentence sample. For example, if one sentence of the second sentence sample is "<S>This computer has a very fast response speed processor<E>", then "<S>This computer's response" is taken as the first part of a sentence in the second sentence sample The words in the first sentence sample are similar to the words before the missing word of a sentence in the first sentence sample; the "fast processor <E>" is used as the last part of a sentence in the second sentence sample, similar to the first sentence The words after the missing word in a sentence of the sample.

If the number of words in the sentence in the second sentence sample is an odd number, the one word in the middle is eliminated when the second sentence sample is divided, and the word in the middle is divided into the front part.

The process of using the second sentence sample to train the first feature extraction model is similar to the process of using the first sentence sample to train the feature extraction model, and will not be repeated here.

107. Use the attribute classification model to identify the attribute words of a plurality of sentences to be recognized, and connect each sentence to be recognized with the attribute words of each sentence to be recognized to obtain the plurality of sentences to be recognized that connect the attribute words .

For example, the attribute tag of the sentence to be recognized "<S>This computer responds very quickly. Processor <E>" is "Processor", and the attribute words of the sentence to be recognized and the sentence to be recognized are connected as "<S>This computer The response speed is very fast <SOE>Processor<E>", where "<SOE>" stands for conjunction.

108. Train an emotion classification model composed of the second feature extraction model and a deep learning model by using the plurality of sentences to be recognized that are connected to attribute words with emotion labels.

The sentiment label may include "positive", "neutral", "negative", etc., and the deep learning model may be CNN, RNN, or LSTM. Training the sentiment classification model is an existing method, and will not be repeated here. Wherein, the output of the second feature extraction model is the input of the deep learning model.

The multiple to-be-recognized sentences of the connected attribute words may be output, and the multiple to-be-recognized sentences of the output connected attribute words may be manually labeled to obtain the multiple to-be-recognized sentences of the connected attribute words with emotion labels , Receiving the plurality of sentences to be recognized of the connection attribute words with emotion tags.

The process of training the second feature extraction model with the plurality of sentences to be recognized with the sentiment label connected attribute words is similar to the process of training the first feature extraction model with the second sentence samples. Go into details again.

109. Use the attribute classification model to identify the attribute words of the sentence to be processed, and the sentiment classification model classifies the sentence to be processed connecting the attribute words, and outputs the attribute words of the sentence to be processed and the emotion type of the sentence to be processed .

For example, the attribute classification model is used to identify the attribute word of the sentence to be processed "this computer is very fast" as "processor", and the sentiment classification model responds very quickly to the sentence "<S>" which is connected to the attribute word. Fast <SOE> processor <E>" to classify, output the attribute word "processor" of the sentence to be processed and the emotion type "positive" of the sentence to be processed.

It should be emphasized that, in order to further ensure the privacy and security of the attribute classification model and the emotion classification model, the attribute classification model and the emotion classification model may also be stored in a node of a blockchain.

The first embodiment realizes emotion classification of sentences, and enhances the accuracy and scene adaptability of emotion classification.

In another embodiment, the U-th encoding module of the n-th forward hidden sublayer converts the U-1th vector of the n-1 intermediate vector sequence of the first encoding sequence, and the first encoding sequence The U-th vector of the n-1 intermediate vector sequence of the second coding sequence and the W-th vector of the n-1 intermediate vector sequence of the second coding sequence are coded as Z _{n, U} ; the nth layer of the backward hidden sublayer is The W encoding modules combine the W-1th vector of the n-1 intermediate vector sequence of the second encoding sequence, the Wth vector of the n-1 intermediate vector sequence of the second encoding sequence, and the The U-th vector of the n-1 intermediate vector sequence of a coding sequence is coded as R _n,W .

In another embodiment, the feature extraction model can be migrated to sentiment classification models in different fields.

Example two

Fig. 2 is a structural diagram of a sentence emotion classification device provided in the second embodiment of the present application. The sentence emotion classification device 20 is applied to computer equipment. The sentence emotion classification device 20 can perform emotion classification on the sentence. As shown in FIG. 2, the sentence emotion classification device 20 may include an acquisition module 201, a conversion module 202, an encoding module 203, a calculation module 204, a first training module 205, a second training module 206, a connection module 207, and a third training module. Module 208, classification module 209.

The obtaining module 201 is configured to obtain a first sentence sample set, and each first sentence sample in the first sentence sample set contains a missing word.

Multiple texts in different fields can be obtained, each text includes multiple sentences, each text is occluded multiple times, part of the words in the text is occluded each time, and a missing part is extracted from the text of each occluded part of the words The sentence of words is used as the first sentence sample.

Multiple texts in various fields such as tourism, electronic products, and patent services can be obtained. Each field includes multiple texts, and each text in each field can include multiple sentences. This embodiment does not limit the size of the field, such as the field of electronic products and the field of notebook computers, and the field of electronic products may include the field of notebook computers.

Each text in multiple texts in each field can be occluded multiple times, and a preset proportion of words in each text can be randomly occluded each time to obtain the first sentence with missing words in multiple texts in each field sample.

The conversion module 202 is configured to, for each first sentence sample, use a feature extraction model to convert the words before the missing words in the first sentence sample into a first word vector sequence in order of words, and convert all the words in the first sentence sample The words following the missing word are converted into a second word vector sequence according to the reverse word order, and the missing words in the first sentence sample are converted into the label vector of the first sentence sample according to a preset vocabulary coding table.

The feature extraction model includes the input layer, the forward hidden layer, the backward hidden layer, and the output layer.

In a specific embodiment, the use of the feature extraction model converts the words before the missing word in the first sentence sample into a first word vector sequence in order of words, and converts the words after the missing word in the first sentence sample The conversion of words into the second word vector sequence according to the reverse word order includes:

Convert the words before the missing word in the first sentence sample into a first coding vector sequence in word order, and convert the words after the missing word in the first sentence sample into a second coding vector in word order Sequence; convert the position number of the word before the missing word in the first sentence sample into a first position vector sequence, and convert the position number of the word after the missing word in the first sentence sample into A second position vector sequence; converting the first coding vector sequence and the first position vector sequence into a first word vector sequence, and converting the second coding vector sequence and the second position vector sequence into a second Sequence of word vectors.

For example, a first sentence sample is "<S>from <mask>Language Processing<E>", where "<S>" means the head word of the first sentence sample, and "<E>" means the first sentence sample The tail word, according to the preset vocabulary coding table, convert the word "<S>自" before the missing word "ran" into the first coding vector sequence {(0,0,0,0,1,0,0 in word order) , 0), (0, 0, 0, 0, 0, 0, 0, 1)}, the word "language processing <E>" after the missing word "ran" is transformed into the second coding vector sequence {( 0, 0, 0, 0, 0, 0, 1, 0), (0, 0, 0, 0, 0, 1, 0, 0), (1, 0, 0, 0, 0, 0, 0, 0), (0, 0, 1, 0, 0, 0, 0, 0), (0, 1, 0, 0, 0, 0, 0, 0)}, the preset vocabulary encoding table may adopt one -Encoding methods such as hot and word2vec. Convert the position number of the word before the missing word "ran" in the first sentence sample into the first position vector sequence {(1, 0, 0, 0, 0, 0, 0, 0), (0, 1, 0) ,0,0,0,0,0)}, convert the position number of the word after the missing word in the first sentence sample into a second position vector sequence {(0,0,0,1,0,0,0 , 0), (0, 0, 0, 0, 1, 0, 0, 0), (0, 0, 0, 0, 0, 1, 0, 0), (0, 0, 0, 0, 0 , 0, 1, 0), (0, 0, 0, 0, 0, 0, 0, 1)}. Add the first code vector in the first code vector sequence corresponding to each word before the missing word "ran" and the first position vector in the first position vector sequence to obtain the first word vector sequence {(1, 0 , 0, 0, 1, 0, 0, 0), (0, 1, 0, 0, 0, 0, 0, 1)}. Add the second code vector in the second code vector sequence corresponding to each word after the missing word "ran" and the second position vector in the second position vector sequence to obtain the second word vector sequence {(0, 0 , 0, 1, 0, 0, 1, 0), (0, 0, 0, 0, 1, 1, 0, 0), (1, 0, 0, 0, 0, 1, 0, 0), (0, 0, 1, 0, 0, 0, 1, 0), (0, 1, 0, 0, 0, 0, 0, 1)}.

According to the preset vocabulary encoding table, the missing word <mask> in the first sentence sample is converted into the label vector (0, 0, 0, 1, 0, 0, 0, 0) of the first sentence sample, namely "Ran" one-hot encoding.

The encoding module 203 is configured to use the feature extraction model to encode the first word vector sequence into a first encoding sequence, and to encode the second word vector sequence into a second encoding sequence.

In this embodiment, the forward hidden layer of the feature extraction model encodes the first word vector sequence into the first coding sequence, and the backward hidden layer of the feature extraction model encodes the second word vector sequence The sequence code is the second coding sequence. The forward hidden layer and the backward hidden layer respectively include N forward hidden sublayers and N backward hidden sublayers, each forward hidden sublayer includes U encoding modules, and each backward hidden sublayer The layer includes W encoding modules; wherein the u-th encoding module of the n-th forward hidden sublayer receives the vector Z _{n-1 output by the u-1th encoding module of the n-1th forward hidden sublayer, u-1} and the vector Z of the n 1-layer of the front output to hidden sublayer u-th encoding module _{n-1, u,} n-th layer before Z _n to the hidden sublayer u-th encoding module output _{vector, u} To the u-th coding module of the forward hidden sublayer of the n+1th layer and the u+1th coding module of the forward hidden sublayer of the n+1th layer, 2≤n≤N, 2≤u≤U. The u-th encoding module of the first layer forward hidden sublayer receives the u-1th word vector of the first word vector sequence and the u-th word vector of the first word vector sequence, and the Nth layer forward The hidden sub-layer is the first coding sequence. The first encoding module of the nth forward hidden sublayer receives the vector Z _n-1,1 output by the first encoding module of the n-1th forward hidden sublayer. The first encoding module of the nth forward hidden sublayer The first encoding module outputs the vector Z _n,1 to the first encoding module of the forward hidden sub-layer of the n+1th layer. The w-th encoding module of the nth layer of backward hidden sublayer receives the vector R _{n-1, w-1} and n-1th layer output by the w-1th encoding module of the n-1th layer of backward hidden sublayer _{The vector R n-1,w} output by the w-th encoding module of the backward hidden sub-layer, the output vector Z _n,w of the w-th encoding module of the n-th backward hidden sub-layer to the n+1-th layer is backward hidden The w-th coding module of the sub-layer and the w+1-th coding module of the backward hidden sub-layer of the n+1 layer, 2≤w≤W. The w-th encoding module of the first layer of backward hidden sublayer receives the w-1th word vector of the second word vector sequence and the wth word vector of the second word vector sequence, and the Nth layer is backward The hidden sublayer is the second coding sequence. The first coding module of the nth layer of backward hidden sublayer receives the vector R _n-1,1 output by the first coding module of the n-1th layer of backward hidden sublayer, and the vector of the nth layer of backward hidden sublayer The first encoding module outputs the vector R _n,1 to the first encoding module of the hidden sub-layer at the n+1th layer.

In a specific embodiment, the encoding of the first word vector sequence into the first encoding sequence by the feature extraction model includes:

(a) The first encoding module of the forward hidden sublayer of the first layer encodes the first word vector of the first word vector sequence according to the first weight matrix subset in the initialized weight matrix set. _{The first vector Z 1,1} of the first intermediate vector sequence of the first coding sequence, the initialized weight matrix set includes N weight matrix subsets, the intermediate vector sequence of the first coding sequence and The intermediate vector sequence of the second coding sequence corresponds one-to-one in order. The forward hidden sublayer of the nth layer and the backward hidden sublayer of the nth layer share the nth weight matrix subset, and each weight matrix The subset includes multiple sets of weight matrixes and a fourth weight matrix, and each set of weight matrixes includes a V weight matrix, a Q weight matrix, and a K weight matrix.

Wherein, the V weight matrix, the Q weight matrix, and the K weight matrix in the multiple sets of weight matrix are used to calculate the first coding sequence and the second coding sequence based on multi-head attention. That is, the first coding sequence represents the above semantic information of the missing word in the first sentence sample, and the second coding sequence represents the following semantic information of the missing word in the first sentence sample information.

The first encoding module of the forward hidden sublayer of the first layer encodes the first word vector of the first word vector sequence into the first weight matrix subset in the initialized weight matrix set. _{The first vector Z 1,1} of the first intermediate vector sequence of the first coding sequence includes:

The first encoding module of the forward hidden sublayer of the first layer multiplies the first word vector of the first word vector sequence by the V in the multiple sets of weight matrixes in the first weight matrix subset. A weight matrix to obtain multiple V weight vectors of the first word vector of the first word vector sequence; connect the multiple V weight vectors of the first word vector of the first word vector sequence, Obtain the combined vector of the first word vector of the first word vector sequence; multiply the combined vector of the first word vector of the first word vector sequence by the fourth weight matrix to obtain the first _{The first vector Z 1,1} of the first intermediate vector sequence of the coding sequence.

The connection between the first coding module of two adjacent layers is similar to an ordinary neuron connection, and no attention mechanism is used.

(b) Starting from the second coding module of the forward hidden sub-layer of the first layer, the u-th coding module of the forward hidden sub-layer of the first layer will one by one according to the first weight matrix subset. The u-1th word vector of the first word vector sequence and the u-th word vector of the first word vector sequence are coded as the u-th vector Z _{1 of the first intermediate vector sequence of the first coding sequence ,u} , obtain the first intermediate vector sequence Z _{1 of} the first coding sequence Z 1 ={Z _1,1 ,...,Z _1,u ,...,Z _1,U }, wherein The u-th vector of the first intermediate vector sequence corresponds to the u-th word vector of the first word vector sequence in a one-to-one correspondence.

The u-th encoding module of the forward hidden sub-layer of the first layer calculates the u-1th word vector of the first word vector sequence and the first word vector sequence one by one according to the first weight matrix subset The u-th word vector encoding is the u-th vector Z _1,u of the first intermediate vector sequence of the first encoding sequence, and the first intermediate vector sequence Z _{1 of} the first encoding sequence Z 1 ={Z _1,1 ,…,Z _1,u ,…,Z _1,U } include:

(1) The second encoding module of the forward hidden sublayer of the first layer multiplies the second word vector of the first word vector sequence by the first group of weight matrix in the first weight matrix subset The V weight matrix in, obtain the V weight vector of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence.

(2) The second encoding module of the forward hidden sublayer of the first layer multiplies the second word vector of the first word vector sequence by the first group of weight matrix in the first weight matrix subset The Q weight matrix in, obtains the Q weight vector of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence.

(3) The second encoding module of the forward hidden sublayer of the first layer multiplies the second word vector of the first word vector sequence by the first group of weight matrix in the first weight matrix subset The K weight matrix in, obtains the K weight vector of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence.

(4) The second encoding module of the forward hidden sublayer of the first layer multiplies the first word vector of the first word vector sequence by the first group of weight matrix in the first weight matrix subset The V weight matrix in, obtains the V'weight vector of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence.

(5) The second encoding module of the forward hidden sublayer of the first layer multiplies the first word vector of the first word vector sequence by the first group of weight matrix in the first weight matrix subset The third K weight matrix in, obtain the K'weight vector of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence.

(6) The second coding module of the forward hidden sublayer of the first layer is based on the Q weight vector _{of the second vector Z 1,2 of the first intermediate vector sequence of the first coding sequence, the} a first intermediate vector of the two vectors encoding a first sequence of the sequence Z K weight vectors _{1 and 2,} the first a first intermediate vector of the two vectors coding sequence of the sequence Z K _1,2 'The weight vector determines the _{attention value of the V weight vector of the second vector Z 1,2} of the first intermediate vector sequence of the first coding sequence and the first intermediate vector sequence of the first coding sequence The attention value of the V'weight vector of the second vector Z _1,2.

(7) The second coding module of the forward hidden sublayer of the first layer is based on the V weight vector _{of the second vector Z 1,2 of the first intermediate vector sequence of the first coding sequence, the} The V'weight vector of the second vector of the first intermediate vector sequence Z _1,2 of the first coding sequence, the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence The attention value of the V weight vector and the attention value of the V'weight vector of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence determine the first coding sequence The first score of the second vector Z _{1,2 of an intermediate vector sequence.}

(8) (1)-(7) is the second encoding module of the forward hidden sublayer of the first layer to obtain the first encoding sequence according to the first set of weight matrices in the first weight matrix subset In the same way, the first score of the second vector Z _{1,2 of the first intermediate vector sequence of} Multiple scores of the second vector Z _{1,2 of an intermediate vector sequence.}

Multiple scores _{of the second vector Z 1,2} of the first intermediate vector sequence of the first coding sequence can be obtained from multiple sets of weight matrixes in the first weight matrix subset at the same time.

(9) The second coding module of the forward hidden sublayer of the first layer connects multiple scores _{of the second vector Z 1,2 of the first intermediate vector sequence of the first coding sequence,} The combined vector _{of the second vector Z 1,2} of the first intermediate vector sequence of the first coding sequence is obtained.

(10) The second coding module of the forward hidden sublayer of the first layer multiplies the combination vector of the _{second vector Z 1,2 of the first intermediate vector sequence of the first coding sequence by the first} A four-weight matrix is used to obtain an intermediate vector of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence.

(11) The feedforward network in the second coding module of the forward hidden sublayer of the first layer performs residual and normalization processing on the second intermediate vector sequence of the first coding sequence. The intermediate vectors of the two vectors Z _1,2 are coded, and the normalization process is performed again to obtain the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence.

(12)(1)-(11) is the second encoding module of the forward hidden sublayer of the first layer, according to the first weight matrix subset, the second word vector of the first word vector sequence And the first word vector of the first word vector sequence is coded as the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence. Similarly, the first layer of forward hiding The u-th coding module of the sub-layer can code to obtain the u-th vector Z _1,u of the first intermediate vector sequence of the first coding sequence, and obtain Z ₁ ={Z _1,1 ,...,Z _1,u ,…,Z _1,U }.

(c) Starting from the forward hidden sub-layer of the second layer, the n-th forward hidden sub-layer is used to calculate the n-1th intermediate vector sequence Z of the first coding sequence according to the n-th weight matrix subset. _{The n-1} code is the n-th intermediate vector sequence Z _n of the first code sequence.

In the same way that the feature extraction model encodes the first word vector sequence as a first encoding sequence, the feature extraction model encodes the second word vector sequence as a second encoding sequence R _n .

The u-th encoding module of the forward hidden sublayer of the first layer can encode the u-th vector Z _1,u of the first intermediate vector sequence of the first encoding sequence.

Each code in the forward hidden sublayer and the backward hidden sublayer of the same layer can run concurrently.

The calculation module 204 is configured to use the feature extraction model to calculate the missing word vector of the first sentence sample according to the first coding sequence and the second coding sequence.

In this embodiment, the output layer of the feature extraction model is used to calculate the missing word vector of the first sentence sample according to the first coding sequence and the second coding sequence.

The vectors in the first coding sequence and the second coding sequence are summed in dimensions, and a sum vector obtained by the summation is multiplied by the output weight matrix and normalized to obtain the first The missing word vector of the sentence sample.

The first training module 205 is configured to train the feature extraction model according to the missing word vector of the first sentence sample and the label vector of the first sentence sample to obtain a first feature extraction model, and create a second feature extraction model, The neural network structure of the second feature extraction model is made consistent with the neural network structure of the first feature extraction model, and the weight of the second feature extraction model is updated with the weight of the first feature extraction model.

The loss value of the missing word vector and the label vector of the first sentence sample may be calculated according to the cross-entropy loss function, and the weight matrix of the feature extraction model may be optimized according to the loss value.

A new intermediate feature extraction model can be created according to the neural network structure of the first feature extraction model. The neural network structure can include the number of neurons, the number of neuron layers, and the way of connection between neurons. The weight of the first feature extraction model can be copied. After the first feature extraction model is trained, the weight of the first feature extraction model enables the first feature extraction model to have strong feature extraction capabilities , Initializing the weight of the intermediate feature extraction model with the weight of the first feature extraction model to obtain the second feature extraction model that is the same as the first feature extraction model.

The second training module 206 is used to train the attribute classification model composed of the first feature extraction model and the fully connected layer using second sentence samples with attribute tags.

For example, for the second sentence sample of a laptop computer, the attribute label may include resolution, processor, sound effects, etc. One sentence of the second sentence sample is "This computer responds quickly", and the attribute label is "Processor". Indicates that this second sample sentence includes the semantics of the processor.

The second sentence sample may be a sentence in a given field with attribute tags. A small number of the second sentence samples can be used to train the attribute classification model. Because the feature extraction model has been trained and can extract semantic information well, it is only necessary to fine-tune and adjust the weight matrix of the feature extraction model. The weight matrix in the fully connected layer is optimized. Wherein, the output of the first feature extraction model is the input of the fully connected layer.

In a specific embodiment, each sentence in the second sentence sample is equally divided into two parts according to the number of words, and the words in the first part of each sentence in the second sentence sample are similar to the first part. The words before the missing word of each sentence in the sentence sample, and the words in the latter part of each sentence of the second sentence sample are similar to the words after the missing word of each sentence in the first sentence sample. For example, if one sentence of the second sentence sample is "<S>This computer has a fast response speed processor<E>", then "<S>This computer's response" is taken as the first part of a sentence in the second sentence sample The words in the first sentence sample are similar to the words before the missing word of a sentence in the first sentence sample; the "fast processor <E>" is used as the last part of a sentence in the second sentence sample, similar to the first sentence The words after the missing word in a sentence of the sample.

If the number of words in the sentence in the second sentence sample is an odd number, the one word in the middle is eliminated when the second sentence sample is divided, and the word in the middle is divided into the front part.

The process of using the second sentence sample to train the first feature extraction model is similar to the process of using the first sentence sample to train the feature extraction model, and will not be repeated here.

The connection module 207 is used to identify the attribute words of a plurality of sentences to be recognized by using the attribute classification model, and connect each sentence to be recognized with the attribute words of each sentence to be recognized to obtain the plurality of connected attribute words. Sentences to be recognized.

For example, the attribute tag of the sentence to be recognized "<S>This computer responds very quickly. Processor <E>" is "Processor", and the attribute words of the sentence to be recognized and the sentence to be recognized are connected as "<S>This computer The response speed is very fast <SOE>Processor<E>", where "<SOE>" stands for conjunction.

The third training module 208 is configured to train an emotion classification model composed of the second feature extraction model and a deep learning model by using the plurality of sentences to be recognized that are connected to attribute words with emotion labels.

The sentiment label may include "positive", "neutral", "negative", etc., and the deep learning model may be CNN, RNN, or LSTM. Training the sentiment classification model is an existing method, and will not be repeated here. Wherein, the output of the second feature extraction model is the input of the deep learning model.

The multiple to-be-recognized sentences of the connected attribute words may be output, and the multiple to-be-recognized sentences of the output connected attribute words may be manually labeled to obtain the multiple to-be-recognized sentences of the connected attribute words with emotion labels , Receiving the plurality of sentences to be recognized of the connection attribute words with emotion tags.

The process of training the second feature extraction model with the plurality of sentences to be recognized with the sentiment label connected attribute words is similar to the process of training the first feature extraction model with the second sentence samples. Go into details again.

The classification module 209 is configured to use the attribute classification model to identify the attribute words of the sentence to be processed. The sentiment classification model classifies the sentence to be processed connecting the attribute words, and outputs the attribute words of the sentence to be processed and the sentence to be processed The emotional type of the sentence.

For example, the attribute classification model is used to identify the attribute word of the sentence to be processed "this computer is very fast" as "processor", and the sentiment classification model responds very quickly to the sentence "<S>" which is connected to the attribute word. Fast <SOE> processor <E>" to classify, output the attribute word "processor" of the sentence to be processed and the emotion type "positive" of the sentence to be processed.

It should be emphasized that, in order to further ensure the privacy and security of the attribute classification model and the emotion classification model, the attribute classification model and the emotion classification model may also be stored in a node of a blockchain.

The second embodiment realizes emotion classification of sentences, and enhances the accuracy and scene adaptability of emotion classification.

In another embodiment, the U-th encoding module of the n-th forward hidden sublayer converts the U-1th vector of the n-1 intermediate vector sequence of the first encoding sequence, and the first encoding sequence The U-th vector of the n-1 intermediate vector sequence of the second coding sequence and the W-th vector of the n-1 intermediate vector sequence of the second coding sequence are coded as Z _{n, U} ; the nth layer of the backward hidden sublayer is The W encoding modules combine the W-1th vector of the n-1 intermediate vector sequence of the second encoding sequence, the Wth vector of the n-1 intermediate vector sequence of the second encoding sequence, and the The U-th vector of the n-1 intermediate vector sequence of a coding sequence is coded as R _n,W .

In another embodiment, the feature extraction model can be migrated to sentiment classification models in different fields.

Example three

This embodiment provides a computer storage medium with a computer program stored on the computer storage medium. The computer storage medium may be non-volatile or volatile. When the computer program is executed by the processor, the steps in the above-mentioned sentence emotion classification method embodiment, such as steps 101-109 shown in FIG. 1, are implemented.

Or, when the computer program is executed by the processor, the function of each module in the above-mentioned device embodiment is realized, for example, the modules 201-209 in FIG. 2.

Example four

FIG. 3 is a schematic diagram of the computer equipment provided in the fourth embodiment of the application. The computer device 30 includes a memory 301, a processor 302, and a computer program 303 stored in the memory 301 and running on the processor 302, such as a sentence emotion classification program. When the processor 302 executes the computer program 303, the steps in the embodiment of the sentence emotion classification method described above are implemented, for example, steps 101-109 shown in FIG. 1.

Alternatively, when the computer program 303 is executed by the processor, the functions of the modules in the above-mentioned device embodiments, such as the modules 201-209 in FIG. 2, are realized.

Exemplarily, the computer program 303 may be divided into one or more modules, and the one or more modules are stored in the memory 301 and executed by the processor 302 to complete the method. The one or more modules may be a series of computer-readable instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 303 in the computer device 30. For example, the computer program 303 can be divided into the acquisition module 201, the conversion module 202, the encoding module 203, the calculation module 204, the first training module 205, the second training module 206, the connection module 207, and the third training in FIG. Module 208, classification module 209, see the second embodiment for the specific functions of each module.

The computer device 30 may be a computing device such as a desktop computer, a notebook, a palmtop computer, and a cloud server. Those skilled in the art can understand that the schematic diagram 3 is only an example of the computer device 30, and does not constitute a limitation on the computer device 30. It may include more or less components than those shown in the figure, or combine certain components, or different components. For example, the computer device 30 may also include input and output devices, network access devices, buses, and so on.

The so-called processor 302 may be a central processing unit (Central Processing Unit, CPU), other general processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor 302 can also be any conventional processor, etc. The processor 302 is the control center of the computer device 30, which uses various interfaces and lines to connect the entire computer device 30. Various parts.

The memory 301 may be used to store the computer program 303, and the processor 302 can implement the computer by running or executing the computer program 303 or module stored in the memory 301 and calling data stored in the memory 301. Various functions of the device 30. The memory 301 may mainly include a storage program area and a storage data area, where the storage program area may store an operating system, an application program required by at least one function (such as a sound playback function, an image playback function, etc.); the storage data area may The data (such as audio data, phone book, etc.) created according to the use of the computer device 30 are stored. In addition, the memory 301 may include non-volatile and volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a Secure Digital (SD) card, and a flash memory card ( Flash Card), at least one magnetic disk storage device, flash memory device, or other storage device.

If the integrated module of the computer device 30 is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, this application implements all or part of the processes in the above-mentioned embodiments and methods, and can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer storage medium. When executed by the processor, the steps of the foregoing method embodiments can be implemented. Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read only memory (ROM), random access memory (RAM) etc.

The blockchain referred to in this application is a new application mode of computer technology such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm. Blockchain, essentially a decentralized database, is a series of data blocks associated with cryptographic methods. Each data block contains a batch of network transaction information for verification. The validity of the information (anti-counterfeiting) and the generation of the next block. The blockchain can include the underlying platform of the blockchain, the platform product service layer, and the application service layer.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the application and not to limit them. Although the application has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the application can be Make modifications or equivalent replacements without departing from the spirit and scope of the technical solution of the present application.

Claims (20)

1. A sentence sentiment classification method, wherein the sentence sentiment classification method includes:

   Acquiring a first sentence sample set, where each first sentence sample in the first sentence sample set contains a missing word;

   For each first sentence sample, use a feature extraction model to convert the words before the missing word in the first sentence sample into a first word vector sequence, and convert the words after the missing word in the first sentence sample Transforming into a second word vector sequence according to the reverse word order, and transforming the missing words in the first sentence sample into the label vector of the first sentence sample according to a preset vocabulary coding table;

   Coding the first word vector sequence into a first coding sequence and coding the second word vector sequence into a second coding sequence by using the feature extraction model;

   Using the feature extraction model to calculate the missing word vector of the first sentence sample according to the first coding sequence and the second coding sequence;

   Train the feature extraction model according to the missing word vector of the first sentence sample and the label vector of the first sentence sample to obtain a first feature extraction model, create a second feature extraction model, and make the second feature extraction model The neural network structure of is consistent with the neural network structure of the first feature extraction model, and the weight of the second feature extraction model is updated with the weight of the first feature extraction model;

   Training an attribute classification model composed of the first feature extraction model and a fully connected layer by using a second sentence sample with attribute labels;

   Identify the attribute words of a plurality of sentences to be recognized by using the attribute classification model, and connect each sentence to be recognized with the attribute words of each sentence to be recognized that are recognized to obtain the plurality of sentences to be recognized that connect the attribute words;

   Training an emotion classification model composed of the second feature extraction model and a deep learning model by using the plurality of sentences to be recognized that are connected attribute words with emotion labels;

   The attribute classification model is used to identify the attribute words of the sentence to be processed, and the emotion classification model classifies the sentence to be processed connecting the attribute words, and outputs the attribute words of the sentence to be processed and the emotion type of the sentence to be processed.
2. The sentence emotion classification method according to claim 1, wherein the feature extraction model includes an input layer, a forward hidden layer, a backward hidden layer, and an output layer.
3. 5. The sentence sentiment classification method according to claim 1, wherein the use of the feature extraction model converts the words before the missing words in the first sentence sample into a first word vector sequence in order to convert the first sentence sample The words after the missing words described in the reverse word order are transformed into the second word vector sequence including:

   Convert the words before the missing word in the first sentence sample into a first coding vector sequence in word order, and convert the words after the missing word in the first sentence sample into a second coding vector in word order sequence;

   Convert the position number of the word before the missing word in the first sentence sample into a first position vector sequence, and convert the position number of the word after the missing word in the first sentence sample into a second Position vector sequence;

   The first coding vector sequence and the first position vector sequence are converted into a first word vector sequence, and the second coding vector sequence and the second position vector sequence are converted into a second word vector sequence.
4. 5. The sentence sentiment classification method according to claim 1, wherein said feature extraction model encoding said first word vector sequence into a first encoding sequence comprises:

   The first encoding module of the forward hidden sublayer of the first layer of the feature extraction model calculates the first word vector of the first word vector sequence according to the first weight matrix subset in the initialized weight matrix set _{Coded as the first vector Z 1,1} of the first intermediate vector sequence of the first coding sequence, the initialized weight matrix set includes N weight matrix subsets, and the middle of the first coding sequence The vector sequence corresponds to the intermediate vector sequence of the second coding sequence in a one-to-one order, and the forward hidden sublayer of the nth layer and the backward hidden sublayer of the nth layer of the feature extraction model share the nth weight. Matrix subset, each weight matrix subset includes multiple sets of weight matrix and a fourth weight matrix, and each set of weight matrix includes V weight matrix, Q weight matrix, and K weight matrix;

   Starting from the second coding module of the forward hidden sublayer of the first layer, the u-th coding module of the forward hidden sublayer of the first layer divides the first coding module one by one according to the first weight matrix subset. The u-1th word vector of the word vector sequence and the u-th word vector of the first word vector sequence are coded as the u-th vector Z _1,u of the first intermediate vector sequence of the first coding sequence, Obtain the first intermediate vector sequence Z ₁ ={Z _1,1 ,...,Z _1,u ,...,Z _1,U } of the first coding sequence, wherein the first one of the first coding sequence The u-th vector of the intermediate vector sequence corresponds to the u-th word vector of the first word vector sequence in a one-to-one correspondence;

   Starting from the forward hidden sub-layer of the second layer of the feature extraction model, the n-th forward hidden sub-layer is used to calculate the n-1th of the first coding sequence according to the n-th weight matrix subset. intermediate vector Z _n-1 encoding sequence for said first sequence of n intermediate vector Z _n coding sequence.
5. The sentence sentiment classification method according to claim 4, wherein the first encoding module of the forward hidden sub-layer of the first layer divides the first weight matrix subset according to the first weight matrix subset in the initialized weight matrix set. The first word vector encoding of a word vector sequence is the first vector Z _1,1 of the first intermediate vector sequence of the first encoding sequence includes:

   The first encoding module of the forward hidden sublayer of the first layer multiplies the first word vector of the first word vector sequence by the V in the multiple sets of weight matrixes in the first weight matrix subset. A weight matrix to obtain multiple V weight vectors of the first word vector of the first word vector sequence;

   Connecting multiple V weight vectors of the first word vector of the first word vector sequence to obtain a combined vector of the first word vector of the first word vector sequence;

   Multiply the combined vector of the first word vector of the first word vector sequence by the fourth weight matrix to obtain the first vector Z _{1,1 of the first intermediate vector sequence of the first coding sequence} .
6. The sentence sentiment classification method according to claim 4, wherein the u-th encoding module of the forward hidden sub-layer of the first layer divides the first word vector sequence of the first word vector sequence one by one according to the first weight matrix subset u-1 word vectors and the u-th word vector of the first word vector sequence are coded as the u-th vector Z _1,u of the first intermediate vector sequence of the first coding sequence, to obtain the first The first intermediate vector sequence Z _{1 of the} coding sequence = {Z _1,1 ,...,Z _1,u ,...,Z _1,U } includes:

   The second encoding module of the forward hidden sublayer of the first layer multiplies the second word vector of the first word vector sequence by V in the first group of weight matrix in the first weight matrix subset. A weight matrix to obtain the V weight vector _{of the second vector Z 1,2} of the first intermediate vector sequence of the first coding sequence;

   The second encoding module of the forward hidden sublayer of the first layer multiplies the second word vector of the first word vector sequence by the Q in the first group of weight matrix in the first weight matrix subset. A weight matrix to obtain the Q weight vector _{of the second vector Z 1,2} of the first intermediate vector sequence of the first coding sequence;

   The second encoding module of the forward hidden sublayer of the first layer multiplies the second word vector of the first word vector sequence by K in the first group of weight matrix in the first weight matrix subset A weight matrix to obtain the K weight vector _{of the second vector Z 1,2} of the first intermediate vector sequence of the first coding sequence;

   The second encoding module of the forward hidden sublayer of the first layer multiplies the first word vector of the first word vector sequence by V in the first group of weight matrix in the first weight matrix subset. A weight matrix to obtain the V'weight vector of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence;

   The second encoding module of the forward hidden sublayer of the first layer multiplies the first word vector of the first word vector sequence by the first weight matrix in the first group of weight matrix subsets in the first weight matrix subset. Three K weight matrix to obtain the K'weight vector of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence;

   The second coding module of the forward hidden sublayer of the first layer is based on the Q weight vector of the second _{vector Z 1,2 of the first intermediate vector sequence of the first coding sequence, and the first coding sequence} The K weight vector of the second vector Z _1,2 of the first intermediate vector sequence of the sequence, the K'weight of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence The vector determines the _{attention value of the V weight vector of the second vector Z 1,2} of the first intermediate vector sequence of the first coding sequence and the second of the first intermediate vector sequence of the first coding sequence. The attention value of the V'weight vector of two vectors Z _1,2;

   The second coding module of the forward hidden sublayer of the first layer is based on the V weight vector of the second _{vector Z 1,2 of the first intermediate vector sequence of the first coding sequence, and the first coding sequence} V'weight vector of the second vector of the first intermediate vector sequence Z _{1,2 of the sequence,} and V weight of the second vector Z 1,2 of the first intermediate vector sequence of the first coding sequence The attention value of the vector and the attention value of the V'weight vector of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence determine the first intermediate vector of the first coding sequence The first score of the second vector Z _{1,2 of the vector sequence;}

   The second coding module of the forward hidden sublayer of the first layer connects multiple scores _{of the second vector Z 1,2 of the first intermediate vector sequence of the first coding sequence to obtain the} The combined vector _{of the second vector Z 1,2} of the first intermediate vector sequence of the first coding sequence;

   The second coding module of the forward hidden sublayer of the first layer multiplies the combined vector of the _{second vector Z 1,2 of the first intermediate vector sequence of the first coding sequence by the fourth weight} Matrix to obtain the intermediate vector of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence;

   The feedforward network in the second coding module of the forward hidden sublayer of the first layer performs residual and normalization processing on the second vector Z of the first intermediate vector sequence of the first coding sequence. _{The intermediate vector of 1,2} is coded, and the normalization process is performed again to obtain the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence.
7. 5. The sentence sentiment classification method according to claim 1, wherein the sentence sentiment classification method further comprises:

   The U-th coding module of the n-th forward hidden sublayer of the feature extraction model converts the U-1th vector of the n-1 intermediate vector sequence of the first coding sequence, and the U-1th vector of the first coding sequence The U-th vector of the n-1 intermediate vector sequence and the W-th vector of the n-1 intermediate vector sequence of the second coding sequence are encoded as Z _{n, U} ; the W-th vector of the n-th layer backward hidden sublayer Encoding modules combine the W-1th vector of the n-1 intermediate vector sequences of the second encoding sequence, the Wth vector of the n-1 intermediate vector sequences of the second encoding sequence and the first The U-th vector of the n-1 intermediate vector sequence of the coding sequence is coded as R _n,W .
8. A sentence emotion classification device, wherein the device includes:

   An obtaining module, configured to obtain a first sentence sample set, each first sentence sample in the first sentence sample set contains a missing word;

   The conversion module is used for each first sentence sample, using a feature extraction model to convert the words before the missing word in the first sentence sample into a first word vector sequence in order of words, and convert the words in the first sentence sample The words after the missing word are converted into a second word vector sequence according to the reverse word order, and the missing words in the first sentence sample are converted into the label vector of the first sentence sample according to a preset vocabulary coding table;

   An encoding module, configured to use the feature extraction model to encode the first word vector sequence into a first encoding sequence, and to encode the second word vector sequence into a second encoding sequence;

   A calculation module, configured to use the feature extraction model to calculate the missing word vector of the first sentence sample according to the first coding sequence and the second coding sequence;

   The first training module is used to train the feature extraction model according to the missing word vector of the first sentence sample and the label vector of the first sentence sample to obtain a first feature extraction model, create a second feature extraction model, and make The neural network structure of the second feature extraction model is consistent with the neural network structure of the first feature extraction model, and the weight of the second feature extraction model is updated with the weight of the first feature extraction model;

   The second training module is used to train the attribute classification model composed of the first feature extraction model and the fully connected layer by using second sentence samples with attribute labels;

   The connection module is used to identify the attribute words of a plurality of sentences to be recognized using the attribute classification model, and connect each sentence to be recognized with the attribute words of each sentence to be recognized to obtain the plurality of connected attribute words Sentence to be recognized;

   The third training module is configured to train an emotion classification model composed of the second feature extraction model and a deep learning model by using the plurality of sentences to be recognized that are connected to attribute words with emotion labels;

   The classification module is used to identify the attribute words of the sentence to be processed using the attribute classification model, and the sentiment classification model classifies the sentence to be processed connecting the attribute words, and outputs the attribute words of the sentence to be processed and the sentence to be processed Type of emotion.
9. A computer device, wherein the computer device includes a processor, and the processor is configured to execute computer-readable instructions stored in a memory to implement the following steps:

   Acquiring a first sentence sample set, where each first sentence sample in the first sentence sample set contains a missing word;

   For each first sentence sample, use a feature extraction model to convert the words before the missing word in the first sentence sample into a first word vector sequence, and convert the words after the missing word in the first sentence sample Transforming into a second word vector sequence according to the reverse word order, and transforming the missing words in the first sentence sample into the label vector of the first sentence sample according to a preset vocabulary coding table;

   Coding the first word vector sequence into a first coding sequence and coding the second word vector sequence into a second coding sequence by using the feature extraction model;

   Using the feature extraction model to calculate the missing word vector of the first sentence sample according to the first coding sequence and the second coding sequence;

   Train the feature extraction model according to the missing word vector of the first sentence sample and the label vector of the first sentence sample to obtain a first feature extraction model, create a second feature extraction model, and make the second feature extraction model The neural network structure of is consistent with the neural network structure of the first feature extraction model, and the weight of the second feature extraction model is updated with the weight of the first feature extraction model;

   Training an attribute classification model composed of the first feature extraction model and a fully connected layer by using a second sentence sample with attribute labels;

   Identify the attribute words of a plurality of sentences to be recognized by using the attribute classification model, and connect each sentence to be recognized with the attribute words of each sentence to be recognized that are recognized to obtain the plurality of sentences to be recognized that connect the attribute words;

   Training an emotion classification model composed of the second feature extraction model and a deep learning model by using the plurality of sentences to be recognized that are connected attribute words with emotion labels;

   The attribute classification model is used to identify the attribute words of the sentence to be processed, and the emotion classification model classifies the sentence to be processed connecting the attribute words, and outputs the attribute words of the sentence to be processed and the emotion type of the sentence to be processed.
10. 9. The computer device of claim 9, wherein the feature extraction model includes an input layer, a forward hidden layer, a backward hidden layer, and an output layer.
11. The computer device according to claim 9, wherein the processor executes the computer-readable instructions stored in the memory to implement the use of a feature extraction model to convert the words before the missing words in the first sentence sample into word order The first word vector sequence, when the words after the missing word in the first sentence sample are converted into the second word vector sequence according to the reverse word order, include:

    Convert the words before the missing word in the first sentence sample into a first coding vector sequence in word order, and convert the words after the missing word in the first sentence sample into a second coding vector in word order sequence;

    Convert the position number of the word before the missing word in the first sentence sample into a first position vector sequence, and convert the position number of the word after the missing word in the first sentence sample into a second Position vector sequence;

    The first coding vector sequence and the first position vector sequence are converted into a first word vector sequence, and the second coding vector sequence and the second position vector sequence are converted into a second word vector sequence.
12. 9. The computer device according to claim 9, wherein the processor executes computer-readable instructions stored in the memory to implement the feature extraction model to encode the first word vector sequence into the first encoding sequence, include:

    The first encoding module of the forward hidden sublayer of the first layer of the feature extraction model calculates the first word vector of the first word vector sequence according to the first weight matrix subset in the initialized weight matrix set _{Coded as the first vector Z 1,1} of the first intermediate vector sequence of the first coding sequence, the initialized weight matrix set includes N weight matrix subsets, and the middle of the first coding sequence The vector sequence corresponds to the intermediate vector sequence of the second coding sequence in a one-to-one order, and the forward hidden sublayer of the nth layer and the backward hidden sublayer of the nth layer of the feature extraction model share the nth weight. Matrix subset, each weight matrix subset includes multiple sets of weight matrix and a fourth weight matrix, and each set of weight matrix includes V weight matrix, Q weight matrix, and K weight matrix;

    Starting from the second coding module of the forward hidden sublayer of the first layer, the u-th coding module of the forward hidden sublayer of the first layer divides the first coding module one by one according to the first weight matrix subset. The u-1th word vector of the word vector sequence and the u-th word vector of the first word vector sequence are coded as the u-th vector Z _1,u of the first intermediate vector sequence of the first coding sequence, Obtain the first intermediate vector sequence Z ₁ ={Z _1,1 ,...,Z _1,u ,...,Z _1,U } of the first coding sequence, where the first one of the first coding sequence The u-th vector of the intermediate vector sequence corresponds to the u-th word vector of the first word vector sequence in a one-to-one correspondence;

    Starting from the forward hidden sub-layer of the second layer of the feature extraction model, the n-th forward hidden sub-layer is used to calculate the n-1th of the first coding sequence according to the n-th weight matrix subset. intermediate vector Z _n-1 encoding sequence for said first sequence of n intermediate vector Z _n coding sequence.
13. The computer device according to claim 12, wherein the processor executes the computer-readable instructions stored in the memory to implement the first encoding module of the first layer forward hiding sublayer according to the initialized weight When the first weight matrix subset in the matrix set encodes the first word vector of the first word vector sequence into the first vector Z _{1,1 of the first intermediate vector sequence of the first encoding sequence} ,include:

    The first encoding module of the forward hidden sublayer of the first layer multiplies the first word vector of the first word vector sequence by the V in the multiple sets of weight matrixes in the first weight matrix subset. A weight matrix to obtain multiple V weight vectors of the first word vector of the first word vector sequence;

    Connecting multiple V weight vectors of the first word vector of the first word vector sequence to obtain a combined vector of the first word vector of the first word vector sequence;

    Multiply the combined vector of the first word vector of the first word vector sequence by the fourth weight matrix to obtain the first vector Z _{1,1 of the first intermediate vector sequence of the first coding sequence} .
14. The computer device of claim 12, wherein the processor executes computer-readable instructions stored in the memory to implement the u-th encoding module of the first layer forward hidden sublayer according to the first weight The value matrix subset encodes the u-1th word vector of the first word vector sequence and the u-th word vector of the first word vector sequence one by one into the first intermediate vector sequence of the first coding sequence When the u-th vector Z _1,u of the first coding sequence is obtained, the first intermediate vector sequence Z ₁ ={Z _1,1 ,...,Z _1,u ,...,Z _1,U } is obtained, including :

    The second encoding module of the forward hidden sublayer of the first layer multiplies the second word vector of the first word vector sequence by V in the first group of weight matrix in the first weight matrix subset. A weight matrix to obtain the V weight vector _{of the second vector Z 1,2} of the first intermediate vector sequence of the first coding sequence;

    The second encoding module of the forward hidden sublayer of the first layer multiplies the second word vector of the first word vector sequence by the Q in the first group of weight matrix in the first weight matrix subset. A weight matrix to obtain the Q weight vector _{of the second vector Z 1,2} of the first intermediate vector sequence of the first coding sequence;

    The second encoding module of the forward hidden sublayer of the first layer multiplies the second word vector of the first word vector sequence by K in the first group of weight matrix in the first weight matrix subset A weight matrix to obtain the K weight vector _{of the second vector Z 1,2} of the first intermediate vector sequence of the first coding sequence;

    The second encoding module of the forward hidden sublayer of the first layer multiplies the first word vector of the first word vector sequence by V in the first group of weight matrix in the first weight matrix subset. A weight matrix to obtain the V'weight vector of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence;

    The second encoding module of the forward hidden sublayer of the first layer multiplies the first word vector of the first word vector sequence by the first weight matrix in the first group of weight matrix subsets in the first weight matrix subset. Three K weight matrix to obtain the K'weight vector of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence;

    The second coding module of the forward hidden sublayer of the first layer is based on the Q weight vector of the second _{vector Z 1,2 of the first intermediate vector sequence of the first coding sequence, and the first coding sequence} The K weight vector of the second vector Z _1,2 of the first intermediate vector sequence of the sequence, the K'weight of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence The vector determines the _{attention value of the V weight vector of the second vector Z 1,2} of the first intermediate vector sequence of the first coding sequence and the second of the first intermediate vector sequence of the first coding sequence. The attention value of the V'weight vector of two vectors Z _1,2;

    The second coding module of the forward hidden sublayer of the first layer is based on the V weight vector of the second _{vector Z 1,2 of the first intermediate vector sequence of the first coding sequence, and the first coding sequence} V'weight vector of the second vector of the first intermediate vector sequence Z _{1,2 of the sequence,} and V weight of the second vector Z 1,2 of the first intermediate vector sequence of the first coding sequence The attention value of the vector and the attention value of the V'weight vector of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence determine the first intermediate vector of the first coding sequence The first score of the second vector Z _{1,2 of the vector sequence;}

    The second coding module of the forward hidden sublayer of the first layer connects multiple scores _{of the second vector Z 1,2 of the first intermediate vector sequence of the first coding sequence to obtain the} The combined vector _{of the second vector Z 1,2} of the first intermediate vector sequence of the first coding sequence;

    The second coding module of the forward hidden sublayer of the first layer multiplies the combined vector of the _{second vector Z 1,2 of the first intermediate vector sequence of the first coding sequence by the fourth weight} Matrix to obtain the intermediate vector of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence;

    The feedforward network in the second coding module of the forward hidden sublayer of the first layer performs residual and normalization processing on the second vector Z of the first intermediate vector sequence of the first coding sequence. _{The intermediate vector of 1,2} is coded, and the normalization process is performed again to obtain the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence.
15. 9. The computer device of claim 9, wherein the processor executes the computer-readable instructions stored in the memory to further implement the following steps:

    The U-th coding module of the n-th forward hidden sublayer of the feature extraction model converts the U-1th vector of the n-1 intermediate vector sequence of the first coding sequence, and the U-1th vector of the first coding sequence The U-th vector of the n-1 intermediate vector sequence and the W-th vector of the n-1 intermediate vector sequence of the second coding sequence are encoded as Z _{n, U} ; the W-th vector of the n-th layer backward hidden sublayer Encoding modules combine the W-1th vector of the n-1 intermediate vector sequences of the second encoding sequence, the Wth vector of the n-1 intermediate vector sequences of the second encoding sequence and the first The U-th vector of the n-1 intermediate vector sequence of the coding sequence is coded as R _n,W .
16. A computer storage medium having computer readable instructions stored thereon, wherein the computer readable instructions implement the following steps when executed by a processor:

    Acquiring a first sentence sample set, where each first sentence sample in the first sentence sample set contains a missing word;

    For each first sentence sample, use a feature extraction model to convert the words before the missing word in the first sentence sample into a first word vector sequence, and convert the words after the missing word in the first sentence sample Transforming into a second word vector sequence according to the reverse word order, and transforming the missing words in the first sentence sample into the label vector of the first sentence sample according to a preset vocabulary coding table;

    Coding the first word vector sequence into a first coding sequence and coding the second word vector sequence into a second coding sequence by using the feature extraction model;

    Using the feature extraction model to calculate the missing word vector of the first sentence sample according to the first coding sequence and the second coding sequence;

    Train the feature extraction model according to the missing word vector of the first sentence sample and the label vector of the first sentence sample to obtain a first feature extraction model, create a second feature extraction model, and make the second feature extraction model The neural network structure of is consistent with the neural network structure of the first feature extraction model, and the weight of the second feature extraction model is updated with the weight of the first feature extraction model;

    Training an attribute classification model composed of the first feature extraction model and a fully connected layer by using a second sentence sample with attribute labels;

    Using the attribute classification model to identify the attribute words of a plurality of sentences to be recognized, and connect each sentence to be recognized with the attribute words of each sentence to be recognized recognized to obtain the plurality of sentences to be recognized that connect the attribute words;

    Training an emotion classification model composed of the second feature extraction model and a deep learning model by using the plurality of sentences to be recognized that are connected attribute words with emotion labels;

    The attribute classification model is used to identify the attribute words of the sentence to be processed, and the emotion classification model classifies the sentence to be processed connecting the attribute words, and outputs the attribute words of the sentence to be processed and the emotion type of the sentence to be processed.
17. The computer storage medium of claim 16, wherein the computer-readable instructions are executed by the processor to implement the use of the feature extraction model to convert the words before the missing words in the first sentence sample into word order The first word vector sequence, when the words after the missing word in the first sentence sample are converted into the second word vector sequence according to the reverse word order, include:

    Convert the words before the missing word in the first sentence sample into a first coding vector sequence in word order, and convert the words after the missing word in the first sentence sample into a second coding vector in word order sequence;

    Convert the position number of the word before the missing word in the first sentence sample into a first position vector sequence, and convert the position number of the word after the missing word in the first sentence sample into a second Position vector sequence;

    The first coding vector sequence and the first position vector sequence are converted into a first word vector sequence, and the second coding vector sequence and the second position vector sequence are converted into a second word vector sequence.
18. 15. The computer storage medium according to claim 16, wherein, when the computer-readable instructions are executed by the processor to implement the feature extraction model to encode the first word vector sequence into the first encoding sequence, the method comprises:

    The first encoding module of the forward hidden sublayer of the first layer of the feature extraction model calculates the first word vector of the first word vector sequence according to the first weight matrix subset in the initialized weight matrix set _{Coded as the first vector Z 1,1} of the first intermediate vector sequence of the first coding sequence, the initialized weight matrix set includes N weight matrix subsets, and the middle of the first coding sequence The vector sequence corresponds to the intermediate vector sequence of the second coding sequence in a one-to-one order, and the forward hidden sublayer of the nth layer and the backward hidden sublayer of the nth layer of the feature extraction model share the nth weight. Matrix subset, each weight matrix subset includes multiple sets of weight matrix and a fourth weight matrix, and each set of weight matrix includes V weight matrix, Q weight matrix, and K weight matrix;

    Starting from the second coding module of the forward hidden sublayer of the first layer, the u-th coding module of the forward hidden sublayer of the first layer divides the first coding module one by one according to the first weight matrix subset. The u-1th word vector of the word vector sequence and the u-th word vector of the first word vector sequence are coded as the u-th vector Z _1,u of the first intermediate vector sequence of the first coding sequence, Obtain the first intermediate vector sequence Z ₁ ={Z _1,1 ,...,Z _1,u ,...,Z _1,U } of the first coding sequence, wherein the first one of the first coding sequence The u-th vector of the intermediate vector sequence corresponds to the u-th word vector of the first word vector sequence in a one-to-one correspondence;

    Starting from the forward hidden sub-layer of the second layer of the feature extraction model, the n-th forward hidden sub-layer is used to calculate the n-1th of the first coding sequence according to the n-th weight matrix subset. intermediate vector Z _n-1 encoding sequence for said first sequence of n intermediate vector Z _n coding sequence.
19. The computer storage medium of claim 18, wherein the computer-readable instructions are executed by the processor to implement the first encoding module of the first layer forward hiding sub-layer to concentrate according to the initialized weight matrix When the first weight matrix subset encodes the first word vector of the first word vector sequence as the first vector Z _1,1 of the first intermediate vector sequence of the first coding sequence, it includes :

    The first encoding module of the forward hidden sublayer of the first layer multiplies the first word vector of the first word vector sequence by the V in the multiple sets of weight matrixes in the first weight matrix subset. A weight matrix to obtain multiple V weight vectors of the first word vector of the first word vector sequence;

    Connecting multiple V weight vectors of the first word vector of the first word vector sequence to obtain a combined vector of the first word vector of the first word vector sequence;

    Multiply the combined vector of the first word vector of the first word vector sequence by the fourth weight matrix to obtain the first vector Z _{1,1 of the first intermediate vector sequence of the first coding sequence} .
20. The computer storage medium of claim 18, wherein the computer-readable instructions are executed by the processor to implement the u-th encoding module of the first layer forward hidden sublayer according to the first weight matrix The subset encodes the u-1th word vector of the first word vector sequence and the uth word vector of the first word vector sequence one by one into the first intermediate vector sequence of the first coding sequence. When u vectors Z _1,u are obtained, the first intermediate vector sequence Z _{1 of} the first coding sequence is obtained = {Z _1,1 ,...,Z _1,u ,...,Z _1,U }, including:

    The second encoding module of the forward hidden sublayer of the first layer multiplies the second word vector of the first word vector sequence by V in the first group of weight matrix in the first weight matrix subset. A weight matrix to obtain the V weight vector _{of the second vector Z 1,2} of the first intermediate vector sequence of the first coding sequence;

    The second encoding module of the forward hidden sublayer of the first layer multiplies the second word vector of the first word vector sequence by the Q in the first group of weight matrix in the first weight matrix subset. A weight matrix to obtain the Q weight vector _{of the second vector Z 1,2} of the first intermediate vector sequence of the first coding sequence;

    The second encoding module of the forward hidden sublayer of the first layer multiplies the second word vector of the first word vector sequence by K in the first group of weight matrix in the first weight matrix subset A weight matrix to obtain the K weight vector _{of the second vector Z 1,2} of the first intermediate vector sequence of the first coding sequence;

    The second encoding module of the forward hidden sublayer of the first layer multiplies the first word vector of the first word vector sequence by V in the first group of weight matrix in the first weight matrix subset. A weight matrix to obtain the V'weight vector of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence;

    The second encoding module of the forward hidden sublayer of the first layer multiplies the first word vector of the first word vector sequence by the first weight matrix in the first group of weight matrix subsets in the first weight matrix subset. Three K weight matrix to obtain the K'weight vector of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence;

    The second coding module of the forward hidden sublayer of the first layer is based on the Q weight vector of the second _{vector Z 1,2 of the first intermediate vector sequence of the first coding sequence, and the first coding sequence} The K weight vector of the second vector Z _1,2 of the first intermediate vector sequence of the sequence, the K'weight of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence The vector determines the _{attention value of the V weight vector of the second vector Z 1,2} of the first intermediate vector sequence of the first coding sequence and the second vector of the first intermediate vector sequence of the first coding sequence. The attention value of the V'weight vector of two vectors Z _1,2;

    The second coding module of the forward hidden sublayer of the first layer is based on the V weight vector of the second _{vector Z 1,2 of the first intermediate vector sequence of the first coding sequence, and the first coding sequence} V'weight vector of the second vector of the first intermediate vector sequence Z _{1,2 of the sequence,} and V weight of the second vector Z 1,2 of the first intermediate vector sequence of the first coding sequence The attention value of the vector and the attention value of the V'weight vector of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence determine the first intermediate vector of the first coding sequence The first score of the second vector Z _{1,2 of the vector sequence;}

    The second coding module of the forward hidden sublayer of the first layer connects multiple scores _{of the second vector Z 1,2 of the first intermediate vector sequence of the first coding sequence to obtain the} The combined vector _{of the second vector Z 1,2} of the first intermediate vector sequence of the first coding sequence;

    The second coding module of the forward hidden sublayer of the first layer multiplies the combined vector of the _{second vector Z 1,2 of the first intermediate vector sequence of the first coding sequence by the fourth weight} Matrix to obtain the intermediate vector of the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence;

    The feedforward network in the second coding module of the forward hidden sublayer of the first layer performs residual and normalization processing on the second vector Z of the first intermediate vector sequence of the first coding sequence. _{The intermediate vector of 1,2} is coded, and the normalization process is performed again to obtain the second vector Z _1,2 of the first intermediate vector sequence of the first coding sequence.

Images