CN111144127B

CN111144127B - Text semantic recognition method, text semantic recognition model acquisition method and related device

Info

Publication number: CN111144127B
Application number: CN201911360687.9A
Authority: CN
Inventors: 肖飞; 宋时德; 胡加学; 赵乾
Original assignee: iFlytek Co Ltd
Current assignee: iFlytek Co Ltd
Priority date: 2019-12-25
Filing date: 2019-12-25
Publication date: 2023-07-25
Anticipated expiration: 2039-12-25
Also published as: CN111144127A

Abstract

The application discloses a text semantic recognition method, a method for acquiring a model of the text semantic recognition method and a related device, wherein the text semantic recognition method comprises the following steps: screening text rules matched with the text to be identified from a plurality of text rules, wherein the text rules are obtained by analyzing a plurality of sample texts marked with text semantics; acquiring a keyword sequence of the text to be recognized based on the matched text rule and the text to be recognized; inputting the text to be recognized and the keyword sequence into a text semantic recognition model obtained through training for semantic understanding, and obtaining text semantics of the text to be recognized; the text semantic recognition model is obtained by training a preset neural network by using a plurality of training texts marked with text semantics and keyword sequences thereof. By the aid of the scheme, accuracy of text semantic recognition can be improved.

Description

Text semantic recognition method, text semantic recognition model acquisition method and related device

Technical Field

The present invention relates to the field of natural language processing technologies, and in particular, to a text semantic recognition method, a method for obtaining a model thereof, and a related device.

Background

With the rapid development of internet technology and the increasing of informatization degree, a machine is adopted to carry out semantic recognition on texts, so that labor investment and mass data sharing are reduced, and the method becomes a research hot spot gradually.

At present, training texts belonging to different text semantics are mainly used for training a neural network, so that the trained neural network is used for carrying out subsequent semantic recognition tasks. However, in the training process, there is a high possibility that the number of training texts belonging to a certain text semantic is too large, and the number of training texts belonging to another text semantic is too small, so that the accuracy of the model obtained by training and the accuracy of the subsequent semantic recognition are reduced. In view of this, how to improve the accuracy of text semantic recognition is a problem to be solved.

Disclosure of Invention

The technical problem to be solved mainly by the application is to provide a text semantic recognition method, a method for acquiring a model of the text semantic recognition method and a related device, so that the accuracy of text semantic recognition can be improved.

In order to solve the above problems, a first aspect of the present application provides a text semantic recognition method, including: screening text rules matched with the text to be identified from a plurality of text rules, wherein the text rules are obtained by analyzing a plurality of sample texts marked with text semantics; acquiring a keyword sequence of the text to be recognized based on the matched text rule and the text to be recognized; inputting the text to be recognized and the keyword sequence into a text semantic recognition model obtained through training for semantic understanding, and obtaining text semantics of the text to be recognized; the text semantic recognition model is obtained by training a preset neural network by using a plurality of training texts marked with text semantics and keyword sequences thereof.

In order to solve the above problem, a second aspect of the present application provides a method for obtaining a text semantic recognition model, including: screening text rules matched with each training text from a plurality of text rules, wherein the text rules are obtained by analyzing a plurality of sample texts marked with text semantics; based on text rules matched with each training text and the training text, respectively acquiring a keyword sequence of each training text; and respectively inputting each training text and the corresponding keyword sequence into a preset neural network for training until the preset training ending condition is met, so as to obtain a text semantic recognition model.

In order to solve the above problem, a third aspect of the present application provides a text semantic recognition device, which includes a memory and a processor coupled to each other, where the memory stores program instructions, and the processor is configured to execute the program instructions to implement the text semantic recognition method in the first aspect.

In order to solve the above problem, a fourth aspect of the present application provides an apparatus for acquiring a text semantic recognition model, which includes a memory and a processor coupled to each other, where the memory stores program instructions, and the processor is configured to execute the program instructions to implement the method for acquiring a text semantic recognition model in the second aspect.

In order to solve the above-mentioned problems, a fifth aspect of the present application provides a storage device storing program instructions executable by a processor for implementing the text semantic recognition method in the above-mentioned first aspect or for implementing the obtaining method of the text semantic recognition model in the above-mentioned second aspect.

According to the scheme, the text rules matched with the text to be recognized are screened from the texts, so that the keyword sequence of the text to be recognized is obtained based on the matched text rules and the text to be recognized, the text to be recognized and the keyword sequence are input into the text semantic recognition model obtained through training for semantic understanding, the text semantic of the text to be recognized is obtained, the text semantic recognition model is obtained by training the preset neural network by utilizing the plurality of training texts marked with the text semantic and the keyword sequence thereof, the text rules can be transmitted into the preset neural network in the training stage through the keyword sequence, the neural network is guided to learn rule information with few sample types better, the problem of sample imbalance can be solved, the text semantic recognition accuracy is improved, in addition, in the recognition stage, the text rules of the text to be recognized can be transmitted into the text semantic recognition model through the keyword sequence, and the text semantic recognition accuracy can be further improved.

Drawings

FIG. 1 is a flow chart of an embodiment of a method for obtaining a text semantic recognition model of the present application;

FIG. 2 is a schematic diagram of an embodiment of the neural network shown in FIG. 1;

FIG. 3 is a flowchart illustrating an embodiment of step S13 in FIG. 1;

FIG. 4 is a flowchart illustrating an embodiment of the step S132 in FIG. 3;

FIG. 5 is a diagram of an embodiment of the encoding process performed by the self-attention mechanism layer of FIG. 2;

FIG. 6 is a flow diagram of one embodiment of a semantic recognition method of the present application;

FIG. 7 is a flow diagram of one embodiment of classifying, by an output layer, a first feature vector and a second feature vector to output text semantics of a text to be recognized;

FIG. 8 is a schematic diagram of a framework of an embodiment of a text semantic recognition device of the present application;

FIG. 9 is a schematic diagram of a framework of an embodiment of an acquisition device for text semantic recognition models of the present application;

FIG. 10 is a schematic diagram of a frame of an embodiment of a storage device of the present application.

Detailed Description

The following describes the embodiments of the present application in detail with reference to the drawings.

In the following description, for purposes of explanation and not limitation, specific details are set forth such as the particular system architecture, interfaces, techniques, etc., in order to provide a thorough understanding of the present application.

The terms "system" and "network" are often used interchangeably herein. The term "and/or" is herein merely an association relationship describing an associated object, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone. In addition, the character "/" herein generally indicates that the front and rear associated objects are an "or" relationship. Further, "a plurality" herein means two or more than two.

Referring to fig. 1, fig. 1 is a flowchart illustrating an embodiment of a method for obtaining a text semantic recognition model according to the present application. Specifically, the method may include the steps of:

step S11: text rules matching each training text are screened from a plurality of text rules respectively.

In this embodiment, the plurality of text rules are obtained by analyzing a plurality of sample texts labeled with text semantics. Text semantic recognition may include, but is not limited to: the intention recognition task, the classification task, and the translation task are not limited herein. Taking intent recognition as an example, the sample text may be dialog text collected in an online man-machine conversation, or dialog text crawled from a web page; the true intent of each dialog text can thus be annotated, for example: booking air tickets, checking weather, booking hotels, checking telephones, booking tickets, checking routes, and the like. Other tasks than the intended recognition task may be analogized, and the present embodiment is not illustrated here. Specifically, the text rule may be a regular expression, for example, taking text semantics as an example for booking an air ticket, the text rule may include: (booking) (from) @ place name (to) @ place name (air ticket). Further, multiple text rules may be included for the same text semantic. Still taking text semantics as an example for booking an air ticket, the text rules may further include: (to) (one|number) @ place name (to) @ place name (air ticket). The text rules mentioned above are merely examples, and may be obtained according to actual sample text analysis when applied in specific applications, and are not limited herein.

Further, the plurality of sample texts may be the same text set as the plurality of training texts, that is, the plurality of sample texts may be included in the plurality of training texts; alternatively, the plurality of training texts may be included in a plurality of sample texts, for example, the plurality of sample texts are divided into a plurality of text subsets according to the labeled text semantics, and then a plurality of texts are respectively extracted from each text subset, so that the extracted plurality of texts are used as a plurality of training texts; alternatively, the plurality of sample texts and the plurality of training texts may be different text sets, which is not limited herein.

Step S12: and respectively acquiring a keyword sequence of each training text based on the text rule matched with each training text and the training text.

In this embodiment, each training text may be matched to obtain a text rule, or may be matched to obtain a plurality of text rules, and accordingly, each training text may obtain a keyword sequence, or may obtain a plurality of keyword sequences, which is not limited herein.

Still taking intention recognition as an example, for a training text "help me reserve air ticket from Beijing to Shanghai", the text rule "(reserve) (from) @ place name (to) @ place name (air ticket)", the keyword sequence corresponding to the training text can be obtained according to the training text and the text rule obtained by matching: (booking, from, beijing, to, shanghai, air ticket). Or, for the training text "i want to order an air ticket from beijing to Shanghai", the text rule "(to be) (one|number) @ place name (to) @ place name (air ticket)", the keyword sequence corresponding to the training text can be obtained according to the training text and the text rule obtained by matching: (want, one, beijing, get, shanghai, air ticket). Or, for the training text "i want to subscribe to a ticket from beijing to Shanghai", the text rule "(book) (from) @ place name (to) @ place name (ticket)", or the text rule "(want) (one|number) @ place name (to) @ place name (ticket)") may be obtained by matching, and then the keyword sequence corresponding to the training text may be obtained according to the training text and the text rule matched with the training text: (want, one, beijing, arrive, shanghai, air ticket), (booking, from, beijing, arrive, shanghai, air ticket). Furthermore, the training text may also be matched to text rules that differ from the text semantics of the annotation, that is, the text semantics of the training text annotation differ from the text semantics to which the matched text rules pertain. For example, for the training text "i am et al meeting reservation of tickets from beijing to shanghai on tomorrow," i am inquire about weather conditions of tomorrow shanghai, "the above text rule" (reservation) (from) @ place name (to) @ place name (ticket) ", and further, a keyword sequence corresponding to the training text may be obtained: (booking, beijing, going, shanghai, air ticket), but the text semantic of the training text is "weather searching", but not the text semantic of the text rule which is obtained by matching is "air ticket booking", in addition, the training text can also be matched to obtain the text rule (query) and place name (weather) which are obtained by matching, so that keyword sequences (query, shanghai, weather) corresponding to the text rule are obtained.

In one implementation scenario, in order to further alleviate the problem of sample imbalance, the matching requirement can be reduced to match to more text rules under the condition that the number of training texts corresponding to the text semantics to which the matched text rules belong is smaller; or, under the condition that the number of training texts corresponding to the text semantics of the matched text rules is large, a plurality of text rules can be removed from the matched text rules so as to be matched with a few text rules. Specifically, the number of texts of the training texts corresponding to each text semantic and the text semantic to which each text rule belongs may be counted, and the text semantic to which the text rule belongs may be the same as the text semantic of the sample text label, for example, the text rule obtained by analysis from the sample text labeled "ticket booking" may also be "ticket booking". On the basis, text semantics of the matched text rules can be obtained, text quantity of training texts corresponding to the text semantics can be obtained, if the text quantity is smaller than a first preset quantity threshold (for example, 200 and 300), more text rules can be matched for the training texts, specifically, text rules meeting the first preset conditions can be selected from a plurality of text rules, and keyword sequences of the training texts can be obtained based on the selected text rules, the matched text rules and the training texts; otherwise, if the number of texts is greater than a second preset number threshold (e.g., 500, 600), and the matched text rule includes a plurality of text rules with the same text semantics, for example, the matched text rule has text semantics of the plurality of text rules all being "ticket booking", then text rules with less text semantics can be matched for the training text, specifically, when the plurality of text rules with the same text semantics conform to a second preset condition, any text rule can be selected from the plurality of text rules with the same text semantics, and based on the selected text rule and training text, a keyword sequence of the training text is obtained.

In a specific implementation scenario, the first preset condition may include at least one of the following: the text semantic of the selected text rule is the same as the text semantic of the matched text rule, and the word overlap ratio between the selected text rule and the training text is larger than a first preset overlap ratio threshold; the selected text rule and the matched text rule are co-occurrence rules, and the co-occurrence frequency is larger than a first preset frequency threshold. For example, if the training text matches the text rule a, the text rule B and the text rule C having a word overlap ratio with the training text greater than a first predetermined overlap ratio threshold (e.g., 50%, 60%, 70%, etc.) may be selected from a plurality of text rules belonging to the same text semantic as the text rule a. Or, for example, the selected text rule D and the matched text rule a are co-occurrence rules, and the co-occurrence frequency is greater than a first preset frequency threshold (for example, 600, 700, 800, etc.), in this embodiment, if a plurality of text rules can be matched to identical training texts, the text rules can be considered as co-occurrence rules, and the number of identical training texts is the co-occurrence frequency, for example, the text rule a can be matched with 600 training texts, and the 600 training texts can also be matched with the text rule D, the text rule a and the text rule D can be considered as co-occurrence rules, and the co-occurrence frequency is 600. Or, for example, a text rule B and a text rule C with a word overlap ratio greater than a first preset overlap ratio threshold value with the training text may be selected from a plurality of text rules belonging to the same text semantic as the matched text rule a, and a text rule D with a co-occurrence frequency greater than a first preset frequency threshold value with the matched text rule a may be selected, so that a keyword sequence of the training text may be obtained based on the matched text rule a, the selected text rule B, C, D and the training text.

In another specific implementation scenario, the second preset condition may include at least one of the following: word overlap ratio between keyword sequences acquired based on text rules and training texts with the same meanings of a plurality of affiliated texts is larger than a second preset overlap ratio threshold; the text rules with the same meanings of the texts are co-occurrence rules, and the co-occurrence frequency is larger than a second preset frequency threshold. For example, the training text is matched to the text rule E, F, G belonging to the same text semantic, and in the word overlap ratio between the three and the keyword sequences obtained by the training text, the word overlap ratio corresponding to the text rule E and the text rule F is greater than a second preset overlap ratio threshold (for example, 90% and 95%), and then one text rule is selected from the text rule E and the text rule F. Or, for example, training the text to match to the text rule E, F, G belonging to the same text semantic, and the text rule E and the text rule G are co-occurrence rules, and the co-occurrence frequency is greater than a second preset frequency threshold (e.g., 900, 1000, etc.), selecting one text rule from the text rule E and the text rule G. Or, for example, the training text is matched to a text rule E, F, G belonging to the same text semantic, and in the word overlap ratio between the keyword sequences obtained respectively based on the three text rules and the training text, the word overlap ratio corresponding to the text rule E and the text rule F is greater than a second preset overlap ratio threshold, the text rule E and the text rule G are co-occurrence rules, and the co-occurrence frequency is greater than a second preset frequency threshold, and then the text rule E can be selected from the three text rules.

The specific values of the first preset number threshold, the second preset number threshold, the first preset overlap ratio threshold, the second preset overlap ratio threshold, the first preset frequency threshold, and the second preset frequency threshold are only examples, and may be set according to actual situations when the method is applied specifically, which is not limited specifically herein.

Step S13: and respectively inputting each training text and the corresponding keyword sequence into a preset neural network for training until the preset training ending condition is met, so as to obtain a text semantic recognition model.

Referring to fig. 2 in combination, fig. 2 is a schematic diagram illustrating an embodiment of the neural network shown in fig. 1. The preset neural network may include an input layer, a long-short term memory network layer, a self-attention mechanism layer, and an output layer connected in sequence. After the training text and the corresponding keyword sequence are input into a preset neural network, word segmentation and vectorization are carried out on the input layer of the preset neural network, coding is carried out on the long-short-term memory network layer and the self-attention mechanism layer, coding feature vectors are obtained after feature vector combination and splicing are carried out on the output layer, each text semantic is output through the full-connection layer, probability values of the training text belonging to each text semantic are obtained through a Softmax function, so that loss values can be calculated, the preset neural network can be trained in a preset training mode, network parameters of the preset neural network can be adjusted by the calculated loss values, training is carried out again until preset training end conditions are met, and a text semantic recognition model is obtained. The specific processes of word segmentation and vectorization, coding processing, vector merging and splicing can refer to the relevant steps in the following embodiments, which are not described herein in detail.

In one implementation scenario, the preset training end condition may include: the loss value is less than the preset loss value threshold and is no longer reduced. In one implementation scenario, the network parameters may include, but are not limited to: hidden layer weight of long-short term memory network layer, related parameters of self-attention mechanism layer, and related parameters of full connection layer in output layer.

According to the scheme, the plurality of sample texts marked with the text semantics are analyzed in advance to obtain the plurality of text rules, and further, in the text semantic recognition process, the text rules matched with the texts to be recognized can be screened from the plurality of texts, so that the keyword sequences of the texts to be recognized are obtained based on the matched text rules and the texts to be recognized, further, the texts to be recognized and the keyword sequences are input into a text semantic recognition model obtained through training to carry out semantic understanding, the text semantics of the texts to be recognized are obtained, the text semantic recognition model is obtained by training a preset neural network by utilizing the plurality of training texts marked with the text semantics and the keyword sequences thereof, the text rules can be transmitted into the preset neural network in the training stage through the keyword sequences, and therefore, the neural network can be guided to learn rule information of few sample categories better, and further, the problem of sample imbalance can be relieved, and the accuracy of text semantic recognition can be improved.

Referring to fig. 3, fig. 3 is a flowchart illustrating an embodiment of step S13 in fig. 1. Specifically, the method may include the steps of:

step S131: and aiming at each training text in the training texts, acquiring a keyword sequence with the same text semantic as the training text from a plurality of keyword sequences of the training text as a positive keyword sequence, and acquiring a keyword sequence with different text semantic from the training text as a negative keyword sequence.

For example, in the training text listed in the above embodiment, the keyword sequence (booking, beijing, to, shanghai, air ticket) is the forward keyword sequence of the training text "help me book air ticket from Beijing to Shanghai"; the keyword sequence (Beijing, shanghai, air ticket) is the forward keyword sequence of the training text "I want an air ticket from Beijing to Shanghai"; the keyword sequence (important, one, beijing, to, shanghai, air ticket), (booking, from, beijing, to, shanghai, air ticket) is the forward keyword sequence of training text "I want to book an air ticket from Beijing to Shanghai"; the keyword sequence (booking, beijing, going to, shanghai, air ticket) is a negative keyword sequence of training text "I'm etc. meeting is to book an air ticket from Beijing to Shanghai on tomorrow, I'm inquire about weather conditions of Shanghai on tomorrow", the keyword sequence (inquiry, shanghai, weather) is a positive keyword sequence of training text "I'm etc. meeting is to book an air ticket from Beijing to Shanghai on tomorrow, I'm inquire about weather conditions of Shanghai on tomorrow". In other application scenarios, the same can be said, and are not exemplified here.

Step S132: and acquiring a loss value and a loss weight corresponding to the training text based on the positive keyword sequence and the negative keyword sequence.

In this embodiment, a union of forward keyword sequences of the training text may be obtained first, then the word length of the obtained union and the word length of the training text may be calculated, and the ratio between the word length of the union and the word length of the training text may be used as the loss weight Pcov corresponding to the training text.

In one implementation scenario, only one forward keyword sequence corresponding to the training text is provided, and the union is the union. Taking the training text "help me reserve air ticket from Beijing to Shanghai" as an example in the above embodiment, the forward keyword sequence is (reserve, from, beijing, to, shanghai, air ticket), the word length of the training text is 13, and the word length of the forward keyword sequence is 10, the loss weight of the training text "help me reserve air ticket from Beijing to Shanghai" is 10/13.

In another implementation scenario, the training text corresponds to a plurality of forward keyword sequences, and then the union of the forward keyword sequences is a set of all elements of the forward keyword sequences. Taking the training text "i want to subscribe to a ticket from Beijing to Shanghai" in the above embodiment as an example, the forward keyword sequence is (want, one, beijing, arrive, shanghai, ticket), (subscribe, from, beijing, arrive, shanghai, ticket), the union of the two is (want, subscribe, one, from, beijing, arrive, shanghai, ticket), the word length is 13, the word length of the training text is 15, and the loss weight of the training text "i want to subscribe to a ticket from Beijing to Shanghai" is 13/15. Other implementation scenarios may be similar, and are not described herein.

In the training process, besides the loss weight corresponding to the training text, the loss value corresponding to the training text needs to be calculated. In order to improve the accuracy of the loss value, and further improve the accuracy of the text semantic recognition model obtained later, besides the cross entropy loss value, the self-attention loss value brought by the self-attention mechanism layer can be further considered. Referring to fig. 4 in combination, fig. 4 is a flowchart illustrating an embodiment of step S132 in fig. 3. Specifically, fig. 4 is a flowchart of an embodiment of obtaining a loss value corresponding to each training text based on a positive keyword sequence and a negative keyword sequence, which includes the following steps:

step S1321: and respectively acquiring a first probability distribution value of each word in the positive keyword sequence and a second probability distribution value of each word in the negative keyword sequence.

For easy understanding, the above-described processes of word segmentation and vectorization, and encoding processing will be described first. Referring to fig. 2 in combination, after obtaining the training text and the corresponding keyword sequences (including the positive keyword sequences and/or the negative keyword sequences), in order to unify the text and the lengths of the sequences for subsequent encoding, the text length may be defined as a preset text length (for example, 60 words) and the sequence length may be defined as a preset sequence length (for example, 40 words), and if the length is insufficient, the preset complement word may be used for complement. On this basis, word segmentation and vectorization processing (for example, word2vec, glove, fasttext is used to perform word segmentation and vectorization, which will not be described herein), so as to obtain a text word vector [ x1 x2 x3 … xn ] corresponding to the training text and a sequence word vector [ f1 f2 f3 … fm ] corresponding to the keyword sequence, and input the text word vector and the sequence word vector into an input layer shown in fig. 2, where n represents a preset text length and m represents a preset sequence length. In this embodiment, the dimension of the word vector may be a predetermined dimension (e.g., 100), and for training text, the text word vector input to the long-short term memory network layer may be considered as a two-dimensional matrix of (60, 100), and the sequence word vector input to the long-short term memory network layer may be considered as a two-dimensional matrix of (40, 100).

The text word vector and the sequence word vector are subjected to coding processing through a long-short-term memory network layer and a self-attention mechanism layer to obtain a text feature vector corresponding to the training text and a sequence feature vector corresponding to the keyword sequence, then the text feature vector and the sequence feature vector are subjected to classification processing through an output layer, and the probability value of each text semantic of the training text is obtained through prediction.

Specifically, the text word vector and the sequence word vector are subjected to a first encoding process through a long-short-term memory network layer, so that a text hidden layer vector [ h1 h2 h3 … hn ] corresponding to the training text and a sequence hidden layer vector [ fh1 fh2 fh3 … fhm ] corresponding to the keyword sequence can be obtained respectively. The text hidden layer vector and the sequence hidden layer vector are respectively subjected to second coding processing through a self-attention mechanism layer, so that a text feature vector corresponding to a training text and a sequence feature vector corresponding to a keyword sequence can be obtained.

Referring to fig. 5 in combination, fig. 5 is a schematic diagram illustrating an embodiment of the coding process performed by the self-attention mechanism layer in fig. 2. Taking the forward keyword sequence as an example, the sequence hidden layer vector corresponding to the forward keyword sequence may be a two-dimensional matrix I of m×l, that is, corresponding to M words, where each word is in L dimensions. The sequence hidden layer vector is firstly subjected to nonlinear transformation sigma (Wx+b) in the self-attention mechanism layer, so that a two-dimensional matrix I' of M x L can be obtained, wherein sigma (·) can be an activation function such as sigmoid or tanh. And carrying out inner product on the two-dimensional matrix I' and an L-dimensional external variable U to obtain an M-dimensional vector V, and calculating the vector V obtained by the inner product by utilizing a Softmax function to obtain a first probability distribution value P (wi) of each word in the forward keyword sequence, wherein I represents an ith word in the forward keyword sequence. The process of calculating the second probability distribution P (xi) for each word in the negative keyword sequence may be analogized, where i represents the i-th word in the negative keyword sequence, and will not be described in detail herein. The weight control of the input hidden layer vector can be realized by introducing an external variable U to carry out inner product with the hidden layer vector. In addition, weight redistribution can be achieved by calculating the vector V obtained by the inner product through the Softmax function, then the obtained first probability distribution value P1 and the hidden layer vector I corresponding to the positive keyword sequence are subjected to dot multiplication, a sequence feature vector corresponding to the positive keyword sequence can be obtained, and the obtained second probability distribution value and the hidden layer vector corresponding to the negative keyword sequence are subjected to dot multiplication, so that a sequence feature vector corresponding to the negative keyword sequence can be obtained. Through the coding process of the self-attention mechanism layer, important words in the keyword sequence can be highlighted, and the influence of non-important words is reduced.

After obtaining the text feature vector corresponding to the training text and the sequence feature vector corresponding to the keyword sequence, the sequence feature vector corresponding to the keyword sequence may be combined, specifically, a positive keyword sequence vector sum of the sequence feature vector corresponding to the positive keyword sequence and a negative keyword sequence vector sum of the sequence feature vector corresponding to the negative keyword sequence may be obtained, and then the positive keyword sequence vector sum and the negative keyword sequence vector sum are subtracted to obtain a combined vector corresponding to the keyword sequence. On the basis, the merging vector and the text feature vector corresponding to the training text are spliced to obtain the coding feature vector corresponding to the training text. By subtracting the positive keyword sequence vector sum from the negative keyword sequence vector sum, the information that the training text belongs to the text semantic of the training text can be enhanced, and the information irrelevant to the text semantic of the training text is eliminated, so that the preset neural network can learn the key information of the text semantic of the training text better.

After the coding feature vector is obtained, all text semantics are output through full connection processing in an output layer, probability values of the training text belonging to all the text semantics are obtained through a Softmax function, and then the cross entropy loss value is obtained through calculation by using the probability values obtained through prediction and the text semantics marked by the training text.

Step S1322: and performing self-attention loss calculation on the first probability distribution value and the second probability distribution value to obtain the self-attention loss value of the training text.

Specifically, the forward penalty value loss can be calculated for each forward keyword sequence using the following formula 1 ₁ And calculate each negative keyword sequence to obtain a negative loss value loss by using the following formula 2 ₂ 。

Wherein p (wi) is a first probability assignment value for the i-th word in a positive keyword sequence, n is the total number of words in a positive keyword sequence, p (xi) is a second probability assignment value for the i-th word in a negative keyword sequence, and m is the total number of words in a negative keyword sequence.

On the basis, the forward loss value loss of each forward keyword sequence ₁ And a negative loss value loss for each negative keyword sequence ₂ Adding to obtain the self-attention loss value loss of the training text _att 。

Step S1323: and obtaining the loss value corresponding to the training text based on the cross entropy loss value and the self-attention loss value of the training text.

In this embodiment, the cross entropy loss value loss _model The text semantic of the true annotation of the training text can be obtained by using the negative log likelihood function of the classifier under the condition of the text semantic of the true annotation of the training text, and the description is omitted here.

In an implementation scenario, in order to facilitate adjustment of importance of the cross entropy loss value and the self-attention loss value in the training process according to actual conditions, a first preset weight may be correspondingly set for the cross entropy loss value, a second preset weight may be correspondingly set for the self-attention loss value, and then the cross entropy loss value and the self-attention loss value are respectively weighted by using the first loss sub-weight and the second loss sub-weight to obtain the loss value corresponding to the training text. For example, if the sum of the first loss sub-weight and the second loss sub-weight is 1, the weighting process may be a weighted sum.

By performing the above-described step S131, step S132, and related steps for each of a plurality of training texts, a penalty value and a penalty weight corresponding to each training text can be obtained.

Step S133: and weighting the loss value of the corresponding training text by using the loss weight of each training text to obtain the loss values of a plurality of training texts.

Specifically, the loss value loss of a plurality of training texts in one training process can be obtained by the following formula _all ：

Wherein N represents the number of training texts and loss _i Representing a loss value, pcov, corresponding to an ith training text in a plurality of training texts _i And representing the loss weight corresponding to the ith training text in the plurality of training texts.

Step S134: and adjusting network parameters of a preset neural network by using the loss values of the training texts until the preset training ending condition is met.

In this embodiment, the network parameters of the preset neural network may include, but are not limited to: parameters W, b in the self-attention mechanism layer, hidden layer weights in the long-short-term memory network layer, and related parameters in the full-connection layer in the output layer.

Different from the foregoing embodiment, in the multiple keyword sequences of each training text, a keyword sequence having the same text semantic meaning as the text semantic meaning of the training text is obtained as a positive keyword sequence, and a keyword sequence having different text semantic meaning as a negative keyword sequence is obtained, so that, based on the positive keyword sequence and the negative keyword sequence, a loss value and a loss weight corresponding to each training text are obtained, and further, the loss value of the corresponding training text is weighted by using the loss weight of each training text, so as to obtain loss values of the multiple training texts, and the network parameters of the preset neural network are adjusted by using the loss values of the multiple training texts until the preset training end condition is satisfied, which is favorable for the preset neural network to learn the key information of the text semantic meaning of the training text, thereby improving the accuracy of the training-obtained text semantic recognition model.

Referring to fig. 6, fig. 6 is a flowchart illustrating an embodiment of a text semantic recognition method according to the present application. Specifically, the method may include the steps of:

step S61: text rules matching the text to be identified are screened from a plurality of text rules.

In this embodiment, the plurality of text rules are obtained by analyzing a plurality of sample texts labeled with text semantics. Reference may be made specifically to the relevant steps in the foregoing embodiments, and details are not repeated here.

In this embodiment, the text to be recognized is text that needs text semantic recognition. In one implementation scenario, the text to be recognized may be obtained through human-machine text chat. In another implementation scenario, the text to be recognized may also be obtained by converting the voice information input by the user into text, which is not particularly limited herein.

In this embodiment, the text to be identified may be matched to one text rule or may be matched to a plurality of text rules. Still taking the task of intention recognition as an example, for the text to be recognized, the real text semantic of which is "check weather", i want to subscribe to the air ticket from Shanghai to Beijing, i ask for the weather condition of next monday Beijing, and can be matched with the text rule "(subscribe) (from) @ place name (to) @ place name (air ticket)", and can also be matched with the text rule "(query) @ place name (weather)". Other application scenarios can be analogized, and this embodiment is not illustrated here.

Step S62: and acquiring a keyword sequence of the text to be recognized based on the matched text rule and the text to be recognized.

Still taking the task of intention recognition as an example, for the text to be recognized, the real text semantic of which is "check weather", i want to subscribe to the air ticket from Shanghai to Beijing, i ask for the weather condition of next monday Beijing, and can be matched with the text rule "(subscribe) (from) @ place name (to) @ place name (air ticket)", and can also be matched with the text rule "(query) @ place name (weather)". On this basis, according to the text rule "(booking) (from) @ place name (to) @ place name (air ticket)") and the text to be recognized, the keyword sequence (query, beijing, weather) thereof can be acquired, and according to the text rule "(booking) (from) @ place name (to) @ place name (air ticket)", the keyword sequence (booking, from, shanghai, to, beijing, air ticket) thereof can be acquired.

In one implementation scenario, in order to further alleviate the problem of sample imbalance in the training process, the matching requirement can be reduced to match to more text rules under the condition that the number of training texts corresponding to the text semantics to which the matched text rules belong is smaller; or, under the condition that the number of training texts corresponding to the text semantics of the matched text rules is large, a plurality of text rules can be removed from the matched text rules so as to be matched with a few text rules. Specifically, the text quantity of training texts corresponding to each text semantic and the text semantic to which each text rule belongs can be counted, the text semantic to which the matched text rule belongs is obtained, the text quantity of training texts corresponding to the text semantic to which the text rule belongs is obtained, if the text quantity is smaller than a first preset quantity threshold, the text rule conforming to a first preset condition is selected from a plurality of text rules, and a keyword sequence of the text to be recognized is obtained based on the selected text rule, the matched text rule and the text to be recognized; if the number of the texts is larger than a second preset number threshold, and the matched text rules comprise a plurality of text rules with the same affiliated text semantics, selecting one text rule from the text rules with the same affiliated text semantics when the text rules with the same affiliated text semantics accord with a second preset condition, and acquiring a keyword sequence of the text to be identified based on the selected text rule and the text to be identified. Reference may be made specifically to the relevant steps in the above embodiments, and this embodiment is not described here again.

Step S63: inputting the text to be recognized and the keyword sequence into a text semantic recognition model obtained through training for semantic understanding, and obtaining the text semantic of the text to be recognized.

In this embodiment, the text semantic recognition model is obtained by training a preset neural network by using a plurality of training texts labeled with text semantics and keyword sequences thereof. The specific training process may refer to the relevant steps in the above embodiments, which are not described herein.

In this embodiment, the text semantic recognition model may specifically include an input layer, a long-short-term memory network layer, a self-attention mechanism layer and an output layer that are sequentially connected, and a specific architecture may refer to a preset neural network as shown in fig. 2. The method is characterized in that after the text semantic recognition model is trained for a plurality of times, network parameters such as hidden layer weights in a long-short-term memory network layer, W and b in a self-attention mechanism layer, relevant parameters in a full-connection layer contained in an output layer and the like are determined.

Corresponding to the training process, the text to be recognized and the keyword sequence can be subjected to word segmentation and vectorization, so that a first word vector corresponding to the text to be recognized and a second word vector corresponding to the keyword sequence are obtained. In one implementation scenario, in order to unify the lengths of the text and the sequence for subsequent encoding, the lengths of the text and the sequence may be defined before the word segmentation and the vectorization, and specifically, reference may be made to the relevant steps in the foregoing embodiments, which are not repeated herein.

The first word vector and the second word vector are respectively input into an input layer, and are subjected to coding processing through a long-short-term memory network layer and a self-attention mechanism layer to obtain a first feature vector corresponding to the text to be recognized and a second feature vector corresponding to the keyword sequence. Specifically, a first word vector and a second word vector can be respectively subjected to first coding processing through a long-term and short-term memory network layer to obtain a first hidden layer vector corresponding to a text to be identified and a second hidden layer vector corresponding to a keyword sequence, and the first hidden layer vector and the second hidden layer vector are respectively subjected to second coding processing through a self-attention mechanism layer to obtain a first feature vector corresponding to the text to be identified and a second feature vector corresponding to the keyword sequence. Reference may be made specifically to the relevant steps in the foregoing embodiments, and details are not repeated here.

On the basis, the first feature vector and the second feature vector are classified through an output layer, so that text semantics of the text to be recognized are output. Specifically, the number of the keyword sequences of the text to be recognized is plural, correspondingly, the number of the second feature vectors corresponding to the keyword sequences is plural, when the classification processing is performed, the plural second feature vectors can be combined, the combined second feature vectors and the first feature vectors are spliced to obtain the coding feature vectors of the text to be recognized, the coding feature vectors are predicted, and the text semantic of the text to be recognized is output. Unlike the above embodiment, in the process of performing text semantic recognition on a text to be recognized, since the text semantic of the text to be recognized is unknown, the positive keyword sequence and the negative keyword sequence in the plurality of keyword sequences are also unknown, in order to accurately perform text semantic recognition on the text to be recognized, the specific process of performing classification processing on the first feature vector and the second feature vector through the output layer to output the text semantic of the text to be recognized may refer to fig. 7, and fig. 7 is a schematic flowchart of an embodiment of performing classification processing on the first feature vector and the second feature vector through the output layer to output the text semantic of the text to be recognized, and specifically may include the following steps:

Step S71: and respectively taking each text semantic to be predicted by the text semantic recognition model as a target text semantic.

For example, text semantics to be predicted by the text semantic recognition mode include: the weather checking, the air ticket booking and the hotel booking can be respectively used as target text semantics. In other application scenarios, the same can be said, and are not exemplified here.

After one of the text semantics to be predicted by the text semantic recognition model is taken as a target text semantic, the following steps may be performed for each target text semantic.

Step S72: and taking the keyword sequences related to the target text semantically in the plurality of keyword sequences as a first positive keyword sequence, and taking other keyword sequences as a first negative keyword sequence.

Taking the example of the text to be identified, "i want to subscribe to the air ticket from Shanghai to Beijing next monday," i inquire about the weather condition of Beijing next monday next, the corresponding keyword sequence includes: corresponding to the text semantic "check weather" (query, beijing, weather), and corresponding to the text semantic "order air ticket" (order, from, shanghai, to, beijing, air ticket). When the target text semantic is "looking up weather", the keyword sequence (query, beijing, weather) may be used as a first positive keyword sequence, and the keyword sequence (reservation, from Shanghai, to Beijing, air ticket) may be used as a first negative keyword sequence. Similarly, when the target text semantic is "order ticket", the keyword sequence (booking, going from Shanghai, going to Beijing, ticket) can be used as a first positive keyword sequence, and the keyword sequence (query, beijing, weather) can be used as a first negative keyword sequence; when the target text semantic is "hotel booking", the keyword sequences (booking, going from, shanghai, going to, beijing, air ticket), (query, beijing, weather) can be used as the first negative keyword sequences. Other application scenarios, and the like, are not exemplified herein.

Step S73: and respectively obtaining a first vector sum among second feature vectors corresponding to the first positive keyword sequences and a second vector sum among the second feature vectors corresponding to the first negative keyword sequences.

In particular, when the first forward keyword sequence is not present, the first vector sum is a zero vector. Still taking the example of the text to be identified, "i want to subscribe to the air ticket from Shanghai to Beijing on monday, help i inquire about the weather condition of Beijing on monday", when the target text semantic is "order hotel", the keyword sequences (subscription, from Shanghai, to Beijing, air ticket), (inquiry, beijing, weather) are all the first negative keyword sequences, so the first vector sum may be zero vector.

By adding the second feature vectors corresponding to the first forward keyword sequences, information that the text to be recognized belongs to the target text semantics can be enhanced, and further accuracy of text semantic recognition is improved.

Step S74: a vector difference between the first vector sum and the second vector sum is obtained.

By subtracting the second vector sum from the first vector sum, other information irrelevant to the target text semantic can be eliminated, and the accuracy of text semantic recognition can be improved.

Step S75: and respectively splicing the vector difference corresponding to each target text semantic with the first feature vector to obtain the coding feature vector corresponding to each target text semantic.

Reference may be made in particular to the relevant steps in the previous embodiments.

Step S76: and respectively utilizing the coding feature vectors corresponding to the semantics of each target text to obtain the probability value of the text to be recognized belonging to the semantics of each target text.

Through the steps, the coding feature vector corresponding to each target text semantic can be obtained. Taking the example of the text to be identified, i'm booking the air ticket from Shanghai to Beijing, helping i to inquire the weather condition of the next monday Beijing, the coding feature vector corresponding to the target text semantic "booking air ticket", the coding feature vector corresponding to the target text semantic "booking hotel" and the coding feature vector corresponding to the target text semantic "checking weather" can be respectively obtained, so that the coding feature vectors are respectively predicted to obtain the probability value of the text to be identified belonging to the target text semantic "booking air ticket", the probability value of the text to be identified belonging to the target text semantic "booking hotel" and the probability value of the text to be identified belonging to the target text semantic "checking weather".

Step S77: and determining the target text semantic corresponding to the highest probability value as the text semantic of the text to be recognized.

Still take the example of the text to be identified, "i want to subscribe to the air ticket from Shanghai to Beijing on monday, help i inquire about the weather condition of Beijing on monday, and can determine the text semantic of the text to be identified as the highest probability value among the probability value of the target text semantic" order air ticket ", the probability value of the target text semantic" order hotel ", and the probability value of the target text semantic" check weather ".

Referring to fig. 8, fig. 8 is a schematic diagram illustrating an embodiment of a text semantic recognition device 80 according to the present application. The text semantic recognition device 80 comprises a memory 81 and a processor 82 coupled to each other, the memory having stored therein program instructions for executing the program instructions to implement the steps of any of the above-described text semantic recognition method embodiments.

In particular, the processor 82 is configured to control itself and the memory 81 to implement the steps of any of the text semantic recognition method embodiments described above. The processor 82 may also be referred to as a CPU (Central Processing Unit ). The processor 82 may be an integrated circuit chip having signal processing capabilities. The processor 82 may also be a general purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a Field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. In addition, the processor 82 may be commonly implemented by a plurality of integrated circuit chips.

In this embodiment, the processor 82 is configured to screen text rules matching with the text to be identified from a plurality of text rules, where the plurality of text rules are obtained by analyzing a plurality of sample texts labeled with text semantics; the processor 82 is further configured to obtain a keyword sequence of the text to be recognized based on the matched text rule and the text to be recognized; the processor 82 is further configured to input the text to be recognized and the keyword sequence into a text semantic recognition model obtained through training to perform semantic understanding, so as to obtain text semantics of the text to be recognized; the text semantic recognition model is obtained by training a preset neural network by using a plurality of training texts marked with text semantics and keyword sequences thereof.

In some embodiments, the text semantic recognition model includes an input layer, a long-short-term memory network layer, a self-attention mechanism layer and an output layer that are sequentially connected, the processor 82 is further configured to segment and vectorize a text to be recognized and a keyword sequence, respectively, to obtain a first word vector corresponding to the text to be recognized and a second word vector corresponding to the keyword sequence, and the processor 82 is further configured to input the first word vector and the second word vector into the input layer, respectively, and perform encoding processing through the long-short-term memory network layer and the self-attention mechanism layer, to obtain a first feature vector corresponding to the text to be recognized and a second feature vector corresponding to the keyword sequence; the processor 82 is further configured to perform a classification process on the first feature vector and the second feature vector through the output layer to output text semantics of the text to be recognized.

In some embodiments, the processor 82 is further configured to perform a first encoding process on the first word vector and the second word vector through the long-short-term memory network layer, to obtain a first hidden layer vector corresponding to the text to be recognized, and a second hidden layer vector corresponding to the keyword sequence; the processor 82 is further configured to perform a second encoding process on the first hidden layer vector and the second hidden layer vector through the self-attention mechanism layer, to obtain a first feature vector corresponding to the text to be recognized, and a second feature vector corresponding to the keyword sequence.

In some embodiments, the keyword sequence of the text to be recognized is plural, and accordingly, the second feature vector corresponding to the keyword sequence is plural, and the processor 82 is further configured to combine the plural second feature vectors; the processor 82 is further configured to splice the second feature vector and the first feature vector after the merging process to obtain a coded feature vector of the text to be identified; the processor 82 is further configured to perform prediction processing on the encoded feature vector, and output text semantics of the text to be recognized.

In some embodiments, the processor 82 is further configured to use each text semantic to be predicted by the text semantic recognition model as a target text semantic, and the processor 82 is further configured to use a keyword sequence related to the target text semantic of the plurality of keyword sequences as a first positive keyword sequence and other keyword sequences as a first negative keyword sequence; the processor 82 is further configured to obtain a first vector sum between second feature vectors corresponding to the first positive keyword sequence and a second vector sum between second feature vectors corresponding to the first negative keyword sequence, respectively; the processor 82 is further configured to obtain a vector difference between the first vector sum and the second vector sum, the processor 82 is further configured to splice the vector difference corresponding to each target text semantic with the first feature vector to obtain a coded feature vector corresponding to each target text semantic, the processor 82 is further configured to obtain a probability value that the text to be recognized belongs to each target text semantic by using the coded feature vector corresponding to each target text semantic, and the processor 82 is further configured to determine the target text semantic corresponding to the highest probability value as the text semantic of the text to be recognized.

Different from the foregoing embodiment, by taking a keyword sequence related to the target text semantic among the plurality of keyword sequences as a first positive keyword sequence and taking other keyword sequences as first negative keyword sequences, respectively obtaining a first vector sum between second feature vectors corresponding to the first positive keyword sequences and a second vector sum between second feature vectors corresponding to the first negative keyword sequences, obtaining a vector difference between the first vector sum and the second vector sum, and performing vector splicing and prediction processing on the basis, information that the text to be recognized belongs to the target text semantic can be enhanced, other information unrelated to the target text semantic can be excluded, and accuracy of text semantic recognition is facilitated to be improved.

In some embodiments, the processor 82 is further configured to count a number of texts of the training text corresponding to each text semantic, and a text semantic to which each text rule belongs, the processor 82 is further configured to obtain a text semantic to which a matched text rule belongs, and obtain a number of texts of the training text corresponding to the text semantic to which the matched text rule belongs, and the processor 82 is further configured to select a text rule meeting a first preset condition from the plurality of text rules when the number of texts is less than a first preset number threshold, and obtain a keyword sequence of the text to be recognized based on the selected text rule, the matched text rule, and the text to be recognized; the processor 82 is further configured to select one of the text rules with the same text semantics from the plurality of text rules with the same text semantics when the number of the texts is greater than a second preset number threshold and the matched text rules include the text rules with the same text semantics, and acquire the keyword sequence of the text to be recognized based on the selected text rule and the text to be recognized.

Different from the foregoing embodiment, by counting the number of texts of the training texts corresponding to each text semantic and the text semantic to which each text rule belongs, and obtaining the text semantic to which the matched text rule belongs and obtaining the number of texts of the training texts corresponding to the text semantic to which the matched text rule belongs, when the number of texts is smaller than a first preset number threshold, selecting a text rule conforming to a first preset condition from a plurality of text rules, and obtaining a keyword sequence of the text to be recognized based on the selected text rule, the matched text rule and the text to be recognized, and when the number of texts is larger than a second preset number threshold and the matched text rule contains a plurality of text rules with the same text semantic to which the matched text rule belongs, when the text rules with the same meanings of the texts conform to the second preset condition, one text rule is selected from the text rules with the same meanings of the texts, and based on the selected text rule and the text to be identified, a keyword sequence of the text to be identified is obtained, so that the matching requirement can be reduced under the condition that the number of training texts corresponding to the text semantics of the matched text rule is small, more text rules can be matched, and under the condition that the number of training texts corresponding to the text semantics of the matched text rule is large, a plurality of text rules are removed from the matched text rules, so that the problem of unbalanced samples can be relieved.

In some embodiments, the processor 82 is further configured to filter text rules from the plurality of text rules that match each training text, respectively; the processor 82 is further configured to obtain a keyword sequence of each training text based on the text rule and the training text matched with each training text, respectively; the processor 82 is further configured to input each training text and the corresponding keyword sequence thereof into a preset neural network, and train the preset neural network in a preset training manner to obtain a text semantic recognition model.

Different from the embodiment, the text rules matched with each training text are screened from a plurality of text rules, and the keyword sequence of each training text is acquired based on the text rules matched with each training text and the training text, so that each training text and the keyword sequence corresponding to each training text are input into a preset neural network, the preset neural network is trained in a preset training mode, a text semantic recognition model is obtained, the text rules can be transmitted into the preset neural network in the training stage through the keyword sequences, the neural network is guided to learn rule information of few sample categories better, the problem of sample imbalance can be relieved, and the accuracy of text semantic recognition is improved.

In some embodiments, the processor 82 is further configured to, for each training text of the plurality of training texts, obtain, from among a plurality of keyword sequences of the training text, a keyword sequence having a text semantic identical to a text semantic of the training text as a positive keyword sequence, and obtain, from among a plurality of keyword sequences of the training text, a keyword sequence having a text semantic different from the text semantic of the training text as a negative keyword sequence, and obtain, based on the positive keyword sequence and the negative keyword sequence, a loss value and a loss weight corresponding to the training text; the processor 82 is further configured to perform a weighting process on the loss value of each training text by using the loss weight of the corresponding training text, so as to obtain loss values of a plurality of training texts; the processor 82 is further configured to adjust a network parameter of the preset neural network using the loss values of the plurality of training texts until a preset training end condition is satisfied.

In some embodiments, the processor 82 is further configured to obtain a first probability distribution value for each word in the positive keyword sequence and a second probability distribution value for each word in the negative keyword sequence, respectively; the processor 82 is further configured to perform a self-attention loss calculation on the first probability distribution value and the second probability distribution value, and obtain a self-attention loss value of the training text; the processor 82 is further configured to obtain a loss value corresponding to the training text based on the cross entropy loss value and the self-attention loss value of the training text.

In some embodiments, the processor 82 is further configured to calculate a forward penalty value loss for each forward keyword sequence using equation 1 below ₁ The negative penalty value loss is calculated for each negative keyword sequence using equation 2 below ₂ ；

Wherein p (wi) is a first probability distribution value of an ith word in a positive keyword sequence, n is a total number of words in the positive keyword sequence, p (xi) is a second probability distribution value of the ith word in a negative keyword sequence, and m is a total number of words in the negative keyword sequence;

the processor 82 is further configured to determine a forward loss value loss for each forward keyword sequence ₁ And a negative loss value loss for each negative keyword sequence ₂ And adding to obtain the self-attention loss value of the training text.

Referring to fig. 9, fig. 9 is a schematic diagram illustrating an embodiment of an apparatus 90 for acquiring a text semantic recognition model according to the present application. The text semantic recognition model obtaining apparatus 90 comprises a memory 91 and a processor 92 coupled to each other, where the memory 91 stores program instructions, and the processor 92 is configured to execute the program instructions to implement the steps of any of the above-described text semantic recognition model obtaining method embodiments.

In particular, the processor 92 is adapted to control itself and the memory 91 to implement the steps in the method embodiment of retrieving any of the text semantic recognition models described above. The processor 92 may also be referred to as a CPU (Central Processing Unit ). The processor 92 may be an integrated circuit chip with signal processing capabilities. The processor 92 may also be a general purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a Field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. In addition, the processor 92 may be commonly implemented by a plurality of integrated circuit chips.

In this embodiment, the processor 92 is configured to screen text rules matching each training text from a plurality of text rules, where the plurality of text rules are obtained by analyzing a plurality of sample texts labeled with text semantics; the processor 92 is further configured to obtain a keyword sequence of each training text based on the text rule and the training text matched with each training text, respectively; the processor 92 is further configured to input each training text and the corresponding keyword sequence thereof into a preset neural network for training until a preset training end condition is met, thereby obtaining a text semantic recognition model.

Referring to fig. 10, fig. 10 is a schematic diagram illustrating a frame of an embodiment of a storage device 100 according to the present application. The storage device 100 stores program instructions 101 that can be executed by the processor, where the program instructions 101 are configured to implement steps in any of the above-described text semantic recognition method embodiments or implement steps in any of the above-described text semantic recognition model acquisition method embodiments.

According to the scheme, the text rule can be transmitted into the preset neural network in the training stage through the keyword sequence, so that the neural network is guided to learn rule information of few sample types better, the problem of sample imbalance can be relieved, the accuracy of text semantic recognition can be improved, in addition, in the recognition stage, the text rule of a text to be recognized can be transmitted into the text semantic recognition model through the keyword sequence, and the accuracy of text semantic recognition can be further improved.

In the several embodiments provided in the present application, it should be understood that the disclosed methods and apparatus may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of modules or units is merely a logical functional division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical, or other forms.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied essentially or in part or all or part of the technical solution contributing to the prior art or in the form of a software product stored in a storage medium, including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the methods of the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

Claims

1. A text semantic recognition method, comprising:

screening text rules matched with a text to be identified from a plurality of text rules, wherein the text rules are obtained by analyzing a plurality of sample texts marked with text semantics;

acquiring a keyword sequence of the text to be recognized based on the matched text rule and the text to be recognized;

inputting the text to be recognized and the keyword sequence into a text semantic recognition model obtained through training for semantic understanding, and obtaining text semantics of the text to be recognized;

the text semantic recognition model is obtained by training a preset neural network by using a plurality of training texts marked with text semantics and keyword sequences thereof, and the keyword sequences are obtained based on text rules matched with the training texts and the training texts.

2. The text semantic recognition method according to claim 1, wherein the text semantic recognition model comprises an input layer, a long-short-term memory network layer, a self-attention mechanism layer and an output layer which are sequentially connected;

the text to be recognized and the keyword sequence are input into a text semantic recognition model obtained through training for semantic understanding, and before the text semantic of the text to be recognized is obtained, the method further comprises the steps of:

Word segmentation and vectorization are respectively carried out on the text to be recognized and the keyword sequence, so that a first word vector corresponding to the text to be recognized and a second word vector corresponding to the keyword sequence are obtained;

inputting the text to be recognized and the keyword sequence into a text semantic recognition model obtained through training to perform semantic understanding, wherein the step of obtaining text semantics of the text to be recognized comprises the following steps:

inputting the first word vector and the second word vector into the input layer respectively, and carrying out coding processing through the long-short-term memory network layer and the self-attention mechanism layer to obtain a first feature vector corresponding to the text to be identified and a second feature vector corresponding to the keyword sequence;

and classifying the first feature vector and the second feature vector through the output layer so as to output text semantics of the text to be recognized.

3. The text semantic recognition method according to claim 2, wherein,

the encoding processing is performed by the long-period memory network layer and the self-attention mechanism layer, and obtaining a first feature vector corresponding to the text to be identified and a second feature vector corresponding to the keyword sequence comprises:

Respectively carrying out first coding processing on the first word vector and the second word vector through the long-short-term memory network layer to obtain a first hidden layer vector corresponding to the text to be identified and a second hidden layer vector corresponding to the keyword sequence;

and respectively carrying out second coding processing on the first hidden layer vector and the second hidden layer vector through the self-attention mechanism layer to obtain a first feature vector corresponding to the text to be identified and a second feature vector corresponding to the keyword sequence.

4. The text semantic recognition method according to claim 2, wherein the number of keyword sequences of the text to be recognized is plural, and correspondingly, the number of second feature vectors corresponding to the keyword sequences is plural;

the classifying the first feature vector and the second feature vector to output text semantics of the text to be recognized includes:

combining the plurality of second feature vectors;

splicing the second feature vector subjected to the merging processing with the first feature vector to obtain the coding feature vector of the text to be identified;

and carrying out prediction processing on the coding feature vector, and outputting text semantics of the text to be recognized.

5. The text semantic recognition method according to claim 4, wherein the combining the plurality of the second feature vectors includes:

taking each text semantic to be predicted by the text semantic recognition model as a target text semantic, and executing the following steps on each target text semantic:

the keyword sequences related to the target text semantically in the keyword sequences are used as first positive keyword sequences, and other keyword sequences are used as first negative keyword sequences;

respectively obtaining a first vector sum among second feature vectors corresponding to the first positive keyword sequences and a second vector sum among second feature vectors corresponding to the first negative keyword sequences;

obtaining a vector difference between the first vector sum and the second vector sum;

the splicing of the second feature vector after the merging processing and the first feature vector to obtain the coding feature vector of the text to be identified comprises the following steps:

respectively splicing the vector difference corresponding to each target text semantic with the first feature vector to obtain a coding feature vector corresponding to each target text semantic;

The predicting processing is carried out on the coding feature vector, and the outputting of the text semantic of the text to be recognized comprises the following steps:

respectively utilizing the coding feature vectors corresponding to each target text semantic to obtain probability values of the texts to be identified belonging to each target text semantic;

and determining the target text semantic corresponding to the highest probability value as the text semantic of the text to be recognized.

6. The text semantic recognition method according to claim 1, wherein before the obtaining the keyword sequence of the text to be recognized based on the text rule and the text to be recognized that are matched, the method further comprises:

counting the text quantity of training texts corresponding to each text semantic and the text semantic of each text rule;

the obtaining the keyword sequence of the text to be recognized based on the matched text rule and the text to be recognized comprises the following steps:

acquiring text semantics of the matched text rule, and acquiring the text quantity of training texts corresponding to the text semantics;

if the number of the texts is smaller than a first preset number threshold, selecting a text rule conforming to a first preset condition from the plurality of text rules, and acquiring a keyword sequence of the text to be identified based on the selected text rule, the matched text rule and the text to be identified;

If the number of the texts is larger than a second preset number threshold, and the matched text rules comprise a plurality of text rules with the same text semantics, selecting one text rule from the text rules with the same text semantics when the text rules with the same text semantics accord with a second preset condition, and acquiring the keyword sequence of the text to be recognized based on the selected text rule and the text to be recognized.

7. The text semantic recognition method according to claim 1, wherein the inputting the text to be recognized and the keyword sequence into a text semantic recognition model obtained through training carries out semantic understanding, and before obtaining text semantics of the text to be recognized, the method further comprises:

screening text rules matched with each training text from the text rules;

based on text rules matched with each training text and the training texts, respectively acquiring a keyword sequence of each training text;

and respectively inputting each training text and the corresponding keyword sequence into the preset neural network, and training the preset neural network by adopting a preset training mode to obtain the text semantic recognition model.

8. The text semantic recognition method according to claim 7, wherein the training by the preset neural network using a preset training mode comprises:

aiming at each training text in the training texts, acquiring a keyword sequence with the same text semantic as the training text from a plurality of keyword sequences of the training text as a positive keyword sequence, and acquiring a keyword sequence with different text semantic as a negative keyword sequence; acquiring a loss value and a loss weight corresponding to the training text based on the positive keyword sequence and the negative keyword sequence;

weighting the loss value of the corresponding training text by using the loss weight of each training text to obtain the loss values of the plurality of training texts;

and adjusting network parameters of the preset neural network by using the loss values of the training texts until a preset training ending condition is met.

9. The text semantic recognition method according to claim 8, wherein the obtaining the penalty value and the penalty weight corresponding to the training text based on the positive keyword sequence and the negative keyword sequence comprises:

Respectively obtaining a first probability distribution value of each word in the positive keyword sequence and a second probability distribution value of each word in the negative keyword sequence;

performing self-attention loss calculation on the first probability distribution value and the second probability distribution value to obtain a self-attention loss value of the training text;

and obtaining the loss value corresponding to the training text based on the cross entropy loss value and the self-attention loss value of the training text.

10. The text semantic recognition method according to claim 9, wherein the performing self-attention loss calculation on the first probability distribution value and the second probability distribution value to obtain a self-attention loss value of the training text comprises:

the forward penalty value loss is calculated for each forward keyword sequence using equation 1 below ₁ The negative penalty value loss is calculated for each negative keyword sequence using equation 2 below ₂ ；

Forward loss value loss of each forward keyword sequence ₁ And a negative loss value loss for each negative keyword sequence ₂ And adding to obtain the self-attention loss value of the training text.

11. The method for acquiring the text semantic recognition model is characterized by comprising the following steps of:

screening text rules matched with each training text from a plurality of text rules, wherein the text rules are obtained by analyzing a plurality of sample texts marked with text semantics;

based on text rules matched with each training text and the training text, respectively acquiring a keyword sequence of each training text;

inputting each training text marked with text semantics and the corresponding keyword sequence into a preset neural network to train until a preset training ending condition is met, and obtaining the text semantic recognition model.

12. A text semantic recognition device comprising a memory and a processor coupled to each other, the memory having program instructions stored therein, the processor being configured to execute the program instructions to implement the text semantic recognition method of any one of claims 1 to 10.

13. An apparatus for obtaining a text semantic recognition model, comprising a memory and a processor coupled to each other, the memory having program instructions stored therein, the processor configured to execute the program instructions to implement the method for obtaining a text semantic recognition model of claim 11.

14. A storage device, characterized in that program instructions executable by a processor for implementing the text semantic recognition method according to any one of claims 1 to 10 or the method for acquiring the text semantic recognition model according to claim 11 are stored.