WO2023005968A1

WO2023005968A1 - Text category recognition method and apparatus, and electronic device and storage medium

Info

Publication number: WO2023005968A1
Application number: PCT/CN2022/108224
Authority: WO
Inventors: 马玉昆; 卜英桐; 程大川
Original assignee: 北京有竹居网络技术有限公司
Priority date: 2021-07-27
Filing date: 2022-07-27
Publication date: 2023-02-02
Also published as: CN113360660A

Abstract

Provided in the present disclosure are a text category recognition method and apparatus, and an electronic device and a storage medium. The method comprises: splitting text to be recognized, so as to obtain a sub-text sequence, and splitting each piece of sub-text in the sub-text sequence, so as to obtain a corresponding sentence sequence; for each sentence in the sentence sequence corresponding to each piece of sub-text, performing feature extraction according to a pre-trained feature extraction model, so as to obtain a sentence feature vector corresponding to the sentence; for each piece of sub-text in the sub-text sequence, executing a first calculation operation to calculate an attention feature vector of the sub-text with respect to the text to be recognized; splicing the attention feature vectors of all the pieces of sub-text in the sub-text sequence with respect to the text to be recognized, so as to obtain a feature vector of the text to be recognized that corresponds to the text to be recognized; and inputting the feature vector of the text to be recognized into a pre-trained classification model, so as to obtain a probability value of the text to be recognized being a preset category of text. Therefore, text to be recognized is automatically classified, thereby reducing the labor costs of text classification.

Description

Text category recognition method, device, electronic device and storage medium

Cross References to Related Applications

This application is based on the Chinese patent application with the application number 202110849917.9, the filing date is July 27, 2021, and the title is "text category recognition method, device, electronic equipment and storage medium", and claims the priority of the Chinese patent application, The entire content of this Chinese patent application is hereby incorporated by reference into this application.

technical field

Embodiments of the present disclosure relate to the technical field of information processing, and specifically relate to a text category recognition method, device, electronic device, and storage medium.

Background technique

Text category recognition refers to pointing out whether the text belongs to a preset category for a piece of text, or giving a probability value that the text belongs to a preset category. For example, the e-commerce platform needs to identify the product introduction text uploaded by the e-commerce company to determine whether the product introduction text meets the requirements and whether there are inappropriate expressions. For another example, the literary works platform needs to identify the text content of literary novels uploaded by users to determine whether the novel text content includes vulgar and indecent content.

Contents of the invention

Embodiments of the present disclosure provide a text category recognition method, device, electronic device and storage medium.

In a first aspect, an embodiment of the present disclosure provides a text category recognition method, the method comprising:

Splitting the text to be recognized to obtain a subtext sequence, and splitting each subtext in the subtext sequence to obtain a corresponding sentence sequence;

For each sentence in the sentence sequence corresponding to each said subtext, perform feature extraction according to a pre-trained feature extraction model to obtain a sentence feature vector corresponding to the sentence;

For each subtext in the subtext sequence, the following first calculation operation is performed: for each sentence in the subtext, based on the sentence feature vector corresponding to each sentence in the sentence sequence corresponding to the subtext, calculate the sentence With respect to the attention feature vector of the subtext; based on the attention feature vector of each sentence relative to the subtext, calculate the attention feature vector of the subtext with respect to the text to be identified;

Splicing the subtext in the subtext sequence relative to the attention feature vector of the text to be recognized to obtain the text feature vector to be recognized corresponding to the text to be recognized;

Inputting the feature vector of the text to be recognized into a pre-trained classification model to obtain a probability value that the text to be recognized belongs to a preset category of text.

In some optional implementation manners, the feature extraction model and the classification model are pre-trained through the following training steps:

Determine the initial feature extraction model and initial classification model;

Obtain a training sample set, wherein the training sample includes sample text and a sample label for representing whether the sample text belongs to a preset category of text;

For the training samples in the training sample set, the following parameter adjustment operations are performed until the preset training end condition is met: the sample text in the training sample is split to obtain a sample subtext sequence, and each of the sample subtext sequences is The subtexts are split to obtain the corresponding sentence sequence; for each sentence in the sentence sequence corresponding to each sample subtext in the sample subtext sequence, perform feature extraction according to the initial feature extraction model to obtain the sentence feature corresponding to the sentence Vector; for each sample subtext in the sample subtext sequence, perform a second calculation operation to obtain the attention feature vector of the sample subtext relative to the sample text: based on the sentence sequence corresponding to the sample subtext The sentence feature vector corresponding to each sentence calculates the attention feature vector of the sentence relative to the sample subtext; based on the attention feature vector of each sentence relative to the sample subtext, calculates the sample subtext relative to the sample The attention feature vector of text; Splicing the attention feature vector of sample subtext in described sample subtext sequence with respect to this sample text, obtain the sample text feature vector corresponding to this sample text; Input the obtained sample text feature vector into the The initial classification model to obtain the probability value that the sample text belongs to the preset category text; based on the difference between the obtained probability value and the sample label in the training sample, adjust the initial feature extraction model and the initial Model parameters for classification models;

Determining the trained initial feature extraction model and the initial classification model as the pre-trained feature extraction model and the classification model.

In some optional implementation manners, the feature extraction model includes a word vector feature extraction model and a sentence vector feature extraction model; and

For each sentence in the sentence sequence corresponding to each of the subtexts, perform feature extraction according to a pre-trained feature extraction model to obtain a sentence feature vector corresponding to the sentence, including:

For each sentence in the sentence sequence corresponding to each of the subtexts, perform feature extraction according to the word vector feature extraction model for each participle in the word segment sequence corresponding to the sentence to obtain a corresponding word vector, and combine the corresponding word vectors of the sentence The word vector corresponding to each word segment in the word segmentation sequence is used to form the sentence feature matrix corresponding to the sentence, and the sentence feature matrix corresponding to the sentence is extracted according to the sentence vector feature extraction model to obtain the sentence feature vector corresponding to the sentence.

In some optional implementation manners, the word vector feature extraction model includes at least one of the following: a long short-term memory network, and a translation model.

In some optional implementation manners, the sentence vector feature extraction model includes at least one of the following: a convolutional neural network, and a bidirectional long-short-term memory network.

In some optional implementation manners, for each sentence in the sentence sequence corresponding to each sample subtext in the sample subtext sequence, perform feature extraction according to the initial feature extraction model to obtain the sentence features corresponding to the sentence vector, including:

For each sentence in the sentence sequence corresponding to each sample subtext in the sample subtext sequence, each word segmentation in the word segmentation sequence corresponding to the sentence is subjected to feature extraction according to the word vector feature extraction model to obtain a corresponding word vector Combining the word vectors corresponding to each word in the word segmentation sequence corresponding to the sentence to form the sentence feature matrix corresponding to the sentence, performing feature extraction on the sentence feature matrix corresponding to the sentence according to the sentence vector feature extraction model to obtain the sentence corresponding to the sentence Feature vector.

In some optional implementation manners, before combining the word vectors corresponding to each word in the word segmentation sequence corresponding to the sentence to form the sentence feature matrix corresponding to the sentence, the training step also includes:

For each word segment in the word segment sequence corresponding to the sentence, in response to determining that the word segment matches a keyword in the preset text category keyword set, the word vector corresponding to the word segment is set as the preset word vector.

In some optional embodiments, the method also includes:

determining whether the probability value is greater than a preset probability threshold;

In response to determining that the value is greater than, first recognition result information for indicating that the text to be recognized is a preset text category is generated.

In some optional embodiments, the method also includes:

In response to determining that it is not greater than, second recognition result information for indicating that the text to be recognized is not a preset text category is generated.

In some optional embodiments, the method also includes:

For each sentence in the sentence sequence corresponding to each subtext in the subtext sequence, based on the attention feature vector of the sentence relative to the subtext, calculate the probability value that the sentence belongs to the preset text category, according to the calculation The obtained probability value determines the presentation manner corresponding to the sentence, and presents the sentence according to the determined presentation manner.

In some optional embodiments, the method also includes:

For each subtext in the subtext sequence, based on the attention feature vector of the subtext relative to the text to be recognized, calculate the probability value that the subtext belongs to the preset text category, according to the calculated probability value A presentation manner corresponding to the subtext is determined, and the subtext is presented according to the determined presentation manner.

In a second aspect, an embodiment of the present disclosure provides a text category recognition device, which includes:

The splitting unit is configured to split the text to be recognized to obtain a subtext sequence, and split each subtext in the subtext sequence to obtain a corresponding sentence sequence;

The feature extraction unit is configured to perform feature extraction for each sentence in the sentence sequence corresponding to each subtext according to a pre-trained feature extraction model to obtain a sentence feature vector corresponding to the sentence;

The calculation unit is configured to perform the following first calculation operation for each subtext in the subtext sequence: for each sentence in the subtext, based on the sentence corresponding to each sentence in the sentence sequence corresponding to the subtext A feature vector, calculating the attention feature vector of the sentence relative to the subtext; calculating the attention feature vector of the subtext relative to the text to be identified based on the attention feature vector of each sentence relative to the subtext;

The splicing unit is configured to splice the attention feature vectors of the subtexts in the subtext sequence relative to the text to be recognized to obtain the text feature vector to be recognized corresponding to the text to be recognized;

The recognition unit is configured to input the feature vector of the text to be recognized into a pre-trained classification model to obtain a probability value that the text to be recognized belongs to a preset category of text.

In some optional implementation manners, the feature extraction model and the classification model are pre-trained as follows:

The feature extraction unit is further configured to:

In some optional embodiments, the device also includes:

a determining unit configured to determine whether the probability value is greater than a preset probability threshold;

The first generating unit is configured to generate first recognition result information indicating that the text to be recognized is a preset text category in response to determining that the value is greater than or equal to greater than 100%.

In some optional embodiments, the device also includes:

The second generating unit is configured to, in response to determining that it is not greater than, generate second recognition result information for indicating that the text to be recognized is not a preset text category.

In some optional embodiments, the device also includes:

The first presentation unit is configured to, for each sentence in the sentence sequence corresponding to each subtext in the subtext sequence, calculate that the sentence belongs to the preset based on the attention feature vector of the sentence relative to the subtext According to the probability value of the text category, the presentation manner corresponding to the sentence is determined according to the calculated probability value, and the sentence is presented according to the determined presentation manner.

In some optional embodiments, the device also includes:

The second presentation unit is configured to, for each subtext in the subtext sequence, calculate the probability that the subtext belongs to the preset text category based on the attention feature vector of the subtext relative to the text to be recognized value, determine the presentation manner corresponding to the subtext according to the calculated probability value, and present the subtext according to the determined presentation manner.

In a third aspect, an embodiment of the present disclosure provides an electronic device, including: one or more processors; a storage device, on which one or more programs are stored. When executed by one or more processors, the above one or more processors implement the method described in any implementation manner of the first aspect.

In a fourth aspect, embodiments of the present disclosure provide a computer-readable storage medium, on which a computer program is stored, wherein, when the computer program is executed by one or more processors, any implementation manner in the first aspect can be realized. described method.

At present, when performing category recognition on long texts (for example, texts with a length of more than 5,000 words) (for example, indicating whether a certain long text content involves a specific category), the following methods are mostly used: 1. Manual labeling; 2. Through keywords Screening; 3. Split the long text into short sentences or paragraphs, and then manually mark the short sentences or paragraphs; 4. Use machine learning models to directly model long texts, but the method is limited to simple models such as the bag of words model, If you need to use a deep semantic model, you need to truncate long text. Among them, 1. The manual marking method has high labor costs; 2. The method of keyword screening may cause accidental injury and leakage, and the efficiency is low; 3. After the long text is split into short text, the text magnitude will be dozens of times An increase of hundreds of times will also cause a lot of labor loss; 4. Using the bag-of-words model to directly model long texts is only based on the statistical information of the frequency of words in long texts, and it is impossible to give specific information in long texts. The content is more related to the probability value of a specific category, which cannot meet richer business needs; and if the deep semantic model is used, it needs to be truncated. At this time, the range of text that can be covered is small, which may also cause omissions.

In order to improve the accuracy of classifying long texts, reduce labor costs, and reduce leakage, etc., the text category recognition method, device, electronic equipment, and storage medium provided by the embodiments of the present disclosure split the text to be recognized into subtexts. The subtext is split into sentences, and then the sentence feature vector is generated for the sentence, and then the attention feature vector of each sentence to the subtext and the attention feature vector of each subtext to be recognized are generated, and each subtext is spliced relative to The attention feature vector of the text to be recognized is obtained to obtain the feature vector of the text to be recognized. Finally, input the feature vector of the text to be recognized into the pre-trained classification model to obtain the probability value that the text to be recognized belongs to the preset category of text. That is, through the attention feature vector of the sentence relative to the subtext and the attention feature vector of the subtext relative to the text to be recognized, the hierarchical attention relationship between the sentence, the subtext and the text to be recognized is established, and then the feature vector of the text to be recognized is generated to calculate the The probability value of the preset text category realizes the automatic classification of the text to be recognized, which reduces the labor cost of text classification; and, optionally, the sentence can also be calculated by using the attention feature vector of the sentence relative to the subtext. The sentence belongs to the probability value of the preset text category, and according to the calculated probability value, the corresponding presentation mode of the sentence is determined, and the sentence is presented according to the determined presentation mode, and then the text with different probability values belonging to the preset text category is realized. Sentences are presented in a corresponding manner for reference during manual labeling to reduce the possibility of missing. Or, optionally, it is also possible to calculate the probability value that the subtext belongs to the preset text category by using the attention feature vector of the subtext relative to the text to be recognized, and determine the corresponding presentation mode of the subtext according to the calculated probability value , and present the subtext according to the determined presentation manner, and then realize the presentation of subtexts with different probability values belonging to the preset text category in a corresponding manner, so as to provide reference for manual marking and reduce the possibility of missing.

Description of drawings

Other features, objects and advantages of the present disclosure will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings. The drawings are only for the purpose of illustrating specific embodiments and are not to be considered as limiting the invention. In the attached picture:

FIG. 1 is an exemplary system architecture diagram to which an embodiment of the present disclosure can be applied;

Fig. 2 is a flowchart of an embodiment of the text category recognition method according to the present disclosure;

FIG. 3 is a schematic diagram of an application scenario of a text category recognition method according to the present disclosure;

FIG. 4 is a flowchart of another embodiment of a text category recognition method according to the present disclosure;

FIG. 5 is a schematic structural diagram of an embodiment of a text category recognition device according to the present disclosure;

FIG. 6 is a structural schematic diagram of a computer system suitable for implementing an electronic device according to an embodiment of the present disclosure.

Detailed ways

The present disclosure will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain related inventions, rather than to limit the invention. It should also be noted that, for the convenience of description, only the parts related to the related invention are shown in the drawings.

It should be noted that, in the case of no conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and embodiments.

FIG. 1 shows an exemplary system architecture 100 to which embodiments of the text category recognition method, device, electronic device and storage medium of the present disclosure can be applied.

As shown in FIG. 1 , a system architecture 100 may include

terminal devices

101 , 102 , 103 , a network 104 and a server 105 . The network 104 is used as a medium for providing communication links between the

terminal devices

101 , 102 , 103 and the server 105 . Network 104 may include various connection types, such as wires, wireless communication links, or fiber optic cables, among others.

Users can use

terminal devices

101 , 102 , 103 to interact with server 105 via network 104 to receive or send messages and the like. Various communication client applications can be installed on the

terminal devices

101, 102, and 103, such as text category recognition applications, speech recognition applications, short video social applications, audio and video conference applications, live video applications, and document editing applications. Applications, input method applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

The

terminal devices

101, 102, and 103 may be hardware or software. When the

terminal devices

101, 102, and 103 are hardware, they can be various electronic devices with display screens, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III, Moving Picture Experts Compression Standard Audio Layer 3), MP4 (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Compression Standard Audio Layer 4) players, laptops and desktop computers, etc. When the

terminal devices

101, 102, and 103 are software, they can be installed in the terminal devices listed above. It can be implemented as multiple software or software modules (for example, for providing text category recognition services), or as a single software or software module. No specific limitation is made here.

In some cases, the text category recognition method provided in the present disclosure may be executed by the

terminal devices

101 , 102 , 103 , and correspondingly, the text category recognition apparatus may be set in the

terminal devices

101 , 102 , 103 . At this time, the system architecture 100 may not include the server 105 .

In some cases, the text category recognition method provided by the present disclosure can be jointly executed by the

terminal device

101, 102, 103 and the server 105, for example, the step of "obtaining the text to be recognized" can be executed by the

terminal device

101, 102, 103, Steps such as “for each sentence in the sentence sequence corresponding to each subtext, perform feature extraction according to a pre-trained feature extraction model to obtain a sentence feature vector corresponding to the sentence” can be performed by the server 105 . The present disclosure does not limit this. Correspondingly, the means for identifying text categories can also be respectively set in the

terminal devices

101, 102, 103 and the server 105.

In some cases, the text category recognition method provided by the present disclosure can be executed by the server 105, and correspondingly, the text category recognition device can also be set in the server 105. At this time, the system architecture 100 may not include the

terminal devices

101, 102 , 103.

It should be noted that the server 105 may be hardware or software. When the server 105 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server. When the server 105 is software, it can be implemented as multiple software or software modules (for example, for providing distributed services), or as a single software or software module. No specific limitation is made here.

It should be understood that the numbers of terminal devices, networks and servers in Fig. 1 are only illustrative. According to the implementation needs, there can be any number of terminal devices, networks and servers.

Continue to refer to FIG. 2 , which shows a process 200 of an embodiment of a text category recognition method according to the present disclosure. The text category recognition method includes the following steps:

Step 201, splitting the text to be recognized to obtain a subtext sequence, and splitting each subtext in the subtext sequence to obtain a corresponding sentence sequence.

In this embodiment, the execution body of the text category recognition method (such as the server 105 shown in FIG. 1 ) can first locally or remotely connect to the above-mentioned execution body from other electronic devices (such as the terminal device shown in FIG. 1 ) 101, 102, 103) Obtain the text to be recognized.

Here, the text to be recognized may be composed of characters of the same language, or may be composed of characters of more than one language, which is not specifically limited in the present disclosure.

The text to be recognized may be text in various situations, which is not specifically limited in the present disclosure.

In some optional implementation manners, the text to be recognized may be any of the following: a part of the news text, some chapters of the novel text, and the like.

The text to be recognized may be relatively long text, for example, the text to be recognized may include at least 400 sentences.

Then, the above execution subject can use various implementation methods to split the text to be recognized to obtain subtext sequences.

In some optional implementation manners, the above-mentioned executive body may split the text to be recognized into a first preset number (for example, 20) of subtexts, wherein, the number of sentences in each subtext may be within a preset number range (for example, , greater than or equal to 20 and less than or equal to 25). When splitting, there can be overlap between two adjacent subtexts, so that the continuous semantic information between subtexts can be maintained in the subsequent process.

Arranging the split subtexts according to the position of the subtexts in the text to be recognized, the subtext sequence can be obtained.

Finally, each subtext in the subtext sequence is split to obtain a corresponding sentence sequence. In practice, for example, the sentence sequence can be obtained by splitting according to the punctuation marks in the subtext.

Step 202, for each sentence in the sentence sequence corresponding to each subtext, perform feature extraction according to a pre-trained feature extraction model to obtain a sentence feature vector corresponding to the sentence.

In this embodiment, for each subtext in the subtext sequence obtained in step 201, the execution subject can perform feature extraction on each sentence in the sentence sequence corresponding to the subtext according to a pre-trained feature extraction model to obtain the The sentence feature vector corresponding to the sentence. Among them, the feature extraction model is used to represent the correspondence between the sentence and the feature vector corresponding to the sentence.

In some optional implementation manners, the feature extraction model may include a word vector feature extraction model and a sentence vector feature extraction model. Based on this, step 202 can be performed as follows: for each sentence in the sentence sequence corresponding to each subtext, first, perform feature extraction for each word segment in the word segment sequence corresponding to the sentence according to the word vector feature extraction model to obtain the corresponding word vector , and then combine the word vectors corresponding to each word in the word segmentation sequence corresponding to the sentence to form the sentence feature matrix corresponding to the sentence. Finally, perform feature extraction on the sentence feature matrix corresponding to the sentence according to the sentence vector feature extraction model to obtain the corresponding sentence. Sentence feature vector.

Among them, various currently known or future word segmentation processing methods can be used to perform word segmentation processing on the sentence to obtain the word segmentation sequence corresponding to the sentence, which will not be repeated here.

Among them, the word vector feature extraction model is used to represent the correspondence between words and word vectors corresponding to the words, that is, the word vector feature extraction model is used to map words to word vectors. As an example, the word vector feature extraction model can be a bag of words model (BAW, Bag Of Words). Optionally, the word vector feature extraction model can include at least one of the following: long-short-term memory (LSTM, Long Short-Term Memory) network, translation (Transformer) model (for example, BERT model, ALBERT model).

However, combining the word vectors corresponding to each word segment in the word segment sequence corresponding to the sentence to form the sentence feature matrix corresponding to the sentence may be combined in sequence according to the position of each word segment in the word segment sequence. For example, the word vector corresponding to each word segment can be a V-dimensional vector, where V is a positive integer, and the word segment sequence corresponding to the sentence can include W word segments, and the word vector corresponding to each word segment in the word segment sequence corresponding to the sentence can be combined. Get a W*V matrix, where each row corresponds to a word vector for a word segment. However, in order to ensure that the sentence feature matrix corresponding to each sentence is a matrix of the same size, the obtained matrix can be expanded to U rows, and U is greater than or equal to W. For each row greater than W, it can be supplemented by padding. For example, it can be greater than W Each row of matrix elements is set to 0. In this way, the sentence feature matrix corresponding to each sentence is a U*V matrix.

Among them, the sentence vector feature extraction model is used to represent the correspondence between the sentence feature matrix and the sentence feature vector corresponding to the sentence, that is, the sentence vector feature extraction model is used to map the sentence feature matrix to the sentence feature vector. Optionally, the sentence vector feature extraction model may include at least one of the following: convolutional neural network (CNN, Convolutional Neural Networks), bidirectional long-term short-term memory network (BiLSTM, Bi-directional Long Short-Term Memory).

By using the word vector feature extraction model and the sentence vector feature extraction model, the sentence feature vector corresponding to the sentence can be extracted. Since the word vector is extracted for each participle in the sentence first, and then combined according to the position of the participle to obtain the sentence feature matrix, Then perform feature extraction on the sentence feature matrix to obtain the sentence feature vector. The extracted sentence feature vector can not only represent the word information in the sentence, but also represent the context between words in the sentence, that is, semantic information, which is more conducive to the text category of the subsequent process. identify.

Step 203, for each subtext in the subtext sequence, perform a first calculation operation.

Here, the execution subject may perform the first calculation operation for each subtext in the subtext sequence obtained in step 201 .

Here, the first computing operation may include sub-step 2031 and sub-step 2032:

Sub-step 2031, for each sentence in the subtext, based on the sentence feature vector corresponding to each sentence in the sentence sequence corresponding to the subtext, calculate the attention feature vector of the sentence relative to the subtext.

Here, assuming that through step 202, the sentence feature vector corresponding to each sentence is an M-dimensional vector, and assuming that the number of sentences in each subtext is at most S (for example, 32) sentences, then each sentence in the sentence sequence corresponding to this subtext The corresponding sentence feature vectors can form a matrix F with a size of S×M, and the matrix F can be regarded as the sub-text feature matrix corresponding to the sub-text. However, calculating the attention feature vector of each sentence in the subtext relative to the subtext can be expressed as follows:

Assume that for the i-th sentence in the sentence sequence corresponding to the subtext, where i is a positive integer between 1 and S, the sentence feature vector F _i corresponding to the i-th sentence is an M-dimensional vector, which can also be considered as 1 *M matrix F _i . And the attention feature vector of the i-th sentence relative to the subtext is matrix B _i , then the Cartesian product of matrix F _i and matrix B _i should be matrix F, which can be expressed as follows:

F _i ×B _i =F formula (1)

It can be seen from the above formula that the matrix B _i is a matrix of S*1, where the element B _i,j,1 in the jth row and the first column of the matrix B _i is used to represent the i-th in the sentence sequence corresponding to the subtext The degree of relevance, importance or attention between the sentence and the jth sentence, j is a positive integer between 1 and S.

When specifically calculating the above-mentioned matrix B _i , the attention feature vector matrix B _i of the i-th sentence in the sentence sequence corresponding to the sub-text relative to the sub-text can be obtained by calculating the known matrix F and matrix F _i , Since the matrix B _i is a matrix of S*1, it can also be considered that the matrix B _i is the attention feature vector of the ith sentence in the sentence sequence corresponding to the sub-text relative to the sub-text.

Sub-step 2032, based on the attention feature vector of each sentence relative to the subtext, calculate the attention feature vector of the subtext relative to the text to be recognized.

Here, after sub-step 2012, the attention feature vector of each sentence in the sentence sequence corresponding to the subtext relative to the subtext has been obtained, and the above assumption is continued, that is, it is assumed that the i-th sentence in the sentence sequence corresponding to the subtext The attention feature vector B _i relative to the subtext is a matrix of S*1, where the element B _i,j,1 in the jth row and column 1 of the matrix B _i is used to represent the sentence sequence corresponding to the subtext The degree of relevance, importance, or degree of attention between the i-th sentence and the j-th sentence. And assuming that the subtext includes S sentences, then according to the position of the sentence in the sentence sequence corresponding to the subtext, the attention feature vector B _i of each sentence relative to the subtext is combined, which can be Obtain an attention expression matrix B whose size is S*S for the subtext, and the element B _i,j in the attention expression matrix is used to represent the relationship between the i-th sentence and the j-th sentence in the sub-text Importance, relevance, or attention.

However, the calculation of the attention feature vector of the subtext relative to the text to be recognized can be expressed as follows:

Suppose there are P subtexts in the subtext sequence corresponding to the text to be recognized. P is a positive integer. Wherein, the attention representation matrix corresponding to each subtext is a matrix with a size of S*S. Assume that for the pth subtext in the subtext sequence, the attention representation matrix corresponding to the pth subtext is C _p , and C _p is a matrix of S*S. According to the position of each subtext in the subtext sequence, the three-dimensional matrix C of P*S*S can be obtained by combining the attention feature matrix C _p corresponding to each subtext. Assume that the attention feature vector of the pth subtext relative to the text to be recognized is a matrix E _p , then the attention feature matrix C _p of the pth subtext and the attention feature matrix E _p of the pth subtext relative to the text to be recognized The Cartesian product should be a matrix C, which can be expressed as follows:

C _p ×E _p =C formula (2)

It can be seen from the above formula that the matrix E _p is a matrix of P*1, where the element E _p,q,1 in the first column of the qth row in the matrix E _p is used to represent the pth subtext and the first subtext in the text to be recognized Relevance, importance or attention among q subtexts.

When specifically calculating the above matrix _Ep , the attention feature matrix Ep of the _pth subtext in the subtext sequence corresponding to the text to be recognized relative to the text to be recognized can be obtained by calculating the known matrix C and matrix _Cp , Since the matrix E _p is a matrix of P*1, it can also be considered that the matrix E _p is the attention feature vector of the p-th subtext in the subtext sequence corresponding to the text to be recognized relative to the text to be recognized.

Step 204, splicing the attention feature vectors of the subtexts in the subtext sequence relative to the text to be recognized to obtain the text feature vector to be recognized corresponding to the text to be recognized.

In this embodiment, for example, according to the position of the subtext in the subtext sequence in the subtext sequence, the execution subject can splice the attention feature vectors of the subtext in the subtext sequence relative to the text to be recognized to obtain the corresponding The feature vector of the text to be recognized.

Here, continuing to use the above example, the attention feature vector E _p of each subtext in the P subtexts in the splicing subtext sequence relative to the text to be recognized can obtain the text feature vector E to be recognized, and the dimension of the text feature vector E to be recognized is P*P.

Step 205, input the feature vector of the text to be recognized into the pre-trained classification model to obtain the probability value that the text to be recognized belongs to the preset category of text.

In this embodiment, the execution subject can input the feature vector of the text to be recognized whose dimension is P*P calculated in step 204 into the pre-trained classification model to obtain the probability value that the text to be recognized belongs to the preset category of text. Wherein, the classification model is used to characterize the corresponding relationship between the text feature vector and the probability value that the text belongs to the preset category text.

In some optional embodiments, the feature extraction model and the classification model may be pre-trained through the training step 300 as shown in Figure 3, and the training step 300 may include the following steps 301 to 3:

Step 301, determining an initial feature extraction model and an initial classification model.

Here, the subject of the training step may be the same as or different from the subject of the text category recognition method. If they are the same, the execution subject of the training step can store the model structure information and the parameter values of the model parameters of the trained feature extraction model and classification model locally after the training obtains the feature extraction model and classification model. If different, the execution subject of the training step can send the model structure information and the parameter values of the model parameters of the trained feature extraction model and classification model to the execution subject of the text category recognition method after training the feature extraction model and the classification model.

Here, since the initial feature extraction model and the initial classification model may include various types of calculation models, the model structure information that needs to be determined is different for different types of calculation models.

Then, the model parameters of the initial feature extraction model and the initial classification model can be initialized. In practice, each model parameter of the initial feature extraction model and the initial classification model can be initialized with some different small random numbers. "Small random number" is used to ensure that the model will not enter a saturated state due to excessive weight, which will cause training failure, and "different" is used to ensure that the model can learn normally.

Optionally, the initial classification model may be a Softmax classifier.

Step 302, acquiring a training sample set.

Here, the training samples in the training sample set include sample text and a sample label used to represent whether the sample text belongs to a preset category of text. In practice, sample labels can be obtained by manual annotation.

Step 303, for the training samples in the training sample set, perform parameter adjustment operations until the preset training end condition is satisfied.

Here, parameter adjustment operations may include:

Step 3031, splitting the sample text in the training sample to obtain a sample subtext sequence, and splitting each subtext in the sample subtext sequence to obtain a corresponding sentence sequence. In practice, the same or similar method in step 201 may be followed.

Step 3032, for each sentence in the sentence sequence corresponding to each sample subtext in the sample subtext sequence, perform feature extraction according to the initial feature extraction model to obtain a sentence feature vector corresponding to the sentence.

Step 3033, for each sample subtext in the sample subtext sequence, perform a second calculation operation to obtain the attention feature vector of the sample subtext relative to the sample text. Wherein, the second computing operation includes the following first to fourth steps:

The first step is to calculate the attention feature vector of the sentence relative to the sample subtext based on the sentence feature vector corresponding to each sentence in the sentence sequence corresponding to the sample subtext.

The second step is to calculate the attention feature vector of the sample subtext relative to the sample text based on the attention feature vector of each sentence relative to the sample subtext.

Here, the specific operations of the first step and the second step are basically the same as the operations of step 2031 and step 2032 respectively, and will not be repeated here.

In the third step, the attention feature vector of the sample subtext in the sample subtext sequence relative to the sample text is spliced to obtain the sample text feature vector corresponding to the sample text.

Here, the specific operation of the third step is basically the same as the operation of step 204, and will not be repeated here.

The fourth step is to input the obtained sample text feature vector into the initial classification model to obtain the probability value that the sample text belongs to the preset category text.

Step 3034, based on the difference between the obtained probability value and the sample label in the training sample, adjust the model parameters of the initial feature extraction model and the initial classification model.

Here, various implementation manners may be adopted to adjust model parameters of the initial feature extraction model and the initial classification model based on the difference between the obtained probability value and the sample label in the training sample. For example, stochastic gradient descent (SGD, Stochastic Gradient Descent), Newton's Method, Quasi-Newton Methods, Conjugate Gradient, heuristic optimization methods and other known Or various optimization algorithms developed in the future.

In step 304, the trained initial feature extraction model and initial classification model are determined as pre-trained feature extraction models and classification models.

In some optional implementation manners, the initial feature extraction model may include a word vector feature extraction model and a sentence vector feature extraction model. Correspondingly, step 3032, for each sentence in the sentence sequence corresponding to each sample subtext in the sample subtext sequence, perform feature extraction according to the initial feature extraction model to obtain the sentence feature vector corresponding to the sentence, which can be performed as follows:

For each sentence in the sentence sequence corresponding to each sample subtext in the sample subtext sequence, first, perform feature extraction on each word segmentation sequence corresponding to the sentence according to the word vector feature extraction model to obtain the corresponding word vector, and then Combine the word vectors corresponding to each word in the word segmentation sequence corresponding to the sentence to form the sentence feature matrix corresponding to the sentence, and finally perform feature extraction on the sentence feature matrix corresponding to the sentence according to the sentence vector feature extraction model to obtain the sentence feature vector corresponding to the sentence . For details, reference may be made to the description of the corresponding optional modes of the word vector feature extraction model and the sentence vector feature extraction model in step 202, which will not be repeated here.

Based on the above optional implementation, optionally, before combining the word vectors corresponding to each word in the word segmentation sequence corresponding to the sentence to form the sentence feature matrix corresponding to the sentence, the execution subject of the training step can also: For each word segment in the word segment sequence, in response to determining that the word segment matches a keyword in the preset text category keyword set, the word vector corresponding to the word segment is set as the preset word vector. As an example, the preset word vector may be a word vector in which all vector components are 0. In this way, by specifying the word vectors corresponding to the words that match the keywords in the preset text category keyword set, the recognition ability of the feature extraction model and classification model for the text of the preset text category can be improved.

Here, the preset text category keyword set can be dynamically learned from a large amount of corpus using machine learning or data mining algorithms, or it can be manually formulated by technicians according to the needs and experience of specific application scenarios. The preset text category keywords The word set can also include both dynamically learned keywords and manually specified keywords.

Using the training steps shown in Figure 3, automatic training can be implemented to obtain the feature extraction model and classification model.

Through steps 201 to 205, the probability value of the text to be recognized belonging to the preset category can be obtained.

In some optional implementation manners, the above execution subject may also perform the following step 206 after step 205:

Step 206, determine whether the probability value is greater than a preset probability threshold.

If determined to be greater than, go to step 207 for execution.

Step 207, generating first recognition result information for indicating that the text to be recognized is a preset text category.

In this way, the above-mentioned first recognition result information can be used to determine that the text to be recognized belongs to the text category.

In some optional implementation manners, the execution subject may also proceed to step 208 for execution if it is determined in step 206 that the value is not greater than.

Step 208, generating second recognition result information for indicating that the text to be recognized is not a preset text category.

In this way, it can be determined that the text to be recognized does not belong to the text category based on the first recognition result information.

In some optional implementation manners, the above execution subject may also perform the following step 209 at other time points after step 2031, for example, before step 204, or before step 205, or after step 205:

Step 209, for each sentence in the sentence sequence corresponding to each subtext in the subtext sequence, based on the attention feature vector of the sentence relative to the subtext, calculate the probability value that the sentence belongs to the preset text category, according to the calculated The probability value of determines the presentation manner corresponding to the sentence, and presents the sentence according to the determined presentation manner.

Continue to use the example in step 2031, that is, assume that the attention feature vector of a certain sentence relative to the subtext to which it belongs is a matrix B _i , and the matrix B _i is a matrix of S*1, wherein the jth row and the first column of the matrix B _i The element B _{i, j, 1} is used to represent the correlation, importance or attention between the i-th sentence and the j-th sentence in the sentence sequence corresponding to the subtext, and j is a positive integer between 1 and S . Then, based on the attention feature vector B _i of the sentence relative to the subtext, the probability value of the sentence belonging to the preset text category is calculated, for example, it can be:

The _sum of the elements of Bi is calculated, or the sum of the squares of the calculated elements is used as the probability value that the sentence belongs to the preset text category.

And according to the calculated probability value to determine the corresponding presentation mode of the sentence, for example, the corresponding relationship between different probability value ranges and corresponding presentation modes can be set in advance, and then when the calculated probability value belongs to the corresponding probability value range, the The presentation manner corresponding to the corresponding probability value range is determined as the presentation manner corresponding to the sentence. For example, when the probability value is greater than 0.8, the presentation method is to use red font. When the probability value is greater than 0.5 and less than 0.8, it will be presented in pink font.

In some optional implementation manners, the above execution subject may also perform the following step 210 at other time points after step 2032, for example, before step 204, or before step 205, or after step 205:

Step 210, for each subtext in the subtext sequence, based on the attention feature vector of the subtext relative to the text to be recognized, calculate the probability value that the subtext belongs to the preset text category, and determine the subtext according to the calculated probability value. The presenting manner corresponding to the text, and presenting the subtext according to the determined presenting manner.

Continue to use the example in step 2032, that is, assume that the attention feature vector of a certain subtext relative to the text to be recognized is a matrix _Ep of P*1, wherein the elements _Ep _{, q, and 1} is used to indicate the degree of relevance, importance or attention between the pth subtext and the qth subtext in the text to be recognized. Then, based on the attention feature vector E _p of the subtext relative to the text to be recognized, the probability value of the subtext belonging to the preset text category is calculated, for example, it may be:

The sum of each element of E _p is calculated, or the calculated sum of the squares of each element is used as a probability value that the subtext belongs to a preset text category.

And according to the calculated probability value to determine the presentation mode corresponding to the subtext, for example, by pre-setting the corresponding relationship between different probability value ranges and corresponding presentation modes, and then when the calculated probability value belongs to the corresponding probability value range, The presentation manner corresponding to the corresponding probability value range is determined as the presentation manner corresponding to the text. For example, when the probability value is greater than 0.9, it is presented in bold font. When the probability value is greater than 0.6 and less than 0.9, the presentation method is normal font.

Continuing to refer to FIG. 4 , FIG. 4 is a schematic diagram of an application scenario of the text category recognition method according to this embodiment. In the application scenario of FIG. 4 , firstly, the server 41 acquires the text 43 to be recognized from the terminal device 42 . Then the server 41 splits the text to be recognized 43 to obtain a subtext sequence 44, and splits the

subtexts

441, 442 and 443 in the subtext sequence 44 to obtain corresponding sentence sequences 451, 452 and 453. The sentence sequence 451 includes Sentence 45101 to sentence 45120, sentence sequence 452 includes sentence 45201 to sentence 45222, and sentence sequence 453 includes sentence 45201 to sentence 45325. For each sentence in 45101 to sentence 45120, sentence 45201 to sentence 45222, and sentence 45301 to sentence 45325, the server 41 performs feature extraction according to the pre-trained feature extraction model to obtain the sentence feature vector 46101 to sentence feature vector 46120 corresponding to the sentence. , sentence feature vector 46201 to sentence feature vector 46222, sentence feature vector 46201 to sentence feature vector 46225. Then, the server 41 respectively performs the first calculation operation on the

subtexts

441, 442 and 443 in the subtext sequence 44, and respectively obtains

attention feature vectors

471, 472 and 473 of 441, 442 and 443 relative to the text to be recognized 43.

Next, the server 41 concatenates the

attention feature vectors

471 , 472 and 473 to obtain the text feature vector 48 corresponding to the text to be recognized. Finally, input the feature vector 48 of the text to be recognized into the pre-trained classification model 49 to obtain the probability value 50 that the text to be recognized belongs to the preset category of text.

The text category recognition method provided by the above-mentioned embodiments of the present disclosure realizes the establishment of hierarchical attention among sentences, subtexts and texts to be recognized through the attention feature vector of the sentence relative to the subtext and the attention feature vector of the subtext relative to the text to be recognized relationship, and then generate the feature vector of the text to be recognized to calculate the probability value belonging to the preset text category, which realizes the automatic classification of the text to be recognized and reduces the labor cost of text classification.

Further referring to FIG. 5 , as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a text category recognition device, which corresponds to the method embodiment shown in FIG. 2 , and the device specifically It can be applied to various electronic devices.

As shown in FIG. 5 , the text category recognition device 500 of this embodiment includes: a splitting unit 501 , a feature extraction unit 502 , a calculation unit 503 , a splicing unit 504 and a recognition unit 505 . Wherein, the splitting unit 501 is configured to split the text to be recognized to obtain a subtext sequence, and split each subtext in the subtext sequence to obtain a corresponding sentence sequence; the feature extraction unit 502 is configured to Each sentence in the sentence sequence corresponding to each sub-text is subjected to feature extraction according to a pre-trained feature extraction model to obtain a sentence feature vector corresponding to the sentence; the calculation unit 503 is configured to for each of the sub-text sequences subtext, perform the following first calculation operation: for each sentence in the subtext, based on the sentence feature vector corresponding to each sentence in the sentence sequence corresponding to the subtext, calculate the attention feature of the sentence relative to the subtext Vector; Based on the attention feature vector of each sentence relative to the subtext, calculate the attention feature vector of the subtext relative to the text to be recognized; splicing unit 504, configured to splice the subtext in the subtext sequence With respect to the attention feature vector of the text to be recognized, the text feature vector to be recognized corresponding to the text to be recognized is obtained; the recognition unit 505 is configured to input the text feature vector to be recognized into a pre-trained classification model to obtain The probability value that the text to be recognized belongs to a preset category of text.

In this embodiment, the specific processing of the splitting unit 501, the feature extraction unit 502, the calculation unit 503, the splicing unit 504, and the recognition unit 505 of the text category recognition device 500 and the technical effects brought about by them can refer to FIG. 2 respectively. The relevant descriptions of step 201, step 202, step 203, step 204 and step 205 in the embodiment will not be repeated here.

In some optional implementation manners, the feature extraction model and the classification model can be obtained through pre-training as follows:

In some optional implementation manners, the feature extraction model may include a word vector feature extraction model and a sentence vector feature extraction model; and

The feature extraction unit 502 may be further configured to:

In some optional implementation manners, the word vector feature extraction model may include at least one of the following: a long short-term memory network, and a translation model.

In some optional implementation manners, the sentence vector feature extraction model may include at least one of the following: a convolutional neural network, and a bidirectional long-short-term memory network.

In some optional implementation manners, for each sentence in the sentence sequence corresponding to each sample subtext in the sample subtext sequence, perform feature extraction according to the initial feature extraction model to obtain the sentence features corresponding to the sentence vector, which can include:

In some optional implementation manners, before combining the word vectors corresponding to each word in the word segmentation sequence corresponding to the sentence to form the sentence feature matrix corresponding to the sentence, the training step may also include:

In some optional implementation manners, the device 500 may also include:

A determining unit 506, configured to determine whether the probability value is greater than a preset probability threshold;

The first generating unit 507 is configured to generate first recognition result information indicating that the text to be recognized is a preset text category in response to determining that the value is greater than or equal to greater than 100%.

In some optional implementation manners, the device 500 may also include:

The second generation unit 508 is configured to, in response to determining that it is not greater than, generate second recognition result information for indicating that the text to be recognized is not a preset text category.

In some optional implementation manners, the device 500 may also include:

The first presentation unit 509 is configured to, for each sentence in the sentence sequence corresponding to each subtext in the subtext sequence, calculate that the sentence belongs to the predetermined sentence based on the attention feature vector of the sentence relative to the subtext. Assuming the probability value of the text category, determine the presentation mode corresponding to the sentence according to the calculated probability value, and present the sentence according to the determined presentation mode.

In some optional implementation manners, the device 500 may also include:

The second presentation unit 510 is configured to, for each subtext in the subtext sequence, calculate the probability that the subtext belongs to the preset text category based on the attention feature vector of the subtext relative to the text to be recognized. A probability value, determining a presentation manner corresponding to the subtext according to the calculated probability value, and presenting the subtext according to the determined presentation manner.

It should be noted that, for the implementation details and technical effects of each unit in the text category recognition device provided by the embodiments of the present disclosure, reference may be made to the descriptions of other embodiments of the present disclosure, and details are not repeated here.

Referring now to FIG. 6 , it shows a schematic structural diagram of a computer system 600 suitable for implementing the electronic device of the present disclosure. The computer system 600 shown in FIG. 6 is only an example, and should not limit the functions and scope of use of the embodiments of the present disclosure.

As shown in FIG. 6, a computer system 600 may include a processing device (eg, a central processing unit, a graphics processing unit, etc.) 601 that may be accessed randomly according to a program stored in a read-only memory (ROM) 602 or loaded from a storage device 608. Various appropriate actions and processes are executed by programs in the memory (RAM) 603 . In the RAM 603, various programs and data necessary for the operation of the computer system 600 are also stored. The processing device 601, ROM 602, and RAM 603 are connected to each other through a bus 604. An input/output (I/O) interface 605 is also connected to the bus 604 .

Generally, the following devices can be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, etc.; output devices 607, including, for example, a liquid crystal display (LCD), speaker, vibrator, etc. ; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609 . The communication means 609 may allow the computer system 600 to communicate with other devices wirelessly or by wire to exchange data. While FIG. 6 shows a computer system 600 of electronic devices having various means, it should be understood that implementing or possessing all of the illustrated means is not a requirement. More or fewer means may alternatively be implemented or provided.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, where the computer program includes program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via communication means 609, or from storage means 608, or from ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are executed.

It should be noted that the above-mentioned computer-readable medium in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. A computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can transmit, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device . Program code embodied on a computer readable medium may be transmitted by any appropriate medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

The above-mentioned computer-readable medium may be included in the above-mentioned electronic device, or may exist independently without being incorporated into the electronic device.

The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device realizes the text shown in the embodiment shown in FIG. 2 and its optional implementation manners. Class recognition method.

Computer program code for carrying out the operations of the present disclosure can be written in one or more programming languages, or combinations thereof, including object-oriented programming languages—such as Java, Smalltalk, C++, and conventional Procedural Programming Language - such as "C" or a similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (such as through an Internet Service Provider). Internet connection).

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or portion of code that contains one or more logical functions for implementing specified executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified functions or operations , or may be implemented by a combination of dedicated hardware and computer instructions.

The units involved in the embodiments described in the present disclosure may be implemented by software or by hardware. Wherein, the name of the unit does not constitute a limitation on the unit itself under certain circumstances, for example, the acquisition unit may also be described as "a unit that acquires the text to be recognized".

The above description is only a preferred embodiment of the present disclosure and an illustration of the applied technical principles. Those skilled in the art should understand that the disclosure scope involved in this disclosure is not limited to the technical solution formed by the specific combination of the above-mentioned technical features, but also covers the technical solutions formed by the above-mentioned technical features or Other technical solutions formed by any combination of equivalent features. For example, a technical solution formed by replacing the above-mentioned features with technical features disclosed in this disclosure (but not limited to) having similar functions.

Claims

A text category recognition method, comprising:

Splitting the text to be recognized to obtain a subtext sequence, and splitting each subtext in the subtext sequence to obtain a corresponding sentence sequence;

For each sentence in the sentence sequence corresponding to each said subtext, perform feature extraction according to a pre-trained feature extraction model to obtain a sentence feature vector corresponding to the sentence;

For each subtext in the subtext sequence, the following first calculation operation is performed: for each sentence in the subtext, based on the sentence feature vector corresponding to each sentence in the sentence sequence corresponding to the subtext, calculate the sentence With respect to the attention feature vector of the subtext; based on the attention feature vector of each sentence relative to the subtext, calculate the attention feature vector of the subtext with respect to the text to be identified;

Splicing the subtext in the subtext sequence relative to the attention feature vector of the text to be recognized to obtain the text feature vector to be recognized corresponding to the text to be recognized;

Inputting the feature vector of the text to be recognized into a pre-trained classification model to obtain a probability value that the text to be recognized belongs to a preset category of text.
The method according to claim 1, wherein the feature extraction model and the classification model are pre-trained through the following training steps:

Determine the initial feature extraction model and initial classification model;

Obtain a training sample set, wherein the training sample includes sample text and a sample label for representing whether the sample text belongs to a preset category of text;

For the training samples in the training sample set, the following parameter adjustment operations are performed until the preset training end condition is met: the sample text in the training sample is split to obtain a sample subtext sequence, and each of the sample subtext sequences is The subtexts are split to obtain the corresponding sentence sequence; for each sentence in the sentence sequence corresponding to each sample subtext in the sample subtext sequence, perform feature extraction according to the initial feature extraction model to obtain the sentence feature corresponding to the sentence Vector; for each sample subtext in the sample subtext sequence, perform a second calculation operation to obtain the attention feature vector of the sample subtext relative to the sample text: based on the sentence sequence corresponding to the sample subtext The sentence feature vector corresponding to each sentence calculates the attention feature vector of the sentence relative to the sample subtext; based on the attention feature vector of each sentence relative to the sample subtext, calculates the sample subtext relative to the sample The attention feature vector of text; Splicing the attention feature vector of sample subtext in described sample subtext sequence with respect to this sample text, obtain the sample text feature vector corresponding to this sample text; Input the obtained sample text feature vector into the The initial classification model to obtain the probability value that the sample text belongs to the preset category text; based on the difference between the obtained probability value and the sample label in the training sample, adjust the initial feature extraction model and the initial Model parameters for classification models;

Determining the trained initial feature extraction model and the initial classification model as the pre-trained feature extraction model and the classification model.
The method according to claim 2, wherein the feature extraction model comprises a word vector feature extraction model and a sentence vector feature extraction model; and

For each sentence in the sentence sequence corresponding to each of the subtexts, perform feature extraction according to a pre-trained feature extraction model to obtain a sentence feature vector corresponding to the sentence, including:

For each sentence in the sentence sequence corresponding to each of the subtexts, perform feature extraction according to the word vector feature extraction model for each participle in the word segment sequence corresponding to the sentence to obtain a corresponding word vector, and combine the corresponding word vectors of the sentence The word vector corresponding to each word segment in the word segmentation sequence is used to form the sentence feature matrix corresponding to the sentence, and the sentence feature matrix corresponding to the sentence is extracted according to the sentence vector feature extraction model to obtain the sentence feature vector corresponding to the sentence.
The method according to claim 3, wherein the word vector feature extraction model includes at least one of the following: a long short-term memory network, and a translation model.
The method according to claim 3, wherein the sentence vector feature extraction model comprises at least one of the following: a convolutional neural network, a bidirectional long-short-term memory network.
The method according to claim 3, wherein, for each sentence in the sentence sequence corresponding to each sample subtext in the sample subtext sequence, perform feature extraction according to the initial feature extraction model to obtain the sentence corresponding to the sentence. Sentence feature vectors, including:

For each sentence in the sentence sequence corresponding to each sample subtext in the sample subtext sequence, each word segmentation in the word segmentation sequence corresponding to the sentence is subjected to feature extraction according to the word vector feature extraction model to obtain a corresponding word vector Combining the word vectors corresponding to each word in the word segmentation sequence corresponding to the sentence to form the sentence feature matrix corresponding to the sentence, performing feature extraction on the sentence feature matrix corresponding to the sentence according to the sentence vector feature extraction model to obtain the sentence corresponding to the sentence Feature vector.
The method according to claim 6, wherein, before the word vector corresponding to each participle in the participle sequence corresponding to the combination of the sentence to form the sentence feature matrix corresponding to the sentence, the training step also includes:

For each participle in the participle sequence corresponding to the sentence, in response to determining that the participle matches the keyword in the preset text category keyword set, the word vector corresponding to the participle is set as the preset word vector.
The method according to claim 1, wherein the method further comprises:

determining whether the probability value is greater than a preset probability threshold;

In response to determining that the value is greater than, first recognition result information for indicating that the text to be recognized is a preset text category is generated.
The method according to claim 8, wherein the method further comprises:

In response to determining that it is not greater than, second recognition result information for indicating that the text to be recognized is not a preset text category is generated.
The method according to claim 1, wherein the method further comprises:

For each sentence in the sentence sequence corresponding to each subtext in the subtext sequence, based on the attention feature vector of the sentence relative to the subtext, calculate the probability value that the sentence belongs to the preset text category, according to the calculation The obtained probability value determines the presentation manner corresponding to the sentence, and presents the sentence according to the determined presentation manner.
The method according to claim 1, wherein the method further comprises:

For each subtext in the subtext sequence, based on the attention feature vector of the subtext relative to the text to be recognized, calculate the probability value that the subtext belongs to the preset text category, according to the calculated probability value A presentation manner corresponding to the subtext is determined, and the subtext is presented according to the determined presentation manner.
A text category recognition device, comprising:

The splitting unit is configured to split the text to be recognized to obtain a subtext sequence, and split each subtext in the subtext sequence to obtain a corresponding sentence sequence;

The feature extraction unit is configured to perform feature extraction for each sentence in the sentence sequence corresponding to each subtext according to a pre-trained feature extraction model to obtain a sentence feature vector corresponding to the sentence;

The calculation unit is configured to perform the following first calculation operation for each subtext in the subtext sequence: for each sentence in the subtext, based on the sentence corresponding to each sentence in the sentence sequence corresponding to the subtext A feature vector, calculating the attention feature vector of the sentence relative to the subtext; calculating the attention feature vector of the subtext relative to the text to be identified based on the attention feature vector of each sentence relative to the subtext;

The splicing unit is configured to splice the attention feature vectors of the subtexts in the subtext sequence relative to the text to be recognized to obtain the text feature vector to be recognized corresponding to the text to be recognized;

The recognition unit is configured to input the feature vector of the text to be recognized into a pre-trained classification model to obtain a probability value that the text to be recognized belongs to a preset category of text.
An electronic device comprising:

one or more processors;

a storage device on which one or more programs are stored,

When the one or more programs are executed by the one or more processors, the one or more processors are made to implement the method according to any one of claims 1-11.
A computer-readable storage medium on which a computer program is stored, wherein the computer program implements the method according to any one of claims 1-11 when executed by one or more processors.