CN112818688B

CN112818688B - Text processing method, device, equipment and storage medium

Info

Publication number: CN112818688B
Application number: CN202110413353.4A
Authority: CN
Inventors: 吴焕钦; 刘维
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2021-04-16
Filing date: 2021-04-16
Publication date: 2021-06-25
Anticipated expiration: 2041-04-16
Also published as: CN112818688A

Abstract

The embodiment of the application relates to the field of artificial intelligence, and discloses a text processing method which comprises the following steps: coding the target text through the trained text processing model to obtain an expression vector of the target text; performing key phrase extraction processing on the representation vector of the target text through a first stacking relation network to obtain a representation vector of a first key phrase corresponding to the target text, and performing key phrase generation processing on the representation vector of the target text through the first stacking relation network to obtain a representation vector of a second key phrase corresponding to the target text; decoding the expression vector of the first key phrase corresponding to the target text to obtain a first key phrase corresponding to the target text, and decoding the expression vector of the second key phrase corresponding to the target text to obtain a second key phrase corresponding to the target text; and generating a key phrase set corresponding to the target text, so that the prediction precision of predicting key phrases according to the text can be improved.

Description

Text processing method, device, equipment and storage medium

Technical Field

The present application relates to the field of artificial intelligence, and in particular, to a text processing method, apparatus, device, and storage medium.

Background

The key phrases refer to phrase-level descriptions capable of describing the text content and the central subject of a certain text (such as an article or a piece of text information), and the prediction of the key phrases from the text is an important research direction in the field of natural language processing in the field of artificial intelligence. Currently, the method for predicting key phrases generally extracts key phrases existing in texts from the texts. However, the key phrases predicted by this method are only phrases existing in the text, resulting in low prediction accuracy.

Disclosure of Invention

The embodiment of the application provides a text processing method, a text processing device and a text processing storage medium, which can improve the prediction precision of predicting key phrases according to texts.

In one aspect, an embodiment of the present application provides a text processing method, including:

coding a target text through a trained text processing model to obtain a representation vector of the target text;

performing key phrase extraction processing on the representation vector of the target text through a first stacking relation network to obtain a representation vector of a first key phrase corresponding to the target text, and performing key phrase generation processing on the representation vector of the target text through the first stacking relation network to obtain a representation vector of a second key phrase corresponding to the target text, wherein the first stacking relation network is a network in the trained text processing model, the first key phrase corresponding to the target text is a key phrase existing in the target text, and the second key phrase corresponding to the target text is a key phrase not existing in the target text;

decoding the representation vector of the first key phrase corresponding to the target text to obtain a first key phrase corresponding to the target text, and decoding the representation vector of the second key phrase corresponding to the target text to obtain a second key phrase corresponding to the target text;

and generating a key phrase set corresponding to the target text, wherein the key phrase set comprises a first key phrase corresponding to the target text and a second key phrase corresponding to the target text.

In one aspect, an embodiment of the present application provides a text processing apparatus, including:

the encoding unit is used for encoding a target text through the trained text processing model to obtain a representation vector of the target text;

a vector processing unit, configured to perform a key phrase extraction process on a representation vector of the target text through a first stacking relationship network to obtain a representation vector of a first key phrase corresponding to the target text, and perform a key phrase generation process on the representation vector of the target text through the first stacking relationship network to obtain a representation vector of a second key phrase corresponding to the target text, where the first stacking relationship network is a network in the trained text processing model, the first key phrase corresponding to the target text is a key phrase existing in the target text, and the second key phrase corresponding to the target text is a key phrase not existing in the target text;

the decoding unit is used for decoding the representation vector of the first key phrase corresponding to the target text to obtain the first key phrase corresponding to the target text, and decoding the representation vector of the second key phrase corresponding to the target text to obtain the second key phrase corresponding to the target text;

a generating unit, configured to generate a set of key phrases corresponding to the target text, where the set of key phrases includes a first key phrase corresponding to the target text and a second key phrase corresponding to the target text.

In one aspect, an embodiment of the present application provides a text processing device, where the text processing device includes an input interface and an output interface, and further includes:

a processor adapted to implement one or more instructions; and the number of the first and second groups,

a computer storage medium having stored thereon one or more instructions adapted to be loaded by the processor and to execute the above-described text processing method.

In one aspect, an embodiment of the present application provides a computer storage medium, where computer program instructions are stored in the computer storage medium, and when the computer program instructions are executed by a processor, the computer storage medium is configured to execute the text processing method.

In one aspect, embodiments of the present application provide a computer program product or a computer program, where the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; the processor of the text processing device reads the computer instructions from the computer readable storage medium, and executes the computer instructions, and the computer instructions, when executed by the processor, are used for executing the text processing method.

In the embodiment of the application, the text processing equipment encodes the target text through the trained text processing model to obtain a representation vector of the target text; performing key phrase extraction processing on the representation vector of the target text through a first stacking relation network to obtain a representation vector of a first key phrase corresponding to the target text, and performing key phrase generation processing on the representation vector of the target text through the first stacking relation network to obtain a representation vector of a second key phrase corresponding to the target text, wherein the first stacking relation network is a network in a trained text processing model, the first key phrase corresponding to the target text is a key phrase existing in the target text, and the second key phrase corresponding to the target text is a key phrase not existing in the target text; decoding the expression vector of the first key phrase corresponding to the target text to obtain a first key phrase corresponding to the target text, and decoding the expression vector of the second key phrase corresponding to the target text to obtain a second key phrase corresponding to the target text; and finally, generating a key phrase set corresponding to the target text, wherein the key phrase set comprises a first key phrase corresponding to the target text and a second key phrase corresponding to the target text. The text processing equipment performs key phrase extraction processing on the representation vector of the target text through the first stacking relation network so as to predict key phrases existing in the target text, performs key phrase generation processing on the representation vector of the target text through the first stacking relation network so as to predict key phrases not existing in the target text, and can predict key phrases of the target text in a mode of combining extraction and generation so as to improve the prediction precision of predicting key phrases according to the text.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

FIG. 1 is a schematic diagram of a text processing model provided by an embodiment of the present application;

FIG. 2 is a schematic diagram of a training process of a text processing model according to an embodiment of the present disclosure;

FIG. 3 is a schematic diagram of a single-layer MLP interworking network provided by an embodiment of the present application;

FIG. 4 is a schematic diagram of a single-layer Co-Attention interaction network provided by an embodiment of the present application;

FIG. 5 is a diagram of a trained text processing model according to an embodiment of the present application;

fig. 6 is a schematic flowchart of a text processing method according to an embodiment of the present application;

FIG. 7 is a schematic structural diagram of a text processing apparatus according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a text processing device according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Artificial Intelligence (AI) is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human Intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making.

The artificial intelligence technology is a comprehensive subject and relates to the field of extensive technology, namely the technology of a hardware level and the technology of a software level. The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.

The embodiment of the application mainly relates to the field of Natural Language Processing (NLP) in the field of artificial intelligence, wherein natural Language processing is an important direction in the fields of computer science and artificial intelligence, and various theories and methods for realizing effective communication between people and computers by using natural Language are researched. Predicting key phrases from text is an important research direction in the field of natural language processing, but currently, the prediction accuracy of the existing method for predicting key phrases from text is low. Based on this, the embodiment of the application provides a text processing scheme, which can encode a target text through a trained text processing model to obtain a representation vector of the target text; and then, performing key phrase extraction processing on the representation vector of the target text through the first stacking relation network to predict key phrases existing in the target text, performing key phrase generation processing on the representation vector of the target text through the first stacking relation network to predict key phrases not existing in the target text, and obtaining a key phrase set corresponding to the target text, wherein the first stacking relation network is a network in a trained text processing model, and the prediction accuracy of predicting key phrases according to the text can be improved.

The key phrases refer to a phrase-level description that can describe the text content of a certain text (e.g., an article or a piece of text information, etc.) and the central subject matter, and the key phrases of the text can be predicted based on the understanding of the text content of the text. Generally, a text generally corresponds to a plurality of key phrases, and the plurality of key phrases of the text are divided into key phrases existing in the text and key phrases not existing in the text according to whether the key phrases exist in the text or not; since both key-phrases present in the text and key-phrases not present in the text may describe the textual content and the central subject matter of the text, there is a correlation between key-phrases present in the text and key-phrases not present in the text. Moreover, as the key phrases can describe the text content and the central subject matter of the text, for the unstructured text, the key phrases can be predicted according to the unstructured text, so that the structured representation of the text can be obtained; the key phrase is predicted according to the text and can be used for relevant processing of user portrait analysis, recall, rough typesetting, fine typesetting and the like of a recommendation system.

For example, if a text is "title: natural language processing techniques for developing a language learning environment. And (3) abstract: up to now, forms of computer-assisted language learning are various. Our research efforts have focused on developing an integrated e-learning environment that can improve language skills in certain situations. An integrated e-learning environment means that it is a world wide Web (Web) based solution, such as a Web browser or email client. It should be accessible. Natural language processing forms the technical basis for building such a learning framework. ", the key phrases of the text are: natural language processing, computer-aided language learning, integrated e-learning, world wide web technology, foreign language learning; the key phrases present in the text are natural language processing, computer-aided language learning, integrated e-learning; the key phrases not present in the text are web technologies, foreign language learning.

The text processing scheme can be realized by a trained text processing model, and the trained text processing model is obtained by training the text processing model in advance by using a large number of training samples, wherein the training samples comprise an initial training text, a first training phrase and a second training phrase, the first training phrase is a key phrase existing in the initial training text, and the second training phrase is a key phrase not existing in the initial training text. As shown in fig. 1, which is a schematic diagram of a text processing model provided in an embodiment of the present application, the text processing model may include a pre-training language model and a second stacking relationship network (relationship Layer). Based on the text processing model, the embodiment of the application provides a training scheme of the text processing model, which can encode a target training text comprising an initial training text and a masked second training phrase through a pre-training language model in the text processing model to obtain a representation vector of the target training text, wherein the masked second training phrase is obtained by masking the second training phrase; performing key phrase extraction processing on the representation vector of the target training text through a second stacking relation network in the text processing model to obtain a representation vector of a first key phrase corresponding to the target training text, and performing key phrase generation processing on the representation vector of the target training text through the second stacking relation network to obtain a representation vector of a second key phrase corresponding to the target training text; decoding the expression vector of the first key phrase corresponding to the target training text to obtain a first key phrase corresponding to the target training text, and decoding the expression vector of the second key phrase corresponding to the target training text to obtain a second key phrase corresponding to the target training text; and then training the text processing model based on a first key phrase corresponding to the target training text, a second key phrase corresponding to the target training text, the first training phrase and the second training phrase to obtain the trained text processing model.

In one embodiment, the training scheme and the text processing scheme of the text processing model may be executed by a text processing device, which may be a computer, a smart phone, a tablet computer, a notebook computer, or the like. Optionally, the training scheme of the text processing model and the text processing scheme may be executed by the same text processing device, that is, the text processing device trains the text processing model in advance to obtain the trained text processing model, and then the text processing scheme may be executed based on the trained text processing model; alternatively, the training scheme of the text processing model and the text processing scheme may be executed by different text processing apparatuses, for example, the training scheme of the text processing model may be executed by a first text processing apparatus, and the text processing scheme may be executed by a second text processing apparatus; the second text processing equipment can acquire the trained text processing model from the first text processing equipment, and further can execute a text processing scheme based on the trained text processing model; or, the first text processing device may train the text processing model, and may also obtain model parameters of the trained text processing model, and the second text processing device may obtain the model parameters from the first text processing device, construct the trained text processing model based on the model parameters, and further may execute the text processing scheme based on the trained text processing model. Optionally, after obtaining the trained text processing model and model parameters, the first text processing device may store the trained text processing model and model parameters to the blockchain, and the second text processing device may directly obtain the trained text processing model or model parameters from the blockchain; further, the second text processing device may process the target text based on the trained text processing model to predict key phrases existing in the target text and to predict key phrases not existing in the target text, so as to obtain a key phrase set corresponding to the target text; and then the second text processing device can store the target text and the key phrase set corresponding to the target text into the block chain, so that when the key phrase set corresponding to the target text needs to be obtained again through the second text processing device, the second text processing device can directly obtain the key phrase set from the block chain without calling the trained text processing model again to process the target text again.

Based on the above training scheme of the text processing model, the embodiment of the present application introduces the training flow of the text processing model in detail. Referring to fig. 2, a schematic diagram of a training process of a text processing model according to an embodiment of the present application is provided. The training process of the text processing model shown in fig. 2 may be performed by a first text processing device, wherein the first text processing device may be a computer. The training process of the text processing model shown in fig. 2 may include the following steps:

s201, obtaining a training sample.

The training samples are used for training a text processing model, the training samples comprise an initial training text, a first training phrase and a second training phrase, the first training phrase is a key phrase existing in the initial training text, the second training phrase is a key phrase not existing in the initial training text, and the first training phrase and the second training phrase are both used for describing text content or a central subject matter of the initial training text.

In one embodiment, the training samples may be user input, and the first text processing device may obtain the training samples input by the user; alternatively, the training samples may be public data sets related to key phrase prediction in the field of natural language processing, and the first text processing device may obtain corpus data from a public corpus to obtain the training samples. For example, the corpus to be published may be LDC corpus (legal Data Consortium), national language commission modern chinese corpus, and the like.

S202, the second training phrase is subjected to covering processing to obtain a covered second training phrase.

In one embodiment, the first text processing device may mask words in the second training phrase for the purpose of training the text processing model to learn to generate masked-out words. Further, the first text processing device may perform a random masking process on the words in the second training phrase to train the text processing model to learn to generate randomly masked words, and the training effect of the text processing model may be better because the learned words are randomly masked. In a specific implementation, the first text processing device may perform random masking processing on the words in the second training phrase by using a random Mask processing (Mask) method.

S203, generating a target training text consisting of the initial training text and the masked second training phrase.

In one embodiment, the first text processing device may concatenate the initial training text and the masked second training phrase to obtain the target training text. Wherein, the obtained target training text comprises predefined mark symbols [ CLS ] and [ SEP ], wherein [ CLS ] is placed at the head of the target training text, and [ SEP ] is used for separating the initial training text and the masked second training phrase. For example, the target training text may be a "[ CLS ] initial training text [ SEP ] obscured second training phrase [ SEP ]".

In an embodiment, the first text processing device may also splice the initial training text and the second training phrase to obtain an intermediate training text; and then, masking the second training phrase included in the middle training text to obtain a target training text, wherein the target training text at the moment includes the initial training text and the masked second training phrase.

And S204, coding the target training text through the text processing model to obtain a representation vector of the target training text.

Referring to the text processing Model shown in fig. 1, the pre-training Language Model may be a pre-training sequence-to-sequence (sequence-to-sequence) Language Model, such as a Unified pre-training Language Model (UNILM). UNILM can be used for joint training of Natural Language Understanding (NLU) and Natural Language Generation (NLG) and is based on a sequence-to-sequence Language model of an encoder (transducer encoder) of a self-attention-transforming network. Specifically, the first text processing device may encode the target training text through UNILM in the text processing model, and obtain a representation vector of the target training text.

S205, performing key phrase extraction processing on the representation vector of the target training text through a second stacking relation network to obtain a representation vector of a first key phrase corresponding to the target training text, and performing key phrase generation processing on the representation vector of the target training text through the second stacking relation network to obtain a representation vector of a second key phrase corresponding to the target training text.

The second stacking relation network is a network in the text processing model, the key phrase extraction processing is carried out on the representation vector of the target training text through the second stacking relation network so as to further obtain key phrases existing in the initial training text, and the key phrase generation processing is carried out on the representation vector of the target training text through the second stacking relation network so as to further obtain key phrases not existing in the initial training text; since there is a correlation between key phrases present in the text and key phrases not present in the text, a second stacked relationship network is proposed to enable interaction of the key phrase extraction process and the key phrase generation process to further learn the correlation between key phrases present in the initial training text and key phrases not present in the initial training text by the text processing model.

In one embodiment, the performing, by the first text processing device, a key phrase extraction process on the representation vector of the target training text through the second stacking relationship network to obtain a representation vector of a first key phrase corresponding to the target training text, and performing a key phrase generation process on the representation vector of the target training text through the second stacking relationship network to obtain a representation vector of a second key phrase corresponding to the target training text may include: performing first conversion processing on the expression vector of the target training text through a second stacking relation network to obtain a first intermediate vector, and performing second conversion processing on the expression vector of the target training text through the second stacking relation network to obtain a second intermediate vector; obtaining a third intermediate vector and a fourth intermediate vector according to the first intermediate vector and the second intermediate vector; obtaining a representation vector of a first key phrase corresponding to the target training text according to the first intermediate vector and the third intermediate vector; and obtaining a representation vector of a second key phrase corresponding to the target training text according to the second intermediate vector and the fourth intermediate vector.

In a specific implementation, the first text processing device performs first transformation processing on the representation vector of the target training text through a second stacking relationship network to obtain a first intermediate vector, and performs second transformation processing on the representation vector of the target training text through the second stacking relationship network to obtain a second intermediate vector, which may include: performing first linear transformation processing on the expression vector of the target training text through a second stacking relation network to obtain a first transformation vector; carrying out residual error and normalization processing on the expression vector of the target training text and the first transformation vector to obtain a first intermediate vector; performing second linear transformation processing on the expression vector of the target training text through a second stacking relation network to obtain a second transformation vector; and carrying out residual error and normalization processing on the representation vector of the target training text and the second transformation vector to obtain a second intermediate vector.

In one embodiment, the second stacking relationship network may be an MLP (multi layer Perceptron) based interaction network, i.e., an MLP interaction network. Optionally, the MLP network may be a single-layer MLP interactive network, or may also be a multilayer MLP interactive network, where the multilayer MLP interactive network is formed by stacking a plurality of single-layer MLP interactive networks; as shown in fig. 3, a schematic diagram of a single-layer MLP interactive network provided for the embodiment of the present application may include a Linear transform (Linear) layer, an Add & Norm (Add & Norm) layer, and a splicing (Concat) layer, where different Linear layers perform different Linear transform processes on an input of the Linear layer, and specifically, the Linear transform is implemented by setting weights and bias terms of the Linear transform, and how to set the weights and bias terms of the Linear transform should be based on specific requirements. With reference to the MLP interactive network shown in fig. 3, a first Linear transformation process may be performed on a representation vector of a target training text through a Linear layer shown as 301 to obtain a first transformation vector; then, carrying out residual error and normalization processing on the representation vector of the target training text and the first transformation vector through an Add & Norm layer shown as 303 to obtain a first intermediate vector; carrying out second Linear transformation processing on the expression vector of the target training text through a Linear layer as shown in 302 to obtain a second transformation vector; then, the representation vector of the target training text and the second transformation vector are subjected to residual error and normalization processing through an Add & Norm layer as shown in 304 to obtain a second intermediate vector.

Further, the obtaining, by the first text processing device, a third intermediate vector and a fourth intermediate vector according to the first intermediate vector and the second intermediate vector through the second stacking relationship network may include: performing first splicing processing on the first intermediate vector and the second intermediate vector to obtain a first spliced vector, and performing linear transformation processing on the first spliced vector to obtain a third intermediate vector; and performing second splicing processing on the first intermediate vector and the second intermediate vector to obtain a second spliced vector, and performing linear transformation processing on the second spliced vector to obtain a fourth intermediate vector. Performing first splicing processing on the first intermediate vector and the second intermediate vector, namely splicing the second intermediate vector behind the first intermediate vector, wherein the obtained first splicing vector is a vector obtained by splicing the second intermediate vector behind the first intermediate vector; and performing second splicing processing on the first intermediate vector and the second intermediate vector, namely splicing the first intermediate vector after the second intermediate vector, namely obtaining a second spliced vector which is obtained by splicing the first intermediate vector after the second intermediate vector.

With reference to the MLP interaction network shown in fig. 3, the first intermediate vector and the second intermediate vector may be subjected to a first splicing process through a Concat layer shown as 305 to obtain a first spliced vector; then, the first splicing vector is subjected to Linear transformation processing through a Linear layer as shown in 307, and a third intermediate vector is obtained; the first intermediate vector and the second intermediate vector may be subjected to a second splicing process through a Concat layer as shown by 306, so as to obtain a second spliced vector; the second stitched vector is then subjected to Linear transformation processing by a Linear layer as shown in 308, resulting in a fourth intermediate vector.

Further, obtaining a representation vector of the first key phrase corresponding to the target training text according to the first intermediate vector and the third intermediate vector may include: and carrying out residual error and normalization processing on the first intermediate vector and the third intermediate vector to obtain a representation vector of the first key phrase corresponding to the target training text. With reference to the MLP interaction network shown in fig. 3, residual error and normalization processing may be performed on the first intermediate vector and the third intermediate vector through Add & Norm layers shown in 309, so as to obtain a representation vector of the first key phrase corresponding to the target training text.

Obtaining a representation vector of a second key phrase corresponding to the target training text according to the second intermediate vector and the fourth intermediate vector, which may include: and carrying out residual error and normalization processing on the second intermediate vector and the fourth intermediate vector to obtain a representation vector of a second key phrase corresponding to the target training text. With reference to the MLP interaction network shown in fig. 3, residual error and normalization processing may be performed on the second intermediate vector and the fourth intermediate vector through Add & Norm layers shown as 310, so as to obtain a representation vector of the second key phrase corresponding to the target training text.

In one embodiment, the second stacking relationship network may be a Co-Attention (Co-Attention) based interaction network, i.e., a Co-Attention interaction network. Optionally, the system can be a single-layer Co-Attention interactive network or a multi-layer Co-Attention interactive network, wherein the multi-layer Co-Attention interactive network is formed by stacking a plurality of single-layer Co-Attention interactive networks; as shown in fig. 4, a schematic diagram of a single-layer Co-Attention interaction network provided for the embodiment of the present application may include a Linear transform (Linear) layer, an Add & Norm (Add & Norm) layer, and a normalization (Softmax) layer, where different Linear layers perform different Linear transform processes on an input of the Linear layer, and specifically, the Linear transform is implemented by setting weights and bias terms of the Linear transform, and how to set the weights and the bias terms of the Linear transform should be based on specific requirements. With reference to the Co-Attention interactive network shown in fig. 4, the expression vector of the target training text may be subjected to a first Linear transformation process through a Linear layer shown as 401, so as to obtain a first transformation vector; then, carrying out residual error and normalization processing on the representation vector of the target training text and the first transformation vector through an Add & Norm layer shown as 403 to obtain a first intermediate vector; carrying out second Linear transformation processing on the expression vector of the target training text through a Linear layer shown as 402 to obtain a second transformation vector; then, the representation vector of the target training text and the second transformation vector are subjected to residual error and normalization processing through an Add & Norm layer as shown in 404 to obtain a second intermediate vector.

Further, the obtaining, by the first text processing device, a third intermediate vector and a fourth intermediate vector according to the first intermediate vector and the second intermediate vector through the second stacking relationship network may include: performing a first arithmetic operation according to the first intermediate vector and the second intermediate vector to obtain a first interactive vector, and performing a second arithmetic operation according to the first interactive vector and the second intermediate vector to obtain a third intermediate vector; and performing third arithmetic operation according to the first intermediate vector and the second intermediate vector to obtain a second interactive vector, and performing fourth arithmetic operation according to the second interactive vector and the first intermediate vector to obtain a fourth intermediate vector.

In a specific implementation, the second intermediate vector may be transposed to obtain a transposed second intermediate vector, and the transposed second intermediate vector is multiplied by the first intermediate vector to obtain a first interactive vector; then, the first interactive vector is normalized to obtain a normalized first interactive vector, and then the normalized first interactive vector is multiplied by the second intermediate vector to obtain a third intermediate vector. Wherein the first interaction vector may be normalized by a Softmax layer as shown in 405. For example, assuming that the first intermediate vector is A and the second intermediate vector is B, the first interaction vector is

Then the third intermediate vector is

。

In a specific implementation, the first intermediate vector may be transposed to obtain a rotationMultiplying the first intermediate vector after the conversion by a second intermediate vector to obtain a second interactive vector; and then, normalizing the second interactive vector to obtain a normalized second interactive vector, and then multiplying the normalized second interactive vector by the first intermediate vector to obtain a fourth intermediate vector. Wherein the second interaction vector may be normalized by the Softmax layer as shown at 406. For example, assuming the first intermediate vector is A and the second intermediate vector is B, the second interaction vector is

Then the third intermediate vector is

。

Further, obtaining a representation vector of the first key phrase corresponding to the target training text according to the first intermediate vector and the third intermediate vector may include: and carrying out residual error and normalization processing on the first intermediate vector and the third intermediate vector to obtain a representation vector of the first key phrase corresponding to the target training text. With reference to the Co-Attention interaction network shown in fig. 4, the first intermediate vector and the third intermediate vector may be subjected to residual error and normalization processing through Add & Norm layer shown in 407, so as to obtain a representation vector of the first key phrase corresponding to the target training text.

Obtaining a representation vector of a second key phrase corresponding to the target training text according to the second intermediate vector and the fourth intermediate vector, which may include: and carrying out residual error and normalization processing on the second intermediate vector and the fourth intermediate vector to obtain a representation vector of a second key phrase corresponding to the target training text. With reference to the Co-Attention interaction network shown in fig. 4, residual error and normalization processing may be performed on the second intermediate vector and the fourth intermediate vector through Add & Norm layer shown in 408, so as to obtain a representation vector of the second key phrase corresponding to the target training text.

S206, decoding the expression vector of the first key phrase corresponding to the target training text to obtain the first key phrase corresponding to the target training text, and decoding the expression vector of the second key phrase corresponding to the target training text to obtain the second key phrase corresponding to the target training text.

The first key phrase corresponding to the target training text is a key phrase existing in the initial training text, the second key phrase corresponding to the target training text is a key phrase not existing in the initial training text, and the first key phrase corresponding to the target training text and the second key phrase corresponding to the target training text have correlation.

In specific implementation, the first text processing device may decode, through the text processing model, a representation vector of a first key phrase corresponding to the target training text to obtain an initial training text after labeling; extracting the marked initial training text to obtain a first key phrase corresponding to the target training text; the first text processing device can decode the representation vector of the second key phrase corresponding to the target training text through the text processing model to obtain the second key phrase corresponding to the target training text. The labeled initial training text may be labeled by using BIO in sequence labeling, specifically, the word labeled by B is used to indicate that the word is the beginning of the first key phrase corresponding to the target training text, the word labeled by I is used to indicate that the word is the middle of the first key phrase corresponding to the target training text, and the word labeled by O is used to indicate that the word does not belong to the first key phrase corresponding to the target training text. Further, the first text processing device extracts and processes the words labeled with B, I in the labeled initial training text through the text processing model, so as to obtain the first key phrase corresponding to the target training text. For example, if the initial training text is "natural language processing technology for developing a language learning environment", and the labeled initial training text is "for _ O, development _ O, language learning _ O, environment _ O, natural language _ B, processing _ I, and technology _ O", the first key phrase corresponding to the target training text is "natural language processing".

In one embodiment, the first text processing device may decode, through the text processing model, a representation vector of a first key phrase corresponding to the target training text, and decode a representation vector of a second key phrase corresponding to the target training text, to obtain a representation form obtained by splicing the labeled initial training text and the second key phrase corresponding to the target training text. Further, the first text processing device may extract, through the text processing model, a representation form obtained by splicing the labeled initial training text and the second key phrase corresponding to the target training text, to obtain a first key phrase corresponding to the target training text and a second key phrase corresponding to the target training text. The expression form after splicing the marked initial training text and the second key phrase corresponding to the target training text comprises predefined mark symbols which are [ CLS ] and [ SEP ], wherein [ CLS ] is arranged at the head of the marked initial training text, and [ SEP ] is used for separating the marked initial training text and the second key phrase corresponding to the target training text. For example, the expression form after the marked initial training text is spliced with the second key phrase corresponding to the target training text can be a [ CLS ] marked initial training text [ SEP ] and a second key phrase [ SEP ] corresponding to the target training text.

And S207, training the text processing model based on the first key phrase corresponding to the target training text, the second key phrase corresponding to the target training text, the first training phrase and the second training phrase to obtain the trained text processing model.

The training of the text processing model is based on a first key phrase corresponding to the target training text, a second key phrase corresponding to the target training text, the first training phrase and the second training phrase, so that the first key phrase corresponding to the target training text obtained through the text processing model is infinitely close to the first training phrase, and the second key phrase corresponding to the target training text is infinitely close to the second training phrase. The first text processing device extracts the labeled initial training text obtained by decoding through the text processing model to obtain a first key phrase corresponding to the target training text, so that the training of the text processing model can also be said to be for training the label of the initial training text, so that the first key phrase corresponding to the target training text extracted according to the labeled initial training text is infinitely close to the first training phrase; and the first text processing device directly processes the target training text spliced by the initial training text and the masked second training phrase through the text processing model to obtain the second key phrase corresponding to the target training text, so that the training of the text processing model can also be said to generate the masked word for training the text processing model, so that the second key phrase corresponding to the target training text is infinitely close to the second training phrase.

In one embodiment, the first text processing device may train the text processing model based on a bag-of-words error generated by the bag-of-words model, and in a specific implementation, the first text processing device may construct a first short-term bag of words based on a first key phrase and a first training phrase corresponding to the target training text; constructing a second phrase bag based on a second key phrase and a second training phrase corresponding to the target training text; obtaining a first word bag error between a first key phrase and a first training phrase corresponding to the target training text based on the first short word bag; obtaining a second word bag error between a second key phrase and a second training phrase corresponding to the target training text based on the second short word bag; generating a word bag total error according to the first word bag error and the second word bag error; and training the text processing model based on the total error of the word bag to obtain the trained text processing model.

Wherein, the words in the first phrase word bag are composed of the words in the first key phrase corresponding to the target training text and the words in the first training phrase; the words in the second phrase bag of words are composed of the words in the second key phrase corresponding to the target training text and the words in the second training phrase, and the bag of words model does not pay attention to the appearance order of each word and the elements such as the syntax of the lexical method. For example, if the first key phrase a1 corresponding to the target training text consists of word 1, word 2, word 3, word 4, word 5, word 6, and word 7, and each word appears only 1 time in the first key phrase corresponding to the target training text; the second key phrase B1 corresponding to the target training text consists of a word 8, a word 9, a word 10, a word 11 and a word 12, and each word only appears for 1 time in the second key phrase corresponding to the target training text; if the first training phrase a2 consists of word 1, word 2, word 3, word 4, word 5, word 13, and word 14, and each word appears only 1 time in the first training phrase; the second training phrase B2 is composed of word 8, word 9, word 10, word 11, and word 15, and each word appears only 1 time in the second training phrase; the first phrase bag includes word 1, word 2, word 3, word 4, word 5, word 6, word 7, word 13, and word 14; the second phrase bag includes word 8, word 9, word 10, word 11, word 12, and word 15.

Further, the obtaining, by the first text processing device, a first bag-of-words error between a first key phrase and a first training phrase corresponding to the target training text based on the first bag of short-term words may include: converting a first key phrase corresponding to a target training text into an N-dimensional vector, converting the first training phrase into an N-dimensional vector, and determining a first word bag error according to the N-dimensional vector corresponding to the first key phrase corresponding to the target training text and the N-dimensional vector corresponding to the first training phrase, wherein N is the number of words included in a first phrase word bag. Assuming that the first key phrase corresponding to the target training text is a1 and the first training phrase is a2, assuming that the mapping relationship between the vectors and the words is { word 1, word 2, word 3, word 4, word 5, word 6, word 7, word 13, word 14}, the N-dimensional vector corresponding to the first key phrase a1 corresponding to the target training text is {1,1,1,1,1,1, 0,0}, and the N-dimensional vector corresponding to the first training phrase a2 is {1,1,1,1,1,1, 0,0,1,1 }. Optionally, the first bag-of-words error may be determined by calculating cosine similarity between an N-dimensional vector corresponding to the first key phrase corresponding to the target training text and an N-dimensional vector corresponding to the first training phrase. The cosine similarity can be represented by calculating a cosine value of an included angle between an N-dimensional vector corresponding to a first key phrase corresponding to the target training text and the N-dimensional vector corresponding to the first training phrase, wherein the closer the cosine value is to 1, the more similar the N-dimensional vector corresponding to the first key phrase corresponding to the target training text is to the N-dimensional vector corresponding to the first training phrase. Further, the first bag-of-words error may be represented by using an inverse number of the cosine similarity, or by subtracting the cosine similarity from a fixed numerical value, for example, when the fixed numerical value is 1, the first bag-of-words error may be 1-cosine similarity, further, the first bag-of-words error may also be represented by using other mathematical forms, which may be set according to specific requirements and scenarios, and the embodiment of the present application is not limited.

Obtaining a second bag-of-words error between a second key phrase and a second training phrase corresponding to the target training text based on the second phrase bag may include: and converting a second key phrase corresponding to the target training text into an M-dimensional vector, converting the second training phrase into an M-dimensional vector, and determining a second word bag error according to the M-dimensional vector corresponding to the second key phrase corresponding to the target training text and the M-dimensional vector corresponding to the second training phrase, wherein M is the number of words included in a second phrase word bag. Assuming that the second key phrase corresponding to the target training text is B1 and the second training phrase is B2, assuming that the mapping relationship between the vectors and the words is { word 8, word 9, word 10, word 11, word 12, word 15}, the M-dimensional vector corresponding to the second key phrase B1 corresponding to the target training text is {1,1,1,1,1,0}, and the M-dimensional vector corresponding to the second training phrase B2 is {1,1,1,1,0,1 }. Optionally, the second bag-of-words error may be determined by calculating cosine similarity between an M-dimensional vector corresponding to a second key phrase corresponding to the target training text and an M-dimensional vector corresponding to the second training phrase. The cosine similarity can be represented by calculating a cosine value of an included angle between an M-dimensional vector corresponding to a second key phrase corresponding to the target training text and the M-dimensional vector corresponding to the second training phrase, wherein the closer the cosine value is to 1, the more similar the M-dimensional vector corresponding to the second key phrase corresponding to the target training text is to the M-dimensional vector corresponding to the second training phrase. Further, the second bag-of-words error may be represented by using an inverse number of the cosine similarity, or by subtracting the cosine similarity from a fixed numerical value, for example, when the fixed numerical value is 1, the second bag-of-words error may be 1-cosine similarity, further, other mathematical forms may also be used to represent the second bag-of-words error, which may be set according to specific requirements and scenarios, and the embodiment of the present application is not limited.

Further, a word bag total error can be generated according to the first word bag error and the second word bag error, and the text processing model is trained based on the word bag total error to obtain a trained text processing model. The total bag-of-words error may be a sum of the first bag-of-words error and the second bag-of-words error, or other arithmetic operations that can jointly affect the total bag-of-words error by the first bag-of-words error and the second bag-of-words error, such as a weighted sum of the first bag-of-words error and the second bag-of-words error, a product of the first bag-of-words error and the second bag-of-words error, and the like. The first text processing device can train the text processing model based on the total bag-of-words errors generated by different training samples, so that the total bag-of-words errors generated by the training samples for testing the text processing model are smaller than a preset error threshold value, and the trained text processing model is obtained. The preset error threshold may be preset according to a requirement, and may represent whether the training of the text processing model can be finished. Specifically, the text processing device may iteratively update the model parameters of the text processing model based on the total bag-of-words errors generated by different training samples, so that the total bag-of-words errors generated by the training samples for the text processing model test based on the updated model parameters are smaller than a preset error threshold, and the trained text processing model is obtained based on the updated model parameters.

In one embodiment, the trained text processing model is compared with a model adopted in an existing method for predicting key phrases according to texts based on the model evaluation index F1 value, and the trained text processing model is found to have a higher F1 value and higher prediction accuracy, wherein the F1 value is an index for judging whether the model is good or bad and can be calculated by the accuracy rate recall 2/(accuracy rate + recall rate). The existing method for predicting key phrases according to texts comprises a key phrase extraction method and a key phrase generation method, wherein the key phrase extraction method adopts an extraction mode to extract key phrases existing in texts from the texts, and the key phrase generation method adopts a generation mode to generate key phrases existing in the texts and generate key phrases not existing in the texts.

As can be seen from the comparison results shown in table 1, the model used in the key phrase extraction method has an F1 value of 0.419 for key phrases present in the text and an F1 value of 0 for key phrases not present in the text; the model adopted by the key phrase generation method has the F1 value of 0.381 for the key phrases existing in the text and the F1 value of 0.020 for the key phrases not existing in the text; further, a method of extracting and generating a combination may be employed that employs a model having an F1 value of 0.437 for key phrases present in the text and an F1 value of 0.022 for key phrases not present in the text.

TABLE 1

Model (model)	F1 value for key phrases present in text	F1 value for key phrases not present in text
			Model adopted by key phrase extraction method	0.419	0.0
Model adopted by key phrase generation method	0.381	0.020
			Method for extracting and generating a combinationModel (III)	0.437	0.022

As shown in the comparison results shown in Table 2, the trained text processing model using the single-layer MLP interactive network has an F1 value of 0.442 for key phrases existing in the text and an F1 value of 0.026 for key phrases not existing in the text; the trained text processing model adopting the multilayer MLP interactive network has an F1 value of 0.439 for key phrases existing in the text and an F1 value of 0.034 for key phrases not existing in the text; the trained text processing model using the single-layer Co-Attention interaction network has a value of 0.448 for F1 for key phrases present in the text, a value of 0.025 for F1 for key phrases not present in the text, a value of 0.452 for F1 for key phrases present in the text, and a value of 0.027 for F1 for key phrases not present in the text.

It can be further seen that, no matter a single-layer MLP interactive network or a multi-layer MLP interactive network or a single-layer Co-Attention interactive network or a multi-layer Co-Attention interactive network is adopted, the F1 values of key phrases existing in the text and the F1 values of key phrases not existing in the text are improved; for the F1 value of the key phrase in the text, the effect is better by adopting a single-layer MLP interactive network than a multi-layer MLP interactive network, and the effect is better by adopting a multi-layer Co-Attention interactive network than a single-layer Co-Attention interactive network; for the F1 value of the key phrase not existing in the text, the effect of adopting the multilayer MLP interactive network is better than that of adopting the single-layer MLP interactive network, and the effect of adopting the multilayer Co-Attention interactive network is better than that of adopting the single-layer Co-Attention interactive network, so that different interactive networks can be adopted according to different application scenes and different requirements on prediction precision.

TABLE 2

Model (model)	F1 value for key phrases present in text	F1 value for key phrases not present in text
			Model for use in a method of extracting and generating a combination	0.437	0.022
Trained text processing model adopting single-layer MLP interactive network	0.442	0.026
			Trained text processing model adopting multilayer MLP interactive network	0.439	0.034
Trained text processing model adopting single-layer Co-Attention interactive network	0.448	0.025
			Trained text processing model adopting multilayer Co-Attention interactive network	0.452	0.027

Further, since the text processing model may be trained using the bag-of-words total error to obtain the trained text processing model, and the model in the extraction and generation combination method obtained by training based on the bag-of-words total error is calculated, it can be known that training the model using the bag-of-words total error can also improve the F1 value, specifically, as shown in table 3, the F1 value for the key phrases existing in the text is 0.442, and the F1 value for the key phrases not existing in the text is 0.024.

TABLE 3

Model (model)	F1 value for key phrases present in text	F1 value for key phrases not present in text
			Model for use in a method of extracting and generating a combination	0.437	0.022
Model in extraction and generation combined method based on bag of words total error training	0.442	0.024

In the embodiment of the application, after acquiring a training sample comprising an initial training text, a first training phrase and a second training phrase, a first text processing device further directly processes a target training text spliced by the initial training text and the masked second training phrase through a text processing model, and further obtains a first key phrase corresponding to the target training text and a second key phrase corresponding to the target training text by adopting a mode of extraction and generation combination through a second stacking relation network in the text processing model; then, the first text processing equipment trains the text processing model based on a first key phrase corresponding to the target training text, a second key phrase corresponding to the target training text, the first training phrase and the second training phrase to obtain a trained text processing model; wherein the first training phrase is a key phrase present in the initial training text and the second training phrase is a key phrase not present in the initial training text. The trained text processing model can predict the key phrases of the text in a mode of extraction and generation combination, and the prediction precision of predicting the key phrases according to the text can be improved.

Based on the description of the training process of the text processing model, an embodiment of the present application provides a trained text processing model, and specifically, as shown in fig. 5, the trained text processing model may include a trained pre-training language model and a first stacking relationship network. Wherein, the trained pre-training language model can be a trained uniform pre-training language model, namely UNILM; the first stacking relation network is a trained second stacking relation network obtained by training a second stacking relation network in the text processing model. In combination with the trained text processing model shown in fig. 5, the target text may be encoded by the trained pre-trained language model in the trained text processing model to obtain a representation vector of the target text; performing key phrase extraction processing on the representation vector of the target text through a first stacking relation network to obtain a representation vector of a first key phrase corresponding to the target text, and performing key phrase generation processing on the representation vector of the target text through the first stacking relation network to obtain a representation vector of a second key phrase corresponding to the target text; decoding the expression vector of the first key phrase corresponding to the target text to obtain a first key phrase corresponding to the target text, and decoding the expression vector of the second key phrase corresponding to the target text to obtain a second key phrase corresponding to the target text; and finally, generating a key phrase set corresponding to the target text, wherein the key phrase set comprises a first key phrase corresponding to the target text and a second key phrase corresponding to the target text.

Based on the above description, an embodiment of the present application provides a text processing method, and refer to fig. 6, which is a flowchart illustrating the text processing method provided in the embodiment of the present application. The text processing method shown in fig. 6 may be performed by a second text processing apparatus, wherein the second text processing apparatus may be a computer. The text processing method shown in fig. 6 may include the steps of:

s601, coding the target text through the trained text processing model to obtain the expression vector of the target text.

S602, performing key phrase extraction processing on the representation vector of the target text through the first stacking relation network to obtain a representation vector of a first key phrase corresponding to the target text, and performing key phrase generation processing on the representation vector of the target text through the first stacking relation network to obtain a representation vector of a second key phrase corresponding to the target text.

S603, decoding the expression vector of the first key phrase corresponding to the target text to obtain the first key phrase corresponding to the target text, and decoding the expression vector of the second key phrase corresponding to the target text to obtain the second key phrase corresponding to the target text.

In steps S601 to S603, the target text is a text to be processed, and may be, for example, a novel text, a news text, a prose text, or the like. The target text can be input by the user, for example, the user wants to obtain a key phrase of a novel text, and the novel text can be input into the second text processing device; after the novel text is acquired, the second text processing device can process the novel text, so that a key phrase corresponding to the novel text is acquired. The first key phrase corresponding to the target text is a key phrase existing in the target text, the second key phrase corresponding to the target text is a key phrase not existing in the target text, and the first key phrase corresponding to the target text and the second key phrase corresponding to the target text have correlation.

The second text processing equipment encodes the target text through the trained text processing model to obtain an expression vector of the target text; and then performing key phrase extraction processing on the representation vector of the target text through the first stacking relation network, and performing key phrase generation processing on the representation vector of the target text through the first stacking relation network, wherein interaction of the key phrase extraction processing and the key phrase generation processing can be realized through the first stacking relation network, so that a first key phrase corresponding to the target text obtained through the trained text processing model has correlation with a second key phrase corresponding to the target text.

Since the first key phrase corresponding to the target text and the second key phrase corresponding to the target text obtained by processing the target text through the trained text processing model in steps S601 to S603 are similar to the first key phrase corresponding to the target training text and the second key phrase corresponding to the target training text obtained by processing the target training text through the text processing model in steps S204 to S206, which are mentioned above, no further description is given here.

S604, generating a key phrase set corresponding to the target text, wherein the key phrase set comprises a first key phrase corresponding to the target text and a second key phrase corresponding to the target text.

In one embodiment, the second text processing device may integrate the first key phrase corresponding to the target text and the second key phrase corresponding to the target text, generate a key phrase set corresponding to the target text, and then output the key phrase set; or directly outputting the first key phrase corresponding to the target text and the second key phrase corresponding to the target text.

In the embodiment of the application, the second text processing device encodes the target text through the trained text processing model to obtain a representation vector of the target text; performing key phrase extraction processing on the representation vector of the target text through a first stacking relation network to obtain a representation vector of a first key phrase corresponding to the target text, and performing key phrase generation processing on the representation vector of the target text through the first stacking relation network to obtain a representation vector of a second key phrase corresponding to the target text, wherein the first stacking relation network is a network in a trained text processing model, the first key phrase corresponding to the target text is a key phrase existing in the target text, and the second key phrase corresponding to the target text is a key phrase not existing in the target text; decoding the expression vector of the first key phrase corresponding to the target text to obtain a first key phrase corresponding to the target text, and decoding the expression vector of the second key phrase corresponding to the target text to obtain a second key phrase corresponding to the target text; and finally, generating a key phrase set corresponding to the target text, wherein the key phrase set comprises a first key phrase corresponding to the target text and a second key phrase corresponding to the target text. The second text processing equipment performs key phrase extraction processing on the representation vector of the target text through the first stacking relation network so as to predict key phrases existing in the target text, performs key phrase generation processing on the representation vector of the target text through the first stacking relation network so as to predict key phrases not existing in the target text, and can predict key phrases of the target text in a mode of combining extraction and generation so as to improve the prediction precision of predicting key phrases according to the text.

Based on the above method embodiments, the present application provides a text processing apparatus. Referring to fig. 7, a schematic structural diagram of a text processing apparatus according to an embodiment of the present disclosure is shown, where the text processing apparatus may include an encoding unit 701, a vector processing unit 702, a decoding unit 703, and a generating unit 704. The text processing apparatus shown in fig. 7 may operate as follows:

the encoding unit 701 is configured to perform encoding processing on a target text through a trained text processing model to obtain a representation vector of the target text;

a vector processing unit 702, configured to perform a key phrase extraction process on the representation vector of the target text through a first stacking relationship network to obtain a representation vector of a first key phrase corresponding to the target text, and perform a key phrase generation process on the representation vector of the target text through the first stacking relationship network to obtain a representation vector of a second key phrase corresponding to the target text, where the first stacking relationship network is a network in the trained text processing model, the first key phrase corresponding to the target text is a key phrase existing in the target text, and the second key phrase corresponding to the target text is a key phrase not existing in the target text;

a decoding unit 703, configured to decode a representation vector of a first key phrase corresponding to the target text to obtain a first key phrase corresponding to the target text, and decode a representation vector of a second key phrase corresponding to the target text to obtain a second key phrase corresponding to the target text;

a generating unit 704, configured to generate a set of key phrases corresponding to the target text, where the set of key phrases includes a first key phrase corresponding to the target text and a second key phrase corresponding to the target text.

In one embodiment, the text processing apparatus may further include an acquisition unit 705, a masking unit 706, and a training unit 707; before the coding unit 701 codes a target text through a trained text processing model to obtain a representation vector of the target text;

the obtaining unit 705 is configured to obtain a training sample, where the training sample includes an initial training text, a first training phrase, and a second training phrase, the first training phrase is a key phrase existing in the initial training text, and the second training phrase is a key phrase not existing in the initial training text;

the masking unit 706 is configured to perform masking processing on the second training phrase to obtain a masked second training phrase;

the generating unit 704 is further configured to generate a target training text composed of the initial training text and the masked second training phrase;

the encoding unit 701 is further configured to encode the target training text through a text processing model to obtain a representation vector of the target training text;

the vector processing unit 702 is further configured to perform, by using a second stacking relationship network, a key phrase extraction process on the representation vector of the target training text to obtain a representation vector of a first key phrase corresponding to the target training text, and perform, by using the second stacking relationship network, a key phrase generation process on the representation vector of the target training text to obtain a representation vector of a second key phrase corresponding to the target training text, where the second stacking relationship network is a network in the text processing model;

the decoding unit 703 is further configured to decode a representation vector of a first key phrase corresponding to the target training text to obtain a first key phrase corresponding to the target training text, and decode a representation vector of a second key phrase corresponding to the target training text to obtain a second key phrase corresponding to the target training text;

the training unit 707 is configured to train the text processing model based on a first key phrase corresponding to the target training text, a second key phrase corresponding to the target training text, the first training phrase, and the second training phrase, so as to obtain the trained text processing model.

In an embodiment, the vector processing unit 702 performs a key phrase extraction process on the representation vector of the target training text through a second stacking relationship network to obtain a representation vector of a first key phrase corresponding to the target training text, and performs a key phrase generation process on the representation vector of the target training text through the second stacking relationship network to obtain a representation vector of a second key phrase corresponding to the target training text, and specifically performs the following operations:

performing first conversion processing on the expression vector of the target training text through the second stacking relation network to obtain a first intermediate vector, and performing second conversion processing on the expression vector of the target training text through the second stacking relation network to obtain a second intermediate vector;

obtaining a third intermediate vector and a fourth intermediate vector according to the first intermediate vector and the second intermediate vector;

obtaining a representation vector of a first key phrase corresponding to the target training text according to the first intermediate vector and the third intermediate vector;

and obtaining a representation vector of a second key phrase corresponding to the target training text according to the second intermediate vector and the fourth intermediate vector.

In an embodiment, when the vector processing unit 702 obtains a third intermediate vector and a fourth intermediate vector according to the first intermediate vector and the second intermediate vector, the following operations are specifically performed:

performing first splicing processing on the first intermediate vector and the second intermediate vector to obtain a first spliced vector, and performing linear transformation processing on the first spliced vector to obtain a third intermediate vector;

and performing second splicing processing on the first intermediate vector and the second intermediate vector to obtain a second spliced vector, and performing linear transformation processing on the second spliced vector to obtain a fourth intermediate vector.

performing a first arithmetic operation according to the first intermediate vector and the second intermediate vector to obtain a first interactive vector, and performing a second arithmetic operation according to the first interactive vector and the second intermediate vector to obtain a third intermediate vector;

and performing third arithmetic operation according to the first intermediate vector and the second intermediate vector to obtain a second interactive vector, and performing fourth arithmetic operation according to the second interactive vector and the first intermediate vector to obtain a fourth intermediate vector.

In an embodiment, when the decoding unit 703 performs decoding processing on the representation vector of the first key phrase corresponding to the target training text to obtain the first key phrase corresponding to the target training text, the following operation is specifically performed:

decoding the expression vector of the first key phrase corresponding to the target training text to obtain an initial training text after marking;

and extracting the marked initial training text to obtain a first key phrase corresponding to the target training text.

In an embodiment, the training unit 707 trains the text processing model based on a first key phrase corresponding to the target training text, a second key phrase corresponding to the target training text, the first training phrase, and the second training phrase, and specifically performs the following operations when obtaining the trained text processing model:

constructing a first phrase bag based on a first key phrase corresponding to the target training text and the first training phrase;

constructing a second phrase bag based on a second key phrase corresponding to the target training text and the second training phrase;

obtaining a first bag-of-words error between a first key phrase corresponding to the target training text and the first training phrase based on the first phrase bag;

obtaining a second word bag error between a second key phrase corresponding to the target training text and the second training phrase based on the second short word bag;

generating a word bag total error according to the first word bag error and the second word bag error;

and training the text processing model based on the total error of the word bag to obtain the trained text processing model.

According to an embodiment of the present application, the steps involved in the text processing methods shown in fig. 2 and fig. 6 may be performed by units in the text processing apparatus shown in fig. 7. For example, step S201 shown in fig. 2 may be executed by the acquisition unit 705 in the text processing apparatus shown in fig. 7, step S202 shown in fig. 2 may be executed by the masking unit 706 in the text processing apparatus shown in fig. 7, and step S203 shown in fig. 2 may be executed by the generation unit 704 in the text processing apparatus shown in fig. 7; step S204 shown in fig. 2 may be performed by the encoding unit 701 in the text processing apparatus shown in fig. 7, step S205 shown in fig. 2 may be performed by the vector processing unit 702 in the text processing apparatus shown in fig. 7, step S206 shown in fig. 2 may be performed by the decoding unit 703 in the text processing apparatus shown in fig. 7, and step S207 shown in fig. 2 may be performed by the training unit 707 in the text processing apparatus shown in fig. 7. As another example, step S601 shown in fig. 6 may be performed by the encoding unit 701 in the text processing apparatus shown in fig. 7, step S602 shown in fig. 6 may be performed by the vector processing unit 702 in the text processing apparatus shown in fig. 7, step S603 shown in fig. 6 may be performed by the decoding unit 703 in the text processing apparatus shown in fig. 7, and step S604 shown in fig. 6 may be performed by the generating unit 704 in the text processing apparatus shown in fig. 7.

According to another embodiment of the present application, the units in the text processing apparatus shown in fig. 7 may be respectively or entirely combined into one or several other units to form one or several other units, or some unit(s) may be further split into multiple units with smaller functions to form one or several other units, which may achieve the same operation without affecting the achievement of the technical effect of the embodiment of the present application. The units are divided based on logic functions, and in practical application, the functions of one unit can be realized by a plurality of units, or the functions of a plurality of units can be realized by one unit. In other embodiments of the present application, the text processing apparatus based on logical function division may also include other units, and in practical applications, these functions may also be implemented by assistance of other units, and may be implemented by cooperation of a plurality of units.

According to another embodiment of the present application, the text processing apparatus shown in fig. 7 may be configured by running a computer program (including program codes) capable of executing the steps involved in the respective methods shown in fig. 2 and fig. 6 on a general-purpose computing device such as a computer including a processing element such as a Central Processing Unit (CPU), a random access storage medium (RAM), a read-only storage medium (ROM), and a storage element, and implementing the text processing method according to the embodiment of the present application. The computer program may be embodied on a computer-readable storage medium, for example, and loaded into and executed by the above-described computing apparatus via the computer-readable storage medium.

In the embodiment of the application, the coding unit 701 codes a target text through a trained text processing model to obtain a representation vector of the target text; the vector processing unit 702 performs a key phrase extraction process on the representation vector of the target text through a first stacking relationship network to obtain a representation vector of a first key phrase corresponding to the target text, and performs a key phrase generation process on the representation vector of the target text through the first stacking relationship network to obtain a representation vector of a second key phrase corresponding to the target text, where the first stacking relationship network is a network in a trained text processing model, the first key phrase corresponding to the target text is a key phrase existing in the target text, and the second key phrase corresponding to the target text is a key phrase not existing in the target text; the decoding unit 703 decodes the representation vector of the first key phrase corresponding to the target text to obtain a first key phrase corresponding to the target text, and decodes the representation vector of the second key phrase corresponding to the target text to obtain a second key phrase corresponding to the target text; the generating unit 704 generates a set of key phrases corresponding to the target text, where the set of key phrases includes a first key phrase corresponding to the target text and a second key phrase corresponding to the target text. The method comprises the steps of performing key phrase extraction processing on a representation vector of a target text through a first stacking relation network to predict key phrases existing in the target text, performing key phrase generation processing on the representation vector of the target text through the first stacking relation network to predict key phrases not existing in the target text, predicting the key phrases of the target text in a mode of combining extraction and generation, and improving prediction accuracy of predicting the key phrases according to the text.

Based on the method embodiment and the device embodiment, the application also provides a text processing device, which may include the first text processing device and the second text processing device. Referring to fig. 8, a schematic structural diagram of a text processing apparatus according to an embodiment of the present application is provided. The text processing device shown in fig. 8 may include at least a processor 801, an input interface 802, an output interface 803, and a computer storage medium 804. The processor 801, the input interface 802, the output interface 803, and the computer storage medium 804 may be connected by a bus or other means.

A computer storage medium 804 may be stored in the memory of the text processing device, the computer storage medium 804 being for storing a computer program comprising program instructions, the processor 801 being for executing the program instructions stored by the computer storage medium 804. The processor 801 (or CPU) is a computing core and a control core of the text Processing apparatus, and is adapted to implement one or more instructions, and specifically, adapted to load and execute the one or more instructions so as to implement the text Processing method flow or the corresponding functions.

An embodiment of the present application further provides a computer storage medium (Memory), which is a Memory device in a text processing device and is used to store programs and data. It is understood that the computer storage medium herein may include a built-in storage medium in the terminal, and may also include an extended storage medium supported by the terminal. The computer storage medium provides a storage space that stores an operating system of the terminal. Also stored in this memory space are one or more instructions, which may be one or more computer programs (including program code), suitable for loading and execution by processor 801. The computer storage medium may be a Random Access Memory (RAM) memory, or a non-volatile memory (non-volatile memory), such as at least one disk memory; and optionally at least one computer storage medium located remotely from the processor.

In one embodiment, one or more instructions stored in a computer storage medium may be loaded and executed by the processor 801 to implement the corresponding steps of the method in the text processing method embodiment described above with respect to fig. 2 and 6, and in particular, one or more instructions stored in a computer storage medium may be loaded and executed by the processor 801 to implement the following steps:

In one embodiment, before the processor 801 performs encoding processing on the target text through the trained text processing model to obtain the representation vector of the target text, the processor 801 is further configured to:

acquiring a training sample, wherein the training sample comprises an initial training text, a first training phrase and a second training phrase, the first training phrase is a key phrase existing in the initial training text, and the second training phrase is a key phrase not existing in the initial training text;

masking the second training phrase to obtain a masked second training phrase;

generating a target training text comprised of the initial training text and the obscured second training phrase;

coding the target training text through a text processing model to obtain a representation vector of the target training text;

performing key phrase extraction processing on the representation vector of the target training text through a second stacking relation network to obtain a representation vector of a first key phrase corresponding to the target training text, and performing key phrase generation processing on the representation vector of the target training text through the second stacking relation network to obtain a representation vector of a second key phrase corresponding to the target training text, wherein the second stacking relation network is a network in the text processing model;

decoding the representation vector of the first key phrase corresponding to the target training text to obtain a first key phrase corresponding to the target training text, and decoding the representation vector of the second key phrase corresponding to the target training text to obtain a second key phrase corresponding to the target training text;

and training the text processing model based on a first key phrase corresponding to the target training text, a second key phrase corresponding to the target training text, the first training phrase and the second training phrase to obtain the trained text processing model.

In an embodiment, when the processor 801 performs a key phrase extraction process on the representation vector of the target training text through a second stacking relationship network to obtain a representation vector of a first key phrase corresponding to the target training text, and performs a key phrase generation process on the representation vector of the target training text through the second stacking relationship network to obtain a representation vector of a second key phrase corresponding to the target training text, the following operations are specifically performed:

In an embodiment, when the processor 801 obtains a third intermediate vector and a fourth intermediate vector according to the first intermediate vector and the second intermediate vector, the following operations are specifically performed:

In an embodiment, when the processor 801 decodes the representation vector of the first key phrase corresponding to the target training text to obtain the first key phrase corresponding to the target training text, the following operation is specifically performed:

In an embodiment, the processor 801 trains the text processing model based on the first key phrase corresponding to the target training text, the second key phrase corresponding to the target training text, the first training phrase, and the second training phrase, and specifically executes the following operations when obtaining the trained text processing model:

Embodiments of the present application provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the text processing device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to cause the text processing device to perform the method embodiments described above as shown in fig. 2 or fig. 6. The computer-readable storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method of text processing, comprising:

masking the second training phrase to obtain a masked second training phrase;

performing first conversion processing on the representation vector of the target training text through a second stacking relation network to obtain a first intermediate vector, and performing second conversion processing on the representation vector of the target training text through the second stacking relation network to obtain a second intermediate vector, wherein the second stacking relation network is a network in the text processing model;

obtaining a representation vector of a second key phrase corresponding to the target training text according to the second intermediate vector and the fourth intermediate vector;

training the text processing model based on a first key phrase corresponding to the target training text, a second key phrase corresponding to the target training text, the first training phrase and the second training phrase to obtain a trained text processing model;

coding a target text through the trained text processing model to obtain a representation vector of the target text;

2. The method of claim 1, wherein said deriving a third intermediate vector and a fourth intermediate vector from the first intermediate vector and the second intermediate vector comprises:

3. The method of claim 1, wherein said deriving a third intermediate vector and a fourth intermediate vector from the first intermediate vector and the second intermediate vector comprises:

4. The method of claim 1, wherein the decoding the representation vector of the first key phrase corresponding to the target training text to obtain the first key phrase corresponding to the target training text comprises:

5. The method of claim 1, wherein the training the text processing model based on a first key phrase corresponding to the target training text, a second key phrase corresponding to the target training text, the first training phrase, and the second training phrase to obtain the trained text processing model comprises:

6. A text processing apparatus, comprising:

the training device comprises an acquisition unit, a comparison unit and a processing unit, wherein the acquisition unit is used for acquiring a training sample, the training sample comprises an initial training text, a first training phrase and a second training phrase, the first training phrase is a key phrase existing in the initial training text, and the second training phrase is a key phrase not existing in the initial training text;

the masking unit is used for masking the second training phrase to obtain a masked second training phrase;

a generating unit, configured to generate a target training text composed of the initial training text and the masked second training phrase;

the coding unit is used for coding the target training text through a text processing model to obtain a representation vector of the target training text;

the vector processing unit is used for performing first conversion processing on the expression vector of the target training text through a second stacking relation network to obtain a first intermediate vector, and performing second conversion processing on the expression vector of the target training text through the second stacking relation network to obtain a second intermediate vector, wherein the second stacking relation network is a network in the text processing model; obtaining a third intermediate vector and a fourth intermediate vector according to the first intermediate vector and the second intermediate vector; obtaining a representation vector of a first key phrase corresponding to the target training text according to the first intermediate vector and the third intermediate vector; obtaining a representation vector of a second key phrase corresponding to the target training text according to the second intermediate vector and the fourth intermediate vector;

the decoding unit is used for decoding the representation vector of the first key phrase corresponding to the target training text to obtain the first key phrase corresponding to the target training text, and decoding the representation vector of the second key phrase corresponding to the target training text to obtain the second key phrase corresponding to the target training text;

a training unit, configured to train the text processing model based on a first key phrase corresponding to the target training text, a second key phrase corresponding to the target training text, the first training phrase, and the second training phrase, so as to obtain a trained text processing model;

the encoding unit is further configured to encode a target text through the trained text processing model to obtain a representation vector of the target text;

the vector processing unit is further configured to perform key phrase extraction processing on the representation vector of the target text through a first stacking relationship network to obtain a representation vector of a first key phrase corresponding to the target text, and perform key phrase generation processing on the representation vector of the target text through the first stacking relationship network to obtain a representation vector of a second key phrase corresponding to the target text, where the first stacking relationship network is a network in the trained text processing model, the first key phrase corresponding to the target text is a key phrase existing in the target text, and the second key phrase corresponding to the target text is a key phrase not existing in the target text;

the decoding unit is further configured to decode a representation vector of a first key phrase corresponding to the target text to obtain a first key phrase corresponding to the target text, and decode a representation vector of a second key phrase corresponding to the target text to obtain a second key phrase corresponding to the target text;

the generating unit is further configured to generate a key phrase set corresponding to the target text, where the key phrase set includes a first key phrase corresponding to the target text and a second key phrase corresponding to the target text.

7. A text processing apparatus, characterized in that the text processing apparatus comprises an input interface and an output interface, and further comprises:

a computer storage medium having stored thereon one or more instructions adapted to be loaded by the processor and to execute the text processing method of any of claims 1-5.

8. A computer storage medium having computer program instructions stored therein for execution by a processor to perform a text processing method according to any one of claims 1-5.