CN112528621A

CN112528621A - Text processing method, text processing model training device and storage medium

Info

Publication number: CN112528621A
Application number: CN202110182270.9A
Authority: CN
Inventors: 周干斌; 林芬
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2021-02-10
Filing date: 2021-02-10
Publication date: 2021-03-19
Anticipated expiration: 2041-02-10
Also published as: CN112528621B

Abstract

The application relates to a text processing method, a text processing device, a computer device and a storage medium. The method comprises the following steps: acquiring an input text, a unit to be replaced and a replacement position; vectorizing the input text and the unit to be replaced to obtain a text vector and a unit vector to be replaced; performing forward semantic extraction and reverse semantic extraction based on the text vector to obtain a forward semantic vector and a reverse semantic vector, and determining a target forward semantic vector and a target reverse semantic vector from the forward semantic vector and the reverse semantic vector based on the replacement position; performing context semantic calculation based on the target forward semantic vector and the target reverse semantic vector to obtain a replacement position context semantic vector; and performing replacement prediction based on the context semantic vector of the replacement position and the vector of the unit to be replaced to obtain a replaceability, wherein the replaceability is used for representing the possibility that the text unit corresponding to the replacement position in the input text is replaced by the unit to be replaced. By adopting the method, the accuracy of text replacement can be improved.

Description

Text processing method, text processing model training device and storage medium

Technical Field

The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for text processing, text processing model training, and search, a computer device, and a storage medium.

Background

With the development of artificial intelligence technology, natural language processing technology has appeared, and text replacement, such as replacement of synonyms in text, can be realized through text processing technology in natural language processing technology. At present, a method for text replacement generally uses a regular expression or a template for matching, and words in a text are replaced after matching is successful. However, in the current method for text replacement by using a regular expression or a template, the problem that the replaced text is partially not smooth is easily caused, and the accuracy of text replacement is low.

Disclosure of Invention

In view of the foregoing, it is desirable to provide a text processing method and apparatus, a text processing model training method and apparatus, a computer device, and a storage medium, which can improve accuracy of text replacement.

A method of text processing, the method comprising:

acquiring an input text and corresponding replacement information, wherein the replacement information comprises a unit to be replaced and a replacement position;

vectorizing an input text to obtain a text vector, and vectorizing a unit to be replaced to obtain a unit vector to be replaced;

performing forward semantic extraction and reverse semantic extraction based on the text vector to obtain a forward semantic vector and a reverse semantic vector, and determining a target forward semantic vector and a target reverse semantic vector from the forward semantic vector and the reverse semantic vector based on the replacement position;

performing context semantic calculation based on the target forward semantic vector and the target reverse semantic vector to obtain a context semantic vector of the replacement position;

and performing replacement prediction based on the context semantic vector of the replacement position and the vector of the unit to be replaced to obtain a replaceability, wherein the replaceability is used for representing the possibility that the text unit corresponding to the replacement position in the input text is replaced by the unit to be replaced.

In one embodiment, the vectorizing the input text to obtain a text vector, and vectorizing the unit to be replaced to obtain a unit to be replaced includes:

carrying out word level division on the input text to obtain each input word, and vectorizing each input word to obtain each input word vector;

and carrying out word level division on the unit to be replaced to obtain each word to be replaced, and vectorizing each word to be replaced to obtain each word vector to be replaced.

In one embodiment, the performing reverse semantic extraction based on the text vector to obtain the reverse semantic vector, and determining a semantic vector corresponding to the start position of the unit to be replaced from the reverse semantic vector to obtain a first reverse semantic vector includes:

acquiring a preset termination hidden vector, and determining a reverse vector of a current word from a text vector according to the sequence from termination to initiation;

performing reverse hidden state calculation based on the preset termination hidden vector and the current word reverse vector to obtain a current word reverse semantic vector corresponding to the current word reverse vector;

taking the current word reverse semantic vector as a preset termination hidden vector, and returning to the step of iterative computation of sequentially determining the current word reverse vector from the text vector according to the sequence from termination to start until obtaining the reverse semantic vector corresponding to each word vector in the text vector;

and determining a target word reverse vector corresponding to the initial position of the unit to be replaced from the word vectors, and taking a reverse semantic vector corresponding to the target word reverse vector as a first reverse semantic vector.

A text processing apparatus, the apparatus comprising:

the text acquisition module is used for acquiring an input text and corresponding replacement information, wherein the replacement information comprises a unit to be replaced and a replacement position;

the vectorization module is used for vectorizing the input text to obtain a text vector and vectorizing the unit to be replaced to obtain a unit vector to be replaced;

the semantic vector determining module is used for performing forward semantic extraction and reverse semantic extraction on the basis of the text vector to obtain a forward semantic vector and a reverse semantic vector, and determining a target forward semantic vector and a target reverse semantic vector from the forward semantic vector and the reverse semantic vector on the basis of the replacement position;

the calculation module is used for performing context semantic calculation based on the target forward semantic vector and the target reverse semantic vector to obtain a context semantic vector of the replacement position;

and the replacement prediction module is used for performing replacement prediction on the basis of the context semantic vector of the replacement position and the unit vector to be replaced to obtain a replaceability, and the replaceability is used for representing the possibility that the text unit corresponding to the replacement position in the input text is replaced by the unit to be replaced.

A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:

A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:

According to the text processing method, the text processing device, the computer equipment and the storage medium, the input text and the corresponding replacement information are obtained, forward semantic extraction and reverse semantic extraction are carried out on the input text to obtain a forward semantic vector and a reverse semantic vector, a target forward semantic vector and a target reverse semantic vector are determined from the forward semantic vector and the reverse semantic vector based on the replacement position, a replacement position context semantic vector is obtained through calculation, replacement prediction is carried out by using the replacement position context semantic vector and the to-be-replaced unit vector to obtain the degree of replacement, and the degree of replacement is used for representing the possibility that the text unit corresponding to the replacement position in the input text is replaced by the to-be-replaced unit. The accuracy of the replaceability is improved by extracting the context semantic vector of the replacement position and then performing replacement prediction by using the context semantic vector of the replacement position and the unit vector to be replaced, and then the accuracy of text replacement can be improved by performing text replacement through the replaceability.

A method of text processing model training, the method comprising:

acquiring a training sample, wherein the training sample comprises a training text, training information and a training weight, and the training information comprises a training unit and a training replacement position;

inputting a training sample into an initial text processing model, performing initial vectorization on the input text by the initial text processing model to obtain an initial text vector, performing initial vectorization on a training unit to obtain an initial training unit vector, performing forward semantic extraction and reverse semantic extraction based on the initial text vector to obtain an initial forward semantic vector and an initial reverse semantic vector, determining a target initial forward semantic vector and a target initial reverse semantic vector from the initial forward semantic vector and the initial reverse semantic vector based on a training replacement position, performing initial context semantic calculation based on the target initial forward semantic vector and the target initial reverse semantic vector to obtain an initial context semantic vector of the training replacement position, and performing initial replacement prediction based on the initial context semantic vector of the training replacement position and the initial training unit vector, obtaining an initial degree of replacement;

and calculating to obtain initial target replaceability based on the initial replaceability and the training weight, and obtaining the text processing model when the initial target replaceability is maximized.

In one embodiment, after the calculating an initial target substitutability based on the initial substitutability and the training weights, the method further includes:

and when the initial target replaceability is not maximized, updating the initial text processing model based on the initial target replaceability to obtain an updated text processing model, taking the updated text processing model as the initial text processing model, and returning to the step of inputting the training sample into the initial text processing model for iterative execution until the initial target replaceability is maximized.

In one embodiment, the training samples include training replacement samples and non-replacement samples, the training replacement samples include training texts, training replacement information and replacement weights, the training replacement information includes training units to be replaced and training replacement positions, the non-replacement samples include the input texts, non-replacement information and non-replacement weights, and the non-replacement information includes non-replacement units and the training replacement positions;

said inputting said training samples into an initial text processing model, comprising

Inputting the training replacement sample and the non-replacement sample into an initial text processing model to obtain a first initial replaceability corresponding to the output training replacement sample and a second initial replaceability corresponding to the non-replacement sample;

the calculating based on the initial replaceability and the training weight to obtain an initial target replaceability, and when the initial target replaceability is maximized, obtaining a text processing model, including:

and calculating to obtain a first initial target alternative degree based on the first initial alternative degree and the replacement weight, calculating to obtain a second initial target alternative degree based on the second initial alternative degree and the non-replacement weight, obtaining a specific initial target alternative degree based on the first initial target alternative degree and the second initial target alternative degree, and obtaining a text processing model when the specific initial target alternative degree is maximized.

A text processing model training apparatus, the apparatus comprising:

the training system comprises a sample acquisition module, a training unit and a training replacement module, wherein the sample acquisition module is used for acquiring a training sample, the training sample comprises a training text, training information and training weight, and the training information comprises a training unit and a training replacement position;

an initial model calculation module for inputting the training samples into an initial text processing model, the initial text processing model performing initial vectorization on the input text to obtain an initial text vector, performing initial vectorization on the training units to obtain an initial training unit vector, performing forward semantic extraction and reverse semantic extraction based on the initial text vector to obtain an initial forward semantic vector and an initial reverse semantic vector, determining a target initial forward semantic vector and a target initial reverse semantic vector from the initial forward semantic vector and the initial reverse semantic vector based on the training replacement position, performing initial context semantic calculation based on the target initial forward semantic vector and the target initial reverse semantic vector to obtain an initial context semantic vector of the training replacement position, and performing initial replacement prediction based on the initial context semantic vector of the training replacement position and the initial training unit vector, obtaining an initial degree of replacement;

and the iteration module is used for calculating to obtain the initial target replaceability based on the initial replaceability and the training weight, and obtaining the text processing model when the initial target replaceability is maximized.

The text processing model training method, the device, the computer equipment and the storage medium have the advantages that the training samples are obtained, the training samples are input into the initial text processing model, then the initial text processing model carries out semantic extraction on the input text to obtain the initial forward semantic vector and the initial reverse semantic vector, the initial forward semantic vector and the initial reverse semantic vector are used for calculating to obtain the initial context semantic vector of the replacement position based on the replacement position, finally, the initial context semantic vector of the replacement position and the initial training unit vector are used for carrying out replacement prediction to obtain the initial replaceability, the initial target replaceability is calculated based on the initial replaceability and the training weight, when the initial target replaceability is maximized, the text processing model is obtained, and the accuracy of the replacement prediction is improved due to the fact that the replacement position context semantic vector is used for replacement prediction, and then when the initial target replaceability is maximized, obtaining the text processing model, thereby improving the accuracy of the trained text processing model.

A method of searching, the method comprising:

receiving a search instruction, wherein the search instruction carries a search text;

sending the search text to a server according to the search instruction, wherein the server receives the search text, searches corresponding replacement information based on the search text, the replacement information comprises a unit to be replaced and a replacement position, vectorizes the search text to obtain a search text vector, and vectorizes the unit to be replaced to obtain a unit vector to be replaced; forward semantic extraction and reverse semantic extraction are carried out on the basis of the search text vector to obtain a forward semantic vector and a reverse semantic vector, and a target forward semantic vector and a target reverse semantic vector are determined from the forward semantic vector and the reverse semantic vector on the basis of the replacement position; performing context semantic calculation based on the target forward semantic vector and the target reverse semantic vector to obtain a context semantic vector of the replacement position; performing replacement prediction on the context semantic vector of the replacement position and the vector of the unit to be replaced to obtain a replaceability, replacing the text unit corresponding to the replacement position in the search text by using the unit to be replaced based on the replaceability to obtain a replacement text, and searching a corresponding search result based on the replacement text and the search text;

and receiving a search result returned by the server, and displaying the search result.

A search apparatus, the apparatus comprising:

the instruction receiving module is used for receiving a search instruction, and the search instruction carries a search text;

the search module is used for sending the search text to the server according to the search instruction, the server receives the search text, searches corresponding replacement information based on the search text, the replacement information comprises a unit to be replaced and a replacement position, vectorizes the search text to obtain a search text vector, and vectorizes the unit to be replaced to obtain a unit vector to be replaced; forward semantic extraction and reverse semantic extraction are carried out on the basis of the search text vector to obtain a forward semantic vector and a reverse semantic vector, and a target forward semantic vector and a target reverse semantic vector are determined from the forward semantic vector and the reverse semantic vector on the basis of the replacement position; performing context semantic calculation based on the target forward semantic vector and the target reverse semantic vector to obtain a context semantic vector of a training replacement position; performing replacement prediction on the context semantic vector of the training replacement position and the vector of the unit to be replaced to obtain a replaceability, replacing a text unit corresponding to the replacement position in the search text by using the unit to be replaced based on the replaceability to obtain a replacement text, and searching a corresponding search result based on the replacement text and the search text;

and the display module is used for receiving the search result returned by the server and displaying the search result.

sending the search text to a server according to the search instruction, wherein the server receives the search text, searches corresponding replacement information based on the search text, the replacement information comprises a unit to be replaced and a replacement position, vectorizes the search text to obtain a search text vector, and vectorizes the unit to be replaced to obtain a unit vector to be replaced; forward semantic extraction and reverse semantic extraction are carried out on the basis of the search text vector to obtain a forward semantic vector and a reverse semantic vector, and a target forward semantic vector and a target reverse semantic vector are determined from the forward semantic vector and the reverse semantic vector on the basis of the replacement position; performing context semantic calculation based on the target forward semantic vector and the target reverse semantic vector to obtain a context semantic vector of a training replacement position; performing replacement prediction on the context semantic vector of the training replacement position and the vector of the unit to be replaced to obtain a replaceability, replacing a text unit corresponding to the replacement position in the search text by using the unit to be replaced based on the replaceability to obtain a replacement text, and searching a corresponding search result based on the replacement text and the search text;

According to the searching method, the searching device, the computer equipment and the storage medium, the searching text is sent to the server, the server searches for corresponding replacement information based on the searching text, the replacement position context semantic vector and the unit vector to be replaced are used for performing replacement prediction to obtain the replaceability, the unit to be replaced is used for replacing the text unit corresponding to the replacement position in the searching text based on the replaceability to obtain the replacement text, and the corresponding searching result is searched based on the replacement text and the searching text.

Drawings

FIG. 1 is a diagram of an application environment of a text processing method in one embodiment;

FIG. 2 is a flow diagram that illustrates a method for text processing in one embodiment;

FIG. 3 is a flow diagram illustrating model prediction in one embodiment;

FIG. 4 is a flow diagram illustrating network prediction in one embodiment;

FIG. 5 is a flow diagram that illustrates obtaining an alternative location context semantic vector, under an embodiment;

FIG. 6 is a flow diagram illustrating a process for obtaining a first forward semantic vector according to one embodiment;

FIG. 7 is a flow diagram illustrating a process for obtaining a first reverse semantic vector according to one embodiment;

FIG. 8 is a flow diagram illustrating the obtaining of an alternative in one embodiment;

FIG. 9 is a flowchart illustrating a method for training a text processing model according to one embodiment;

FIG. 10 is a schematic diagram of a process for obtaining training samples in one embodiment;

FIG. 11 is a flow diagram illustrating a search method in one embodiment;

FIG. 12 is a flowchart illustrating a method for text processing in an exemplary embodiment;

FIG. 13 is a diagram of a search page in one embodiment;

FIG. 14 is a diagram illustrating model prediction in one embodiment;

FIG. 15 is a block diagram showing a configuration of a text processing apparatus according to an embodiment;

FIG. 16 is a block diagram showing the construction of a text processing model training apparatus according to an embodiment;

FIG. 17 is a block diagram showing the structure of a search device in one embodiment;

FIG. 18 is a diagram showing an internal structure of a computer device in one embodiment;

fig. 19 is an internal configuration diagram of a computer device in another embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

Natural Language Processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that enable efficient communication between humans and computers using natural language. Natural language processing is a science integrating linguistics, computer science and mathematics. Therefore, the research in this field will involve natural language, i.e. the language that people use everyday, so it is closely related to the research of linguistics. Natural language processing techniques typically include text processing, semantic understanding, machine translation, robotic question and answer, knowledge mapping, and the like.

The scheme provided by the embodiment of the application relates to the technologies of text processing and the like of artificial intelligence, and is specifically explained by the following embodiment:

the text processing method provided by the application can be applied to the application environment shown in fig. 1. Wherein a user terminal 102 communicates with a server 104 over a network. The server 104 acquires an input text sent by the terminal 102, and the server 104 acquires replacement information corresponding to the input text from the database 106, wherein the replacement information comprises a unit to be replaced and a replacement position; the server 104 vectorizes the input text to obtain a text vector, and vectorizes the unit to be replaced to obtain a unit vector to be replaced; the server 104 performs forward semantic extraction and reverse semantic extraction based on the text vector to obtain a forward semantic vector and a reverse semantic vector, and determines a target forward semantic vector and a target reverse semantic vector from the forward semantic vector and the reverse semantic vector based on the replacement position; the server 104 performs context semantic calculation based on the target forward semantic vector and the target reverse semantic vector to obtain a context semantic vector of the replacement position; the server 104 performs replacement prediction based on the context semantic vector of the replacement position and the unit vector to be replaced, to obtain a replaceability, where the replaceability is used to represent a possibility that a text unit corresponding to the replacement position in the input text is replaced by the unit to be replaced. The terminal 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices, and the server 104 may be implemented by an independent server or a server cluster formed by a plurality of servers.

In an embodiment, as shown in fig. 2, a text processing method is provided, which is described by taking the method applied to the server in fig. 1 as an example, and it is understood that the text processing method may also be applied to a terminal, and may also be applied to a blockchain node, and in this embodiment, the method includes the following steps:

step 202, acquiring an input text and corresponding replacement information, wherein the replacement information comprises a unit to be replaced and a replacement position.

The input text refers to a text to be replaced, and the input text may be a sentence, an exercise, or a chapter. The input text may be text of different language types, for example, chinese text, japanese text, english text, german text, and so forth. The text replacement may be to replace a word, word or sentence, etc. in the input text. The replacement information is used for representing specific information of replacement, and includes a unit to be replaced and a replacement position, the unit to be replaced refers to a text unit waiting for replacement, and is a unit which needs to predict whether replacement can be performed, and the unit to be replaced can be a sentence, a word, a symbol, an expression and the like. The unit to be replaced may be a synonymous unit, a unit obtained by using a replacement template, or a unit obtained by using a regular expression. The replacement position is used for indicating the position of the unit to be replaced in the input text, namely the position of the replaced unit in the input text. The replaced units in the input text refer to the text units in the input text which are replaced by the units to be replaced.

Specifically, the server may retrieve the input text and the corresponding replacement information from the database. The server can also acquire the input text input by the user from the terminal and then find the corresponding replacement information according to the input text. For example, the server may segment the input text, find the matched word to be replaced from the database according to the segmentation result, store the word to be replaced and the replacement word with the pre-established matching relationship in the database, and then determine the word replacement position according to the word to be replaced matched with the segmentation result. In one embodiment, the server may obtain the input text and the corresponding replacement information sent by a third party server, which is a server for providing business services.

In one embodiment, the unit to be replaced may also be a sentence to be replaced or a word to be replaced. The sentence to be replaced refers to a sentence which needs to be predicted whether to be replaced into the input text, and the word to be replaced refers to a word which needs to be predicted whether to be replaced into the input text. And then the replacement information acquired by the server comprises sentence replacement positions corresponding to the sentences to be replaced or word replacement positions corresponding to the words to be replaced.

And step 204, vectorizing the input text to obtain a text vector, and vectorizing the unit to be replaced to obtain a unit vector to be replaced.

The text vector refers to a vector corresponding to an input text, and the unit vector to be replaced refers to a vector corresponding to a unit to be replaced.

Specifically, the server may perform vectorization on the input text and the unit to be replaced, respectively, to obtain a text vector and a unit to be replaced, where the word-level vectorization may be performed, that is, the input text and the unit to be replaced are divided into words, and then the words are vectorized. Word-level vectorization, i.e., dividing the input text and the unit to be replaced into words, and then vectorizing the words, may also be performed. In an embodiment, when the unit to be replaced is an emoticon, an emoticon vector corresponding to the preset emoticon may be acquired.

And step 206, performing forward semantic extraction and reverse semantic extraction based on the text vector to obtain a forward semantic vector and a reverse semantic vector, and determining a target forward semantic vector and a target reverse semantic vector from the forward semantic vector and the reverse semantic vector based on the replacement position.

The forward semantic extraction refers to semantic extraction sequentially from a starting position to an ending position in a text vector. The reverse semantic extraction refers to semantic extraction sequentially from the end position to the end position in the text vector. The forward semantic vector refers to a vector obtained by performing semantic extraction from a starting position to an ending position based on a text vector. The reverse semantic vector is a vector obtained by sequentially performing semantic extraction from an end position to a start position based on a text vector. The target forward semantic vector refers to a vector derived from the forward semantic vector according to the replacement position. The target reverse semantic vector refers to a vector derived from the reverse semantic vector according to the replacement position.

Specifically, the server extracts forward semantics of the text vector according to the sequence from the initial position to the final position and extracts reverse semantics according to the sequence from the final position to the initial position to obtain a forward semantic vector and a reverse semantic vector, then determines a semantic vector corresponding to the context at the replacement position from the forward semantic vector according to the replacement position to obtain a target forward semantic vector, and determines a semantic vector corresponding to the context at the replacement position from the reverse semantic vector to obtain a target reverse semantic vector.

And 208, performing context semantic calculation based on the target forward semantic vector and the target reverse semantic vector to obtain a context semantic vector of the replacement position.

The context semantic vector of the replacement position is used for representing the semantic vector corresponding to the context of the replacement position in the input text. The context of the alternative position refers to a text obtained by text screening from the input text according to the alternative position. For example, the text before the start position of the unit to be replaced may contain the replacement position and the text before the end position of the word or the text after the start position of the unit to be replaced, and so on.

Specifically, the server may perform weighted calculation on the target forward semantic vector and the target reverse semantic vector to obtain a context semantic vector of the replacement location, for example, the target forward semantic vector may be weighted by using a preset forward weight, the target reverse semantic vector may be weighted by using a preset reverse weight, and then the weighted sum is calculated to obtain the context semantic vector of the replacement location. The preset forward weight and the preset reverse weight may be preset weights, or weights obtained after training using a neural network.

And 210, performing replacement prediction based on the context semantic vector of the replacement position and the unit vector to be replaced to obtain a replaceability, wherein the replaceability is used for representing the possibility that the text unit corresponding to the replacement position in the input text is replaced by the unit to be replaced.

The replaceability is used to represent the possibility that the text unit corresponding to the replacement position in the input text is replaced by the unit to be replaced, that is, the probability that the text unit corresponding to the replacement position in the input text is replaced by the unit to be replaced.

Specifically, the server calculates a word distribution vector corresponding to the unit to be replaced by using the context semantic vector of the replacement position and the unit to be replaced, and obtains the degree of replacement according to the word distribution vector, where the word distribution vector is used to represent the replacement probability corresponding to each division result after the unit to be replaced is divided, for example, when the division result is a word, the word distribution vector includes the replacement probability corresponding to each word. And then calculating to obtain the replaceability according to the replacement probability corresponding to each division result. The input text is replaced based on the degree of replaceability. In an embodiment, the server may replace the text unit corresponding to the replacement position in the input text by using the unit to be replaced based on the degree of replaceability to obtain a replaced text, that is, compare the degree of replaceability with a preset degree of replaceability threshold, and replace the word corresponding to the replacement position in the input text by using the unit to be replaced when the degree of replaceability exceeds the preset degree of replaceability threshold to obtain a replaced text, where the replaced text is obtained by replacing the input text by using the unit to be replaced. When the degree of replaceability does not exceed the preset degree of replaceability threshold, the input text remains unchanged.

According to the text processing method, the text processing device, the computer equipment and the storage medium, the input text and the corresponding replacement information are obtained, then forward semantic extraction and reverse semantic extraction are carried out on the input text to obtain a forward semantic vector and a reverse semantic vector, a target forward semantic vector and a target reverse semantic vector are determined from the forward semantic vector and the reverse semantic vector based on the replacement position, then a replacement position context semantic vector is obtained through calculation, and finally replacement prediction is carried out by using the replacement position context semantic vector and the unit vector to be replaced to obtain the replaceability. The method comprises the steps of extracting a context semantic vector of a replacement position, and then performing replacement prediction by using the context semantic vector of the replacement position and a unit vector to be replaced, so that the accuracy of the obtained replaceability is improved, and then when text replacement is performed through the replaceability, the accuracy of text replacement can be improved.

In one embodiment, as shown in fig. 3, the text processing method further includes:

step 302, inputting an input text and replacement information into a text processing model, vectorizing the input text by the text processing model to obtain a text vector, vectorizing a unit to be replaced to obtain a unit vector to be replaced, performing forward semantic extraction and reverse semantic extraction based on the text vector to obtain a forward semantic vector and a reverse semantic vector, and determining a target forward semantic vector and a target reverse semantic vector from the forward semantic vector and the reverse semantic vector based on a replacement position; performing context semantic calculation based on the target forward semantic vector and the target reverse semantic vector to obtain a context semantic vector of a replacement position, and performing replacement prediction based on the context semantic vector of the replacement position and a unit vector to be replaced to obtain a replaceability; the text processing model is obtained by training by using a training text and corresponding training replacement information based on a neural network algorithm.

And 304, replacing the text unit corresponding to the replacement position in the input text by using the unit to be replaced based on the replaceability to obtain a replacement text.

The text processing model is obtained by training by using a training text and corresponding training replacement information based on a neural network algorithm, wherein the training text refers to an input text used during training, and the training replacement information refers to replacement information corresponding to the training text used during training. The Neural Network algorithm may be an RNN algorithm, an RNN (Recurrent Neural Network) algorithm, and an RNN is a Recurrent Neural Network in which sequence data is input, recursion (recursion) is performed in an evolution direction of the sequence, and all nodes (Recurrent units) are connected in a chain manner. The Neural network algorithm may also be a BERT (Bidirectional Encoder from converters) algorithm, a CNN (Convolutional Neural Networks) algorithm, or the like. The replacement text refers to a text obtained by replacing the input text with a unit to be replaced.

Specifically, the server uses a training text and corresponding training replacement information to perform training in advance based on a neural network algorithm to obtain a text processing model, and then deploys the text processing model for use. When the server obtains the input text and the corresponding replacement information, the input text and the replacement information are input into a text processing model, the text processing model carries out replacement prediction to obtain output replaceability, and when the replaceability exceeds a preset replaceability threshold value, a unit to be replaced is used for replacing the text corresponding to the replacement position in the input text to obtain a replacement text.

In the embodiment, the efficiency of text replacement prediction can be improved by inputting the input text and the replacement information into the text processing model for text replacement prediction.

In one embodiment, the text processing model includes an encoding network and a decoding network.

As shown in fig. 4, step 302, inputting the input text and the replacement information into the text processing model, includes:

step 402, inputting an input text and replacement information into a coding network, vectorizing the input text by the coding network to obtain a text vector, vectorizing a unit to be replaced to obtain a unit vector to be replaced, performing forward semantic extraction and reverse semantic extraction based on the text vector to obtain a forward semantic vector and a reverse semantic vector, determining a target forward semantic vector and a target reverse semantic vector from the forward semantic vector and the reverse semantic vector based on a replacement position, and performing context semantic calculation based on the target forward semantic vector and the target reverse semantic vector to obtain a context semantic vector of the replacement position.

And 404, inputting the context semantic vector of the replacement position and the unit vector to be replaced into a decoding network, and performing replacement prediction on the decoding network based on the context semantic vector of the replacement position and the unit vector to be replaced to obtain the degree of replacement.

Wherein an encoding network, which may be a bi-directional recurrent neural network, is used to extract the alternative location context semantic vector. I.e. both the text before and after the replacement location will influence the semantic information of the replacement location. The decoding network is used for carrying out replacement prediction to obtain the replaceability. The decoding network can be a recurrent neural network, and the replacement probability corresponding to the division result of the unit to be replaced is calculated by using the semantic vector of the replacement position context and the vector of the unit to be replaced in a sequence generation mode.

Specifically, the server inputs the input text and the replacement information into a coding network, the coding network outputs a context semantic vector of the replacement position through calculation, then the context semantic vector of the replacement position and a unit vector to be replaced are input into a decoding network, the decoding network carries out replacement prediction on the context semantic vector of the replacement position and the unit vector to be replaced, and the obtained replaceability is output.

In the above embodiment, the alternative location context semantic vector is extracted through the encoding network, and then the replaceability is obtained through prediction by the decoding network, so that the accuracy of the obtained replaceability is improved.

In one embodiment, vectorizing the input text and the unit to be replaced to obtain a text vector and a unit to be replaced vector, includes:

carrying out word level division on an input text to obtain each input word, and vectorizing each input word to obtain each input word vector; and carrying out word level division on the unit to be replaced to obtain each word to be replaced, and vectorizing each word to be replaced to obtain each word vector to be replaced.

The input words refer to words in the input text, and the input word vectors refer to vectors corresponding to the input words. The word to be replaced refers to a word in the unit to be replaced, and the word vector to be replaced refers to a vector corresponding to the word to be replaced.

Specifically, the server performs word level division on an input text to obtain each input word, then performs vectorization on each input word, performs word level division on a unit to be replaced to obtain each word to be replaced, and performs vectorization on each word to be replaced to obtain each word vector to be replaced. Any vectorization method may be used, such as dictionary vectorization, vectorization using word set models, vectorization using bag of words models, vectorization using one-hot encoding, vectorization using neural network, and so on.

In one embodiment, the input text is subjected to word level division to obtain each input word, each input word is vectorized to obtain each input word vector, the unit to be replaced is subjected to word level division to obtain each unit to be replaced, and each unit to be replaced is vectorized to obtain each unit to be replaced.

In the embodiment, the input text and the unit to be replaced are subjected to word level division and then vectorization, so that each input word vectorization and each word vector to be replaced are obtained, and subsequent use is facilitated.

In one embodiment, the replacement position includes a start position of the unit to be replaced and a length of the unit to be replaced;

as shown in fig. 5, forward semantic extraction and reverse semantic extraction are performed based on a text vector to obtain a forward semantic vector and a reverse semantic vector, and a target forward semantic vector and a target reverse semantic vector are determined from the forward semantic vector and the reverse semantic vector based on a replacement position, including:

step 502, forward semantic extraction is performed based on the text vector to obtain a forward semantic vector, and a semantic vector before the initial position of the unit to be replaced is determined from the forward semantic vector to obtain a first forward semantic vector.

The starting position of the unit to be replaced refers to a corresponding starting position when the unit to be replaced is replaced in the input text. The length of the unit to be replaced is the length of the unit to be replaced, and can be obtained according to the number of the symbols in the unit to be replaced, for example, if the input text is "please explain what time you mom birthday is", the unit to be replaced is "mom", and the starting position of the unit to be replaced can be the position of the fifth word "mom" in the input text. The length of the unit to be replaced is 2. The forward semantic vector is a vector obtained by performing semantic extraction on the text vector in sequence from the starting position to the ending position. The first forward semantic vector refers to a semantic vector corresponding to a text before the starting position of the unit to be replaced in the input text, that is, semantic information for representing the text from the left side of the input text to the left side of the replacement position. When the unit to be replaced is an emoticon, one emoticon is 1 length. The length of the emoticon may be preset.

Specifically, the forward semantic vector refers to performing semantic extraction on vectors corresponding to each division result in the text vector in sequence from the starting position to the ending position to obtain each forward semantic vector, for example, performing semantic extraction on each word vector in the text vector to obtain a forward semantic vector corresponding to each word vector. And then determining a corresponding forward semantic vector before the initial position of the unit to be replaced from each forward semantic vector to obtain a first forward semantic vector. For example, a semantic vector of a word "you" before the position of "mom" is obtained by calculation, and a first forward semantic vector is obtained, and the first forward semantic vector is used for representing semantic information of "please explain you" in the input text.

In one embodiment, the target text is obtained by adding characters for representing the beginning of the text and characters for representing the END of the text to the input text, for example, adding a special segment < END > to each of the beginning position and the END position of the input text. And then the server vectorizes the target text to obtain a target text vector. And performing forward semantic extraction by using the target text vector to obtain a first forward semantic vector.

Step 504, determining a target position based on the initial position of the unit to be replaced and the length of the unit to be replaced, and determining a semantic vector corresponding to the target position from the forward semantic vectors to obtain a second forward semantic vector.

The target position refers to a calculated position where the second forward semantic vector can be calculated, that is, the target position refers to a termination position of the unit to be replaced in the input text, for example, the termination position corresponding to the unit to be replaced "mom" in the input text "please explain when your mom birthday is" is six. The second forward semantic vector is used for representing the semantic vector corresponding to the word vector of the target position in the input text, namely the second forward semantic vector is used for representing semantic information from the left side of the input text to the right side of the replacement position.

Specifically, the server determines a target position based on the starting position of the unit to be replaced and the length of the unit to be replaced, and then determines semantic vectors corresponding to word vectors before the target position from forward semantic vectors corresponding to word vectors, so as to obtain a second forward semantic vector. For example, a forward semantic vector of the sixth word "mom" is obtained through calculation, and a first forward semantic vector is obtained, where the first forward semantic vector is used to represent semantic information of "please explain your mom" in the input text.

Step 506, performing reverse semantic extraction based on the text vector to obtain a reverse semantic vector, and determining a semantic vector corresponding to the initial position of the unit to be replaced from the reverse semantic vector to obtain a first reverse semantic vector.

The reverse semantic vector is obtained by performing semantic extraction on the text vector in sequence from the end position to the end of the initial position. The first reverse semantic vector refers to a semantic vector corresponding to a text before the starting position of the unit to be replaced in the input text, that is, semantic information for representing the text from the right side of the input text to the left side of the replacement position.

Specifically, the server sequentially performs semantic extraction on the text vectors according to the sequence from the end position to the end position of the start position to obtain each reverse semantic vector, and then determines the reverse semantic vector corresponding to the start position of the unit to be replaced from each reverse semantic vector to obtain a first reverse semantic vector. For example, the initial position corresponding to the unit to be replaced "mom" in the text "please explain when your mom birthday is five", then the reverse semantic vector of the fifth word "mom" is calculated to obtain the first reverse semantic vector, and the first reverse semantic vector is used for representing the semantic information of "time-of-day-of-life mom".

Step 508, determining a semantic vector behind the target position from the reverse semantic vectors to obtain a second reverse semantic vector.

The second reverse semantic vector is used for representing the reverse semantic vector behind the target position, and is semantic information from the right side of the input text to the right side of the replacement position.

Specifically, the server determines a semantic vector behind the target position from the reverse semantic vectors to obtain a second reverse semantic vector. For example, the termination position corresponding to the unit to be replaced "mom" in the text "please explain when your mom birthday is" is five, then the reverse semantic vector corresponding to the "birthday" word after the sixth word "mom" is calculated to obtain the first reverse semantic vector, and the first reverse semantic vector is used for representing the semantic information of "how long you are birthday".

Performing context semantic calculation based on the target forward semantic vector and the target reverse semantic vector to obtain a context semantic vector of the replacement position, wherein the context semantic vector comprises:

step 510, calculating a weighted sum based on the first forward semantic vector, the second forward semantic vector, the first reverse semantic vector and the second reverse semantic vector to obtain a context semantic vector of the replacement position.

Specifically, the server calculates a weighted sum of a first forward semantic vector, a second forward semantic vector, a first reverse semantic vector and a second reverse semantic vector, wherein the first forward semantic vector, the second forward semantic vector, the first reverse semantic vector and the second reverse semantic vector may have different weights, may be preset, or may be obtained through neural network training. A context semantic vector for the alternate location is then derived.

In the embodiment, the replaced position context semantic vector is obtained by calculating the first forward semantic vector, the second forward semantic vector, the first reverse semantic vector and the second reverse semantic vector, so that the replaced position context semantic vector obtained by calculation is more accurate, and the accuracy of the obtained recommendability can be improved.

In one embodiment, as shown in fig. 6, step 502, performing forward semantic extraction based on the text vector to obtain a forward semantic vector, determining a semantic vector before the start position of the word from the forward semantic vector to obtain a first forward semantic vector, includes:

step 602, obtaining a preset starting hidden vector, and determining a current word vector from the text vectors according to the sequence from the start to the end.

The preset starting hidden vector is a preset vector used for hidden state calculation from a starting position to an ending position. The current word vector refers to a word vector of which the semantic vector needs to be calculated currently in the text vectors obtained by dividing according to the word level. In one embodiment, a current word forward vector may be determined from the text vectors in order from beginning to end, where the current word vector refers to a word vector of which semantic vectors currently need to be calculated in the text vectors obtained by word-level division.

Specifically, the service ticket obtains a preset starting hidden vector, and determines a current word vector from the text vector in an order from start to end, for example, determining a starting word vector corresponding to a starting word from the text vector.

Step 604, performing forward hidden state calculation based on the preset initial hidden vector and the current word vector to obtain a current word forward semantic vector corresponding to the current word vector.

Specifically, the server uses a recurrent neural network to perform forward hidden state calculation based on a preset initial hidden vector and a current word vector, so as to obtain a current word forward semantic vector corresponding to a previous word vector.

And 606, taking the forward semantic vector of the current word as a preset initial hidden vector, and returning to the step of determining the current word vector from the text vector in sequence from the beginning to the end for iterative computation until the forward semantic vector corresponding to each word vector in the text vector is obtained.

Specifically, the server takes the forward semantic vector of the current word as a preset initial hidden vector, and returns to the step of iterative computation of sequentially determining the current word vector from the text vector according to the sequence from the beginning to the end until the forward semantic vector corresponding to each word vector in the text vector is obtained, that is, when the forward semantic vector corresponding to the end vector in the text vector is obtained, the iteration is finished.

In a specific embodiment, in the bidirectional recurrent neural network, forward semantic extraction is performed sequentially from the start position to the end position of the input text, and a forward semantic vector can be obtained by using formula (1) for extraction.

Formula (1)

Wherein the content of the first and second substances,

representing the forward semantic vector corresponding to the t-th word,

representing the forward semantic vector corresponding to the t-1 th word,

representing a word vector in the text vector corresponding to the t-th word.

It is meant that the default starting hidden vector may be set to an initial value, such as 0, or a trainable vector or other value. I.e. server usage

And calculating a forward semantic vector corresponding to the initial word vector by using a formula (1) for the initial word vector in the text vector, then iterating the forward semantic vector corresponding to the word vector after the initial word vector by using the forward semantic vector corresponding to the initial word vector, and ending the iteration until the forward semantic vector corresponding to the ending word vector is obtained by calculationSegment of<END>Corresponding word vector is obtained by calculation<END>And when the word vector of (1) corresponds to the forward semantic vector, the iteration is ended.

Step 608, determining a target word vector before the start position of the unit to be replaced from the word vectors, and using the forward semantic vector corresponding to the target word vector as the first forward semantic vector.

Specifically, the target word vector refers to a vector corresponding to a word before the start position of the unit to be replaced in the input text. And the server takes the forward semantic vector corresponding to the target word vector as a first forward semantic vector. For example, if the starting position of the unit to be replaced is B and the length of the unit to be replaced is L, the first forward semantic vector is the forward semantic vector corresponding to the B-1 th position.

In one embodiment, a word vector corresponding to the target position is determined from the word vectors, and a forward semantic vector corresponding to the word vector corresponding to the target position is taken as a second forward semantic vector. For example, if the starting position of the unit to be replaced is B and the length of the unit to be replaced is L, the second forward semantic vector is the forward semantic vector corresponding to the B-1+ L position.

In the above embodiment, the forward semantic vector corresponding to each word vector in the text vector is obtained by using the preset initial hidden vector and the text vector through iterative computation, and then the forward semantic vector is determined from the forward semantic vector corresponding to each word vector based on the replacement information, so that the accuracy of the obtained forward semantic vector is improved.

In an embodiment, as shown in fig. 7, in step 506, performing reverse semantic extraction based on the text vector to obtain a reverse semantic vector, and determining a semantic vector corresponding to the start position of the unit to be replaced from the reverse semantic vector to obtain a first reverse semantic vector, where the method includes:

step 702, obtaining a preset termination hidden vector, and determining a current word reverse vector from the text vector according to the sequence from termination to start.

The preset termination hidden vector is a preset vector used for hidden state calculation from the termination position to the starting position, and may be the same as or different from the preset starting hidden vector. The current word reverse vector refers to a vector corresponding to the current word determined from the text vector at the time of reverse calculation. The current word reverse vector is the vector that currently needs to compute the reverse semantic information. Reverse refers to from the end position to the start position and forward refers to from the start position to the end position.

Specifically, the server obtains a set termination hidden vector, and determines a current word reverse vector from the text vector according to a sequence from termination to start, for example, a termination word vector corresponding to a termination word in the text vector is used as the current word reverse vector.

Step 704, performing reverse hidden state calculation based on the preset termination hidden vector and the current word reverse vector to obtain a current word reverse semantic vector corresponding to the current word reverse vector.

Step 706, taking the reverse semantic vector of the current word as a preset termination hidden vector, and returning to the step of determining the reverse vector of the current word from the text vector in sequence from termination to start for iterative computation until the reverse semantic vector corresponding to each word vector in the text vector is obtained.

Specifically, the server performs reverse hidden state calculation using the following formula (2) to obtain a current word reverse semantic vector corresponding to the current word reverse vector.

Formula (2)

Wherein the content of the first and second substances,

representing the reverse semantic vector corresponding to the t-th word,

representing the reverse semantic vector corresponding to the t-1 th word,

representing a word vector in the text vector corresponding to the t-th word.

It is meant that the default termination hidden vector may be set to an initial value, such as 0, or a trainable vector or other value. I.e. server usage

And calculating a reverse semantic vector corresponding to the terminating word vector in the text vector by using a formula (2), then iteratively calculating a reverse semantic vector corresponding to a word vector before the terminating word vector by using the reverse semantic vector corresponding to the terminating word vector, and ending the iteration until the reverse semantic vector corresponding to the starting word vector is obtained by calculation<END>Corresponding word vector is obtained by calculation<END>And when the word vector of (1) corresponds to the reverse semantic vector, the iteration is ended.

Step 708, determining a target word reverse vector corresponding to the initial position of the unit to be replaced from each word vector, and taking a reverse semantic vector corresponding to the target word reverse vector as a first reverse semantic vector.

Specifically, the target word reverse vector is a vector of a word corresponding to the initial position of the unit to be replaced in the input text, and the server takes the reverse semantic vector corresponding to the target word reverse vector as a first reverse semantic vector. For example, if the starting position of the unit to be replaced is B and the length of the unit to be replaced is L, the first reverse semantic vector is a forward semantic vector corresponding to the B-th position.

In one embodiment, word vectors after the target position are determined from the word vectors, and the reverse semantic vector corresponding to the word vector after the target position is taken as the second reverse semantic vector. For example, if the start position of the unit to be replaced is B and the length of the unit to be replaced is L, the second reverse semantic vector is a forward semantic vector corresponding to the B + L-th position.

In the above embodiment, the reverse semantic vector corresponding to each word vector in the text vector is obtained by using the preset termination hidden vector and the text vector through iterative computation, and then the reverse semantic vector is determined from the reverse semantic vector corresponding to each word vector based on the replacement information, so that the accuracy of the obtained reverse semantic vector is improved.

In a specific embodiment, the alternative location context semantic vector may be calculated using equation (3) as shown below.

Formula (3)

Wherein C refers to an alternate location context semantic vector,

refers to the first forward semantic vector,

is the weight of the first forward semantic vector,

refers to the second forward semantic vector,

refers to the weight of the second forward semantic vector.

Refers to the first reverse semantic vector that is,

refers to the weight of the first reverse semantic vector,

refers to a second reverse semantic vector that is,

refers to the weight of the second reverse semantic vector. Wherein the content of the first and second substances,

、

、

and

either a specified value or using recurrent neural network training. The replacement position context semantic vector is obtained by calculation through the formula, so that the efficiency and the accuracy of obtaining the replacement position context semantic vector are improved.

In one embodiment, performing replacement prediction based on the replacement location context semantic vector and the unit to be replaced vector to obtain the replaceability, including:

and performing hidden state calculation based on the context semantic vector of the replacement position and the unit vector to be replaced to obtain a hidden vector to be replaced, and performing regression calculation based on the hidden vector to be replaced to obtain the replaceability.

The hidden vector to be replaced refers to a hidden state vector corresponding to the unit vector to be replaced, and each word vector in the unit vector to be replaced has a corresponding hidden vector to be replaced.

Specifically, the server performs hidden state calculation using the replacement position context semantic vector and the unit vector to be replaced, for example, the hidden state calculation may be performed using a recurrent neural network to obtain a hidden vector to be replaced, and then performs regression calculation on the hidden vector to be replaced to obtain the degree of replacement.

In one embodiment, as shown in fig. 8, performing hidden state calculation based on the context semantic vector of the replacement position and the unit vector to be replaced to obtain a hidden vector to be replaced, and performing regression calculation based on the hidden vector to be replaced to obtain the degree of replacement includes:

step 802, obtaining a preset target hidden vector, and determining a current word vector to be replaced from the unit vectors to be replaced according to the sequence from the beginning to the end.

The preset target hidden vector is a preset starting hidden vector used for calculating the hidden vector to be replaced. The current word vector to be replaced refers to a vector corresponding to the word to be replaced, which needs to be subjected to the calculation of the hidden vector to be replaced currently, in the unit to be replaced.

Specifically, the server obtains a preset target hidden vector, then determines an initial word vector to be replaced corresponding to an initial word to be replaced from the unit vectors to be replaced according to the sequence from the beginning to the end, and takes the initial word vector to be replaced as the current word vector to be replaced.

Step 804, calculating a current word hidden vector to be replaced corresponding to the current word vector to be replaced based on the preset target hidden vector, the context semantic vector of the replacement position and the current word vector to be replaced.

Specifically, the server calculates to obtain a word hidden vector to be replaced corresponding to each word to be replaced, for example, the word hidden vector to be replaced may be calculated by using the following formula (4).

Formula (4)

Wherein the content of the first and second substances,

and representing a word hidden vector to be replaced corresponding to the t-th word to be replaced.

Representing the t-1 th latent vector of the word to be replaced.

Is a preset target hidden vector.

And representing the word vector to be replaced corresponding to the t-th word to be replaced. C represents the alternate location context semantic vector. Performing hidden state calculation by using a recurrent neural network based on a preset target hidden vector, a replacement position context semantic vector and a current word vector to be replaced to obtain a current word vector to be replaced corresponding to the current word vector to be replacedAnd replacing the hidden vector.

Step 806, performing regression calculation based on the hidden vector of the current word to be replaced to obtain the replacement probability of the current word to be replaced corresponding to the vector of the current word to be replaced.

Specifically, the server performs regression calculation by using the word hidden vector to be replaced corresponding to each word to be replaced, so as to obtain the word replacement probability to be replaced corresponding to each word vector to be replaced. For example, the word distribution vector corresponding to the unit to be replaced can be calculated by using the following formula (5), so as to obtain the replacement probability of each word to be replaced.

Formula (5)

Wherein the content of the first and second substances,

represents the replacement probability corresponding to the t-th word to be replaced under the condition of C,

and representing the distribution vector corresponding to the t-th word to be replaced.

Representing the hidden vector of the word to be replaced corresponding to the t-th word to be replaced, U is a network parameter and is obtained by training,

where R is a real number and V refers to the total number of words in the unit to be replaced. The Softmax function refers to a normalized exponential function that "compresses" a K-dimensional vector z containing arbitrary real numbers into another K-dimensional real vector σ (z) such that each element ranges between (0,1) and the sum of all elements is 1.

And 808, taking the hidden vector of the word to be replaced as a preset target hidden vector, and returning to the step of determining the current word vector to be replaced from the unit vector to be replaced according to the sequence from the beginning to the end for iterative execution until the replacement probability of the word to be replaced corresponding to each word vector to be replaced is obtained.

Specifically, the server takes the hidden vector of the word to be replaced currently as a preset target hidden vector, and returns to step 802 to perform iterative execution of the step of determining the vector of the word to be replaced currently from the vector of the unit to be replaced according to the sequence from the beginning to the end until the replacement probability of the word to be replaced corresponding to each vector of the word to be replaced is obtained.

And 810, performing cumulative multiplication calculation based on the replacement probability of the word to be replaced corresponding to each word vector to be replaced to obtain the degree of replacement.

Specifically, the server performs cumulative multiplication calculation on the replacement probability of the word to be replaced corresponding to each word vector to be replaced to obtain the degree of replacement. For example, the degree of replaceability can be calculated using equation (6) shown below.

Formula (6)

Wherein the content of the first and second substances,

the probability that the word with the replacement position including the starting position B of the unit to be replaced and the length L of the unit to be replaced is replaced by the unit to be replaced is determined under the condition that the text q is input. In one embodiment, the server may also calculate a weighted sum of the replacement probabilities of the words to be replaced, to obtain the degree of replacement.

In the embodiment, the replacement probability corresponding to each word to be replaced is obtained by calculating the context semantic vector of the replacement position and the word vector to be replaced, and the degree of replacement is further obtained by calculation, so that the obtained degree of replacement is more accurate.

In one embodiment, as shown in fig. 9, a text processing model training method is provided, which is described by taking the method as an example applied to the server in fig. 1, and includes the following steps:

step 902, a training sample is obtained, wherein the training sample comprises a training text, training information and training weights, and the training information comprises training units and training replacement positions.

The training samples refer to samples used in training a text processing model. The training text refers to input text used in training. The training information refers to the replacement information corresponding to the training text during training. The signal weight refers to a weight corresponding to the training text, the training weight is used for representing the number of times of presentation of the training text, and the number of times of presentation is obtained through statistics. A training unit refers to a unit of text used in training. The training alternative position refers to an alternative position corresponding to a text unit used in training.

Specifically, the server may directly obtain the training sample from the database, may also collect the corpus from the internet, extract the training sample through the corpus, and may also directly obtain the training sample from a service provider providing the training sample.

Step 904, inputting the training samples into an initial text processing model, the initial text processing model performing initial vectorization on the input text to obtain an initial text vector, performing initial vectorization on the training units to obtain an initial training unit vector, performing forward semantic extraction and reverse semantic extraction based on the initial text vector to obtain an initial forward semantic vector and an initial reverse semantic vector, determining a target initial forward semantic vector and a target initial reverse semantic vector from the initial forward semantic vector and the initial reverse semantic vector based on the training replacement positions, performing initial context semantic calculation based on the target initial forward semantic vector and the target initial reverse semantic vector to obtain an initial context semantic vector of the training replacement positions, performing initial replacement prediction based on the initial context semantic vector of the training replacement positions and the initial training unit vector, an initial degree of replaceability is obtained.

The initial text processing model refers to a text processing model with initialized model parameters. The initial text processing model is built using a recurrent neural network. The initial text vector refers to a vector corresponding to a training text obtained by vectorizing the initial text processing model. The initial training unit vector is a vector corresponding to a training unit obtained by vectorizing the initial text processing model. The initial forward semantic vector and the initial reverse semantic vector are vectors obtained by performing semantic extraction by using initial semantic extraction parameters in the initial text processing model. The initial context semantic vector of the training replacement position refers to the context semantic vector of the training replacement position calculated by using the initial text processing model. The initial replaceability refers to the replaceability corresponding to the training unit predicted by using the initial text processing model.

Specifically, the server inputs a training sample into an initial text processing model for training, the initial text processing model obtains a context semantic vector of a training replacement position through bidirectional cyclic neural network calculation, then a unidirectional cyclic neural network is used for replacement prediction according to the context semantic vector of the training replacement position and an initial training unit vector, and initial replaceability is output.

And step 906, calculating to obtain the initial target replaceability based on the initial replaceability and the training weight.

Wherein, the initial target replaceability refers to a target replaceability calculated by using the initial replaceability.

Specifically, the server performs weighted calculation on the initial replaceability by using the training weight to obtain the initial target replaceability.

Step 908, determining whether the initial target alternative is maximized, executing step 910a when the initial target alternative is maximized, executing step 910b when the initial target alternative is not maximized, and returning to step 904 for execution.

Step 910a, a text processing model is obtained.

And step 910b, when the initial target replaceability is not maximized, updating the initial text processing model based on the initial target replaceability to obtain an updated text processing model, taking the updated text processing model as the initial text processing model, and returning to the step of inputting the training samples into the initial text processing model to carry out the steps until the initial target replaceability is maximized.

The step of judging whether the initial target replaceability is maximized refers to judging whether the total likelihood error corresponding to the initial target replaceability is maximized.

Specifically, the server may determine whether the initial target substitutability is maximized using equation (7) as shown below.

Formula (7)

Where W represents a training weight.

Representing an initial target degree of replaceability. When the initial target replaceability is not maximized, updating the initial text processing model based on the initial target replaceability, wherein the initial text processing model can be updated reversely by using a gradient descent method to obtain an updated text processing model, and other optimization algorithms can be used to update the initial text processing model, such as an AdaGrad (Adaptive learning rate) algorithm, an Adam (Adaptive moment estimation) algorithm, and the like. And taking the updated text processing model as an initial text processing model, returning the step of inputting the training sample into the initial text processing model, and carrying out execution until the initial target replaceability is maximized, and taking the model with the maximized initial target replaceability as the text processing model obtained by training.

The text processing model training method comprises the steps of obtaining a training sample, inputting the training sample into an initial text processing model, performing semantic extraction on an input text by the initial text processing model to obtain an initial forward semantic vector and an initial reverse semantic vector, calculating to obtain an initial context semantic vector of a training replacement position based on a replacement position by using the initial forward semantic vector and the initial reverse semantic vector, performing replacement prediction by using the initial context semantic vector of the training replacement position and an initial training unit vector to obtain an initial replaceability, calculating to obtain an initial target replaceability based on the initial replaceability and a training weight, obtaining the text processing model when the initial target replaceability is maximized, and improving the accuracy of the replacement prediction due to the fact that the replacement position context semantic vector is used for replacement prediction, and then when the initial target replaceability is maximized, obtaining the text processing model, thereby improving the accuracy of the trained text processing model.

In one embodiment, the training samples comprise training replacement samples and non-replacement samples, the training replacement samples comprise training texts, training replacement information and replacement weights, the training replacement information comprises training units to be replaced and training replacement positions, the non-replacement samples comprise the training texts, the non-replacement information and the non-replacement weights, and the non-replacement information comprises the non-replacement units and the training replacement positions;

step S904, inputting the training sample into the initial text processing model, including

And inputting the training replacement samples and the non-replacement samples into the initial text processing model to obtain a first initial replaceability corresponding to the output training replacement samples and a second initial replaceability corresponding to the non-replacement samples.

The training text refers to an input text used in training. The replacement weight is used for representing a weight for replacing the training text under the condition of the unit to be replaced, namely the weight corresponding to the training replacement text. The non-replacement information is used for indicating that the word corresponding to the unit to be replaced of the training text is replaced by the word, namely, the word is kept unchanged. The non-replaced weight is used for representing the weight of the training text which is kept unchanged under the condition that the unit is to be replaced, namely the weight corresponding to the training text. An un-replaced element refers to an element of text that replaces a location in the training text.

Specifically, the server inputs training replacement samples and non-replacement samples into an initial text processing model, the initial text processing model carries out initial vectorization on the training texts, the units to be replaced and the non-replacement units to obtain initial text vectors, initial unit vectors to be replaced and initial unit vectors to be replaced, forward semantic extraction and reverse semantic extraction are carried out on the basis of the initial text vectors to obtain initial forward semantic vectors and initial reverse semantic vectors, target initial forward semantic vectors and target initial reverse semantic vectors are determined from the initial forward semantic vectors and the initial reverse semantic vectors on the basis of the training replacement positions, initial context semantic calculation is carried out on the basis of the target initial forward semantic vectors and the target initial reverse semantic vectors to obtain initial context semantic vectors of the training replacement positions, and initial replacement prediction is carried out on the basis of the initial context vectors of the training replacement positions and the initial unit vectors to be replaced, and obtaining a first initial replaceability corresponding to the training replacement sample, and performing initial replacement prediction based on the initial context semantic vector of the training replacement position and the initial un-replaced word vector to obtain a second initial replaceability corresponding to the un-replaced sample.

Step 906, calculating to obtain an initial target alternative degree based on the initial alternative degree and the training weight, and obtaining a text processing model when the initial target alternative degree is maximized, wherein the method comprises the following steps:

The first initial target replaceability refers to the replaceability obtained by weighting and calculating the first initial replaceability by using the replacement weight. The second initial target replaceability is the replaceability obtained by weighting and calculating the second initial replaceability by using the non-replacement weight.

Specifically, the server calculates the product of the first initial replaceability and the replacement weight to obtain the first initial target replaceability. And calculating the product of the second initial replaceability and the un-replaced weight to obtain a second initial target replaceability, then adding the first initial target replaceability and the second initial target replaceability to obtain a specific initial target replaceability, and when the specific initial target replaceability is maximized, obtaining a text processing model. In one embodiment, the text processing model is derived when the calculated first initial target alternative and the second initial target alternative are both maximized.

In the above embodiment, the text processing model is trained by using the replacement samples and the non-replacement samples, so that the trained text processing model can improve the accuracy of the replacement prediction.

In one embodiment, as shown in FIG. 10, step 902, obtaining training samples comprises:

step 1002, obtaining a training corpus, and performing sentence-level division on the training corpus to obtain training sentences.

The training corpus refers to a corpus from which training samples can be extracted. For example, it may be a long text, a search log, a microblog log, a query log, etc. The training sentence refers to sentence text used when the training sample is extracted.

Specifically, the server collects the training corpus from the internet, and then splits the training corpus, for example, sentence-level division is performed on the training corpus to obtain each training sentence.

And 1004, determining a replacement training matching pair according to a preset replacement relation based on the training sentence, wherein the replacement training matching pair comprises a training input sentence and a corresponding training replacement sentence.

The replacing training matching pair is used for representing a pair of texts capable of being replaced and comprises a training input sentence and a corresponding training replacing sentence. The training input sentence refers to the training sentence itself and to the input text used in training. The training replacement sentence is a replacement text obtained by replacing words in the training sentence with units to be replaced. The preset replacement relationship refers to a preset replacement relationship, and includes synonym replacement, template replacement, regular expression replacement and the like.

Specifically, the server extracts all possible alternative ways for each training sentence to obtain each alternative training matching pair. For example, the training input sentence is q, the training alternative sentence is q', and the alternative is (q, x, y, B, L). Where x refers to a word in the training input sentence and y refers to a word that replaces the training input sentence x. B denotes the starting position of x in the training input sentence and L denotes the length of the word x in the training input sentence. For example, for a string q = "wife of XX president", if "wife" is allowed to be replaced by "husband", then the replacement information for q is S = { (x = wife, y = husband, B =6, L =2) }.

Step 1006, counting the occurrence frequency of the input sentence of the training input sentence in the training corpus, and calculating the retention weight based on the occurrence frequency of the input sentence.

Specifically, the server counts the number of occurrences of an input sentence of a training input sentence in the corpus, and then calculates the retention weight using the number of occurrences of the input sentence. Wherein the holding weight can be calculated using equation (8) shown below.

Formula (8)

Wherein V represents the number of occurrences of the input sentence,

refers to saving the weights.

And step 1008, counting the occurrence times of the replacement sentences of the training replacement sentences in the training corpus, and calculating replacement weights based on the occurrence times of the replacement sentences.

Specifically, the server counts the number of occurrences of the replacement sentence of the training replacement sentence in the corpus, and calculates the replacement weight using the number of occurrences of the replacement sentence, wherein the replacement weight may be calculated using formula (9) described below.

Formula (9)

Wherein the content of the first and second substances,

it is referred to as the replacement weight,

refers to the number of occurrences of the replacement sentence.

The attenuation coefficient is set in advance.

And step 1010, obtaining a retention sample based on the training input sentence and the retention weight, and obtaining a training replacement sample based on the training replacement sentence and the replacement weight.

Specifically, the server obtains a retention sample using the training input sentence, the retention information, and the retention weight, and obtains a replacement sample using the training replacement sentence, the replacement information, and the replacement weight. For example, the retained sample obtained may be

The replacement sample may be

Wherein q refers to the training input sentence, x refers to the word in q, and y refers to the word replacing q. B refers to the starting position of the unit to be replaced, and L refers to the length of the unit to be replaced. Wherein when the number of occurrences of the replacement sentence is greater than the number of occurrences of the input sentence, the input sentence is apt to be replaced with the replacement sentence. And because the attenuation coefficient limits the replacement weight, only when the occurrence frequency of the replacement sentence is more than that of the input sentence, the probability that the input sentence is changed into q is possibly higher than the probability that the input sentence is not replaced, thereby reducing the interference of low-frequency noise data during training the model and avoiding the false high probability of long-tail replacement. In a specific embodiment, the "health code" 0.5 × ln v =5 "is calculated, and the" key health code "ln v' =0.69, and meanwhile, we have a replacement relationship of" key health "to" health ", and then the following two training samples are obtained:

in the embodiment, the text processing model obtained by training is more accurate by obtaining the holding sample and the replacing sample, obtaining the training sample through the holding sample and the replacing sample, and then obtaining the text processing model through training the training sample.

In one embodiment, as shown in fig. 11, a searching method is provided, which is described by taking the method as an example applied to the terminal in fig. 1, and includes the following steps:

step 1102, receiving a search instruction, wherein the search instruction carries a search text.

Specifically, the search instruction refers to an instruction to perform a search using a search text, and the search text may be a search word or a search sentence. The search text can be a text input by a user, the search text can also be a text clicked by the user through a click operation, the search text is a text of an associated network address, a text terminal of the user clicking the associated network address receives a search instruction, the search instruction carries the search text, and then the search instruction is sent to the associated network address. The network address may be an address of a server.

Step 1104, sending the search text to a server according to the search instruction, the server receiving the search text, finding corresponding replacement information based on the search text, the replacement information including a unit to be replaced and a replacement position, vectorizing the search text to obtain a search text vector, and vectorizing the unit to be replaced to obtain a unit vector to be replaced; forward semantic extraction and reverse semantic extraction are carried out on the basis of the search text vector to obtain a forward semantic vector and a reverse semantic vector, and a target forward semantic vector and a target reverse semantic vector are determined from the forward semantic vector and the reverse semantic vector on the basis of the replacement position; performing context semantic calculation based on the target forward semantic vector and the target reverse semantic vector to obtain a context semantic vector of the replacement position; and performing replacement prediction on the context semantic vector based on the replacement position and the vector of the unit to be replaced to obtain a replaceability, replacing the text unit corresponding to the replacement position in the search text by using the unit to be replaced based on the replaceability to obtain a replacement text, and searching for a corresponding search result based on the replacement text and the search text.

Specifically, the terminal sends the search text to the server according to the search instruction, the server receives the text to be searched, corresponding replacement information is searched in a database based on the search text, and the database has various replacement information obtained after replacement is performed by using a replacement mode. For example, the server may traverse each word in the search text, search for a unit to be replaced corresponding to each word, and then obtain replacement information according to the unit to be replaced and the replacement position. Vectorizing the search text and the unit to be replaced to obtain a search text vector and a unit vector to be replaced; forward semantic extraction and reverse semantic extraction are carried out on the basis of the search text vector to obtain a forward semantic vector and a reverse semantic vector, and a target forward semantic vector and a target reverse semantic vector are determined from the forward semantic vector and the reverse semantic vector on the basis of the replacement position; performing context semantic calculation based on the target forward semantic vector and the target reverse semantic vector to obtain a context semantic vector of the replacement position; and replacing and predicting the context semantic vector and the unit vector to be replaced based on the replacement position to obtain a replaceability, and replacing the text unit corresponding to the replacement position in the search text by using the unit to be replaced based on the replaceability to obtain a replacement text. In one embodiment, the server may directly input the search text and the corresponding replacement information into the text processing model for replacement prediction, so as to obtain the output replaceability. In an embodiment, the server may use the text processing method in any of the above embodiments to perform replacement prediction based on the search text and the corresponding replacement information, so as to obtain the degree of replaceability.

And the server searches the corresponding search result by using the replacement text and the search text under the condition that the replaceability exceeds the prediction replaceability threshold value, and searches the corresponding search result by using the search text under the condition that the replaceability does not exceed the prediction replaceability threshold value. And then returning the searched search result to the terminal. s

And step 1106, receiving the search result returned by the server, and displaying the search result.

Specifically, the terminal receives the search result returned by the server, and may display the search result, for example, the search result may be displayed through a web page, may be displayed in a voice prompt manner, and may be displayed through various manners such as a list and a video.

In the searching method, when a searching instruction is received, the searching text is sent to the server, then the server determines whether to replace the searching text by calculating the replaceability of the searching text and the replacing information, and then the replacing text and the searching text can be used for searching the searching result together, so that the obtained searching result is more accurate.

In a specific embodiment, as shown in fig. 12, a text processing method is provided, which specifically includes the following steps:

step 1202, an input text and corresponding replacement information are acquired, the replacement information comprises a unit to be replaced and a replacement position, and the input text and the replacement information are input into a coding network of a text processing model.

Step 1204, the coding network performs word level division on the input text to obtain each input word, vectorizes each input word to obtain each input word vector, performs word level division on the unit to be replaced to obtain each word to be replaced, vectorizes each word to be replaced to obtain each word vector to be replaced.

And 1206, the coding network acquires a preset initial hidden vector, determines a current word vector from the text vector according to the sequence from the beginning to the end, and performs forward hidden state calculation based on the preset initial hidden vector and the current word vector to obtain a current word forward semantic vector corresponding to the current word vector.

And 1208, taking the forward semantic vector of the current word as a preset initial hidden vector by the coding network, and returning to the step of determining the current word vector from the text vector in sequence from the beginning to the end for iterative computation until the forward semantic vector corresponding to each word vector in the text vector is obtained.

In step 1210, the coding network determines a target word forward vector before the replacement position from each word vector, and takes a forward semantic vector corresponding to the target word forward vector as a first forward semantic vector. And determining a target position based on the initial position of the unit to be replaced and the length of the unit to be replaced, and determining a semantic vector before the target position from the forward semantic vectors to obtain a second forward semantic vector.

Step 1212, the coding network obtains a preset termination hidden vector, and determines a current word reverse vector from the text vector according to a sequence from termination to start; and performing reverse hidden state calculation based on the preset termination hidden vector and the current word reverse vector to obtain a current word reverse semantic vector corresponding to the current word reverse vector.

Step 1214, the coding network uses the reverse semantic vector of the current word as a preset termination hidden vector, and returns to the step of determining the reverse vector of the current word from the text vector in sequence from termination to start for iterative computation until the reverse semantic vector corresponding to each word vector in the text vector is obtained.

Step 1216, the coding network determines a target word reverse vector corresponding to the initial position of the unit to be replaced from each word vector, takes the reverse semantic vector corresponding to the target word reverse vector as a first reverse semantic vector, and determines a semantic vector behind the target position from the reverse semantic vectors to obtain a second reverse semantic vector.

In step 1218, the coding network calculates a weighted sum based on the first forward semantic vector, the second forward semantic vector, the first reverse semantic vector, and the second reverse semantic vector to obtain a replacement location context semantic vector.

Step 1220, inputting the context semantic vector of the replacement position and the unit vector to be replaced into a decoding network, acquiring a preset target hidden vector by the decoding network, and determining a current word vector to be replaced from the unit vector to be replaced according to the sequence from the beginning to the end; calculating a current word hidden vector to be replaced corresponding to the current word vector to be replaced based on a preset target hidden vector, a context semantic vector of a replacement position and the current word vector to be replaced; and performing regression calculation based on the hidden vector of the current word to be replaced to obtain the replacement probability of the current word to be replaced corresponding to the vector of the current word to be replaced.

Step 1222, the decoding network uses the hidden vector of the word to be replaced as a preset target hidden vector, and returns to the step of determining the current word vector to be replaced from the unit vector to be replaced according to the sequence from the beginning to the end, and the iteration is performed until the replacement probability of the word to be replaced corresponding to each word vector to be replaced is obtained.

Step 1224, performing a cumulative multiplication calculation based on the word replacement probabilities corresponding to the word vectors to be replaced to obtain the degree of replacement. And replacing the text unit corresponding to the replacement position in the input text by using the unit to be replaced based on the replaceability to obtain a replacement text.

The application also provides an application scene, and the application scene applies the text processing method. Specifically, the application of the text processing method in the application scenario is as follows:

as shown in fig. 13, for the search interface schematic diagram, a user inputs text from an input box of the search interface, the input text may be "who XXX wife is", and when the terminal receives a search instruction, the input text may be "who XXX wife is" and is sent to the server. The server receives the input text, finds out that the unit to be replaced corresponding to the 'XXX wife' is 'lady' and the replacement position comprises that the start position of the unit to be replaced is 3 (the start position of the unit to be replaced starts from 0) and the length of the unit to be replaced is 2, and inputs the 'XXX wife' and the replacement information into a text processing model, as shown in fig. 14, the text processing model is a schematic diagram for performing replacement prediction. That is, the forward semantic vector of each word is calculated forward by the encoder, and the reverse semantic vector of each word is calculated backward. Then based on the replacement information, the context semantic vector of the replacement position is calculated by using the corresponding forward semantic vector and reverse semantic vector, and then the decoder calculates the replacement probability of 'Fu' based on the context semantic vector of the replacement position and the vector of the unit to be replaced

And replacement probability of "person

The method comprises the steps of multiplying the replacement probability of 'husband' and the replacement probability of 'person' to obtain the replaceability, obtaining a replacement text 'XXX frogman is who' when the replaceability exceeds a replaceability threshold value, searching by using 'XXX frogman is who' and 'XXX wife is' to obtain a search result, sending the search result to a terminal, and displaying the search result by the terminal.

The application further provides an application scenario applying the text processing method. Specifically, the application of the text processing method in the application scenario is as follows:

in the process of judging synonyms in the text, obtaining the text needing synonym judgment, inputting the text needing synonym judgment and the synonyms into a text processing model for replacement prediction to obtain the replaceability, and when the replaceability reaches a preset synonym threshold value, obtaining the judgment result that the synonyms and the words in the text are the synonyms.

It should be understood that although the various steps in the flow charts of fig. 2-12 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2-12 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed in turn or alternately with other steps or at least some of the other steps.

In one embodiment, as shown in fig. 15, there is provided a text processing apparatus 1500, which may be a part of a computer device using software modules or hardware modules, or a combination of the two, and specifically includes: a text acquisition module 1502, a vectorization module 1504, a semantic vector determination module 1506, a calculation module 1508, and an alternative prediction module 1510, wherein:

a text obtaining module 1502, configured to obtain an input text and corresponding replacement information, where the replacement information includes a unit to be replaced and a replacement position;

the vectorization module 1504 is used for vectorizing the input text to obtain a text vector, and vectorizing the unit to be replaced to obtain a unit vector to be replaced;

the semantic vector determining module 1506 is used for performing forward semantic extraction and reverse semantic extraction on the basis of the text vector to obtain a forward semantic vector and a reverse semantic vector, and determining a target forward semantic vector and a target reverse semantic vector from the forward semantic vector and the reverse semantic vector on the basis of the replacement position;

a calculating module 1508, configured to perform context semantic calculation based on the target forward semantic vector and the target reverse semantic vector to obtain a context semantic vector of the replacement position;

and the replacement prediction module 1510 is configured to perform replacement prediction on the context semantic vector of the replacement position and the unit vector to be replaced, so as to obtain a degree of replaceability, where the degree of replaceability is used to represent a possibility that a text unit corresponding to the replacement position in the input text is replaced by the unit to be replaced.

In one embodiment, the text processing apparatus 1500 further includes:

the model prediction module is used for inputting an input text and replacement information into the text processing model, the text processing model vectorizes the input text to obtain a text vector, vectorizes a unit to be replaced to obtain a unit to be replaced, performs forward semantic extraction and reverse semantic extraction based on the text vector to obtain a forward semantic vector and a reverse semantic vector, and determines a target forward semantic vector and a target reverse semantic vector from the forward semantic vector and the reverse semantic vector based on a replacement position; performing context semantic calculation based on the target forward semantic vector and the target reverse semantic vector to obtain a context semantic vector of a replacement position, and performing replacement prediction based on the context semantic vector of the replacement position and a unit vector to be replaced to obtain a replaceability; the text processing model is obtained by training by using a training text and corresponding training replacement information based on a neural network algorithm;

and replacing the text unit corresponding to the replacement position in the input text by using the unit to be replaced based on the replaceability to obtain a replacement text.

In one embodiment, the text processing model includes an encoding network and a decoding network; a model prediction module comprising:

the encoding network vectorizes the input text to obtain a text vector, vectorizes the unit to be replaced to obtain a unit vector to be replaced, performs forward semantic extraction and reverse semantic extraction based on the text vector to obtain a forward semantic vector and a reverse semantic vector, determines a target forward semantic vector and a target reverse semantic vector from the forward semantic vector and the reverse semantic vector based on a replacement position, and performs context semantic calculation based on the target forward semantic vector and the target reverse semantic vector to obtain a context semantic vector of the replacement position;

and the decoding unit is used for inputting the context semantic vector of the replacement position and the vector of the unit to be replaced into a decoding network, and the decoding network carries out replacement prediction based on the context semantic vector of the replacement position and the vector of the unit to be replaced to obtain the replaceability.

In one embodiment, the vectorization module 1504 is further configured to perform word level division on the input text to obtain each input word, and vectorize each input word to obtain each input word vector; and carrying out word level division on the unit to be replaced to obtain each word to be replaced, and vectorizing each word to be replaced to obtain each word vector to be replaced.

a semantic vector determination module 1506, comprising:

the first forward vector determining unit is used for extracting forward semantics based on the text vector to obtain a forward semantic vector, and determining a semantic vector before the initial position of the unit to be replaced from the forward semantic vector to obtain a first forward semantic vector;

the second forward direction vector determining unit is used for determining a target position based on the initial position of the unit to be replaced and the length of the unit to be replaced, and determining a semantic vector before the target position from the forward direction semantic vectors to obtain a second forward direction semantic vector;

the first reverse vector determining unit is used for performing reverse semantic extraction based on the text vector to obtain a reverse semantic vector, and determining a semantic vector corresponding to the initial position of the unit to be replaced from the reverse semantic vector to obtain a first reverse semantic vector;

the second reverse vector determining unit is used for determining a semantic vector behind the target position from the reverse semantic vectors to obtain a second reverse semantic vector;

the calculating module 1508 is further configured to calculate a weighted sum based on the first forward semantic vector, the second forward semantic vector, the first reverse semantic vector, and the second reverse semantic vector, resulting in a context semantic vector of the alternative location.

In one embodiment, the first forward vector determining unit is further configured to obtain a preset starting hidden vector, and determine a current word vector from the text vectors in an order from start to end; performing forward hidden state calculation based on a preset initial hidden vector and a current word vector to obtain a current word forward semantic vector corresponding to the current word vector; taking the forward semantic vector of the current word as a preset initial hidden vector, and returning to the step of determining the current word vector from the text vector in sequence from the beginning to the end for iterative computation until the forward semantic vector corresponding to each word vector in the text vector is obtained; and determining a target word forward vector before the replacement position from each word vector, and taking a forward semantic vector corresponding to the target word forward vector as a first forward semantic vector.

In one embodiment, the first reverse vector determining unit is further configured to obtain a preset termination hidden vector, and determine a current word reverse vector from the text vector according to an order from termination to start; performing reverse hidden state calculation based on a preset termination hidden vector and a current word reverse vector to obtain a current word reverse semantic vector corresponding to the current word reverse vector; taking the reverse semantic vector of the current word as a preset termination hidden vector, and returning to the step of determining the reverse vector of the current word from the text vector in sequence from termination to start for iterative computation until the reverse semantic vector corresponding to each word vector in the text vector is obtained; and determining a target word reverse vector corresponding to the initial position of the unit to be replaced from each word vector, and taking the reverse semantic vector corresponding to the target word reverse vector as a first reverse semantic vector.

In one embodiment, the replacement prediction module 1510 includes:

and the replacement degree calculation module is used for performing hidden state calculation based on the context semantic vector of the replacement position and the unit vector to be replaced to obtain a hidden vector to be replaced, and performing regression calculation based on the hidden vector to be replaced to obtain the degree of replacement.

In one embodiment, the replacement degree calculation module is further configured to obtain a preset target hidden vector, and determine a current word vector to be replaced from the unit vectors to be replaced according to a sequence from start to end; calculating a current word hidden vector to be replaced corresponding to the current word vector to be replaced based on a preset target hidden vector, a context semantic vector of a replacement position and the current word vector to be replaced; performing regression calculation based on the hidden vector of the current word to be replaced to obtain the replacement probability of the current word to be replaced corresponding to the vector of the current word to be replaced; taking the hidden vector of the current word to be replaced as a preset target hidden vector, and returning to the step of determining the vector of the current word to be replaced from the vector of the unit to be replaced according to the sequence from the beginning to the end for iterative execution until the replacement probability of the word to be replaced corresponding to each vector of the word to be replaced is obtained; and performing cumulative multiplication calculation based on the replacement probability of the words to be replaced corresponding to each word vector to be replaced to obtain the degree of replacement.

A text processing model training apparatus 1600, as shown in fig. 16, comprising:

a sample obtaining module 1602, configured to obtain a training sample, where the training sample includes a training text, training information, and training weights, and the training information includes training units and training replacement positions;

an initial model calculation module 1604, configured to input training samples into an initial text processing model, where the initial text processing model performs initial vectorization on the input text to obtain an initial text vector, and performs initial vectorization on the training units to obtain an initial training unit vector, performs forward semantic extraction and reverse semantic extraction based on the initial text vector to obtain an initial forward semantic vector and an initial reverse semantic vector, determines a target initial forward semantic vector and a target initial reverse semantic vector from the initial forward semantic vector and the initial reverse semantic vector based on a training replacement position, performs initial context semantic calculation based on the target initial forward semantic vector and the target initial reverse semantic vector to obtain an initial context semantic vector of the training replacement position, and performs initial replacement prediction based on the initial context semantic vector of the training replacement position and the initial training unit vector, an initial degree of replaceability is obtained.

And the iteration module 1606 is configured to calculate an initial target alternative based on the initial alternative and the training weight, and obtain the text processing model when the initial target alternative is maximized.

In one embodiment, the iteration module 1606 is further configured to update the initial text processing model based on the initial target replaceability when the initial target replaceability is not maximized, to obtain an updated text processing model, to use the updated text processing model as the initial text processing model, and to return to the step of inputting the training samples into the initial text processing model to carry out until the initial target replaceability is maximized.

In one embodiment, the training samples comprise training replacement samples and non-replacement samples, the training replacement samples comprise training texts, training replacement information and replacement weights, the training replacement information comprises training units to be replaced and training replacement positions, the non-replacement samples comprise the training texts, the non-replacement information and the non-replacement weights, and the non-replacement information comprises non-replacement words and the training replacement positions;

the initial model calculation module 1604 is further configured to input the training replacement sample and the un-replaced sample into the initial text processing model, and obtain a first initial substitutability corresponding to the output training replacement sample and a second initial substitutability corresponding to the un-replaced sample;

the iteration module 1606 is further configured to calculate a first initial target alternative based on the first initial alternative and the replacement weight, calculate a second initial target alternative based on the second initial alternative and the non-replacement weight, obtain a specific initial target alternative based on the first initial target alternative and the second initial target alternative, and obtain the text processing model when the specific initial target alternative is maximized.

In an embodiment, the sample obtaining module 1602 is further configured to obtain a training corpus, and perform sentence-level division on the training corpus to obtain a training sentence; determining a replacement training matching pair according to a preset replacement relation based on the training sentence, wherein the replacement training matching pair comprises a training input sentence and a corresponding training replacement sentence; counting the occurrence times of input sentences of training input sentences in the training corpus, and calculating a retention weight based on the occurrence times of the input sentences; counting the occurrence times of the replacement sentences of the training replacement sentences in the training corpus, and calculating replacement weights based on the occurrence times of the replacement sentences; and obtaining a retention sample based on the training input sentence and the retention weight, and obtaining a training replacement sample based on the training replacement sentence and the replacement weight.

A search apparatus 1700, as shown in fig. 17, the apparatus comprising:

an instruction receiving module 1702, configured to receive a search instruction, where the search instruction carries a search text;

a search module 1704, configured to send the search text to a server according to the search instruction, where the server receives the search text, searches for corresponding replacement information based on the search text, where the replacement information includes a unit to be replaced and a replacement position, vectorizes the search text to obtain a search text vector, and vectorizes the unit to be replaced to obtain a unit vector to be replaced; forward semantic extraction and reverse semantic extraction are carried out on the basis of the search text vector to obtain a forward semantic vector and a reverse semantic vector, and a target forward semantic vector and a target reverse semantic vector are determined from the forward semantic vector and the reverse semantic vector on the basis of the replacement position; performing context semantic calculation based on the target forward semantic vector and the target reverse semantic vector to obtain a context semantic vector of the replacement position; performing replacement prediction on the context semantic vector of the replacement position and the vector of the unit to be replaced to obtain a replaceability, replacing the text unit corresponding to the replacement position in the search text by using the unit to be replaced based on the replaceability to obtain a replacement text, and searching a corresponding search result based on the replacement text and the search text;

the display module 1706 is configured to receive the search result returned by the server, and display the search result.

For specific limitations of the text processing apparatus, the text processing model training apparatus, and the search apparatus, reference may be made to the above limitations of the text processing method, the text processing model training method, and the search method, which are not described herein again. The modules in the text processing device, the text processing model training device and the searching device can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 18. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The computer device may be a node on a blockchain. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used to store replacement information and training samples. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a text processing method and a text processing model training method.

In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 19. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, an operator network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a search method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.

It will be appreciated by those skilled in the art that the configurations shown in fig. 18 and 19 are block diagrams of only some of the configurations relevant to the present disclosure, and do not constitute a limitation on the computing devices to which the present disclosure may be applied, and that a particular computing device may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, a computer device is further provided, which includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above method embodiments when executing the computer program.

In an embodiment, a computer-readable storage medium is provided, in which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned method embodiments.

In one embodiment, a computer program product or computer program is provided that includes computer instructions stored in a computer-readable storage medium. The computer instructions are read by a processor of a computer device from a computer-readable storage medium, and the computer instructions are executed by the processor to cause the computer device to perform the steps in the above-mentioned method embodiments.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. A method of text processing, the method comprising:

vectorizing the input text to obtain a text vector, and vectorizing the unit to be replaced to obtain a unit vector to be replaced;

performing forward semantic extraction and reverse semantic extraction on the basis of the text vector to obtain a forward semantic vector and a reverse semantic vector, and determining a target forward semantic vector and a target reverse semantic vector from the forward semantic vector and the reverse semantic vector on the basis of the replacement position;

and performing replacement prediction based on the context semantic vector of the replacement position and the unit vector to be replaced to obtain a replaceability, wherein the replaceability is used for representing the possibility that the text unit corresponding to the replacement position in the input text is replaced by the unit to be replaced.

2. The method of claim 1, further comprising:

inputting the input text and the replacement information into a text processing model, vectorizing the input text by the text processing model to obtain a text vector, vectorizing the unit to be replaced to obtain a unit vector to be replaced, performing forward semantic extraction and reverse semantic extraction based on the text vector to obtain a forward semantic vector and a reverse semantic vector, and determining a target forward semantic vector and a target reverse semantic vector from the forward semantic vector and the reverse semantic vector based on the replacement position; performing context semantic calculation based on the target forward semantic vector and the target reverse semantic vector to obtain a context semantic vector of the replacement position, and performing replacement prediction based on the context semantic vector of the replacement position and the unit vector to be replaced to obtain a replaceability; the text processing model is obtained by training by using a training text and corresponding training replacement information based on a neural network algorithm;

3. The method of claim 2, wherein the text processing model comprises an encoding network and a decoding network;

the inputting the input text and the replacement information into a text processing model includes:

inputting the input text and the replacement information into the coding network, vectorizing the input text by the coding network to obtain a text vector, vectorizing the unit to be replaced to obtain a unit vector to be replaced, performing forward semantic extraction and reverse semantic extraction based on the text vector to obtain a forward semantic vector and a reverse semantic vector, determining a target forward semantic vector and a target reverse semantic vector from the forward semantic vector and the reverse semantic vector based on the replacement position, and performing context semantic calculation based on the target forward semantic vector and the target reverse semantic vector to obtain a context semantic vector of the replacement position;

and inputting the context semantic vector of the replacement position and the unit vector to be replaced into the decoding network, and performing replacement prediction by the decoding network based on the context semantic vector of the replacement position and the unit vector to be replaced to obtain the replaceability.

4. The method according to any one of claims 1-3, wherein the replacement position comprises a start position of the unit to be replaced and a length of the unit to be replaced;

the method comprises the steps of performing forward semantic extraction and reverse semantic extraction on the basis of the text vector to obtain a forward semantic vector and a reverse semantic vector, and determining a target forward semantic vector and a target reverse semantic vector from the forward semantic vector and the reverse semantic vector on the basis of the replacement position, and comprises the following steps:

performing forward semantic extraction based on the text vector to obtain a forward semantic vector, and determining a semantic vector before the initial position of the unit to be replaced from the forward semantic vector to obtain a first forward semantic vector;

determining a target position based on the initial position of the unit to be replaced and the length of the unit to be replaced, and determining a semantic vector corresponding to the target position from the forward semantic vectors to obtain a second forward semantic vector;

performing reverse semantic extraction based on the text vector to obtain a reverse semantic vector, and determining a semantic vector corresponding to the initial position of the unit to be replaced from the reverse semantic vector to obtain a first reverse semantic vector;

determining a semantic vector behind the target position from the reverse semantic vectors to obtain a second reverse semantic vector;

the performing context semantic calculation based on the target forward semantic vector and the target reverse semantic vector to obtain the context semantic vector of the replacement position includes:

calculating a weighted sum based on the first forward semantic vector, the second forward semantic vector, the first reverse semantic vector, and the second reverse semantic vector to obtain a context semantic vector of the replacement location.

5. The method according to claim 4, wherein the performing forward semantic extraction based on the text vector to obtain the forward semantic vector, and determining a semantic vector before a start position of the unit to be replaced from the forward semantic vector to obtain a first forward semantic vector comprises:

acquiring a preset starting hidden vector, and determining a current word vector from text vectors according to the sequence from the beginning to the end;

performing forward hidden state calculation based on the preset initial hidden vector and the current word vector to obtain a current word forward semantic vector corresponding to the current word vector;

taking the forward semantic vector of the current word as a preset initial hidden vector, and returning to the step of iterative computation of sequentially determining the current word vector from the text vector according to the sequence from the beginning to the end until the forward semantic vector corresponding to each word vector in the text vector is obtained;

and determining a target word forward vector before the replacement position from the word vectors, and taking a forward semantic vector corresponding to the target word forward vector as a first forward semantic vector.

6. The method according to any one of claims 1 to 3, wherein the performing replacement prediction based on the replacement location context semantic vector and the unit to be replaced vector to obtain a degree of replacement comprises:

and performing hidden state calculation based on the context semantic vector of the replacement position and the unit vector to be replaced to obtain a hidden vector to be replaced, and performing regression calculation based on the hidden vector to be replaced to obtain the degree of replacement.

7. The method of claim 6, wherein the performing hidden state computation based on the replacement position context semantic vector and the to-be-replaced unit vector to obtain to-be-replaced hidden vectors, and performing regression computation based on the to-be-replaced hidden vectors to obtain the degree of replacement comprises:

acquiring a preset target hidden vector, and determining a current word vector to be replaced from the unit vector to be replaced according to the sequence from the beginning to the end;

calculating a current word hidden vector to be replaced corresponding to the current word vector to be replaced based on the preset target hidden vector, the context semantic vector of the replacement position and the current word vector to be replaced;

performing regression calculation based on the hidden vector of the current word to be replaced to obtain the replacement probability of the current word to be replaced corresponding to the vector of the current word to be replaced;

taking the hidden vector of the current word to be replaced as a preset target hidden vector, and returning to the step of determining the vector of the current word to be replaced from the vector of the unit to be replaced according to the sequence from the beginning to the end for iterative execution until the replacement probability of the word to be replaced corresponding to each vector of the word to be replaced is obtained;

and performing cumulative multiplication calculation based on the replacement probability of the words to be replaced corresponding to each word vector to be replaced to obtain the degree of replacement.

8. A method for training a text processing model, the method comprising:

inputting the training samples into an initial text processing model, performing initial vectorization on the input text by the initial text processing model to obtain an initial text vector, performing initial vectorization on the training units to obtain an initial training unit vector, performing forward semantic extraction and reverse semantic extraction on the basis of the initial text vector to obtain an initial forward semantic vector and an initial reverse semantic vector, determining a target initial forward semantic vector and a target initial reverse semantic vector from the initial forward semantic vector and the initial reverse semantic vector on the basis of the training replacement positions, performing initial context semantic calculation on the basis of the target initial forward semantic vector and the target initial reverse semantic vector to obtain an initial context semantic vector of the training replacement positions, and performing initial replacement prediction on the basis of the initial context semantic vector of the training replacement positions and the initial training unit vector, obtaining an initial degree of replacement;

and calculating to obtain an initial target alternative degree based on the initial alternative degree and the training weight, and obtaining a text processing model when the initial target alternative degree is maximized.

9. The method of claim 8, wherein the obtaining training samples comprises:

obtaining a training corpus, and carrying out sentence-level division on the training corpus to obtain training sentences;

determining a replacement training matching pair according to a preset replacement relation based on the training sentence, wherein the replacement training matching pair comprises a training input sentence and a corresponding training replacement sentence;

counting the occurrence times of input sentences of the training input sentences in the training corpus, and calculating a retention weight based on the occurrence times of the input sentences;

counting the occurrence times of the replacement sentences of the training replacement sentences in the training corpus, and calculating replacement weights based on the occurrence times of the replacement sentences;

and obtaining a retention sample based on the training input sentence and the retention weight, and obtaining a training replacement sample based on the training replacement sentence and the replacement weight.

10. A method of searching, the method comprising:

sending the search text to a server according to the search instruction, wherein the server receives the search text, searches corresponding replacement information based on the search text, the replacement information comprises a unit to be replaced and a replacement position, vectorizes the search text to obtain a search text vector, and vectorizes the unit to be replaced to obtain a unit vector to be replaced; forward semantic extraction and reverse semantic extraction are carried out on the basis of the search text vector to obtain a forward semantic vector and a reverse semantic vector, and a target forward semantic vector and a target reverse semantic vector are determined from the forward semantic vector and the reverse semantic vector on the basis of the replacement position; performing context semantic calculation based on the target forward semantic vector and the target reverse semantic vector to obtain a context semantic vector of the replacement position; performing replacement prediction on the context semantic vector of the replacement position and the vector of the unit to be replaced to obtain a replaceability, replacing a text unit corresponding to the replacement position in the search text by using the unit to be replaced on the basis of the replaceability to obtain a replacement text, and searching a corresponding search result on the basis of the replacement text and the search text;

and receiving the search result returned by the server, and displaying the search result.

11. A text processing apparatus, characterized in that the apparatus comprises:

the system comprises a text acquisition module, a text processing module and a text processing module, wherein the text acquisition module is used for acquiring an input text and corresponding replacement information, and the replacement information comprises a unit to be replaced and a replacement position;

12. A text processing model training apparatus, the apparatus comprising:

an initial model calculation module, configured to input the training sample into an initial text processing model, where the initial text processing model performs initial vectorization on the input text to obtain an initial text vector, performs initial vectorization on the training unit to obtain an initial training unit vector, performs forward semantic extraction and reverse semantic extraction based on the initial text vector to obtain an initial forward semantic vector and an initial reverse semantic vector, determines a target initial forward semantic vector and a target initial reverse semantic vector from the initial forward semantic vector and the initial reverse semantic vector based on the training replacement position, performs initial context semantic calculation based on the target initial forward semantic vector and the target initial reverse semantic vector to obtain an initial context semantic vector of the training replacement position, and performs initial replacement based on the initial context semantic vector of the training replacement position and the initial training unit vector Changing the prediction to obtain the initial degree of replacement

And the iteration module is used for calculating to obtain an initial target replaceability based on the initial replaceability and the training weight, and obtaining a text processing model when the initial target replaceability is maximized.

13. A search apparatus, characterized in that the apparatus comprises:

the search module is used for sending the search text to a server according to the search instruction, the server receives the search text, searches corresponding replacement information based on the search text, the replacement information comprises a unit to be replaced and a replacement position, vectorizes the search text to obtain a search text vector, and vectorizes the unit to be replaced to obtain a unit vector to be replaced; forward semantic extraction and reverse semantic extraction are carried out on the basis of the search text vector to obtain a forward semantic vector and a reverse semantic vector, and a target forward semantic vector and a target reverse semantic vector are determined from the forward semantic vector and the reverse semantic vector on the basis of the replacement position; performing context semantic calculation based on the target forward semantic vector and the target reverse semantic vector to obtain a context semantic vector of the replacement position; performing replacement prediction on the context semantic vector of the replacement position and the vector of the unit to be replaced to obtain a replaceability, replacing a text unit corresponding to the replacement position in the search text by using the unit to be replaced on the basis of the replaceability to obtain a replacement text, and searching a corresponding search result on the basis of the replacement text and the search text;

14. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor realizes the steps of the method of any one of claims 1 to 10 when executing the computer program.

15. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 10.