CN113297833A

CN113297833A - Text error correction method and device, terminal equipment and computer storage medium

Info

Publication number: CN113297833A
Application number: CN202010110410.7A
Authority: CN
Inventors: 姚林霞; 孟函可; 祝官文
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2020-02-21
Filing date: 2020-02-21
Publication date: 2021-08-24
Also published as: WO2021164310A1

Abstract

The application is applicable to the technical field of artificial intelligence, and provides a text error correction method, a text error correction device, terminal equipment and a computer storage medium. In the text error correction method of the application, before decoding, a decoder in an encoder-decoder model needs to perform label classification on each input word vector by using an error correction judgment model to obtain an error correction label of each input word vector. The error correction label is used for indicating whether the corresponding word needs to be corrected or not. After the terminal equipment obtains the error correction labels corresponding to the input word vectors in the input text, the error correction labels corresponding to the input word vectors are input into the decoder, so that the decoder can perform targeted decoding according to the error correction labels corresponding to the input word vectors, the decoding process is regulated and controlled, the misjudgment condition of the decoder is reduced, the accuracy of text error correction is improved, and the problems that the decoding process of the current encoder-decoder model is uncontrollable and the misjudgment condition is easily generated are solved.

Description

Text error correction method and device, terminal equipment and computer storage medium

Technical Field

The present application belongs to the technical field of artificial intelligence, and in particular, to a text error correction method, apparatus, terminal device, and computer storage medium.

Background

In the current field of text processing, a coder-decoder model is commonly used to implement text processing functions such as text correction, text translation, document extraction, question and answer systems, and the like.

In the encoder-decoder model, an encoder and a decoder are provided. When text error correction is carried out, a user can input a text needing error correction into an encoder of an encoder-decoder model, the encoder converts the text input by the user into a semantic vector, then the encoder transmits the semantic vector into a decoder of the encoder-decoder model, and the decoder decodes the semantic vector to obtain the text after error correction and outputs the text to the user.

However, in the current encoder-decoder model, the decoding process of the model is uncontrollable, and misjudgment is easily caused, and some correct words may be misjudged as incorrect words for error correction, or some incorrect words may be misjudged as correct words without error correction.

Disclosure of Invention

The embodiment of the application provides a text error correction method, a text error correction device, terminal equipment and a computer storage medium, and can solve the problems that the decoding process of the current encoder-decoder model is uncontrollable and misjudgment is easy to generate.

A first aspect of an embodiment of the present application provides a text error correction method, including:

the method comprises the steps that terminal equipment carries out word vector conversion on an input text to obtain a word vector sequence corresponding to the input text, wherein the word vector sequence comprises input word vectors corresponding to all words in the input text;

the terminal equipment inputs the word vector sequence into an encoder of an encoder-decoder model to obtain a semantic vector;

the terminal equipment inputs the word vector sequence into an error correction judgment model to obtain an error correction label corresponding to each input word vector;

and the terminal equipment inputs the word vector sequence, the semantic vectors and the error correction labels corresponding to the input word vectors into a decoder of the encoder-decoder model to obtain the text after error correction.

Before decoding, the terminal device inputs the word vector sequence into an error correction decision model for error correction decision to obtain an error correction tag corresponding to each word in the input text. The error correction label is used for indicating whether each word in the input text needs to be corrected or not.

In the decoding process, the decoder can perform targeted decoding according to the error correction label of each word in the input text, and the decoding process is regulated, so that the misjudgment condition of the decoder is reduced, and the accuracy of text error correction is improved.

In a possible implementation manner of the first aspect, the inputting, by the terminal device, the word vector sequence, the semantic vector, and the error correction tag corresponding to each input word vector into a decoder of the encoder-decoder model, and obtaining an error-corrected text includes:

the terminal equipment sequentially inputs the input word vectors in the word vector sequence into a decoder of the coder-decoder model;

after the input word vector is input into the decoder every time, the terminal device calculates an attention vector corresponding to the input word vector and a second hidden layer vector corresponding to a next input word vector according to the input word vector and the second hidden layer vector corresponding to the input word vector, wherein the second hidden layer vector is a hidden layer vector of the decoder, and the semantic vector is a second hidden layer vector corresponding to a first input word vector;

if the error correction label corresponding to the input word vector is the first label, the terminal device controls the decoder to take the word corresponding to the input word vector as the decoded word corresponding to the input word vector, wherein the error correction label comprises the first label and the second label;

if the error correction label corresponding to the input word vector is a second label, the terminal device constructs a first vector according to the error correction label corresponding to the input word vector, the attention vector corresponding to the input word vector and a second hidden layer vector corresponding to the input word vector;

the terminal equipment carries out similarity calculation on the first vector and second vectors corresponding to all words in a preset dictionary to obtain first similarities corresponding to all the words in the preset dictionary;

the terminal equipment determines a decoding word corresponding to the input word vector according to the first similarity;

and the terminal equipment determines the text after error correction according to the decoding words corresponding to the input word vectors.

It should be noted that, when the decoder decodes using the similarity comparison method, the computational complexity of decoding can be reduced, the loss of system performance can be reduced, and the processing time can be reduced.

In a possible implementation manner of the first aspect, the determining, by the terminal device, a decoded word corresponding to the input word vector according to the first similarity includes:

and the terminal equipment takes the word with the highest first similarity in the preset dictionary as the decoding word corresponding to the input word vector.

It should be noted that the terminal device may directly use the word with the highest first similarity in the preset dictionary as the decoded word corresponding to the input word vector, so as to reduce the complexity of decoding calculation.

In another possible implementation manner of the first aspect, the input word vector includes a pinyin word vector and a font word vector;

correspondingly, the determining, by the terminal device, the decoded word corresponding to the input word vector according to the first similarity includes:

the terminal equipment carries out similarity calculation on the pinyin word vectors in the input word vectors and the pinyin word vectors corresponding to all the words in the preset dictionary to obtain the pinyin similarity corresponding to all the words in the preset dictionary;

the terminal equipment carries out similarity calculation on the font word vector in the input word vector and the font word vector corresponding to each word in the preset dictionary to obtain the font similarity corresponding to each word in the preset dictionary;

the terminal equipment calculates the editing distance between the words corresponding to the input word vectors and each word in the preset dictionary to obtain the editing distance corresponding to each word in the preset dictionary;

the terminal equipment respectively carries out weighted summation on the first similarity, the pinyin similarity, the font similarity and the editing distance corresponding to each word in the preset dictionary to obtain the target similarity corresponding to each word in the preset dictionary;

and the terminal equipment takes the word with the highest target similarity in the preset dictionary as the decoding word corresponding to the input word vector.

It should be noted that, when the terminal device needs to improve the accuracy of decoding, the first similarity, the pinyin similarity, the font similarity, the edit distance and other domain knowledge may be combined to perform comprehensive evaluation to obtain the target similarity.

And then, the terminal equipment takes the word with the highest target similarity in the preset dictionary as the decoding word corresponding to the input word vector, so that the decoding accuracy of the decoder is improved.

In one possible implementation manner of the first aspect, the error correction decision model includes a bidirectional coding characterization model and a classifier;

correspondingly, the step of inputting the word vector sequence into an error correction judgment model by the terminal device to obtain an error correction label corresponding to each input word vector includes:

the terminal equipment sequentially inputs each input word vector in the word vector sequence into an error correction judgment model to obtain a first output value corresponding to each input word vector;

and the terminal equipment respectively inputs the first output values corresponding to the input word vectors into the two classifiers to obtain the error correction labels corresponding to the input word vectors.

It should be noted that the bidirectional coding representation model has the advantages of high accuracy, convenient use, high adjustment speed and the like, and the construction difficulty and the training difficulty of the error correction judgment model can be reduced by using the bidirectional coding representation model and the two classifiers.

A second aspect of an embodiment of the present application provides a text error correction apparatus, including:

the embedded module is used for performing word vector conversion on an input text to obtain a word vector sequence corresponding to the input text, wherein the word vector sequence comprises input word vectors corresponding to all words in the input text;

the semantic module is used for inputting the word vector sequence into an encoder of an encoder-decoder model to obtain a semantic vector;

the label module is used for inputting the word vector sequence into an error correction judgment model to obtain an error correction label corresponding to each input word vector;

and the error correction module is used for inputting the word vector sequence, the semantic vectors and the error correction labels corresponding to the input word vectors into a decoder of the encoder-decoder model to obtain an error-corrected text.

In a possible implementation manner of the second aspect, the error correction module includes:

a vector input submodule for sequentially inputting the input word vectors in the word vector sequence into a decoder of the encoder-decoder model;

a hidden update sub-module, configured to calculate, after the input word vector is input into the decoder each time, an attention vector corresponding to the input word vector and a second hidden layer vector corresponding to a next input word vector according to the input word vector and the second hidden layer vector corresponding to the input word vector, where the second hidden layer vector is a hidden layer vector of the decoder, and the semantic vector is a second hidden layer vector corresponding to a first input word vector;

a first output sub-module, configured to control the decoder to use a word corresponding to the input word vector as a decoded word corresponding to the input word vector if an error correction tag corresponding to the input word vector is a first tag, where the error correction tag includes a first tag and a second tag;

the first vector quantity module is used for constructing a first vector according to the error correction label corresponding to the input word vector, the attention vector corresponding to the input word vector and the second hidden layer vector corresponding to the input word vector if the error correction label corresponding to the input word vector is the second label;

the first calculation submodule is used for calculating the similarity of the first vector and a second vector corresponding to each word in a preset dictionary to obtain a first similarity corresponding to each word in the preset dictionary;

the second output submodule is used for determining a decoding word corresponding to the input word vector according to the first similarity;

and the text integration sub-module is used for determining the text after error correction according to the decoding words corresponding to the input word vectors.

In a possible implementation manner of the second aspect, the second output sub-module is specifically configured to use a word with a highest first similarity in the preset dictionary as a decoded word corresponding to the input word vector.

In another possible implementation manner of the second aspect, the input word vector includes a pinyin word vector and a font word vector;

correspondingly, the second output submodule includes:

the second calculation submodule is used for carrying out similarity calculation on the pinyin word vectors in the input word vectors and the pinyin word vectors corresponding to all the words in the preset dictionary to obtain the pinyin similarity corresponding to all the words in the preset dictionary;

the third computation submodule is used for carrying out similarity calculation on the font word vector in the input word vector and the font word vector corresponding to each word in the preset dictionary to obtain the font similarity corresponding to each word in the preset dictionary;

the fourth calculation submodule is used for calculating the editing distance between the words corresponding to the input word vectors and each word in the preset dictionary to obtain the editing distance corresponding to each word in the preset dictionary;

the target calculation submodule is used for respectively carrying out weighted summation on the first similarity, the pinyin similarity, the font similarity and the editing distance corresponding to each word in the preset dictionary to obtain the target similarity corresponding to each word in the preset dictionary;

and the target output submodule is used for taking the word with the highest target similarity in the preset dictionary as the decoding word corresponding to the input word vector.

In one possible implementation manner of the second aspect, the error correction decision model includes a bidirectional coding characterization model and a classifier;

correspondingly, the label module comprises:

the pre-error correction sub-module is used for sequentially inputting each input word vector in the word vector sequence into the bidirectional coding representation model to obtain a first output value corresponding to each input word vector;

and the label classification submodule is used for respectively inputting the first output values corresponding to the input word vectors into the two classifiers to obtain the error correction labels corresponding to the input word vectors.

A third aspect of the embodiments of the present application provides a terminal device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the terminal device is enabled to implement the steps of the method as described above.

A fourth aspect of embodiments of the present application provides a computer-readable storage medium, which stores a computer program that, when executed by a processor, causes a terminal device to implement the steps of the method as described above.

A fifth aspect of embodiments of the present application provides a computer program product, which, when run on a terminal device, causes the terminal device to implement the steps of the method as described above.

Compared with the prior art, the embodiment of the application has the advantages that:

the embodiment of the application provides a text error correction method, wherein before a decoder in an encoder-decoder model decodes, an error correction judgment model is required to perform label classification on each input word vector to obtain an error correction label of each input word vector. The error correction label is used for indicating whether the corresponding word needs to be corrected or not. After the terminal equipment obtains the error correction labels corresponding to the input word vectors in the input text, the error correction labels corresponding to the input word vectors are input into the decoder, so that the decoder can perform targeted decoding according to the error correction labels corresponding to the input word vectors, the decoding process is regulated and controlled, the misjudgment condition of the decoder is reduced, the accuracy of text error correction is improved, and the problems that the decoding process of the current encoder-decoder model is uncontrollable and the misjudgment condition is easily generated are solved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

Fig. 1 is a schematic flowchart of a text error correction method according to an embodiment of the present application;

FIG. 2 is a schematic structural diagram of a text correction system according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a word vector embedding model provided in an embodiment of the present application;

FIG. 4 is a schematic diagram of an encoder provided in an embodiment of the present application;

FIG. 5 is a schematic diagram of an operation of an error correction decision model provided in an embodiment of the present application;

FIG. 6 is a diagram of a predetermined dictionary provided in an embodiment of the present application;

fig. 7 is a schematic structural diagram of a text error correction apparatus according to an embodiment of the present application;

fig. 8 is a schematic diagram of a terminal device provided in an embodiment of the present application.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It should also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to" determining "or" in response to detecting ". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".

Furthermore, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used for distinguishing between descriptions and not necessarily for describing or implying relative importance.

Reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather "one or more but not all embodiments" unless specifically stated otherwise. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless expressly specified otherwise.

The text error correction method provided by the embodiment of the application can be applied to terminal devices such as a mobile phone, a tablet personal computer, a wearable device, a vehicle-mounted device, an Augmented Reality (AR)/Virtual Reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a Personal Digital Assistant (PDA), and the like, and the embodiment of the application does not limit the specific type of the terminal device at all.

For example, the terminal device may be a Station (ST) in a WLAN, which may be a cellular phone, a cordless phone, a Session Initiation Protocol (SIP) phone, a Wireless Local Loop (WLL) station, a Personal Digital Assistant (PDA) device, a handheld device with Wireless communication capability, a computing device or other processing device connected to a Wireless modem, a vehicle-mounted device, a vehicle-mounted networking terminal, a computer, a laptop, a handheld communication device, a handheld computing device, a satellite Wireless device, a Wireless modem card, a television set-top box (STB), a Customer Premises Equipment (CPE), and/or other devices for communicating over a Wireless system and a next generation communication system, such as a Mobile terminal in a 5G Network or a Public Land Mobile Network (future evolved, PLMN) mobile terminals in the network, etc.

In the encoder-decoder model, an encoder and a decoder are provided. When text error correction is carried out, a user can input a text needing error correction into an encoder of an encoder-decoder model, the encoder converts the text input by the user into a semantic vector, then the encoder transmits the semantic vector to a decoder of the encoder-decoder model, and the decoder decodes the text according to the semantic vector and the input text to obtain the text after error correction and outputs the text to the user.

In view of this, embodiments of the present application provide a text error correction method, before a decoder in an encoder-decoder model performs decoding, a label classification needs to be performed on each input word vector by using an error correction decision model to obtain an error correction label of each input word vector. The error correction label is used for indicating whether the corresponding word needs to be corrected or not. After the terminal equipment obtains the error correction labels corresponding to the input word vectors in the input text, the error correction labels corresponding to the input word vectors are input into the decoder, so that the decoder can perform targeted decoding according to the error correction labels corresponding to the input word vectors, the decoding process is regulated and controlled, the misjudgment condition of the decoder is reduced, the accuracy of text error correction is improved, and the problems that the decoding process of the current encoder-decoder model is uncontrollable and the misjudgment condition is easily generated are solved.

Next, the contents of the text error correction method improved by the present embodiment will be described from the viewpoint of the terminal device. Please refer to the flowchart of the text error correction method described in fig. 1, which includes:

s101, performing word vector conversion on an input text by terminal equipment to obtain a word vector sequence corresponding to the input text, wherein the word vector sequence comprises input word vectors corresponding to all words in the input text;

referring to the system diagram of the text correction system shown in fig. 2, a word vector embedding model 201, a coder-decoder model 202, and an error correction decision model 203 may be included in the text correction system.

The word vector embedding model 201 is configured to perform word vector conversion on an input text, and this process may also be referred to as word vector embedding (embedding), and converts the input text from a natural language into an input word vector of a first preset length.

The type of word vector embedding model 201 may be selected according to the actual situation. For example, assuming that the input text is a chinese text, the terminal device may select any one or a combination of plural kinds of models such as a pinyin word vector model, a font word vector model, and an n-gram language model as the word vector embedding model 201.

Here, the language model (language model) is a model for calculating the probability of one sentence. The language model has wide application in the fields of machine translation, Chinese word segmentation, grammar analysis and the like. The language model mainly used by people at present is an n-gram language model. n in the n-element language model is a preset numerical value. In the n-gram language model, a word is most related to only its first n-1 words. When n is 1, the n-gram language model indicates that each word in the sentence is independent of the preceding words. When n is 2, it means that a word is related to only one word preceding it. When n is 3, it means that a word is related to only the two words preceding it.

Further, the word vector embedding model 201 may be a single model, or the word vector embedding model 201 may be a combination of a plurality of models.

For example, when the terminal device selects a combination of three models, namely a pinyin word vector model, a font word vector model, and an n-ary language model, as the word vector model, the input text may be input into the pinyin word vector model, the font word vector model, and the n-ary language model, respectively, to obtain a pinyin word vector, a font word vector, and a context word vector corresponding to each word in the input text. Then, the terminal equipment respectively splices the pinyin word vector, the font word vector and the context word vector corresponding to each word to obtain the input word vector of each word in the input text.

When the input word vector is obtained by combining various word vectors, the knowledge and the field characteristics in different fields can be fully utilized, and the accuracy of text error correction is improved.

After determining the word vector embedding model 201, the terminal device may embed the input text word vector in the model 201 to obtain a word vector sequence corresponding to the input text. The word vector sequence includes input word vectors for each word in the input text.

It should be understood that the words (tokens) mentioned in the above description may be defined according to the language type of the input text and the content pre-configured by the user. For example, in a chinese text, the above-mentioned word may be a word, or a combination of multiple words; in the english text, the above-mentioned words may be one word or a phrase consisting of a plurality of words. The present embodiment does not limit the definition of words.

S102, the terminal equipment inputs the word vector sequence into an encoder 2021 of an encoder-decoder model 202 to obtain a semantic vector;

the encoder-decoder model 202 is a model applied to Sequence-to-Sequence (Seq 2Seq) questions, and can be applied to the field of text processing such as text translation, document extraction, question-answering system, and the like. The inputs and outputs of the encoder-decoder model 202 represent different meanings in different text processing domains. For example, in the field of text translation, the input to the encoder-decoder model 202 is the text to be translated, and the output of the encoder-decoder model 202 is the translated text; in the field of question-answering systems, the input to the encoder-decoder model 202 is a question and the output of the encoder-decoder model 202 is an answer.

In the encoder-decoder model 202, an encoder 2021 and a decoder 2022 are provided. The encoder 2021 is configured to convert the input sequence into a fixed-length vector, and the decoder 2022 is configured to convert the fixed-length vector generated by the encoder 2021 into the output sequence.

The type of encoder 2021 and the type of decoder 2022 may be selected as appropriate.

For example, the type of the encoder 2021 and the type of the decoder 2022 may be any one of a Recurrent Neural Network (RNN), a Long Short-Term Memory (LSTM) model, a Gated Recurrent Unit (GRU) model, a Text convolution Neural Network (TextCNN) model, a transform (transformer) model, and the like.

Also, the type of the encoder 2021 may be the same as that of the decoder 2022, or the type of the encoder 2021 may not be the same as that of the decoder 2022.

For example, the type of encoder 2021 and the type of decoder 2022 may both be LSTM models. Alternatively, the encoder 2021 may be of the LSTM type and the decoder 2022 may be of the transform type.

After obtaining the word vector sequence of the input text, the terminal device may sequentially input the input word vectors in the word vector sequence into the encoder 2021 of the encoder-decoder model 202 to obtain semantic vectors.

In the process that the terminal device inputs the input word vectors in the word vector sequence to the encoder 2021 in sequence, each time an input word vector is input, the encoder 2021 updates the current first hidden layer vector according to the input word vector to obtain a new first hidden layer vector. The first hidden layer vector is the hidden layer vector of the encoder 2021. After the terminal device inputs the last input word vector into the encoder 2021, the encoder 2021 updates the first hidden layer vector for the last time according to the last input word vector to obtain a semantic vector.

S103, the terminal equipment inputs the word vector sequence into an error correction judgment model 203 to obtain an error correction label corresponding to each input word vector;

in the text error correction scheme of the existing encoder-decoder model 202, after the encoder 2021 encodes the word vector sequence to obtain the semantic vector, the terminal device inputs the semantic vector output by the encoder 2021 and the word vector sequence into the decoder 2022, and the decoder 2022 performs decoding operation according to the semantic vector and the word vector sequence to output the text after error correction.

However, in this decoding method, the decoding process is not controllable, i.e. each word in the input text, whether it is the correct word or not, may be corrected. Therefore, in the current encoder-decoder model 202, the decoding process is not controllable, erroneous judgment is easily caused, and some words not requiring error correction may be subjected to error correction or some words requiring error correction may not be subjected to error correction.

In contrast, in the error correction method of this embodiment, before the decoder 2022 performs decoding, the word vector sequence is input into the error correction determination model 203 to perform error correction determination, so as to obtain an error correction label corresponding to each word in the input text.

The error correction determination model 203 is configured to identify whether each word in the input text is a correct word, so as to determine which words in the input text need to be corrected and which words do not need to be corrected, and obtain an error correction label corresponding to each word in the input text.

The error correction label is used for indicating whether each word in the input text needs to be corrected or not, and the error correction label may include a first label and a second label, where the first label indicates that no error correction is needed, and the second label indicates that error correction is needed. The form of the error correction label can be set according to actual conditions. For example, in some embodiments, a first tag may be represented by a 0 and a second tag may be represented by a 1.

The structure of the error correction determination model 203 may be set according to actual circumstances. In some embodiments, a combination of a Bidirectional encoding representation (Bert) model and a two-classifier may be used as the error correction decision model 203.

The terminal device may sequentially input the input word vectors in the word vector sequence into the Bert model to obtain first output values corresponding to the input word vectors. And then, the terminal equipment inputs the first output values corresponding to the input word vectors into the two classifiers respectively for classification, and the error correction labels corresponding to the input word vectors in the input text are obtained. And the first output value of the Bert model and the input word vector are in one-to-one correspondence relationship.

The Bert model has the advantages of high accuracy, convenience in use, high adjusting speed and the like, and when the Bert model and the classifier are used for constructing the error correction judgment model 203, the construction difficulty and the training difficulty of the error correction judgment model 203 can be reduced.

In other embodiments, other models may be selected as the error correction decision model 203. For example, the terminal device may select a RNN model, an LSTM model, a GRU model, a TextCNN model, a transform model, or the like to construct the error correction determination model 203. The specific structure of the error correction determination model 203 may be set according to actual conditions.

S104, the terminal device inputs the word vector sequence, the semantic vectors and the error correction labels corresponding to the input word vectors into the decoder 2022 of the encoder-decoder model 202 to obtain an error-corrected text.

After obtaining the error correction label of each word in the input text, the terminal device may input the word vector sequence, the semantic vector, and the error correction label of each word in the input text into the decoder 2022 of the encoder-decoder model 202, and may perform decoding by the decoder 2022.

In the decoding process, the decoder 2022 can perform targeted decoding according to the error correction label of each word in the input text, and regulate the decoding process, thereby reducing the misjudgment condition of the decoder and improving the accuracy of text error correction.

In addition, the decoder 2022 usually performs decoding using softmax function during decoding, and this decoding method will occupy more computational performance of the system and take a long processing time. Therefore, in some possible implementations, the decoder 2022 may perform decoding by using a similarity comparison method, so as to reduce the computational complexity of decoding, reduce the loss of system performance, and reduce the processing time.

In the decoding process, the terminal device may use the above semantic vector as an initial value of a second hidden layer vector of the decoder 2022, where the semantic vector is the second hidden layer vector corresponding to the first input word vector. The second concealment layer vector is the concealment layer vector of the decoder 2022.

The terminal device sequentially inputs the input word vector in the word vector sequence to the decoder 2022, and calculates the attention vector corresponding to the input word vector and the second hidden layer vector corresponding to the next input word vector according to the input word vector and the second hidden layer vector corresponding to the input word vector.

If the error correction tag corresponding to the input word vector is the first tag, which indicates that the word corresponding to the input word vector does not need to be corrected, the terminal device controls the decoder 2022 to use the word corresponding to the input word vector as the decoded word corresponding to the input word vector, where the decoded word is the output value of the decoder.

If the error correction label corresponding to the input word vector is the second label, which indicates that the word corresponding to the input word vector needs to be corrected, the terminal device constructs the first vector according to the error correction label corresponding to the input word vector, the attention vector corresponding to the input word vector and the second hidden layer vector corresponding to the input word vector. The construction mode of the first vector can be set according to actual conditions. For example, in some embodiments, the error correction label corresponding to the input word vector, the attention vector corresponding to the input word vector, and the second hidden layer vector corresponding to the input word vector may be directly spliced into the first vector.

And the terminal equipment calculates the similarity of the first vector and the second vector corresponding to each word in the preset dictionary to obtain the first similarity corresponding to each word in the preset dictionary, and determines the decoding word corresponding to the input word vector according to the first similarity.

In some possible implementations, the terminal device may directly control the decoder 2022 to determine the word with the highest first similarity in the preset dictionary as the decoded word corresponding to the input word vector.

In other possible implementation manners, in order to improve the accuracy of decoding, the terminal device may further perform comprehensive comparison with knowledge in other fields to obtain a target similarity, and determine a decoded word corresponding to the input word vector according to the target similarity. The other domain knowledge may include one or more of pinyin similarity, font similarity, edit distance, and other domain knowledge.

For example, taking Chinese text as an example, assume that the input word vectors include a Pinyin word vector and a glyph word vector.

The terminal equipment can carry out similarity calculation according to the pinyin word vector in the input word vector and the pinyin word vector corresponding to each word in the preset dictionary to obtain the pinyin similarity corresponding to each word in the preset dictionary.

The terminal equipment can perform similarity calculation according to the font word vectors and the font word vectors corresponding to all words in the preset dictionary to obtain font similarity corresponding to all words in the preset dictionary.

And the terminal equipment calculates the Edit Distance (Edit Distance) between the words corresponding to the input word vector and each word in the preset dictionary to obtain the Edit Distance corresponding to each word in the preset dictionary.

The edit distance is the minimum number of edit operations between two character strings required to convert one character string into another character string. The application range of the edit distance is very wide, especially on similarity problems such as text error correction, plagiarism recognition and the like.

The editing in the edit distance includes three operations: insertion, deletion, and replacement. The edit distance between two strings is typically calculated using a dynamic programming algorithm.

And the terminal equipment performs weighted summation on the first similarity, the pinyin similarity, the font similarity and the editing distance corresponding to each word in the preset dictionary to obtain the target similarity corresponding to each word in the preset dictionary. Wherein, the first weight value corresponding to the first similarity, the second weight value corresponding to the pinyin similarity, the third weight value corresponding to the font similarity and the fourth weight value corresponding to the edit distance are preset values.

It is understood that, in the above-mentioned similarity calculation process, an appropriate similarity calculation method may be selected for calculation according to actual situations. For example, the similarity may be calculated by selecting a way of calculating the cosine distance; or, the similarity can be calculated by selecting a mode of calculating the Euclidean distance; alternatively, other similarity algorithms may be selected to calculate the similarity.

After the target similarity is obtained through calculation, the terminal device may determine a word with the highest target similarity in a preset dictionary as a decoded word corresponding to the input word vector.

And after the terminal equipment obtains the decoded words corresponding to the input word vectors, determining the text after error correction according to the decoded words corresponding to the input word vectors.

The text error correction method of the present embodiment is described below with reference to specific application scenarios:

it is assumed that the text error correction system includes an encoder-decoder model, an error correction decision model, and a word vector embedding model. In the encoder-decoder model, an encoder and a decoder are both LSTM models, an error correction judgment model is a Bert model followed by a classifier, and a word vector embedding model comprises a pinyin word vector model, a font word vector model and an n-element language model.

And before text correction, training the text correction system by using the training corpus. The corpus may include a tagged text corpus recognized by collected Automatic Speech Recognition (ASR), public data in a newspaper, a general entity vocabulary, and other public corpora, such as a Sighan Bakeoff corpus.

In the process of obtaining the corpus, the corpus may be preprocessed, including:

1.1, collecting and labeling corpora;

1.2, converting the traditional Chinese characters in the collected corpus into simplified Chinese characters;

1.3, converting the Chinese characters into pinyin;

and 1.4, corpus quality screening and corpus analysis statistics.

After the preprocessing process, the quality of the training corpus can be improved. And then, converting the training corpus into a training word vector through a word vector embedding model, and training the encoder, the decoder and the error correction judgment module by using the training word vector.

In each round of training, a first loss value corresponding to the encoder, a second loss value corresponding to the decoder and a third loss value corresponding to the error correction judging module are calculated, the encoder is iteratively updated according to the first loss value, the decoder is iteratively updated according to the second loss value, and the error correction judging module is iteratively updated according to the third loss value.

And repeating the training until a preset training stopping condition is reached. The training suspension condition can be set according to actual conditions. For example, the training suspension condition may be that the training number reaches a preset iteration number; or the training suspension condition may be that the first loss value, the second loss value and the third loss value are less than a preset loss threshold; alternatively, the training suspension condition may be another condition.

After training is completed, the text correction can be performed using the trained text correction system.

As shown in fig. 3, assuming that the input text is "song before listening to learning," a word is used as a word, and the input word vector of the input text is embedded in the model to obtain a word vector sequence corresponding to the error-corrected text, where the word vector sequence includes the input word vector corresponding to each word (i.e., word vector a1 to word vector a6 in fig. 3), and each input word vector is obtained by splicing a pinyin word vector, a font word vector, and a context word vector.

The terminal apparatus inputs the word vector a1 through the word vector a6 one by one into the encoder. Each time the input word vector is input to the encoder, the encoder updates the first hidden layer vector of the encoder according to the input word vector.

As shown in fig. 4, the initial value of the first hidden layer vector of the encoder is h 0. After the terminal equipment inputs the word vector A1 into the encoder, the encoder updates h0 to h1 according to the word vector A1; after the terminal equipment inputs the word vector A2 into the encoder, the encoder updates h1 to h2 according to the word vector A2; by analogy, after the terminal device inputs the word vector a6 into the encoder, the encoder updates h5 to h6 according to the word vector a 6. The terminal device takes h6 as a semantic vector and controls the encoder to input h6 into the decoder as the initial value s1 of the second hidden layer vector of the decoder.

As shown in fig. 5, the terminal device inputs the word vectors a1 through a6 into the Bert model in the error correction determination model one by one, the Bert model outputs the first output values corresponding to the word vectors a1 through a6, and the first output values are input into the two classifiers to obtain the error correction labels corresponding to the word vectors a1 through a 6. The error correction tag comprises a first tag and a second tag, wherein the value of the first tag is 0, and the value of the second tag is 1.

At this time, the error correction labels corresponding to the word vector a1, the word vector a5, and the word vector a6 are 0, which means that the words corresponding to the word vector a1, the word vector a5, and the word vector a6 do not need to be corrected; the error correction labels corresponding to the word vector a2, the word vector A3, and the word vector a4 are 1, which indicates that the words corresponding to the word vector a2, the word vector A3, and the word vector a4 need error correction.

Then, the terminal device inputs the word vectors a 1-a 6 and the error correction tags corresponding to the input word vectors into a decoder, so as to obtain decoded words corresponding to the input word vectors. The specific process is as follows:

2.1, the terminal equipment inputs the word vector A1 and the error correction label corresponding to the word vector A1 into a decoder, and the decoder updates s1 to s2 according to the word vector A1. Since the error correction flag corresponding to the word vector a1 is 0, the decoder outputs the word "listen" corresponding to the word vector a 1.

2.2, the terminal equipment inputs the word vector A2 and the error correction label corresponding to the word vector A2 into a decoder, and the decoder updates s2 to s3 according to the word vector A2. Since the error correction label corresponding to the word vector a2 is 1, the decoder calculates the attention vector b1 corresponding to the word vector a2 according to s2 and h1 to h6, and then constructs the first vector c1 corresponding to the word vector a2 according to the error correction label corresponding to the word vector a2, s2 and b 1.

As shown in fig. 6, it is assumed that m words exist in the preset dictionary, and m is a preset positive integer. Then there is a corresponding second vector for each word. For example, "snow" corresponds to second vector d1, "Xue" corresponds to second vector d2, "days" corresponds to second vector d3, "crystal" corresponds to second vector d4, and "split" corresponds to second vector dm.

And the terminal equipment calculates the similarity between the first vector c1 and the second vector of each word in the preset dictionary to obtain the first similarity corresponding to each word in the preset dictionary. At this time, the first vector c1 and the second vector d2 have the highest first similarity, and the decoder outputs the word "xu" corresponding to the second vector d 2.

2.3, the terminal equipment inputs the word vector A3 and the error correction label corresponding to the word vector A3 into a decoder, and the decoder updates s3 to s4 according to the word vector A3. Since the error correction label corresponding to the word vector A3 is 1, the decoder calculates the attention vector b2 corresponding to the word vector A3 according to s3 and h1 to h6, and then constructs the first vector c2 corresponding to the word vector A3 according to the error correction label corresponding to the word vector A3, s3 and b 2.

The terminal equipment calculates the similarity of the first vector c2 and the second vector of each word in the preset dictionary to obtain the first similarity corresponding to each word in the preset dictionary, and the decoder outputs the word 'it' with the highest first similarity in the preset dictionary.

2.4, the terminal equipment inputs the word vector A4 and the error correction label corresponding to the word vector A4 into a decoder, and the decoder updates s4 to s5 according to the word vector A4. Since the error correction label corresponding to the word vector a4 is 1, the decoder calculates the attention vector b3 corresponding to the word vector a4 according to s4 and h1 to h6, and then constructs the first vector c3 corresponding to the word vector a4 according to the error correction label corresponding to the word vector a4, s4 and b 3.

The terminal equipment calculates the similarity of the first vector c3 and the second vector of each word in the preset dictionary to obtain the first similarity corresponding to each word in the preset dictionary, and the decoder outputs the word 'pretty' with the highest first similarity in the preset dictionary.

2.5, the terminal equipment inputs the word vector A5 and the error correction label corresponding to the word vector A5 into a decoder, and the decoder updates s5 to s6 according to the word vector A5. Since the error correction flag corresponding to the word vector a5 is 0, the decoder outputs the word "of" corresponding to the word vector a 5.

2.6, the terminal equipment inputs the word vector A6 and the error correction label corresponding to the word vector A6 into a decoder, and the decoder updates s6 to s7 according to the word vector A6. Since the error correction flag corresponding to the word vector a6 is 0, the decoder outputs the word "song" corresponding to the word vector a 6.

After the terminal device obtains the decoding words "listen", "xue", "zhi", "humble", "singing" corresponding to the word vector a1 to the word vector a6, the decoding words are arranged in sequence to obtain the text after error correction, i.e. "listen to the humble song".

In summary, the embodiment of the present application provides a text error correction method, where before a decoder in an encoder-decoder model decodes, an error correction decision model needs to be used to perform label classification on each input word vector to obtain an error correction label of each input word vector. The error correction label is used for indicating whether the corresponding word needs to be corrected or not. After the terminal equipment obtains the error correction labels corresponding to the input word vectors in the input text, the error correction labels corresponding to the input word vectors are input into the decoder, so that the decoder can perform targeted decoding according to the error correction labels corresponding to the input word vectors, the decoding process is regulated and controlled, the misjudgment condition of the decoder is reduced, the accuracy of text error correction is improved, and the problems that the decoding process of the current encoder-decoder model is uncontrollable and the misjudgment condition is easily generated are solved.

After the input text is corrected by the text correction method, the corrected text can be obtained. The corrected text can be widely applied to various downstream tasks, such as word segmentation tasks, part-of-speech tagging tasks, entity recognition tasks, intention classification tasks, slot filling tasks, conversation management tasks, text generation tasks and the like.

It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

Referring to fig. 7, an embodiment of the present application provides a text correction apparatus, only a portion related to the present application is shown for convenience of description, and as shown in fig. 7, the text correction apparatus includes,

an embedding module 701, configured to perform word vector conversion on an input text to obtain a word vector sequence corresponding to the input text, where the word vector sequence includes an input word vector corresponding to each word in the input text;

a semantic module 702, configured to input the word vector sequence into an encoder of an encoder-decoder model to obtain a semantic vector;

a label module 703, configured to input the word vector sequence into an error correction decision model, to obtain an error correction label corresponding to each input word vector;

and an error correction module 704, configured to input the word vector sequence, the semantic vector, and the error correction tag corresponding to each input word vector into a decoder of the encoder-decoder model, so as to obtain an error-corrected text.

Further, the error correction module 704 comprises:

Further, the second output sub-module is specifically configured to use a word with the highest first similarity in the preset dictionary as a decoded word corresponding to the input word vector.

Further, the input word vector comprises a pinyin word vector and a font word vector;

correspondingly, the second output submodule includes:

Further, the error correction decision model comprises a bidirectional coding representation model and a classifier;

accordingly, the label module 703 includes:

It should be noted that, for the information interaction, execution process, and other contents between the above-mentioned devices/units, the specific functions and technical effects thereof are based on the same concept as those of the embodiment of the method of the present application, and specific reference may be made to the part of the embodiment of the method, which is not described herein again.

Referring to fig. 8, an embodiment of the present application further provides a terminal device, where the terminal device 8 includes: a processor 80, a memory 81 and a computer program 82 stored in said memory 81 and executable on said processor 80. The processor 80, when executing the computer program 82, implements the steps in the above-described embodiments of the number privacy protection method, such as the steps S101 to S104 shown in fig. 1. Alternatively, the processor 80, when executing the computer program 82, implements the functions of each module/unit in each device embodiment described above, for example, the functions of the modules 701 to 704 shown in fig. 7.

Illustratively, the computer program 82 may be partitioned into one or more modules/units that are stored in the memory 81 and executed by the processor 80 to accomplish the present application. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution of the computer program 82 in the terminal device 8. For example, the computer program 82 may be divided into an embedding module, a semantic module, a labeling module, and an error correction module, each of which functions specifically as follows:

The terminal device 8 may be a desktop computer, a notebook, a palm computer, a cloud server, or other computing devices. The terminal device may include, but is not limited to, a processor 80, a memory 81. Those skilled in the art will appreciate that fig. 8 is merely an example of a terminal device 8 and does not constitute a limitation of terminal device 8 and may include more or fewer components than shown, or some components may be combined, or different components, e.g., the terminal device may also include input-output devices, network access devices, buses, etc.

The Processor 80 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The storage 81 may be an internal storage unit of the terminal device 8, such as a hard disk or a memory of the terminal device 8. The memory 81 may also be an external storage device of the terminal device 8, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the terminal device 8. Further, the memory 81 may also include both an internal storage unit and an external storage device of the terminal device 8. The memory 81 is used for storing the computer program and other programs and data required by the terminal device. The memory 81 may also be used to temporarily store data that has been output or is to be output.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the above-described embodiments of the apparatus/terminal device are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow in the method of the embodiments described above can be realized by a computer program, which can be stored in a computer-readable storage medium and can realize the steps of the embodiments of the methods described above when the computer program is executed by a processor. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.

The above-mentioned embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.

Claims

1. A text error correction method, comprising:

2. The method as claimed in claim 1, wherein the step of inputting the word vector sequence, the semantic vector and the error correction tag corresponding to each input word vector into a decoder of the encoder-decoder model by the terminal device to obtain the error-corrected text comprises:

3. The text error correction method of claim 2, wherein the determining, by the terminal device, the decoded word corresponding to the input word vector according to the first similarity comprises:

4. The text error correction method of claim 2, wherein the input word vector includes a pinyin word vector and a font word vector;

5. A method of text error correction according to claim 1, wherein the error correction decision model comprises a bi-directional coding characterization model and a classifier;

6. A text correction apparatus, comprising:

7. The text correction apparatus of claim 6, wherein the correction module comprises:

8. The apparatus according to claim 7, wherein the second output sub-module is configured to use a word with the highest first similarity in the preset dictionary as the decoded word corresponding to the input word vector.

9. The text correction apparatus of claim 7, wherein the input word vector includes a pinyin word vector and a font word vector;

correspondingly, the second output submodule includes:

10. The apparatus for correcting text according to claim 6, wherein the correction decision model includes a bidirectional coding representation model and a classifier;

correspondingly, the label module comprises:

11. A terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, causes the terminal device to carry out the steps of the method according to any of claims 1 to 5.

12. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, causes a terminal device to carry out the steps of the method according to any one of claims 1 to 5.