WO2025016237A1 - 文本识别方法、装置、电子设备及可读存储介质 - Google Patents

文本识别方法、装置、电子设备及可读存储介质 Download PDF

Info

Publication number
WO2025016237A1
WO2025016237A1 PCT/CN2024/104175 CN2024104175W WO2025016237A1 WO 2025016237 A1 WO2025016237 A1 WO 2025016237A1 CN 2024104175 W CN2024104175 W CN 2024104175W WO 2025016237 A1 WO2025016237 A1 WO 2025016237A1
Authority
WO
WIPO (PCT)
Prior art keywords
text
recognized
character
pronunciation
polyphonetic
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2024/104175
Other languages
English (en)
French (fr)
Inventor
冉佳诺
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Vivo Mobile Communication Co Ltd
Original Assignee
Vivo Mobile Communication Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Vivo Mobile Communication Co Ltd filed Critical Vivo Mobile Communication Co Ltd
Publication of WO2025016237A1 publication Critical patent/WO2025016237A1/zh
Anticipated expiration legal-status Critical
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/10—Text processing
    • G06F40/12—Use of codes for handling textual entities
    • G06F40/126—Character encoding
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]

Definitions

  • the present application belongs to the field of natural language processing, and specifically relates to a text recognition method, device, electronic device and readable storage medium.
  • Speech synthesis is a technology that converts text into speech.
  • speech synthesis technology provides users with a new way to interact with information.
  • Chinese speech synthesis broadcast scenario because Chinese characters contain a large number of polyphones, speech synthesis may make mistakes when processing polyphones.
  • Text encoding refers to converting natural language text into a vector form for computer understanding and processing.
  • the character encoding method commonly used in text encoding is to encode word by word, and only consider the content of the current character.
  • the purpose of the embodiments of the present application is to provide a text recognition method, device, electronic device and readable storage medium, which can better utilize local semantic information and global semantic information to analyze the pronunciation of long and difficult sentences and complex texts, thereby improving the accuracy of predicting the pronunciation of polyphones in the text.
  • an embodiment of the present application provides a text recognition method, the method comprising: inputting a first text to be recognized into a first text encoding model, the first text to be recognized containing a first polyphonetic character, and the first text encoding model comprising a convolutional layer; extracting a first text feature sequence corresponding to the first text to be recognized; using a convolutional layer to perform segmented convolution processing on the first text feature sequence to obtain a second text feature sequence, the second text feature sequence at least containing global semantic information of the first text to be recognized and local semantic information of a first text segment in the first text to be recognized, the first text segment containing a first polyphonetic character; based on the second text feature sequence, obtaining first encoding information corresponding to the first text to be recognized, and obtaining a recognition result of the first text to be recognized based on the above-mentioned first encoding information.
  • an embodiment of the present application provides a text recognition device, the device comprising: a processing module and an extraction module, wherein: the processing module is used to input a first text to be recognized into a first text
  • the first text to be recognized contains a first polyphone
  • the first text encoding model includes a convolution layer
  • an extraction module is used to extract a first text feature sequence corresponding to the first text to be recognized
  • the processing module is also used to use the convolution layer to perform segmented convolution processing on the first text feature sequence to obtain a second text feature sequence, the second text feature sequence at least contains global semantic information of the first text to be recognized and local semantic information of a first text segment in the first text to be recognized, and the first text segment contains the first polyphone
  • the processing module is also used to obtain first encoding information corresponding to the first text to be recognized based on the second text feature sequence, and obtain a recognition result of the first text to be recognized based on the above-mentioned first encoding information
  • an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.
  • an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.
  • an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.
  • an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.
  • a text recognition device inputs a first text to be recognized into a first text encoding model, where the first text to be recognized contains a first polyphonetic character, and the first text encoding model includes a convolutional layer. Then, a first text feature sequence corresponding to the first text to be recognized is extracted, and the convolutional layer is used to perform segmented convolution processing on the above-mentioned first text feature sequence to obtain a second text feature sequence, where the above-mentioned second text feature sequence at least contains global semantic information of the first text to be recognized and local semantic information of a first text segment in the first text to be recognized, and the above-mentioned first text segment contains a first polyphonetic character.
  • first encoding information corresponding to the first text to be recognized is obtained, and based on the above-mentioned first encoding information, a recognition result of the first text to be recognized is obtained.
  • a text recognition device performs a convolution operation on the global text feature sequence of the text through the convolution layer in the first text encoding model to obtain a text feature sequence containing global semantic information and local semantic information, and obtains a more accurate recognition result based on the text feature sequence, thereby enhancing the ability to model the local information of the text sequence, and can make up for the deficiencies of the existing model in extracting local feature information, so as to more accurately analyze the pronunciation of polyphones in long and difficult sentences and complex texts, thereby improving the accuracy of pronunciation prediction of polyphones in the text.
  • FIG1 is a flow chart of a text recognition method provided in an embodiment of the present application.
  • FIG2 is a schematic diagram of the structure of the CNN layer in the improved DeBERTa model provided in an embodiment of the present application;
  • FIG3 is a schematic diagram of the structure of an improved DeBERTa model provided in an embodiment of the present application.
  • FIG4 is a flow chart of a text recognition method provided in an embodiment of the present application.
  • FIG5 is a schematic diagram of the structure of a text recognition device provided in an embodiment of the present application.
  • FIG6 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.
  • FIG. 7 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application.
  • first, second, etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first”, “second”, etc. are generally of one type, and the number of objects is not limited.
  • the first object can be one or more.
  • “and/or” in the specification and claims represents at least one of the connected objects, and the character “/" generally indicates that the objects associated with each other are in an "or” relationship.
  • Speech synthesis is a technology that converts text into speech.
  • speech synthesis technology provides users with a new way to interact with information. Compared with visual information interaction, in some specific scenarios, such as driving or sports, speech synthesis can convey information more conveniently and quickly in the form of sound.
  • speech synthesis helps users better understand information by simulating the intonation, tone and emotion of language, providing users with a more realistic interactive experience.
  • Text encoding refers to converting natural language text into a vector form for computer understanding and processing.
  • character encoding refers to encoding character by character and only considering the content of the current character.
  • the character encoding method has a relatively small amount of calculation and high efficiency, but in some complex and ambiguous scenarios, polyphone pronunciation prediction errors may occur due to the connection of the context.
  • Sentence encoding takes into account the context and semantics before and after, so it can more accurately predict the pronunciation of polyphones, but the sentence encoding method is slow to calculate and requires a large amount of corpus for training.
  • the existing sentence encoding method captures the contextual relationship in the sequence through a global self-attention mechanism. Each position of the input sequence can pay attention to the information of all other positions in the sequence, but it is also easily affected by other positions when processing local information, and it is difficult to extract fine-grained local features. In this way, sentence encoding is not effective for polyphones in complex texts such as long and difficult sentences. The accuracy of pronunciation prediction for Chinese characters is low.
  • this application proposes a text recognition method, which adds a convolution module, i.e., a convolution layer, to the DeBERTa (Decoding-enhanced BERT with disentangled attention) model, and extracts local feature information of the text to be recognized based on the global text feature sequence of the text to be recognized through the convolution layer, thereby enhancing the modeling ability of local information of the text sequence.
  • the convolution network obtains local features by performing convolution operations on sequence segments through local connections and convolution kernel weight sharing.
  • Adding a convolution module after the Transformer layer can make up for the shortcomings of the Transformer layer in extracting local features, so that local information and global information can be better used to analyze the pronunciation of long and difficult sentences and complex texts, thereby improving the accuracy of pronunciation prediction for polyphones in complex texts such as long and difficult sentences.
  • DeBERTa that is, BERT with decentralized decoding enhancement.
  • the DeBERTa model is an improved model based on the BERT (Bidirectional Encoder Representation from Transformers) model, BERT, that is, the encoding of the bidirectional Transformer.
  • the BERT model and the DeBERTa model are general new language models that provide support for different natural language processing (NLP) tasks.
  • NLP natural language processing
  • the encoder structure of the DeBERTa model is a multi-layer Transformer superposition. Transformer is a model widely used in text and time series modeling tasks. It models the entire sequence and captures the contextual relationship in the sequence through a global self-attention mechanism. Each position of the input sequence can pay attention to the information of all other positions in the sequence.
  • FIG. 1 is a flow chart of a text recognition method provided in an embodiment of the present application. As shown in FIG. 1 , the text recognition method may include the following steps S201 to S204:
  • Step S201 The text recognition device inputs a first text to be recognized into a first text encoding model.
  • the above-mentioned first text to be recognized includes a first polyphonetic character
  • the above-mentioned first text encoding model includes a convolutional layer.
  • the above-mentioned first text encoding model can be a DeBERTa model including a CNN (Convolutional Neural Networks) layer.
  • CNN Convolutional Neural Networks
  • the first text to be recognized may be text input by a user, or text already in a page, etc., and this embodiment of the present application does not limit this.
  • the first text to be recognized may include: at least one sentence or at least one paragraph.
  • the first text to be recognized may be a text that requires speech synthesis.
  • the first polyphonetic character may include one or more than two polyphonetic characters.
  • the above-mentioned convolutional layer includes a convolutional kernel.
  • the window length of the convolution window corresponding to the above-mentioned convolution kernel may be 3, and the moving step length of the convolution kernel may be 1.
  • the convolution window corresponding to the above convolution kernel refers to the window length when the convolution kernel performs a convolution operation based on the step size.
  • the convolution kernel size is 3 ⁇ 1, that is, the window length corresponding to each movement of the convolution kernel is 3.
  • the convolution kernel is used to extract local features of the first text to be recognized.
  • the above-mentioned convolution kernel may include at least one convolution kernel with a fixed size.
  • convolution window also known as convolution layer window, convolution window, etc.
  • convolution layer window also known as convolution layer window, convolution window, etc.
  • Step S202 The text recognition device extracts a first text feature sequence corresponding to a first text to be recognized.
  • the text recognition device may input the first text to be recognized into the Transformer layer of the DeBERTa model, perform text feature extraction on the first text to be recognized, and obtain a first text feature sequence corresponding to the text to be recognized.
  • the above Transformer layer is a bidirectional long short-term memory network based on a multi-head self-attention mechanism, which includes two parts: encoding and decoding.
  • the above-mentioned first text feature sequence can be: an attention result obtained by the Transformer layer of the DeBERTa model based on the contextual semantic features of the first text to be recognized, and the attention result can be represented by H0 .
  • the content information and position information of the characters in the text are represented by two vectors respectively to calculate the attention weights between characters at different positions. Then, the weighted sum of each position in the character sequence is calculated through the attention weights, and different weights are assigned to different positions in the sequence to obtain the above-mentioned attention result.
  • each character in the text can be converted into a vector form of fixed dimension to represent the content information corresponding to each character in the text, and the position information of each character can be recorded using absolute position coding.
  • each character of the input text is converted into a vector of fixed dimension to represent the content information corresponding to each character in the text, and the position information of each character is recorded using absolute position encoding.
  • the DeBERTa model takes into account the importance of the relative position between characters. For example, when the words "depth” and "learning" appear adjacent to each other in a sentence, they have a stronger correlation than when they appear separately in other different positions in the sentence. Therefore, the content information and position information of the characters are represented by two vectors respectively to calculate the attention weight.
  • ⁇ H ij ⁇ represents the embedding vector of the i-th character, that is, the character is converted into a vector of fixed dimension.
  • j ⁇ represents the relative position of the i-th character and the j-th character, so the attention weight between the two characters can be expressed by the following formula: Show:
  • the attention weights Ai ,j between characters can be decomposed into four parts: “content to content”, “content to position”, “position to content”, and “position to position”. Since relative position encoding is used, "position to position” itself has no meaning, so it is omitted in subsequent calculations.
  • the embodiment of the present application calculates attention weights based on the DeBERTa model, taking into account the information of "position to content”, and using position information to help the model better understand the semantic content in the sequence.
  • content to content refers to the connection between the semantics of one character and another character, for example, the connection between the semantics of character A and the semantics of character B in the text to be recognized;
  • content to position refers to the connection between the content of one character and the position of another character, which can reflect the closeness of the connection between the characters;
  • position to content refers to the connection between the position of one character and the content of another character;
  • position to position refers to the relationship between the position of one character and the position of another character, for example, the position of character A in the text to be recognized and the distance between the position of character B in the text.
  • H represents the input vector
  • Q, K and V are the content vectors obtained by mapping H
  • H0 is the output result of the self-attention mechanism
  • Wq , Wk , Wv are learnable mapping matrices
  • A is the attention weight matrix
  • N is the length of the input vector
  • d is the dimension of the input vector.
  • H is a real number matrix with a dimension of N ⁇ d, which can be expressed as H ⁇ R N ⁇ d
  • W v is a real number matrix with a dimension of d ⁇ d, which can be expressed as W v ⁇ R d ⁇ d
  • A is a real number matrix with a dimension of N ⁇ N, which can be expressed as A ⁇ R N ⁇ N .
  • the attention weights can be used to calculate the weighted value of each position in the sequence, thereby helping the model to better capture the correlation in the sequence and obtain an accurate text feature sequence.
  • each element in the sequence is multiplied by the attention weight corresponding to the element to obtain the weighted value of each element in the sequence. That is, by assigning different weights to different positions in the sequence, the attention result H 0 is obtained, thereby obtaining a more accurate text feature sequence.
  • the first text to be recognized may be a text obtained by concatenating multiple first texts.
  • the multiple first texts may be text fragments in the first text to be recognized.
  • the first text feature sequence may include multiple text feature sequences, and one text feature sequence corresponds to a text segment in the first text to be recognized.
  • the text recognition device converts the first text to be recognized into a vector through the DeBERTa model, obtains the character embedding vector and the type embedding vector corresponding to each text segment in the first text to be recognized, and uses the sum of the character embedding vector and the type embedding vector as the input of the Transformer layer to obtain the attention result H0 .
  • the above character embedding vector is a vector expression of the character content;
  • the type embedding vector is a vector obtained by manually marking each concatenated sentence and then converting the mark into a vector.
  • the type embedding vector can enable the model to distinguish different sentences.
  • the input vector is a concatenation of multiple sentences input into the model, and the different sentences in the concatenated sentences are marked by the type embedding vector, so that the model can distinguish different sentences.
  • the above H0 may include M text feature sequences, each sequence includes up to N elements, that is, the length of each sequence may be N. Among them, one element may correspond to a character embedding vector, and the dimensions of each character embedding vector may be the same.
  • each text feature sequence includes a text feature sequence corresponding to a text segment in the first text to be recognized.
  • the first text to be recognized is the text obtained by concatenating text 1, text 2 and text 3.
  • the attention result H0 obtained includes the sequence corresponding to text 1, the sequence corresponding to text 2 and the sequence corresponding to text 3, where each sequence can include 8 elements, that is, the length of the sequence is 8.
  • M and N are only examples and do not constitute a limitation to the present application.
  • the specific values of M and N can be determined according to actual needs.
  • Step S203 the text recognition device uses a convolution layer to perform segmented convolution processing on the first text feature sequence to obtain a second text feature sequence.
  • the second text feature sequence at least includes global semantic information of the first text to be recognized and local semantic information of a first text segment in the first text to be recognized, and the first text segment includes the first polyphonetic character.
  • the text recognition device may input the above-mentioned first text feature sequence into the CNN layer of the DeBERTa model, and perform segmented convolution processing on the first text feature sequence through the CNN layer to obtain a second text feature sequence.
  • the above-mentioned output vector H 0 sequence can be segmented convolution processed by a convolution kernel of a fixed size.
  • the convolution kernel of a fixed size is segmented and convolved with the output vector H 0 to obtain a local convolution result.
  • the convolution kernel is moved forward and convolved with the next segment of the sequence, and the convolution operation is repeated to traverse the entire sequence, and finally the overall convolution result is output.
  • FIG2 is a schematic diagram of the structure of the CNN layer in the improved DeBERTa model provided in an embodiment of the present application.
  • the DeBERTa model at least includes a CNN input layer, a convolution layer and an output layer, wherein the CNN input layer is used to input the first text feature sequence, that is, the output vector H 0 of the Transformer layer, and a sequence of the output vector H 0 includes eight elements X1, X2, X3, X4, X5, X6, X7 and X8, each element represents a character embedding vector, and the dimension of each character embedding vector is 3.
  • the convolution layer is used to perform segmented convolution on the first text feature sequence through a convolution kernel, and the output layer is used to output the convolution result.
  • each column in the input layer matrix in Figure 2 represents the dimension of the corresponding character embedding vector.
  • the local convolution result is obtained by segmenting the convolution calculation with the output vector H 0 through the convolution kernel of a fixed size. After the calculation is completed, the convolution kernel is moved forward to convolve with the next sequence, and the convolution operation is repeated to traverse the entire sequence, and finally the overall convolution result is output.
  • the size of the convolution kernel is much smaller than the sequence length of the output vector H 0 , so when using the convolution kernel to convolve with the vector, the convolution can be performed in segments.
  • the convolution kernel size is 3 ⁇ 1 and the size of H0 is 42 ⁇ 1
  • the convolution kernel is multiplied by the 1st, 2nd, and 3rd positions of H0 and the sum is taken to obtain a convolution result.
  • the convolution kernel is moved forward, and the corresponding convolution results are obtained by multiplying the convolution kernel by the elements at the 2nd, 3rd, and 4th positions of the sequence.
  • the final convolution result is obtained.
  • the text recognition device may determine a text segment corresponding to the local semantic information based on the obtained local semantic information, thereby determining the semantic information of the text segment where the polyphone is located.
  • the global semantic information of the text obtained through the Transformer layer may be “we are walking”
  • the global semantic information of the text further obtained through the CNN layer may be “we are walking”
  • the local semantic information obtained may include “we are walking” and "are in the same industry”.
  • the pronunciation of “ ⁇ ” in the text may all be predicted as “xing2”, resulting in poor accuracy.
  • the pronunciation of " ⁇ ” at the first position can be predicted as “xing2” based on the semantic information of the text segment where " ⁇ " appears, and the pronunciation of " ⁇ ” at the second position can be predicted as "hang2" based on the semantic information of the text segment where " ⁇ " appears.
  • Step S204 The text recognition device obtains first encoding information corresponding to the first text to be recognized based on the second text feature sequence.
  • the text recognition device may input the above-mentioned second text feature sequence into the encoding module of the DeBERTa model to obtain the above-mentioned first encoding information.
  • FIG3 is a schematic diagram of the structure of the improved DeBERTa model provided in the embodiment of the present application.
  • the DeBERTa model at least includes: a Transformer layer, a CNN layer and an encoding module, namely, a Mask Decoder.
  • the text recognition device splices a plurality of different texts as model input information to the DeBERTa model, and obtains the character embedding vector and the type embedding vector corresponding to the spliced text, and then uses the sum of the character embedding vector and the type embedding vector as the Transformer layer input, and obtains the text feature vector representing the global semantics corresponding to the spliced text through the Transformer layer combined with the relative position embedding vector calculation, and then inputs the text feature vector into the CNN layer for segmented convolution processing to obtain the text feature vector representing the local semantics and the global semantics, namely, the second text feature sequence, and finally inputs the text feature vector representing the local semantics and the global semantics into the encoding module to obtain the final sentence encoding result.
  • the model can not only obtain global semantic information from Transformer, but also better model local sequence information through CNN, taking into account the possible strong local correlation information of sequence semantic information, so that the model can make judgments based on global and local information when predicting polyphones.
  • the output of the CNN output layer is passed through Decoder to obtain the final sentence encoding result.
  • the pronunciation prediction results based on word and sentence encoding and the annotated pronunciation results are compared respectively.
  • the statistical results show that the accuracy of the polyphone prediction results based on sentence encoding generated by the improved DeBERTA is much higher than the polyphone pronunciation results based on word encoding.
  • the text recognition method is that the text recognition device inputs the first text to be recognized into the first text encoding model, the first text to be recognized contains the first polyphone, and the first text encoding model includes a convolution layer, then extracts the first text feature sequence corresponding to the first text to be recognized, and uses the convolution layer to perform segmented convolution processing on the first text feature sequence to obtain a second text feature sequence, the second text feature sequence at least contains the global semantic information of the first text to be recognized and the local semantic information of the first text segment in the first text to be recognized, the first text segment contains the first polyphone, and finally, based on the second text feature sequence, obtains the first encoding information corresponding to the first text to be recognized.
  • the text recognition device performs a convolution operation on the global text feature sequence of the text through the convolution layer in the first text encoding model to obtain a text feature sequence containing global semantic information and local semantic information, and obtains a more accurate recognition result based on the text feature sequence, thereby enhancing the ability to model the local information of the text sequence, and can make up for the shortcomings of the existing model in extracting local feature information, so that the pronunciation of polyphones in long and difficult sentences and complex texts can be analyzed more accurately, thereby improving the prediction of the pronunciation of polyphones in the text.
  • the accuracy of the measurement is a convolution operation on the global text feature sequence of the text through the convolution layer in the first text encoding model to obtain a text feature sequence containing global semantic information and local semantic information, and obtains a more accurate recognition result based on the text feature sequence, thereby enhancing the ability to model the local information of the text sequence, and can make up for the shortcomings of the existing model in extracting local feature information, so that the pronunciation of polyphones in long and
  • step S203 may include the following steps S203a and S203b:
  • Step S203a The text recognition device performs segmented convolution processing on the first text feature sequence through the convolution window corresponding to the convolution kernel in the convolution layer to obtain N sequence segments.
  • the text feature sequence corresponding to any sequence segment contains the local semantic information of the text segment corresponding to any sequence segment.
  • the second text feature sequence is composed of the N sequence fragments.
  • the first text feature sequence may include multiple text feature sequences, and each of the text feature sequences includes at least one text feature element arranged in sequence.
  • the text recognition device can select at least two feature elements in a first text feature sequence for convolution processing each time according to the window length of the convolution window, and move the convolution window according to the convolution step size to select feature elements for the next convolution processing.
  • the number of text feature elements contained in a sequence segment is determined based on the window length of the convolution window.
  • the size of the convolution kernel is 3*1
  • the convolution step is 1, when performing segmented convolution processing, the convolution kernel is multiplied and summed with the element X1 at the first position, the element X2 at the second position, and the element X3 at the third position of the text feature sequence to obtain a convolution result R1.
  • the convolution kernel is moved forward, and the convolution kernel is multiplied and summed with the elements at the second, third, and fourth positions of the sequence (i.e., X2, X3, and X4) to obtain the corresponding convolution result R1.
  • the convolution kernel is multiplied and summed with the elements at the sixth, seventh, and eighth positions of the sequence (i.e., X6, X7, and X8) to obtain the corresponding convolution result R6, and then the final convolution result is obtained based on the sum of each convolution result.
  • the text recognition method provided in the embodiment of the present application further includes the following steps C1 to C3:
  • Step C1 The text recognition device receives a first input from a user.
  • Step C2 In response to the first input, the text recognition device stores the text to be recognized corresponding to the first input into a first file.
  • Step C3 when the number of the to-be-recognized texts stored in the first file exceeds a first threshold, the text recognition device synthesizes the to-be-recognized texts stored in the first file into the first to-be-recognized texts.
  • the first input may be a user's touch input
  • Any feasible input such as voice input or gesture input is not limited in the embodiments of the present application.
  • the touch input may be a click input, a long press input, or a sliding input by the user.
  • the first threshold may be 50, 100 or 150, etc., which is not limited in the embodiment of the present application.
  • the text recognition device may perform text splicing on the text to be recognized stored in the first file to obtain the first text to be recognized.
  • the text recognition device obtains the text and automatically writes the text into a fixed directory file D.
  • the number of texts in the directory file D reaches a preset number
  • the pronunciation correction system will be automatically started, and the accumulated texts will be spliced to obtain the above-mentioned first text to be recognized, and the first text to be recognized will be input into the first text encoding model for processing.
  • the encoding device when a user requests to read aloud a text on a mobile phone, the encoding device can store the text requested to be read aloud each time into a file, and after the amount of text exceeds a certain threshold, input the text into the first text encoding model for processing, so that the first text encoding model can accurately semantically recognize and encode the text that the user usually needs to read aloud, and obtain more accurate encoding information, so that the pronunciation of polyphones in the text can be more accurately predicted based on the encoding in the future.
  • the text recognition method provided in the embodiment of the present application further includes the following step S205:
  • Step S205 The text recognition device inputs the first encoding information into a first polyphonetic character model, performs pronunciation prediction on the first polyphonetic character, and obtains first pronunciation prediction information of the first polyphonetic character.
  • the first polyphone model can be obtained by training a bidirectional long short-term memory neural network and a linear neural network.
  • the text recognition device after the text recognition device inputs the above-mentioned first encoding information into the first polyphonetic character model, it infers the polyphonetic character model based on the sentence encoding to obtain the corresponding polyphonetic character pronunciation prediction result.
  • the first text For example, take the first text to be recognized as " ⁇ " as an example.
  • the encoding information is input into the first polyphone model.
  • the first polyphone model predicts that the probability of the polyphone " ⁇ ” in the text being pronounced as “zhao1” is 85%, which is greater than the probability of being pronounced as "zhe5", and then outputs the pronunciation prediction information "zhao1" of the text.
  • "zhao" in the pronunciation prediction information "zhao1” represents pinyin
  • "1" represents tone, i.e., the first tone.
  • the text recognition method provided in the embodiment of the present application further includes the following step S206:
  • Step S206 the text recognition device uses the first text to be recognized as a preset text, and stores it in a text pronunciation database in association with the first pronunciation prediction information.
  • the text pronunciation library contains a plurality of preset texts and pronunciation prediction information corresponding to each preset text, and each preset text contains at least one polyphonic character.
  • the above-mentioned text pronunciation library can be called an error correction intervention library.
  • the data format of the data stored in the above-mentioned text pronunciation library is: ⁇ text>
  • the data stored in the text pronunciation library includes: "He is dressed very ordinary
  • the text recognition device obtains the first pronunciation prediction information of the first polyphonetic character, and takes the first text to be recognized as the preset text, and after storing it in a text pronunciation library in association with the first pronunciation prediction information, it can obtain erroneous pronunciation prediction information in the text pronunciation library, and add the erroneous pronunciation prediction information to the training data set of the above-mentioned first polyphonetic character model, so as to train the first polyphonetic character model through the erroneous pronunciation prediction information, thereby iteratively updating and optimizing the model.
  • the text recognition device can check the pronunciation prediction information corresponding to the preset text stored in the text pronunciation library.
  • the erroneous pronunciation prediction information is added to the training data set of the above-mentioned first polyphonetic character model to train the first polyphonetic character model through the erroneous pronunciation prediction information.
  • the text recognition device adds erroneous pronunciation prediction information to the training set, iteratively trains the polyphone model through the accumulated data set, repairs examples of incorrect pronunciation in the original online reading system, enhances the generalization ability of the model, and thus periodically improves the pronunciation prediction accuracy of the polyphone model.
  • a pronunciation prediction method provided in the embodiment of the present application may include the following steps A1 to A4:
  • Step A1 The text recognition device inputs the second text to be recognized into the second text encoding model, and extracts a third text feature sequence corresponding to the second text to be recognized.
  • the second text to be recognized contains a second polyphonetic character, and the second text encoding model is different from the first text encoding model.
  • Step A2 The text recognition device obtains the second encoding information corresponding to the second text to be recognized based on the third text feature sequence.
  • Step A3 The text recognition device inputs the second encoding information into a second polyphonetic character model, performs pronunciation prediction on the second polyphonetic character, and obtains third pronunciation prediction information of the second polyphonetic character.
  • Step A4 when the target preset text exists in the text pronunciation database, the text recognition device updates the third pronunciation prediction information to the fourth pronunciation prediction information corresponding to the target preset text.
  • the above-mentioned target preset text is the text in the above-mentioned text pronunciation library that matches the second text to be recognized.
  • the second text encoding model may be a character encoding model.
  • the polyphone model can be obtained by training a bidirectional long short-term memory neural network and a linear neural network.
  • the encoding device obtains the text request and sends the text request to the speech synthesis engine in real time, and the speech synthesis engine encodes the text using a character encoding method, and the encoded result is input into the second polyphonetic character model.
  • the speech synthesis engine when performing character encoding processing, uses a Focal Loss function to mitigate the impact of data imbalance, thereby improving the pronunciation accuracy of polyphones, and the formula is as follows:
  • Focal Loss - ⁇ t (1-p t ) ⁇ log (p t ) (4)
  • the imbalance of the number of positive and negative samples is suppressed by the weight factor ⁇ , and the value of the weight factor ⁇ can be ⁇ 0,1 ⁇ , that is, ⁇ [0,1]; (1-p t ) ⁇ is a modulation factor to assist in distinguishing difficult and easy samples.
  • the Focal Loss loss function improves the problem of sample category imbalance.
  • the amount of data for the pronunciation "ba3" in the training set is ten times that of "ba4", which causes the model prediction results to tend to predict "ba3” and mispredict some "ba4" pronunciations.
  • the model structure and loss function of the second polyphonetic character model are the same as those of the first polyphonetic character model.
  • the third pronunciation prediction information of the second text to be recognized is obtained based on the character encoding
  • the second text to be recognized is matched with a preset text in a text pronunciation library.
  • the third pronunciation prediction information is used to update the fourth pronunciation prediction information corresponding to the target preset text in the text pronunciation library.
  • the third pronunciation prediction information of the second text to be recognized is the same as the fourth pronunciation prediction information corresponding to the target preset text. If they are the same, the third pronunciation prediction information is output; if they are different, the third pronunciation prediction information is updated to the fourth pronunciation prediction information.
  • the second text to be recognized is “His clothes are very ordinary” as an example, assuming that the pronunciation prediction result of the second text to be recognized based on the character encoding is "His clothes (zhe5) are very ordinary", the second text to be recognized is matched with the preset text in the text pronunciation library, and the pronunciation prediction result corresponding to the matching text is "His clothes (zhuo2) are very ordinary", then the pronunciation prediction result of the second text to be recognized is corrected to "His clothes (zhuo2) are very ordinary”.
  • steps A1 to A4 may be performed before step S201 or after step S201.
  • the text recognition method provided in the embodiment of the present application combines the advantages of sentence encoding and character encoding. While ensuring low-latency voice reading service, it can correct erroneous pronunciations in a timely and rapid manner. According to the error correction data of the error correction system, the polyphone model is periodically iterated to improve the accuracy of polyphone pronunciation, which greatly reduces the manpower and time costs required for manual error checking and correction.
  • the text recognition method provided in the embodiment of the present application further includes the following step B1:
  • Step B1 When the text recognition device detects that the third pronunciation prediction information is erroneous pronunciation prediction information, the text recognition device adds the third pronunciation prediction information to the training data set.
  • the training data set is used to train the second polyphone model.
  • the third pronunciation prediction information of the second text to be recognized is different from the fourth pronunciation prediction information matched from the text pronunciation library
  • the third pronunciation prediction information is added as erroneous pronunciation prediction information to the training set of the above-mentioned second polyphonetic character model to perform model iterative update and optimization of the second polyphonetic character model.
  • the text recognition device adds the erroneous pronunciation prediction information obtained based on the character encoding to the training set of the polyphonetic character model that predicts the pronunciation based on the encoding result of the character encoding, and trains the polyphonetic character model based on the accumulated erroneous pronunciation prediction information to repair the examples of incorrect pronunciation in the original online reading system, enhance the generalization ability of the model, and thus periodically improve the pronunciation prediction accuracy of the polyphonetic character model.
  • FIG4 is a flow chart of a text recognition method provided by an embodiment of the present application. As shown in FIG4 , the text recognition method may include the following steps 101 to 109:
  • Step 101 Encode the text requested to be recognized by the user using a second text encoding model.
  • the text request is sent to the speech synthesis engine in real time, the text is encoded, and the encoded result is input into the polyphone model of step 102 .
  • Step 102 Obtain pronunciation prediction results through a second polyphonetic character model.
  • Step 103 Match the query target text in the intervention database.
  • step 104 if the text exists in several pre-libraries and the matching result is True, the result corresponding to the dry library is used and the process jumps to step 104 ; if the text does not exist in several pre-libraries, the matching result is False and the process jumps to step 105 .
  • Step 104 Pronunciation result intervention and repair.
  • step 104 For example, if the matching result in step 104 is True, the corresponding document in the intervention library is used. The result of this pronunciation jumps to step 105 to complete the correction of the wrong pronunciation.
  • Step 105 Output the final pronunciation.
  • Step 106 Text accumulation.
  • the pronunciation correction system operates in non-real time, and the text requested to be read aloud by the user on the mobile phone is automatically written into a fixed directory file D.
  • the pronunciation correction system is automatically started and the accumulated text is sent to step 107.
  • Step 107 Encode the text sentence based on the large model DeBERTa.
  • the text accumulated in step 106 is read, and the text sentence encoding results are output in batches using the pre-trained large model DeBERTa, which is the first text encoding model mentioned above.
  • Step 108 Obtain a pronunciation prediction result based on the second phonetic character model of the sentence encoding.
  • Step 109a iteratively update the first polyphonetic character model.
  • Step 109b iteratively update the second polyphonetic character model.
  • the text recognition method provided in the embodiment of the present application can automatically correct the pronunciation of polyphonetic characters in online speech synthesis through the automatic error correction system for polyphonetic characters' pronunciation, and the sentence encoding generated by the improved DeBERTa pre-training model can better express and capture context and semantic information, ensuring the high accuracy of pronunciation.
  • the polyphonetic character model trained by the pre-trained large model sentence encoding guides the iterative update of the online speech synthesis polyphonetic character model, thereby improving the pronunciation accuracy of the online polyphonetic character model itself.
  • error correction is performed through the automatic error correction system, avoiding the high time and labor costs required for manual error correction, while improving the error correction efficiency.
  • the text recognition method provided in the embodiment of the present application can be executed by a text recognition device.
  • a text recognition device executing the text recognition method is taken as an example to illustrate the text recognition device provided in the embodiment of the present application.
  • FIG5 is a schematic diagram of the structure of a text recognition device provided in an embodiment of the present application.
  • the text recognition device 500 includes an execution module 501 and a processing module 502, wherein:
  • the execution module 501 is used to input a first text to be recognized into a first text encoding model, wherein the first text to be recognized includes a first polyphonetic character, and the first text encoding model includes a convolutional layer;
  • the processing module 502 is used to extract a first text feature sequence corresponding to the first text to be recognized
  • the processing module 502 is further configured to perform segmented convolution processing on the first text feature sequence using the convolution layer to obtain a second text feature sequence, wherein the second text feature sequence at least includes global semantic information of the first text to be recognized and local semantic information of a first text segment in the first text to be recognized, wherein the first text segment includes the first polyphonetic character;
  • the processing module 502 is further configured to obtain first coding information corresponding to the first text to be recognized based on the second text feature sequence, and obtain a recognition result of the first text to be recognized based on the first coding information.
  • the processing module is specifically configured to perform the convolution operation
  • the convolution window corresponding to the convolution kernel in the layer performs segmented convolution processing on the above-mentioned first text feature sequence to obtain text feature sequences corresponding to N sequence fragments, the number of text feature elements contained in the above-mentioned sequence fragment is determined based on the window length of the above-mentioned convolution window, and the text feature sequence corresponding to any sequence fragment contains the local semantic information of the text fragment corresponding to the above-mentioned any sequence fragment; wherein the above-mentioned second text feature sequence is composed of the above-mentioned N sequence fragments.
  • the processing module is specifically used to input the first coding information into a first polyphonetic character model, perform pronunciation prediction on the first polyphonetic character, and obtain first pronunciation prediction information of the first polyphonetic character.
  • the above-mentioned device further includes: a storage module
  • the storage module is used for, after the processing module inputs the first coding information into the first polyphone model, predicting the pronunciation of the first polyphone, and obtaining the first pronunciation prediction information of the first polyphone, taking the first text to be recognized as a preset text, and storing it in a text pronunciation library in association with the first pronunciation prediction information;
  • the text pronunciation library contains a plurality of preset texts and pronunciation prediction information corresponding to each preset text, and each of the preset texts contains at least one polyphonic character.
  • the above-mentioned processing module is also used to input the second text to be recognized into a second text encoding model, extract a third text feature sequence corresponding to the above-mentioned second text to be recognized, the second text to be recognized contains a second polyphonetic character, and the above-mentioned second text encoding model is different from the above-mentioned first text encoding model; the above-mentioned processing module is also used to obtain the second encoding information corresponding to the above-mentioned second text to be recognized based on the above-mentioned third text feature sequence; the above-mentioned processing module is also used to input the above-mentioned second encoding information into a second polyphonetic character model, perform pronunciation prediction on the above-mentioned second polyphonetic character, and obtain the third pronunciation prediction information of the above-mentioned second polyphonetic character; the above-mentioned device also includes: an updating module; the above-mentione
  • the processing module is further used to add the third pronunciation prediction information to a training data set when it is detected that the third pronunciation prediction information is erroneous pronunciation prediction information; wherein the training data set is used to train the second polyphonetic character model.
  • the above-mentioned device also includes: a receiving module and a storage module, wherein: the above-mentioned receiving module is used to receive a first input from a user; the above-mentioned storage module is used to store the text to be recognized corresponding to the above-mentioned first input in a first file in response to the above-mentioned first input received by the above-mentioned receiving module; the above-mentioned processing module is also used to synthesize the text to be recognized stored in the above-mentioned first file into the above-mentioned first text to be input when the number of texts to be recognized stored in the above-mentioned first file exceeds a first threshold.
  • the above-mentioned receiving module is used to receive a first input from a user
  • the above-mentioned storage module is used to store the text to be recognized corresponding to the above-mentioned first input in a first file in response to the above-mentioned first input received by the above-ment
  • the text recognition device inputs a first text to be recognized into a first text encoding model, wherein the first text to be recognized includes a first polyphonetic character, and the first text encoding model includes a convolution layer, then extracts a first text feature sequence corresponding to the first text to be recognized, and uses the convolution layer to perform segmented convolution processing on the first text feature sequence to obtain A second text feature sequence, the second text feature sequence at least includes the global semantic information of the first text to be recognized and the local semantic information of the first text segment in the first text to be recognized, the first text segment includes the first polyphone, and finally, based on the second text feature sequence, the first encoding information corresponding to the first text to be recognized is obtained, and the recognition result of the first text to be recognized is obtained based on the first encoding information.
  • the text recognition device performs a convolution operation on the global text feature sequence of the text through the convolution layer in the first text encoding model to obtain a text feature sequence containing global semantic information and local semantic information, and obtains a more accurate recognition result based on the text feature sequence, thereby enhancing the ability to model the local information of the text sequence, and can make up for the deficiencies of the existing model in extracting local feature information, so as to more accurately analyze the pronunciation of polyphones in long and difficult sentences and complex texts, thereby improving the accuracy of the pronunciation prediction of polyphones in the text.
  • the text recognition device in the embodiment of the present application can be an electronic device or a component in the electronic device, such as an integrated circuit or a chip.
  • the electronic device can be a terminal or other devices other than a terminal.
  • the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a car-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR)/virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc.
  • NAS Network Attached Storage
  • PC personal computer
  • TV television
  • teller machine a self-service machine
  • the text recognition device in the embodiment of the present application may be a device having an operating system.
  • the operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.
  • the text recognition device provided in the embodiment of the present application can implement each process implemented in the method embodiments of Figures 1 to 4, and will not be described again here to avoid repetition.
  • an embodiment of the present application also provides an electronic device 600, including a processor 601 and a memory 602, and the memory 602 stores a program or instruction that can be executed on the above-mentioned processor 601.
  • the program or instruction is executed by the processor 601
  • the various steps of the above-mentioned text recognition method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
  • the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.
  • FIG. 7 is a schematic diagram of the hardware structure of an electronic device implementing an embodiment of the present application.
  • the electronic device 1000 includes but is not limited to: a radio frequency unit 1001, a network module 1002, an audio output unit 1003, an input unit 100, a sensor 1005, a display unit 1006, a user input unit 1007, an interface unit 1008, a memory 1009, and a processor 1010 and other components.
  • the electronic device 1000 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 1010 through a power management system, so that the power management system can manage charging, discharging, power consumption, and other functions.
  • a power source such as a battery
  • the electronic device structure shown in the figure does not constitute a limitation on the electronic device.
  • the electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.
  • the processor 1010 is used to input a first text to be recognized into a first text encoding model, wherein the first text to be recognized includes a first polyphonetic character, and the first text encoding model includes a convolutional layer;
  • the processor 1010 is used to extract a first text feature sequence corresponding to a first text to be recognized
  • the processor 1010 is further configured to perform a segmented convolution process on the first text feature sequence using the convolution layer to obtain a second text feature sequence, wherein the second text feature sequence at least includes global semantic information of the first text to be recognized and local semantic information of a first text segment in the first text to be recognized, wherein the first text segment includes the first polyphonetic character;
  • the processor 1010 is further configured to obtain first encoding information corresponding to the first text to be recognized based on the second text feature sequence, and obtain a recognition result of the first text to be recognized based on the first encoding information.
  • the processor 1010 is specifically used to perform segmented convolution processing on the text feature sequence through a convolution window corresponding to the convolution kernel in the convolution layer to obtain text feature sequences corresponding to N sequence fragments, wherein the text feature sequence corresponding to any sequence fragment contains local semantic information of the text fragment corresponding to any of the sequence fragments; wherein the second text feature sequence is composed of the N sequence fragments.
  • the processor 1010 is specifically configured to input the first coding information into a first polyphonetic character model, perform pronunciation prediction on the first polyphonetic character, and obtain first pronunciation prediction information of the first polyphonetic character.
  • the memory 1009 is used to input the first coding information into the first polyphone model, perform pronunciation prediction on the first polyphone, and obtain first pronunciation prediction information of the first polyphone, and then use the first text to be recognized as a preset text, and store it in a text pronunciation library in association with the first pronunciation prediction information;
  • the text pronunciation library contains a plurality of preset texts and pronunciation prediction information corresponding to each preset text, and each of the preset texts contains at least one polyphonic character.
  • the processor 1010 is further used to input a second text to be recognized into a second text encoding model, extract a third text feature sequence corresponding to the second text to be recognized, the second text to be recognized includes a second polyphonetic character, and the second text encoding model is different from the first text encoding model; the processor 1010 is further used to obtain second encoding information corresponding to the second text to be recognized based on the third text feature sequence; the processor 1010 is further used to input the second encoding information into a second polyphonetic character model, perform pronunciation prediction on the second polyphonetic character, and obtain third pronunciation prediction information of the second polyphonetic character; the device further includes: an update module; the update module is used to update the third pronunciation prediction information to the fourth pronunciation prediction information corresponding to the target preset text when there is a target preset text in the text pronunciation library; wherein the target preset text is a text in the text pronunciation library that matches the second text to be recognized.
  • the processor 1010 is further configured to add the third pronunciation prediction information to a training data set when it is detected that the third pronunciation prediction information is erroneous pronunciation prediction information; wherein the training data set is used to train the second polyphonetic character model.
  • the user input unit 1007 is used to receive a first input from a user; the memory 1009 is used to store the text to be recognized corresponding to the first input in a first file in response to the first input received by the user input unit 1007; the processor 1010 is also used to synthesize the text to be recognized stored in the first file into the first text to be input when the amount of the text to be recognized stored in the first file exceeds a first threshold.
  • An electronic device inputs a first text to be recognized into a first text encoding model, the first text to be recognized includes a first polyphone, and the first text encoding model includes a convolution layer. Then, a first text feature sequence corresponding to the first text to be recognized is extracted, and the first text feature sequence is subjected to segmented convolution processing using the convolution layer to obtain a second text feature sequence.
  • the second text feature sequence at least includes global semantic information of the first text to be recognized and local semantic information of a first text segment in the first text to be recognized, the first text segment includes a first polyphone.
  • a text recognition device performs a convolution operation on the global text feature sequence of the text through the convolution layer in the first text encoding model to obtain a text feature sequence containing global semantic information and local semantic information, and obtains a more accurate recognition result based on the text feature sequence, thereby enhancing the ability to model the local information of the text sequence, and can make up for the deficiencies of the existing model in extracting local feature information, so as to more accurately analyze the pronunciation of polyphones in long and difficult sentences and complex texts, thereby improving the accuracy of pronunciation prediction of polyphones in the text.
  • the input unit 1004 may include a graphics processing unit (GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode.
  • the display unit 1006 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc.
  • the user input unit 1007 includes a touch panel 1071 and at least one of other input devices 1072.
  • the touch panel 1071 is also called a touch screen.
  • the touch panel 1071 may include two parts: a touch detection device and a touch controller.
  • Other input devices 1072 may include, but are not limited to, a physical keyboard, function keys (such as a volume control key, a switch key, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.
  • the memory 1009 can be used to store software programs and various data.
  • the memory 1009 can mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area can store an operating system, an application program or instruction required for at least one function (such as a sound playback function, an image playback function, etc.).
  • the memory 1009 can include a volatile memory or a non-volatile memory, or the memory 1009 can include a volatile and a non-volatile memory. Both.
  • the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory.
  • the volatile memory can be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous connection dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM).
  • the memory 1009 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.
  • the processor 1010 may include one or more processing units; optionally, the processor 1010 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 1010.
  • An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored.
  • a program or instruction is stored.
  • the various processes of the above-mentioned text recognition method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
  • the processor is the processor in the electronic device described in the above embodiment.
  • the readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.
  • An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned text recognition method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
  • the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
  • An embodiment of the present application provides a computer program product, which is stored in a storage medium.
  • the program product is executed by at least one processor to implement the various processes of the above-mentioned text recognition method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
  • the method may be performed in a substantially simultaneous manner or in a reverse order according to the functions involved, for example, the method may be performed in an order different from that described, and various steps may be added, omitted, or combined.
  • features described with reference to certain examples may be combined in other examples.
  • the technical solution of the present application can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM/RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.
  • a storage medium such as ROM/RAM, a disk, or an optical disk
  • a terminal which can be a mobile phone, a computer, a server, or a network device, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Character Discrimination (AREA)
  • Machine Translation (AREA)

Abstract

本申请公开了一种文本识别方法、装置、电子设备及可读存储介质，属于自然语言处理领域。该方法包括：将第一待识别文本输入第一文本编码模型，第一待识别文本包含第一多音字，第一文本编码模型包括卷积层；提取第一待识别文本对应的第一文本特征序列；采用卷积层对第一文本特征序列进行分段卷积处理，得到第二文本特征序列，第二文本特征序列至少包含第一待识别文本的全局语义信息和第一待识别文本中的第一文本片段的局部语义信息，第一文本片段包含第一多音字；基于第二文本特征序列，得到第一待识别文本对应的第一编码信息，并基于上述第一编码信息获得第一待识别文本的识别结果。

Description

文本识别方法、装置、电子设备及可读存储介质
相关申请的交叉引用
本申请主张在2023年07月14日在中国提交的中国专利申请号No.202310873503.9的优先权，其全部内容通过引用包含于此。
技术领域
本申请属于自然语言处理领域，具体涉及一种文本识别方法、装置、电子设备及可读存储介质。
背景技术
语音合成是一种将文本转换为语音的技术，在用户与电子设备交互的场景中，语音合成技术为用户提供了一种新的信息交互方式。目前，在中文语音合成播报场景中，由于汉字里包含大量多音字，语音合成在处理多音字时可能会出现读错的情况。
在相关技术中，为了让电子设备正确地读出一段文本，通常需要使用多音字模型来预测文本读音，多音字模型需要根据文本编码后的结果来进行读音预测，文本编码指将自然语言文本转换成向量的形式便于计算机理解和处理。目前在进行文本编码时通常采用的字编码方式，即逐字进行编码，只考虑当前字符的内容。
然而，在采用字编码方式处理一些语言结构比较复杂的文本时，可能会出现多音字读音预测错误，从而导致对文本中的多音字的读音预测的准确率较低。
发明内容
本申请实施例的目的是提供一种文本识别方法、装置、电子设备及可读存储介质，能够更好地利用局部语义信息和全局语义信息分析长难句、复杂文本读音，进而提高对文本中的多音字读音预测的准确率。
第一方面，本申请实施例提供了一种文本识别方法，该方法包括：将第一待识别文本输入第一文本编码模型，第一待识别文本包含第一多音字，第一文本编码模型包括卷积层；提取第一待识别文本对应的第一文本特征序列；采用卷积层对第一文本特征序列进行分段卷积处理，得到第二文本特征序列，第二文本特征序列至少包含第一待识别文本的全局语义信息和第一待识别文本中的第一文本片段的局部语义信息，第一文本片段包含第一多音字；基于第二文本特征序列，得到第一待识别文本对应的第一编码信息，并基于上述第一编码信息获得第一待识别文本的识别结果。
第二方面，本申请实施例提供了一种文本识别装置，该装置包括：处理模块和提取模块，其中：处理模块，用于将第一待识别文本输入第一文 本编码模型，第一待识别文本包含第一多音字，第一文本编码模型包括卷积层；提取模块，用于提取第一待识别文本对应的第一文本特征序列；处理模块，还用于采用卷积层对第一文本特征序列进行分段卷积处理，得到第二文本特征序列，第二文本特征序列至少包含第一待识别文本的全局语义信息和第一待识别文本中的第一文本片段的局部语义信息，第一文本片段包含第一多音字；处理模块，还用于基于第二文本特征序列，得到第一待识别文本对应的第一编码信息，并基于上述第一编码信息获得第一待识别文本的识别结果。
第三方面，本申请实施例提供了一种电子设备，该电子设备包括处理器和存储器，所述存储器存储可在所述处理器上运行的程序或指令，所述程序或指令被所述处理器执行时实现如第一方面所述的方法的步骤。
第四方面，本申请实施例提供了一种可读存储介质，所述可读存储介质上存储程序或指令，所述程序或指令被处理器执行时实现如第一方面所述的方法的步骤。
第五方面，本申请实施例提供了一种芯片，所述芯片包括处理器和通信接口，所述通信接口和所述处理器耦合，所述处理器用于运行程序或指令，实现如第一方面所述的方法。
第六方面，本申请实施例提供一种计算机程序产品，该程序产品被存储在存储介质中，该程序产品被至少一个处理器执行以实现如第一方面所述的方法。
在本申请实施例中，文本识别装置将第一待识别文本输入第一文本编码模型，该第一待识别文本包含第一多音字，该第一文本编码模型包括卷积层，然后，提取第一待识别文本对应的第一文本特征序列，并采用所述卷积层对上述第一文本特征序列进行分段卷积处理，得到第二文本特征序列，上述第二文本特征序列至少包含所述第一待识别文本的全局语义信息和第一待识别文本中的第一文本片段的局部语义信息，上述第一文本片段包含第一多音字，最后，基于上述第二文本特征序列，得到第一待识别文本对应的第一编码信息，并基于上述第一编码信息获得第一待识别文本的识别结果。通过该方法，文本识别装置通过第一文本编码模型中的卷积层对文本的全局文本特征序列进行卷积操作获得包含全局语义信息和局部语义信息的文本特征序列，并基于该文本特征序列得到更为准确的识别结果，从而加强对文本序列局部信息建模能力，能够弥补现有模型对于局部特征信息提取的不足，从而能够更为准确地分析长难句、复杂文本中多音字的读音，进而提高对文本中的多音字读音预测的准确率。
附图说明
图1为本申请实施例提供的文本识别方法的流程示意图；
图2为本申请实施例提供的改进后的DeBERTa模型中的CNN层的结构示意图；
图3为本申请实施例提供的改进的DeBERTa模型的结构示意图；
图4为本申请实施例提供的文本识别方法的流程图；
图5为本申请实施例提供的文本识别装置的结构示意图；
图6为本申请实施例提供的电子设备的结构示意图；
图7为本申请实施例提供的电子设备的硬件结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员获得的所有其他实施例，都属于本申请保护的范围。
本申请的说明书和权利要求书中的术语“第一”、“第二”等是用于区别类似的对象，而不用于描述特定的顺序或先后次序。应该理解这样使用的术语在适当情况下可以互换，以便本申请的实施例能够以除了在这里图示或描述的那些以外的顺序实施，且“第一”、“第二”等所区分的对象通常为一类，并不限定对象的个数，例如第一对象可以是一个，也可以是多个。此外，说明书以及权利要求中“和/或”表示所连接对象的至少其中之一，字符“/”，一般表示前后关联对象是一种“或”的关系。
语音合成是一种将文本转换为语音的技术，在用户与手机交互的场景中，语音合成技术为用户提供了一种新的信息交互方式。相比于视觉上的信息交互，在一些特定场景，例如，驾驶场景或者运动场景，语音合成以声音的形式能够更方便快捷地传递信息。其次，语音合成通过模拟语言的语调、语气和情感帮助用户更好地理解信息，为用户提供更加真实的交互体验。
在中文语音合成播报场景中，由于汉字里包含大量多音字，语音合成在处理多音字时可能会出现读错的情况，这种误读向用户传递了错误信息进而对信息产生了误解，降低了语音信息传递的可靠性和准确性。因此对错误读音进行纠正非常重要，尤其是针对多音字等重难点字词读音的识别与校正。
在相关技术中，为了让电子设备正确地读出一段文本，通常需要使用词典规则和多音字模型来预测文本读音，其中多音字模型需要根据文本编码后的结果来进行读音预测，文本编码指将自然语言文本转换成向量的形式便于计算机理解和处理。目前有两种编码方式：字编码和句编码。字编码指的是逐字进行编码只考虑当前字符的内容，采取字编码方式计算量比较小，效率高，但在一些复杂、有歧义的场景下由于上下文的联系可能会出现多音字读音预测错误。句编码考虑了前后语境、语义，因此能够更准确地预测多音字的读音，但句编码方式计算速度较为缓慢，需要大量的语料库进行训练，以及，现有的句编码方式通过基于全局的自注意力机制捕捉序列中的上下文关系，输入序列的每个位置能够关注到序列中其他所有位置的信息，但也因此在处理局部信息时容易受到其他位置的影响，难以提取细粒度的局部特征，如此，基于句编码对长难句等复杂文本中的多音 字进行读音预测的准确性低。
针对上述问题，本申请提出一种文本识别方法，通过在DeBERTa(Decoding-enhanced BERT with disentangled attention)模型新增卷积模块，即，卷积层，并通过该卷积层基于待识别文本的全局文本特征序列，提取待识别文本的局部特征信息，从而加强对文本序列局部信息建模能力，卷积网络通过局部连接和卷积核权值共享的方式，对序列分段进行卷积操作获得局部特征。在Transformer层后新增卷积模块，能够弥补Transformer层对于局部特征提取的不足，从而能够更好地利用局部信息和全局信息分析长难句、复杂文本读音，进而能够提高对长难句等复杂文本中的多音字进行读音预测的准确性。
需要说明的是，DeBERTa，即，带有分散注意力的解码增强型BERT。DeBERTa模型为基于BERT(Bidirectional Encoder Representation from Transformers)模型改进后得到的模型，BERT，即，双向Transformer的编码。
需要说明的是，BERT模型和DeBERTa模型是为不同的自然语言处理(Natural Language Processing，NLP)任务提供支持的通用的新型语言模型。DeBERTa模型的编码器结构为多层Transformer叠加，Transformer是一种在文本、时间序列建模任务中广泛使用的模型，它对整个序列进行建模，通过基于全局的自注意力机制捕捉序列中的上下文关系，输入序列的每个位置能够关注到序列中其他所有位置的信息。
下面结合附图，通过具体的实施例及其应用场景对本申请实施例提供的文本识别方法进行详细地说明。
图1为本申请实施例提供的文本识别方法的流程示意图，如图1所示，该文本识别方法可以包括以下步骤S201至步骤S204：
步骤S201：文本识别装置将第一待识别文本输入第一文本编码模型。
其中，上述第一待识别文本包含第一多音字，上述第一文本编码模型包括卷积层。
可选地，在本申请实施例中，上述第一文本编码模型可以为包括CNN(卷积神经网络，Convolutional Neural Networks)层的DeBERTa模型。
需要说明的是，对DeBERTa模型的解释说明可以参见上文描述，此处不再赘述。
可选地，在本申请实施例中，上述第一待识别文本可以为用户输入的文本，或者页面中已有的文本等，本申请实施例对此不作限定。
可选地，在本申请实施例中，上述第一待识别文本可以包括：至少一个句子或者至少一个段落。
可选地，在本申请实施例中，上述第一待识别文本可以为需要进行语音合成的文本。
可选地，在本申请实施例中，上述第一多音字可以包括一个或者两个以上多音字。
在本申请实施例中，上述卷积层包括卷积核(convolutional kernel)。
可选地，在本申请实施例中，上述卷积核对应的卷积窗口的窗长可以为3，所述卷积核的移动步长可以为1。
需要说明的是，上述卷积核对应的卷积窗口指的是卷积核执行卷积操作时，基于步长移动时的窗口长度。例如，卷积核尺寸为3×1，即卷积核每移动一次对应的窗口长度为3。
进一步可选地，上述卷积核用于对第一待识别文本进行局部特征提取。
进一步可选地，上述卷积核可以包括至少一个尺寸固定的卷积核。
需要说明的是，上述卷积窗口，又名卷积层窗口，卷积窗等，上述卷积窗口用于提取输入数据中的特征信息。
步骤S202：文本识别装置提取第一待识别文本对应的第一文本特征序列。
可选地，在本申请实施例中，文本识别装置可以将第一待识别文本输入至DeBERTa模型的Transformer层，对该第一待识别文本进行文本特征提取，得到该待识别文本对应的第一文本特征序列。
需要说明的是，上述Transformer层是一个基于多头自注意力机制的双向长短期记忆网络，包括编码和解码两个部分。
可选地，在本申请实施例中，上述第一文本特征序列可以为：通过DeBERTa模型的Transformer层基于第一待识别文本的上下文语义特征得到的注意力结果，该注意力结果可以用H0表示。
进一步可选地，在通过DeBERTa模型的Transformer层对输入的文本进行处理时，将文本中的字符的内容信息和位置信息分别用两个向量表示，用以计算不同位置的字符间的注意力权重，然后，通过注意力权重，计算字符序列中每个位置的加权和，并在序列不同位置上分配不同的权重，得到上述注意力结果。
进一步地，文本中的每个字符可以转换成一个固定维度的向量形式，用以表示文本中每个字符对应的内容信息，并使用绝对位置编码记录每个字符的位置信息。
需要说明的是，在BERT中，输入文本的每个字符都转换成一个固定维度的向量形式，用以表示文本中每个字符对应的内容信息，使用绝对位置编码记录每个字符的位置信息。而DeBERTa模型考虑了字符与字符间的相对位置的重要性，例如“深度”和”学习”两个词在句子中相邻出现时，比它们分别出现在句子其他不同位置时具有更强的关联性，因此将字符的内容信息和位置信息分别用两个向量表示，用以计算注意力权重。
以下通过具体的实施例对通过DeBERTa模型的Transformer层，对第一待识别文本进行文本特征提取的过程进行解释说明。
示例性地，以处理文本中第i个字符为例，{Hij}表示第i个字符的嵌入向量，即将字符转换为一个固定维度的向量。{Pi|j}表示第i个字符和第j个字符的相对位置,由此两个字符之间的注意力权重可用以下公式进行表 示：
由上式可知，字符与字符间的注意力权重Ai，j可以分解为“内容到内容”，“内容到位置”，“位置到内容”，“位置到位置”四个部分，由于使用相对位置编码，“位置到位置”本身不具备意义，因此在后续计算中略去。相比于相关技术中通过BERT计算注意力权重时仅考虑“内容到内容”和“内容到位置”，本申请实施例基于DeBERTa模型计算注意力权重，考虑了“位置到内容”的信息，通过位置信息帮助模型更好的理解序列中的语义内容。
需要说明的是，上述“内容到内容”指的是一个字符和另一个字符的语义之间的联系，例如，待识别的文本中的字符A的语义和字符B的语义之间的联系；上述“内容到位置”指的是一个字符的内容和另一个字符的位置之间的联系，能够反映字符之间的联系紧密程度；上述“位置到内容”指的是一个字符的位置与另一个字符的内容之间的联系；上述“位置到位置”指的是一个字符的位置与另一个字符之间的位置之间的关系，例如，待识别的文本中的字符A的位置和字符B的在文本中的位置的远近。
结合上述公式(1)，标准的注意力机制计算公式如下：
其中，H表示输入向量，Q、K和V为通过对H作映射得到的内容向量，H0为自注意力机制输出结果,Wq，Wk，Wv为可学习的映射矩阵，A为注意力权重矩阵，N为输入向量长度，d为输入向量的维度。
需要说明的是，H为一个维度为N×d的实数矩阵，可以表示为H∈RN×d；Wv是一个维度为d×d的实数矩阵，可以表示为Wv∈Rd×d；A是一个维度为N×N的实数矩阵，可以表示为A∈RN×N。
对于上述公式(2)，使用公式(1)中的方式将其进一步分解公式(3)。其中Qc,Kc,Vc表示由矩阵Wq,c,Wk,c,Wv,c映射后的内容向量,Qr,Kr表示Wq,r,Wk,r由映射后的相对位置向量，公式如下：
其中，为注意力矩阵中的元素，表示第i个字符到第j个字符间的注意力权重，对应公式(1)中前三项的计算公式，为Qc的第i行向量，为Qc的第j行向量，δ(i|j)表示文本中的字符i和字 符j间的相对距离，为尺度因子，d为输入向量的维度。
需要说明的是，通过注意力权重，可以计算序列中每个位置的加权值，从而帮助模型更好地捕捉序列中的相关性，得到准确的文本特征序列。
示例性地，在通过上述公式(2)计算得到注意力权重后，将序列中每个元素与该元素对应的注意力权重相乘，得到序列中每个元素的加权值，也就是说，通过在序列的不同位置上分配不同的权重，获得注意力结果H0，从而得到更加准确的文本特征序列。
可选地，在本申请实施例中，第一待识别文本可以为将多个第一文本进行拼接后得到的文本，换言之，上述多个第一文本可以为第一待识别文本中的文本片段。
可选地，在本申请实施例中第一文本特征序列可以包括多个文本特征序列，一个文本特征序列对应第一待识别文本中的一个文本片段。
进一步可选地，在通过DeBERTa模型的Transformer层对上述拼接后得到的文本进行处理时，文本识别装置在第一待识别文本输入至DeBERTa模型后，通过该DeBERTa模型将该第一待识别文本转换为向量，得到第一待识别文本中的每个文本片段对应的字符嵌入向量以及类型嵌入向量，并将字符嵌入向量与类型嵌入向量的和作为Transformer层的输入，得到注意力结果H0。
进一步地，上述字符嵌入向量为字符内容的向量表达形式；类型嵌入向量为人工给每个拼接的句打上标记，然后将标记转换为向量，而得到的向量，类型嵌入向量可以使模型能够区分不同的句子。
需要说明的是，输入向量为多个句子拼接输入至模型，通过类型嵌入向量给拼接句子里的不同句子打标记，使模型能够区分不同句子。
进一步地，上述H0可以包括M个文本特征序列，每个序列中包括至N个元素，即，每个序列的长度可以为N。其中，一个元素可以对应一个字符嵌入向量，每个字符嵌入向量的维度可以相同。
进一步地，每个文本特征序列包含上述第一待识别文本中的一个文本片段对应的文本特征序列。
举例说明，以第一待识别文本为对文本1、文本2和文本3进行拼接后得到的文本，在通过Transformer层对该文本处理后，得到的注意力结果H0包括文本1对应的序列，文本2对应的序列和文本3对应的序列，其中每个序列可以包括8个元素，即序列的长度为8。
需要说明的是，上述M、N的取值仅为一种示例，并不构成对本申请的限定，M和N的取值的具体可以根据实际需求确定。
步骤S203：文本识别装置采用卷积层对第一文本特征序列进行分段卷积处理，得到第二文本特征序列。
其中，上述第二文本特征序列至少包含第一待识别文本的全局语义信息和第一待识别文本中的第一文本片段的局部语义信息，上述第一文本片段包含上述第一多音字。
可选地，在本申请实施例中，文本识别装置可以将上述第一文本特征序列输入至DeBERTa模型的CNN层，并通过CNN层对该第一文本特征序列进行分段卷积处理，得到第二文本特征序列。
示例性地，以第一文本特征序列为Transformer层的输出向量H0为例，文本识别装置在通过CNN层对该第一文本特征序列进行分段卷积处理时，可以通过一个固定尺寸的卷积核对上述输出向量H0的序列进行分段卷积处理。具体地，将固定大小的卷积核与输出向量H0分段进行卷积计算获得局部的卷积结果，当计算结束后，前移卷积核与下一段序列进行卷积，重复该卷积操作遍历整个序列，最后输出整体的卷积结果。
以下结合附图对本申请实施例中的分段卷积处理过程进行举例说明。
图2为本申请实施例提供的改进后的DeBERTa模型中的CNN层的结构示意图，如图2所示，上述DeBERTa模型至少包括CNN输入层、卷积层和输出层，其中，上述CNN输入层用于输入上述第一文本特征序列，即，Transformer层的输出向量H0，该输出向量H0的一个序列包括X1、X2、X3、X4、X5、X6、X7以及X8以上八个元素，每个元素表示一个字符嵌入向量，每个字符嵌入向量的维度为3。上述卷积层用于通过卷积核对上述第一文本特征序列进行分段卷积，上述输出层用于输出卷积结果。
需要说明的是，图2中的输入层矩阵中的每一列表示对应字符嵌入向量的维度。
结合上述图2，在将上述输出向量H0输入至卷积层后，通过固定大小的卷积核与输出向量H0分段进行卷积计算获得局部的卷积结果，当计算结束后，前移卷积核与下一段序列进行卷积，重复该卷积操作遍历整个序列，最后输出整体的卷积结果。比如，卷积核的大小远小于输出向量H0的序列长度，因此使用卷积核与向量做卷积时，可以分段进行卷积。
举例说明，卷积核大小为3×1，H0大小为42×1，那么第一次卷积运算时，卷积核与H0的第1、2、3个位置做乘法并求和，得到一个卷积结果。之后前向移动卷积核，通过卷积核与序列的第2、3、4个位置的元素做乘法，分别得到对应的卷积结果。然后基于每个卷积结果的和，得到最终的卷积结果。如此，通过控制卷积层中的卷积核大小，在做卷积运算时能够仅关注指定范围内的内容，从而获得准确的局部语义信息。
可选地，在本申请实施例中，文本识别装置可以根据得到的局部语义信息，确定与该局部语义信息对应的文本片段，从而确定多音字所在的文本片段的语义信息。
示例性地，以第一待识别文本为“我们一起同行的人都是同行”为例，则通过Transformer层获取到该文本的全局语义信息可以为“我们在行走”，进一步通过CNN层获取到该文本的全局语义信息可以为“我们在行走”，获取到的局部语义信息可以包括“我们同行”和“是同行”。
可以理解的是，后续通过该文本的全局语义信息进行多音字预测时，可能会将该文本中的“行”的读音均预测为“xing2”，导致准确性较差。 而通过局部语义信息进行多音字预测时，可以根据第一个位置出现的“行”所在的文本片段的语义信息“我们同行”预测该处的“行”的读音为“xing2”，根据第二个位置出现的“行”所在的文本片段的语义信息预测该处的“行”的读音为“hang2”。
步骤S204：文本识别装置基于上述第二文本特征序列，得到第一待识别文本对应的第一编码信息。
可选地，在本申请实施例中，文本识别装置可以将上述第二文本特征序列输入至DeBERTa模型的编码模块，得到上述第一编码信息。
以下结合附图对本申请实施例提供的编码方法进行解释说明。
图3为本申请实施例提供的改进的DeBERTa模型的结构示意图，如图3所示，该DeBERTa模型至少包括：Transfomer层、CNN层和编码模块，即Mask Decoder。如图3所示，文本识别装置将多个不同的文本拼接，作为模型输入信息输入至DeBERTa模型，并得到拼接文本对应的字符嵌入向量和类型嵌入向量，然后将字符嵌入向量与类型嵌入向量的和作为Transformer层输入，通过Transformer层结合相对位置嵌入向量计算得到拼接文本对应的表征全局语义的文本特征向量后，将该文本特征向量输入至CNN层进行分段卷积处理，得到表征局部语义和全局语义的文本特征向量，即第二文本特征序列，最后将该表征局部语义和全局语义的文本特征向量输入至编码模块，得到最终的句编码结果。如此，通过将Transformer与CNN相结合，使模型不但能从Transformer获得全局语义信息，也能通过CNN更好地对局部序列信息进行建模，考虑了序列语义信息的可能存在的局部强相关信息，使模型在预测多音字时能够综合全局和局部信息进行判断。将CNN输出层输出的结果经过Decoder获得最终的句编码结果，在已有标注的验证集测试中，分别对比了基于字、句编码的读音预测结果和标注的读音结果，统计结果表明，基于改进的DeBERTA生成句编码的多音字预测结果准确率远高于基于字编码的多音字读音结果。
本申请实施例提供的文本识别方法，文本识别装置将第一待识别文本输入第一文本编码模型，该第一待识别文本包含第一多音字，该第一文本编码模型包括卷积层，然后，提取第一待识别文本对应的第一文本特征序列，并采用所述卷积层对上述第一文本特征序列进行分段卷积处理，得到第二文本特征序列，上述第二文本特征序列至少包含所述第一待识别文本的全局语义信息和第一待识别文本中的第一文本片段的局部语义信息，上述第一文本片段包含第一多音字，最后，基于上述第二文本特征序列，得到第一待识别文本对应的第一编码信息。通过该方法，文本识别装置通过第一文本编码模型中的卷积层对文本的全局文本特征序列进行卷积操作获得包含全局语义信息和局部语义信息的文本特征序列，并基于该文本特征序列得到更为准确的识别结果，从而加强对文本序列局部信息建模能力，能够弥补现有模型对于局部特征信息提取的不足，从而能够更为准确地分析长难句、复杂文本中多音字的读音，进而提高对文本中的多音字读音预 测的准确率。
可选地，在本申请实施例中，上述步骤S203可以包括以下步骤S203a和步骤S203b：
步骤S203a：文本识别装置通过上述卷积层中的卷积核对应的卷积窗口，对上述第一文本特征序列进行分段卷积处理，得到N个序列片段。
其中，任一序列片段对应的文本特征序列包含上述任一序列片段对应文本片段的局部语义信息。
可选地，在本申请实施例中，上述第二文本特征序列由上述N个序列片段组成。
可选地，在本申请实施例中，上述第一文本特征序列可以包括多个文本特征序列，上述每个文本特征序列包含至少一个按序排列的文本特征元素。
需要说明的是，对于卷积层的相关解释说明可以参见上文，此处不再赘述，
可选地，在本申请实施例中，文本识别装置可以根据卷积窗口的窗长，每次选择一个第一文本特征序列中的至少两个特征元素进行卷积处理，并且根据卷积步长移动卷积窗口，以选择特征元素进行下一次的卷积处理。
可选地，在本申请实施例中，一个序列片段中包含的文本特征元素的数量是基于卷积窗口的窗长确定的，
示例性地，结合上述实施例，以文本特征序列包括X1、X2、X3、X4、X5、X6、X7以及X8以上八个元素、卷积核的尺寸为3*1，卷积步长为1为例，在进行分段卷积处理时，卷积核与文本特征序列的第1个位置的元素X1、第2个位置的元素X2和第3个位置的元素X3做乘法并求和，得到一个卷积结果R1。之后前向移动卷积核，通过卷积核与序列的第2、3、4个位置的元素(即，X2、X3、X4)做乘法并求和，得到对应的卷积结果R1，以此类推，通过卷积核与序列的第6、7、8个位置的元素(即，X6、X7、X8)做乘法并求和，得到对应的卷积结果R6，然后基于每个卷积结果的和，得到最终的卷积结果。
如此，通过控制卷积层中的卷积核大小，在做卷积运算时能够仅关注指定范围内的内容，从而获得准确的局部语义信息。
可选地，在本申请实施例中，上述步骤S201之前，本申请实施例提供的文本识别方法还包括以下步骤C1至步骤C3：
步骤C1：文本识别装置接收用户的第一输入。
步骤C2：文本识别装置响应于上述第一输入，将第一输入对应的待识别文本存储至第一文件中。
步骤C3：文本识别装置在第一文件中存储的待识别文本的数量超过第一阈值的情况下，将第一文件中存储的待识别文本合成为上述第一待识别文本。
可选地，在本申请实施例中，上述第一输入可以为用户的触控输入、 语音输入或者手势输入等任意具备可行性的输入，本申请实施例对此不作限定。
进一步地，上述触控输入可以为用户的点击输入、长按输入或者滑动输入等。
可选地，在本申请实施例中，上述第一阈值可以为50条、100条或者150条等，本申请实施例对此不作限定。
可选地，在本申请实施例中，文本识别装置可以将上述第一文件中存储的待识别文本进行文本拼接，得到上述第一待识别文本。
示例性地，以第一输入为用户对待识别文本的点击输入为例，用户点击电子设备的界面上显示的需要进行语音合成的文本后，文本识别装置获取该文本，并将该文本自动写入固定目录文件D，当该目录文件D中文本数量达到预先设定的数量时，将会自动启动读音纠正系统，并将所积累的文本进行拼接得到上述第一待识别文本，并将该第一待识别文本输入至第一文本编码模型进行处理。
本申请实施例提供的文本识别方法，用户在手机端请求朗读文本时，编码装置可以将每次请求朗读的文本存储至文件中，并在文本数量超过一定阈值后，将文本输入至第一文本编码模型进行处理，从而能够第一文本编码模型对用户惯常需要朗读的文本进行准确地语义识别和编码，得到更为准确的编码信息，从而在后续能够基于编码更为准确地预测文本中多音字的读音。
可选地，在本申请实施例中，上述步骤S204之后，本申请实施例提供的文本识别方法还包括以下步骤S205：
步骤S205：文本识别装置将上述第一编码信息输入至第一多音字模型，对该第一多音字进行读音预测，得到该第一多音字的第一读音预测信息。
可选地，在本申请实施例中，上述第一多音字模型可以由双向长短时记忆神经网络和线性神经网络训练获得。
可选地，在本申请实施例中，文本识别装置将上述第一编码信息输入至第一多音字模型后，基于句编码的多音字模型推理，得到对应的多音字读音预测结果。
举例说明，以第一待识别文本包括“差点着了他的道”为例，在基于上述第一文本编码模型得到该文本的编码信息后，将该编码信息输入至第一多音字模型，通过第一多音字模型预测该文本中的多音字“着”发音为“zhao1”的概率为85％，大于发音为“zhe5”的概率，则输出该文本的读音预测信息“zhao1”。需要说明的是，读音预测信息“zhao1”中的“zhao”表示拼音，“1”表示声调，即一声。
进一步可选地，在本申请实施例中，上述步骤S205之后，本申请实施例提供的文本识别方法还包括以下步骤S206：
步骤S206：文本识别装置将上述第一待识别文本作为预设文本，与该第一读音预测信息关联存储至文本读音库。
其中，上述文本读音库中包含多个预设文本以及每个预设文本对应的读音预测信息，每个预设文本中包含至少一个多音字。
为了便于理解，上述文本读音库可以称为纠错干预库。
可选地，在本申请实施例中，上述文本读音库中存储的数据的数据格式为：<文本>|<带目标多音字拼音的文本>。例如，文本读音库中存储的数据包括：“他的穿着很普通|他的穿着(zhuo2)很普通”，“差点着了他的道|差点着zhao1了他的道”，“家里着火了|家里着(zhao2)火了”。
如此，通过对累积的多个待识别文本基于句编码结果进行读音预测，从而得到更为准确的读音预测结果，并且，将得到的更为准确的读音预测信息预存至文本读音库中，使得后续进行在线语音合成时，可以通过文本读音库对基于字编码结果进行读音预测后得到的读音预测结果进行纠正，从而在保证处理速度的同时保证了读音预测的高准确率。
进一步可选地，在本申请实施例中，文本识别装置得到该第一多音字的第一读音预测信息，并将第一待识别文本作为预设文本，与该第一读音预测信息关联存储至文本读音库之后，可以获取文本读音库中错误的读音预测信息，并将错误的读音预测信息添加到上述第一多音字模型的训练数据集，以通过错误的读音预测信息对该第一多音字模型进行训练，从而对模型迭代更新优化。
可选地，文本识别装置可以对文本读音库中保存的预设文本对应的读音预测信息进行检查，在检查到预设文本对应的读音预测信息为错误信息时，将该错误的读音预测信息添加到上述第一多音字模型的训练数据集，以通过错误的读音预测信息对该第一多音字模型进行训练。
示例性地，文本识别装置将错误的读音预测信息分别加入训练集中训练，通过积累的数据集迭代训练多音字模型，修复原本在线朗读系统中读音出错的例子，增强模型的泛化能力，从而实现周期性地提高多音字模型读音预测准确率。
可选地，在本申请实施例中，本申请实施例提供的一种读音预测方法，该读音预测方法可以包括以下步骤A1至步骤A4：
步骤A1：文本识别装置将第二待识别文本输入第二文本编码模型，提取第二待识别文本对应的第三文本特征序列。
其中，上述第二待识别文本包含第二多音字，上述第二文本编码模型与上述第一文本编码模型不同。
步骤A2：文本识别装置基于上述第三文本特征序列，得到上述第二待识别文本对应的第二编码信息。
步骤A3：文本识别装置将上述第二编码信息输入至第二多音字模型，对上述第二多音字进行读音预测，得到上述第二多音字的第三读音预测信息。
步骤A4：文本识别装置在文本读音库中存在目标预设文本的情况下，将上述第三读音预测信息更新为目标预设文本对应的第四读音预测信息。
其中，上述目标预设文本为上述文本读音库中与第二待识别文本匹配的文本。
可选地，在本申请实施例中，上述第二文本编码模型可以为字编码模型。
可选地，在本申请实施例中，该多音字模型可以由双向长短时记忆神经网络和线性神经网络训练获得。
示例性地，在用户需要对文本进行朗读时，用户点击该文本后，编码装置获取文本请求并将该文本请求实时发送到语音合成引擎端，通过语音合成引擎端采用字编码方式对文本编码处理，并将编码后的编码结果输入到第二多音字模型中。
可选地，在本申请实施例中，在进行字编码处理时，语音合成引擎端使用Focal Loss损失函数以减缓数据不平衡带来的影响，从而提高多音字读音准确率,其公式如下：
Focal Loss ＝ -αt(1-pt)γlog (pt)      (4)
其中，通过权重因子α抑制正负样本的数量失衡的情况，权重因子α的取值可以为{0,1}，即，α∈[0，1]；(1-pt)γ为调制因子，辅助难、易样本的区分。通过Focal Loss损失函数，改善了样本类别不均衡问题。
需要说明的是，由于多音字读音的使用频率及场景不同，因此在训练数据中每种读音样本数据难以保持均衡，例如训练集中把的读音“ba3”数据量是“ba4”的十倍，这导致模型预测结果会倾向于预测“ba3”，将部分“ba4”读音预测错误。
可选地，在本申请实施例中，上述第二多音字模型的模型结构及损失函数与上述第一多音字模型相同。
可选地，在本申请实施例中，在得到第二待识别文本基于字编码得到的第三读音预测信息的情况下，将该第二待识别文本与文本读音库中的预设文本进行匹配，在文本读音库中存在与上述第二待识别文本匹配的目标预设文本的情况下，将上述第三读音预测信息更新文本读音库中目标预设文本对应的第四读音预测信息。
进一步可选地，在文本读音库中存在与上述第二待识别文本匹配的目标预设文本的情况下，可以判断该第二待识别文本的第三读音预测信息与目标预设文本对应的第四读音预测信息是否相同，若相同，则输出上述第三读音预测信息，若不同，则将第三读音预测信息更新为第四读音预测信息。
举例说明，以第二待识别文本为“他的穿着很普通”为例，假设第二待识别文本基于字编码得到读音预测结果为“他的穿着(zhe5)很普通”，将该第二待识别文本与文本读音库中的预设文本进行匹配，得到与其匹配的文本对应的读音预测结果为“他的穿着(zhuo2)很普通”，则将该第二待识别文本的读音预测结果纠正为“他的穿着(zhuo2)很普通”。
需要说明的是，读音“zhe5”中的“zhe”为拼音，“5”为声调，即 轻声。同理，读音“zhuo2”中的“zhuo”为拼音，“2”为声调，表示二声。
需要说明的是，上述步骤A1至步骤A4可以在步骤S201之前执行，或者在上述步骤S201之后执行。
本申请实施例提供的文本识别方法，结合了句编码和字编码的优势，在保证低延迟语音朗读服务的同时，能够及时快速纠正错误读音；根据纠错系统的纠错数据，周期性地迭代多音字模型，提高多音字读音准确率，极大减小了人工查错纠错所需的人力和时间成本。
进一步可选地，在本申请实施例中，上述步骤A3之后，本申请实施例提供的文本识别方法还包括以下步骤B1：
步骤B1：文本识别装置在检测到上述第三读音预测信息为错误读音预测信息的情况下，将上述第三读音预测信息添加至训练数据集。
其中，上述训练数据集用于训练上述第二多音字模型。
可选地，在本申请实施例中，在第二待识别文本的第三读音预测信息与从文本读音库中匹配到的第四读音预测信息不同的情况下，将该第三读音预测信息作为错误的读音预测信息添加至上述第二多音字模型的训练集，以对第二多音字模型进行模型迭代更新优化。
示例性地，文本识别装置将基于字编码得到的错误读音预测信息加入根据字编码的编码结果预测读音的多音字模型的训练集中，并基于积累的错误读音预测信息训练该多音字模型，以修复原本在线朗读系统中读音出错的例子，增强模型的泛化能力，从而实现周期性地提高多音字模型读音预测准确率。
以下结合流程图通过具体的实施例对本申请实施例提供的文本识别方法进行说明。
图4为本申请实施例提供的文本识别方法的流程图，如图4所示，该文本识别方法可以包括以下步骤101至步骤109：
步骤101：通过第二文本编码模型对用户请求识别的文本进行字编码。
示例性地，用户点击文本进行语音朗读，此时该文本请求会实时发送到语音合成引擎端，对文本进行字编码处理，并将编码后的结果输入到步骤102的多音字模型中。
步骤102：通过第二多音字模型获得读音预测结果。
需要说明的是，对于该步骤的解释说明可以参见上文描述，此处不再赘述。
步骤103：干预库内匹配查询目标文本。
示例性地，若干预库中存在该文本，匹配结果为True，则使用干库对应的结果，并跳转至步骤104；若干预库中不存在该文本，则匹配结果为False，跳转至步骤105。
步骤104：读音结果干预修复。
示例性地，若步骤104匹配结果为True，则使用干预库中所对应的文 本读音结果，跳转至步骤105，完成错误读音的纠正。
步骤105：输出最终的读音。
步骤106：文本积累。
示例性地，读音纠正系统非实时运行，用户在手机端请求朗读的文本会自动写入固定目录文件D，当请求文本数量达到预先设定的数量时，将会自动启动读音纠正系统，并将所积累的文本送入步骤107。
步骤107：基于大模型DeBERTa进行文本句编码。
示例性地，读取步骤106中积累的文本，使用预训练大模型DeBERTa批量输出文本句编码结果，预训练大模型DeBERTa，即上述第一文本编码模型。
步骤108：基于句编码的第二音字模型获得读音预测结果。
步骤109a：对第一多音字模型进行迭代更新。
步骤109b：对第二多音字模型进行迭代更新。
需要说明的是，对该实施例中的解释说明可以参见上文，此处不再赘述。
本申请实施例提供的文本识别方法，一方面，通过多音字读音自动化纠错系统，能够自动纠错线上语音合成多音字读音，通过改进的DeBERTa预训练模型生成的句编码能够更好地表达和捕捉语境和语义信息，保证了读音的高准确率。另一方面，通过预训练大模型句编码训练的多音字模型指导线上语音合成多音字模型迭代更新，提高线上多音字模型自身的读音准确率。再一方面，通过自动纠错系统进行纠错，避免了人工纠错所需要的高昂时间和人力成本，同时提高了纠错效率。
本申请实施例提供的文本识别方法，执行主体可以为文本识别装置。本申请实施例中以文本识别装置执行文本识别方法为例，说明本申请实施例提供的文本识别装置。
图5为本申请实施例提供的文本识别装置的结构示意图，如图5所示，该文本识别装置500包括执行模块501和处理模块502，其中：
上述执行模块501，用于将第一待识别文本输入第一文本编码模型，上述第一待识别文本包含第一多音字，上述第一文本编码模型包括卷积层；
上述处理模块502，用于提取第一待识别文本对应的第一文本特征序列；
上述处理模块502，还用于采用上述卷积层对上述第一文本特征序列进行分段卷积处理，得到第二文本特征序列，上述第二文本特征序列至少包含上述第一待识别文本的全局语义信息和上述第一待识别文本中的第一文本片段的局部语义信息，上述第一文本片段包含上述第一多音字；
上述处理模块502，还用于基于上述第二文本特征序列，得到上述第一待识别文本对应的第一编码信息，并基于上述第一编码信息获得上述第一待识别文本的识别结果。
可选地，在本申请实施例中，上述处理模块，具体用于通过上述卷积 层中的卷积核对应的卷积窗口，对上述第一文本特征序列进行分段卷积处理，得到N个序列片段对应的文本特征序列，一个上述序列片段中包含的文本特征元素的数量是基于上述卷积窗口的窗长确定的，任一序列片段对应的文本特征序列包含上述任一序列片段对应文本片段的局部语义信息；其中，上述第二文本特征序列由上述N个序列片段组成。
可选地，在本申请实施例中，上述处理模块，具体用于将上述第一编码信息输入至第一多音字模型，对上述第一多音字进行读音预测，得到上述第一多音字的第一读音预测信息。
可选地，在本申请实施例中，上述装置还包括：存储模块；
上述存储模块，用于在上述处理模块将上述第一编码信息输入至第一多音字模型，对上述第一多音字进行读音预测，得到上述第一多音字的第一读音预测信息之后，将上述第一待识别文本作为预设文本，与上述第一读音预测信息关联存储至文本读音库；
其中，上述文本读音库中包含多个预设文本以及每个预设文本对应的读音预测信息，每个上述预设文本中包含至少一个多音字。
可选地，在本申请实施例中，上述处理模块，还用于将第二待识别文本输入第二文本编码模型，提取上述第二待识别文本对应的第三文本特征序列，第二待识别文本包含第二多音字，上述第二文本编码模型与上述第一文本编码模型不同；上述处理模块，还用于基于上述第三文本特征序列，得到上述第二待识别文本对应的第二编码信息；上述处理模块，还用于将上述第二编码信息输入至第二多音字模型，对上述第二多音字进行读音预测，得到上述第二多音字的第三读音预测信息；上述装置还包括：更新模块；上述更新模块，用于在上述文本读音库中存在目标预设文本的情况下，将上述第三读音预测信息更新为上述目标预设文本对应的第四读音预测信息；其中，上述目标预设文本为上述文本读音库中与上述第二待识别文本匹配的文本。
可选地，在本申请实施例中，上述处理模块，还用于在检测到上述第三读音预测信息为错误读音预测信息的情况下，将上述第三读音预测信息添加至训练数据集；其中，上述训练数据集用于训练上述第二多音字模型。
可选地，在本申请实施例中，上述装置还包括：接收模块和存储模块，其中：上述接收模块，用于接收用户的第一输入；上述存储模块，用于响应于上述接收模块接收的上述第一输入，将上述第一输入对应的待识别文本存储至第一文件中；上述处理模块，还用于在上述第一文件中存储的待识别文本的数量超过第一阈值的情况下，将上述第一文件中存储的待识别文本合成为上述第一待输入文本。
本申请实施例提供的文本识别装置，文本识别装置将第一待识别文本输入第一文本编码模型，该第一待识别文本包含第一多音字，该第一文本编码模型包括卷积层，然后，提取第一待识别文本对应的第一文本特征序列，并采用上述卷积层对上述第一文本特征序列进行分段卷积处理，得到 第二文本特征序列，上述第二文本特征序列至少包含上述第一待识别文本的全局语义信息和第一待识别文本中的第一文本片段的局部语义信息，上述第一文本片段包含第一多音字，最后，基于上述第二文本特征序列，得到第一待识别文本对应的第一编码信息，并基于上述第一编码信息获得第一待识别文本的识别结果。通过该方法，文本识别装置通过第一文本编码模型中的卷积层对文本的全局文本特征序列进行卷积操作获得包含全局语义信息和局部语义信息的文本特征序列，并基于该文本特征序列得到更为准确的识别结果，从而加强对文本序列局部信息建模能力，能够弥补现有模型对于局部特征信息提取的不足，从而能够更为准确地分析长难句、复杂文本中多音字的读音，进而提高对文本中的多音字读音预测的准确率。
本申请实施例中的文本识别装置可以是电子设备，也可以是电子设备中的部件，例如集成电路或芯片。该电子设备可以是终端，也可以为除终端之外的其他设备。示例性的，电子设备可以为手机、平板电脑、笔记本电脑、掌上电脑、车载电子设备、移动上网装置(Mobile Internet Device，MID)、增强现实(augmented reality，AR)/虚拟现实(virtual reality，VR)设备、机器人、可穿戴设备、超级移动个人计算机(ultra-mobile personal computer，UMPC)、上网本或者个人数字助理(personal digital assistant，PDA)等，还可以为服务器、网络附属存储器(Network Attached Storage，NAS)、个人计算机(personal computer，PC)、电视机(television，TV)、柜员机或者自助机等，本申请实施例不作具体限定。
本申请实施例中的文本识别装置可以为具有操作系统的装置。该操作系统可以为安卓(Android)操作系统，可以为ios操作系统，还可以为其他可能的操作系统，本申请实施例不作具体限定。
本申请实施例提供的文本识别装置能够实现图1至图4的方法实施例实现的各个过程，为避免重复，这里不再赘述。
可选地，如图6所示，本申请实施例还提供一种电子设备600，包括处理器601和存储器602，存储器602上存储有可在上述处理器601上运行的程序或指令，该程序或指令被处理器601执行时实现上述文本识别方法实施例的各个步骤，且能达到相同的技术效果，为避免重复，这里不再赘述。
需要说明的是，本申请实施例中的电子设备包括上述的移动电子设备和非移动电子设备。
图7为实现本申请实施例的一种电子设备的硬件结构示意图。
该电子设备1000包括但不限于：射频单元1001、网络模块1002、音频输出单元1003、输入单元100、传感器1005、显示单元1006、用户输入单元1007、接口单元1008、存储器1009、以及处理器1010等部件。
本领域技术人员可以理解，电子设备1000还可以包括给各个部件供电的电源(比如电池)，电源可以通过电源管理系统与处理器1010逻辑相连，从而通过电源管理系统实现管理充电、放电、以及功耗管理等功能。图7 中示出的电子设备结构并不构成对电子设备的限定，电子设备可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置，在此不再赘述。
其中，上述处理器1010，用于将第一待识别文本输入第一文本编码模型，上述第一待识别文本包含第一多音字，上述第一文本编码模型包括卷积层；
上述处理器1010，用于提取第一待识别文本对应的第一文本特征序列；
上述处理器1010，还用于采用上述卷积层对上述第一文本特征序列进行分段卷积处理，得到第二文本特征序列，上述第二文本特征序列至少包含上述第一待识别文本的全局语义信息和上述第一待识别文本中的第一文本片段的局部语义信息，上述第一文本片段包含上述第一多音字；
上述处理器1010，还用于基于上述第二文本特征序列，得到上述第一待识别文本对应的第一编码信息，并基于上述第一编码信息获得上述第一待识别文本的识别结果。
可选地，在本申请实施例中，上述处理器1010，具体用于通过上述卷积层中的卷积核对应的卷积窗口，对上述文本特征序列进行分段卷积处理，得到N个序列片段对应的文本特征序列，任一序列片段对应的文本特征序列包含上述任一序列片段对应文本片段的局部语义信息；其中，上述第二文本特征序列由上述N个序列片段组成。
可选地，在本申请实施例中，上述处理器1010，具体用于将上述第一编码信息输入至第一多音字模型，对上述第一多音字进行读音预测，得到上述第一多音字的第一读音预测信息。
可选地，在本申请实施例中，上述存储器1009，用于在将上述第一编码信息输入至第一多音字模型，对上述第一多音字进行读音预测，得到上述第一多音字的第一读音预测信息之后，将上述第一待识别文本作为预设文本，与上述第一读音预测信息关联存储至文本读音库；
其中，上述文本读音库中包含多个预设文本以及每个预设文本对应的读音预测信息，每个上述预设文本中包含至少一个多音字。
可选地，在本申请实施例中，上述处理器1010，还用于将第二待识别文本输入第二文本编码模型，提取上述第二待识别文本对应的第三文本特征序列，第二待识别文本包含第二多音字，上述第二文本编码模型与上述第一文本编码模型不同；上述处理器1010，还用于基于上述第三文本特征序列，得到上述第二待识别文本对应的第二编码信息；上述处理器1010，还用于将上述第二编码信息输入至第二多音字模型，对上述第二多音字进行读音预测，得到上述第二多音字的第三读音预测信息；上述装置还包括：更新模块；上述更新模块，用于在上述文本读音库中存在目标预设文本的情况下，将上述第三读音预测信息更新为上述目标预设文本对应的第四读音预测信息；其中，上述目标预设文本为上述文本读音库中与上述第二待识别文本匹配的文本。
可选地，在本申请实施例中，上述处理器1010，还用于在检测到上述第三读音预测信息为错误读音预测信息的情况下，将上述第三读音预测信息添加至训练数据集；其中，上述训练数据集用于训练上述第二多音字模型。
可选地，在本申请实施例中，上述用户输入单元1007，用于接收用户的第一输入；上述存储器1009，用于响应于上述用户输入单元1007接收的上述第一输入，将上述第一输入对应的待识别文本存储至第一文件中；上述处理器1010，还用于在上述第一文件中存储的待识别文本的数量超过第一阈值的情况下，将上述第一文件中存储的待识别文本合成为上述第一待输入文本。
本申请实施例提供的电子设备，电子设备将第一待识别文本输入第一文本编码模型，该第一待识别文本包含第一多音字，该第一文本编码模型包括卷积层，然后，提取第一待识别文本对应的第一文本特征序列，并采用上述卷积层对上述第一文本特征序列进行分段卷积处理，得到第二文本特征序列，上述第二文本特征序列至少包含上述第一待识别文本的全局语义信息和第一待识别文本中的第一文本片段的局部语义信息，上述第一文本片段包含第一多音字，最后，基于上述第二文本特征序列，得到第一待识别文本对应的第一编码信息，并基于上述第一编码信息获得第一待识别文本的识别结果。通过该方法，文本识别装置通过第一文本编码模型中的卷积层对文本的全局文本特征序列进行卷积操作获得包含全局语义信息和局部语义信息的文本特征序列，并基于该文本特征序列得到更为准确的识别结果，从而加强对文本序列局部信息建模能力，能够弥补现有模型对于局部特征信息提取的不足，从而能够更为准确地分析长难句、复杂文本中多音字的读音，进而提高对文本中的多音字读音预测的准确率。
应理解的是，本申请实施例中，输入单元1004可以包括图形处理器(Graphics Processing Unit，GPU)1041和麦克风1042，图形处理器1041对在视频捕获模式或图像捕获模式中由图像捕获装置(如摄像头)获得的静态图片或视频的图像数据进行处理。显示单元1006可包括显示面板1061，可以采用液晶显示器、有机发光二极管等形式来配置显示面板1061。用户输入单元1007包括触控面板1071以及其他输入设备1072中的至少一种。触控面板1071，也称为触摸屏。触控面板1071可包括触摸检测装置和触摸控制器两个部分。其他输入设备1072可以包括但不限于物理键盘、功能键(比如音量控制按键、开关按键等)、轨迹球、鼠标、操作杆，在此不再赘述。
存储器1009可用于存储软件程序以及各种数据。存储器1009可主要包括存储程序或指令的第一存储区和存储数据的第二存储区，其中，第一存储区可存储操作系统、至少一个功能所需的应用程序或指令(比如声音播放功能、图像播放功能等)等。此外，存储器1009可以包括易失性存储器或非易失性存储器，或者，存储器1009可以包括易失性和非易失性存储 器两者。其中，非易失性存储器可以是只读存储器(Read-Only Memory，ROM)、可编程只读存储器(Programmable ROM，PROM)、可擦除可编程只读存储器(Erasable PROM，EPROM)、电可擦除可编程只读存储器(Electrically EPROM，EEPROM)或闪存。易失性存储器可以是随机存取存储器(Random Access Memory，RAM)，静态随机存取存储器(Static RAM，SRAM)、动态随机存取存储器(Dynamic RAM，DRAM)、同步动态随机存取存储器(Synchronous DRAM，SDRAM)、双倍数据速率同步动态随机存取存储器(Double Data Rate SDRAM，DDRSDRAM)、增强型同步动态随机存取存储器(Enhanced SDRAM，ESDRAM)、同步连接动态随机存取存储器(Synch link DRAM，SLDRAM)和直接内存总线随机存取存储器(Direct Rambus RAM，DRRAM)。本申请实施例中的存储器1009包括但不限于这些和任意其它适合类型的存储器。
处理器1010可包括一个或多个处理单元；可选的，处理器1010集成应用处理器和调制解调处理器，其中，应用处理器主要处理涉及操作系统、用户界面和应用程序等的操作，调制解调处理器主要处理无线通信信号，如基带处理器。可以理解的是，上述调制解调处理器也可以不集成到处理器1010中。
本申请实施例还提供一种可读存储介质，所述可读存储介质上存储有程序或指令，该程序或指令被处理器执行时实现上述文本识别方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
其中，所述处理器为上述实施例中所述的电子设备中的处理器。所述可读存储介质，包括计算机可读存储介质，如计算机只读存储器ROM、随机存取存储器RAM、磁碟或者光盘等。
本申请实施例另提供了一种芯片，所述芯片包括处理器和通信接口，所述通信接口和所述处理器耦合，所述处理器用于运行程序或指令，实现上述文本识别方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
应理解，本申请实施例提到的芯片还可以称为系统级芯片、系统芯片、芯片系统或片上系统芯片等。
本申请实施例提供一种计算机程序产品，该程序产品被存储在存储介质中，该程序产品被至少一个处理器执行以实现如上述文本识别方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。此外，需要指出的是，本申请实施方式中的方法和装置的范围不限按示出或讨论的 顺序来执行功能，还可包括根据所涉及的功能按基本同时的方式或按相反的顺序来执行功能，例如，可以按不同于所描述的次序来执行所描述的方法，并且还可以添加、省去、或组合各种步骤。另外，参照某些示例所描述的特征可在其他示例中被组合。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以计算机软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端(可以是手机，计算机，服务器，或者网络设备等)执行本申请各个实施例所述的方法。
上面结合附图对本申请的实施例进行了描述，但是本申请并不局限于上述的具体实施方式，上述的具体实施方式仅仅是示意性的，而不是限制性的，本领域的普通技术人员在本申请的启示下，在不脱离本申请宗旨和权利要求所保护的范围情况下，还可做出很多形式，均属于本申请的保护之内。

Claims (18)

  1. 一种文本识别方法，所述方法包括：
    将第一待识别文本输入第一文本编码模型，所述第一待识别文本包含第一多音字，所述第一文本编码模型包括卷积层；
    提取第一待识别文本对应的第一文本特征序列，所述第一文本特征序列包含所述第一待识别文本的语义信息；
    采用所述卷积层对所述第一文本特征序列进行分段卷积处理，得到第二文本特征序列，所述第二文本特征序列至少包含所述第一待识别文本的全局语义信息和所述第一待识别文本中的第一文本片段的局部语义信息，所述第一文本片段包含所述第一多音字；
    基于所述第二文本特征序列，得到所述第一待识别文本对应的第一编码信息，并基于所述第一编码信息获得所述第一待识别文本的识别结果。
  2. 根据权利要求1所述的方法，其中，所述采用所述卷积层对所述第一文本特征序列进行卷积处理，得到第二文本特征序列，包括：
    通过所述卷积层中的卷积核对应的卷积窗口，对所述第一文本特征序列进行分段卷积处理，得到N个序列片段，任一序列片段对应的文本特征序列包含所述任一序列片段对应文本片段的局部语义信息；
    其中，所述第二文本特征序列由所述N个序列片段组成。
  3. 根据权利要求1或2所述的方法，其中，所述基于所述第一编码信息获得所述第一待识别文本的识别结果，包括：
    将所述第一编码信息输入至第一多音字模型，对所述第一多音字进行读音预测，得到所述第一多音字的第一读音预测信息。
  4. 根据权利要求3所述的方法，其中，所述将所述第一编码信息输入至第一多音字模型，对所述第一多音字进行读音预测，得到所述第一多音字的第一读音预测信息之后，所述方法还包括：
    将所述第一待识别文本作为预设文本，与所述第一读音预测信息关联存储至文本读音库；
    其中，所述文本读音库中包含多个预设文本以及每个预设文本对应的读音预测信息，每个所述预设文本中包含至少一个多音字。
  5. 根据权利要求4所述的方法，其中，所述方法还包括：
    将第二待识别文本输入第二文本编码模型，提取所述第二待识别文本对应的第三文本特征序列，第二待识别文本包含第二多音字，所述第二文本编码模型与所述第一文本编码模型不同；
    基于所述第三文本特征序列，得到所述第二待识别文本对应的第二编码信息；
    将所述第二编码信息输入至第二多音字模型，对所述第二多音字进行读音预测，得到所述第二多音字的第三读音预测信息；
    在所述文本读音库中存在目标预设文本的情况下，将所述第三读音预测信息更新为所述目标预设文本对应的第四读音预测信息；其中，所述目 标预设文本为所述文本读音库中与所述第二待识别文本匹配的文本。
  6. 根据权利要求5所述的方法，其中，所述基于所述第二编码信息，得到所述第二多音字的第三读音预测信息之后，所述方法还包括：
    在检测到所述第三读音预测信息为错误读音预测信息的情况下，将所述第三读音预测信息添加至训练数据集；
    其中，所述训练数据集用于训练所述第二多音字模型。
  7. 根据权利要求1所述的方法，其中，所述将第一待识别文本输入文本编码模型之前，所述方法还包括：
    接收用户的第一输入；
    响应于所述第一输入，将所述第一输入对应的待识别文本存储至第一文件中；
    在所述第一文件中存储的待识别文本的数量超过第一阈值的情况下，将所述第一文件中存储的待识别文本合成为所述第一待输入文本。
  8. 一种文本识别装置，所述装置包括：执行模块和处理模块，其中：
    所述执行模块，用于将第一待识别文本输入第一文本编码模型，所述第一待识别文本包含第一多音字，所述第一文本编码模型包括卷积层；
    所述处理模块，用于提取第一待识别文本对应的第一文本特征序列；
    所述处理模块，还用于采用所述卷积层对所述第一文本特征序列进行分段卷积处理，得到第二文本特征序列，所述第二文本特征序列至少包含所述第一待识别文本的全局语义信息和所述第一待识别文本中的第一文本片段的局部语义信息，所述第一文本片段包含所述第一多音字；
    所述处理模块，还用于基于所述第二文本特征序列，得到所述第一待识别文本对应的第一编码信息，并基于所述第一编码信息获得所述第一待识别文本的识别结果。
  9. 根据权利要求8所述的装置，其中，所述处理模块，具体用于通过所述卷积层中的卷积核对应的卷积窗口，对所述第一文本特征序列进行分段卷积处理，得到N个序列片段，任一序列片段对应的文本特征序列包含所述任一序列片段对应文本片段的局部语义信息；
    其中，所述第二文本特征序列由所述N个序列片段组成。
  10. 根据权利要求8或9所述的装置，其中，所述处理模块，具体用于将所述第一编码信息输入至第一多音字模型，对所述第一多音字进行读音预测，得到所述第一多音字的第一读音预测信息。
  11. 根据权利要求10所述的装置，其中，所述装置还包括：存储模块；
    所述存储模块，用于在所述处理模块将所述第一编码信息输入至第一多音字模型，对所述第一多音字进行读音预测，得到所述第一多音字的第一读音预测信息之后，将所述第一待识别文本作为预设文本，与所述第一读音预测信息关联存储至文本读音库；
    其中，所述文本读音库中包含多个预设文本以及每个预设文本对应的读音预测信息，每个所述预设文本中包含至少一个多音字。
  12. 根据权利要求11所述的装置，其中，所述处理模块，还用于将第二待识别文本输入第二文本编码模型，提取所述第二待识别文本对应的第三文本特征序列，第二待识别文本包含第二多音字，所述第二文本编码模型与所述第一文本编码模型不同；
    所述处理模块，还用于基于所述第三文本特征序列，得到所述第二待识别文本对应的第二编码信息；
    所述处理模块，还用于将所述第二编码信息输入至第二多音字模型，对所述第二多音字进行读音预测，得到所述第二多音字的第三读音预测信息；
    所述装置还包括：更新模块；
    所述更新模块，用于在所述文本读音库中存在目标预设文本的情况下，将所述第三读音预测信息更新为所述目标预设文本对应的第四读音预测信息；
    其中，所述目标预设文本为所述文本读音库中与所述第二待识别文本匹配的文本。
  13. 根据权利要求12所述的装置，其中，所述处理模块，还用于在检测到所述第三读音预测信息为错误读音预测信息的情况下，将所述第三读音预测信息添加至训练数据集；
    其中，所述训练数据集用于训练所述第二多音字模型。
  14. 根据权利要求8所述的装置，其中，所述装置还包括：接收模块和存储模块，其中：
    所述接收模块，用于接收用户的第一输入；
    所述存储模块，用于响应于所述接收模块接收的所述第一输入，将所述第一输入对应的待识别文本存储至第一文件中；
    所述处理模块，还用于在所述第一文件中存储的待识别文本的数量超过第一阈值的情况下，将所述第一文件中存储的待识别文本合成为所述第一待输入文本。
  15. 一种电子设备，包括处理器和存储器，所述存储器存储可在所述处理器上运行的程序或指令，所述程序或指令被所述处理器执行时实现如权利要求1-7任一项所述的文本识别方法的步骤。
  16. 一种可读存储介质，所述可读存储介质上存储程序或指令，所述程序或指令被处理器执行时实现如权利要求1-7任一项所述的文本识别方法的步骤。
  17. 一种计算机程序产品，所述计算机程序产品被存储在存储介质中，所述计算机程序产品被至少一个处理器执行以实现如权利要求1至7任一项所述的文本识别方法。
  18. 一种芯片，所述芯片包括处理器和通信接口，所述通信接口和所述处理器耦合，所述处理器用于运行程序或指令，实现如权利要求1至7任一项所述的文本识别方法。
PCT/CN2024/104175 2023-07-14 2024-07-08 文本识别方法、装置、电子设备及可读存储介质 Pending WO2025016237A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310873503.9 2023-07-14
CN202310873503.9A CN116861851A (zh) 2023-07-14 2023-07-14 文本识别方法、装置、电子设备及可读存储介质

Publications (1)

Publication Number Publication Date
WO2025016237A1 true WO2025016237A1 (zh) 2025-01-23

Family

ID=88221287

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/104175 Pending WO2025016237A1 (zh) 2023-07-14 2024-07-08 文本识别方法、装置、电子设备及可读存储介质

Country Status (2)

Country Link
CN (1) CN116861851A (zh)
WO (1) WO2025016237A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116861851A (zh) * 2023-07-14 2023-10-10 维沃移动通信有限公司 文本识别方法、装置、电子设备及可读存储介质
WO2025111923A1 (zh) * 2023-11-30 2025-06-05 京东方科技集团股份有限公司 文本识别方法、装置、电子设备和可读存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050192807A1 (en) * 2004-02-26 2005-09-01 Ossama Emam Hierarchical approach for the statistical vowelization of Arabic text
CN110277085A (zh) * 2019-06-25 2019-09-24 腾讯科技（深圳）有限公司 确定多音字发音的方法及装置
CN112989821A (zh) * 2021-04-13 2021-06-18 北京世纪好未来教育科技有限公司 多音字的注音方法及计算机存储介质
CN113823259A (zh) * 2021-07-22 2021-12-21 腾讯科技（深圳）有限公司 将文本数据转换为音素序列的方法及设备
CN116861851A (zh) * 2023-07-14 2023-10-10 维沃移动通信有限公司 文本识别方法、装置、电子设备及可读存储介质

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110889290B (zh) * 2019-11-13 2021-11-16 北京邮电大学 文本编码方法和设备、文本编码有效性检验方法和设备
CN111798834B (zh) * 2020-07-03 2022-03-15 北京字节跳动网络技术有限公司 多音字的识别方法、装置、可读介质和电子设备
CN115512682B (zh) * 2022-09-19 2025-07-18 北京世纪好未来教育科技有限公司 多音字读音预测方法、装置、电子设备及存储介质
CN115547295B (zh) * 2022-10-11 2025-11-14 平安银行股份有限公司 理财产品销售语音的播报方法及其系统、计算机设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050192807A1 (en) * 2004-02-26 2005-09-01 Ossama Emam Hierarchical approach for the statistical vowelization of Arabic text
CN110277085A (zh) * 2019-06-25 2019-09-24 腾讯科技（深圳）有限公司 确定多音字发音的方法及装置
CN112989821A (zh) * 2021-04-13 2021-06-18 北京世纪好未来教育科技有限公司 多音字的注音方法及计算机存储介质
CN113823259A (zh) * 2021-07-22 2021-12-21 腾讯科技（深圳）有限公司 将文本数据转换为音素序列的方法及设备
CN116861851A (zh) * 2023-07-14 2023-10-10 维沃移动通信有限公司 文本识别方法、装置、电子设备及可读存储介质

Also Published As

Publication number Publication date
CN116861851A (zh) 2023-10-10

Similar Documents

Publication Publication Date Title
US12008336B2 (en) Multimodal translation method, apparatus, electronic device and computer-readable storage medium
US11238845B2 (en) Multi-dialect and multilingual speech recognition
CN111581229B (zh) Sql语句的生成方法、装置、计算机设备及存储介质
CN110457713B (zh) 基于机器翻译模型的翻译方法、装置、设备和存储介质
CN112528637B (zh) 文本处理模型训练方法、装置、计算机设备和存储介质
CN105068998B (zh) 基于神经网络模型的翻译方法及装置
CN110415679B (zh) 语音纠错方法、装置、设备和存储介质
JP2023509031A (ja) マルチモーダル機械学習に基づく翻訳方法、装置、機器及びコンピュータプログラム
CN113627158B (zh) 基于多表征和多预训练模型的中文拼写纠错方法及装置
WO2025016237A1 (zh) 文本识别方法、装置、电子设备及可读存储介质
CN111062217A (zh) 语言信息的处理方法、装置、存储介质及电子设备
CN110600002A (zh) 语音合成方法、装置及电子设备
CN115587583A (zh) 噪声的检测方法、装置及电子设备
CN117709367A (zh) 翻译方法及相关设备
CN119443087A (zh) 文本纠错方法、装置、设备及存储介质
CN116579327A (zh) 文本纠错模型训练方法、文本纠错方法、设备及存储介质
CN115563959A (zh) 面向中文拼音拼写纠错的自监督预训练方法、系统及介质
CN118733758A (zh) 优化提示词的方法、装置、设备及介质
CN112559725A (zh) 文本匹配方法、装置、终端和存储介质
CN112926334B (zh) 确定词表示向量的方法、装置及电子设备
CN116013307A (zh) 一种标点预测方法、装置、设备以及计算机存储介质
CN113012685A (zh) 音频识别方法、装置、电子设备及存储介质
CN119089395B (zh) 多模态信息提取方法、装置、计算机设备及存储介质
CN112989821B (zh) 多音字的注音方法及计算机存储介质
CN114580446A (zh) 基于文档上下文的神经机器翻译方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24842232

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE