CN115168588A

CN115168588A - Text processing method and device

Info

Publication number: CN115168588A
Application number: CN202210901566.6A
Authority: CN
Inventors: 弓源; 李长亮
Original assignee: Beijing Kingsoft Digital Entertainment Co Ltd
Current assignee: Beijing Kingsoft Digital Entertainment Co Ltd
Priority date: 2022-03-16
Filing date: 2022-03-16
Publication date: 2022-10-11
Also published as: CN114328848B; CN114328848A

Abstract

The application provides a text processing method and a text processing device, wherein the text processing method comprises the following steps: performing retranslation processing on the written language text to obtain a retranslated written language text corresponding to the written language text; determining a written language text to be processed in the written language text and the retranslated written language text, and sequentially performing conversion processing of sentence forming units on the written language text to be processed according to a plurality of conversion processing strategies of a preset execution sequence to obtain a spoken language text; constructing sample linguistic data based on the corresponding relation between the written language text and the retranslated written language text and the spoken language text respectively; and training the initial written language rewriting model through the sample corpus until the written language rewriting model meeting the training stopping condition is obtained.

Description

Text processing method and device

Technical Field

The present application relates to the field of artificial intelligence of information technology, and in particular, to a text processing method, and also relates to a text processing apparatus, a computing device, and a computer-readable storage medium.

Background

Artificial Intelligence (AI) refers to the ability of an engineered (i.e., designed and manufactured) system to perceive the environment, as well as the ability to acquire, process, apply, and represent knowledge. The development conditions of key technologies in the field of artificial intelligence comprise key technologies such as machine learning, knowledge maps, natural language processing, computer vision, human-computer interaction, biological feature recognition, virtual reality/augmented reality and the like. Natural Language Processing (NLP) refers to the operation and Processing of information such as the shape, sound, meaning, etc. of Natural Language, i.e. the input, output, recognition, analysis, understanding, generation, etc. of characters, words, sentences and sections, by using a computer.

The text generation task is an important research direction in the field of natural language processing, wherein machine translation, abstract generation, text style migration and the like are important tasks in the field of natural text generation. The rewriting of the spoken language text-written language text is one of the important tasks in the field of natural text generation, and has important application in daily work and life. For example, in analysis application scenarios involving spoken texts, such as analysis of recorded texts, conference speech text summary, and transfer of important written language material documents, the transfer quality of the transfer of spoken texts into written language texts is of great importance. However, the task of transferring the spoken text to the written language text is a huge challenge due to the lack of sample corpus of the spoken text-written language text and the influence of many factors such as great uncertainty and discontinuity of the text generation task to generate the result text.

Disclosure of Invention

In view of this, embodiments of the present application provide a text processing method, and the present application also relates to a text processing apparatus, a computing device, and a computer readable storage medium, so as to solve technical defects existing in the prior art.

According to a first aspect of embodiments of the present application, there is provided a text processing method, including:

performing retranslation processing on the written language text to obtain a retranslated written language text corresponding to the written language text;

determining a written language text to be processed in the written language text and the retranslated written language text, and sequentially performing conversion processing of sentence forming units on the written language text to be processed according to a plurality of conversion processing strategies of a preset execution sequence to obtain a spoken language text;

constructing a sample corpus based on the corresponding relation between the written language text and the retranslated written language text and the spoken language text;

and training the initial written language rewriting model through the sample corpus until the written language rewriting model meeting the training stopping condition is obtained.

According to a second aspect of embodiments of the present application, there is provided a text processing apparatus including:

the first interpretation module is configured to perform interpretation processing on the written language text to obtain a back-translated written language text corresponding to the written language text;

the first conversion module is configured to determine a written language text to be processed in the written language text and the retranslated written language text, and sequentially perform conversion processing of sentence forming units on the written language text to be processed according to a plurality of conversion processing strategies of a preset execution sequence to obtain a spoken language text;

a construction module configured to construct a sample corpus based on correspondence of the written language text and the translated back written language text with the spoken language text;

and the model training module is configured to train the initial written language rewriting model through the sample corpus until the written language rewriting model meeting the training stopping condition is obtained.

According to a third aspect of embodiments herein, there is provided a computing device comprising a memory, a processor and computer instructions stored on the memory and executable on the processor, the processor implementing the steps of the text processing method when executing the computer instructions.

According to a fourth aspect of embodiments herein, there is provided a computer readable storage medium storing computer instructions which, when executed by a processor, implement the steps of the text processing method.

According to a fifth aspect of embodiments of the present application, there is provided a chip storing computer instructions which, when executed by the chip, implement the steps of the text processing method.

In the embodiment of the application, the written language text is obtained and is subjected to the retranslation treatment, so that the retranslation written language text corresponding to the written language text is obtained, and the written language material is expanded in a retranslation mode. In order to further expand the language materials, the written language text to be processed can be determined in the written language text and the retranslated written language text, then, each conversion processing strategy is executed in sequence according to a plurality of conversion processing strategies of a preset execution sequence, the conversion processing of sentence forming units is realized on the written language text to be processed, so that the spoken language text is obtained, and the richness of the language materials is improved in the preprocessing stage. And then constructing a sample corpus based on the corresponding relation between the written language text and the translated written language text and the spoken language text. The method and the device have the advantages that the written language corpus expanded based on the translation and the converted spoken language text provide a large amount of sample corpora of the spoken text-written language text for model training, and the initial written language rewriting model is trained into the written language rewriting model meeting requirements on the basis of the sample corpora, so that the training difficulty of the model is simplified, time and labor are avoided being consumed manually to collect and process a large amount of text data, and time cost and labor cost are saved.

Drawings

FIG. 1 is a block diagram of a computing device provided by an embodiment of the present application;

FIG. 2 is a diagram illustrating a text processing method according to an embodiment of the present application;

FIG. 3 is a flowchart of a text processing method according to an embodiment of the present application;

FIG. 4 is a diagram illustrating written language rewrite in a text processing method according to an embodiment of the present application;

FIG. 5 is a flow chart of a method for generating spoken text according to an embodiment of the present application;

fig. 6 is a processing flow chart of a text processing method applied to an actual scene according to an embodiment of the present application;

FIG. 7 is a schematic structural diagram of a text processing apparatus according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a spoken text generation apparatus according to an embodiment of the present application.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. This application is capable of implementation in many different ways than those herein set forth and of similar import by those skilled in the art without departing from the spirit of this application and is therefore not limited to the specific implementations disclosed below.

The terminology used in the one or more embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of the present application. As used in one or more embodiments of the present application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used in one or more embodiments of the present application is intended to encompass any and all possible combinations of one or more of the associated listed items.

It will be understood that, although the terms first, second, etc. may be used herein in one or more embodiments of the present application to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, a first aspect may be termed a second aspect, and, similarly, a second aspect may be termed a first aspect, without departing from the scope of one or more embodiments of the present application. The word "if," as used herein, may be interpreted as "responsive to a determination," depending on the context.

First, the noun terms to which one or more embodiments of the present invention relate are explained.

Seq2Seq (Sequence to Sequence) model: a series of machine learning methods for natural language processing are commonly used in application fields such as machine translation, image description, dialogue models and text summarization.

Transformer model: a deep learning model adopts an attention mechanism to carry out differential weighting on the importance of each part of input data, and is widely applied to various natural language processing tasks.

Text classification: meaning that in a given classification system, text is assigned to be classified into one or several categories.

Natural Language Generation (NLG, natural Language Generation): as part of natural language processing, natural language text is generated from a machine expression system such as a knowledge base or logical form.

Text style migration: text in one stylistic form is transcribed to produce text in another stylistic form.

And (3) abstract generation: through the technical scheme, the process of compressing, summarizing and summarizing the long text is realized, so that the short text with the generalized meaning is formed.

And (3) machine translation: a process for converting one natural language (source language) to another natural language (target language) using a computer.

Entity: refers to a description of a word or phrase of an entity having a particular meaning in the text.

Part of speech tagging: the method is a process of judging the grammar category of each word in a given sentence, determining the part of speech of each word and labeling, and is also a very important basic work in Natural Language Processing (NLP).

And (3) syntactic analysis: is one of the key underlying technologies in Natural Language Processing (NLP), and the basic task is to determine the syntactic structure of a sentence or the dependency relationship between words in the sentence.

In the present application, a text processing method is provided, and the present application relates to a text processing apparatus, a spoken language generating method, a spoken language generating apparatus, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

FIG. 1 shows a block diagram of a computing device 100 according to an embodiment of the present application. The components of the computing device 100 include, but are not limited to, a memory 110 and a processor 120. The processor 120 is coupled to the memory 110 via a bus 130 and a database 150 is used to store data.

Computing device 100 also includes access device 140, access device 140 enabling computing device 100 to communicate via one or more networks 160. Examples of such networks include the Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the internet. Access device 140 may include one or more of any type of network interface (e.g., a Network Interface Card (NIC)) such as IEEE802, wired or wireless. 11 Wireless Local Area Network (WLAN) wireless interface, global microwave internet access (Wi-MAX) interface, ethernet interface, universal Serial Bus (USB) interface, cellular network interface, bluetooth interface, near Field Communication (NFC) interface, and the like.

In one embodiment of the present application, the above-mentioned components of the computing device 100 and other components not shown in fig. 1 may also be connected to each other, for example, by a bus. It should be understood that the block diagram of the computing device architecture shown in FIG. 1 is for purposes of example only and is not limiting as to the scope of the present application. Those skilled in the art may add or replace other components as desired.

Computing device 100 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., tablet, personal digital assistant, laptop, notebook, netbook, etc.), a mobile phone (e.g., smartphone), a wearable computing device (e.g., smartwatch, smartglasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or PC. Computing device 100 may also be a mobile or stationary server.

In practical application, because the change of the spoken text-written language text has important application in daily work and life, in the prior art, in order to realize the change of the spoken text-written language text, the spoken text can be changed into the written and spoken text in an artificial way; or the method of regular rewriting can be adopted to rewrite and replace part of the processable spoken language expression; in addition, the spoken text can be directly translated into written language text by text translation.

The manual rewriting mode is adopted, a large amount of manpower is consumed, and the quality and the result of text transcription are not uniform; the method adopts a regular mode for rewriting, can only process a limited small number of spoken words and fixed text forms, and has higher processing complexity of rewritten logic rules; the method can realize the transcription effect to a certain degree, but is not suitable for the task of transcribing the spoken language to the written language on the whole.

Therefore, in order to accurately rewrite the spoken text-written language text, a written language rewrite model trained in advance can be used to rewrite the spoken text, however, in the process of training the written language rewrite model, a large amount of sample corpora of the spoken text-written language text needs to be used for model training, but in specific implementation, the difficulty of model training for written language rewrite of the spoken text is large due to the lack of sample corpora of the spoken text-written language text. Therefore, an effective solution to solve the above problems is needed.

Referring to fig. 2, fig. 2 is a schematic diagram illustrating a text processing method according to an embodiment of the present application. After the written language text is obtained, in order to further expand the written language text, the written language text can be translated back, and in the translation process, the sentence in the written language text is analyzed according to lexical grammar, and the key entity words in the sentence are returned (replaced) according to the analysis result, so that the obtained translated language material is ensured to be consistent with the key information in the written language text. And then the retranslate linguistic data and the written language text are jointly used as a data source to be input into a spoken language data generating module for spoken language conversion. The spoken data generation module comprises the following steps of converting written language sentences in a data source into clause-level, word-level, character-level and symbol-level sentences.

The clause-level conversion processing comprises the conversion processing of clause repetition, clause generation, clause disorder and the like of written language sentences at the clause level; the word-level conversion processing comprises the conversion processing of adding words, repeating words, disordering words and the like on the written language sentences at the word level; the character-level conversion processing includes conversion processing such as character disordering processing for written-language sentences at the character level, and the symbol-level conversion processing includes conversion processing such as symbol deletion and symbol insertion for written-language sentences at the symbol level.

After the data source is subjected to spoken language conversion through the spoken language data generation module, an initial spoken language text can be obtained, the initial spoken language text is subjected to data cleaning, abnormal information (namely information containing error information, error data or error punctuations) in the initial spoken language text is removed, and the spoken language text corresponding to the data source can be output.

The embodiment of the application provides a written language-spoken language alignment text data generation method by researching, analyzing and summarizing text structures and syntactic and grammatical structural characteristics of spoken texts, the written language texts based on standards are subjected to translation processing, the extended written language texts are converted after being extended, spoken expressions of corresponding written language texts are generated, sample corpora of written language rewriting models are extended, and efficiency and richness of obtaining the sample corpora rewritten by the written language are improved.

Fig. 3 shows a flowchart of a text processing method according to an embodiment of the present application, which specifically includes the following steps:

step 302: and acquiring written language text.

Written language text refers to text formed using the language that people use when writing and reading an article, with words being the main components. The written language text can be written language text in any field, such as written language text in the medical field, written language text in the chemical field, written language text in the sales field, written language text in the daily life field, written language text in the travel field, and the like. And the number of the text of the written language text may be one or more, which is not limited herein.

In the embodiment, the written language text LT, in which the obtained written language text is used as the language body of the art, is taken as an example to explain the text processing method, and the processing procedures of other types of written language texts can refer to the same or similar descriptions in the embodiment, which is not limited herein.

Step 304: and obtaining a retranslate written language text corresponding to the written language text by performing retranslate processing on the written language text.

In specific implementation, considering that the written language text is simply converted to generate the corresponding spoken language text, the expansion of the sample corpus may still be limited, and in order to further expand the sample corpus, the written language text may be expanded in a manner of translating the written language text back, and then the text conversion is performed on the expanded written language text to convert the text into the spoken language text corresponding to the written language text.

The translation processing refers to a process of translating the text in the language a into the language B and then translating the text in the language B back into the language a. In practical applications, since the retraced written language text generated through the retracing process can generate a text expression differentiated from the original written language text, the retraced written language text generated through the retracing process can expand the written language text.

Further, in consideration that the generated retranslate written language text after the retranslation processing may have a large difference from the original written language text and may lose the meaning to be expressed by the original written language text, in order to ensure that the key information in the retranslate written language text and the written language text remains unchanged, the retranslate written language text generated by the retranslation may be replaced by the key words in the written language text, which is specifically implemented in the following manner in the embodiment of the present application:

translating the written language text into a translated text written language text corresponding to a preset language;

the translated written language text is translated back into the target language to which the written language text belongs, and an initial translated written language text is obtained;

and replacing target key words corresponding to the key words in the initial retranslate written language text by the key words in the written language text to obtain the retranslate written language text.

The predetermined language may be any one or more of english, french, korean, german, etc., without limitation. Accordingly, the target language refers to the language to which the characters in the written language text belong.

In practical application, the written language text is firstly translated into text of other languages, namely, the translated written language text. And translating the written text of the translated text back to the language to which the written text of the translated text belongs to obtain the initial translated written text. This initial translation of the written language text may deviate significantly from the expressed meaning of the written language text due to the translation process. In order to keep the key information unchanged for the two texts, the corresponding words (namely the target key words) in the initial translation written language text can be replaced by the key words in the written language text, so that the translation written language text with the key information consistent with the written language text is generated.

The key words can be words which are selected from the written language texts in advance and are considered to be important to the written language texts, and in practical application, the key words can be selected according to a preset selection rule, wherein the preset selection rule can be selected according to parts of speech or entity types of the words. In addition, the key words can be selected through a pre-established key word library, and the words in the key word library contained in the written language text are used as the key words.

In specific implementation, target key words corresponding to the key words in the initial translation written language text are replaced by the key words in the written language text, and the target key words corresponding to the key words need to be determined first. Specifically, the determination manner may be various, for example, the determination manner may be determined according to a position relationship between the key word and the target key word in the text sentence, or may be determined by searching for a corresponding synonym of the key word in the initial translated back written language text, taking the synonym as the target key word, or may be determined by a sentence component to which the key word belongs in the text sentence, taking a word belonging to the same sentence component as the target key word (for example, a subject, a predicate, an object, a fixed language, a resultant, or a complement in the sentence may be taken as the key word, and a word having the same component as the target key word is selected in the initial translated back written language text). In practical application, a suitable mode can be selected according to an actual scene to determine a target key word corresponding to the key word.

After the target key words corresponding to the key words are determined, the corresponding target key words in the initial retranslate written language text are replaced through the key words, and then the retranslate written language text can be obtained.

Following the above example, on the basis of determining that the language to which the written text LT belongs is the chinese language and the preset language is the german language, the written text LT of the chinese language is translated into the german language to obtain the translated written text LT1 of the german language, and then the translated written text LT1 of the german language is translated into: chinese, the initial retraced written language text LT2 of chinese is obtained. It is assumed that the written language text LT contains a written language sentence S1, and the written language sentence S1 is specifically "my hometown is shanxi, where is beautiful". The key words in the written sentence S1 are geographical location entities "shanxi", and under the condition that the written sentence S11 corresponding to the written sentence S1 in the initial retracing written sentence text LT2 is "shanxi is my hometown, which is very beautiful", the target key word corresponding to the key word in the written sentence S11 corresponding to the written sentence S1 is geographical location entities "shanxi", and the "shanxi" in the written sentence S11 is replaced by the "shanxi" to obtain a retracing written sentence text LT3, and the retracing written sentence LT3 includes the written sentence S12 "shanxi" after the written sentence S11 is replaced, which is my hometown, which is very beautiful.

In conclusion, in the retracing process, the corresponding target key words in the retraced written language text are replaced by the key words in the written language text, so that the consistency of the retraced written language text and the key information in the written language text is ensured under the condition of performing language material expansion on the written language text. The accuracy of the back translation of the written language text is improved.

In specific implementation, in consideration that it is important to accurately determine target key words corresponding to the key words for keeping consistency between the translated written language text and text meanings in the written language text, in order to avoid determining that wrong target key words are replaced by the key words, it is possible to ensure that the target key words corresponding to the key words can be accurately obtained and replaced by adding position marks to the key words in the written language text, in the embodiment of the present application, before translating the written language text into the translated written language text corresponding to the preset language, the method further includes:

identifying key words with parts of speech being preset parts of speech in the written language text by analyzing the parts of speech of the written language text;

marking the positions of the key words in the written language text;

correspondingly, target key words corresponding to the key words in the initial retranslate written language text are replaced by the key words in the written language text, and the retranslate written language text is obtained and comprises the following steps:

and replacing corresponding target key words in the initial retranslate written language text by the key words based on the position marks to obtain the retranslate written language text.

Specifically, the part-of-speech analysis of the written language text may be to determine what part-of-speech words are in the written language text by means of part-of-speech tagging of the words in the written language text. The part-of-speech tagging can adopt a part-of-speech tagging method based on rules, a part-of-speech tagging method based on a statistical model, and a part-of-speech tagging method based on a combination of a statistical method and a rule method. Accordingly, the part of speech refers to the characteristic of a word as the basis for dividing the part of speech, and the part of speech may be a noun part of speech, a verb part of speech, an adjective part of speech, a digit part of speech, and the like. In practical applications, since written language texts may belong to different fields, and words with parts of speech considered important in different fields (i.e. key words) may be different, for example, in the chemical field, words with parts of speech of numbers are considered as key words, and in the daily life field, words with parts of speech of nouns are considered as key words.

In specific implementation, the positions of the key words are marked by using signs such as braces { } or asterisks. In practical applications, the position mark may be added at positions before and after the keyword. For example, the key terms are: and the mobile phone marks the position of the keyword through braces, { }, and the marked key word is { mobile phone }.

Before the written language text is translated, the keyword words in the written language text are marked in position. The position mark can be still kept in the initial retranslate written language text obtained after the written language text is retranslated, and the word marked by the position mark in the initial retranslate written language text is the target key word. Namely, the target key words corresponding to the key words can be accurately positioned in a position marking mode, so that the target key words can be accurately replaced. When a plurality of key words exist in one written sentence, the target key word corresponding to the key word may be determined according to the similarity between the word marked by the position mark and the key word, or a mark word having the same sentence component may be determined as the target key word corresponding to the key word according to the component (for example, the sentence component such as subject, predicate, object) of the word marked by the position mark in the sentence.

In addition, in order to facilitate subsequent text processing on the written language text and the replaced text after replacement, the position marks in the written language text and the replaced text can be deleted, and the retranslate written language text can be obtained by deleting the position marks in the replaced text.

Taking a written language sentence S1 in the written language text LT as an example for explanation, performing part-of-speech analysis on the written language sentence S1, and recognizing key words with preset parts-of-speech as nouns in the written language sentence S1 includes: the term "home country" and "shanxi" are marked by the position mark { } to obtain the marked written language sentence S1, and the marked written language sentence S1 is "my { home country } is { shanxi }, which is beautiful there. And the written language sentence S11 in the initial translated written language text LT2 corresponding to the marked written language sentence S1 is "my { hometown } is { shanxi }, which is very beautiful", the "hometown" of the target key word corresponding to the position mark "{ }" in the written language sentence S11 is replaced by "hometown", and the "shanxi" of the target key word corresponding to the position mark "{ }" in the written language sentence S11 is replaced by "shanxi", so that the written language sentence S11 after replacement is "my { hometown } is { shanxi }, which is very beautiful", the position mark in the written language sentence S11 after replacement is deleted, and the written language sentence S12 after deletion is "my hometown is shanxi", which is very beautiful ".

In conclusion, after the keyword words in the written language text are subjected to position marking before translation, the target keyword words are determined and replaced through the position marking, and the replacement accuracy and efficiency are improved.

Step 306: and respectively carrying out conversion processing of sentence composition units on the written language text and the retranslated written language text to obtain a spoken language text.

Specifically, on the basis of obtaining the retraced written language text corresponding to the written language text by performing retracing processing on the written language text, in order to further expand the spoken language text, conversion processing may be performed on the written language text and the retraced written language text, respectively, so as to obtain the corresponding spoken language text.

Optionally, the sentence component unit comprises at least one of: clause unit, word unit, character unit and symbol unit.

In practical applications, since written language text is usually composed of written language sentences, written language sentences are usually composed of a plurality of sentence constituting units, which include: clause units (clauses), word units (words), character units (characters), symbol units (punctuation marks), and the like. Each sentence component unit may have a difference between written language expression and spoken language expression, and therefore, the written language sentence can be converted for each sentence component unit, so that the written language sentence has more characteristics of spoken language expression in clause units, word units, character units, symbol units, and the like.

The clause unit refers to a clause in a written sentence, for example, when the written sentence is "today is sunny, there is no clouds in all, and is suitable for play at home", the written sentence includes 3 clauses, where clause 1 is: "today's weather is sunny", clause 2 is: "Wanliwuyun", clause 3 is: "suitable for play out", the 3 clauses are separated by commas in the written sentence. Correspondingly, word units refer to words in written language sentences. The character unit refers to a character in a written language sentence, and the character can be understood as a word in english or a single word in chinese, and is not limited herein. The symbol unit refers to punctuation marks in written language sentences, such as commas, quotation marks, dashes, and the like, and is not limited herein. In a specific embodiment, the written language sentence in the written language text may be subjected to a treatment such as adjustment or rewriting at a clause level, and/or a treatment such as adjustment or rewriting at a word level, and/or a treatment such as adjustment or rewriting at a character level, and/or a treatment such as adjustment or rewriting at a symbol level, and/or a treatment such as rewriting at a symbol level, and the like.

In specific implementation, the written language text and the translated written language text are used as different written language materials to construct the sample language material, so that the written language text and the translated written language text need to be converted by sentence forming units respectively to obtain corresponding spoken language texts, which is specifically realized by the following method in the embodiment of the application:

performing sentence composition unit conversion processing on the written language text to obtain a first spoken language text corresponding to the written language text;

performing sentence component unit conversion processing on the retranslated written language text to obtain a second spoken language text corresponding to the retranslated written language text;

and taking the first spoken text and the second spoken text as the spoken text.

The first spoken language text is a spoken language text obtained by converting the written language text. The second spoken text is obtained by converting the retranslated written language text.

Following the above example, the written language text LT is subjected to the conversion processing of the sentence constituting unit to obtain the first spoken text ST1 corresponding to the written language text LT, and the translated written language text LT3 is subjected to the conversion processing of the sentence constituting unit to obtain the second spoken text ST2 corresponding to the translated written language text LT3, and the first spoken text ST1 and the second spoken text ST2 are taken as spoken texts.

In conclusion, the corresponding spoken texts are obtained by respectively performing the conversion processing of the sentence composition units on the written language texts and the retraced written language texts, namely two spoken texts are obtained, and the spoken texts are expanded.

In practical applications, although there may be many differences between spoken expressions and written expressions, these differences are not reflected in each sentence, but appear with a certain probability according to the expression habits of speakers, in order to make the converted written language text more consistent with the spoken language features, a corresponding conversion processing probability may be set for each conversion processing policy, and whether to execute the conversion processing policy may be determined according to the conversion processing probability, which is specifically implemented as follows:

determining conversion processing probability corresponding to a conversion processing strategy of the written language text to be processed;

determining a target conversion processing strategy to be executed in the conversion processing strategies based on the conversion processing probability;

and performing conversion processing of sentence composition units on the written text to be processed by executing a target conversion processing strategy to obtain the spoken text corresponding to the written text to be processed.

The conversion processing strategy refers to a preset method (strategy) for performing conversion processing on the written texts to be processed. Specifically, the conversion processing policy may include at least one of the following: a clause conversion processing policy (a processing policy of performing clause units on written language sentences), a word conversion processing policy (a policy of performing word unit conversion processing on written language sentences), a character conversion processing policy (a policy of performing character unit conversion processing on written language sentences), and a sign conversion processing policy (a policy of performing sign unit conversion processing on written language sentences).

The clause conversion processing policy may be copy processing of clauses (i.e., a copy clause conversion processing policy), out-of-order processing, and/or flip processing, etc. The word conversion processing strategy can be adding processing, repeated processing, out-of-order processing and/or the like of words. The character conversion processing policy may be character out-of-order processing or the like. The symbol conversion processing strategy may be deleting symbol processing, adding symbol processing, and/or modifying symbol processing, etc.

Specifically, the conversion processing probability corresponding to the conversion processing policy refers to the probability of executing the conversion processing policy. In practical applications, each conversion processing strategy may have a corresponding conversion processing probability. Further, a target conversion processing strategy to be executed is determined in the conversion processing strategies based on the conversion processing probability. Taking the conversion processing strategy a as an example, the conversion processing probability corresponding to the conversion processing strategy a is 10%. A value range may be set, where the value range is 1 to 100 (or 1 to 10, etc.), a value range having a value probability that is the same as the conversion processing probability corresponding to the conversion processing policy a, such as 1 to 10 (or 90 to 100), is set in the value range, and then any value in the value range of 1 to 100 is randomly generated. If the generated numerical value is 9 and the numerical value is between 1 and 10, the numerical value meets the value probability of 10 percent, namely meets the conversion processing probability corresponding to the conversion processing strategy A, the conversion processing strategy A is determined to be executed, and the conversion processing strategy A is used as a target conversion processing strategy; if the generated value is 50, the value is between 11 and 100, which means that the value does not satisfy the value probability of 10%, that is, does not satisfy the conversion processing probability corresponding to the conversion processing policy a, and thus it is determined that the conversion processing policy is not executed. Similarly, other conversion processing strategies may be processed correspondingly in the above manner.

Further, the determined target conversion processing strategy can be one or more. In the case where the target conversion processing strategies are plural, the conversion processing may be performed by sequentially executing the target conversion processing strategies on the written language text to be processed in a preset execution order.

It should be noted that, because each conversion processing strategy has a corresponding conversion processing probability, and each conversion processing strategy also has a certain randomness, multiple conversion processes are performed on the same written language text to be processed, and the finally generated spoken language text is likely to be different. Therefore, in order to further expand the corpus, the conversion processing of the sentence forming unit can be performed on at least one written language text to be processed for multiple times, so as to obtain multiple spoken language texts corresponding to the written language text to be processed.

In addition, considering that a higher conversion processing probability is set for the conversion processing policy, the complexity of the sample corpus may be increased. The higher the complexity of the sample corpus, the more complicated the written language rewrite of the rewrite model obtained by training the sample corpus. Therefore, when there are a plurality of rewrite models for different text types, it is not suitable to perform complicated rewrite for a target spoken text of a fuzzy text type. Therefore, in the case of constructing a sample corpus of a rewriting model corresponding to a target spoken language text of a fuzzy text type, a lower conversion processing probability can be set for the conversion processing policy.

Following the above example, assuming that there are 4 conversion processing strategies for the written language text, the conversion processing probabilities corresponding to each conversion processing strategy for the written language text LT are determined as: 2%,6%,0.8%,8%. Then, for each conversion processing strategy, a numerical range may be set for the corresponding conversion processing probability, and a value range corresponding to the conversion processing probability is set, and a number is randomly generated, and if the number is within the value range, the conversion processing strategy corresponding to the conversion processing probability is determined as a target conversion processing strategy, and the target conversion processing strategy is executed to perform conversion processing of a sentence component unit on the written language text LT, so as to obtain a spoken language text corresponding to the written language text LT.

In summary, the corresponding conversion processing probability is set for each conversion strategy, that is, each conversion strategy is executed according to a certain execution probability, so that it is not necessary to execute each conversion processing strategy intentionally, thereby ensuring the naturalness and reasonability of the written language conversion.

In particular, since the generation of the translated back written language text is to expand the written language text, it is necessary to perform spoken language conversion on the written language text and the translated back written language text, respectively. Therefore, any one of the written language text and the translated written language text can be used as the written language text to be processed, and the conversion processing of the sentence constituting unit is performed on the written language text to be processed, and in the case that the sentence constituting unit is a sub-sentence unit, the following steps 3062 to 3066 are specifically implemented:

step 3062, performing sentence recognition on the written language text to be processed to obtain written language sentences contained in the written language text to be processed.

The sentence recognition is carried out on the written text to be processed, and the sentence division processing can be understood to be carried out on the text to be processed. In practical application, the sentence dividing (identifying) is performed through the sentence dividing symbol to obtain at least one written language sentence contained in the written language to be processed.

Step 3064, converting the written sentence unit to obtain the converted written sentence.

Furthermore, clause units are respectively converted for each identified written language sentence, and the converted written language sentence corresponding to each written language sentence can be obtained.

Specifically, because the conversion method for converting the clause unit into the written language sentence included in the written language text to be processed is various, in the embodiment of the present application, the written language sentence may be converted by the following two methods or a combination of the following two methods, including:

the method comprises the following steps: performing clause sampling on the written sentence according to a preset clause sampling rule to obtain a target clause in the written sentence; and converting the target clause in the written language sentence to obtain the converted written language sentence.

In practical application, since a written sentence may contain a plurality of clauses, and the clauses do not necessarily have expression differences between written languages and spoken languages, the clauses that need to be converted by clause units may be selected from the written sentences, and then the selected clauses may be converted.

The preset clause sampling rule refers to a preset sampling rule for sampling clauses in written sentences, and the preset clause sampling rule may be random sampling, or sampling according to positions, for example, a clause with a sampling position arranged at a first position in a written sentence, or sampling according to the number of characters, for example, a clause with a number of characters less than 5 in a sampling clause, and the like, and is not limited herein. Correspondingly, the target clause refers to a clause obtained by sampling a written language sentence according to a preset clause sampling rule.

In order to increase the naturalness and richness of the converted written sentence, the method for converting the target clause may be implemented by any combination of the following three conversion methods or the following three conversion methods, including:

mode A: and copying the target clause to obtain a copied target clause, and inserting the copied target clause into the written language sentence according to a preset clause inserting position to obtain the converted written language sentence.

In practical applications, when spoken language expression is used, some spoken language expression that is not included in written language sentences may occur, for example: good for the right, good and the like. In order to make the written language more accord with the characteristics of the spoken language, some spoken clauses can be added to the written language sentence.

Specifically, the preset clause inserting position refers to a preset position for inserting the target clause into the written sentence, and the preset position may be set according to the actual spoken language characteristics, for example, the preset clause inserting position may be a beginning or an end of the written sentence, or may be before or after the position of the target clause in the written sentence.

Following the above example, assuming that the written language text LT is used as the written language text to be processed, sentence recognition is performed on the written language text LT, and n written language sentences included in the written language text LT are obtained, which are written language sentence S1 and written language sentence S2 … … written language sentence Sn. Taking the written sentence S1 as an example for explanation, the written sentence S1 "my hometown is shanxi, where is beautiful" is sampled by clauses, and the target clause in the written sentence S1 is obtained as "my hometown is shanxi". The target clause is copied in the written language sentence S1 to obtain a copy target clause 'my hometown is Shanxi', and under the condition that the preset clause inserting position is before the position of the target clause, the copy target clause 'my hometown is Shanxi' is inserted into the written language sentence S1, and the converted written language sentence S13 is obtained as follows: "my home town is shanxi, where it is beautiful".

Mode B: deleting the target clause in the written language sentence; and inserting the target clause into the deleted written sentence according to a preset clause insertion rule to obtain the converted written sentence.

In practical applications, the expression order of the clauses may not be intended in the case of spoken language expression, and therefore, the expression order of the clauses may not match the expression order of the written language sentence in the spoken language sentence. In order to make the converted written language more consistent with the characteristics of spoken language, some clauses of the written language sentence can be subjected to position adjustment processing.

Specifically, the preset clause insertion rule refers to a preset rule for inserting a target clause, and the rule may be set according to actual experience, for example, the preset clause insertion rule may be random insertion (that is, the preset clause is randomly inserted before or after any clause in the written sentence), may be inserted after the first clause, may also be inserted at the end of the sentence, and the like.

Since the conversion processing of method a and the conversion processing of method B can be selectively executed, the conversion processing of method B can be executed on the converted written language sentence obtained by method a, the conversion processing of method B can be executed directly on the original written language sentence, the conversion processing of method a can be executed on the converted written language sentence obtained by method B, and other conversion processing can be selectively executed and/or sequentially executed.

Following the above example, or taking the written sentence S1 as an example, the written sentence S1 "my hometown is shanxi, where it is beautiful" is sampled by clauses, and the target clause in the written sentence S1 is obtained as "my hometown is shanxi". Deleting the target clause in the written language sentence S1, and randomly inserting the target clause 'my hometown is Shanxi' into any clause of the written language sentence S1 under the condition that a preset clause inserting rule is random inserting, and then obtaining a converted written language sentence S13 as follows: "my home town is Shanxi, where it is beautiful, and my home town is Shanxi".

Mode C: carrying out syntactic analysis on the target clause to obtain a syntactic structure corresponding to the target clause; and converting the target clause according to the target syntactic structure corresponding to the syntactic structure to obtain the converted written language sentence.

In practical applications, even though the grammatical structure (syntactic structure) of a clause is inconsistent, the meaning of the expression is the same, and thus, the word order in the clause may be inconsistent with the grammatical structure in the written language sentence in the spoken language sentence. Therefore, in order to make the converted written language more consistent with the characteristics of the spoken language, the grammatical structure of some clauses of the written language sentence can be changed, such as: and (5) flip-chip processing.

Specifically, the syntax analysis may be performed on the sampled target clause, and a syntax structure corresponding to the target clause may be obtained by using a rule-based syntax analysis method or a statistics-based syntax analysis method, where the syntax structure may be a predicate-object structure or a predicate-object structure, and the like, which is not limited herein. Accordingly, the target syntax structure refers to a syntax structure corresponding to the syntax structure of the target clause, which is set in advance. In particular, the syntax structure and the target syntax structure can be converted. For example, the syntax structure may be an active syntax structure of a principal predicate, and the target syntax structure may be a passive syntax structure of the principal predicate.

Following the above example, or randomly sampling the clauses of the written phrase sentence S1 "my hometown is shanxi, where is beautiful", and taking the target clause in the written phrase sentence S1 as "my hometown is shanxi" as an example for explanation. The syntax structure of the target clause is a main predicate object structure, and the target syntax structure corresponding to the syntax structure is an object predicate main structure. Then the target clause is converted into a guest-predicate main structure, and the converted target clause becomes: "Shanxi is my hometown. Accordingly, the converted written sentence S13 is: "Shanxi is my hometown, where it is beautiful".

The second method comprises the following steps: determining clause position probability distribution corresponding to preset clauses contained in a preset clause set; determining a target preset clause and a clause adding position corresponding to the target preset clause in the preset clauses based on the clause position probability distribution; and adding the target preset clause into the written language sentence according to the clause adding position to obtain the converted written language sentence.

The preset clause set refers to a preset set containing at least one spoken clause. Correspondingly, the preset clause refers to a clause included in the preset clause set. The phrase position probability distribution refers to position probability distribution of each preset phrase obtained by counting the occurrence positions (such as the beginning, end, or middle position of the phrase) of the preset phrases in a certain spoken language corpus in advance. In practical application, the frequency of each preset clause appearing at each position can be counted, and then the position probability distribution is calculated according to the counted frequency.

It is assumed that the preset clause set includes 3 preset clauses, and the 3 preset clauses are preset clause 1, preset clause 2, and preset clause 3, respectively. According to the statistics of the spoken language corpus in the sales field, if a sub-sentence 1 appears 60 times at the beginning of a sentence, a sub-sentence 2 appears 20 times at the end of the sentence, and a sub-sentence 3 appears 20 times at the beginning of the sentence, the probability that the sub-sentence 1 is added to the beginning of the sentence is preset as follows: 60/(60 + 20) =60%, the probability of adding a preset clause 2 to the end of a sentence is 20/(60 + 20) =20%, and the probability of adding a preset clause 3 to the beginning of a sentence is 20/(60 + 20) =20%. The above 3 probabilities are the probability distribution of clause positions corresponding to the preset clauses.

Further, based on the clause position probability distribution, a target preset clause and a clause adding position corresponding to the target preset clause (a position for adding the target preset clause in the written sentence) can be determined in the preset clause. In specific implementation, a value range may also be preset, where the value range is 1-100 (or 1-10, etc.), and value intervals with value probabilities the same as the position probability distribution of the clauses, such as 1-60, 61-80, and 81-100, are set in the value range, and then any value in the value range of 1-100 is randomly generated. If the generated numerical value is 9 and the numerical value is between 1 and 60, the numerical value satisfies 60% of the value-taking probability, that is, the probability of adding the preset clause 1 to the sentence head is satisfied, and it is determined that the target preset clause is the preset clause 1 and the clause adding position corresponding to the target preset clause is the sentence head.

Still further, in a case that the target preset clause is "pairwise" the target preset clause "pairwise" is added to the beginning of the written sentence S1, and the converted written sentence S13 is obtained as: "Pair, my hometown is Shanxi, where it is beautiful".

Step 3066, a spoken text is determined based on the converted written language sentence.

When there are a plurality of converted written language sentences, the converted written language sentences may be combined in the order of arrangement of the original written language sentences in the written language text to generate a spoken language text.

Following the above example, at least one conversion process is performed on at least one written sentence among n written sentences included in the written sentence text LT to obtain n converted written sentences, which are written sentence S13, written sentence S23, … …, and written sentence Sn3, respectively, and the n converted written sentences are combined to generate the spoken text ST1.

In conclusion, the written sentence is rewritten in spoken language by converting the written sentence by copying clauses, disordering clauses and/or adding clauses in clause units, so that the converted written sentence is more consistent with the characteristics of the spoken language.

In addition, in the case that the sentence component unit is a word unit, the conversion method of performing the conversion processing of the clause unit on the written language text to be processed is also diversified, and the first implementation manner provided by the embodiment of the present application is specifically implemented as follows:

performing sentence recognition on the written text to be processed to obtain written sentences contained in the written text to be processed;

determining word position probability distribution corresponding to preset words contained in a preset word set;

determining a target preset word and a word adding position corresponding to the target preset word in the preset words according to the word position probability distribution, and inserting the target preset word into the written language sentence according to the word adding position to obtain a converted written language sentence;

determining spoken text based on the converted written language sentence.

In practical applications, since some spoken words are randomly added to the spoken expression, the spoken words may include: conjunctions, linguistic or other spoken words, etc., such as: ouabase, real, etc., these spoken words are not normally present in written language sentences. In order to make the written language more consistent with the characteristics of the spoken language, some addition processing of spoken words can be carried out on the written language sentences.

Specifically, the preset term set refers to a preset set including at least one spoken term. Correspondingly, the preset words refer to words contained in the preset word set. The term position probability distribution refers to position probability distribution of each preset term obtained by counting the occurrence positions (such as the beginning, end, or middle position of a sentence) of the preset terms in a certain speech corpus set in advance. In practical application, the occurrence frequency of each preset word at each position can be counted, and the position probability distribution is calculated according to the counted frequency.

Suppose that the preset word set includes 2 preset words, and the 2 preset words are preset word 1 and preset word 2, respectively. According to the statistics of the spoken language corpus in the sales field, the preset word 1 appears 80 times at the beginning of the sentence, the preset word 2 appears 20 times at the end of the sentence, and the probability that the preset word 1 is added to the beginning of the sentence is as follows: 80/(80 + 20) =80%, and the probability of adding the preset word 2 to the sentence end is 20%, then the 2 probabilities are the word position probability distribution corresponding to the preset word.

Specifically, the specific implementation of the target preset word and the word adding position corresponding to the target preset word (the position where the target preset word is added in the written sentence) in the preset words is determined according to the word position probability distribution, and the specific implementation of the target preset clause and the clause adding position corresponding to the target preset clause in the preset clauses based on the clause position probability distribution is referred to above, which is not described herein again.

On the basis of determining the target preset word and the word adding position corresponding to the target preset word, the target preset word can be added to the word adding position in the written sentence, and the converted written sentence is obtained, further, the specific implementation of the spoken text is determined based on the converted written sentence by referring to the conversion processing part in the clause unit, and the specific implementation of the spoken text is determined based on the converted written sentence, which is not limited herein.

Following with the example, on the basis of n written sentence that contains in written language text LT, to written sentence S1 wherein explain as the example, contained 2 in the preset word set and preset the word, these 2 are preset word 1, preset word 2 respectively, and the word position probability distribution of these 2 preset words is: the probability of adding a preset word 1 to the beginning of a sentence is 80%, and the probability of adding a preset word 2 to the beginning of a sentence is 20%. It is assumed that a target preset word is determined as a preset word 1 and a word adding position corresponding to the target preset word is determined as a sentence start in a preset word set according to the word position probability distribution. In the case that the preset word 1 is "true", the preset word 1 is added to the beginning of the written sentence S1, and the converted written sentence S13 is obtained as follows: "actually my hometown is Shanxi, where it is beautiful". Then, the n converted written sentences are combined to generate a spoken text ST1.

In conclusion, the word adding processing of the word unit is carried out on the written sentence, so that the written sentence is orally rewritten, and the converted written sentence is more consistent with the characteristics of the oral language.

In specific implementation, because some spoken words may be habitually repeated after being added during the expression of the spoken language, in order to make the converted written language more conform to the characteristics of the spoken language, the added words in the written language sentence may be copied, which is specifically implemented by the following steps:

copying the target preset words added in the converted written sentence to obtain copied words;

inserting the copied words into the converted written sentence according to a preset word insertion rule to obtain the inserted written sentence;

determining spoken text based on the inserted written language sentence.

Specifically, the preset word insertion rule refers to a preset rule for inserting the target preset word into the written language sentence, and the rule may be set according to actual spoken language characteristics, for example, the preset word insertion rule may be a position before or after the target preset word is inserted into the target preset word, or may be other positions where the target preset word is inserted into the written language sentence, which is not limited herein.

Following the above example, the written language sentence S13 after obtaining the conversion is: on the basis of "the real my home country is shanxi and there is beauty", the target preset word "real" is copied to obtain a copied word "real", and in the case that the preset word insertion rule is inserted before the target preset word, the copied word "real" is added to the converted written language sentence S13 to obtain an inserted written language sentence S14, and the written language sentence S14 is "the real my home country is shanxi and there is beauty". The n inserted written sentences are combined to generate a spoken text ST1.

In conclusion, the added words are repeatedly processed on the aspect of processing word addition processing of the word units on the written language sentences, so that the converted written language sentences can better accord with the characteristics of the spoken language.

In the case that the sentence component unit is a word unit, in addition to the conversion processing of the word unit, the second implementation manner provided by the embodiment of the present application is specifically implemented by the following processing manner:

carrying out sentence recognition on the written language text to be processed to obtain written language sentences contained in the written language text;

carrying out word sampling on words in the written language sentences according to preset word sampling rules to obtain target words in the written language sentences;

deleting the target words in the written language sentences, and inserting the target words into a preset insertion range corresponding to the target words in the deleted written language sentences to obtain converted written language sentences;

determining spoken text based on the converted written language sentence.

Since the expression order of words may not be intended in the spoken language expression, the expression order of words may not coincide with the expression order of written language sentences in the spoken language sentence. In order to make the converted written language more consistent with the characteristics of the spoken language, position adjustment processing can be performed on some words of the written language sentence.

Specifically, the preset word sampling rule refers to a preset sampling rule for sampling words to be out of order in written sentences, and the preset word sampling rule can be random sampling or sampling according to the number of preset characters, for example, the number of the sampled characters in the written sentences is 3 characters in terms of random sampling. Accordingly, the target word refers to a word sampled in the written language sentence by the word sampling rule.

The preset insertion range refers to a range in which insertion processing is performed, which is set in advance. The preset insertion range can be preset according to actual experience or spoken language expression habits, specifically, the preset insertion range corresponding to the target word can be a character interval from 3 characters of the target word before the position of the written language sentence to 3 characters of the target word after the position of the written language sentence, the character interval can be abbreviated as [ -3,3], in addition, the preset insertion range can also be a clause range to which the word belongs, and the like. Further, the target words are randomly inserted in a preset insertion range.

In the above example, on the basis of n written sentences included in the written language text LT, the written language sentence S1 is taken as an example for explanation, the target sentence is obtained as "there" by randomly sampling the words in the written language sentence S1, the target sentence is deleted in the written language sentence S1, and the deleted written language sentence S1 is "my hometown is shanxi, and is beautiful". And under the condition that the preset insertion range is 'the clause range to which the target word belongs', the target word is inserted into the preset insertion range of the written sentence S1, and the converted written sentence S13 is obtained as follows: "my hometown is Shanxi, where is much America". Then, the n converted written sentences are combined to generate a spoken text ST1.

In conclusion, the written sentence is converted into the word unit, so that the converted written sentence is more in line with the characteristics of the spoken language.

In the case where sentence component units are character units, it is considered that sometimes the expression order of characters is not strictly followed in the spoken language expression process, and therefore a case may occur in which the expression order of characters does not coincide with the expression order of characters in a written language sentence. In order to make the converted written language more accord with the characteristics of spoken language, some characters of written language sentences can be subjected to position adjustment treatment, and the embodiment of the application is realized by the following specific method:

performing sentence recognition on the written language text to be processed to obtain written language sentences contained in the written language text to be processed;

carrying out character sampling on characters in the written language sentence according to a preset character sampling rule to obtain target characters in the written language sentence;

deleting the target characters in the written sentence, and inserting the target characters into a preset character insertion range corresponding to the target characters in the deleted written sentence to obtain a converted written sentence;

determining spoken text based on the converted written language sentence.

Specifically, the preset character sampling rule refers to a preset sampling rule for sampling a character to be scrambled in a written sentence, and the preset character sampling rule may be a random sampling rule or a sampling rule according to a preset character position, for example, randomly sampling a character positioned at the 5 th position in the written sentence, and is not limited herein.

Accordingly, the preset character insertion range corresponding to the target character may be a character interval from the target character 3 characters before the position of the written sentence to the target word 3 characters after the position of the written sentence, and the character interval may be abbreviated as [ -3,3], and in addition, the preset character insertion range may also be a clause range where the target character is located, and the like, which is not limited herein.

Following the above example, on the basis of n written sentences included in the written language text LT, the written language sentence S1 is taken as an example for explanation, characters are randomly sampled in the written language sentence S1, a target character is "american", the target character is deleted in the written language sentence S1, and the deleted written language sentence S1 is "my hometown is shanxi, where is very good". And under the condition that the preset character insertion range corresponding to the target character is the 'clause range to which the target character belongs', inserting the target character 'American' into the preset character insertion range of the written language sentence S1, and obtaining a converted written language sentence S13 as follows: "my hometown is Shanxi, where Mei is great". Then, the n converted written sentences are combined to generate a spoken text ST1.

In conclusion, the written language sentences are subjected to character unit disorder processing, so that the converted written language sentences better accord with the characteristics of spoken language.

In the case that the sentence component units are symbol units, the symbols appearing in the spoken sentences may not be consistent with the symbols appearing in the written language sentences because the spoken expressions may not have an explicit division for the disconnection or connection of the sentences, or the division is arbitrary. In order to make the converted written language more conform to the characteristics of the spoken language, the conversion processing of the symbolic unit can be performed on the text of the written language to be processed by the following two modes or the combination of the following two modes, including:

the first conversion method comprises the following steps: performing sentence recognition on the written text to be processed to obtain written sentences contained in the written text to be processed; carrying out symbol sampling on the written sentence according to a preset symbol sampling rule to obtain a target punctuation mark in the written sentence, deleting the target punctuation mark in the written sentence, and obtaining the converted written sentence; determining spoken text based on the converted written language sentence.

Specifically, the preset symbol sampling rule refers to a preset sampling rule for sampling a symbol to be deleted in a written sentence. The preset symbol sampling rule may be random sampling, or sampling according to a preset position, such as sampling punctuation marks after a first clause in a written sentence, which is not limited herein. Correspondingly, the target punctuation mark refers to punctuation marks sampled from written language sentences according to a preset symbol sampling rule.

In the above example, on the basis of n written sentences contained in the written language text LT, the written sentence S1 is taken as an example for explanation, and the written sentence S1 is subjected to symbol sampling at random, so as to obtain a comma after the target punctuation in the written sentence S1 is the clause "my hometown is shanxi". Deleting the target punctuation mark in the written sentence S1, and obtaining a converted written sentence S13 as follows: "my hometown is Shanxi where it is beautiful". Then, the n converted written sentences are combined to generate a spoken text ST1.

And a second conversion method comprises the following steps: performing sentence recognition on the written language text to be processed to obtain written language sentences contained in the written language text to be processed; performing symbol clause sampling on the written language sentence according to a preset symbol clause sampling rule to obtain a target symbol clause in the written language sentence, and inserting a preset punctuation mark into the target symbol clause to obtain a converted written language sentence; determining spoken text based on the converted written language sentence.

Specifically, the preset symbol clause sampling rule refers to a preset sampling rule for sampling a clause added with a symbol in a written sentence. The preset symbol clause sampling rule may be random sampling, or may be sampling according to the number of characters of a clause, for example, sampling a clause with the largest number of sampled characters in a written sentence, which is not limited herein. Correspondingly, the preset punctuation mark refers to a punctuation mark which is preset for insertion, and in practical application, the preset punctuation mark can be randomly inserted into the target symbol clause, or can be inserted according to a preset position, and is not limited herein; the target symbol clause refers to a clause sampled in the written language sentence according to a preset symbol clause sampling rule.

In the above example, on the basis of n written sentences included in the written language text LT, the written language sentence S1 is taken as an example for explanation, and in the case where the preset symbolic clause sampling rule is the longest-character-sampled clause, symbolic clause sampling is performed on the written language sentence S1, and the target symbolic clause in the written language sentence S1 is obtained as "shanxi" in my hometown. In the default punctuation mark is "! "in case of inserting the preset punctuation mark into the written sentence S1, the converted written sentence S13 is obtained as follows: "My home town is Shanxi! There is a great beauty ". Then, the n converted written sentences are combined to generate a spoken text ST1.

In conclusion, the written sentence is subjected to conversion processing of deleting symbols and adding symbols of symbol units, so that the written sentence is subjected to spoken language rewriting, and the converted written sentence is more in line with the characteristics of spoken language.

Step 308: and constructing a sample corpus based on the corresponding relation between the written language text and the retranslated written language text and the spoken language text.

Specifically, on the basis of the above-mentioned acquisition of the spoken text, since the acquired spoken text is obtained by converting the written language text or the translated back written language text, there is a correspondence between the spoken text and the written language text or the translated back written language text. Based on the corresponding relation, sample corpus aligned with written language-spoken language text can be generated.

The sample corpus refers to a training sample pair for model training. In practice, a written language rewrite model for training spoken text to written language text may be used by generating a training sample pair of written language text-spoken text. And under the condition of training the written language rewriting model, taking the spoken language text in the sample corpus as a training sample, and taking the written language text in the sample corpus as a sample label corresponding to the spoken language text.

In practical application, because some abnormal data may exist in the spoken language text obtained through conversion processing, the existence of the abnormal data seriously affects the quality of the spoken language text, and in order to guarantee the quality of the generated spoken language text, the abnormal data in the spoken language text can be subjected to data cleaning, which is specifically implemented in the following way in the embodiment of the application:

identifying abnormal information in the spoken language text;

cleaning the spoken language text according to the abnormal information to obtain a cleaned spoken language text;

and constructing a sample corpus based on the corresponding relation between the written language text and the translated written language text and the cleaned spoken language text.

The abnormal information may be abnormal information such as wrongly written characters, repeated punctuations, chinese punctuations mixed with English punctuations, special symbols, stop words, and the like. In addition, the abnormal information may be semantically fuzzy or semantically unreasonable information, which is not limited herein. In practical application, the abnormal information in the spoken language text can be identified through a preset abnormal identification rule, and the abnormal information in the spoken language text can also be identified based on a pre-trained text cleaning model. In a specific implementation, the text cleaning model can be a deep context model for syntax error correction to perform syntax detection.

Furthermore, after the abnormal information in the spoken text is identified, in the case that there are a plurality of spoken texts, the spoken text with the abnormal information can be deleted directly, so as to obtain the spoken text without the abnormal information (i.e. the cleaned spoken text). In addition, the abnormal information in the spoken language text may also be deleted or corrected, so as to obtain the cleaned spoken language text, which is not limited herein. If any spoken text is deleted, the corresponding written text or translated back written text needs to be deleted.

In specific implementation, the written language text and the spoken language text can be subjected to data cleaning by considering that the written language text possibly adopted also contains abnormal information.

Following the above example, the spoken text ST1 is obtained by converting the written language text LT as the written language text to be processed, the spoken text ST2 is obtained by converting the translated written language text LT3 as the text to be processed, and the abnormal information in the spoken text ST1 is recognized as ",! If the spoken text ST2 is not abnormal, the spoken text ST1 is subjected to data cleaning according to the abnormal information to obtain a cleaned spoken text ST1, and the spoken text ST2 is directly used as the cleaned spoken text ST2. And constructing a sample corpus pair 1 by using the written language text LT and the cleaned spoken language text ST1 based on the corresponding relation between the written language text LT and the cleaned spoken language text ST1. And constructing a sample corpus pair 2 by the written language text LT3 and the cleaned spoken language text ST2 based on the corresponding relation between the translated written language text LT3 and the cleaned spoken language text ST2, and taking the sample corpus pair 1 and the sample corpus pair 2 as sample corpora.

In conclusion, the data of the spoken language text after the conversion processing is cleaned, and the sample corpus is constructed through the cleaned spoken language text, so that the quality of the sample corpus is guaranteed, and the accuracy of model training is further improved.

On the basis of constructing the sample corpus, if the number of the written language texts is large enough, the constructed sample corpus is also rich enough, namely, the problem that the constructed sample corpus is lack can be solved, therefore, model training can be carried out based on the constructed sample corpus, and the embodiment of the application is realized by the following method:

The initial written language rewriting model may be a written language rewriting model to be trained, which is constructed based on a Seq2Seq model, wherein both an encoder and a decoder in the Seq2Seq model may be constructed by using a Transformer model. Accordingly, the training stopping condition may be that a loss value between the predicted written language text generated by rewriting the written language of the spoken language text in the sample corpus through the model and the sample written language text is smaller than a preset loss value, or that the training iteration number reaches a preset iteration number, such as 5 times or 6 times, and is not limited herein. Accordingly, the written language rewrite model may be understood as a trained written language rewrite model for a spoken language text.

In the practical application, the loss function for calculating the model loss value can be a 0-1 loss function, an absolute value loss function, a square loss function, a cross entropy loss function and the like in the practical application, and the absolute value loss function is taken as an example for explanation, and the following formula 1 is referred to:

l (Y, f (x)) = | Y-f (x) | equation 1

Wherein L represents a loss value, f (X) represents a predicted written language text, and Y represents a sample written language text, and in the present application, the selection of the loss function is not limited, subject to practical application.

After the model loss value is calculated, the model parameters of the initial text classification model can be reversely adjusted according to the model loss value, the initial text classification model is continuously trained by sampling the sample corpora of the next batch until the training stopping condition is reached, and the written language rewriting model after training can be obtained.

In specific implementation, the written language rewrite model is subjected to model training by adopting rich sample corpora, and can be used for processing more complicated sentence rewrite, so that the written language rewrite model can be called a written language standard rewrite model.

In practical application, because the target spoken language text to be rewritten in written language may have different quality, in this case, the written language text is directly rewritten in written language, and the quality of the rewriting may not be guaranteed, therefore, in order to guarantee the effect of written language rewriting, text classification may be performed on the target spoken language text, so that corresponding rewriting measures are implemented on different types of target spoken language texts, and the embodiment of the present application is implemented specifically by the following manner:

acquiring a target spoken language text, inputting the acquired target spoken language text into a text classification model for classification processing, and acquiring a text type corresponding to the target spoken language text;

under the condition that the text type is a clear text type, inputting the target spoken text into a written language standard rewriting model to rewrite the written language, and obtaining a first target written language text corresponding to the target spoken text;

and under the condition that the text type is the fuzzy text type, inputting the target spoken text into a written language micro-rewriting model to rewrite the written language, and obtaining a second target written language text corresponding to the target spoken text.

The target spoken language text is the spoken language text to be rewritten. The text classification model is a model which is trained in advance and used for classifying the target spoken language text, can be a two-classification model, and is used for classifying the target spoken language text into a clear text type or a fuzzy text type by performing text semantic definition recognition on the target spoken language text. The clear text type means that the text semantic expression of the target spoken language text is clear; the fuzzy text type means that the text semantic expression of the target spoken language text is fuzzy.

Furthermore, it makes no sense to rewrite this type of text in written language, considering that the target spoken text may also be text that does not contain semantic information. Therefore, the target spoken language text can be classified into a clear text type, a fuzzy text type or an invalid text type by adopting a three-classification model.

Further, the text classification model may be trained by: acquiring a sample spoken language text and a semantic definition label corresponding to the sample spoken language text; constructing a training sample pair based on the sample spoken language text and the semantic definition label; and performing model training on the initial text classification model through the training samples until the text classification model meeting the classification training stopping condition is obtained.

In practical applications, if the text classification model is a binary classification model, the semantic definition tag includes: a clear text type and a fuzzy text type. If the text classification model is a three-classification model, the semantic definition tag comprises: a clear text type, a fuzzy text type, and an invalid text type.

It should be noted that the semantic definition label of the sample spoken language text needs to be labeled in advance, and then the sample spoken language text and the semantic definition label are used as text corpus semantic definition labeling data to perform model training on the initial text type model. The initial text classification model may be a text classification model to be trained, which is previously constructed by CNN (convolutional neural network), RNN (cyclic neural network), LSTM (long-term memory network), fastText, textCNN, HAN models, and the like.

In the practical application, the loss function for calculating the model loss value can be a 0-1 loss function, an absolute value loss function, a square loss function, a cross entropy loss function and the like in the practical application, and here, the 0-1 loss function is taken as an example for explanation, which is shown in the following formula 2:

wherein L represents a loss value, f (X) represents a predicted text type, and Y represents a sample text type, and in the present application, the selection of the loss function is not limited, subject to practical application.

After the model loss value is calculated, the model parameters of the initial text classification model can be reversely adjusted according to the model loss value, and the initial text classification model is continuously trained by sampling the semantic definition annotation data of the text corpus of the next batch until reaching a classification training stopping condition, specifically, the classification training stopping condition can be that the model loss value is smaller than a preset threshold value or the training iteration number reaches the preset iteration number, and the like, and is not limited herein.

In specific implementation, the target spoken text for the invalid text type can be directly filtered, i.e. written language rewriting of the target spoken text is not required. Aiming at the target spoken language text of the clear text type, the trained written language standard rewriting model can be input to rewrite the written language, and the target written language text (namely the first target written language text) corresponding to the target spoken language text is obtained. And aiming at the target spoken language text of the fuzzy text type, an additional written language rewriting model (namely, a written language micro-rewriting model) for slightly rewriting the spoken language text needs to be input for written language rewriting, and a target written language text (namely, a second target written language text) corresponding to the target spoken language text is obtained. This is because the semantic expression of the target spoken language text is fuzzy, and if the target spoken language text is rewritten in a complicated way, the semantic expression of the target spoken language text may be fuzzy or prone to deviation. Therefore, for the target spoken language text of the fuzzy text type, simple rewriting of spoken words, tone words and the like can be performed by using the written language micro-rewriting model.

For example: acquiring a target spoken text TST, inputting the target spoken text TST into a text classification model, and acquiring that the text type corresponding to the target spoken text TST output by the text classification model is a clear text type, and then inputting the target spoken text TST into a written language standard rewriting model to acquire a target written language text TLT1 output by the written language standard rewriting model;

and if the text type corresponding to the target spoken text TST output by the text classification model is a fuzzy text type, inputting the target spoken text TST into a written language micro-rewriting model to obtain a target written language text TLT2 output by the written language micro-rewriting model.

In conclusion, the target spoken texts are classified, the target spoken texts in the clear text types are subjected to standard rewriting through the written language standard rewriting model, and the target spoken texts in the fuzzy text types are slightly rewritten through the written language micro-rewriting model, so that the reasonable rewriting of the spoken texts in different types is realized, and the written language rewriting quality is guaranteed.

Specifically, the written language micro-rewriting model can be obtained by performing model training on another sample corpus (which may be referred to as micro-sample corpus) constructed based on the corresponding relationship between the written language text and the spoken language text, and the micro-sample corpus is relatively simplified compared with the constructed sample corpus. When the method is specifically implemented, the initial written language rewriting model is trained through the micro sample corpus until the written language rewriting model meeting the training stopping condition is obtained.

In practical applications, in order to further increase the rewriting difference between the two models, a lower conversion processing probability may be set for each conversion processing strategy for written language texts in the process of constructing the micro sample corpus. The change between the spoken text and the written text generated after the conversion process is thus relatively small. Therefore, the written language micro-rewriting model trained by the micro-sample corpus has more tiny rewriting to the spoken language text.

Fig. 4 is a schematic diagram illustrating written language rewriting in a text processing method according to an embodiment of the present application. The method comprises the steps of obtaining a target spoken language text, and inputting the target spoken language text into a text recognition model (namely a text classification model), wherein the text recognition model is obtained by training a text recognition model through pre-collected text corpus semantic definition labeling data. Further, the text recognition model outputs the predicted text type corresponding to the target spoken language text by performing text classification on the input target spoken language text. And determining a rewriting model (namely a written language standard rewriting model or a written language micro-rewriting model) corresponding to the target spoken language text according to the predicted text type, wherein if the predicted text type is a clear text type, the rewriting model is the written language standard rewriting model, and if the predicted text type is a fuzzy text type, the rewriting model is the written language micro-rewriting model, and the rewriting model is obtained by performing model training through sample linguistic data constructed by spoken language text-written language text alignment data. Furthermore, the rewriting model obtains the target written language text output by the rewriting model by performing written language rewriting on the input target spoken language text.

Furthermore, in order to further ensure the accuracy of written language rewriting of the written language standard rewriting model and the written language micro-rewriting model and avoid the situation that the text generation result is uncontrollable, a character-level mask operation can be adopted in the process of rewriting the two models. Through the character-level mask operation, the generation result of written language rewriting is guaranteed to be mainly from an input text, in the embodiment of the application, a written language standard rewriting model is used as the written language rewriting model, the written language rewriting model comprises an encoding layer and a decoding layer, and the written language rewriting model is used for rewriting a target spoken language text and is realized by adopting the following specific method:

carrying out sentence splitting processing on the target spoken language text to obtain a sentence sequence contained in the target spoken language text;

sequentially inputting oral sentence units in a sentence sequence into a coding layer of a written language rewriting model for coding, and obtaining sentence characteristic vectors and vocabulary vectors corresponding to the oral sentence units, wherein the vocabulary vectors are obtained by mapping the oral sentence units and the vocabulary;

and calculating a vector product between the sentence characteristic vector and the word list vector, inputting the vector product into a decoding layer of the written language rewriting model for decoding, and obtaining a target written language text corresponding to the target spoken language text.

The word list refers to a word list. Specifically, the vocabulary may be generated by counting the frequency of words/characters appearing in the sample corpus during the training process of the written language rewrite model (for example, adding characters/words with the frequency of appearance greater than a threshold in the training sample corpus into the vocabulary), may be carried by the model itself, and may be generated in other ways. The sentence sequence is a sequence formed by arranging the spoken sentences contained in the target spoken language text according to the sequence in the target spoken language text. Accordingly, a spoken sentence unit refers to a spoken sentence included in a sentence sequence.

In specific implementation, the sentence sequence is mapped to a word list, which means that characters/words in the spoken sentence are matched with the characters/words in the word list; if any character/word in the word list is hit by the character/word in the spoken sentence, the vector bit corresponding to the hit character/word in the word list is set to 1, and the vector bit corresponding to the missed character/word in the word list is set to 0, so that the word list vector can be obtained. For example, a vocabulary table includes 5000 characters, 4 characters in a spoken sentence 1 are mapped with the vocabulary table, wherein the 1 st character maps the 3 rd character in the vocabulary table, the 2 nd character maps the 6 th character in the vocabulary table, the 3 rd character maps the 9 th character in the vocabulary table, and the 4 th character maps the 5 th character in the vocabulary table, so that an obtained vocabulary vector is 00101100100 … ….

Furthermore, the vector product between the word list vector and the sentence characteristic vector is calculated, decoding is carried out based on the vector product, the input text characters/words are counted, constraint limitation is carried out during decoding output, the text characters generated by the written language rewriting model are mainly from an input text source, and semantic deviation of the rewriting result of the written language rewriting model is greatly avoided.

Similarly, the written language micro-rewriting model may also be used as a written language rewriting model, and written language rewriting is performed by performing the above steps. Therefore, the specific steps of writing the written language can be realized by referring to the written language rewriting model, and are not repeated herein.

In conclusion, the written language text is obtained and the retranslated written language text corresponding to the written language text is obtained by obtaining the written language text and performing retranslation processing on the written language text, so that the written language material is expanded in a retranslation mode. And then the written language text and the retraced written language text are respectively subjected to sentence component unit conversion processing to obtain the spoken language text, so that the written language material based on retracing expansion is converted. And constructing a sample corpus based on the corresponding relation between the written language text and the translated written language text and the spoken language text. The method and the device have the advantages that the written language corpus after the retracing expansion and the converted spoken language text are used for providing a large number of sample corpora of the spoken text-written language text for model training, the training difficulty of the model is simplified, the phenomenon that a large amount of text data are collected and processed manually and time and labor are wasted is avoided, and the time cost and the labor cost are saved.

Fig. 5 is a flowchart illustrating a method for generating a spoken language text according to an embodiment of the present application, which specifically includes the following steps:

step 502: acquiring a written language text;

step 504: obtaining a retranslated written language text corresponding to the written language text by performing retranslation processing on the written language text;

step 506: and respectively carrying out conversion processing of sentence forming units on the written language text and the retraced written language text to obtain a spoken language text.

The corresponding spoken language text can be obtained by performing the conversion processing of the sentence component unit on the written language text and the retraced written language text respectively. In specific implementation, the spoken language text with relatively accurate semantic expression can be obtained by further screening the spoken language text obtained after conversion, and the spoken language text is used as the spoken language text corresponding to the written language text. Further improving the accuracy of the spoken transcription of the written language text.

In practice, it is considered that written or spoken language expressions may be used when the user asks or consults. By generating the corresponding spoken language text, not only the question bank can be expanded, but also the accuracy of semantic analysis on user questions or consultations can be improved.

On the basis of obtaining the spoken text corresponding to the written language text and retranslating the spoken text corresponding to the written language text, the sample corpus can be constructed according to the corresponding relationship between the written language text and the spoken text and the corresponding relationship between the retranslated written language text and the spoken text. And a written language rewriting model for performing written language rewriting on the target spoken language text is trained by the sample corpus.

An optional embodiment, further comprising:

acquiring a target spoken language text;

classifying the target spoken language text to obtain a text type corresponding to the target spoken language text;

under the condition that the text type is a standard text type, selecting a corresponding written language rewriting model according to the standard text type;

inputting the target spoken language text into the written language rewriting model for processing to obtain a target written language text corresponding to the target spoken language text;

the written language rewriting model is obtained by training a spoken language text obtained by retracing and converting the written language text based on the written language text.

Optionally, after obtaining the text type corresponding to the target spoken language text, the method further includes:

under the condition that the text type is a fuzzy text type, selecting a corresponding written language conversion model according to the fuzzy text type;

inputting the target spoken language text into the written language conversion model for processing to obtain a converted written language text corresponding to the target spoken language text;

the written language conversion model is obtained by training a basic spoken language text obtained by converting the written language text based on the written language text.

Optionally, the training of the written-to-speech model comprises:

acquiring a written language text;

carrying out conversion processing of sentence composition units on the written language text to obtain a basic spoken language text;

constructing a basic sample corpus based on the corresponding relation between the written language text and the basic spoken language text;

training an initial written language conversion model through the basic sample corpus until the written language conversion model meeting a second training stop condition is obtained.

Optionally, the classifying the target spoken language text to obtain a text type corresponding to the target spoken language text includes:

inputting the target spoken language text into a text classification model for classification processing to obtain a text type corresponding to the target spoken language text; wherein the training of the text classification model comprises:

acquiring a sample spoken language text and a semantic definition label corresponding to the sample spoken language text;

constructing a training sample pair based on the sample spoken language text and the semantic definition label;

and performing model training on the initial text classification model through the training sample until the text classification model meeting the classification training stopping condition is obtained.

Optionally, the inputting the target spoken language text into the written language rewriting model for processing to obtain a target written language text corresponding to the target spoken language text includes:

sequentially inputting the oral sentence units in the sentence sequence into the coding layer of the written language rewriting model for coding processing to obtain sentence characteristic vectors and vocabulary vectors corresponding to the oral sentence units, wherein the vocabulary vectors are obtained by mapping the oral sentence units and the vocabulary;

and calculating a vector product between the statement feature vector and the word list vector, and inputting the vector product into a decoding layer of the written language rewriting model for decoding to obtain a target written language text corresponding to the target spoken language text.

Optionally, the written language rewriting model is obtained by training a spoken language text obtained by performing a retracing and conversion process on a written language text, and includes:

acquiring a written language text;

obtaining a retranslate written language text corresponding to the written language text by performing retranslate processing on the written language text;

respectively carrying out conversion processing of sentence forming units on the written language text and the retranslated written language text to obtain a spoken language text;

and training an initial written language rewriting model through the sample corpus until the written language rewriting model meeting a second training stop condition is obtained.

Optionally, the obtaining a retranslated written language text corresponding to the written language text by performing retranslation processing on the written language text includes:

translating the translated written language text back into a target language to which the written language text belongs to obtain an initial translated written language text;

and replacing target key words corresponding to the key words in the initial retranslate written language text by the key words in the written language text to obtain a retranslate written language text.

Optionally, before the translating the written language text into the translated written language text corresponding to the preset language, the method further includes:

marking the position of the key word in the written language text;

correspondingly, the replacing the target key words corresponding to the key words in the initial retranslate written language text by the key words in the written language text to obtain a retranslate written language text, which includes:

and replacing corresponding target key words in the initial retranslate written language text by the key words based on the position marks to obtain a retranslate written language text.

Optionally, taking any one of the written language text and the retranslated written language text as a written text to be processed, and performing a conversion process of a sentence component unit on the written text to be processed, including:

determining conversion processing probability corresponding to the conversion processing strategy of the written language text to be processed;

and performing sentence component unit conversion processing on the written language text to be processed by executing the target conversion processing strategy to obtain the spoken language text corresponding to the written language text to be processed.

and deleting the target spoken language text under the condition that the text type is an invalid text type.

In conclusion, the written language text is obtained and is subjected to retracing processing, so that the retracing written language text corresponding to the written language text is obtained, and the written language materials are expanded in a retracing mode. And then the written language text and the retraced written language text are respectively subjected to sentence component unit conversion processing to obtain the spoken language text, so that the conversion processing based on the retraced and expanded written language corpus is realized, the spoken language text is expanded, the conversion accuracy of the written language text to the spoken language text is also improved, downstream related processing tasks are further performed based on the expanded written language corpus and/or the expanded spoken language text, a text basis is provided for the task processing, and the processing efficiency of the task processing is improved.

The above is a schematic scheme of a spoken language text generation method according to this embodiment. It should be noted that the technical solution of the spoken language text generation method and the technical solution of the text processing method belong to the same concept, and details of the technical solution of the spoken language text generation method, which are not described in detail, can be referred to the description of the technical solution of the text processing method.

The following describes the text processing method with reference to fig. 6 by taking an application of the text processing method provided in the present application in an actual scene as an example. Fig. 6 shows a processing flow chart of a text processing method applied to an actual scene according to an embodiment of the present application, which specifically includes the following steps:

step 602: and acquiring written language text.

Specifically, the written language text may be written language text of any field, such as written language text of medical field, written language text of chemical field, written language text of sales field, written language text of daily life field, written language text of travel field, and the like, without limitation. And the number of the text of the written language text may be one or more, which is not limited herein.

Taking the sales field as an example, written language text T of the sales field is obtained.

Step 604: and identifying key words with parts of speech being preset parts of speech in the written language text by analyzing the parts of speech of the written language text.

And performing part-of-speech analysis on each word contained in the written language text T to obtain the part-of-speech of each word in the written language text T. And under the condition that the preset part of speech is the part of speech of a noun, identifying the words of the part of speech of the noun in the written language text T as key words.

Step 606: the positions of the key words are marked in the written language text.

Based on this, assuming that the key words identified in the written language text T are "computer" and "speed", the written language sentences SS to which these key words belong in the written language text T are: "i use the computer, the speed is very fast, and it is very convenient", carry on the position mark in the written language text T through the asterisk "+", the written language text T after the mark is finished is got after the mark. The written language sentence SS in the marked written language text T is changed to: "i use computer, speed is fast and very convenient".

Step 608: and translating the marked written language text into a translated written language text corresponding to the preset language.

Specifically, the preset language may be any one or more languages such as english, french, korean, and the like, which is not limited herein.

Based on this, under the condition that the preset language is English, the marked written language text T is translated into English, and an English translation written language text T1 corresponding to the marked written language text T is obtained.

Step 610: and translating the translated written language text into the target language to which the written language text belongs to obtain an initial retraced written language text.

Specifically, since the target language to which the text content in the written language text T belongs is chinese, the english translation written language text T1 is translated into chinese, and an initial retracing written language text T2 corresponding to the english translation written language text T1 is obtained, where a written language sentence SS2 corresponding to a written language sentence SS in the initial retracing written language text T2 is updated as follows: "i adopt computers, efficiency is fast and very convenient".

Step 612: and replacing the target key words corresponding to the position marks in the initial retranslate written language text by the key words to obtain the retranslate written language text.

Specifically, the target keyword corresponding to the position mark refers to a word marked by the position mark in the initial translated written language text, and the target keyword also corresponds to the keyword. In practical application, in combination with part-of-speech analysis of the written language text, the position of a word with a specific part-of-speech in the written language sentence is marked and replaced in the retracing process, so that the retracing of the written language text and key information in the written language text is ensured to be unchanged as much as possible.

Based on this, the target key words corresponding to the mark positions in the initial retracing written language text T2 are 'computer' and 'efficiency'; replacing the 'computer' in the initial retranslate written language text T2 by the keyword word 'computer', and replacing the 'efficiency' in the initial retranslate written language text T2 by the keyword word 'speed' to obtain a retranslate written language text T3, wherein a written language sentence SS3 corresponding to the written language sentence SS in the retranslate written language text T3 is updated as follows: 'I adopts a computer to carry out operation, has high speed and is very convenient'.

Step 614: and taking each written language text in the written language text and the retranslated written language text as the written text to be processed, and performing sentence recognition on each written language text to be processed to obtain written language sentences contained in each written language text to be processed.

Specifically, each written language text in the written language text T and the retranslated written language text T3 is used as a written text to be processed, and sentence recognition is performed on each text to be processed in sequence to obtain written language sentences contained in each text to be processed. Further, the following steps 616 to 622 are performed for each written sentence in each text to be processed.

Based on this, it is assumed that the written language text T is used as the text T to be processed, the text T to be processed is subjected to sentence recognition, n written language sentences included in the text T to be processed are obtained, the n written language sentences are written language sentence 1, written language sentence 2, … …, and written language sentence n, and the following steps 616 to 622 are performed on the n written language sentences, respectively.

Step 616: and performing clause unit conversion processing on the written language sentence to obtain a converted A4 written language sentence.

Specifically, the clause unit conversion processing is performed on any one written language sentence, and is specifically realized by executing the following steps 616-1 to 616-18:

step 616-1: and determining the copy clause conversion processing probability corresponding to the copy clause conversion processing strategy in each clause conversion processing strategy of the written language sentence.

The phrase conversion processing policy refers to a processing policy for copying phrases in written sentences, and the phrase conversion processing probability refers to a preset probability for executing the phrase conversion processing policy. The probability of the sentence transfer processing for duplication may be preset according to actual experience or spoken language expression habits, for example, the probability of the sentence transfer processing for duplication may be 10%, 20%, 30%, or the like, and is not limited herein. In the case where the probability of the duplicate clause conversion processing is 10%, it indicates that the duplicate clause conversion processing policy is executed with a probability of 10% for the written language sentence.

Based on this, assuming that the written sentence 1 is the written sentence SS "i use a computer, which is fast and convenient", and the probability of the duplicate clause conversion processing corresponding to the duplicate clause conversion processing policy is 10%, the probability of the duplicate clause conversion processing for executing the duplicate clause conversion processing policy with respect to the written sentence 1 is 10%.

Step 616-2: whether to execute a copy clause conversion processing policy for the written language sentence is determined based on the copy clause conversion processing probability.

Specifically, if it is determined to execute the duplicate clause transformation processing policy based on the duplicate clause transformation processing probability, the following step 616-3 is executed; if it is determined that the copy clause line feed processing policy is not to be executed, the written sentence is directly used as the A1 st written sentence, and the following step 616-5 is executed.

Step 616-3: under the condition that a strategy of executing the conversion processing of the copy clauses is determined, clause sampling is carried out on the written sentence according to a first preset sampling rule, and a first target clause in the written sentence is obtained.

Specifically, the first preset sampling rule is a preset sampling rule for sampling a clause to be copied in a written language sentence. The first preset sampling rule may be random sampling, or sampling according to positions, for example, a clause with a sampling position arranged at a first position in a written sentence, or sampling according to the number of characters, for example, a clause with a number of characters less than 5 in a sampling clause. The first preset sampling rule may be the same as the preset clause sampling rule in the method embodiment, and may also be understood as one of the preset clause sampling rules in the method embodiment. Accordingly, the first target clause refers to a clause sampled in the written language sentence according to the first preset sampling rule, and may also be understood as a target clause in the above method embodiment.

Based on this, in the case where it is determined to execute the strategy of duplicate clause conversion processing, clause sampling is randomly performed on the written sentence 1, and the first target clause in the written sentence 1 is obtained as "fast speed".

Step 616-4: and copying the first target clause to obtain a copy target clause, and inserting the copy target clause into the written language sentence according to a preset clause inserting position to obtain the converted A1 written language sentence.

Specifically, the preset clause inserting position refers to a preset position for inserting the target first clause into the written sentence, and the preset position may be set according to the actual spoken language characteristics, for example, the preset clause inserting position may be a beginning or an end of the written sentence, or may be before or after the position of the first target clause in the written sentence, which is not limited herein.

Based on this, the first target clause is copied at a high speed, the obtained copied target clause is also copied at a high speed, and under the condition that the preset clause inserting position is before the position of the first target clause, the copied target clause is inserted before the position of the first target clause in the written sentence 1, and the converted A1-th written sentence is obtained as follows: "I use the computer, it is fast, very fast, and very convenient".

Step 616-5: and determining the added clause conversion processing probability corresponding to the added clause conversion processing strategy in each clause conversion processing strategy of the A1 st written language sentence.

The addition clause conversion processing strategy refers to a processing strategy for adding clauses to written language sentences. Accordingly, the added clause conversion processing probability refers to a preset probability of executing processing related to the added clause conversion processing policy, and the added clause conversion processing probability may also be preset according to actual experience or spoken language expression habits, for example, the added clause conversion processing probability may be 15%, 20%, or the like, and is not limited herein. In the case where the added clause conversion processing probability is 15%, it indicates that the added clause conversion processing policy is executed with a probability of 15% for the written language sentence.

Based on this, the added clause conversion processing probability corresponding to the added clause conversion processing policy of the A1 st written language sentence is determined to be 15%.

Step 616-6: whether to execute an add clause conversion processing policy for the A1 st written language sentence is determined based on the add clause conversion processing probability.

Specifically, the specific implementation manner of determining whether to execute the add clause conversion processing policy for the A1 st written sentence based on the add clause conversion processing probability is similar to the specific implementation manner of determining whether to execute the copy clause conversion processing policy for the written sentence based on the copy clause conversion processing probability, and the specific implementation manner of determining whether to execute the copy clause conversion processing policy for the written sentence with reference to the copy clause conversion processing probability is not repeated herein.

In specific implementation, if it is determined to execute the add clause transformation processing policy, the following step 616-7 is executed; if it is determined that the add clause conversion processing policy is not to be executed, the following step 616-9 is executed with the A1 st written language sentence as the A2 nd written language sentence.

Based on this, it is assumed that the add clause conversion processing policy is determined to be executed for the A1 st written language sentence based on the add clause conversion processing probability of 15%.

Step 616-7: and under the condition that the strategy of adding clause conversion processing is determined to be executed, determining clause position probability distribution corresponding to the preset clauses contained in the preset clause set.

Specifically, the preset clause set includes 3 preset clauses, and the 3 preset clauses are preset clause 1, preset clause 2, and preset clause 3, respectively. The clause position probability distribution corresponding to the 3 preset clauses is as follows: the probability of adding preset clause 1 to the beginning of a sentence is 60%, the probability of adding preset clause 2 to the end of a sentence is 20%, and the probability of adding preset clause 3 to the beginning of a sentence is 20%.

Step 616-8: and determining a target preset clause and a clause adding position corresponding to the target preset clause in the preset clauses based on clause position probability distribution, and adding the target preset clause to the A1 st written language sentence according to the clause adding position to obtain the A2 nd written language sentence after conversion.

Specifically, based on the clause position probability distribution, it is determined that the target preset clause is preset clause 1 and the clause adding position corresponding to the preset clause 1 is the clause head in the preset clause, the preset clause 1 is added to the clause head of the A1-th written language sentence, and under the condition that the preset clause 1 is 'pair-to-pair', the converted A2-th written language sentence is obtained as 'pair-to-pair', i use a computer, which is fast, fast and convenient.

Step 616-9: and determining the out-of-order clause conversion processing probability corresponding to the out-of-order clause conversion processing strategy in each clause conversion processing strategy of the A2 nd written language sentence.

The out-of-order clause conversion processing strategy refers to a processing strategy for out-of-order clauses in written language sentences. Accordingly, the out-of-order clause transformation processing probability refers to a preset probability for executing the processing related to the out-of-order clause transformation processing strategy, and may also be preset according to actual experience or spoken language expression habits, for example, the out-of-order clause transformation processing probability may be 5%, 10%, and the like, and is not limited herein. In the case where the out-of-order clause conversion processing probability is 5%, it indicates that the out-of-order clause conversion processing policy is executed with a probability of 5% for written language sentences.

Based on this, the unordered clause conversion processing probability corresponding to the unordered clause conversion processing strategy of the A2 nd written sentence is determined to be 5%.

Step 616-10: and determining whether to execute the disorder clause conversion processing strategy aiming at the A2 written language sentence based on the disorder clause conversion processing probability.

Specifically, the specific implementation manner of determining whether to execute the disorder clause conversion processing policy for the A2 nd written sentence based on the disorder clause conversion processing probability is similar to the specific implementation manner of determining whether to execute the copy clause conversion processing policy for the written sentence based on the copy clause conversion processing probability, and the specific implementation manner of determining whether to execute the copy clause conversion processing policy for the written sentence with reference to the copy clause conversion processing probability is not repeated herein.

In specific implementation, if the out-of-order clause conversion processing strategy is determined to be executed, the following step 616-11 is executed; if it is determined that the out-of-order clause conversion processing policy is not to be executed, the A2 nd written sentence is directly used as the A3 rd written sentence, and the following steps 616-13 are executed.

Based on this, it is assumed that the out-of-order clause conversion processing policy is determined to be executed for the A2 nd written language sentence based on the added clause conversion processing probability of 5%.

Step 616-11: and under the condition that the out-of-order clause conversion processing strategy is determined to be executed, clause sampling is carried out on the A2 th written language sentence according to a second preset sampling rule, and a second target clause in the A2 th written language sentence is obtained.

Specifically, the second preset sampling rule refers to a preset sampling rule for sampling a clause to be disorderly in a written sentence, and the second preset sampling rule may be random sampling, or sampling according to a position, for example, a clause with a sampling position arranged at the last position in the written sentence, or sampling according to a number of characters, for example, a clause with a number of characters less than 5 in the sampling clause, and the like, which is not limited herein. In practical applications, the second preset sampling rule may be the same as the first preset sampling rule, or may be different from the first preset sampling rule, which is not limited herein. The second preset sampling rule may also be the same as the preset clause sampling rule in the above method embodiment, or may be understood as one of the preset clause sampling rules in the above method embodiment. Accordingly, the second target clause refers to a clause sampled in the written language sentence according to the second preset sampling rule, and may also be understood as a target clause in the above method embodiment.

Based on this, when the out-of-order clause conversion processing strategy is determined to be executed, clause sampling is randomly carried out on the A2 th written language sentence, and the second target clause in the A2 nd written language sentence is obtained as the 'pairwise-combination'.

Step 616-12: and deleting the second target clause from the A2 written language sentence, and inserting the second target clause into the deleted A2 written language sentence according to a preset clause insertion rule to obtain a converted A3 written language sentence.

Specifically, the second target clause "pairwise" is deleted in the A2 nd written language sentence, and the deleted A2 nd written language sentence is obtained: "I use the computer, it is fast, very fast, and very convenient". And randomly inserting the second target clause 'Pair' into the deleted A2 written language sentence, and obtaining a converted A3 written language sentence as follows: "I use the computer, the speed is very fast, and it is very convenient, to right".

Step 616-13: and determining flip clause conversion processing probability corresponding to the flip clause conversion processing strategy in each clause conversion processing strategy of the A3 rd written language sentence.

The inverted clause conversion processing strategy refers to a processing strategy for inverting the word order of clauses in written language sentences (for example, inverting a main predicate object structure into a guest predicate object structure). In practical applications, although the word order of the clauses is not consistent, the meaning of the expression is the same, so that the word order in the clauses may not be consistent with the word order in the written language sentences in the spoken language sentences. In order to make the converted written language more consistent with the characteristics of the spoken language, the language order of some clauses of the written language sentence can be reversely processed. Accordingly, the probability of the flip-chip clause transformation processing is a preset probability of executing the processing related to the policy of the flip-chip clause transformation processing, and the probability of the flip-chip clause transformation processing may also be preset according to practical experience or oral expression habits, for example, the probability of the flip-chip clause transformation processing may be 3% or 5%, and is not limited herein. In the case where the flip-chip clause conversion processing probability is 3%, it indicates that the flip-chip clause conversion processing policy is performed with a probability of 3% for written language sentences.

Based on this, the flip clause conversion processing probability corresponding to the flip clause conversion processing policy of the A3 rd written language sentence is determined to be 3%.

Step 616-14: based on the flip clause conversion processing probability, it is determined whether or not a flip clause conversion processing policy is executed for the A3 rd written language sentence.

Specifically, the specific implementation manner of determining whether to execute the inversion clause conversion processing strategy for the A3 rd written sentence based on the inversion clause conversion processing probability is similar to the specific implementation manner of determining whether to execute the inversion clause conversion processing strategy for the written sentence based on the inversion clause conversion processing probability, and the specific implementation manner of determining whether to execute the inversion clause conversion processing strategy for the written sentence based on the inversion clause conversion processing probability is determined by referring to the specific implementation manner of determining whether to execute the inversion clause conversion processing strategy for the written sentence based on the inversion clause conversion processing probability, which is not described herein again.

In specific implementation, if it is determined to execute the flip-chip clause transformation processing policy, the following steps 616-15 are executed; if it is determined that the flip-chip clause conversion processing strategy is not to be executed, the following step 618 is executed with the A3 rd written language sentence as the A4 th written language sentence.

Step 616-15: and under the condition that the strategy of executing the inversion clause conversion processing is determined, clause sampling is carried out on the A3 th written language sentence according to a third preset sampling rule, and a third target clause in the A3 th written language sentence is obtained.

Specifically, the third preset sampling rule refers to a preset sampling rule for sampling a clause to be flipped in a written sentence, and the third preset sampling rule may be random sampling, or sampling according to a position, for example, a clause with a sampling position arranged at a beginning position of the written sentence, or sampling according to a number of characters, for example, a clause with a number of characters greater than 5 in the sampling clause, and the like, which is not limited herein. In practical applications, the third preset sampling rule may be the same as the first preset sampling rule or the second preset sampling rule, or may be different from the first preset sampling rule or the second preset sampling rule, which is not limited herein. The third preset sampling rule may also be the same as the preset clause sampling rule in the above method embodiment, or may be understood as one of the preset clause sampling rules in the above method embodiment. Accordingly, the third target clause refers to a clause sampled in the written language sentence according to the third preset sampling rule, and may also be understood as a target clause in the above method embodiment.

Based on this, in the case of determining to execute the strategy of inversion clause conversion processing, clause sampling is randomly performed on the A3 rd written sentence, and the third target clause in the A3 rd written sentence is obtained as 'I use computer'.

Step 616-17: and carrying out syntactic analysis on the third target clause to obtain a syntactic structure corresponding to the third target clause.

Specifically, the syntax analysis is performed on the third target clause, and a syntax structure corresponding to the third target clause is obtained as a major-predicate structure.

Step 616-18: and converting the third target clause according to the target syntactic structure corresponding to the syntactic structure to obtain a converted A4 written language sentence.

Specifically, under the condition that the target syntax structure corresponding to the major-predicate object structure is the minor-predicate main structure, the third target clause is converted according to the minor-predicate main structure, and the A4 written language sentence after conversion is obtained is: "the computer is used by me, and is very fast, and very convenient, to right".

Step 618: and performing word unit conversion processing on the A4 th written language sentence to obtain a B3 th written language sentence.

Specifically, on the basis of performing clause unit conversion processing on the written language sentence to obtain an A4 th written language sentence, performing word unit conversion processing on the A4 th written language sentence, specifically, the following steps 618-1 to 618-12 are performed:

step 618-1: and determining the added word conversion processing probability corresponding to the added word conversion processing strategy in each word conversion processing strategy of the A4 th written language sentence.

Specifically, the adding of the word conversion processing policy refers to a processing policy of adding words to the written sentence. Accordingly, the addition word conversion processing probability refers to a probability of executing an addition word conversion processing policy set in advance. The added word conversion processing probability may be preset according to actual experience or spoken language expression habits, for example, the added word conversion processing probability may be 10% or 13%, and is not limited herein. In the case where the added word conversion processing probability is 10%, it indicates that the added word conversion processing policy is executed with a probability of 10% for the written language sentence.

Based on this, the A4 written language sentence is determined: the "computer is used by me, and is fast, and very convenient, and the added word conversion processing probability corresponding to the added word conversion processing policy in the word conversion processing policy for the pair" is 10%, and the added word conversion processing probability of executing the added word conversion processing policy for the A4 th written language sentence is 10%.

Step 618-2: based on the addition word conversion processing probability, it is determined whether or not an addition word conversion processing policy is executed for the A4 th written language sentence.

Specifically, the specific implementation manner of determining whether to execute the adding word conversion processing policy for the A4 th written sentence based on the adding word conversion processing probability is similar to the specific implementation manner of determining whether to execute the copying clause conversion processing policy for the written sentence based on the copying clause conversion processing probability, and the specific implementation manner of determining whether to execute the copying clause conversion processing policy for the written sentence based on the copying clause conversion processing probability is determined with reference to the specific implementation manner of determining whether to execute the copying clause conversion processing policy for the written sentence, which is not described herein again.

In specific implementation, if the adding word conversion processing strategy is determined to be executed, the following step 618-3 is executed; if it is determined that the add word conversion processing policy is not to be executed, the following step 618-5 is executed by directly regarding the A4 th written language sentence as the B1 th written language sentence.

Based on this, it is assumed that the addition word conversion processing policy is determined to be executed for the A4 th written language sentence based on the addition word conversion processing probability of 10%.

Step 618-3: and under the condition that the added word conversion processing strategy is determined to be executed, determining word position probability distribution corresponding to preset words contained in the preset word set.

Specifically, 2 preset words are included in the preset word set, and the 2 preset words are preset word 1 and preset word 2 respectively. According to the statistics of the spoken language corpus in the sales field, the probability distribution of the word positions corresponding to the two preset words is as follows: the probability of adding word 1 to the beginning of a sentence is preset to be 80%, and the probability of adding word 2 to the end of a sentence is preset to be 20%.

Step 618-4: and determining a target preset word and a word adding position corresponding to the target preset word in the preset words according to the word position probability distribution, and adding the target preset word to the A4 th written language sentence according to the word adding position to obtain the converted B1 th written language sentence.

Specifically, based on the word position probability distribution, the target preset word is preset word 1 in the preset words, and the preset adding position corresponding to the preset word 1 is a sentence head, then the preset word 1 is added to the sentence head of the A4 written language sentence, and under the condition that the preset word 1 is 'ousse', the converted B1 written language sentence is obtained as 'computer used by me, ousse, speed is very high, and the method is very convenient and fast and is in opposite pair'.

Step 618-5: and determining the conversion processing probability of the duplicate words corresponding to the conversion processing strategy of the duplicate words in each word conversion processing strategy of the B1 written language sentence.

The duplication term conversion processing policy refers to a processing policy for duplicating the target preset term added in the step 618-4 in the written sentence. Accordingly, the duplication term conversion processing probability refers to a preset probability for executing the processing related to the duplication term conversion processing policy, and the duplication term conversion processing probability may also be preset according to actual experience or spoken language expression habits, for example, the duplication term conversion processing probability may be 8% or 12%, and is not limited herein. In the case where the probability of duplicate word conversion processing is 8%, it indicates that the duplicate word conversion processing policy is executed with a probability of 8% for the written language sentence.

Based on this, the duplicate word conversion processing probability corresponding to the duplicate word conversion processing policy of the B1 th written language sentence was determined to be 8%.

Step 618-6: based on the duplication word conversion processing probability, it is determined whether or not a duplication word conversion processing policy is executed for the B1 st written language sentence.

Specifically, the specific implementation manner of determining whether to execute the duplication term conversion processing policy for the written sentence according to the B1 th written sentence based on the duplication term conversion processing probability is similar to the specific implementation manner of determining whether to execute the duplication term conversion processing policy for the written sentence based on the duplication clause conversion processing probability, and the specific implementation manner of determining whether to execute the duplication term conversion processing policy for the written sentence according to the duplication term conversion processing probability is determined, which is not described herein again.

In specific implementation, if the duplicate word conversion processing policy is determined to be executed, the following step 618-7 is executed; if it is determined that the duplicate word conversion processing policy is not to be executed, the B1 st written language sentence is directly used as the B2 nd written language sentence, and the following step 618-8 is executed.

Based on this, it is assumed that the duplication word conversion processing policy is determined to be executed for the B1 th written language sentence based on the duplication word conversion processing probability of 8%.

Step 618-7: under the condition that a strategy of converting and processing the copied words is determined to be executed, the target preset words added in the B1 th written language sentence are copied to obtain the copied words, the copied words are inserted into the B1 th written language sentence according to preset word insertion rules, and the inserted B2 th written language sentence is obtained.

Based on this, under the condition that it is determined to execute the duplication word conversion processing strategy, the target preset word "java" in the B1 th written language sentence is duplicated, the obtained duplication word is also "java", and under the condition that the preset word insertion rule is that the target preset word is inserted before the position where the target preset word is located in the B1 th written language sentence, the inserted B2 th written language sentence is obtained by: 'computer is used by me, and the Java is quick, fast, convenient and opposite'.

Step 618-8: and determining the out-of-order word conversion processing probability corresponding to the out-of-order word conversion processing strategy in all the word conversion processing strategies of the B2 written language sentence.

The out-of-order word conversion processing strategy refers to a processing strategy for out-of-order words in written language sentences. Correspondingly, the out-of-order word conversion processing probability refers to a preset probability for executing relevant processing of an out-of-order word conversion processing strategy, and the out-of-order word conversion processing probability may also be preset according to actual experience or a spoken language expression habit, for example, the out-of-order word conversion processing probability may be 6%, 9%, and the like, which is not limited herein. In the case where the out-of-order word conversion processing probability is 6%, it indicates that the out-of-order word conversion processing policy is executed with a probability of 6% for the written language sentence.

Based on this, the out-of-order word conversion processing probability corresponding to the out-of-order word conversion processing policy of the B2 nd written language sentence is determined to be 6%.

Step 618-9: and determining whether to execute the out-of-order word conversion processing strategy aiming at the B2 written language sentence based on the out-of-order word conversion processing probability.

Specifically, the specific implementation manner of determining whether to execute the out-of-order word conversion processing policy for the B2 nd written sentence based on the out-of-order word conversion processing probability is similar to the specific implementation manner of determining whether to execute the copy clause conversion processing policy for the written sentence based on the copy clause conversion processing probability, and the specific implementation manner of determining whether to execute the copy clause conversion processing policy for the written sentence based on the copy clause conversion processing probability is determined with reference to the specific implementation manner of determining whether to execute the copy clause conversion processing policy for the written sentence, which is not described herein again.

In specific implementation, if the out-of-order word conversion processing strategy is determined to be executed, the following step 618-10 is executed; if it is determined that the out-of-order word conversion processing strategy is not to be executed, the B2 written language sentence is directly used as the B3 written language sentence, and the following step 620 is executed.

Based on this, it is assumed that the out-of-order word conversion processing policy is determined to be executed for the B2 nd written language sentence based on the out-of-order word conversion processing probability 5%.

Step 618-10: under the condition that the out-of-order word conversion processing strategy is determined to be executed, word sampling is carried out on words in the B2 written language sentence according to a preset word sampling rule, and a target word in the B2 written language sentence is obtained. Based on this, in the case of determining the out-of-order word conversion processing strategy to be executed, 2-character word sampling is randomly performed on the B2 nd written language sentence, and the target word in the B2 nd written language sentence is obtained as "use".

Step 618-11: and deleting the target words in the B2 th written language sentence, and inserting the target words into a preset insertion range corresponding to the target words in the deleted B2 th written language sentence to obtain the converted B3 th written language sentence.

Specifically, the target word "use" is deleted in the B2 written language sentence, and the deleted B2 written language sentence is obtained: "the computer is my, fast, very fast and very convenient". And randomly inserting the target word 'computing' into the character interval of [ -3,3] in the deleted B2 written language sentence to obtain a converted B3 written language sentence as follows: ' computer use is my, java and Java ' plug, the speed is very fast, and it is very convenient, to right '.

Step 620: and performing character unit conversion processing on the B3 written language sentence to obtain a C written language sentence.

Specifically, on the basis of performing the word unit conversion processing on the written sentence to obtain the B3 th written sentence, the character unit conversion processing is performed on the B3 th written sentence, and the following steps 620-1 to 620-4 are performed as follows:

step 620-1: and determining the out-of-order character conversion processing probability corresponding to the out-of-order character conversion processing strategy of the B3 written language sentence.

Specifically, the out-of-order character conversion processing strategy refers to a processing strategy for out-of-order characters in written language sentences. Accordingly, the out-of-order character conversion processing probability refers to a preset probability for executing the relevant processing of the out-of-order character conversion processing strategy, and the out-of-order character conversion processing probability may also be preset according to actual experience or spoken language expression habits, for example, the out-of-order character conversion processing probability may be 5%, 9%, and the like, and is not limited herein. In the case where the out-of-order character conversion processing probability is 5%, it indicates that the out-of-order character conversion processing policy is executed with a probability of 6% for the written language sentence.

Based on this, the out-of-order character conversion processing probability corresponding to the out-of-order character conversion processing policy of the B3 th written language sentence is determined to be 5%.

Step 620-2: determining whether to execute an out-of-order character conversion processing strategy for the B3 written language sentence based on the out-of-order character conversion processing probability.

Specifically, the specific implementation manner of determining whether to execute the out-of-order character conversion processing strategy for the B3 rd written sentence based on the out-of-order character conversion processing probability is similar to the specific implementation manner of determining whether to execute the copy clause conversion processing strategy for the written sentence based on the copy clause conversion processing probability, and the specific implementation manner of determining whether to execute the copy clause conversion processing strategy for the written sentence based on the copy clause conversion processing probability is determined with reference to the specific implementation manner of determining whether to execute the copy clause conversion processing strategy for the written sentence, which is not described herein again.

In specific implementation, if the out-of-order character conversion processing strategy is determined to be executed, the following step 620-3 is executed; if it is determined that the out-of-order character conversion processing strategy is not to be executed, the following step 622 is executed by directly regarding the B3 written language sentence as the C written language sentence.

Based on this, it is assumed that the out-of-order character conversion processing policy is determined to be executed for the B3 rd written language sentence based on the out-of-order character conversion processing probability 5%.

Step 620-3: under the condition that the out-of-order character conversion processing strategy is determined to be executed, character sampling is carried out on characters in the B3 written language sentence according to a preset character sampling rule, and target characters in the B3 written language sentence are obtained. Based on this, in the case of determining the execution out-of-order character conversion processing strategy, the B3 th written language sentence is randomly subjected to character sampling, and the target character in the B3 th written language sentence is obtained as "AND".

Step 620-4: and deleting the target characters in the B3 th written language sentence, and inserting the target characters into a preset character insertion range corresponding to the target characters in the deleted B3 th written language sentence to obtain the converted C written language sentence. Specifically, the target character "make" is deleted in the B3 written language sentence, and the deleted B3 written language sentence is obtained: 'computer is used by me, wassen, fast, convenient, right to right'. And randomly inserting the target character into the clause range to which the target character belongs in the deleted B3 written language sentence to obtain a C written language sentence after conversion as follows: 'the computer is used by me, the Java plug and the Java plug, the speed is high, and the method is very convenient and fast and is paired'.

Step 622: and performing symbol unit conversion processing on the C written language sentence to obtain a D2 written language sentence.

Specifically, on the basis of performing the character unit conversion processing on the written language sentence to obtain the C written language sentence, the symbol unit conversion processing is performed on the C written language sentence, and the following steps 622-1 to 622-8 are performed specifically:

step 622-1: and determining the deleted symbol conversion processing probability corresponding to the deleted symbol conversion processing strategy in the symbol conversion processing strategy of the C written language sentence.

Specifically, the delete symbol conversion processing policy refers to a processing policy for deleting symbols from written sentences. Accordingly, the erasure symbol conversion processing probability refers to a preset probability for executing the erasure symbol conversion processing policy related processing, and the erasure symbol conversion processing probability may also be preset according to actual experience or spoken language expression habits, for example, the erasure symbol conversion processing probability may be 8% or 12%, and is not limited herein. In the case where the erasure conversion processing probability is 8%, it indicates that the erasure conversion processing policy is executed with a probability of 8% for the written language sentence.

Based on this, it is determined that the erasure conversion processing probability corresponding to the erasure conversion processing policy in the symbol conversion processing policy of the C-th written language sentence is 8%, and the probability of executing the erasure conversion processing policy for the C-th written language sentence is 8%.

Step 622-2: and determining whether to execute a delete symbol conversion processing strategy for the C written language sentence based on the delete symbol conversion processing probability.

Specifically, the specific implementation manner of determining whether to execute the deletion symbol conversion processing policy for the C-th written sentence based on the deletion symbol conversion processing probability is similar to the specific implementation manner of determining whether to execute the copy clause conversion processing policy for the written sentence based on the copy clause conversion processing probability, and the specific implementation manner of determining whether to execute the copy clause conversion processing policy for the written sentence with reference to the copy clause conversion processing probability is determined, which is not described herein again.

In specific implementation, if it is determined to execute the delete symbol conversion processing policy, the following step 622-3 is executed; if it is determined that the delete symbol conversion processing policy is not to be executed, the C-th written language sentence is directly used as the D1-th written language sentence, and the following step 622-5 is executed.

Based on this, it is assumed that the delete symbol conversion processing policy is determined to be executed for the C written language sentence based on the delete symbol conversion processing probability of 8%.

Step 622-3: and under the condition that the deletion symbol conversion processing strategy is determined to be executed, symbol sampling is carried out on the C written language sentence according to a preset symbol sampling rule, and a target punctuation mark in the C written language sentence is obtained. Based on the method, under the condition that a strategy for executing the symbol deletion conversion processing is determined, symbol sampling is randomly carried out on the C written language sentence, and a comma of which the target punctuation in the C written language sentence is the first 'fast speed' clause is obtained.

Step 622-4: and deleting the target punctuation marks in the C written language sentence to obtain a converted D1 written language sentence.

Specifically, the comma after the first 'fast' clause is deleted in the C written language sentence, and the D1 written language sentence after deletion is obtained: 'the computer is used by me, the Java plug and the Java plug are fast and convenient and are in opposite pairs'.

Step 622-5: and determining the added symbol conversion processing probability corresponding to the added symbol conversion processing strategy in the symbol conversion processing strategy of the D1 written language sentence.

The adding symbol conversion processing strategy refers to a processing strategy for adding symbols to written language sentences. Accordingly, the add symbol conversion processing probability refers to a preset probability for executing the processing related to the add symbol conversion processing strategy, and the add symbol conversion processing probability may also be preset according to actual experience or spoken language expression habits, for example, the add symbol conversion processing probability may be 2%, 5%, and the like, and is not limited herein. In the case where the probability of the addition symbol conversion processing is 2%, it is indicated that the addition symbol conversion processing policy is executed with a probability of 2% for the written language sentence.

Based on this, the adding symbol conversion processing probability corresponding to the adding symbol conversion processing strategy of the D1 th written language sentence is determined to be 2%.

Step 622-6: and determining whether to execute the adding symbol conversion processing strategy for the D1 written language sentence based on the adding symbol conversion processing probability.

Specifically, the specific implementation manner of determining whether to execute the add symbol conversion processing policy for the D1 st written sentence based on the add symbol conversion processing probability is similar to the specific implementation manner of determining whether to execute the copy clause conversion processing policy for the written sentence based on the copy clause conversion processing probability, and the specific implementation manner of determining whether to execute the copy clause conversion processing policy for the written sentence with reference to the copy clause conversion processing probability is determined, which is not described herein again.

In specific implementation, if it is determined to execute the add symbol conversion processing strategy, the following step 622-7 is executed; if it is determined that the addition symbol conversion processing strategy is not to be executed, the following step 624 is executed by directly regarding the D1 th written language sentence as the D2 th written language sentence.

Based on this, it is assumed that the addition symbol conversion processing policy is determined to be executed for the D1 written language sentence based on the addition symbol conversion processing probability 2%.

Step 622-7: under the condition that the strategy of adding symbol conversion processing is determined to be executed, symbol clause sampling is carried out on the D1 written language sentence according to a preset symbol clause sampling rule, and a target symbol clause in the D1 written language sentence is obtained.

Based on this, under the condition that the strategy of adding symbol conversion processing is determined to be executed, the clauses with the largest number of sampled characters in the statement of the D1 written language include 'fast and fast' and 'very convenient', and one clause is randomly selected from the two clauses to obtain a target character clause, wherein the target character clause is 'fast and fast'.

Step 622-8: and randomly inserting preset punctuation marks into the target symbolic clauses to obtain the converted D2 written language sentences, and taking the D2 written language sentences as target spoken language sentences.

Specifically, the predetermined punctuation mark is "! "in the case of the target symbol clause" fast enough "a predetermined punctuation symbol"! ", the converted D2 written language sentence is obtained as: ' the computer is used by me, the Java plug and the Java plug are fast and fast! And is very convenient and fast to pair. And taking the D2 written language sentence as a target spoken language sentence.

Step 624: and combining the D2 written language sentences corresponding to each written sentence in each text to be processed to obtain the spoken text corresponding to each text to be processed.

Specifically, after the steps 616 to 622 are respectively executed on n written sentences included in the text T to be processed, the target spoken sentence corresponding to each written sentence in the text T to be processed may be obtained, and then the n target spoken sentences are combined according to the sentence order of the n written sentences in the text T to be processed, so as to obtain the spoken text corresponding to the text T to be processed. Wherein, the spoken text comprises the target spoken sentence corresponding to the written sentence 1: ' the computer is used by me, the Java plug and the Java plug are fast and fast! And is very convenient, to pair ".

Step 626: abnormal information in the spoken text is identified.

Specifically, the method includes the steps of recognizing that the abnormal information in the target spoken sentence corresponding to the written sentence 1 in the spoken text is'! "

Step 628: and cleaning the spoken language text according to the abnormal information to obtain the cleaned spoken language text.

In practical application, the spoken language text is subjected to data cleaning according to the abnormal information, and the abnormal information identified in the spoken language text can be filtered or adjusted.

Specifically, based on the exception information! "data cleaning is performed on the target spoken sentence corresponding to the written sentence 1 in the spoken text, and the target spoken sentence corresponding to the written sentence 1 in the spoken text after cleaning is obtained is changed into: ' the computer is used by me, the Java plug and the Java plug are fast! And is very convenient and fast to pair.

Step 630: and constructing a first sample corpus based on the corresponding relation between the written language text and the translated written language text and the cleaned spoken language text.

In practical applications, the corresponding cleaned spoken text can be obtained for each written or translated written text. Therefore, a large number of written language texts can be obtained, and the steps 602 to 628 are performed for each written language text, so as to obtain the translated back written language text corresponding to each written language text and the cleaned spoken language text corresponding to the written language text. And combining the written language text and the cleaned spoken language text which have corresponding relation into sample corpus pairs, and combining the sample corpus pairs into a first sample corpus.

Further, since each conversion processing strategy has a respective conversion processing probability, whether each conversion processing strategy is executed or not is not fixed in each execution of the above steps 616 to 622, and therefore, the spoken texts generated in each execution of the above steps 616 to 622 are very different in probability. Based on this, the above steps 616 to 622 can be executed for each text to be processed multiple times, so as to generate a plurality of spoken texts for each text to be processed, thereby further expanding the first sample corpus.

Specifically, m written language texts in the sales field (the m written language texts include the written language text T) are obtained, the retranslation processing from the step 604 to the step 612 is performed on each written language text, m retranslated written language texts are obtained, the m written language texts and the m retranslated written language texts are used as texts to-be-processed, the step 614 to the step 628 is performed, m cleaned spoken texts corresponding to the m written language texts and m cleaned spoken texts corresponding to the m retranslated written language texts are obtained, the m written language texts are used as training samples, the m cleaned spoken texts corresponding to the m written language texts are used as sample tags, the m retranslated written language texts are used as training samples, and the m cleaned spoken texts corresponding to the m translated written language texts are used as sample tags, so as to construct a first sample corpus, wherein the first sample corpus can be understood as a sample corpus in the method embodiment.

Step 632: and training the initial written language rewrite model through the first sample corpus until a first written language rewrite model meeting a first training stop condition is obtained.

The first written language rewrite model may be understood as the written language standard rewrite model in the above method embodiment. Specifically, the initial written language rewrite model is trained through the first sample corpus, and under the condition that the training meets the preset i iterations, the training is stopped, and a first written language rewrite model M1 is obtained.

In addition, on the basis of setting a lower probability value for the plurality of conversion processing probabilities in the above steps 616 to 622, the above steps 616 to 628 are performed again on the written language sentences included in the above m written language texts to obtain m cleaned spoken language texts SST corresponding to the m written language texts, and then a second sample corpus is constructed based on the m written language texts as training samples and the m cleaned spoken language texts SST corresponding to the training samples as sample tags. The second sample corpus may be understood as the micro sample corpus in the above method embodiment. And then training the initial written language rewriting model by using a second sample corpus until a second written language rewriting model M2 meeting a second training stop condition is obtained, wherein the second written language rewriting model can be understood as a written language micro-rewriting model in the method embodiment.

In a specific implementation, the second training stopping condition is a training stopping condition for training the initial written language rewriting model by using the second sample corpus, and the second training stopping condition may be the same as or different from the first training stopping condition, which is not limited herein. The second written-text rewrite model may be understood as a model of a slightly written-text rewrite of a spoken text after training. In concrete implementation, the second written language rewrite model is model-trained using a sample corpus simpler than that of the first written language rewrite model, and therefore, the second written language rewrite model can be used to deal with a lighter sentence rewrite.

Step 634: and acquiring a target spoken language text.

Specifically, a target spoken language text T4 is obtained. The target spoken text T4 may be any spoken text in the sales field.

Step 636: and carrying out text classification on the target spoken language text through a text classification model to obtain a text type.

The text classification model refers to a pre-trained model for classifying spoken texts, and in practical applications, the text classification model may be a CNN (convolutional neural network), RNN (cyclic neural network), LSTM (long-term memory network), fastText, textCNN, HAN model, and the like, which is not limited herein.

In practical application, a large number of spoken texts can be obtained, and the spoken texts are labeled to obtain a text label corresponding to each spoken text, where the text label includes: invalid text type, fuzzy text type, standard text type, etc., without limitation. And then constructing a training sample through the spoken text and the text label corresponding to the spoken text, and training an initial text classification model through the training sample to obtain the trained text classification model.

Specifically, the target spoken language text T4 is subjected to text classification by a text classification model trained in advance, and a text type corresponding to the target spoken language text T4 output by the text classification model is obtained.

Step 638: and deleting the target spoken text in the case that the text type is an invalid text type.

Specifically, if the text type is an invalid text type, the target spoken text T4 may be deleted.

Step 640: and under the condition that the text is classified into a clear text type, inputting the target spoken text into a first written language rewriting model to rewrite the written language, and obtaining a first target written language text output by the first written language rewriting model.

Specifically, assuming that the text type is a clear text type, the target spoken text T4 is input to the first written language rewrite model M1 to rewrite the written language, and the first target written language text T5 output by the first written language model M1 is obtained.

Step 642: and under the condition that the text type is the fuzzy text type, inputting the target spoken language text into the second written language rewriting model to rewrite the written language, and obtaining a second target written language text output by the second written language rewriting model.

Specifically, assuming that the text type is a fuzzy text type, the target spoken text T4 is input to the second written language rewriting model M2 to be rewritten in written language, and the second target written language text T6 output by the second written language model M2 is obtained.

In summary, the text processing method provided in the embodiment of the present application obtains the retranslate written text by performing retranslation processing on the written text, so as to expand the original written text with the retranslate written text. On the basis, the conversion processing of clause level, word level, character level and symbol level is carried out on the written language text by the preset conversion processing probability, so that the spoken language text corresponding to the written language text is further expanded. And the expanded written language text and the spoken language text are combined to generate the sample linguistic data, so that the sample linguistic data of the written language rewrite model are automatically generated, the sample linguistic data of the written language rewrite model are enriched, the generation efficiency of the sample linguistic data is improved, and the rewrite accuracy of the written language rewrite model is improved indirectly by enriching the sample linguistic data.

Corresponding to the above method embodiment, the present application further provides a text processing apparatus embodiment, and fig. 7 shows a schematic structural diagram of the text processing apparatus provided in an embodiment of the present application. As shown in fig. 7, the apparatus includes:

a first obtaining module 702 configured to obtain written language text;

a first translation module 704 configured to obtain a translated back written language text corresponding to the written language text by performing translation back processing on the written language text;

a first conversion module 706 configured to perform conversion processing of sentence forming units on the written language text and the retraced written language text, respectively, to obtain a spoken language text;

a construction module 708 configured to construct a sample corpus based on the correspondence between the written language text and the translated back written language text and the spoken language text.

Optionally, the first translation module 704 includes:

the translation sub-module is configured to translate the written language text into a translated text written language text corresponding to a preset language;

the translation module is configured to translate the written translated text back into a target language to which the written text belongs to obtain an initial translated written text;

and the replacing submodule is configured to replace target key words corresponding to the key words in the initial retranslate written language text by the key words in the written language text to obtain a retranslate written language text.

Optionally, the first translation module 704 further includes:

the part-of-speech analysis submodule is configured to identify key words of which the parts of speech are preset parts of speech in the written language text by performing part-of-speech analysis on the written language text;

a marking sub-module configured to mark the position of the key words in the written language text;

accordingly, the replacement sub-module is further configured to:

and replacing the target key words corresponding to the position marks in the initial retranslate written language text through the key words to obtain the retranslate written language text.

Optionally, in a case that the sentence component unit is a clause unit, the first conversion module 706 includes:

the first recognition submodule is configured to perform sentence recognition on the written language text to be processed to obtain written language sentences contained in the written language text to be processed;

a clause conversion submodule configured to perform clause unit conversion processing on the written language sentence to obtain a converted written language sentence;

a first determination module configured to determine a spoken text based on the converted written language sentence.

Optionally, the clause conversion submodule is further configured to:

performing clause sampling on the written sentence according to a preset clause sampling rule to obtain a target clause in the written sentence; converting the target clause in the written sentence to obtain a converted written sentence; and/or

Determining clause position probability distribution corresponding to preset clauses contained in a preset clause set; determining a target preset clause and a clause adding position corresponding to the target preset clause in the preset clauses based on the clause position probability distribution; and adding the target preset clause into the written sentence according to the clause adding position to obtain the converted written sentence.

Optionally, the clause conversion sub-module is further configured to:

copying the target clause to obtain a copied target clause, and inserting the copied target clause into the written language sentence according to a preset clause inserting position to obtain a converted written language sentence; and/or the presence of a gas in the atmosphere,

deleting the target clause in the written language sentence; inserting the target clause into the deleted written language sentence according to a preset clause insertion rule to obtain a converted written language sentence; and/or

Carrying out syntactic analysis on the target clause to obtain a syntactic structure corresponding to the target clause; and converting the target clause according to the target syntax structure corresponding to the syntax structure to obtain the converted written language sentence.

Optionally, in a case that the sentence component unit is a word unit, the first conversion module 706 includes:

the second identification submodule is configured to perform sentence identification on the written language text to be processed to obtain written language sentences contained in the written language text to be processed;

the distribution determining submodule is configured to determine word position probability distribution corresponding to preset words contained in the preset word set;

the word adding sub-module is configured to determine a target preset word and a word adding position corresponding to the target preset word in the preset words according to the word position probability distribution, insert and add the target preset word into the written language sentence according to the word adding position, and obtain the converted written language sentence;

a second determination module configured to determine a spoken text based on the converted written language sentence.

Optionally, in a case that the sentence component unit is a word unit, the first conversion module 706 is further configured to:

performing sentence recognition on the written language text to be processed to obtain written language sentences contained in the written language text;

performing word sampling on words in the written language sentences according to preset word sampling rules to obtain target words in the written language sentences;

deleting the target words in the written sentence, and inserting the target words into a preset insertion range corresponding to the target words in the deleted written sentence to obtain a converted written sentence;

determining spoken text based on the converted written language sentence.

Optionally, the first conversion module 706 is further configured to:

the word copying sub-module is configured to copy the target preset words added in the converted written language sentences to obtain copied words;

the inserting word sub-module is configured to insert the copied words into the converted written language sentences according to preset word inserting rules to obtain the inserted written language sentences;

a third determination module configured to determine the spoken text based on the inserted written language sentence.

Optionally, in a case that the sentence component unit is a character unit, the first conversion module 706 is further configured to:

deleting the target characters in the written language sentence, and inserting the target characters into a preset character insertion range corresponding to the target characters in the deleted written language sentence to obtain a converted written language sentence;

determining spoken text based on the converted written language sentence.

Optionally, in a case that the sentence component unit is a symbol unit, the first conversion module 706 is further configured to:

performing sentence recognition on the written language text to be processed to obtain written language sentences contained in the written language text to be processed; symbol sampling is carried out on the written sentence according to a preset symbol sampling rule, a target punctuation mark in the written sentence is obtained, the target punctuation mark is deleted from the written sentence, and the converted written sentence is obtained; determining a spoken text based on the converted written language sentence; and/or

Performing sentence recognition on the written language text to be processed to obtain written language sentences contained in the written language text to be processed; carrying out symbol clause sampling on the written language sentence according to a preset symbol clause sampling rule to obtain a target symbol clause in the written language sentence, and inserting a preset punctuation symbol into the target symbol clause to obtain a converted written language sentence; determining spoken text based on the converted written language sentence.

Optionally, the first conversion module 706 is further configured to:

Optionally, the text processing apparatus further includes:

an identification information module configured to identify abnormal information in the spoken text;

the cleaning module is configured to perform data cleaning on the spoken language text according to the abnormal information to obtain a cleaned spoken language text;

and the sample corpus building module is configured to build a sample corpus based on the corresponding relation between the written language text and the retranslated written language text and the cleaned spoken language text.

Optionally, the text processing apparatus further includes:

and the training module is configured to train the initial written language rewriting model through the sample corpus until the written language rewriting model meeting the training stopping condition is obtained.

Optionally, the first conversion module 706 is further configured to:

performing conversion processing of sentence composition units on the written language text to obtain a first spoken language text corresponding to the written language text;

performing sentence component unit conversion processing on the retranslate written language text to obtain a second spoken language text corresponding to the retranslate written language text;

and taking the first spoken text and the second spoken text as the spoken text.

In conclusion, the written language text is obtained and the retranslated written language text corresponding to the written language text is obtained by obtaining the written language text and performing retranslation processing on the written language text, so that the written language material is expanded in a retranslation mode. And then the written language text and the retraced written language text are respectively subjected to sentence component conversion processing to obtain the spoken language text, so that the written language materials expanded based on retracing are converted, and the written language text is rewritten. And constructing a sample corpus based on the corresponding relation between the written language text and the translated written language text and the spoken language text. The method and the device have the advantages that the written language corpus after the retracing expansion and the converted spoken language text are used for providing a large number of sample corpora of the spoken text-written language text for model training, the training difficulty of the model is simplified, the phenomenon that a large amount of text data are collected and processed manually and time and labor are wasted is avoided, and the time cost and the labor cost are saved.

The above is a schematic scheme of a text processing apparatus of the present embodiment. It should be noted that the technical solution of the text processing apparatus and the technical solution of the text processing method belong to the same concept, and details that are not described in detail in the technical solution of the text processing apparatus can be referred to the description of the technical solution of the text processing method.

Corresponding to the above method embodiment, the present application further provides an embodiment of a spoken text generation apparatus, and fig. 8 shows a schematic structural diagram of the spoken text generation apparatus provided in an embodiment of the present application. As shown in fig. 8, the apparatus includes:

a second obtaining module 802 configured to obtain written language text;

a second retranslation module 804 configured to obtain a retranslated written language text corresponding to the written language text by performing retranslation processing on the written language text;

a second conversion module 806, configured to perform conversion processing of sentence composing units on the written language text and the retraced written language text respectively, to obtain a spoken language text.

On the basis of obtaining the spoken text corresponding to the written language text and retranslating the spoken text corresponding to the written language text, the sample corpus can be constructed according to the corresponding relationship between the written language text and the spoken text and the corresponding relationship between the retranslated written language text and the spoken text. And further training a written language rewriting model for performing written language rewriting on the target spoken language text through the sample corpus.

Optionally, the apparatus further comprises:

an acquisition module configured to acquire a target spoken language text;

the classification module is configured to classify the target spoken language text to obtain a text type corresponding to the target spoken language text;

the selection module is configured to select a corresponding written language rewriting model according to the standard text type under the condition that the text type is the standard text type;

the processing module is configured to input the target spoken language text into the written language rewriting model for processing to obtain a target written language text corresponding to the target spoken language text; the written language rewriting model is obtained by training a spoken language text obtained by retracing and converting the written language text based on the written language text.

Optionally, the apparatus further comprises:

the selection model module is configured to select a corresponding written language conversion model according to the fuzzy text type under the condition that the text type is the fuzzy text type;

inputting the target spoken language text into the written language conversion model for processing to obtain a converted written language text corresponding to the target spoken language text; the written language conversion model is obtained by training a basic spoken language text obtained by converting the written language text based on the written language text.

Optionally, the training of the written language conversion model is implemented by operating the following modules:

a first acquisition module configured to acquire written language text;

the first conversion module is configured to perform conversion processing of sentence composition units on the written language text to obtain a basic spoken language text;

the first construction module is configured to construct a basic sample corpus based on the corresponding relation between the written language text and the basic spoken language text;

a first training module configured to train an initial written-to-speech conversion model through the base sample corpus until the written-to-speech conversion model satisfying a first training stop condition is obtained.

Optionally, the classification module is further configured to:

inputting the target spoken language text into a text classification model for classification processing to obtain a text type corresponding to the target spoken language text; the training of the text classification model is realized by operating the following modules:

the system comprises an acquisition sample module, a semantic definition tag and a semantic definition tag, wherein the acquisition sample module is configured to acquire a sample spoken language text and the semantic definition tag corresponding to the sample spoken language text;

a construct sample pair module configured to construct a training sample pair based on the sample spoken text and the semantic clarity label;

and the model training module is configured to perform model training on the initial text classification model through the training sample pair until the text classification model meeting the classification training stopping condition is obtained.

Optionally, the processing module is further configured to:

Optionally, the written language rewrite model is trained by operating the following modules:

a second acquisition module configured to acquire written language text;

the retranslation module is configured to obtain a retranslated written language text corresponding to the written language text by performing retranslation processing on the written language text;

the second conversion module is configured to respectively perform conversion processing of sentence forming units on the written language text and the retraced written language text to obtain a spoken language text;

a second construction module configured to construct a sample corpus based on correspondence between the written language text and the translated back written language text and the spoken language text;

and the second training module is configured to train the initial written language rewriting model through the sample corpus until a written language rewriting model meeting a second training stop condition is obtained.

Optionally, the translation module includes:

the translation sub-module is configured to translate the written translated language text into a target language to which the written language text belongs, and obtain an initial translation written language text;

Optionally, the translation module further includes:

a marking submodule configured to position mark positions of the key words in the written language text;

accordingly, the replacement sub-module is further configured to:

Optionally, in a case that the sentence component unit is a clause unit, the second conversion module includes:

the first identification submodule is configured to perform sentence identification on the written language text to be processed to obtain written language sentences contained in the written language text to be processed;

Optionally, the clause conversion sub-module is further configured to:

Determining clause position probability distribution corresponding to preset clauses contained in a preset clause set; determining a target preset clause and a clause adding position corresponding to the target preset clause in the preset clauses based on the clause position probability distribution; and adding the target preset clause into the written language sentence according to the clause adding position to obtain the converted written language sentence.

Optionally, the clause conversion submodule is further configured to:

copying the target clause to obtain a copied target clause, and inserting the copied target clause into the written sentence according to a preset clause insertion position to obtain a converted written sentence; and/or the presence of a gas in the gas,

Optionally, in a case that the sentence component unit is a word unit, the second conversion module includes:

Optionally, in a case that the sentence component unit is a word unit, the second conversion module is further configured to:

determining spoken text based on the converted written language sentence.

Optionally, the second conversion module is further configured to:

a third determination module configured to determine a spoken text based on the inserted written language sentence.

Optionally, in a case that the sentence component unit is a character unit, the second conversion module is further configured to:

determining spoken text based on the converted written language sentence.

Optionally, in a case that the sentence component unit is a symbol unit, the second conversion module is further configured to:

Optionally, the second conversion module is further configured to:

Optionally, the apparatus further comprises:

Optionally, the second conversion module is further configured to:

performing sentence component unit conversion processing on the written language text to obtain a first spoken language text corresponding to the written language text;

and taking the first spoken text and the second spoken text as the spoken text.

Optionally, the apparatus further comprises:

a deletion module configured to delete the target spoken text if the text type is an invalid text type.

In conclusion, the written language text is obtained and the retranslated written language text corresponding to the written language text is obtained by obtaining the written language text and performing retranslation processing on the written language text, so that the written language material is expanded in a retranslation mode. And then the written language text and the retraced written language text are respectively subjected to sentence component unit conversion processing to obtain the spoken language text, so that the conversion processing based on the retraced and expanded written language corpus is realized, the spoken language text is expanded, the conversion accuracy of the written language text to the spoken language text is also improved, downstream related processing tasks are further performed based on the expanded written language corpus and/or the expanded spoken language text, a text basis is provided for the task processing, and the processing efficiency of the task processing is improved.

The above is a schematic scheme of a spoken text generation apparatus according to this embodiment. It should be noted that the technical solution of the spoken language text generating device and the technical solution of the spoken language text generating method belong to the same concept, and details of the technical solution of the spoken language text generating device, which are not described in detail, can be referred to the description of the technical solution of the spoken language text generating method.

It should be noted that the components in the device claims should be understood as functional blocks which are necessary to implement the steps of the program flow or the steps of the method, and each functional block is not actually defined by functional division or separation. The device claims defined by such a set of functional modules are to be understood as a functional module framework for implementing the solution mainly by means of a computer program as described in the specification, and not as a physical device for implementing the solution mainly by means of hardware.

An embodiment of the present application further provides a computing device, which includes a memory, a processor, and computer instructions stored on the memory and executable on the processor, where the processor implements the steps of the text processing method or the spoken text generation method when executing the computer instructions.

The above is an illustrative scheme of a computing device of the present embodiment. It should be noted that the technical solution of the computing device and the technical solution of the text processing method or the spoken text generation method belong to the same concept, and details that are not described in detail in the technical solution of the computing device can be referred to the description of the technical solution of the text processing method or the spoken text generation method.

An embodiment of the present application further provides a computer readable storage medium storing computer instructions, which when executed by a processor, implement the steps of the text processing method or the spoken text generation method as described above.

The above is an illustrative scheme of a computer-readable storage medium of the present embodiment. It should be noted that the technical solution of the storage medium belongs to the same concept as the technical solution of the text processing method or the spoken text generating method, and details that are not described in detail in the technical solution of the storage medium can be referred to the description of the technical solution of the text processing method or the spoken text generating method.

The embodiment of the application discloses a chip, which stores computer instructions, and the computer instructions are executed by a processor to realize the steps of the text processing method or the spoken text generation method.

The above is a schematic scheme of a chip of this embodiment. It should be noted that the technical scheme of the chip and the technical scheme of the text processing method or the spoken text generation method belong to the same concept, and details that are not described in detail in the technical scheme of the chip can be referred to the description of the technical scheme of the text processing method or the spoken text generation method.

The foregoing description of specific embodiments of the present application has been presented. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

The computer instructions comprise computer program code which may be in the form of source code, object code, an executable file or some intermediate form, or the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, read-Only Memory (ROM), random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.

It should be noted that, for the sake of simplicity, the above-mentioned method embodiments are described as a series of acts or combinations, but those skilled in the art should understand that the present application is not limited by the described order of acts, as some steps may be performed in other orders or simultaneously according to the present application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

The preferred embodiments of the present application disclosed above are intended only to aid in the explanation of the application. Alternative embodiments are not exhaustive and do not limit the invention to the precise embodiments described. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the application and its practical applications, to thereby enable others skilled in the art to best understand and utilize the application. The application is limited only by the claims and their full scope and equivalents.

Claims

1. A method of text processing, comprising:

constructing sample linguistic data based on the corresponding relation between the written language text and the retranslated written language text and the spoken language text respectively;

2. The text processing method of claim 1, wherein the plurality of conversion processing strategies includes at least one of: clause conversion processing strategy, word conversion processing strategy, character conversion processing strategy and symbol conversion processing strategy;

the sentence component unit includes at least one of: clause unit, word unit, character unit and symbol unit.

3. The text processing method according to claim 2, wherein in a case where the clause conversion processing policy is executed, the performing of the conversion processing of the sentence component unit on the written text to be processed includes:

converting the written sentence into a clause unit to obtain a converted written sentence;

determining a spoken text based on the converted written language sentence.

4. The method according to claim 3, wherein the converting the written sentence into a clause unit to obtain a converted written sentence, comprises:

5. The text processing method of claim 4, wherein the converting the target clause in the written sentence to obtain a converted written sentence comprises:

6. The text processing method according to claim 2, wherein the performing of the conversion processing of the sentence component unit on the written language text to be processed in the case of executing the word conversion processing policy comprises:

determining spoken text based on the converted written language sentence.

7. The text processing method according to claim 2, wherein the performing of the conversion processing of the sentence component unit on the written language text to be processed in the case of executing the word conversion processing policy comprises:

determining spoken text based on the converted written language sentence.

8. The method of claim 7, wherein the inserting the target preset word into the written sentence according to the word adding position further comprises, after obtaining the converted written sentence:

inserting the copied words into the converted written language sentences according to preset word insertion rules to obtain the inserted written language sentences;

determining spoken text based on the inserted written language sentence.

9. The text processing method according to claim 2, wherein in a case where the character conversion processing policy is executed, the performing of the conversion processing of the sentence component unit on the written language text to be processed includes:

determining spoken text based on the converted written language sentence.

10. The text processing method according to claim 2, wherein the performing of the sentence component unit conversion processing on the written language text to be processed in the case of executing the symbol conversion processing policy comprises:

11. The method according to any one of claims 1 to 10, wherein said performing a translation process on the written language text to obtain a translated back written language text corresponding to the written language text comprises:

translating the translated written language text back into the target language to which the written language text belongs to obtain an initial translated written language text;

12. The method of claim 11, wherein before translating the written language text into a translated written language text corresponding to a predetermined language, the method further comprises:

position marking the position of the key words in the written language text;

replacing target key words corresponding to the key words in the initial retranslate written language text by the key words in the written language text to obtain a retranslate written language text, wherein the method comprises the following steps:

13. The method according to claim 1, wherein said converting the written language text to be processed into a sentence component unit to obtain a spoken language text further comprises:

identifying abnormal information in the spoken text;

carrying out data cleaning on the spoken language text according to the abnormal information to obtain the cleaned spoken language text;

and constructing a sample corpus based on the corresponding relation between the written language text and the retranslated written language text and the cleaned spoken language text.