TWI635483B - Method and system for generating prosody by using linguistic features inspired by punctuation - Google Patents

Abstract

The present invention discloses a method and system for generating a linguistic feature inspired by punctuation and applied to a national prosody generation, including a main punctuation prediction unit and a quotation phrase prediction unit, and the main punctuation and prosody breakpoint correlation are related to It is very important to construct the rhythm structure of synthetic speech. On the other hand, the special rhythm changes in the phonetic expression of the quotation marks play a key role in the semantic understanding of the synthesized speech. Therefore, the two messages can improve the text-to-speech The prosody in the system generates the performance of the handler and further enhances the synthesized speech quality. The invention utilizes the conditional random field technique to predict the main punctuation marks and quoted phrases for the input characters and generate corresponding confidence prediction values, and finally integrates the two confidence prediction values and the basic language features as the prosody generation processing. Input features to predict four prosody parameters including volume, length, pitch, and pause length.

Description

Method and system for generating prosody by using linguistic features inspired by punctuation

The invention belongs to a method and system for generating linguistic features inspired by punctuation marks and applied to national language prosody, in particular to using a conditional random field technique to predict and generate correspondence between main punctuation marks and quotation marks for input text. The confidence prediction value finally integrates the two confidence prediction values and the basic language features as input features of the prosody generation processing.

Punctuation marks are roughly divided into labels and dot numbers. Their roles in text-to-speech (TTS) systems can be as follows: one is the grammar structure or the prosody structure in the expression sentence, and the other is The quotation marks are marked with special meaning or emphasized tone. Observing the writing habits of ordinary people, although there are punctuation marks between the lines of most of the texts, there are still two problems. In fact, there are still many problems with insufficient punctuation, and the other is inconsistent, not only different people. There is a different perception of the punctuation given, even the same person is difficult to maintain consistency and occasional errors occur.

In the Mandarin-to-speech system, the function of prosody generation processing converts language features into prosodic features, while the performance of prosody generation processing is related to two factors, one is the ability to predict the prosody model, and the other is the language used. Features, related language features can be classified into the following six categories according to the hierarchy: (1) pronunciation information, (2) text information, (3) word segmentation information, (4) word class information, (5) basic phrase information, (6) Grammatical structure information. The above four language features are the basic language features that most Mandarin-to-speech systems use. The first type of pronunciation information and the second type of text information can be said to be the basic elements required for prosody generation processing. The three types of word-breaking information and the fourth-type word-class information feature are the products of the word-breaking and word-class labeling processing for the input text. The fifth-class and sixth-class features are higher-order language features, and the two types of features are required. The profiling model must rely on experts with linguistic knowledge to pre-mark a large amount of text data. Because this labeling process is extremely time-consuming and there are consistency issues among different labelers, the training data is not easy to obtain, which in turn affects the performance of the profiling model.

In addition, the traditional TTS system does not make good use of the information provided by the quotation marks. Because the phrases or short sentences in the quotation marks are mostly important semantic meanings, they have a certain degree of influence on the comprehensibility of synthesized speech, and it is worth further exploration to enrich the rhythm. Predict the language features required.

In view of the shortcomings derived from the above-mentioned conventional methods, the inventor of the present invention has improved and innovated, and after years of painstaking research, he finally succeeded in researching and developing the language features inspired by punctuation and applying it to the generation of Mandarin rhythm. Method and system.

The purpose of the present invention is to propose a machine-extracted linguistic feature for use in a Mandarin-to-speech-to-speech system, one of which is a linguistic feature as the main punctuation confidence, which is used to measure the possibility of inserting a main punctuation between a word and a word. The second is that the linguistic feature is the confidence of the quoted phrase. The function is to measure the quotation marks before and after a certain string, so that the string becomes a special meaning or The possibility of a unit with an emphasis on tone.

In order to achieve the above object, the present invention proposes a statistical feature called main punctuation confidence, which focuses on replacing the basic piece information and the grammatical structure information feature with different degrees of pause prediction confidence value. The main punctuation marks defined by the present invention include a period (.), an exclamation point (!), a question mark (?), a semicolon (;), a colon (:), a comma (,), and generally, the higher the pause prediction confidence value The representative is the main pause breakpoint, most of which is the end of the phrase or short sentence and the pause time is longer; the lower pause prediction confidence value represents the secondary pause breakpoint, mostly the word boundary and the pause time is short. The main punctuation confidence value is generated from a main punctuation generation model based on the conditional random field (CRF) technique, and a large number of texts containing various major punctuation marks are required to establish the model. There is no need for manual labeling. Applying this model will provide a predictive value for the probability of occurrence of major punctuation between words and words. A higher value indicates that the position is more likely to insert a pause.

Using a large amount of textual data containing punctuation marks, machine learning technology is used to establish consistent punctuation prediction ability for text analysis processing in text-to-speech system and output corresponding confidence value to provide subsequent prosody prediction processing. Rich information.

The main punctuation generation model based on CRF technology deals with a mark labeling problem. The establishment procedure of this model is to first define a set of marks. The words of the input text need to be marked with this mark set, and the best mark In the process of sequence determination, different fields composed of different combinations of input features are considered, and Viterbi search technology is used to search for the best path, and then the prediction of each mark is related to the main punctuation. value.

The role of punctuation in the text-to-speech system, except In addition to the pause, there are also marking functions, especially the text strings with special meaning and emphasis. Through the large number of transcripts, the types of quotation marks and the function of quotation marks have the following findings: First, the punctuation marks associated with quotation marks can be generally classified into 10 categories, namely "()", "{}", "[]", "" "","""","<>","【】","""",""""versus"""".

The first type of symbol "()", which is mostly used for sorting, is not the target symbol to be processed by the present invention.

The second type of symbol "{}" is used to indicate the title of the book or the name of the article, which is the target symbol to be processed by the present invention.

The third type of symbol "[]" is used to indicate the annotation or supplementary description of the article, which is not the target symbol to be processed by the present invention.

The fourth type of symbol """ and the fifth type of symbols """ and """ are arranged according to the number of words contained in the quoted phrase, followed by short sentences, proper nouns, idioms and buzzwords. These two categories are the main sampling sources for the target symbols to be processed in the present invention, and the probability of occurrence is the highest.

The sixth to tenth symbols, "<>", "[]", """, """ and "〝〞" are based on proper nouns, idioms and buzzwords. Appears in short sentences, but is the target symbol to be processed by the present invention.

Because the phrases or short sentences in the quotes are mostly important semantics, the corresponding voices will show the following obvious changes: the quoted phrase boundaries have obvious pauses and pitch resets, and the quoted phrases are mostly non-stop. And there is a volume increase phenomenon. In order to enable these changes to be effectively learned by the prosody prediction model of the text-to-speech system, the present invention hopes to establish a quotation phrase confidence prediction model, and provides predictive values for the possibility of quotation marks between words and words to assist the prosody generation process. The invention also uses the CRF technology to establish the confidence prediction model of the quoted phrase, and the required training materials are a large number of cited Number of texts.

The above two confidence prediction values are then subjected to prosody generation processing in combination with the basic language features to predict prosody parameters including volume, pitch, length and pause length.

A system inspired by punctuation and applied to the system of Chinese prosody generation, which includes a word segmentation unit and a word class indicator unit, which is used to process the input text, and to break the word content, and to perform word class tagging and tagging words. Constituting components to provide information needed for subsequent processing units; the main punctuation prediction unit is to input the input information provided by the word segmentation unit and the word class labeling unit simultaneously with the quoted phrase prediction unit to generate confidence of the main punctuation symbol; The prediction unit is configured to process the input information provided by the word segmentation unit and the word class labeling unit simultaneously with the main punctuation prediction unit to generate the confidence of the quoted phrase phrase; the context analysis unit is the input provided by the receiving processing word segment and the word class labeling unit. Information to generate basic language features; integrated feature vector unit, which integrates main punctuation confidence, quoted phrase confidence, and basic language features; prosody generation processing unit is based on multi-layer perceptron (MLP) Receiving the integrated information of the integrated feature vector unit, Generating a syllable of the prosodic units, while the text before and after analysis and prosody generating unit to establish a processing unit, the voice data must be used to adjust the parameters of.

The input information of the main punctuation prediction unit is a word (WORD) and a part of speech (POS), and the input information of the quoted phrase prediction unit and the input information of the before and after analysis units are all words, word classes and punctuation. Punctuation mark (PM).

Establishing speech data used by the context analysis unit and the prosody generation processing unit, including training sets and development sets (development set), and testing set, wherein the training set is a training MLP related parameter, including the connection weight between different layer nodes; the development set is determined before and after The optimal number of adjacent words to be set by the text analysis unit, and the number of nodes included in the hidden layer of the prosody message generation processing unit; and the testing set is to evaluate the prosody information generation processing unit in pitch, length, Estimated performance of four prosody parameters for volume and pause length.

A method and system for generating a linguistic feature inspired by punctuation and applied to a prosody generation of a national language, comprising: step one: performing word class tagging after the input text content is broken; and step 2, simultaneously establishing main punctuation prediction The model and the quotation phrase prediction model are established; in step 3, the post-analysis module is established before considering the adjacent words to generate the basic language features; and the fourth step is to establish the prosody message generation processing module based on the MLP.

The above steps 3 and 4 must use the voice data to adjust the relevant parameters.

The first step is to generate information about the components of the word break, the word class mark and the word.

The punctuation prediction model of step 2 uses the 11-marking method or the 2-marking method, and uses the input features including words, word lengths and word classes, and uses the CRF++ model to establish a corresponding probability field according to different combinations of input characteristics. This probability field is used to obtain the optimal marker sequence, and the CRF is combined with the finite state machine rule method to perform the main punctuation estimation and generate the corresponding confidence prediction value. The length of the word is the number of syllables contained in the word.

The description of each of the 11 marking methods is as follows: B1 (first , B2 (second), B3 (third), B4 (fourth), I (intermediate term when the number of words is less than nine and odd), M (the middle of the number of words is greater than or equal to nine o'clock) Term), E4 (the fourth to last), E3 (the third to the last), E2 (the second to last), E1 (the first to the last), S (single); can also use the 2 mark method, in After each term in the sentence, two tags are used to indicate whether the main punctuation exists.

The quotation phrase prediction model of step 2 is that the mark prediction related to the quotation mark group can be performed by using the 8 mark method or the 19 mark method, and the input features used include words, word length, word class and other punctuation marks except quotation marks. The CRF++ model is used to obtain the optimal marker sequence. At the same time, the CRF method is combined with the finite state machine rule method to predict the quoted phrase and generate the corresponding confidence prediction value.

The descriptions of the above-mentioned eight-marking method are as follows: B (first), B2 (second), B3 (third), I (intermediate term with three words), M (number of words) Greater than three o'clock intermediate term), E (last), S (single one), O (words other than quoted phrase).

The descriptions of the above-mentioned 19-marking methods are as follows: the first 7 marks including the 8-marking method describe the structure in the quoted phrase and the newly added set of 12 marks describing the structure of the non-quoted phrase in the sentence, the latter adopted The format is Xy, and the set of X belongs to {P, M, F}, which in turn represents the quotation marks before, after the quotation marks and after the quotation marks, and the y belongs to the set {b, m, e, s}, The order represents the starting word, the middle word, the ending word and the single word.

The voice data used in the third step and the fourth step includes a training set, a development set and a testing set, wherein the training set is a training related MLP related parameter, including different The connection weight between the nodes; the development set is the optimal number of adjacent words required to determine the context analysis unit, and the prosody message generation. The number of nodes included in the hidden layer of the processing unit; and the test set is an estimated performance of the prosody signal generation processing unit in terms of pitch, length, volume, and pause length.

In the above steps, steps 1 and 2 only need to use text data, and steps 3 and 4 must use voice data again.

In summary, the input text is first processed by word segmentation and word class labeling, and the output information is simultaneously input into three processings: CRF-based main punctuation prediction unit, CRP-based quotation phrase prediction unit and context analysis unit. The role of the analysis unit is to generate basic language features by considering adjacent words in the context, that is, including pronunciation information, text information, word segmentation information, and word class information. The main punctuation prediction unit and the quoted phrase prediction unit are respectively Generates confidence in the main punctuation and confidence in the quoted phrase. The integrated feature vector generated by integrating the above basic language features, main punctuation confidence and quoted phrase confidence is input into the prosody generation processing unit, which contains 4 sets of independent neural networks, each group The architecture of a neural network is a three-layer architecture based on MLP, including an input layer, a hidden layer and an output layer. The input features contained in the input layer are the integrated feature vectors described above, and the output layer is in syllables. The relevant prosodic parameters of the unit are described as follows: the output of the first set of neural networks is the fourth-order orthogonal transform coefficient describing the pitch trajectory, the output of the second set of neural networks is the sound length, and the third set of neural networks The output of the network is the volume, and the output of the fourth group of neural networks is the pause length between the syllables.

The invention provides a method and a system for using the linguistic features inspired by punctuation marks and applied to the prosody generation of the national language, and has the following advantages when compared with other conventional techniques:

1. The first type of object processed by the present invention is a main punctuation mark, such a class The symbol is related to the pause breakpoint that expresses the prosody structure. Different from the traditional TTS system, the language feature is generally only used, or the syntax parser is needed to generate high-order language features. Punctuation marks represent pause breakpoints to replace higher-order language features.

2. The second type of object processed by the present invention is a quotation mark. The traditional TTS system does not make good use of the information provided by such punctuation to assist in prosody prediction and speech synthesis processing. The present invention hopes to further enhance the intelligibility of synthesized speech by using this information to enrich the input language features predicted by prosody.

3. The present invention uses a conditional random field (CRF) technique to establish a prediction system for the above two types of punctuation, for a long sentence without a main punctuation or a sentence lacking a quotation mark, by machine learning techniques. That is, the possibility of these two types of punctuation marks can be predicted.

110‧‧‧Words and word class

120‧‧‧main punctuation prediction unit

130‧‧‧ quoted phrase prediction unit

140‧‧‧ before and after analysis unit

150‧‧‧Integrated feature vector unit

160‧‧‧prosody generation processing unit

S210~S240‧‧‧ Process

The detailed description of the present invention and the accompanying drawings will be further understood, and the technical contents of the present invention and the functions thereof can be further understood. The related drawings are: FIG. 1 is a language feature inspired by punctuation marks and applied to Mandarin. Schematic diagram of a method and system for prosody generation; FIG. 2 is a flow chart of a method and system for generating prosody in Chinese by the punctuation-inspired linguistic features of the present invention; FIG. 3 is inspired by punctuation in the present invention. The linguistic features are applied to the method of generating the prosody of the national language and the schematic diagram of the multi-layered cognition processing of the rhythm generation processing of the system.

The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

Hereinafter, the present invention will be further described with reference to the accompanying drawings:

Please refer to FIG. 1 , which is an architectural diagram of a method and system for generating a prosody generated by a punctuation mark in accordance with the present invention, which includes a word segmentation and word class indicator unit 110 for processing input text. And the word content is broken, and the constituent components of the word class mark and the mark word are provided to provide information required by the subsequent processing unit; the main punctuation prediction unit 120 performs the process of breaking the word and the word class at the same time with the quoted phrase prediction unit 130. The input information provided by the unit to generate the main punctuation confidence; the quotation phrase prediction unit 130 is configured to process the input information provided by the word segmentation unit and the word class indicator unit simultaneously with the main punctuation prediction unit 120 to generate the quotation phrase confidence. The contextual analysis unit 140 receives the input information provided by the processing word segmentation and the word class labeling unit to generate a basic language feature; the integrated feature vector unit 150 integrates the main punctuation confidence, the quotation phrase confidence, and the basic language. Feature; prosody generation processing unit 160 is a receiver integrated feature vector unit integration The latter information, and produces prosodic parameters in syllables. The establishment of the context analysis unit and the prosody generation processing unit must use the voice data to adjust the relevant parameters.

The input information of the main punctuation prediction unit 120 is a word and a word class, and the input information of the quoted phrase prediction unit 130 and the input information of the context analysis unit 140 are words, word classes and punctuation marks.

Established before and after the text analysis unit and prosody generation processing unit The voice data includes a training set, a development set, and a testing set, wherein the training set is a training MLP related parameter, including a connection between different layer nodes. The set of developments is the number of optimal neighbors required to determine the context analysis unit, and the number of nodes in the hidden layer of the prosody message generation processing unit; and the testing set. It is the estimated performance of the prosody signal generation processing unit in the four prosody parameters of pitch, length, volume and pause length.

Please refer to FIG. 2, which is a flow chart of a method and system for generating a prosody of a national language by the punctuation-inspired language feature of the present invention, which includes: Step 1: After S210 breaks the input text content, Word class tag; Step 2: Simultaneously establish S221 to establish the main punctuation prediction model and S222 to establish the quotation phrase prediction model; Step 3, S230 establishes the context analysis module before considering the adjacent words to generate the basic language features; and Step 4 The S240 establishes a prosody message generation processing module based on the MLP.

Among them, step three S230 and step four S240 must use the voice data to adjust related parameters.

Step 1 of S210 is to generate related information about the components of the word break, the word class mark and the word.

The main punctuation prediction model of step 2 of S221 is to use the 11 mark method or the 2 mark method, and the input features used include words, word lengths and word classes, and use the CRF++ model to establish a corresponding probability according to different combinations of input features. Field, taking the probability field to find the best mark sequence, and using CRF The main punctuation prediction is carried out in combination with the finite state machine rule method and the corresponding confidence prediction value is generated. The length of the word is the number of syllables contained in the word.

The description of each mark of the 11 mark method is as follows: B1 (first), B2 (second), B3 (third), B4 (fourth), I (when the number of words is less than nine and is odd) Intermediate term), M (intermediate term with words greater than or equal to nine o'clock), E4 (fourth from the last), E3 (third from the last), E2 (second from the penultimate), E1 (first from the last) , S (single one); can also use the 2 mark method, after each word in the sentence, use two marks to indicate whether there is a main punctuation mark.

The quotation phrase prediction model of step S222 of S222 is that the mark prediction related to the quotation mark group can be performed by using the 8 mark method or the 19 mark method, and the input features used include words, word length, word class and other punctuation except quotation marks. Symbols, and use CRF++ model to obtain the best mark sequence, and use CRF method combined with finite state machine rule method to make quoted phrase prediction and produce corresponding confidence prediction value.

The descriptions of the above-mentioned eight-marking method are as follows: B (first), B2 (second), B3 (third), I (intermediate term with three words), M (number of words) Greater than three o'clock intermediate term), E (last), S (single one), O (words other than quoted phrase).

The descriptions of the above-mentioned 19-marking methods are as follows: the first 7 marks including the 8-marking method describe the structure in the quoted phrase and the newly added set of 12 marks describing the structure of the non-quoted phrase in the sentence, the latter adopted The format is Xy, and the set of X belongs to {P, M, F}, which in turn represents the quotation marks before, after the quotation marks and after the quotation marks, and the y belongs to the set {b, m, e, s}, The order represents the starting word, the middle word, the ending word and the single word.

The voice data used in step S230 of S230 and step 4 of S240 includes a training set and a development set. And the test set, wherein the training set is a training MLP related parameter, including the connection weight between different layer nodes; the development set is the best adjacent word determined by the decision analysis unit. The number, and the number of nodes included in the hidden layer of the prosody message generation processing unit; and the test set are estimates of the prognostic performance of the prosody information generation processing unit in terms of pitch, length, volume, and pause length.

In the above steps, steps 1 and 2 only need to use text data, and steps 3 and 4 must use voice data again.

In summary, the input text is first processed by word segmentation and word class labeling, and the output information is simultaneously input into three processings: CRF-based main punctuation prediction unit, CRP-based quotation phrase prediction unit and context analysis unit. The role of the analysis unit is to generate basic language features by considering adjacent words in the context, that is, including pronunciation information, text information, word segmentation information, and word class information. The main punctuation prediction unit and the quoted phrase prediction unit are respectively Generates confidence in the main punctuation and confidence in the quoted phrase. The integrated feature vector is to integrate the above basic language features, the main punctuation confidence and the confidence of the quoted phrase, and then input to the prosody generation processing unit. The prosody generation processing unit contains 4 sets of independent neural networks, each group. The architecture of the neural network is a three-layer architecture based on MLP. Please refer to Figure 3, including the input layer, hidden layer and output layer. The input features contained in the input layer are the integrated feature vectors mentioned above. The output layer is the relevant prosody parameters in syllables and is described as follows: the output of the first set of neural networks is the fourth-order orthogonal transform coefficient describing the pitch trajectory, and the output of the second set of neural networks is the sound. Long, the output of the third group of neural networks is the volume, and the output of the fourth group of neural networks is the pause length between the syllables.

The linguistic features inspired by punctuation and applied to the rhyme of the national language The method and system for generating the law are divided into four parts. The first part belongs to the traditional text analysis processing, including the word segmentation and word class labeling processing. The second part is the main punctuation prediction unit based on CRF technology, the third part is the CRF technology based quotation phrase prediction unit, and the fourth part includes the consideration of the adjacent words to generate the basic language. The feature analysis unit before and after, and the prosody message generation processing unit established by the MLP.

The implementation is as follows:

First, the first part treats the input text through the word segmentation and the word class labeling unit. The word segmentation and word class indicator unit breaks the word content and then performs the component of the word class tag and the tag word to provide the information needed for the subsequent processing module.

Furthermore, the second part of the main punctuation prediction unit can use the simplest design method, that is, the 2 mark method. After each word in the sentence, the two marks are used to indicate whether there is a main punctuation mark, and it is found through experiments. The practice only helps to segment the long sentences. If you want the model to have better primary punctuation prediction ability, you must use a more complex set of tags that can represent the overall structure of the sentence.

In view of this, the present invention uses the 11-marking method, and the description of each mark is as follows: B1 (first), B2 (second), B3 (third), B4 (fourth), I (less than the number of words) Nine (the middle term in the odd number), M (the middle term in the number of words greater than or equal to nine), E4 (the fourth in the last), E3 (the third in the last), E2 (the second to last), E1 (the last one), S (single), the following is a further description of the way the sentence is marked:

If the length is one unit, it is marked as: S.

If the length is two units, it is marked as: B1, E1.

If the length is three units, it is marked as: B1, I, E1.

If the length is four units, it is marked as: B1, B2, E2, E1.

If the length is five units, it is marked as: B1, B2, I, E2, E1.

If the length is six units, it is marked as: B1, B2, B3, E3, E2, E1.

If the length is seven units, it is marked as: B1, B2, B3, I, E3, E2, E1.

If the length is eight units, it is marked as: B1, B2, B3, B4, E4, E3, E2, E1.

If the length is more than nine units, it is marked as: B1, B2, B3, B4, M...M, E4, E3, E2, E1.

Please refer to Table 1 below for the training examples and marking materials of the main punctuation prediction unit, where (a) is the text of the training example sentence, and (b) is the labeling data obtained by using the 2 marking method, where 2 marks are y ₀ and y ₁ , (c) is the labeling data obtained by using the 11-marking method. The input features used to establish the main punctuation prediction unit include the word, the word length (the number of syllables contained in the word) and the word class. The CRF++ model is used to establish the corresponding probability field according to different combinations of input features, thereby taking the probability field to obtain the most Good marker sequence.

Then, the third part of the quotation phrase prediction unit can take two In the first method, the labeling method is {B, B2, B3, M, I, E, S, O}. This method focuses on the structure prediction in the quoted phrase, and the words other than the quoted phrase are Given the mark O, the role of the remaining 7 marks is as follows: B (first), B2 (second), B3 (third), I (intermediate term with three words), M (The number of words is greater than three in the middle term), E (last), S (single), the following is a further description of the way the phrase is marked in quotes:

If the length is one unit, it is marked as: S.

If the length is two units, it is marked as: B, E.

If the length is three units, it is marked as: B, I, E.

If the length is four units, it is marked as: B, B2, M, E.

If the length is five units, it is marked as: B, B2, M, M, E.

If the length is more than six units, it is marked as: B, B2, B3, M...M, E.

The second method is the 19-marking method, which is different from the first method, which predicts both the structure within the quoted phrase and the structure of the non-quoted phrase in the sentence. This practice can be considered as an improved version of the first method, using additional marks to represent the positional information of the words of the non-quoted phrase in the sentence, that is, the mark used includes the first seven of the first practice. Tag and new non-quoted phrase tag set, the format of the non-quoted phrase tag set is Xy, X belongs to the set {P, M, F}, respectively, before the quoted phrase, between the quoted phrase and the quoted phrase After that, the set of y belongs to {b, m, e, s}, which respectively represent the starting word, the middle word, the ending word and the single word.

Please refer to Table 2 below for the training example sentences and marking materials of the quoted phrase prediction unit, where (a) is the text of the training example sentence, (b) is the labeling data obtained by using the 8 marking method, and (c) is the 19 marking using the marking method. Marking information obtained by law. The input features used by the quoted phrase prediction unit include words, word lengths, and word classes. And other punctuation marks except quotation marks, and use CRF++ model to obtain the best mark sequence.

The final fourth part includes the analysis of the preceding and subsequent words to generate the basic language features before the analysis unit, the integration of the relevant language features, and the prosody message generation processing unit established by the MLP. Different from the establishment of the main punctuation prediction unit and the quotation phrase prediction unit is the use of text data, the fourth part of the analytic unit and the prosody message generation processing unit need to use the speech-containing data, the relevant corpus will be divided into training sets (training set), development set and testing set, wherein the role of the training set is to train the known MLP related parameters, see the connection weight w _ji between the different layer nodes shown in Figure 3. w _kj , the role of the development set is used to determine the optimal number of adjacent words required for the context analysis unit, and the number of nodes in the hidden layer of the prosody message generation processing unit. The test set is used to evaluate the rhythm. The estimated performance of the message generation processing unit in four prosody parameters such as pitch, length, volume and pause length. For the four independent neural networks used, please refer to the MLP architecture of Figure 3, the output of the first group of MLPs. 4 is described in order of the track pitch orthogonal transform coefficients, the output layer nodes K is 4, the second group to the fourth group of MLP output sequentially as sound length, volume and between syllables Dayton length, which are output layer nodes K 1.

In summary, in order to establish an implementation manner of the system related to the present invention, The content includes adding a main punctuation prediction function and a quotation phrase prediction function for text analysis processing, thereby adding two corresponding confidence prediction values as input features of the prosody message generation processing unit, and the above processing is added to the traditional Mandarin text-to-speech system. There are the following flexible implementations. The first method is to assist the prosody generation process to predict the relevant prosody parameters by the added confidence prediction value of the present invention, and the original punctuation marks of the input text are still retained. Another approach is to set an appropriate threshold to generate the main punctuation and quoted phrases by high confidence predictions and replace the relevant punctuation of the original input text. The related application of the latter method can be applied to the speech-to-text processing in addition to the speech-to-speech field discussed in the present invention, and the addition of appropriate punctuation marks to the voice file, and the auxiliary punctuation in the document processing system. Correction processing of symbols.

The detailed description of the present invention is intended to be illustrative of a preferred embodiment of the invention, and is not intended to limit the scope of the invention. The patent scope of this case.

To sum up, this case is not only innovative in terms of technical thinking, but also has many of the above-mentioned functions that are not in the traditional methods of the past. It has fully complied with the statutory invention patent requirements of novelty and progressiveness, and applied for it according to law. Approved this invention patent application, in order to invent invention, to the sense of virtue.

Claims (11)

A system inspired by punctuation and applied to the system of Chinese prosody generation, which includes: a word segmentation unit and a word class labeling unit, which is to process the input text, and to break the word content, and to perform word class tagging and tagging words. The composition component to provide the information needed for the subsequent processing unit; the main punctuation prediction unit simultaneously processes the input information provided by the word segmentation unit and the word class labeling unit to generate the main punctuation confidence. Wherein, the main punctuation confidence function acts to measure the possibility of inserting a main punctuation mark between the word and the word; the quoted phrase prediction unit is processed simultaneously with the main punctuation prediction unit by the word segment and the word class indicating unit. input information to generate confidence phrase in quotation marks, in which the phrase in quotation marks to measure the role of confidence there will be a word quotation marks around the string, so that the possibility of becoming a word string has a special meaning or tone with emphasis on the unit; The context analysis unit receives and processes the input provided by the word segmentation and the word class indicator unit. Information to generate basic language features; integrating feature vector units, integrating the main punctuation confidence, the confidence of the quoted phrase, and the basic language feature; the prosody generation processing unit receives the integrated information of the integrated feature vector unit And generating a prosody parameter in units of syllables; the establishment of the context analysis unit and the prosody generation processing unit, the speech data must be used to adjust the relevant parameters. For example, the linguistic features inspired by punctuation as described in claim 1 of the patent application are applied to a system for generating a prosody of a national language, wherein the input information of the main punctuation prediction unit is a word and a word class. The linguistic features inspired by punctuation as described in claim 1 of the patent application and applied to the system for generating a prosody of the national language, wherein the input information of the quotation phrase prediction unit is a word, a word class and a punctuation symbol. For example, the language features inspired by punctuation as described in the first paragraph of the patent application are applied to the system for generating the prosody of the national language, wherein the input information of the analysis unit is a word, a word class and a punctuation symbol. The linguistic feature inspired by punctuation as described in claim 1 of the patent application and applied to the system for generating a prosody of the national language, wherein the linguistic analysis unit and the phonological data used by the prosody generation processing unit are: The training set is a training MLP related parameter, including the connection weight between different layer nodes; the development set is the optimal number of adjacent words required to determine the context analysis unit. And the number of nodes included in the hidden layer of the prosody message generation processing unit; and the testing set is an estimated performance of the prosody signal generation processing unit in terms of pitch, length, volume, and pause length. . A method for generating a linguistic feature inspired by punctuation and applied to a prosody generation of a national language includes: step one: after the word content of the input text is broken, the word class is marked; and step 2, simultaneously establishing a main punctuation prediction model and Establish a quotation phrase prediction model; Step 3: Establish a analytic module before considering the adjacent words to generate the basic language features; and Step 4, establish a prosody message generation processing module based on the MLP; The punctuation prediction model uses the 11-marking method or the 2-labeling method, and uses the input features including words, word lengths, and word classes, and uses the CRF ⁺⁺ model to establish a corresponding probability field according to different combinations of input characteristics. The field obtains the optimal mark sequence, and uses CRF combined with the finite state machine rule method to predict the main punctuation and produces a corresponding confidence prediction value, wherein the word length is the number of syllables contained in the word; Quoted phrase prediction model, which is able to use the 8 mark method or the 19 mark method to perform mark prediction related to the quoted phrase, and will make The additional features include punctuation input, except the word, long word, parts of speech, and quotation marks, and obtaining the best model using CRF ⁺⁺ marker sequence, while the use of CRF binding method finite state machine rule method and generates a corresponding estimated quoted phrase of The confidence prediction value, wherein the word length is the number of syllables included in the word; and the third step and the fourth step must use the voice data to adjust related parameters. For example, the linguistic features inspired by punctuation and the method for generating the prosody of the national language are described in the sixth paragraph of the patent application, wherein the first step is to generate information about the components of the word break, the word class mark and the word. The method for applying the linguistic features inspired by the punctuation marks and the method for generating the prosody of the national language according to the sixth aspect of the patent application, wherein the speech data used in the third step and the fourth step includes: a training set (training set) The system is a training MLP related parameter, including the connection weight between different layer nodes; the development set is the optimal number of adjacent words required to determine the context analysis unit, and the prosody message generation processing The number of nodes in the hidden layer of the unit; and the testing set, which is the evaluation prosody message generation processing unit Estimated performance of four prosody parameters for pitch, length, volume, and pause length. The method of applying the linguistic features inspired by punctuation and applying to the pronodicity of the national language as described in claim 6 of the patent application, wherein the description of each of the 11 marks is as follows: B1 (first), B2 (first) Two), B3 (third), B4 (fourth), I (intermediate term when the number of words is less than nine and odd), M (intermediate term with the number of words greater than or equal to nine), E4 ( The fourth to the last), E3 (the third to the last), E2 (the second to last), E1 (the first one), S (single); can also use the 2 mark method, after the words in the sentence Use two markers to indicate whether there is a main punctuation mark. The method of applying the linguistic features inspired by punctuation as described in claim 6 of the patent application and applying the method for generating the prosody of the national language, wherein the description of each mark of the 8 mark method is as follows: B (first), B2 (first) Two), B3 (third), I (intermediate term with three words), M (intermediate term with more than three words), E (last), S (single), O (words other than quoted phrases). The method of applying the linguistic features inspired by punctuation as described in claim 6 and applying the method to the pronodicity of the national language, wherein the description of each of the 19 marks is as follows: including the first 7 marks of the 8 mark Describe the structure within the quoted phrase and the new set of 12 markers to describe the structure of the non-quoted phrase in the sentence. The format adopted by the latter is Xy, and the set of X belongs to {P, M, F}, which respectively represent Before the quoted phrase, after the quoted phrase and after the quoted phrase, and the set of y belongs to {b, m, e, s}, respectively, the start word, the middle word, the ending word and the single word.