KR20050021567A - Concatenative text-to-speech conversion - Google Patents

Abstract

The present invention provides a method of text-to-speech (S300) comprising dividing text into divided speech units (S303) and then identifying (S304) suitable sound units for each of the speech units. do. Each acoustic unit AU represents acoustic segments forming a speech cluster that is determined by the acoustic similarity of the acoustic segments. The method S300 then performs a step S305 of determining the prosodic parameters of the acoustic unit AU and the variations between the respective speech units. Generating acoustic parameters from the speech parameters of the acoustic unit and associated variations (S306) is then performed and then affects providing an output speech signal based on the acoustic parameters (S307). The present invention may provide improved synthetic sound quality, system performance, and a moderately reduced memory cost for portable devices.

Description

Concatenative text-to-speech conversion}

The present invention relates to a concatenated text-to-speech (TTS) transformation. The present invention is particularly useful for, but not necessarily limited to, cascading TTS synthesis with prosodic control.

Reading large amounts of text documents stored on computers, mobile phones, or personal data assistants (PDAs) may easily affect vision tiredness. And sometimes, it is inconvenient to read data on an electronic screen in a moving vehicle. Thus, to solve these problems, it is desirable to convert text documents into speech that is played for the listener.

Currently, almost all high quality text-to-speech TTS synthesis techniques are based on the speech waveform chain of each corresponding letter, word, or phrase. Preferred speech waveforms are usually derived from a set of speech waveforms, where the speech waveform set stores various sentences, phrases and their corresponding speech waveforms. Preferred quality of synthetic vocalization depends on the same size as the aggregate.

1 illustrates an existing typical chain TTS system. The system includes three parts: a text processor, an acoustic segment base, and a voice synthesizer. The system first breaks up sentences and words into word segments and then assigns the corresponding letters to phonetic symbols with the assistance of a Lexicon. Then, a series of divided speech symbols are matched with acoustic segments from the speech or phrase waveform set, thus obtaining the best matched acoustic segments. Eventually, the acoustic segments selected to obtain the output voice will be chained to the insertion of the appropriate brakes.

Such conventional TTS systems usually store voice waveforms directly. However, in order to obtain a voice effect very close to human speech, it will be required to store large amounts of speech waveforms in all kinds of speech environments, including speech characteristics of most situations. The storage of huge amounts of speech waveforms requires a lot of memory space. High quality text-to-speech systems typically require hundreds of megabytes of memory capacity. For portable devices such as mobile phones or PDAs, the memory capacity is usually only a few megabytes due to hardware and cost constraints. Thus, in those portable devices, it is difficult to have high quality text-voice. This limits the use of text-to-speech conversions in these technical fields.

1 illustrates a prior art text-to-speech system.

2 illustrates a text-to-speech system according to the present invention.

3 is a flowchart illustrating a text-to-speech method according to the present invention.

Summary of the Invention

The present invention provides a text-to-speech conversion method, the method comprising: dividing text into divided speech units and identifying a suitable sound unit for each of the speech units, wherein each sound unit Denoting the acoustic segments forming a speech cluster determined by acoustic similarity, determining the prosodic parameters of the acoustic unit and the variations between each of the speech units; Generating acoustic parameters from the speech parameters and associated variations of the acoustic unit to select a, and providing an output speech signal based on the acoustic segment.

Suitably, the speech parameters include pitch, duration or energy.

Preferably, the determining step is based on a phrase or sentence, co-articulation, phrase length or the position of the acoustic unit in adjacent characters of the acoustic unit.

The dividing step may be characterized by dividing the sentences of the text into syllables. Suitably, the speech units are syllables. The speech units may be assigned to speech symbols. Suitably, the negative symbol is a pinyin representation.

In another form, there is a text-to-speech system provided. The system includes a text processor that forms sentences of speech symbols after word segmentation based on input text. The text-to-speech conversion system further comprises an acoustic and speech controller comprising at least a speech annotation set and a sound unit index (AU index) and a speech vectors (PV) selection device. The speech annotation set includes at least acoustic unit (AU) indices and speech vectors PV. The acoustic unit index (AU index) and speech vector (PV) selection device receive successive speech symbols after word division, and receive a series of control data comprising acoustic unit (AU) indexes and speech vectors PV. Create The text-to-speech conversion system also includes a synthesizer that includes at least an acoustic parameter base, the synthesizer responsive to control data from the sound / rhyme controller, and thus synthesize speech.

The present invention also provides a method for converting text entries into corresponding synthesized speech via a cascaded text-to-speech system. The method for generating successive segmented speech symbols comprises processing and transforming text input and a speech annotation comprising at least acoustic unit (AU) indices to find the maximum match that fetches a matched annotation context. Searching for a set, replacing matched portions of the successive segmented speech symbols with AU indexes and rhyme vectors, generating continuous control data with at least AU indexes and rhyme vectors, and Generating a synthesized speech in response to the control data.

The invention also provides a method of forming a symbolic aggregate. The method comprises the steps of slicing speech into acoustic segments (AS), grouping the AS into clusters taking into account voice classification and acoustic similarity, and an acoustic unit (AS) representing all acoustic segments in a cluster. Selecting, converting the acoustic units into parameters of respective sequences per pixel, vector-quantizing the pixel parameters of each AU into consecutive vector indices, frame-based scalar parameters and vector Forming an AU parameter base comprising the indices, finding a matched AU for all ASs, determining respective rhyme vectors between the AUs and ASs, and forming a speech annotation set instead of the original AS waveform set Replacing sound segments with speech symbols, AU indices, and rhyme vectors. In this way, based on the actual human utterance for the set, the present invention groups the utterance or acoustic segments and replaces all acoustic segments only in a cluster and in the difference between the acoustic segments and the acoustic unit. Only the sound unit is stored, and parameters representing the original speech waveforms are used, thereby effectively reducing the amount of data stored in the speech annotation set.

According to the present invention, the speech symbols are used to replace any acoustic segments of each cluster, thus effectively reducing the number of required data in the memory and saving memory space. In addition, using such parameters in place of the acoustic unit waveform, the present invention converts each acoustic unit waveform into a series of parameters to form an acoustic unit parameter base, and thus also the memory required to store the acoustic units. Reduce space By utilizing the difference between the acoustic units and acoustic segments, the present invention represents the acoustic segments and instead replaces the waveform of the acoustic segments with the speech symbols of each acoustic segment and its corresponding acoustic unit parameters and the difference between them. do. This can express the speech information of the syllable corresponding to each acoustic segment, thereby reducing the distortion.

The present invention provides a high-efficiency text-to-speech method and apparatus, and provides high quality synthesized speech. The system performance and memory space required make it suitable for small portable devices as well as for ordinary computers.

Detailed description of the preferred embodiment

Referring to Figure 1, a prior art TTS conversion system is shown. The system includes three main parts: text processor 100, acoustic segment base 200, and synthesizer 300. The main function of the text processor 100 is to have normalized and split input text, and then assign characters of the text to corresponding phonetic symbols. The system utilizes the obtained speech symbol subsequent to matching the speech symbol sequence stored in the acoustic segment base 200 and then replaces the speech symbols corresponding to the acoustic segments of the corresponding speech or phrases. In turn, synthesizer 300 chains these acoustic segments in accordance with the text with the insertion of the appropriate breaks, thus obtaining the required audio output. The acoustic segment base 200 stores a huge amount of text content information and utterances of the text content. The more speech information there is, the closer the synthesized speech is to the actual human speech. If the sentence of the input text matches completely and directly with the sentence stored in the acoustic segment base, the waveform of this stored sentence can be used directly for the speech output, ie the actual utterance recorded. In most cases, however, the system cannot find such a perfectly matched sentence. In this case, partial matching of words and phrases in this sentence is required and thus word division is essential. The corresponding acoustic segments are then identified to provide a TTS transform.

In FIG. 1, input text is first normalized using text normalization unit 110. Thereafter, the word dividing unit 130, guided by the dictionary 120, performs sentence division by oral identification and word dividing procedures. After the word division, speech symbol assignment unit 140 and acoustic segment selection unit 250 use speech or phrase set 260 to search for and select acoustic segments within acoustic segment base 200. The selected segments are sent to the brake generating circuit 380 and to the acoustic segment chain unit 370. The brake generation unit 30 generates brake information provided to the acoustic segment chain unit 370. The acoustic segment chain unit 370 connects and adds appropriate brakes and outputs speech signals to the waveform post-processing device. The waveform post-processing unit 390 then outputs the synthesized converted speech signals.

For a concatenated TTS conversion method or system, the quality of intrinsic pronunciation depends on the size of the speech waveform set and the selection of suitable acoustic segments. In order to save memory space, the present invention mainly stores the parameters of the spoken waveforms and then uses these parameters to synthesize the required speech, thus reducing memory storage overheads.

The present invention provides a method of forming a set of voiced tin. The method includes the following steps of forming a set of speech waveforms. It first records human utterances while reading various texts and stores these utterances in a row utterance waveform set. These speeches are carefully selected to produce a set of behavioral waveforms with good speech and rhyme balance.

The spoken waveforms are divided into a plurality of acoustic segments AS. Each acoustic segment AS typically corresponds to the utterance of characters in a certain language environment. Each acoustic segment is a detailed representation of syllables or sub-syllables in the detailed text and has finite speech means. Usually, in other language environments the phonetic symbols of each letter may correspond to many different acoustic segments. The purpose of the acoustic chain is to find the required suitable acoustic segment of each letter, word, or phrase in a detailed language environment, and then connect the acoustic segments together.

According to the speech classification and acoustic similarity of the acoustic segments AS, the acoustic segments AS are grouped into clusters CR which are determined by their acoustic similarity. In each cluster CR, one acoustic segment AS called an acoustic unit AU is selected as a representation of all acoustic segments AS in the cluster CR. All acoustic units AU form an acoustic unit parameter base 231. Compared with the prior art, the present invention uses an acoustic unit (AU) to represent a cluster (CR), and all other acoustic segments (AS) in the cluster (CR) are associated with the acoustic segment of the cluster (CR). Stored by offset parameters that indicate rhyme variations in comparison. Referring to this, there is a relatively small variation between all acoustic segments AS in the cluster CR. Each acoustic unit AU is thus converted into a series of parameters per frame and stored in the acoustic unit parameter base 231. Using frame vector codebook 232, the "frame parameters" of each acoustic unit will be vector-quantized as a sequence of vector indices and acoustic unit parameters. In this case, acoustic unit indices are used to replace the actual acoustic unit data, thus reducing the necessary stored data. During speech concatenation and synthesis, using acoustic unit indices will be guided to vector indices and acoustic unit parameters, which will then be guided to frame parameters of the original speech waveforms. The human's first speech waveforms can then be synthesized using the frame parameters.

For example, in Chinese, the frames representing acoustic units AU, with AU implied tones 1-5, are stored in the acoustic unit parameter base 231 in the following form:

Frame_AU_n_ (pitch, duration, energy); In this embodiment the pitch ranges from 180 to 330 (Hz); The duration ranges from 165 to 452 ms; The energy ranges from 770-7406 derived from processed and digitized utterances that change the measured RMS (root mean square) power value.

As will be apparent to those skilled in the art, sebum, energy and duration are rhyme features expressed as rhyme vectors or parameters. Thus, the sound unit AU for the voice or pinyin "Yu (2)" may be stored as frame_AU_51_ (254,251,3142) and "Mao (1)" may be stored as frame_AU_1001_ (280,190,2519). It may be.

Each acoustic segment AS of each cluster CR of the speech waveform set is mapped to corresponding acoustic unit indices of the acoustic unit parameter base. Each acoustic segment may be obtained via an acoustic unit AU representing one of the clusters CR of acoustic segments AS.

The rhyme vector between the sound segment and its corresponding sound unit can be derived. The rhyme vector represents the difference in parameters between the sound segments of each cluster and the sound unit representing the cluster. Such parameter differences are based on their differences at physical moments. Therefore, the acoustic segment can be found through the representative sound unit and some rhyme vector. The speech annotation set is thus produced by its rhyme vector in lieu of the speech symbols of each segment, its corresponding acoustic unit indices and acoustic segment waveforms.

Referring to Fig. 2, a text-voice chain synthesis is described. The text-to-speech chain includes three main parts: text processing, sound and rhyme control, and speech synthesis. Through the text processing, the input text is converted into speech symbols used for sound and rhyme control. Through data-driven control, the sound and rhyme control uses the speech annotation set to match sound unit indices and speech symbols for conversion to rhyme vectors, and then through rule-driven control, the sound Unmatched speech symbols from the annotation set will be converted to the required sound unit indices and rhyme vectors. In the speech synthesizer, the obtained sound unit indices and rhyme vectors will be converted into pixel parameters of the eigenwave waveform through the sound unit parameter base and the pixel vector codebook and then concatenated into synthesized speech.

First, text processing is briefly described. Similar to existing concatenated text-to-speech conversions, the input text of the present invention is first processed in text processor 201. Through text normalization unit 211, input irregular text is classified and converted into the system's normalized text format. The word dividing unit 212 then divides the normalized text into series of word segments according to the dictionary 213 and the associated rule base (not shown). After the division, the speech symbol assignment unit 214 converts the characters and words of the input text into a series of speech symbols. Considering Chinese, phonetic symbols will be represented by Pinyin representation. Thus if the letter "魚" (fish in Chinese) is received at unit 211 this will be converted to pinyin "Yu (2)" at unit 214, where (2) represents the pronunciation of the second tone of Yu. will be.

The acoustic and speech controller 202 of the present invention performs the analysis and processing of the obtained sequence of speech symbols. The sound and voice controller 202 includes a voice annotation set 221, a sound unit index and rhyme vector selection unit 222, a rhyme rule base 223, and a rhyme detailing unit 224. The present invention utilizes multiple controls of sound and rhyme to generate sound and rhyme information. The control includes two stages: data-driven control and rule-driven control.

In the prior art, for each voice symbol in the input text, one must first search for matching acoustic segments in the speech waveform set as output. The present invention does not use a speech waveform set directly, but uses a speech annotation set to search for parameters of matching acoustic segments.

In the data-driven control stage, for the sequence of speech symbols obtained from the control division, the sound unit index and rhyme vector selection unit 222 first obtains from the speech annotation set 221 by using text relation or rhyme relation. Find the match. The matching speech symbol is replaced with a speech vector in the corresponding sound unit index and speech annotation set. If the matched part comprises one or more brakes, then the specific acoustic unit representing the brake is inserted so as to contain the parameters brake information of the acoustic unit.

During the data-driven step for unmatched speech symbols, input text is first normalized using text normalization unit 110. Thereafter, the word dividing unit 130, guided by the dictionary 120, performs sentence division by oral identification and word dividing procedures. After the word division, speech symbol assignment unit 140 and acoustic segment selection unit 250 use speech or phrase set 260 to search for and select acoustic segments within acoustic segment base 200. The selected segments are sent to the brake generating circuit 380 and to the acoustic segment chain unit 370.

The output of the acoustic and speech controller 202 includes a series of control data reflecting the speech features of the acoustic unit, speech vectors and necessary break symbols. For example, the brake generation unit 30 generates brake information provided to the acoustic segment chain unit 370.

The system also has a speech waveform synthesizer 203 that includes a speech unit parameter base 231. The acoustic segment chain unit 370 connects and adds appropriate brakes and outputs speech signals to the waveform post-processing device. The waveform post-processing unit 390 then outputs the synthesized converted speech signals.

For a concatenated TTS conversion method or system, the quality of intrinsic pronunciation depends on the size of the speech waveform set and the selection of suitable acoustic segments. In order to save memory space, the present invention mainly stores the parameters of the spoken waveforms and then uses these parameters to synthesize the required speech, thus reducing memory storage overheads.

Based on the sound and voice control data output, the present invention provides a method of forming a voice annotation set. The method includes the following steps of forming a set of speech waveforms. It first records human utterances while reading various texts and stores these utterances in a row utterance waveform set. These speeches are carefully selected to produce a set of behavioral waveforms with good speech and rhyme balance.

As described above, the spoken waveforms are divided into a plurality of acoustic segments AS. Each acoustic segment AS typically corresponds to the utterance of characters in a certain language environment. Each acoustic segment is a detailed representation of syllables or sub-syllables in the detailed text and has finite speech means. Usually, in other language environments the phonetic symbols of each letter may correspond to many different acoustic segments.

In the frame vector codebook 232, the series of vector indices is the purpose of the acoustic chain to find the required appropriate acoustic segment of each letter, word, or phrase in a detailed language environment, and then connect the acoustic segments together. According to the speech classification and acoustic similarity of the acoustic segments AS, the acoustic segments AS are grouped into clusters CR which are determined by their acoustic similarity. In each cluster CR, one acoustic segment AS called an acoustic unit AU is selected as a representation of all acoustic segments AS in the cluster CR.

The sound unit parameter array generating unit 233 has sound units AU forming a sound unit parameter base 231. Compared with the prior art, the present invention uses an acoustic unit (AU) to represent a cluster (CR), and all other acoustic segments (AS) in the cluster (CR) are associated with the acoustic segment of the cluster (CR). Stored by offset parameters that indicate rhyme variations in comparison. Referring to this, there is a relatively small variation between all acoustic segments AS in the cluster CR.

In this respect, acoustic features representative of acoustic segments are obtained. Each acoustic unit AU is thus converted into a series of parameters per frame and stored in the acoustic unit parameter base 231. Using frame vector codebook 232, the "frame parameters" of each acoustic unit will be vector-quantized as a sequence of vector indices and acoustic unit parameters. In this case, acoustic unit indices are used to replace the actual acoustic unit data, thus reducing the necessary stored data. During speech concatenation and synthesis, using sound unit indices will be guided by vector indices and sound unit parameters.

The acoustic unit parameter changing unit 234 will guide the vector indices to frame parameters of the original speech waveforms. The human's first speech waveforms can then be synthesized using the frame parameters. For example, in Chinese, the frames representing the acoustic units AU, with AU implied tones 1-5, are stored in the acoustic unit parameter base 231 in the following form: frame_AU_n_ (pitch , Duration, energy); In this embodiment the pitch ranges from 180 to 330 (Hz); The duration ranges from 165 to 452 ms; The energy ranges from 770-7406 derived from processed and digitized utterances that change the measured RMS (root mean square) power value.

The purpose of synthesizing the speech is to reproduce acoustic segments in a group of speech waveforms, or to generate acoustic segments by low distortion based on the rhythm rule base 223. As will be apparent to those skilled in the art, sebum, energy and duration are rhyme features expressed as rhyme vectors or parameters. Thus, the sound unit AU for the voice or pinyin "Yu (2)" may be stored as frame_AU_51_ (254,251,3142) and "Mao (1)" may be stored as frame_AU_1001_ (280,190,2519). It may be. Each acoustic segment AS of each cluster CR of the speech waveform set is mapped to corresponding acoustic unit indices of the acoustic unit parameter base. Each acoustic segment may be obtained via an acoustic unit AU representing one of the clusters CR of acoustic segments AS.

In the prior art, the data drive is a rhyme vector between the acoustic segment and its corresponding acoustic unit can be derived. The rhyme vector represents the difference in parameters between the sound segments of each cluster and the sound unit representing the cluster. Such parameter differences are based on their differences at physical moments.

In order to obtain the intrinsic speech effect, the present invention also utilizes data-driven control. The difference is that the acoustic segments can be found through representative sound units and some rhyme vectors. The speech annotation set is thus produced by its rhyme vector in lieu of the speech symbols of each segment, its corresponding acoustic unit indices and acoustic segment waveforms. In the speech annotation group, only syllables and statements of the acoustic unit base are stored.

3, the present invention is also described. In Fig. 3, a text-negative chain synthesis is described. The text-to-speech chain includes three main parts: text processing, sound and rhyme control, and speech synthesis. Through the text processing, the input text is converted into speech symbols used for sound and rhyme control. Through data-driven control, the sound and rhyme control uses the speech annotation set to match sound unit indices and speech symbols for conversion to rhyme vectors, and then through rule-driven control, the sound Unmatched speech symbols from the annotation set will be converted to the required sound unit indices and rhyme vectors. In the speech synthesizer, the obtained sound unit indices and rhyme vectors will be converted into pixel parameters of the eigenwave waveform through the sound unit parameter base and the pixel vector codebook and then concatenated into synthesized speech. First, text processing is briefly described. Similar to existing concatenated text-to-speech conversions, the input text of the present invention is first processed in text processor 201. Through text normalization unit 211, input irregular text is classified and converted into the system's normalized text format. The word dividing unit 212 then divides the normalized text into series of word segments according to the dictionary 213 and the associated rule base (not shown). After the division, the speech symbol assignment unit 214 converts the characters and words of the input text into a series of speech symbols. Considering Chinese, phonetic symbols will be represented by Pinyin representation. Thus if the letter "魚" (fish in Chinese) is received at unit 211 this will be converted to pinyin "Yu (2)" at unit 214, where (2) represents the pronunciation of the second tone of Yu. will be.

After step S305, the acoustic and speech controller 202 of the present invention performs the analysis and processing of the obtained sequence of speech symbols. The sound and voice controller 202 includes a voice annotation set 221, a sound unit index and rhyme vector selection unit 222, a rhyme rule base 223, and a rhyme detailing unit 224. The present invention utilizes multiple controls of sound and rhyme to generate sound and rhyme information. The control includes two stages: data-driven control and rule-driven control. In the prior art, for each voice symbol in the input text, one must first search for matching acoustic segments in the speech waveform set as output. The present invention does not use a speech waveform set directly, but uses a speech annotation set to search for parameters of matching acoustic segments. In the data-driven control stage, for the sequence of speech symbols obtained from the control division, the sound unit index and rhyme vector selection unit 222 first obtains from the speech annotation set 221 by using text relation or rhyme relation. Find the match. The method then affects test step S308 and terminates at step S309 or returns to step S302 to determine whether there is more text going on.

Advantageously, the present invention provides for allowing a relatively small number of acoustic units representing clusters. Thus this provides memory overheads.

The detailed description provides only preferred exemplary embodiments and is not intended to limit the scope, applicability, or configuration of the present invention. Rather, the detailed description of the preferred exemplary embodiments provides those skilled in the art with a possible description for implementing the preferred exemplary embodiments of the invention. It is to be understood that various changes may be made in function and that an arrangement of elements may be made without departing from the spirit and scope of the invention as described in the appended claims.

Claims (14)

In the text-to-speech method, Dividing the text into divided speech units; Identifying a suitable acoustic unit for each of the speech units, wherein each acoustic unit represents acoustic segments forming a speech cluster determined by acoustic similarity; Determining variances between the prosodic parameters of the acoustic unit and each of the speech units; Generating acoustic parameters from the rhyme parameters and associated variations of the acoustic unit to select a acoustic segment, and Providing an output speech signal based on the acoustic segment. The method of claim 1, And the rhyme parameters comprise a pitch. The method of claim 1, And the rhyme parameters comprise a duration. The method of claim 1, And the rhyme parameters comprise energy. The method of claim 1, And the determining step is based on the position of the acoustic unit in a phrase or sentence. The method of claim 1, And the determining step is based on co-articulation. The method of claim 1, And the determining step is based on a phrase length. The method of claim 1, And the determining step is based on adjacent characters of the sound unit. The method of claim 1, And wherein said dividing comprises dividing sentences of text into syllables. The method of claim 1, And the speech units are syllables. The method of claim 1, And the speech units are assigned to speech symbols. The method of claim 11, And the speech symbol is a pinyin representation. The method of claim 1, And the output speech signal is a selection of concatenated selected acoustic segments. The method of claim 11, And the output speech signal is a selection of concatenated selected acoustic segments.