JP4503853B2 - Speech synthesizer based on variable rate speech coding - Google Patents

Abstract

An apparatus and method for speech synthesis based on variable rate vocoding is presented. An input speech signal is encoded by a variable rate vocoder (202), and the parameters of the speech signal are stored in memory. To synthesize speech, a variable rate decoder (208) decodes the parameters to produce speech samples. A codec (210) converts the speech samples from a digital signal to an analog signal, which is broadcast through a speaker (212).

Description

[0001]
Background of the Invention
I. Field of Invention
The present invention relates to speech synthesis. In particular, the present invention relates to synthesis of speech encoded by a variable rate vocoder (speech analysis / synthesis device). Furthermore, the present invention relates to the use of speech synthesis with a wireless communication device.
II. Explanation of related technology
Electronic speech synthesis is useful in many applications. Computers and other electronic devices are about to have voice prompt options as user interfaces. For example, voice is useful for reading electronic mail messages, generating spoken words in a voice response system, or directing a driver to a car.
[0002]
There are two types of speech synthesizers or techniques used to create speech. The first type belongs to the text-to-speech (TTS) speech synthesizer and is based on grammar. The TTS system converts ordinary text into easy-to-understand natural speech. This is useful for applications that require automatic conversion to convert any input text into an easy-to-understand natural speech output. This is particularly useful when there is a lot of vocabulary and / or constantly changing data. TTS is useful for applications that provide automatic voice alerts and responses, calibration, telephone access to databases, and conversion of email to voice mail or audio output. Because TTS is flexible and powerful, it is useful for many applications. However, implementation of a TTS system requires a huge memory capacity and processing capacity. In addition, if the speech synthesizer does not imitate the human speech inflection, it will become a mechanical tone. Therefore, TTS is not a practical choice for applications with limited memory capacity and processing capability, such as small portable wireless devices, remote communication devices or computers.
[0003]
The second type of speech synthesizer is based on a voice coder (vocoder). A vocoder compresses a speech or audio signal by extracting parameters relating to a human speech generation model. The vocoder was developed to compress digitally converted input speech at a rate of 64 kilobits per second (kbps) to a low rate of 13 kbps, 8 kbps, or less. A vocoder-based speech synthesizer generates certain parameters for or for speech to be synthesized. This parameter is stored in some kind of memory, preferably flash memory, and decoded based on speech synthesis. Since the parameters of all synthesized words must be stored in memory, vocoder-based speech synthesizers are more suitable for applications that do not require a large amount of vocabulary. This speech synthesizer is particularly suitable for systems having limited memory capacity and processing capability.
[0004]
In a vocoder-based speech synthesizer, it is necessary to optimize the use of memory while maintaining good speech quality. For some applications, it is desirable to maximize the number of vocabularies for a given memory capacity. It is also desirable to use signal processing resources that are already utilized in a given communication system design to perform speech synthesis. A speech synthesizer having these and other characteristics is provided by the present invention described below.
[0005]
Summary of the Invention
The present invention is an apparatus and method for speech synthesis based on variable rate speech coding. The synthesized speech is encoded by a variable rate speech vocoder. A variable rate vocoder encodes a speech frame at one of a set of predetermined rates based on speech movement occurring within the speech frame. In one embodiment, the variable rate vocoder is a code-excited linear prediction (CELP) encoder with a 4-bit rate. In this way, the input speech signal is encoded into speech parameters at one of four rates by the CELP coding scheme at the selected rate. Speech parameters are generally provided in a decoder that performs a variable rate decoding method corresponding to a commonly used variable rate coding method. The decoder generates speech samples that are fed to an encoder-decoder or a digital to analog conversion codec. The resulting analog signal generated by the codec is then broadcast as synthesized speech through speakers or other known audio output devices.
[0006]
The speech synthesizer of the present invention is particularly suitable for use in a wireless communication system in which variable rate speech coding has already been performed. In these systems, existing speech coding resources can be used for speech synthesis. Alternatively, DSP elements that are already present and easily installed can be used with a small amount of memory to provide speech synthesizer functionality. In addition, a speech synthesizer based on variable rate speech coding can provide good speech quality without requiring a large capacity memory. The compression level provided by the variable rate voice vocoder is suitable for applications with limited memory.
[0007]
The features, objects and advantages of the present invention will become more apparent from the detailed description set forth below when taken in conjunction with the drawings in which like reference characters identify correspondingly.
[0008]
Detailed Description of the Preferred Embodiment
The present invention provides a speech synthesizer and speech synthesis method that are very useful when used with a wireless communication device. The present invention utilizes existing signal processing resources in a wireless communication device or uses minimal additional hardware to synthesize speech in a manner that provides high voice quality and requires small memory capacity. Can do.
[0009]
The present invention is very useful when used with a variety of known communication devices and will be described below with respect to a CDMA wireless communication system. In addition, it is particularly suitable for special applications, such as hands-free car kits used to mount and operate wireless devices on vehicles. However, those skilled in the art will recognize that this is not a limitation of the present invention and other devices, including devices that communicate in wired telegraph or optical cable systems, and devices that use other signal modulation techniques. It will be readily appreciated that it can be used with any type of communication device.
[0010]
Typical wireless communication systems use code division multiple access (CDMA) modulation techniques. While other techniques are known, such as time division multiple access (TDMA), frequency division multiple access (FDMA), and amplitude modulation (AM) such as amplitude companding single sideband (ACSSB), CDMA is the other technology. Has a great advantage over. The use of CDMA technology in a multiple access communication system is attributed to U.S. Pat. No. 4,901,307 entitled “Spread Spectrum Using Satellite or Terrestrial Repeaters”, which is incorporated herein by reference. Multiple Access Communication System ".
[0011]
For various reasons, a speech synthesizer can be implemented in a wireless communication device and equipment. For example, speech synthesis is part of a “hands-free” car kit used to assist operation in a wireless telephone speech recognition system or vehicle. The speech synthesizer can provide information in an audible form when the user or driver of the device cannot see the output screen or pointing device on the device. For example, information can be provided to drive or output a device when a vehicle driver or machine driver approaches and cannot safely view the communication device. The speech synthesizer also allows hands-free operation of the device by giving voice instructions for work to be performed. For example, the voice synthesizer automatically asks the device to dial a phone number to ask, dial, remember, open mail, hang up, or hang up (shut down) It is possible to ask the command (command) to be implemented.
[0012]
In one embodiment, the present invention uses existing vocoder circuitry in some wireless devices such as wireless telephones and other products utilized by communication service subscribers to generate voice. In particular, the speech recognition device of the present invention is based on a variable rate vocoder. Variable rate vocoders use voice motion to change the instantaneous data rate. When speech is moving, the vocoder encoder uses a large number of bits to encode the speech sample. In silence periods, the vocoder encoder uses a small number of bits or less to encode background noise. An exemplary embodiment of a variable rate vocoder is described in US Pat. No. 5,414,796 “Variable Rate Vocoder”, which is assigned to the right holder of the present invention and incorporated herein by reference.
[0013]
Variable rate vocoders are commonly used in CDMA communication systems to increase system capacity by reducing the number of bits commonly used by each communication signal. The variable rate vocoder can be implemented, for example, in the CDMA communication system of the aforementioned Patent No. 4,901,307. In a CDMA communication system, other users use the same band but communicate using different code channels. A variable rate vocoder in a CDMA communication system takes advantage of the fact that the user is actually speaking about 40% of the time on a given channel. By sending a few bits when the user is not speaking, the variable rate vocoder allows more users to share the same band.
[0014]
A schematic block diagram of a typical variable rate vocoder is shown in FIG. Although the vocoder shown in FIG. 1 uses four different data rates, it should be understood that a different number of data rates may be employed instead, as is well known in the art. If the highest rate is 13.2 kbps in a set of four data rates, the total rate corresponds to 13.2 kbps, the 1/2 rate corresponds to about 6.2 kbps, and the 1/4 rate corresponds to about 2. Corresponding to 7 kbps, and 1/8 rate corresponds to about 1.0 kbps. The actual bit rate other than the full rate is approximate because it uses overhead bits, as is well known in the art.
[0015]
Referring to FIG. 1, the variable rate vocoder 100 includes an encoder 102 and a decoder 104. The encoder 102 receives as input an audio sample for a frame of audio data, for example as an 8-bit PCM sample at a data rate of 64 kbps in mu-law or a-law format. The encoder 102 encodes these speech samples into speech parameters at one of four data rates according to speech motion. The input voice sample is also supplied to the rate determining unit 106.
[0016]
The rate determining unit 106 includes several rate determining algorithms. In one embodiment, an energy threshold for the background noise energy level is used to determine speech movement and thereby the rate at which input samples are encoded. If the current frame of the speech sample is far above the background noise energy, the rate determining unit 106 determines to encode the frame at full rate. If the current frame of the speech sample is close to the background noise energy, the rate determiner 106 determines to encode the frame at an eighth rate, etc., as is well known.
[0017]
No. 5,056,056, filed on Sep. 10, 1993, which is hereby incorporated by reference into the present invention and incorporated herein by reference. Another rate determination technique is disclosed in 08 / 286,842, “Method and Apparatus for Variable Rate Speech Coding with Reduced Rate”. This technique provides a set of rate determination criteria referred to as mode measures. The first mode measure is the target matching signal-to-noise ratio (TMSNR) from the previous coding frame, which provides information on how well the coding model is performed by comparing the synthesized speech signal with the input signal. To do. The second mode measure is a normalized autocorrelation function (NACF), which measures the periodicity of speech frames. The third mode measure is a zero crossing (ZC) parameter, which measures the high frequency components of the input speech frame. A fourth mode measure, Predictive Gain Differentiation (PGD), determines whether the encoder maintains predictive efficiency. The fifth mode measure is the energy derivative (ED), which compares the current frame energy with the average frame energy.
[0018]
Using the aforementioned mode measure, the rate determination logic selects the coding rate for the input speech data. The values for the various modes select one of four or more modes to be activated. That is, the value detected for each mode measure with respect to a threshold or other criteria determines the coding rate selection based on a preselected pattern or hierarchy. For example, if the NACF value is less than a preselected threshold and ZC is greater than a second preselected threshold, one rate is selected. However, although these conditions are not met, a quarter rate is selected if the ED is lower than the third threshold. TSNR value is large, PGD If it is small and the NACF is greater than the fourth, fifth, and sixth thresholds, respectively, a half rate is selected. Various such combinations and thresholds have been used by those skilled in the art to select code rates.
[0019]
It should be noted that other rate determination techniques may be employed in the rate determination unit 106.
[0020]
Referring to FIG. 1, a signal indicating the data rate determined by the rate determining unit 106 is supplied to the switch 108. The switch 108 is selected from among the full rate coding unit 110, the half rate coding unit 112, the quarter rate coding unit 114, and the eighth rate coding unit 116 specified by the data signal. An encoding unit for encoding a frame of the input speech sample is selected. The selected encoder encodes the speech sample to generate a signal of the encoded data packet. The rate determination unit 106 supplies a signal indicating the data rate to the switch 118 that selects the same encoding unit as the switch 108, and therefore, the signal of the encoded data packet generated by the selected encoding unit is variable. Supplied to the output of the rate vocoder.
[0021]
Each encoding unit 110, 112, 114, and 116 is arranged for encoding speech using a predetermined encoding method. Linear predictive coding methods such as code-excited linear prediction (CELP) are used in the preferred embodiment. The CELP encoder is based on the content of Thomas E.I., published in Proceeding of Mobile Satellite Conference 1998. Tremain et al., “4.8 Kbps Code Excited Linear Predictive Encoder”. Linear predictive encoders compress speech by removing the inherent redundancy inherent in speech. Speech generally shows short-term redundancy due to mechanical movement of the lips and tongue and long-term redundancy due to vocal cord vibration. The linear prediction method models these operations as filters, removes redundancy, and then models the resulting residual signal as white Gaussian noise. Therefore, the linear prediction coder achieves a bit rate reduced by the transmission filter coefficients and quantization noise rather than a full bandwidth speech signal.
[0022]
A linear predictive coding method employing a variable rate further reduces the bit rate without compromising speech quality. In FIG. 1, the full rate encoding unit 110 encodes the parameters of the input speech signal using many bits in order to keep the input characteristics well. During the period in which speech is not detected, there is almost no content to be captured or useful information, so the eighth rate encoding unit 116 encodes the parameter using a small number of bits. The change between the period of speech with motion and the period when speech is not detected is encoded by the half rate encoding unit 112 and the quarter rate encoding unit 114.
[0023]
Referring now to the decoding unit of the variable rate vocoder, the decoder 104 receives the encoded speech parameter signal as well as the signal indicating the rate used to encode the speech. The rate extractor 128 receives this input signal and determines the audio data rate. The data rate signal is supplied to a switch 130 that selects a decoding unit from a set of decoding units in order to correctly decode the input parameters. In FIG. 1, there are four possible rates for four decoding units: full rate decoding unit 120, half rate decoding unit 122, quarter rate decoding unit 124, and eighth rate decoding unit 126. Is provided for decoding speech parameters. The selected decoding unit decodes the input parameters based on the data rate to produce a decoded sample signal, typically 64 kbps pulse code modulation (PCM) samples. The data rate signal determined by the rate extraction unit 128 is supplied to the switch 132. The switch 132 selects the same decoding unit as the switch 130, and as a result, the decoded sample signal is supplied to the output of the vocoder.
[0024]
Referring to FIG. 2, a block diagram of a speech synthesis system that operates in accordance with the principles of the present invention with a variable rate vocoder is shown. This speech synthesis system includes a variable rate encoder 202 and a speech synthesizer 204. An example of the variable rate encoder 202 is the encoder 102 of FIG. Variable rate encoder 202 receives the speech signal as input and encodes the speech with one of a set of predetermined rates. In the preferred embodiment, the variable rate encoder 202 is a CELP encoder that generates speech parameters at one of the rates based on speech movement in the speech input segment.
[0025]
The present invention uses a variable rate vocoder as described in US Pat. No. 5,414,796, such as a commercially available 13 kbps vocoder product from Qualcomm. In a preferred embodiment, the variable rate vocoder is a high performance variable rate vocoder described for the IS127 standard.
[0026]
In one embodiment of the present invention, code rate determination is based on the “mode major” discussed above. Various combinations of criteria used to make rate selections are used to create what is referred to as a “reduced rate mode” or “mode” and as understood by those skilled in the art. Also, simply referred to as mode 0, mode 1, mode 2, etc. The present invention can utilize such a mode for the purpose of speech synthesis.
[0027]
The speech received by the variable rate encoder 202 is a word or phrase from a pre-selected vocabulary that is designed for synthesis by a communication device such as a telephone, car kit, or other communication device. The vocabulary may include immediate answers and warnings to be given to the user of the device. For example, by extracting and synthesizing five vocabulary words: ｀ call ', ｀ radial', ｀ program ', ｀ or', and ｀ exit ', speech synthesis is performed by 誘導 call, It can be designed to provide immediate, program, or, and exit 'answers. Alternatively, the speech synthesizer should be designed to provide previously stored information to device users in response to various device inputs, including audio, such as in a phone book, lookup table, or database. You can also. The speech received by the variable rate encoder 202 is encoded and the encoded parameters are supplied to a memory element or circuit 206 of the speech synthesizer 204 for storage.
[0028]
Memory 206 holds or stores parameters over a period of time of desired device operation. However, the vocabulary can generally have stored parameters to improve or replace parameters, such as when it is changed to change conditions or needs to be improved with respect to the characteristics of the device. desirable. Accordingly, the memory 206 is arranged in the form of a non-volatile and rewritable memory and is accomplished using flash memory elements as is well known in the art.
[0029]
As one will appreciate, the operation of loading parameters is performed during the manufacture of the communication device in which the present invention is used. Since the immediate answers and warnings to be synthesized are predetermined, they are encoded at the time of manufacture and can be stored in flash memory prior to use. The parameters can be changed or replaced during device service or through newly developed aerial programming techniques for wireless devices.
[0030]
Alternatively, variable rate encoder 202 can receive an audio signal input during operation of the communication device. For example, the user can make a voice response to the immediate answer from the voice synthesizer. The variable rate encoder 202 then encodes the user's speech and the encoded parameters are supplied to a flash memory 206 for storage and / or speech (not shown) for speech recognition purposes. Supplied to the recognition device. In this way, parameters can be entered after the input, such as by building a personal vocabulary library for each device (vocoder) user, such as when the device immediately enters or exits a valid service with respect to the user's request. It is a product of.
[0031]
The flash memory 206 is large enough to store pre-selected vocabulary parameters as well as speech parameters expected by the user. In this manner, the capacity of the flash memory 206 can be changed based on a special application requirement. Post-manufacturing storage has the advantage of reducing memory requirements where each device user does not require as much vocabulary as what the manufacturer has to install to encompass the entire large device market. A speech synthesizer can record names or other words such as ｀ Fred Smith 'by detecting the end of the target or desired phrase or speech, removing silence or redundancy, and encoding it . Thus, the audio is recorded “on-line” and can be used later to synthesize the audio output.
[0032]
It should be noted that the variable rate encoder 202 can be deployed based on available memory and required voice quality. In a four-rate system with a total rate of 13 kbps, the average rate is typically 5.88 kbps based on 40% speech movement. The use of variable rate provides high voice quality. However, if the memory capacity is limited, the variable rate encoder 202 will be arranged to operate at a fixed half rate of, for example, about 800 bytes per second. Otherwise, select from a subset of a given rate set instead of the entire rate set. For example, the reduced rate mode discussed above can be used to select various modes. In one embodiment of the present invention, the rates are classified into a set of four modes with modes 0, 1, 2, and 3. Using a fixed rate according to this mode, a rate of about 1800 bytes per second, a rate of 1540 bytes per second, a rate of 1400 bytes per second, and a rate of 1100 bytes per second can be used. With such a fixed reduced rate, it is possible to deliver very high quality voice given a pre-defined data rate, which is close to the quality of the landline. These four modes provide the best trade-off between synthesized speech quality and memory requirements.
[0033]
Further, the variable rate encoder 202 can switch between various operating modes (variable rate, full-half rate, variable rate subset, etc.) based on the instantaneous demands of the application. Since there is a trade-off between audio quality and memory capacity, the configuration employed depends on the application used.
[0034]
The speech parameters stored in the flash memory 206 are supplied to the variable rate decoder 208 when speech synthesis is requested. The variable rate decoder 208 is arranged to decode the speech parameters generated by the corresponding variable rate encoder 202. An example of a variable rate decoder is the decoder 104 of FIG.
[0035]
In general, the variable rate decoder 208 will be incorporated as part of a digital signal processor (DSP) used in a communication device. Such a DSP is used as or for constituting a control element for signal encoding / decoding, CDMA encoding, power adjustment and the like. Since such elements are commonly used in wireless devices and many other devices that the invention provides, their presence can be exploited for incorporation into the present invention at a very efficient cost.
[0036]
In order to have the decoding function for the present invention, only a small amount of memory is required for or connected to the DSP. Standalone decoders within or using DSPs require very small amounts of memory (both program and data) to obtain speech synthesis capability. The speech synthesizer can be incorporated using well-known DSP circuits and devices such as those commercially available from Analog Devices and Qualcomm.
[0037]
Decoding parameters, typically in the form of pulse code modulation (PCM) samples, are supplied to codec 210. The codec 210 converts the PCM sample from a digital format to an analog signal. The analog signal is supplied to a speaker or other well-known audio output device 212 that produces or broadcasts synthesized audio to the surrounding audible device environment.
[0038]
As a result, a speech synthesizer based on variable rate speech coding is provided by the present invention. This speech synthesizer is particularly suitable for use with wireless devices that already have a variable rate vocoder. In other words, existing variable rate vocoders employed by speech synthesizers through appropriate use translate into programs or operating instructions or use of control hardware. Alternatively, the use of variable rate coding allows the compression achieved to store a predetermined vocabulary in a limited amount of memory associated with the wireless device or other devices to which it is connected. Furthermore, the trade-off between speech quality and memory capacity is considered when placing a variable rate vocoder to provide a speech synthesizer with the desired speech quality and memory capacity.
[0039]
The present invention can be used in various communication devices and interface devices. The above examples can be used for user terminals, subscriber phones, mobile (mobile) stations, or wireless such as cellular and satellite phones, often referred to simply as “users”, “mobiles”, or “subscribers”. A communication device is described. In addition, other devices such as, for example, a message receiver and a data transmission device (for example, a portable computer, a personal information terminal (PDA), a modem, a machine controller), or a public telephone network or a dedicated communication line are also conceivable.
[0040]
The present invention can be implemented using a separate circuit in the form of a dedicated device or application specific IC (ASIC) to construct a speech synthesizer for installation in a desired device. Alternatively, it can be incorporated into other ASICs and devices by using a small amount of additional memory to operate with existing digital signal processor elements.
[0041]
The previous description of the preferred embodiment allows anyone skilled in the art to make or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the principles defined therein can be applied to other embodiments without using their original capabilities. Thus, the present invention is not limited to the embodiments shown herein, but extends to a wide range of fields that are compatible with the principles and novel features disclosed therein.
[Brief description of the drawings]
FIG. 1 is a block diagram of a variable rate vocoder.
FIG. 2 is a block diagram of a speech synthesizer according to the present invention.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 100 ... Variable rate vocoder 102 ... Encoder 104 ... Decoder 106 ... Rate determination part 108 ... Switch 110 ... Full rate encoding part 112 ... Half rate encoding part 114 ... Quarter rate encoding part 116 ... Eight Fraction rate encoding unit 118 ... Switch 120 ... Full rate decoding unit 122 ... Half rate decoding unit 124 ... Quarter rate decoding unit 126 ... Eighth rate decoding unit 128 ... Rate extraction unit 130 ... Switch DESCRIPTION OF SYMBOLS 132 ... Switch 202 ... Variable rate encoder 204 ... Speech synthesizer 206 ... Flash memory 208 ... Variable rate decoder 210 ... Codec 212 ... Audio output device

Claims (16)

In a wireless communication system, an apparatus for synthesizing a preselected vocabulary encoded by a variable rate encoder at a set of variable rates,
A memory for storing a set of speech parameters representing the encoded preselected vocabulary;
Receiving the word input from the user, and a processor configured to select a subset of the words the set of speech parameters or rales voice parameters according to the input from the user,
Extracting the variable rate used by the variable rate encoder from the encoded speech parameter subset signal and decoding the selected speech parameter subset based on the extracted variable rate; A variable rate decoder for generating decoded speech samples;
A device comprising a digital-to-analog converter for converting the audio sample into an analog signal for broadcasting as synthesized speech.   The apparatus of claim 1, wherein the variable rate encoder is based on linear prediction.   The apparatus of claim 1, wherein the variable rate decoder is based on linear prediction.   The apparatus of claim 1, wherein the set of speech parameters is encoded in a variable rate set comprising a full rate, a half rate, a quarter rate, and an eighth rate.   The total rate is 13.2 kbps, the half rate is about 6.2 kbps, the quarter rate is about 2.7 kbps, and the eighth rate is about 1.0 kbps. The apparatus of claim 4.   5. The apparatus of claim 4, wherein a coding rate is selected according to speech quality and memory capacity requirements.   The apparatus of claim 1, wherein the wireless communication system is a CDMA system.   The apparatus of claim 1, wherein the variable rate encoder comprises an enhanced variable rate encoder. In a wireless communication system, a method of synthesizing a preselected vocabulary encoded by a variable rate encoder at a set of variable rates, comprising:
Receive user input of words,
Retrieving a set of speech parameters stored in memory that is the encoded preselected vocabulary corresponding to a user input of the word;
The variable rate used by the variable rate encoder is extracted from the encoded speech parameter subset signal, and the extracted one is extracted using a variable rate decoding scheme based on the extracted variable rate. Decoding a set of speech parameters to generate a decoded speech sample;
Converting the audio sample into an analog signal for broadcast as synthesized speech. The method of claim 9 , wherein the variable rate coding scheme is based on linear prediction. The method of claim 9 , wherein the variable rate decoding scheme is based on linear prediction. 10. The method of claim 9 , wherein the set of speech parameters is encoded with a variable rate set comprising a full rate, a half rate, a quarter rate, and an eighth rate. The total rate is 13.2 kbps, the half rate is about 6.2 kbps, the quarter rate is about 2.7 kbps, and the eighth rate is about 1.0 kbps. The method of claim 12 . 13. The method of claim 12 , wherein a code rate is selected according to speech quality and memory capacity requirements. The method of claim 9 , wherein the wireless communication system is a CDMA system. 10. The method of claim 9 , further comprising the step of encoding the user input of the word and adding the user input of the encoded word to the memory as part of the set of speech parameters.

Images