KR20020081374A - Closed-loop multimode mixed-domain linear prediction speech coder - Google Patents

Abstract

A closed-loop, multimode, mixed-domain linear prediction (MDLP) speech coder includes a high-rate, time-domain coding mode, a low-rate, frequency-domain coding mode, and a closed-loop mode-selection mechanism for selecting a coding mode for the coder based upon the speech content of frames input to the coder. Transition speech (i.e., from unvoiced speech to voiced speech, or vice versa) frames are encoded with the high-rate, time-domain coding mode, which may be a CELP coding mode. Voiced speech frames are encoded with the low-rate, frequency-domain coding mode, which may be a harmonic coding mode. Phase parameters are not encoded by the frequency-domain coding mode, and are instead modeled in accordance with, e.g., a quadratic phase model. For each speech frame encoded with the frequency-domain coding mode, the initial phase value is taken to be the initial phase value of the immediately preceding speech frame encoded with the frequency-domain coding mode. If the immediately preceding speech frame was encoded with the time-domain coding mode, the initial phase value of the current speech frame is computed from the decoded speech frame information of the immediately preceding, time-domain-encoded speech frame. Each speech frame encoded with the frequency-domain coding mode may be compared with the corresponding input speech frame to obtain a performance measure. If the performance measure falls below a predefined threshold value, the input speech frame is encoded with the time-domain coding mode.

Description

CLOSED-LOOP MULTIMODE MIXED-DOMAIN LINEAR PREDICTION SPEECH CODER}

Voice transmission by digital technology has become widespread, particularly in long distance and digital cordless telephone applications. In turn, this has generated interest in determining the minimum amount of information that can be transmitted over a channel while maintaining the perceived quality of the reproduced speech. When voice is transmitted by simply sampling and digitizing, a data rate of 64 kilobits per second (kbps) is required to achieve the voice quality of a conventional analog telephone. However, the data rate can be significantly reduced using voice analysis, followed by appropriate coding, transmission, and re-synthesis at the receiver.

An apparatus using a technique of extracting parameters related to a human speech generation model and compressing a language is called a speech coder. The speech coder divides the incoming speech signal into time blocks or analysis frames. Generally, a speech coder has an encoder and a decoder. The encoder analyzes the incoming speech frame to extract certain relevant parameters and then quantizes the parameters into a binary representation, i. E., A set of bits or a binary data packet. The data packet is transmitted to the receiver or decoder via the communication channel. The decoder processes the data packets, quantizes them to generate parameters, and uses the dequantized parameters to reconstruct the speech frames.

The function of the speech coder is to compress the digitized speech signal into a low bit rate signal by eliminating all natural redundancies inherent in the speech. Digital compression can be achieved by expressing an input speech frame as a parameter set and using quantization, expressing the parameters in a bit set. When the input voice frame has the number of bits N _i and the data packet generated by the voice coder has the number of bits N _o , the compression rate obtained by the voice coder is to be. What should be tried is to maintain the sound quality of the decoded speech while maintaining the target compression rate. The performance of the speech coder depends on (1) how well the speech model or a combination of the analysis and synthesis processing described above is performed, and (2) how well the parameter quantization process is performed at the target bit rate of N ₀ bits per frame. Therefore, the purpose of the speech model is to obtain the characteristics of the speech signal and the target speech quality with a small set of parameters for each frame.

The speech coder may be implemented as a time domain coder, which obtains a time domain speech waveform by using a high resolution processing that encodes a small segment of speech (typically 5 milliseconds (ms) subframes) at one time. For each subframe, high-precision representative values from the codebook region can be known through various search algorithms known in the art. Alternatively, the speech coder may be implemented as a frequency domain coder, which captures the short-term speech spectrum of the input speech frame with a set of parameters and uses the corresponding synthesis processing to reproduce the speech waveform from the spectral parameters. The parameter quantizer preserves the parameters by expressing the parameters in a stored codevector representation according to the known quantization technique described in A. Gersho & R. M. Gray, Vector Quantization and Signal Compression (1992).

A known time domain speech coder is the Code Excited Linear Predictive (CELP) coders described in LB Rabiner & RW Schafer, Digital Processing of Speech Signals, 396-453 (1978). In this CELP coder, short-term correlation and redundancy of speech signals are removed by linear prediction (LP) analysis which obtains coefficients of a short-term formant filter. The LP residual signal is generated using a short-term prediction filter on the incoming speech frame, and further modeled and quantized with the long-term prediction filter parameter and a subsequent probabilistic codebook. Thus, CELP coding divides the task of encoding the time domain speech waveform into separate operations that encode LP short term filter coefficients and encode the LP residual components. Time-domain coding can be performed at a fixed rate (i.e., using the same number of bits N _0, for each frame) or a variable rate (using a different bit rate relative to the frame can be of different types). The variable-rate coder uses the amount of bits needed to encode the codec parameters at a level suitable for obtaining the target quality. A representative variable rate CELP coder, which is assigned to the assignee of the present invention and is described in U.S. Patent No. 5,414,796, which is incorporated herein by reference.

In general, time-domain coders such as the CELP coders, in order to maintain the accuracy of the time-domain speech waveform, uses a number of bits N ₀ per high frame. In general, these coders provide a relatively large number of bits per frame N ₀ (e.g., 8 kbps or higher), thus having excellent voice quality. However, at low bit rates (4 kbps or less), time domain coders can not maintain high quality and good performance due to a limited number of available bits. At low bitrates, this limited codebook space degrades the waveform matching capabilities of conventional time-domain coders that are successfully used in high-rate commercial applications.

Currently, there is a research interest and strong commercial need to develop high quality voice coders that operate at medium to low bit rates (i.e., in the 2.4 to 4 kbps range). Applications include wireless telephony, satellite communications, Internet telephony, various multimedia, and voice streaming applications, voice mail, and other voice storage systems. Its driving force is the demand for high capacity and good performance under packet loss conditions. A variety of recent voice coding standardization efforts are another direct driving force for research and development of low rate speech coding algorithms. A low rate voice coder creates more channels or users per allowable application bandwidth and a low rate voice coder in combination with an additional layer of appropriate channel coding can satisfy the bit budget of the overall coder specification and is superior Performance.

For coding at low bitrates, spectrum or frequency domain speech coding has been developed in various ways to analyze speech signals into time-varing evolution of the spectrum (see, e.g., RJ McAulay & TF Quatieri, Sinusodial In Spectrum Coders, the purpose is not to accurately mimic a time-varying speech waveform, but rather to use a set of spectral parameters (e.g., And then generates the output speech frame with the decoded parameters. The resulting synthesized speech does not match the original input speech waveform, but provides similar recognition quality Examples of frequency domain coders known in the art include a multiband excitation coder (MBE), a sine wave transformer Coder (STC), and harmonic coder (HC). These frequency domain coders provide a high quality parameter model with a compact set of parameters that can be accurately quantized with a small number of bits available at low bit rates .

However, low bit rate coding limits the efficiency of a single coding mechanism by imposing critical constraints on limited coding resolution or limited codebook space, so that coders can not represent different types of speech segments with the same accuracy under various background conditions . For example, conventional low bit rate, frequency domain coders do not transmit phase information for voice frames. Instead, the phase information is reproduced using a random, initial phase value of artificial creation and a linear interpolation technique. See, for example, H. YANG et al., Quadratic Phase Interpolation for Speech Synthesis in the MBE Model in 29 Electronic Letters 856-57 (May 1993). Since the phase information is artificially generated, even if the amplitude of the sine wave is completely preserved by the quantization-dequantization processing, the output speech generated by the frequency domain coder does not coincide with the original input speech (i.e., ). Thus, for example, it has been demonstrated that in a frequency domain coder, it is difficult to use any closed-loop performance measures such as signal-to-noise ratio (SNR) or perceptual SNR (SNR).

Multimodal coding techniques are used to perform low rate speech coding by open loop mode determination processing. An example of such a multimode technique is Amitava Das et al., Speech Coding and Synthesis ch. in Multimode and Variable-Rate Coding of Speech (WB Kleijin & KK Paliwal eds. 1995). Conventional multimode coders apply different modes or encoding-decoding algorithms for different input speech frames. Each mode or encoding-decoding process may be tailored to suit the most efficient manner of representing certain types of speech segments, e.g., voiced speech, unvoiced speech, or background noise (non-speech) Loses. Typically, the external, open-loop mode determination mechanism examines the input speech frame to determine what mode to apply to the frame. In general, the open loop mode determination is performed by extracting a plurality of parameters from an input frame, evaluating parameters with respect to a certain time and spectral characteristics, and making a mode determination based on the evaluation. Thus, the mode determination is made without knowing in advance the actual conditions of the output speech, i.e. how close the output speech is to the input speech with respect to the speech quality or other performance measures.

Based on the above, it is desirable to provide low bit rate, frequency domain coders that more accurately evaluate the phase information. It is also desirable to provide a multi-mode, mixed-region coder that time-domain encodes some voice frames and frequency-domain encodes other voice frames based on the voice content of the frame. It is also desirable to provide a mixed area coder in which some speech frames may be time domain encoded and other language frames may be frequency domain encoded in accordance with the closed loop coding mode determination mechanism. Therefore, a closed loop, multi-mode, mixed region voice coder is required that ensures time synchronization between the output speech generated by the coder and the original speech input by the coder.

In general, the present invention relates to speech processing, and more particularly, to a closed loop, multimode, mixed region speech coding method and apparatus.

1 is a block diagram of a communication channel terminated by a voice coder at each stage.

2 is a block diagram of an encoder that may be used for a multimode, mixed region linear prediction (MDLP) speech coder.

3 is a block diagram of a decoder that can be used in a multimode, MDLP speech coder.

4 is a flow chart illustrating MDLP encoding steps performed by an MDLP encoder usable in the encoder of FIG.

5 is a flow chart showing a speech coding determination process.

6 is a block diagram of a closed-loop, multi-mode, MDLP speech coder.

FIG. 7 is a block diagram of a spectrum coder that may be used with the coder of FIG. 6 or the encoder of FIG. 2;

8 is an amplitude vs. frequency graph showing the amplitude of a sine wave in a harmonic coder.

9 is a flow chart showing a mode determination process of the multi-mode, MDLP voice coder.

FIG. 10A is a speech signal amplitude versus time graph, and FIG. 10B is a linear prediction (LP) residue component amplitude versus time graph.

11B is a perceptual signal-to-noise ratio (PSNR) versus frame index graph for a closed-loop determination, and FIG. 11C is a plot of the rate / mode versus frame index for closed- And a PSNR-to-frame index graph.

The present invention relates to a closed loop, multimode, mixed region voice coder that ensures time synchronization between the output speech produced by the coder and the original speech input to the coder. Thus, in one aspect of the invention, preferably, the multimode, mixed region, speech processor comprises a coder having one or more time domain coding modes and one or more frequency domain coding modes, and a coder coupled to the coder, And a closed loop mode selection device configured to select a coding mode for the coder based on the content of the frame.

In another aspect of the present invention, preferably, the frame processing method includes applying an open-loop coding mode selection process to each successive input frame to generate a time-domain coding mode or a frequency-domain coding mode Selecting one; Frequency-domain coding an input frame when the audio content of the input frame indicates a stable voice voice; Comparing the frequency domain coded frame with an input frame to obtain a performance measure; And time-domain coding the input frame if the performance measure falls below a predetermined threshold.

In another aspect of the present invention, preferably, the multimode, mixed region, speech processor applies an open loop coding mode selection process to an input frame to generate a time domain coding mode or a frequency domain coding Means for selecting one of the modes; Means for frequency-domain coding an input frame if the audio content of the input frame represents a steady state voice; Means for time-domain coding the input frame if the audio content of the input frame is not a steady state voice; Means for comparing a frequency domain coded frame with an input frame to obtain a performance measure; And means for time-domain coding the input frame if the performance measure falls below a predetermined threshold.

1, a first encoder 10 receives a digitized speech sample s (n) and generates a sample s (n) for transmission to the first decoder 14 over a transmission medium 12 or communication channel 12, ). Decoder 14 decodes the encoded speech samples and synthesizes the output speech signal s _SYNTH (n). For transmission in the reverse direction, the second encoder 16 encodes the digitized speech samples s (n) transmitted over the communication channel 18. The second decoder 20 receives and decodes the encoded speech samples to produce a synthesized output speech signal s _SYNTH (n).

For example, the speech samples s (n) may be encoded by various methods known in the art, including pulse code modulation (PCM), compassed μ-law or A-law, Thus representing speech signals that are digitized and quantized. As is known in the art, a speech sample s (n) consists of an input data frame, where each frame contains a predetermined number of digital speech samples s (n). In a preferred embodiment, each 20 ms frame uses a sampling rate of 8 kHz including 160 samples. In the preferred embodiment described below, it is preferable to change the data transfer rate from 8 kbps (full rate) to 4 kbps (half rate) to 2 kbps (1/4 rate) to 1 kbps (1/8 rate) . Other data rates may also be used. The term " full rate " or " high rate ", as used herein, refers to a data rate that is typically 8 kbps or higher and a " half rate " It is desirable to change the data rate because the low bit rate can selectively use frames that contain relatively less audio information. Other sampling rates, frame sizes, and data transfer rates may be used, as will be appreciated by those skilled in the art.

The first encoder 10 and the second decoder 20 jointly comprise a first speech coder or a language codec. Similarly, the second encoder 16 and the first decoder 14 have a second voice coder in common. Those skilled in the art will recognize that voice coders can be implemented in a digital signal processor (DSP), an application specific integrated circuit (ASIC), discrete gate logic, firmware, or any conventional programmable software module and microprocessor. The software module may be a RAM memory, a flash memory, a register, or any other type of recordable storage medium known in the art. Alternatively, the microprocessor can be replaced with any conventional processor, controller, or state machine. Representative ASICs specifically designed for speech coding are disclosed in U. S. Patent Application Serial No. 08 / 197,417, entitled " VOCODER ASIC ", filed February 16, 1994, assigned to the assignee of the present invention and incorporated herein by reference Lt; / RTI >

2, a multimode, mixed region linear prediction (MDLP) encoder 100 that may be used for a voice coder includes a mode determination module 102, a pitch estimation module 104, a linear prediction (LP) analysis module 106, an LP analysis filter 108, an LP quantization module 110, and an MDLP residual encoder 112. The input speech frame s (n) is provided to the mode determination module 102, the pitch estimation module 104, the linear prediction (LP) analysis module 106, and the LP analysis filter 108. The mode determination module 102 determines the mode index I _M , the mode M, and the energy, spectral tilt, zero crossing rate, etc. of each input language frame s (n) based on the periodicity Other extraction parameters are generated. Various methods of classifying speech frames according to periodicity are described in U. S. Patent Application Serial No. 10 / 030,131, entitled METHOD AND APPARATUS FOR PERFORMING REDUCED RATE VARIABLE RATE VOCODING, filed on March 11, 1997, assigned to the assignee of the present invention, Lt; RTI ID = 0.0 > 08 / 815,354. ≪ / RTI > These methods are also included in TIA / EIA IS-127 and TIA / EIA IS-733 provisional standards of the Telecommunications Industry Association.

The pitch estimation module 104 generates based on the pitch index I _p and the lag value P _o based on each input speech frame s (n). The LP analysis module 106 performs a linear prediction analysis on each input speech frame s (n) to generate an LP parameter a. The LP parameter a is provided to the LP quantization module 110. In addition, the LP quantization module 110 receives the mode M and performs quantization processing according to the mode. The LP quantization module 110 receives the LP index I _LP and the quantized LP parameters . The LP analysis filter 108 includes a quantized LP parameter < RTI ID = 0.0 > . The LP analysis filter 108 generates an LP residual signal R [n], which is a quantized linear prediction parameter And an error between the input speech frame s (n) and the reconstructed speech. The LP residual component R [n], the mode M, and the quantized LP parameter Is provided to the MDLP residual encoder (112). Based on these values, the MDLP residual encoder 112 calculates the residual index I _R and the quantized residual signal &_lt; _{RTI ID} = 0.0 _> In accordance with the method described below with reference to the flowchart of Fig.

3, a decoder 200 that can be used in a speech coder includes an LP parameter decoding module 202, a residual decoding module 204, a mode decoding module 206, and an LP synthesis filter 208. Mode decoding module 206 produces a mode M from it receives the mode index I _M. The LP parameter decoding module 202 receives the mode M and the LP index I _LP . The LP parameter decoding module 202 decodes the received value to obtain a quantized LP parameter . The residual decoding module 204 receives the residual index I _R , the pitch index I _P , and the mode index I _M. The residual decoding module 204 decodes the received value and outputs the quantized residual signal < RTI ID = 0.0 > . Quantized residual signal And a quantized LP parameter Is provided to an LP synthesis filter 208 and an LP synthesis filter 208 filters the decoded output speech signal < RTI ID = 0.0 > .

Except for the MDLP residual encoder 112, the operation and implementation of the various modules of the encoder 100 of FIG. 2 and the encoder 200 of FIG. 3 are described in the aforementioned US Pat. Nos. 5,414,796 and L.B. Rabiner & R.W. Schafer, Digital Processing of Speech Signals 396-453 (1978).

According to one embodiment, an MDLP encoder (not shown) performs the steps shown in the flow chart of FIG. The MDLP encoder may be the MDLP residual encoder 112 of FIG. In step 300, the MDLP encoder checks whether the mode M is full rate (FR), quarter rate (QR), or 1/8 rate (ER). If mode M is FR, QR, or ER, then the MDLP encoder proceeds to step 302. In step 302, the MDLP encoder applies FR, QR, or ER depending on the value of the corresponding rate M to the residual index I _R. In FR mode, time-domain coding, which is high accuracy, high rate coding, and preferably CELP coding, is applied to the LP residual frame or voice frame. And then transmits the frame (after further signal processing, including digital-to-analog conversion and modulation). In one embodiment, the frame is an LP residual frame indicating a prediction error. In another embodiment, the frame is a speech frame representing a speech sample.

On the other hand, in step 300, if the mode M is not FR, QR, or ER (i.e., if the mode M is a half rate (HR)), the MDLP encoder proceeds to step 304. In step 304, spectral coding, preferably harmonic coding, is used as a half rate for the LP residual or speech signal. The MDLP encoder then proceeds to step 306. In step 306, a distortion measure D is obtained by decoding the encoded speech and comparing it to the original input frame. The MDLP encoder then proceeds to step 308. In step 308, the distortion measure D is compared to a predetermined threshold T. In step 308, the distortion measure D is compared with a predetermined threshold T. If the distortion measure D is greater than the threshold T, then the corresponding quantized parameters for the half-rate, spectrally encoded frame are modulated and transmitted. On the other hand, if the distortion measure D is not greater than the threshold T, the MDLP encoder proceeds to step 310. In step 310, the decoded frame is re-encoded in the time domain at full rate. Preferably, any conventional high rate, high accuracy coding algorithm such as CELP coding can be used. And then modulates and transmits FR mode quantized parameters associated with the frame.

As shown in the flow chart of FIG. 5, a closed-loop, multi-mode, MDLP voice coder, in accordance with an embodiment of the present invention, follows a series of steps for processing samples for transmission. In step 400, the speech coder receives a digital sample of a speech signal that is a continuous frame. Upon receiving the given frame, the voice coder proceeds to step 402. In step 402, the coder detects the energy of the frame. Energy is a measure of the voice activity of the frame. Voice detection is performed by summing the magnitudes of the magnitudes of digitized voice samples and comparing the summed energy to a threshold. In one embodiment, the threshold is adopted based on the level of change in background noise. A representative variable threshold audio activity detector is disclosed in the aforementioned U.S. Patent No. 5,414,796. The sound of some unvoiced speech may be extremely low energy samples and may be incorrectly encoded as background noise. To prevent this, the spectrum tilt of the low energy samples can be used to distinguish the unvoiced language from the background noise, as disclosed in the above-mentioned U.S. Patent No. 5,414,796.

After detecting the energy of the frame, the speech coder proceeds to step 404. In step 404, the voice coder determines if the detected frame energy is sufficient to classify the frame as containing voice information. If the detected frame energy is lower than a predetermined threshold level, the speech coder proceeds to step 406. In step 406, the speech coder encodes the frame as background noise (i.e., non-speech or silence). In one embodiment, the background noise frame is time domain encoded at 1/8 rate or 1 kbps. In step 404, if the detected frame energy coincides with or exceeds a predetermined threshold level, the frame is classified as speech and the speech coder proceeds to step 408.

In step 408, the voice coder determines if the frame is periodic. Various known methods for periodicity determination include, for example, the use of zero crossing and the use of a normalized auto correlation function (NACF). In particular, the use of code transition points and NACFs to detect periodicity is assigned to the assignee of the present invention and is incorporated herein by reference in its entirety, filed March 11, 1997, entitled METHOD AND APPARATUS FOR PERFORMING REDUCED RATE VARIABLE RATE VOCODING &Quot; in U. S. Patent Application Serial No. 08 / 815,354. Also, the above methods for distinguishing voice speech from unvoiced speech are included in the Telecommunication Industry Association provisional standards TIA / EIA IS-127 and TIA / EIA IS-733. In step 410, if the frame is not determined to be periodic, the speech coder proceeds to step 410. In step 410, the speech coder encodes the frame into an unvoiced speech. In one embodiment, the unvoiced speech frame is time domain encoded at either quarter rate or 2 kbps. If it is determined in step 408 that the frame is periodic, the speech coder proceeds to step 412.

In step 412, the voice coder determines whether the frame is sufficiently periodic, for example, using a period detection method known in the art, as described in U.S. Patent Application Serial No. 08 / 815,354, supra. If it is determined that the frame is not sufficiently periodic, the speech coder proceeds to step 414. In step 414, the frame is temporally encoded with transition speech (i.e., transition from unvoiced speech to voice speech). In one embodiment, the transition voice frame is time domain encoded at full rate or 8 kbps.

In step 412, if the voice coder determines that the frame is sufficiently periodic, the voice coder proceeds to step 416. [ In step 416, the voice coder encodes the frame as a voice voice. In one embodiment, the voice speech frame is spectrally encoded at half rate or at 4 kbps. Preferably, the voice speech frame is spectrally encoded with a harmonic coder as described below with reference to Fig. Alternatively, other spectral coders can be used, such as, for example, a sinusoidal transcoder, a multi-band excursion coder, as known in the art. The voice coder then proceeds to step 418. In step 418, the speech coder decodes the encoded voice speech frame. Thereafter, the speech coder proceeds to step 420. At step 420, the decoded voice speech frame is compared to a corresponding input speech sample for that frame to measure the composite speech distortion and determine if the half rate, voice speech, or spectral coding model is operating within tolerance. The voice coder then proceeds to step 422.

In step 422, the voice coder determines whether the error between the decoded voice speech frame and the input speech samples corresponding to that frame is below a predetermined threshold. According to one embodiment, this determination is performed in the manner described below with reference to FIG. If the encoding distortion is below a certain threshold, the speech coder proceeds to step 424. In step 424, the voice coder transmits the frame as a voice voice using the parameters of step 416. If the encoding distortion in step 422 coincides with or exceeds a predetermined threshold, the speech coder proceeds to step 414 and time-domain encodes the frame of the digitized voice sample received in step 400 as a transition speech at full rate.

Steps 400 to 410 include an open loop encoding determination mode. On the other hand, steps 412 to 426 include a closed loop, encoding determination mode.

In one embodiment, a closed loop, multimode, MDLP voice coder is shown coupled to an analog-to-digital converter (A / D) 500 coupled to a frame buffer 502 coupled to a control processor 504, Respectively. An energy calculator 506, a voice speech detector 508, a background noise encoder 510, a high rate, a time domain encoder 512 and a low rate spectrum encoder 514 are coupled to the control processor 504. The spectrum decoder 516 is coupled to a spectrum encoder 514 and the error calculator 518 is coupled to a spectrum decoder 516 and a control processor 504. [ The threshold comparator 520 is coupled to the error calculator 518 and the control processor 504. The buffer 522 is coupled to a spectral encoder 514, a spectrum decoder 516, and a threshold comparator 520.

In the embodiment of FIG. 6, the voice coder components are preferably implemented as firmware or other software-driven modules within the voice coder that are themselves installed in the DSP or ASIC. Voice coder components can be implemented as well in a number of other known ways. Preferably, control processor 504 may be a microprocessor, but may alternatively be implemented as a controller, state machine, or discrete logic.

In the multimode coder of Fig. 6, the audio signal is provided to the A / D 500. The A / D 500 converts the analog signal into a frame of digitized speech samples S (n). The digitized speech samples are provided to a frame buffer 502. The control processor 504 takes the digitized speech samples from the frame buffer 502 and provides them to the energy calculator 506. [ The energy calculator 506,

, Where the frame is 20 ms long and the sampling rate is 8 kHz.

The control processor 504 compares the calculated speech energy with the speech activity threshold. If the calculated energy is below the voice activity threshold, the control processor 504 sends the digitized voice samples from the frame buffer 502 to the background noise encoder 510. The background noise encoder 510 encodes the frame using the minimum number of bits needed to maintain an estimate of the background noise.

If the calculated energy is greater than or equal to the voice activity threshold, the control processor 504 sends the digitized voice samples from the frame buffer 502 to the voice voice detector 508. The voice speech detector 508 determines if the periodicity level of the voice frame allows effective coding using low rate spectrum encoding. Methods for determining the periodicity level of a voice frame are known in the art and include, for example, the use of a normalized autocorrelation function (NACF) and a code transition point. These and other methods are disclosed in the above-mentioned U.S. Serial No. 08 / 815,354.

The voice speech detector 508 provides a signal to the control processor 504 indicating whether the speech frame contains sufficient periodic speech that is effectively encoded by the spectrum encoder 514. [ When the voice voice detector 508 determines that the voice frame does not have sufficient periodicity, the control processor 504 sends the digitized voice samples to the high rate encoder 512, Time domain encoding at the maximum data rate. In one embodiment, the predetermined maximum data rate is 8 kbps and the high rate encoder 512 is a CELP coder.

If the voice voice detector 508 initially determines that the voice signal has sufficient periodicity to be effectively encoded by the spectrum encoder 514, then the control processor 504 may send the digitized voice samples from the frame buffer 502 to the spectral encoder < RTI ID = 0.0 & (514). A typical spectral encoder will be described below with reference to Fig.

The spectrum encoder 514 extracts the estimated pitch frequency F ₀ , the amplitude A _I of the harmonic of the pitch frequency, and the voice information V _c . The spectral encoder 514 provides these parameters to the buffer 522 and the spectral decoder 516. Preferably, the spectrum decoder 516 is similar to a decoder in an encoder of a conventional CELP encoder. The spectral decoder 516, in accordance with the spectral decoding format (described below with reference to Figure 7)

And provides the synthesized speech samples to error calculator 518. [ The control processor 504 sends the speech samples S (n) to the error calculator 518.

Error calculator 518

Calculates the mean square error (MSE) between each speech sample, S (n), and each corresponding sync speech sample. The calculated MSE is provided to a threshold comparator 520, which determines whether the distortion level is within an acceptable boundary, i. E., The level of distortion is below a predetermined threshold.

If the calculated MSE is within an acceptable limit, then the threshold comparator 520 provides a signal to the buffer 502 and the spectrally encoded data is output from the speech coder. On the other hand, if the MSE is not within an acceptable limit The threshold comparator 520 provides a signal to the control processor 504 which in turn sends the digitized samples from the frame buffer 502 to the high rate, time domain encoder 512. The time domain encoder 512 encodes the frame at a predetermined maximum rate and discards the contents of the buffer 522.

In the embodiment of Fig. 6, the type of spectral coding used is harmonic coding as described below with reference to Fig. 7, but in a different way, for example, any type of spectral coding such as sinusoidal transform coding or multi- Lt; / RTI > The use of multi-band excitation coding is disclosed, for example, in U.S. Patent No. 5,195,166, and the use of sinusoidal transform coding is disclosed, for example, in U.S. Patent No. 4,865,068.

For a transition frame and for a voice frame with a phase distortion threshold equal to or less than the periodicity parameter, the multimode coder of FIG. 6 is preferably a full-rate, 8 kbps CELP coding is used. Alternatively, any other known high-rate, time-domain coding may be used for these frames. Thus, a transition frame (and a sufficiently non-periodic voice frame) can be precisely coded to well match the waveforms at the input and output while maintaining good phase information. In one embodiment, after processing a predetermined number of consecutive voice frames whose thresholds exceed the periodicity, regardless of the decision of the threshold comparator 520, the multi-coder may perform a full rate CELP Switching to coding.

Energy calculator 506 and voice voice detector 508 in conjunction with control processor 504 include an open loop encoding determination. Alternatively, in connection with the control processor 504, the spectral encoder 514, the spectrum decoder 516, the error calculator 518, the threshold comparator 520, and the buffer 522 comprise closed loop encoding decisions.

In one embodiment described in connection with FIG. 7, spectral coding, and preferably harmonic coding, is used to encode a voice frame with sufficient periodicity at a low bit rate. In general, a spectral coder is defined as an algorithm that maintains temporal evolution of speech spectral characteristics in a perceptually meaningful way by modeling and encoding each speech frame in the frequency domain. The essential parts of these algorithms are: (1) spectrum analysis or parameter estimation; (2) parameter quantization, and (3) synthesis of the output speech waveform and the decoded parameter. It is therefore an object to maintain the important characteristics of the short-term speech spectrum with a set of spectral parameters, to encode the parameters, and to synthesize the output speech using the decoded spectral parameters. Generally, the output speech is synthesized as a weighted sum of sinusoids. The amplitude, frequency, and phase of the sine wave are spectral parameters estimated during analysis.

&Quot; Analysis by synthesis " is a known technique in CELP coding, but this technique is not used in spectral coding. The main reason for not applying the synthesis analysis to the spectrum coder is that although the speech model functions properly from a perceptible point of view, the mean square energy of the synthesized speech (MSE: Mean Square Energy ) Will be able to `. Thus, another benefit of accurately generating the initial phase is the ability to make a determination as to whether the speech model accurately encodes the speech frame by directly comparing the speech sample with the reconstructed speech.

In spectral coding,

S [n] = S _v [n] + S _uv [n], n = 1, 2, ..., N

Synthesis and, where N is the number of samples per frame, S _v and S _uv are each a voice and a voice component as a frozen. In sum-of-sinusoid synthesis,

And generating a, where L is the total number of sinusoidal functions, f _k is the frequency of interest in the short-term spectrum, A is the amplitude of the (k, n) is a sine wave, θ (k, n) is a phase of a sine wave. The amplitude, frequency, and phase parameters are estimated from the short-term spectrum of the input frame by spectral analysis processing. The unvoiced components may be added together with the voice component in a single sinusoidal synthesis or separately calculated by a dedicated unvoiced process and added back to S _v .

In the embodiment of FIG. 7, a specific type of spectral coder, referred to as a harmonic coder, is used to spectrally encode a sufficiently periodic voice frame at a low bit rate. The harmonic coder analyzes small segments of the frame by characterizing the frame as a sinusoidal sum. Each sine wave in the sinusoidal sum has a frequency that is an integral multiple of the pitch F ₀ of the frame. In embodiments where the particular type of spectral coder used is not a harmonic coder, the sinusoidal frequency for each frame is taken from a real set of numbers between 0 and 2 [pi]. In the embodiment of FIG. 7, the amplitude and phase of each sine wave in the summed state are selected and matched to the signal for one period as best shown in the graph of FIG. In general, the harmonic coder uses an external classification that marks each input speech frame as "voiced" or "unvoiced". For a voice frame, the frequency of the sinusoids is limited to the harmonic of the estimated pitch (F ₀ ), i.e., f _k = kF ₀ . For unvoiced speech, the short-term spectral peak is used to determine the sine wave. The amplitude and phase are similar to their evolution for the frame

A (k, n) = C ₁ (k) * n + C ₂ (k)

θ (k, n) = B 1 (k) * n 2 + B 2 (k) * n + B 3 (k)

It is inserted as, where coefficients [Ci (k), Bi ( k)] is from the short-term Fourier transform (STFT) of a windowed processing the input speech frame, the amplitude and frequency of the specific frequency position f _k (= kf _0), and It is estimated from the instantaneous value of phase. The parameters to be transmitted for each sinusoidal wave are amplitude and frequency. This phase is not transmitted, but instead is modeled according to one of several known techniques, including for example a quadratic phase model.

7, the harmonic coder has a pitch extractor 600 and harmonic analysis logic 604 coupled to window processing logic 602 and a discrete Fourier transform (DFT). Pitch extractor 600, which also receives speech samples S (n) as input, is coupled to DFT and harmonic analysis logic 604. The DFT and harmonic analysis logic 604 is coupled to the residual encoder 606. Pitch extractor 600, DFT and harmonic analysis logic 604, and residual encoder 606 are coupled to parameter quantizer 608, respectively. A parameter quantizer 608 is coupled to a channel encoder 610, which is coupled to a transmitter 612. Transmitter 612 is coupled to receiver 614 by a standard radio frequency (RF) interface, such as, for example, a CDMA over-the-air interface. The receiver 614 is coupled to a channel decoder 616 and the channel decoder 616 is coupled to a dequantizer 618. The inverse quantizer 618 is coupled to the sinusoidal sum speech synthesizer 620. In addition, phase estimator 622, which receives previous frame information as input, is coupled to sinusoidal sum speech synthesizer 620. The sinusoidal sum speech synthesizer 620 is configured to generate the synthesized speech output, s _SYNTH (n).

The pitch extractor 600, the window processing logic 602, the DFT and harmonic analysis logic 604, the residual encoder 606, the parameter quantizer 608, the channel encoder 606, the sinusoidal sum speech synthesizer 620, The phase estimator 622 may be implemented in various ways known in the art, including, for example, firmware or software modules. Transmitter 612 and receiver 614 may be implemented with any equivalent standard RF component known in the art.

In the harmonic coder of FIG. 7, the input sample S (n) is received by the pitch extractor 600, which extracts pitch frequency information F _0. The voice processing frame 602 then samples the voice frame by multiplying it with the appropriate window function to enable analysis of small segments. Using the pitch of the information supplied by the pitch extractor 608, the DFT and harmonic analysis logic 604 computes the DFT of the samples to determine the complex spectral point at which the harmonic amplitude A < ₁ > (complex spectrum point). In FIG. 8, L represents the total number of harmonics. The DFT is provided to the residual encoder 606 which extracts the speech information V _c .

As shown in Fig. 8, the V _c parameter indicates a point on the frequency axis, and above this point, the spectrum is characteristic of the unvoiced speech signal and is not a harmonic number. Alternatively, under point V _c , the spectrum is a harmonic and characterizes the voice.

The A ₁ , F ₀ , and V _c components are provided to a parameter quantizer 608 that quantizes information. The quantized information is provided to the channel encoder 610 in a packet form, and the channel encoder 610 quantizes the packet at a low bit rate, such as a half rate, 4 kbps, or the like. The packet is provided to a transmitter 612 which modulates the packet and wirelessly transmits the modulated signal to the receiver 614. [ Receiver 614 receives and demodulates the signal and forwards the encoded packet to a channel decoder 616. The channel decoder 616 decodes the packet and provides it to the dequantizer 618 as a decoded packet. The dequantizer 618 dequantizes the information. Information is provided to the sinusoidal sum speech synthesizer 620. [

The sinusoidal sum speech synthesizer 620 is configured to synthesize a plurality of sine waves for modeling the short-term speech spectrum according to the above equation for S [n]. The frequency of the sine wave, f _k, is a multiple or harmonic of the fundamental frequency F ₀ , which is the frequency with pitch periodicity for the pseudo periodic (i.e., transition) voice speech segment.

In addition, the sinusoidal sum speech synthesizer 620 receives the phase information from the phase estimator 622. In addition, phase estimator 622 receives previous frame information, i.e., A ₁ , F ₀ , and V _c parameters for the immediately preceding frame. In addition, phase estimator 622 receives the reproduced N samples of the previous frame, where N is the frame length (i.e., N is the number of samples per frame). The phase estimator 622 determines the initial phase for the frame based on information about the previous frame. The initial phase determination is provided to sinusoidal sum speech synthesizer 620. On the basis of the initial phase calculation performed by the phase estimator 622 based on the information about the current frame and the past frame information, the sinusoidal sum speech synthesizer 620 generates a synthesized speech frame as described below.

As described above, the harmonic coder uses previous frame information and synthesizes or reproduces the speech frame by predicting that the phase will change linearly from frame to frame. In the synthetic model described above, generally referred to as a second phase model, the coefficient B ₃ (k) represents the initial phase for the current voice frame to be synthesized. When determining the phase, the conventional harmonic coder sets the initial phase to zero or generates the initial phase value randomly or in some pseudorandom generation manner. In order to more accurately predict the phase, the phase estimator 622 determines whether the previous frame is a voice voice frame (i.e., a frame with sufficient periodicity) or a transition voice frame, . If the previous frame is a voice speech frame, the last estimated phase value of the frame is used as the initial phase value of the current frame. On the other hand, if the previous frame is classified as a transition frame, the initial phase value for the current frame is obtained from the spectrum of the previous frame obtained by performing the DFT of the decoder output for the previous frame. Thus, the phase estimator 622 uses the precise phase information already available (since the previous frame, which is a transition frame, was processed at full rate).

In one embodiment, the closed-loop, multimode, MDLP speech coder follows the speech processing steps shown in the flow chart of Fig. The voice coder encodes the LP residual components of each input voice frame by selecting the optimal encoding mode. Some modes encode the LP residual or negative residual in the time domain while other modes show the LP residual or negative residual in the frequency domain. The set of modes is a full rate, time domain (T mode) for transition frames; Half rate, frequency domain (V mode) for voice frames; Quarter rate, time domain (U mode) for unvoiced frames; And a time domain (N mode) for a 1/8 rate, noise frame.

It is possible to encode the speech signal or the corresponding LP residual component according to the steps shown in FIG. The waveform characteristics of noise, unvoiced, transition and voice speech are as shown by the time function in the graph of FIG. 10A. The noise, unvoiced, transition, and voice LP residual components are as a function of time in the graph of FIG. 10B.

In step 700, the open loop mode determination is made for one of the four modes (T, V, U, or N) that apply to the input speech residual component S (n). When applying the T mode, the audio residual component is processed in the T-mode, i.e., the full rate, in the time domain at step 702. [ When the U mode is applied, the audio residual component is processed in the time domain in the U-mode, that is, the 1/4 rate in step 704. In case of applying the N mode, the audio residual component is processed in N mode in the time domain at step 706, that is, 1/8 rate. When applying the V mode, the audio residual component is processed in step 708 in the frequency domain in V mode, i.e., a half rate.

In step 710, the speech encoded in step 708 is decoded and compared with the input speech residual component S (n) to calculate the performance measure D. In step 712, the performance measure D is compared to a predetermined threshold T. If the performance measure D is greater than or equal to the threshold T, the spectrally encoded voice residual component of step 708 is acknowledged in step 714. Alternatively, if the performance measure D is less than the threshold T, the input speech residual component S (n) is processed in the T mode in step 716. [ In another embodiment, no performance measure is computed and no threshold is defined. Instead, after processing a predetermined number of remaining audio component frames in the V mode, the next frame is processed in the T mode.

Preferably, the decision steps shown in Fig. 9 allow high bit rate T mode to be used only when necessary, so that by switching at full rate when not properly performing the V mode, Rate V mode to utilize the periodicity of the voice voice segment. Thus, a very high quality sound that reaches full-rate sound quality can be generated at a significantly lower average rate than the full rate. In addition, the target speech quality can be controlled by the selected performance measure and the selected threshold.

In addition, " update " to the T mode keeps the model phase track close to the phase track of the input voice, thereby improving the performance of the subsequent application of the V mode. When performance in the V mode is inadequate, the closed loop performance check at steps 710 and 712 switches to T mode to " refresh " the initial phase value to improve the performance of subsequent V mode processing, Lt; / RTI > to approach this initial input speech phase track again. For example, as shown in the graphs of Figures 11A-C, the fifth frame from the start does not work properly in the V-mode, as evidenced by the PSNR distortion measurements used. As a result, there is no closed loop determination and update, so that the phase track modeled as shown in Fig. 11C deviates significantly from the original input voice phase track and is severely degraded at PSNR. In addition, the performance of the subsequent frame processed in the V mode also deteriorates. However, under the closed loop determination, the fifth frame is switched to T mode processing as shown in FIG. 11A. As shown in Figure 11B, the performance of subsequent frames processed under the V mode is also improved, as evidenced by the improvement in PSNR. Also, the performance of the subsequent frame processed in the V mode is improved.

The decision step shown in FIG. 9 provides a very accurate initial phase estimate to improve the quality of the V-mode representation so that the enhanced V-mode synthesized speech residual signal is precisely time aligned with the original input speech residual, S (n). The initial phase for the first V-processed voice residual segment is derived from the immediately preceding decoded frame in the following manner. For each harmonic, if the previous frame is processed under V mode, the initial phase is set equal to the last estimated phase of the previous frame. For each harmonic, if the previous frame is processed in T mode, the initial phase is set equal to the actual harmonic phase of the previous frame. The actual harmonic phase of the previous frame can be derived by taking the DFT of the residual component that was previously decoded using the previous full frame. Alternatively, the actual harmonic phase of the previous frame can be derived by taking the DFT of the past decoded frames in a pitch synchronization manner by processing the previous frame of various pitch periods.

A new closed loop, multimode, mixed region linear prediction (MDLP) speech coder has been described. The various illustrative logical blocks and algorithm steps described in connection with the present embodiment may be implemented as a digital signal processor (DSP), an application specific integrated circuit (ASIC), a discrete gate or transistor logic, discrete hardware components such as registers and FIFOs, A processor executing firmware instructions, or any conventional programmable software module and processor. Preferably, the processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The software module may be a RAM memory, a flash memory, a register, or any other type of recordable storage medium known in the art. Preferably, data, instructions, commands, information, signals, bits, symbols, and chips may be represented throughout the description by voltage, current, electromagnetic waves, magnetic fields or particles, optical fields or particles, .

The preferred embodiments of the present invention have been shown and described. However, various modifications may be made to the embodiments disclosed herein without departing from the spirit and scope of the invention. Accordingly, the invention is not limited except as by the following claims.

Claims (26)

In a multimode, mixed region, speech processor, A coder having at least one time domain coding mode and at least one frequency domain coding mode; And a closed loop mode selection device coupled to the coder and configured to select a coding mode for the coder based on the content of the frame processed by the speech processor. The method according to claim 1, And wherein the coder encodes the speech frames. The method according to claim 1, And wherein the coder encodes a linear prediction residual component of the speech frame. The method according to claim 1, Wherein the one or more time domain coding modes include a coding mode for coding frames at a first coding rate, Wherein the at least one frequency-domain coding mode includes a coding mode for coding frames at a second coding rate, And the second coding mode is smaller than the first coding mode. The method according to claim 1, Wherein the at least one frequency-domain coding mode comprises a harmonic coding mode. The method according to claim 1, A comparison circuit coupled to the coder and comparing the uncoded frame with the coded frame in one or more frequency domain coding modes and generating a performance measure based on the comparison, The coder applies one or more time-domain coding modes only if the performance measure falls below a predetermined threshold, and otherwise applies one or more frequency-domain coding modes. The method according to claim 1, Wherein the coder applies one or more time-domain coding modes to each of the frames immediately following a predetermined number of consecutive processed frames coded in one or more frequency-domain coding modes. The method according to claim 1, The at least one frequency-domain coding mode expresses the short-term spectrum of each frame with a plurality of sinusoids having a parameter set including frequency, phase, and amplitude, The phase is modeled as a polynomial representation and an initial phase value, The initial phase values are (1) the last estimated phase value of the previous frame if the previous frame is coded in one or more frequency-domain coding modes, or (2) the estimated phase value of the previous frame if the previous frame is coded in more than one time- Is one of the phase values derived from the short-term spectrum. 9. The method of claim 8, Wherein the sinusoidal frequency for each frame is an integral multiple of the pitch frequency of the frame. 9. The method of claim 8, Wherein the sinusoidal frequency is taken from a set of real numbers between 0 and < RTI ID = 0.0 > 2. ≪ / RTI > In the frame processing method, Applying an open loop coding mode selection process to each successive input frame to select one of a time domain coding mode or a frequency domain coding mode based on the audio content of the input frame; Frequency-domain coding an input frame if the audio content of the input frame represents a steady state voice; Time-domain coding an input frame if the audio content of the input frame indicates something other than the steady-state voice voice; Comparing the frequency domain coded frame with an input frame to obtain a performance measurement; And And when the performance measure falls below a predetermined threshold, time-domain coding the input frame. 12. The method of claim 11, Wherein the frame is a linear prediction residual frame. 12. The method of claim 11, Wherein the frame is a voice frame. 12. The method of claim 11, The time domain coding step comprises coding the frame at a first coding rate, The frequency domain coding step includes coding the frame at a second coding rate, And the second coding rate is less than the first coding rate. 12. The method of claim 11, Wherein the frequency domain coding step comprises harmonic coding. 12. The method of claim 11, The frequency domain coding step comprises representing the short-term spectrum of each frame with a plurality of sinusoids having a set of parameters including frequency, phase and amplitude, The phase is modeled as a polynomial representation and an initial phase value, The initial phase values are (1) the last estimated phase value of the previous frame if the previous frame is more than one frequency domain coded, or (2) the phase derived from the short-term spectrum of the previous frame if the previous frame is more than one time- ≪ / RTI > 17. The method of claim 16, Wherein the sinusoidal frequency for each frame is an integral multiple of the pitch frequency of the frame. 17. The method of claim 16, Wherein the sinusoidal frequency for each frame is taken from a set of real numbers between 0 and 2 [pi]. In a multimode, mixed region, speech processor, Means for applying open-loop coding mode selection processing to an input frame to select one of a time-domain coding mode or a frequency-domain coding mode based on the audio content of the input frame; Means for frequency-domain coding an input frame if the audio content of the input frame represents a steady state voice; Means for time-domain coding the input frame if the audio content of the input frame indicates something other than a steady state voice; Means for comparing a frequency domain coded frame with an input frame to obtain a performance measure; And And means for time-domain coding the input frame if the performance measure falls below a predetermined threshold. 20. The method of claim 19, Wherein the input frame is a linear prediction residual frame. 20. The method of claim 19, Wherein the input frame is a voice frame. 20. The method of claim 19, The time domain coding means comprises means for coding the frame at a first coding rate, The frequency domain coding means comprises means for coding the frame at a second coding rate, And the second coding rate is less than the first coding rate. 20. The method of claim 19, Wherein the frequency domain coding means comprises a harmonic coder. 20. The method of claim 19, The frequency domain coding means comprises means for representing the short term spectrum of each frame with a plurality of sinusoids having a parameter set comprising frequency, phase and amplitude, The phase is modeled as a polynomial representation and an initial phase value, The initial phase value is (1) the final estimated phase value of the immediately preceding frame when the previous frame is frequency-domain coded, or (2) the phase value derived from the short-term spectrum of the immediately preceding frame when the previous frame is time- ≪ / RTI > 25. The method of claim 24, Wherein the sinusoidal frequency for each frame is an integral multiple of the pitch frequency of the frame. 25. The method of claim 24, Wherein the sinusoidal frequency for each frame is taken from a set of real numbers between 0 and 2 [pi].