JP2011090311A - Linear prediction voice coder in mixed domain of multimode of closed loop - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method for coding voice in a mixed domain of a multimode of a closed loop. <P>SOLUTION: A voice coder of mixed domain linear prediction (MDLP) of the multimode of the closed loop, includes a high bit rate time domain coding mode, and a low bit rate frequency domain coding mode, and a mode selecting mechanism for selecting a mode based on a voice content of an input frame. A frame of transit voice is coded by a high bit rate code excited linear prediction (CELP) mode. A frame of voiced sound voice is coded by a low bit rate harmonics mode. A phase parameter is modeled by a quasi phase model. An initial phase value becomes the initial phase value of a voice frame just before the frame which is coded by the frequency domain mode, and it is calculated from voice frame information of the frame which is coded in time domain just before that. Each voice frame which is coded in the frequency domain mode is compared with a corresponding input voice frame, and when a performance scale is lower than a predetermined threshold, it is coded by the time domain mode. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates generally to the field of speech processing, and more particularly to a method and apparatus for encoding speech in a closed-loop multi-mode mixed domain.

  The transmission of voice through digital technology has become widespread, especially in long-distance digital radiotelephone applications. This has generated interest in determining the minimum amount of information that can be sent on a channel while maintaining the perceived quality of the reconstructed speech. When voice is simply sampled and sent in digital form, data rates on the order of 64 kilobit seconds per second (kbps) are required to achieve the voice quality of conventional analog telephones. However, the data rate can be significantly reduced by using speech analysis and then properly encoding, transmitting and recombining at the receiver.

  A device that employs a technology for compressing speech by extracting parameters related to a human speech generation model is called a speech coder. The speech coder divides the input speech signal into blocks of time, ie analysis frames. In general, a speech coder includes an encoder and a decoder. The encoder parses the input speech frame, extracts certain relevant parameters, and quantizes the parameters into a binary representation, ie a set of bits or binary data packets. Data packets are sent over a communication channel to a receiver and a decoder. The decoder processes the data packet, unquantizes to generate parameters, and re-synthesizes the speech frame using the unquantized parameters.

The function of the voice coder is to compress the digitized voice signal into a low bit rate signal by removing all of the inherent redundancy that the voice inherently has. Digital compression is implemented by representing an input speech frame with a set of parameters, employing quantization and representing the parameters with a set of bits. When the input speech frame has a number of bits N _i and the data packet generated by the speech coder has a number of bits N ₀ , the compression factor obtained by the speech coder is C _r = N _i / N ₀ . The challenge is to maintain the high voice quality of the decoded speech while obtaining the target compression factor. The performance of a speech coder is: (1) how well the speech model, ie the combination of the above analysis and synthesis processes, is performed, and (2) how much the parameter quantization process is at a target bit rate of N ₀ bits per frame. Depends on proper execution. The speech model is therefore aimed at obtaining the essence of the speech signal, ie the target speech quality, using a small set of parameters for each frame.

  Speech coders employ time-domain coders, ie, advanced time-resolution processing that encodes a small segment of speech (typically a millisecond, ms subframe) at a time. Can be configured as a time domain coder attempting to obtain a time domain speech waveform. For each subframe, a high-precision representative is found from the codebook space by various search algorithms known in the art. Instead, the speech coder may be configured as a frequency domain coder, using a set of parameters to capture (analyze) the short-term speech spectrum of the input speech frame and employ a corresponding synthesis process. Then, try to reproduce the speech waveform from the spectral parameters. The parameter quantizer uses a stored representation of the code vector to parameterize according to known quantization techniques described in the literature (A Gersho & RM Gray, Vector Quantization and Signal Compression (1992)). Save those parameters by representing them.

A well-known time domain speech coder is the Code Excited Linear Predictive (CELP) coder, which is described in the literature by LB Rabiner & RW Schafer (Digital Processing of Speech Signals 396-453 (1978)). This is a general reference here. The CELP coder finds short-term formant filter coefficients by linear prediction (LP) analysis, and removes short-term correlations, ie, redundancy, in the speech signal. A short-term prediction filter is applied to the input speech frame to generate an LP residual signal, which is modeled with a longer-term prediction filter parameter and the following probability codebook: Quantize. Thus, CELP coding divides the task of coding the time-domain speech waveform into separate tasks: the task of coding the short-term filter coefficients of LP and the task of coding the remainder of LP. Time domain coding is used at a fixed rate (ie, a rate that uses the same number of bits N ₀ for each frame) or at a variable rate (ie, different bit rates for different types of frame content). At a rate). A variable rate coder attempts to use only the amount of bits necessary to code the codec parameters to a level suitable to achieve the target quality. An exemplary variable rate CELP coder is described in US Pat. No. 5,414,796, which is assigned to the assignee of the present invention and is generally incorporated herein by reference.

Time domain coders, such as CELP coders, typically rely on a number of bits N ₀ per frame to maintain the accuracy of the time domain speech waveform. Such a coder usually conveys excellent voice quality when the number of bits N ₀ per frame is relatively high (eg, 8 kilobit seconds or more). However, at low bit rates (below 4 kilobit seconds), time domain coders do not maintain high quality and robust performance due to the limited number of available bits. Because the codebook space is limited at low bit rates, it removes the ability to match the waveforms provided by traditional time domain coders and runs such coders in higher rate commercial applications. Succeeded to do.

  Currently, research interest and active commercial demands have increased rapidly, developing high quality speech coders that operate at moderate to low bit rates (ie, in the 2.4 to 4 kilobit second range and below). It was. Applications include wireless telephony, satellite communications, internet telephony, various multimedia and voice streaming applications, voice mail, and other voice storage systems. With respect to driving force, a large capacity is required, and robust performance is required under circumstances where packets are lost. The efforts to standardize various recent speech coding are devoted to another direct driving force that drives the research and development of low-rate speech coding algorithms. A low-rate speech coder generates more channels, or users, for each acceptable application bandwidth, and combines the low-rate speech coder with an additional layer of appropriate channel coding to make the overall coder It can be adapted to the specifications of the correct bit budget, and the channel can be made robust even under the wrong circumstances.

  In order to code at lower bit rates, various methods of coding the speech spectrum, ie in the frequency domain, have been developed, in which the speech signal is a time-varying evolution spectrum. of spectra). For example, see RJ McAulay & TF Quatieri's document (Sinusoidal Coding, in Speech Coding and Synthesis ch. 4 (WB Kleijin & KK Paliwal eds., 1995). Rather, it aims to model or predict the short-term speech spectrum of each input frame of speech using a set of spectral parameters, where the spectral parameters are coded and the speech output frame is decoded. The generated synthesized speech does not match the original input speech waveform, but provides similar perceptual quality.Examples of frequency domain coders well known in the art Includes multiband excitation coder (MBE), sinusoidal transform coder (STC), harmonic coder harmonic coder, including HC). coder of such frequency range, give a high-quality parametric model of having a compact set of parameters that can be accurately quantized with fewer bits available at low bit rates.

  Nevertheless, low bit rate coding adds significant constraints to limited coding resolution, i.e., limited codebook space, limiting the effectiveness of a single coding mechanism, Various types of speech segments cannot be represented under various background conditions of equal precision. For example, conventional low bit rate frequency domain coders do not send audio frame phase information. Instead, the phase information is reconstructed by using random artificially generated initial phase values and a linear interpolation technique. See, for example, H. Yang et al. (Quadratic Phase Interpolation for Voiced Speech Synthesis in the MBE Model, in 29 Electronic Letters 856-57 (May 1993)). Since the phase information is artificially generated, the output speech produced by the frequency domain coder does not match the original input speech even when the sinusoid amplitude is fully preserved by the quantization-dequantization process ( For example, most pulses are not synchronized). Thus, it has been found difficult for frequency domain coders to employ closed-loop performance measures, such as signal-to-noise ratio (SNR) or perceptual SNR.

  Multi-mode coding techniques have been employed to perform low rate speech coding in connection with the open loop mode decision process. One such multimode coding technique is described in Amitava Das, et al. (Multimode and Variable-Rate Coding of Speech, in Speech Coding and Synthesis ch. 7 (WB Kleijin & KK Paliwal eds., 1995)). Has been. Conventional multi-mode coders apply different modes, ie encoding-decoding algorithms, to different types of input speech frames. Each mode, ie the encoding-decoding process, represents a certain type of speech segment such as voiced speech, unvoiced speech, or background noise (nonspeech) in the most efficient manner. Specialized in. An external open loop mode decision mechanism examines the input speech frame to determine which mode to apply to the frame. Typically, open loop mode determination is performed by extracting a number of parameters from the input frame, evaluating parameters related to certain time and spectral characteristics, and based on the mode determination for this evaluation. Therefore, the mode decision is made without knowing in advance how the output speech is extracted, i.e., how close the output speech is to the input speech with respect to speech quality or other performance measure.

  Based on the above, it is desirable to provide a low bit rate frequency domain coder that more accurately estimates phase information. It is further advantageous to provide a multi-mode mixed domain coder that codes certain speech frames in the time domain and other speech frames in the frequency domain based on the speech content of the frame. It is even more desirable to have a mixed domain coder that can code certain speech frames in the time domain and code other speech frames in the frequency domain according to a closed loop coding mode determination mechanism. Therefore, there is a need for a closed-loop, multi-mode mixed domain speech coder that ensures time synchrony between the output speech generated by the coder and the original speech input to the coder.

  The present invention relates to a closed-loop multimode mixed domain speech coder that ensures time synchrony between the output speech generated by the coder and the original speech input to the coder. Accordingly, in one aspect of the present invention, a multi-mode mixed domain speech processor is connected to a coder having at least one time domain coding mode and at least one frequency domain coding mode, and the speech processor. And a closed-loop mode selection device configured to select a coding mode of the coder based on the frame content processed by.

  In another aspect of the present invention, a method of processing a frame applies a open-loop coding mode selection process to each successive input frame to select a time domain coding mode based on the audio content of the input frame, or Selecting one of the frequency domain coding modes, and when the audio content of the input frame indicates a voiced voice in a steady state, the step of encoding the input frame in the frequency domain, and the audio content of the input frame When indicating something other than steady-state voiced speech, encoding the input frame in the time domain, comparing the frequency-coded frame with the input frame, and determining a performance measure; Advantageously, encoding the input frame in the time domain when the performance measure is below a predetermined threshold.

  In another aspect of the present invention, the multi-mode mixed domain speech processor applies an open loop coding mode selection process to the input frame to determine whether the time domain coding mode is based on the speech content of the input frame, or Or means for selecting one of the frequency domain coding modes, and means for encoding the input frame in the frequency domain when the voice content of the input frame indicates a voiced voice in a steady state, and the voice content of the input frame Means that the input frame is coded in the time domain and the frame coded in the frequency domain and the input frame are compared to obtain a performance measure And means for encoding the input frame in the time domain when the performance measure is below a predetermined threshold.

FIG. 3 is a block diagram of a communication channel that is terminated at each end by a voice coder. Block diagram of an encoder that can be used in multi-mode mixed-domain linear prediction (MDLP) speech coders. FIG. 3 is a block diagram of a decoder that can be used in a multi-mode MDLP speech coder. 3 is a flowchart illustrating MDLP encoding steps performed by an MDLP encoder that can be used in the encoder of FIG. 6 is a flowchart showing a voice coding determination process. Block diagram of a closed-loop multi-mode MDLP speech coder. FIG. 7 is a block diagram of a spectral coder that can be used in the coder of FIG. 6 or the encoder of FIG. Amplitude versus frequency graph showing the amplitude of the sinusoid of the harmonic coder. 6 is a flowchart illustrating a mode determination process in a multi-mode MDLP speech coder. A graph of speech signal amplitude versus time (FIG. 10a) and a linear prediction (LP) residual amplitude versus time graph (FIG. 10b). Graph of rate / mode vs. frame index under closed loop coding decision (FIG. 11a), perceptual signal-to-noise ratio (PSNR) vs. frame under closed loop decision Indicator graph (FIG. 11b), both rate / mode and PSNR vs. frame indicator graph without closed loop coding decisions (FIG. 11c).

In FIG. 1, a first encoder 10 receives a digital audio sample s (n), encodes the sample s (n), and transmits it to the first decoder 14 on the transmission medium 12, ie the communication channel 12. send. The decoder 14 decodes the encoded audio sample and synthesizes the output audio signal S _SYNTH (n). For transmission in the opposite direction, the second encoder 16 encodes a digitally formatted audio sample s (n) and sends it over the communication channel 18. The second decoder 20 receives and decodes the encoded audio samples and generates a synthesized output audio signal S _SYNTH (n).

  The audio sample s (n) can be obtained by various methods known in the art, such as pulse code modulation (PMC), the compounded μ-method, ie the A-law (companded μ-law, or A-law). ) Represents a speech signal that has been digitized and quantized according to a method including: As is known in the art, speech samples s (n) are organized into frames of input data, each containing a predetermined number of digitally formatted speech samples s (n). In the exemplary embodiment, a sampling rate of 8 kilohertz is employed and each 20 millisecond frame includes 160 samples. In the separately described embodiment, the data transmission rate is from 8 kilobit seconds (full rate) to 4 kilobit seconds (half rate), 2 kilobit seconds (quarter rate), 1 kilobit second (8 minutes) per frame. It is convenient to change to 1 rate). Instead, other data rates may be used. As used herein, the term “full rate” or “high rate” usually refers to a data rate of 8 kilobit seconds or more, “half rate” or The term “low rate” usually refers to a data rate of 4 kilobit seconds or less. It is advantageous to change the data transmission rate because a lower bit rate can be selectively employed for frames containing relatively little audio information. Other sampling rates, frame sizes, and data transmission rates may be used, as will be appreciated by those skilled in the art.

  Both the first encoder 10 and the second decoder 20 include a first speech coder or speech codec. Similarly, both the second encoder 16 and the first decoder 14 include a second speech coder. Voice coders can be digital signal processors (DSPs), application-specific integrated circuits (ASICs), discrete gate logic, firmware, or traditional programmable software modules and It will be appreciated that it may consist of a microprocessor. The software modules reside in RAM memory, flash memory, registers, or other forms of writable storage media known in the art. Alternatively, a conventional processor, controller, or state machine may be replaced with a microprocessor. An example of an ASIC specifically designed for speech coding is US Pat. No. 5,727,123, assigned to the assignee of the present invention, and hereby fully incorporated by reference, and February 16, 1994. No. 08 / 197,417 (Title of Invention: VOCODER ASIC), filed on the same day, assigned to the assignee of the present invention, and here fully incorporated by reference.

In accordance with one embodiment, a multi-mode mixed-domain linear prediction (MDLP) encoder 100 that can be used in a speech coder, as shown in FIG. It includes an estimation module 104, a linear prediction (LP) analysis module 106, an LP analysis filter 108, an LP quantization module 110, and an MDLP residual encoder 112. The input speech frame s (n) is supplied to the mode determination module 102, the pitch estimation module 104, the LP analysis module 106, and the LP analysis filter 108. The mode determination module 102 generates a mode indicator I _M and mode M based on the periodicity of each input speech frame s (n) and other extraction parameters such as energy, spectral tilt, zero crossing rate, and the like. Various methods for classifying speech frames according to periodicity are described in US patent application Ser. No. 08 / 815,354 (invention name: METHOD AND APPARATUS FOR PERFORMING REDUCED RATE VARIABLE RATE VOCODING), which was published in March 1997. Filed on the 11th and assigned to the assignee of the present invention, which is hereby fully incorporated by reference. Such a method is also used in TIA / EIA IS-127 and TIA / EIA IS-733 of the Telecommunication Industry Association Industry Interim Standards.

  Except for the MDLP residual encoder 112, the operation and configuration of the various modules of the encoder 100 of FIG. 2 and the decoder 200 of FIG. 3 are known in the art and are described in US Pat. No. 5,414,796 and LB. (Digital Processing of Speech Signals 396-453 (1978)).

According to one embodiment, an MDLP encoder (not shown) performs the steps shown in the flowchart of FIG. The MDLP encoder may be the MDLP residual encoder 112 of FIG. In step 300, the MDLP encoder determines whether mode M is full rate (FR), quarter rate (QR), or eighth rate (ER). Check for it. When mode M is FR, QR, or ER, the MDLP encoder proceeds to step 302. In step 302, MDLP encoder (depending on the value of M -FR, QR, or ER,) corresponding rate to apply to the residual index _{I R.} The time-domain coding is advantageously high-precision, high-rate coding in the FR mode and CELP coding, but this time-domain coding is the LP residual frame, or its Instead, it is applied to the audio frame. The frame is then sent (after another signal processing including digital to analog conversion and modulation). In one embodiment, the frame is an LP residual frame that represents the prediction error. In an alternative embodiment, the frame is an audio frame that represents an audio sample.

  On the other hand, at step 300, when mode M is not FR, QR, or ER (ie, when mode M is half rate, HR), the MDLP encoder proceeds to step 304. In step 304, spectral coding, preferably harmonic coding, is applied to the LP residual, or alternatively to the speech signal at a rate of one half. The MDLP encoder then proceeds to step 306. In step 306, a distortion measure D is obtained by decoding the encoded speech and comparing it to the original input frame. The MDLP encoder then proceeds to step 308. In step 308, the strain measure D is compared to a predetermined threshold T. When the distortion measure D is greater than the threshold T, the corresponding quantized parameter is modulated and sent for a half-rate spectrally encoded frame. On the other hand, if the distortion measure D is less than or equal to the threshold T, the MDLP encoder proceeds to step 310. In step 310, the decoded frame is re-encoded at full rate in this time domain. Conventional high-rate and high-precision coding algorithms, such as preferably CELP coding, may be used. Next, the FR mode quantized parameters associated with the frame are modulated and sent.

  As shown in the flowchart of FIG. 5, a closed-loop multi-mode MDLP speech coder then follows a set of steps for processing and sending speech samples according to one embodiment. In step 400, the speech coder receives digital samples of the speech signal in successive frames. Upon receipt of a given frame, the voice coder proceeds to step 402. In step 402, the speech coder detects the energy of the frame. Energy is a measure of the speech activity of the frame. Speech detection is performed by adding the squares of the amplitudes of the speech samples in digital form and comparing the generated energy to a threshold value. In one embodiment, a threshold is employed based on the background noise change level. An exemplary variable threshold voice activity detector is described in the aforementioned US Pat. No. 5,414,796. Some unvoiced speech is a very low energy sample and may be erroneously encoded as background noise. To prevent this from happening, the silent tilt of the spectrum of low energy samples is used to distinguish unvoiced speech from background noise, as described in US Pat. No. 5,414,796 mentioned above.

  After detecting the energy of the frame, the speech coder proceeds to step 404. In step 404, the speech coder determines whether the detected frame energy is sufficient to classify the frame as containing speech information. If the detected frame energy is below a predetermined threshold level, the speech coder proceeds to step 406. In step 406, the voice coder encodes the frame as background noise (ie, non-voice or silence). In one embodiment, the background noise frame is a time domain encoded at 1/8 rate, or 1 kilobit second. In step 404, when the detected frame energy is greater than or equal to a predetermined threshold level, the frame is classified as speech and the speech coder proceeds to step 408.

  In step 408, the speech coder determines whether the frame is periodic. Various known methods of determining periodicity include, for example, the use of zero crossings and the use of normalized autocorrelation functions (NACF). In particular, detection of periodicity using zero crossings and NACF is described in US application Ser. No. 08 / 815,354 (invention: METHOD AND APPARATUS FOR PERFORMING REDUCED RATE VARIABLE RATE VOCODING). Filed on March 11, 1997, assigned to the assignee of the present invention, which is hereby fully incorporated by reference. In addition, the above-described method used to distinguish voiced speech from unvoiced speech is based on TIA / EIA IS-127 of the Telecommunication Industry Association Industry Interim Standards and TIA / EIA IS-127. Used in TIA / EIA IS-733. When it is determined at step 408 that the frame is not periodic, the speech coder proceeds to step 410. In step 410, the speech coder encodes the frame as unvoiced speech. In one embodiment, the voice frame of unvoiced sound is a time domain encoded at a quarter rate, ie, 2 kilobit seconds. In step 408, when the frame is determined to be periodic, the speech coder proceeds to step 412.

  In step 412, the speech coder uses a periodicity detection method known in the art, for example as described in the above-mentioned US patent application Ser. No. 08 / 815,354, to ensure that the frame is sufficiently periodic. Determine if there is. If it is determined that the frame is not sufficiently periodic, the speech coder proceeds to step 414. In step 414, the frame is coded in the time domain as transition speech (ie, transition from unvoiced to voiced speech). In one embodiment, the transitional speech frames are encoded in the time domain at full rate, ie 8 kilobit seconds.

  If the voice coder determines in step 412 that the frame is sufficiently periodic, it proceeds to step 416. In step 416, the speech coder encodes the frame as voiced speech. In one embodiment, voiced speech frames are spectrally encoded, especially at a half rate, ie 4 kilobit seconds. As described separately with reference to FIG. 7, a voiced speech frame is advantageously spectrally encoded with a harmonic coder. Instead, other spectral coders can advantageously be used, for example as a sinusoidal transmission coder or a multiband excitation coder, as is known in the art. The voice coder then proceeds to step 418. In step 418, the speech coder decodes the encoded voiced speech frame. The voice coder then proceeds to step 420. In step 420, the decoded voice frame of the voiced sound is compared with the corresponding input voice sample of the frame to obtain a distortion measure of the synthesized voice to obtain a half-rate voiced voice spectral code. Determine whether the optimization model is operating within acceptable limits. The voice coder then proceeds to step 422.

  In step 422, the speech coder determines whether the error between the decoded voice frame of voiced sound and the input speech frame corresponding to this frame is less than a predetermined threshold. In one embodiment, this determination is made in the manner described separately with reference to FIG. If the coding distortion is below a predetermined threshold, the speech coder proceeds to step 426. In step 426, the speech coder uses the parameters of step 416 to send the frame as voiced speech. At step 422, if the coding distortion is greater than or equal to a predetermined threshold, the speech coder proceeds to step 414 and encodes the frame of the digital speech sample received at step 400 as a transition speech at full rate in the time domain. .

  Note that steps 400 through 410 include an open loop coding decision mode. On the other hand, steps 412 to 426 include a closed loop coding decision mode.

  In one embodiment, as shown in FIG. 6, the closed-loop multi-mode MDLP speech coder includes an analog-to-digital converter (A / D) 500, where the A / D 500 is a frame buffer. The frame buffer 502 is connected to the control processor 504. Energy calculator 506, voiced speech detector 508, background noise encoder 510, high rate time domain encoder 512, and low rate spectral encoder 514 are connected to control processor 504. The spectrum decoder 516 is connected to the spectrum encoder 514, and the error calculator 518 is connected to the spectrum decoder 516 and the control processor 504. The threshold comparator 520 is connected to the error calculator 518 and the control processor 504. Buffer 522 is connected to spectrum encoder 514, spectrum decoder 516, and threshold comparator 520.

  In the embodiment of FIG. 6, the components of the voice coder are conveniently configured as firmware or other software-driven modules within the voice coder, and the voice coder itself may be within a DSP or ASIC. Convenient. Those skilled in the art will appreciate that the components of the speech coder can be similarly configured in a number of other known ways as well. The control processor 504 is conveniently a microprocessor, but may be configured with a controller, state machine, or discrete logic.

In the multi-mode coder of FIG. 6, the audio signal is supplied to the A / D 500. The A / D 500 converts an analog signal into a digital audio sample S (n). The audio samples in digital form are supplied to the frame buffer 502. The control processor 504 obtains digital audio samples from the frame buffer 502 and provides them to the energy calculator 506. The energy calculator 506 calculates the energy E of the speech sample according to the following formula:

      The frame is 20 milliseconds long and the sampling rate is 8 kilohertz. The calculated energy E is sent to the control processor 504.

  The control processor 504 compares the calculated speech energy with a speech activity threshold. When the calculated energy is less than the voice activity threshold, the control processor 504 sends digitally formatted voice samples from the frame buffer 502 to the background noise encoder 510. Background noise encoder 510 encodes the frame using the minimum number of bits necessary to hold an estimate of background noise.

  When the calculated energy is greater than or equal to the voice activity threshold, the control processor 504 directs the digital audio sample from the frame buffer 502 to the voiced audio detector 508. Voiced speech detector 508 determines whether the periodicity of the speech frame allows efficient coding using low bit rate spectral coding. Methods for determining the level of periodicity within a speech frame are well known in the art and include, for example, the use of normalized autocorrelation functions (NACF) and zero crossings. These and other methods are described in the aforementioned US patent application Ser. No. 08 / 815,354.

  Voiced speech detector 508 provides a signal to control processor 504 that indicates whether the speech frame contains speech with sufficient periodicity for spectral encoder 514 to efficiently encode. When the voiced speech detector 508 determines that the speech frame lacks sufficient periodicity, the control processor 504 directs the digitally formatted speech samples to the high rate encoder 512, which in turn has a predetermined maximum Encode speech in time domain at data rate. In one embodiment, the predetermined maximum data rate is 8 kilobit seconds and the high rate encoder 512 is a CELP coder.

When voiced speech detector 508 first determines that the speech signal has sufficient periodicity for spectral encoder 514 to efficiently encode, control processor 504 may transmit spectral encoder 514 from frame buffer 502. Direct audio samples in digital form. An exemplary spectral encoder is described in further detail with reference to FIG.

  When the calculated MSE is within an acceptable range, the threshold comparator 520 provides a signal to the buffer 522 and the spectrally encoded data is output from the speech coder. On the other hand, if the MSE is not within acceptable limits, the threshold comparator 520 sends a signal to the control processor 504 which directs the digital format samples from the frame buffer 502 to the high rate time domain encoder 512. . The time domain encoder 512 encodes the frame at a predetermined maximum rate, and the contents of the buffer 522 are discarded.

  In the embodiment of FIG. 6, the type of spectral encoding employed is harmonic encoding, which will be described separately with reference to FIG. 7, but in an alternative embodiment, encoding of the sinusoid transform Or it may be a type of spectral coding, such as multi-band excitation coding. The use of multiband excitation coding is described in US Pat. No. 5,195,166, and the use of sinusoidal transform coding is described, for example, in US Pat. No. 4,865,068.

  For transitional frames and voiced frames whose phase distortion threshold is less than or equal to the periodicity parameter, the multimode coder of FIG. 6 employs CELP coding with a high rate time domain coder 512 at full rate, ie 8 kbps. Convenient to do. Instead, other known forms of high rate time domain coding may be used for such frames. Thus, transition frames (and voiced frames that are not sufficiently periodic) are coded with high accuracy, waveforms at the input and output are properly matched, and phase information is properly maintained. In one embodiment, the multimode coder, after processing a predetermined number of consecutive voiced frames whose threshold exceeds the periodicity measure, 2 for each frame, regardless of the decision of the threshold comparator 520. Switch from one-rate spectral coding to full-rate CELP coding.

  It should be noted that in connection with control processor 504, energy calculator 506 and voiced speech detector 508 include open loop coding decisions. In contrast, in connection with control processor 504, spectral encoder 514, spectral decoder 516, error calculator 518, threshold comparator 520, and buffer 522 include closed-loop coding decisions.

  In one embodiment described with reference to FIG. 7, spectral encoding, preferably harmonic encoding, is used to encode a sufficiently periodic voiced sound frame at a low bit rate. Spectral coders generally maintain time-evolution over time of speech spectral characteristics in a perceptually important manner by modeling and coding each speech frame in the frequency domain. Specified as an algorithm that tries to The essential parts of such an algorithm are (1) spectrum analysis or parameter estimation, (2) parameter quantization, and (3) synthesis of the output speech waveform and decoded parameters. Accordingly, it is intended to preserve the important characteristics of the short-term speech spectrum with a set of spectral parameters and to synthesize the output speech using the decoded spectral parameters. Normally, the output speech is synthesized as a weighted sum of sinusoids. The sinusoid amplitude, frequency, and phase are spectral parameters estimated during analysis.

  “Analysis by synthesis” is a well-known technique in CELP coding, but this technique has not been used for spectral coding. The main reason that synthesis analysis does not apply to spectrum coders is that the loss of the initial phase information causes the mean square energy, even if the speech model is functioning properly from a perceptual point of view. This is because MSE) is high. Thus, generating the initial phase correctly has another advantage in that it can directly compare the speech samples and the reconstructed speech to determine whether the speech model accurately encodes the speech frame.

In spectral coding, the output speech frames can be synthesized as follows: S [n] = S _v [n] + S _uv [n], n = 1, 2,. . . , N, Incidentally, N is the number of samples per frame, _{S v} and _{S uv} are respectively voiced component and unvoiced components. The sum-of-sinusoid synthesis process generates a voiced sound component as shown in the following equation:

Amplitude, frequency, and phase parameters are estimated from the short-term spectrum of the input frame by a spectral analysis process. Unvoiced components, either generated along with the voiced part in a single sinusoid sum synthetic or computed separately by a dedicated unvoiced synthesis process is added back to S _v.

In the embodiment of FIG. 7, a particular type of spectral coder called a harmonic coder is used to spectrally encode a sufficiently periodic voiced sound frame at a low bit rate. The harmonic coder characterizes the frame as a sinusoidal sum and analyzes small segments of the frame. Each sinusoid in the sinusoidal sum has a frequency of an integral multiple of the pitch F ₀ of the frame. In an alternative embodiment, a specific type of spectral coder other than a harmonic coder is used, and the sinusoid frequency for each frame is derived from a set of real numbers from 0 to 2π. In the embodiment of FIG. 7, it is convenient that the amplitude and phase of each sinusoid in the sum is selected, so that the sum is best matched to the signal in one period, as shown by the graph of FIG. To do. Harmonic coders typically employ external classification, and each input speech frame is displayed as voiced or unvoiced. In a voiced sound frame, the sinusoid frequency is limited to harmonics of the estimated pitch (F ₀ ), ie f _k = kF ₀ . In unvoiced speech, short-term spectral peaks are used to determine sinusoids. The amplitude and phase are interpolated to mimic the gradual change in the frame, as shown in the following equation:

  The parameters sent for each sinusoid are amplitude and frequency. The phase is not sent, but instead is modeled according to several known techniques including, for example, a quadratic phase model, or a conventional polynomial representation of the phase.

As shown in FIG. 7, the harmonic coder includes a pitch extractor 600 that is connected to windowing logic 602, which is a discrete Fourier transform (DFT), And to harmonic analysis logic 604. The pitch extractor 600 that receives the audio sample S (n) as input is also connected to the DFT and harmonic analysis logic 604. DFT and harmonic analysis logic 604 is connected to residual encoder 606. Pitch extractor 600, DFT and harmonic analysis logic 604, and residual encoder 606 are connected to parameter quantizer 608, respectively. Parameter quantizer 608 is connected to channel encoder 610, which is connected to transmitter 612. Transmitter 612 receives over an over-the-air interface with a standard radio-frequency (RF) interface, eg, code division multiple access (CDMA). Connected to machine 614. Receiver 614 is connected to channel decoder 616, which is connected to dequantizer 618. Dequantizer 618 is connected to sinusoidal sum speech synthesizer 620. Further connected to the sinusoidal sum speech synthesizer 620 is a phase estimator 622, which receives the previous frame information as an input. The sinusoidal sum speech synthesizer 620 is configured to generate a synthesized speech output S _SYNTH (n).

  Pitch extractor 600, windowing logic 602, DTF and harmonic analysis logic 604, residual encoder 606, parameter quantizer 608, channel encoder 610, channel decoder 616, non-quantizer 618, sinusoidal sum speech synthesizer 620, and The phase estimator 622 can be configured in a variety of different ways familiar to those skilled in the art including, for example, firmware or software modules. Transmitter 612 and receiver 614 may be implemented with corresponding standard RF components known to those skilled in the art.

The harmonic coder of FIG. 7, input samples S (n) is received by the pitch extractor 600, the pitch extractor 600 extracts a pitch frequency information F _0. The samples are then multiplied by an appropriate windowing function by windowing logic 602 to allow analysis of small segments of speech frames. Using the pitch information provided by the pitch extractor 600, the DFT and harmonic analysis logic 604 calculates the DFT of the sample to generate composite spectral points from which the graph of FIG. As shown in FIG. 8, the harmonic amplitude A _I is extracted, and in FIG. 8, L indicates the total number of harmonics. DFT is fed to the residual encoder 606, the residual encoder 606 extracts the audio information (voicing information) V _c.

The V _c parameter indicates a point on the frequency axis as shown in FIG. 8, and note that as V _c becomes higher, the spectrum is characteristic of an unvoiced speech signal and is no longer a harmonic. Should. In contrast, below the point V _c , the spectrum is harmonic and exhibits voiced voice characteristics.

The components of A _I , F ₀ , and V _c are supplied to the parameter quantizer 608, which quantizes the information. The quantized information is supplied to the channel encoder 610 in the form of a packet, and the channel encoder 610 quantizes the packet at, for example, a half rate, that is, a low bit rate such as 4 kilobit seconds. The packet is provided to transmitter 612, which modulates the packet and sends the generated signal to the receiver 614 over the air. Receiver 614 receives the signal, demodulates it, and sends the encoded packet to channel decoder 616. The channel decoder 616 decodes the packet and supplies the decoded packet to the non-quantizer 618. Dequantizer 618 dequantizes the information. The information is supplied to the sinusoidal sum speech synthesizer 620.

The sinusoidal sum speech synthesizer 620 is configured to synthesize a plurality of sinusoidal modeling that models the short-term speech spectrum according to the above equation for S [n]. The frequency of the sinusoid f _k is a multiple or harmonic of the fundamental frequency F ₀ and is a frequency with a periodicity of pitch relative to the quasi-periodic (ie, transitional) voiced speech segment.

In addition, the sinusoidal sum speech synthesizer 620 receives phase information from the phase estimator 622. Phase estimator 622 receives the previous frame information, ie, the parameters of A _I , F ₀ , and V _c for the previous frame. Phase estimator 622 also receives N reconstructed samples of the previous frame, where N is the frame length (ie, N is the number of samples per frame). The phase estimator 622 determines the initial phase of the frame based on the information of the previous frame. The determination of the initial phase is supplied to the sinusoidal sum speech synthesizer 620. Based on the information about the current frame and the calculation of the initial phase performed by the phase estimator 622 based on the past frame information, the sinusoidal sum speech synthesizer 620 generates a speech frame as described above.

As already described, the harmonic coder uses the previous frame information to synthesize or reconstruct the speech frame by predicting that the phase will change linearly from frame to frame. The above synthesis model is generally called a quasi-phase model, and in such a synthesis model, the coefficient B ₃ (k) indicates that the initial phase of the current voiced sound frame is synthesized. Yes. When determining the phase, a conventional harmonic coder sets the initial phase to zero, or generates the initial phase value randomly or using a pseudo-random generation method. To more accurately predict the phase, the phase estimator 622 depends on whether the previous frame is a voiced speech frame (ie, a sufficiently periodic frame) or a transitional speech frame. Use one of two possible methods to determine the initial phase. When the previous frame is a voiced voice frame, the estimated final phase value of this frame is used as the initial phase value of the current frame. On the other hand, when the previous frame is classified as a transition frame, the initial phase value of the current frame is obtained from the spectrum of the previous frame, which is obtained by performing a DFT of the decoder output of the previous frame. Thus, phase estimator 622 can use accurate phase information that is already available (since the previous frame, which is a transition frame, has been processed at full rate).

  In one embodiment, a closed loop multi-mode MDLP speech coder follows the speech processing steps shown in the flowchart of FIG. The speech coder encodes the LP remainder of each input speech frame by selecting the most appropriate coding mode. Certain modes encode LP residuals, i.e. speech residuals, in the time domain, while other modes represent LP residuals, i.e. speech residuals, in the frequency domain. Mode set includes full rate time domain for transition frames (T mode); half rate frequency domain for voiced frames (V mode); quarter rate time domain for unvoiced frames (U mode) And there is a 1/8 rate time domain (N mode) for noise frames.

  Those skilled in the art will appreciate that following the steps shown in FIG. 9 encodes the speech signal or the corresponding LP residue. The waveform characteristics of noise, unvoiced sounds, transitions, and voiced sounds can be referenced as a function of time in the graph of FIG. 10a. The residual waveform characteristics of the noise, unvoiced, transition, and voiced LPs can be referenced as a function of time in the graph of FIG. 10b.

  In step 700, an open-loop mode decision is made for any one of the four modes (T, V, U, or N) and applied to the residual S (n) of the input speech. When T mode is applied, in step 702, the residual of speech is processed in the time mode in T mode, i.e., full rate. When the U mode is applied, at step 704, the remainder of the speech is processed in the time domain in the U mode, ie, a quarter rate. When the N mode is applied, in step 706, the remainder of the speech is processed in the N mode, ie 1/8 rate, in the time domain. When the V mode is applied, in step 708, the residual audio is processed in the V mode in the frequency domain, i.e. at a half rate.

  In step 710, the speech encoded in step 708 is decoded and compared with the residual S (n) of the input speech and a performance measure D is calculated. In step 712, the performance measure D is compared to a predetermined threshold T. If the performance measure D is greater than or equal to the threshold T, then in step 714, the remainder of the spectrally encoded speech in step 708 is allowed to be transmitted. On the other hand, when the performance measure D is smaller than the threshold T, in step 716, the residual S (n) of the input speech is processed in the T mode. In another embodiment, no performance measure is calculated and no threshold is defined. Instead, after a predetermined number of audio residual frames have been processed in V mode, the next frame is processed in T mode.

  In the decision step shown in FIG. 9, the high bit rate T mode can be used only when needed to take advantage of the periodicity of voiced speech segments in the lower bit rate V mode, while When V mode is not performed properly, it is advantageous to prevent quality degradation by switching to full rate. Thus, very high voice quality approaching full rate voice quality can be generated at an average rate substantially lower than the full rate. Furthermore, the target voice quality can be controlled by the selected performance measure and the selected threshold.

  “Updating” to the T mode can improve the operation of applying the V mode later by keeping the model phase tracking close to the phase tracking of the input speech. When the V-mode performance is inadequate, the closed-loop performance check in steps 710 and 712 switches to T-mode and “refreshes” the initial phase value so that the model phase tracking becomes the original input speech phase tracking. By approaching again, the processing performance of the next V mode can be improved. For example, as shown in the graphs of FIGS. 11a-c, the fifth frame from the start does not work properly in V mode, as evidenced by the PSNR distortion measure being used. As a result, when there is no closed-loop determination and update, the modeled phase tracking deviates significantly from the original input speech phase tracking, and degrades the PSNR considerably, as shown in FIG. 11c. Furthermore, the performance of the next frame processed in the V mode is degraded. However, under the closed loop decision, the fifth frame is switched to T-mode processing, as shown in FIG. 11a. The performance of the fifth frame is significantly improved by updating, as demonstrated by the improvement in PSNR, as shown in FIG. 11b. In addition, the performance of the next frame processed under the V mode is also improved.

  The decision step shown in FIG. 9 improves the V-mode representation quality by giving a very accurate initial phase estimate, and the generated V-mode synthesized speech residual signal is the original input speech. It is guaranteed to be exactly in time alignment with the remaining S (n). The initial phase in the remaining segment of speech processed in the first V mode is determined from the previous decoded frame in the following manner. At each harmonic, when the previous frame is processed in V mode, the initial phase is set equal to the estimated final phase of the previous frame. For each harmonic, when the previous frame is processed in T mode, the initial phase is set equal to the phase of the actual harmonic of the previous frame. The phase of the actual harmonics of the previous frame is determined by taking the past decoded residual DFT using all previous frames. Instead, the phase of the actual harmonics of the previous frame is determined by taking the DFT of the previous decoded frame in a pitch-synchronized manner by processing the various pitch periods of the previous frame.

  The present description describes a novel closed-loop multi-mode mixed-domain linear prediction (MDLP) speech coder. Those skilled in the art will recognize that the various exemplary logic blocks and algorithm steps described in connection with the embodiments disclosed herein are digital signal processors (DSPs), application specific integrated circuits. circuit, ASIC), discrete gate or transistor logic, eg, discrete hardware components such as registers and FIFOs, a processor that executes a set of firmware instructions, or a conventional programmable software module and processor. You will see that you can. The processor is conveniently a microprocessor, but may alternatively be a conventional processor, controller, microprocessor, or state machine. The software modules may be in RAM memory, flash memory, registers, or other forms of writable storage media known in the art. Those skilled in the art will further understand that the data, instructions, commands, information, signals, bits, symbols, and chips referred to throughout the above description are voltage, current, electromagnetic, magnetic or magnetic particles, optical ranges or particles. or particles), or a combination thereof, will be conveniently represented.

  Herein, preferred embodiments of the present invention have been shown and described. However, one of ordinary skill in the art appreciates that many modifications can be made to the embodiments described herein without departing from the spirit or scope of the invention. Accordingly, the invention is not limited except as according to the claims.

Claims (26)

A coder having at least one time domain coding mode and at least one frequency domain coding mode;
A multi-mode mixed domain speech processor, comprising: a closed-loop mode selection device connected to the coder and configured to select a coding mode of the coder based on frame content processed by the speech processor. The speech processor of claim 1, wherein the coder encodes speech frames. The speech processor of claim 1, wherein the coder encodes a linear prediction residual of the speech frame. At least one time domain coding mode includes a coding mode that encodes a frame at a first coding rate, and at least one frequency domain coding mode encodes a frame at a second coding rate. The speech processor of claim 1, including a coding mode, wherein the second coding rate is lower than the first coding rate. The speech processor of claim 1, wherein the at least one frequency domain coding mode comprises a harmonic coding mode. A comparison circuit connected to the coder for comparing an uncoded frame with a frame coded in at least one frequency domain coding mode and generating a performance measure based on the comparison. Further comprising: the coder applies at least one time-domain coding mode only when the performance measure is below a predetermined threshold, otherwise the coder applies at least one frequency-domain coding mode. The voice processor described. The coder applies at least one time domain coding mode to each frame immediately following a predetermined number of consecutively processed frames coded in at least one frequency domain coding mode. Voice processor. At least one frequency domain coding mode represents a short-term spectrum of each frame with a plurality of sinusoids with a set of parameters including frequency, phase, and amplitude, where the phase is modeled with a polynomial representation and an initial phase value. And the initial phase value is (1) the estimated final phase value of the previous frame when the previous frame was coded in at least one frequency domain coding mode, or (2) the previous frame 2. A speech processor according to claim 1, wherein when coded in at least one time domain coding mode, is a phase value determined from a short-term spectrum of the previous frame. 9. A speech processor according to claim 8, wherein the frequency of the sinusoid in each frame is an integral multiple of the pitch frequency of the frame. 9. A speech processor according to claim 8, wherein the frequency of the sinusoid in each frame is obtained from a set of real numbers from 0 to 2π. A method of processing a frame,
Applying an open-loop coding mode selection process to each successive input frame to select either a time-domain coding mode or a frequency-domain coding mode based on the audio content of the input frame;
Encoding the input frame in the frequency domain when the audio content of the input frame indicates a voiced voice in a steady state;
Encoding the input frame in the time domain when the audio content of the input frame indicates something other than steady-state voiced speech;
Comparing the frequency-coded frame with the input frame to determine a performance measure;
Encoding the input frame in the time domain when the performance measure is below a predetermined threshold. The method of claim 11, wherein the frame is a linear prediction residual frame. The method of claim 11, wherein the frame is an audio frame. Encoding in the time domain includes encoding the frame at a first encoding rate, and encoding in the frequency domain includes encoding the frame at a second encoding rate; The method of claim 11, wherein the second coding rate is lower than the first coding rate. The method of claim 11, wherein encoding in the frequency domain comprises encoding with harmonics. The step of coding in the frequency domain represents the short-term spectrum of each frame with multiple sinusoids with a set of parameters including frequency, phase and amplitude, the phase being modeled with a polynomial representation and initial phase values The initial phase value is (1) when the previous frame is coded in the frequency domain, or is the estimated final phase value of the previous frame, or (2) the previous frame is coded in the time domain The method according to claim 11, wherein the phase value is obtained from a short-term spectrum of the previous frame. The method of claim 16, wherein the sinusoid frequency of each frame is an integer multiple of the pitch frequency of the frame. The method of claim 16, wherein the sinusoid frequency of each frame is obtained from a set of real numbers from 0 to 2π. A multi-mode mixed-domain audio processor,
Means for applying an open-loop coding mode selection process to the input frame to select either a time-domain coding mode or a frequency-domain coding mode based on the audio content of the input frame;
Means for encoding the input frame in the frequency domain when the audio content of the input frame indicates a voiced voice in a steady state;
Means for encoding the input frame in the time domain when the audio content of the input frame indicates something other than a steady-state voiced sound;
Means for comparing a frame coded in the frequency domain with an input frame to obtain a performance measure;
A multi-mode mixed domain speech processor including means for encoding the input frame in the time domain when the performance measure is below a predetermined threshold. The speech processor of claim 19, wherein the frame is a linear prediction residual frame. The speech processor of claim 19, wherein the input frame is a speech frame. Means for encoding in the time domain includes means for encoding the frame at a first encoding rate, and means for encoding in the frequency domain includes means for encoding the frame at a second encoding rate; The speech processor of claim 19, wherein the second coding rate is lower than the first coding rate. The speech processor of claim 19, wherein the means for encoding in the frequency domain comprises a harmonic coder. Means for encoding in the frequency domain includes means for representing a short-term spectrum of each frame with a plurality of sinusoids having a set of parameters including frequency, phase and amplitude, wherein the phase is modeled with a polynomial representation and an initial phase value And the initial phase value is (1) when the immediately preceding frame is coded in the frequency domain, or is the estimated final phase value of the immediately preceding frame, or (2) the immediately preceding frame is time 20. A speech processor according to claim 19, wherein when coded in a region, the speech value is a phase value determined from a short-term spectrum of the immediately preceding frame. The speech processor of claim 24, wherein the sinusoid frequency of each frame is an integer multiple of the pitch frequency of the frame. 25. A speech processor according to claim 24, wherein the sinusoidal frequency of each frame is obtained from a set of real numbers from 0 to 2π.

Images