KR100711047B1 - Closed-loop multimode mixed-domain linear prediction speech coder - Google Patents

Abstract

A closed-loop, multimode, mixed-domain linear prediction (MDLP) speech coder includes a high-rate, time-domain coding mode, a low-rate, frequency-domain coding mode, and a closed-loop mode-selection mechanism for selecting a coding mode for the coder based upon the speech content of frames input to the coder. Transition speech (i.e., from unvoiced speech to voiced speech, or vice versa) frames are encoded with the high-rate, time-domain coding mode, which may be a CELP coding mode. Voiced speech frames are encoded with the low-rate, frequency-domain coding mode, which may be a harmonic coding mode. Phase parameters are not encoded by the frequency-domain coding mode, and are instead modeled in accordance with, e.g., a quadratic phase model. For each speech frame encoded with the frequency-domain coding mode, the initial phase value is taken to be the initial phase value of the immediately preceding speech frame encoded with the frequency-domain coding mode. If the immediately preceding speech frame was encoded with the time-domain coding mode, the initial phase value of the current speech frame is computed from the decoded speech frame information of the immediately preceding, time-domain-encoded speech frame. Each speech frame encoded with the frequency-domain coding mode may be compared with the corresponding input speech frame to obtain a performance measure. If the performance measure falls below a predefined threshold value, the input speech frame is encoded with the time-domain coding mode.

Description

Closed-loop multimode mixed region linear prediction (MDD) voice coder {CLOSED-LOOP MULTIMODE MIXED-DOMAIN LINEAR PREDICTION SPEECH CODER}

In general, the present invention relates to speech processing, and more particularly, to closed loop, multimode, mixed region speech coding methods and apparatus.

Voice transmission by digital technology is particularly prevalent in long distance and digital cordless telephone applications. In turn, this has generated interest in determining the minimum amount of information that can be transmitted over a channel while maintaining the recognition quality of the reproduced speech. When voice is simply transmitted by sampling and digitization, in order to achieve voice quality of a conventional analog telephone, a data rate of 64 kilobytes per second (kbps) is required. However, speech analysis and subsequent appropriate coding, transmission, and resynthesis at the receiver can be used to significantly reduce the data rate.

A device that uses the technology of language compression by extracting parameters related to the human speech generation model is called a speech coder. The speech coder splits the input speech signal into time blocks or analysis frames. Generally, a voice coder has an encoder and a decoder. The encoder analyzes the input speech frame to extract certain relevant parameters and then quantizes the parameters into a binary representation, ie a set of bits or a binary data packet. Data packets are transmitted to receivers and decoders over communication channels. The decoder processes the data packets, quantizes them to generate parameters, and resynthesizes the speech frames using the dequantized parameters.

이다. 문제는 목표 압축률을 유지하면서 디코딩된 음성의 음질을 유지하는 것이다. 음성 코더의 성능은 (1) 음성 모델 또는 위에서 설명한 분석 및 합성 프로세스의 조합이 얼마나 잘 수행되느냐, (2) 프레임당 N₀ 비트의 목표 비트레이트에서 파라미터 양자화 프로세스가 얼마나 잘 수행되느냐에 의존한다. 따라서, 음성 모델의 목적은, 각각의 프레임에 대하여 적은 파라미터 세트로 음성 신호의 특징이나 목표 음성 품질을 획득하는 것이다.The function of the speech coder is to compress the digitized speech signal into a low bit rate signal by removing all inherent redundancy inherent in speech. Digital compression can be accomplished by representing the input speech frame in a set of parameters and using quantization to represent the parameters in bitsets. When the input speech frame has the number of bits N _i and the data packet generated by the speech coder has the number of bits N _o , the compression rate obtained by the speech coder is

to be. The problem is to maintain the sound quality of the decoded speech while maintaining the target compression rate. The performance of the speech coder depends on how well (1) the speech model or the combination of the analysis and synthesis processes described above is performed, and (2) how well the parametric quantization process is performed at the target bitrate of N ₀ bits per frame. Therefore, the purpose of the speech model is to obtain the characteristic or target speech quality of the speech signal with a small set of parameters for each frame.

The speech coder can be implemented as a time domain coder, which uses a high resolution process to encode small segments of speech (typically 5 millisecond (ms) subframes) at one time, thereby obtaining time domain speech waveforms. For each subframe, the high precision representative value from the codebook region can be known through various search algorithms known in the art. Alternatively, the speech coder can be implemented as a frequency domain coder, which captures the short-term speech spectrum of the input speech frame with a set of parameters and uses the corresponding synthesis process to reproduce the speech waveform from the spectral parameters. The parametric quantizer preserves the parameters by representing the parameters in a stored codevector representation according to known quantization techniques described in A. Gersho & R.M. Gray, Vector Quantization and Signal Compression (1992).

A known time domain speech coder is the Code Excited Linear Predictive (CELP) coder described in Digital Processing of Speech Signals 396-453 (1978) by LB Rabiner & RW Schafer. In this CELP coder, short-term correlation and redundancy of speech signals are removed by linear prediction (LP) analysis, which obtains the coefficients of a short-term formant filter. An LP residual signal is generated using a short-term prediction filter on the input speech frame, and further modeled and quantized with long-term prediction filter parameters and subsequent probabilistic code books. Therefore, CELP coding divides the encoding of the time-domain speech waveform into a separate operation of encoding LP short-term filter coefficients and an encoding of LP residual components. Time-domain coding can be performed at a fixed rate (ie, using the same number of bits N ₀ for each frame) or at a variable rate (using a different bit rate for different types of frame content). The variable-rate coder uses as many bits as necessary to encode the codec parameters at a level suitable to achieve the target quality. An exemplary variable rate CELP coder, assigned to the assignee of the present invention and described in US Pat. No. 5,414,796, incorporated herein by reference.

In general, time domain coders, such as CELP coders, use a high number of bits per frame N ₀ to maintain the accuracy of the time domain speech waveform. In general, these coders provide a relatively large number of bits N ₀ per frame (eg, 8 kbps or more) and thus have good speech quality. However, at low bit rates (4 kbps or less), time domain coders cannot maintain high quality and good performance due to the limited number of bits used. At low bit rates, this limited codebook space degrades the waveform matching capability of conventional time domain coders successfully used for high rate commercial applications.

Currently, there is a strong commercial need and research interest to develop high quality voice coders that operate at medium to low bit rates (ie, in the range of 2.4 to 4 kbps). Applications include wireless telephones, satellite communications, internet telephony, various multimedia, and voice streaming applications, voice mail, and other voice storage systems. Its driving force is the demand for high capacity and good performance under packet loss situations. Various recent speech coding standardization efforts are another direct driving force for the research and development of low rate speech coding algorithms. The low rate voice coder creates more channels or users per allowable application bandwidth, and the low rate voice coder coupled with an additional layer of appropriate channel coding can meet the bit balance of the overall coder specification and under channel error conditions. Has excellent performance.

For coding at low bit rates, various methods of spectral or frequency domain speech coding have been developed (eg RJ McAulay & TF Quatieri, Sinusodial ) which analyze the speech signal with time-varing evolution of the spectrum. Coding, in Speech Coding and Synthesis ch. 4 (see WB Kleijin & KK Paliwal eds., 1995) .In a spectral coder, the goal is to accurately convert each input speech frame into a set of spectral parameters instead of accurately Model or predict the short-term speech spectrum, then encode the spectral parameters and generate an output speech frame with the decoded parameters The synthesized speech produced does not match the original input speech waveform, but provides similar recognition quality. Examples of frequency domain coders known in the art include multiband excitation coders (MBE), sinusoidal variations. Coder (STC), and harmonic coder (HC), these frequency domain coders provide a high quality parametric model with a compact parameter set that can be accurately quantized with the small number of bits available at low bit rates. .

However, because low bit rate coding imposes limited coding resolution or critical limitations of limited codebook space, thereby limiting the efficiency of a single coding mechanism, coders cannot express different types of speech segments with the same accuracy under different background conditions. . For example, conventional low bit rate, frequency domain coders do not transmit phase information for speech frames. Instead, the phase information is reproduced using random, artificially generated initial phase values and a linear interpolation technique. See, for example, Quadratic Phase Interpolation for Voice Speech Synthesis in the MBE Model in 29 Electronic Letters 856-57 (May 1993). Because of artificially generating phase information, even if the amplitude of the sinusoid is completely preserved by the quantization-dequantization process, the output speech generated by the frequency domain coder does not match the original input speech (ie, the main pulses are not synchronized). Do). Thus, for example, it has proved difficult to use any closed loop performance measurements, such as signal-to-noise ratio (SNR) or perceptual SNR, for example in frequency-domain coders.

Multimode coding techniques have been used to perform low rate speech coding by the open loop mode determination process. An example of such a multimode technique is Amitava Das et al. Speech Coding and Synthesis ch. in Multimode and Variable-Rate Coding of Speech (WB Kleijin & KK Paliwal eds. 1995). Conventional multimode coders apply different modes or encoding-decoding algorithms to different input speech frames. Each mode or encoding-decoding process is tailored to represent a certain type of speech segment, such as voiced speech, unvoiced speech, or background noise (non-voice) in the most efficient manner. Lose. In general, an external, open loop mode determination mechanism examines an input speech frame to determine which mode to apply to the frame. In general, open loop mode determination is performed by extracting a plurality of parameters from an input frame, evaluating the parameters with respect to constant time and spectral characteristics, and making a mode determination based on the evaluation. Thus, the mode decision is made without knowing in advance how close the output voice is to the input voice with respect to the actual conditions of the output voice, i.e., voice quality or other performance measures.

Based on the above, it is desirable to provide low bit rate, frequency domain coders that more accurately evaluate phase information. It is also desirable to provide a multimode, mixed domain coder that encodes some speech frames in time-domain encoding and others in frequency-domain encoding based on the speech content of the frame. It is also desirable to provide a mixed domain coder capable of time domain encoding and other language frames frequency domain encoding in accordance with the closed loop coding mode determination mechanism. Accordingly, there is a need for a closed loop, multimode, mixed region voice coder that guarantees time synchronization between the output voice generated by the coder and the original voice input to the coder.

The present invention relates to a closed loop, multimode, mixed domain voice coder that guarantees time synchronization between the output voice generated by the coder and the original voice input to the coder. Thus, in one aspect of the invention, preferably, the multimode, mixed domain, speech processor is coupled to a coder having one or more time domain coding modes and one or more frequency domain coding modes, and processed by the speech processor. And a closed loop mode selection device configured to select a coding mode for the coder based on the content of the frame.

In another aspect of the present invention, preferably, the frame processing method applies an open loop coding mode selection process to each successive input frame to generate a time domain coding mode or a frequency domain coding mode based on the speech content of the input frame. Selecting one; Frequency domain coding the input frame when the voice content of the input frame represents a voice voice in a stable state; Comparing the frequency domain coded frame with the input frame to obtain a performance measure; And time-domain coding the input frame if the performance measure is lower than a predetermined threshold.

In another aspect of the present invention, preferably, the multimode, mixed domain, speech processor applies an open loop coding mode selection process to the input frame, thereby applying a time domain coding mode or frequency domain coding based on the speech content of the input frame. Means for selecting one of the modes; Means for frequency domain coding the input frame when the speech content of the input frame represents a steady state voice speech; Means for time-domain coding the input frame if it indicates that the speech content of the input frame is not a steady state voice speech; Means for comparing a frequency domain coded frame and an input frame to obtain a performance measure; And means for time-domain coding the input frame when the performance measure is lower than a predetermined threshold.

1 is a block diagram of a communication channel terminated by a voice coder at each stage.

2 is a block diagram of an encoder that can be used in a multimode, mixed region linear prediction (MDLP) speech coder.

3 is a block diagram of a decoder that can be used in a multimode, MDLP voice coder.

4 is a flowchart illustrating MDLP encoding steps performed by an MDLP encoder that can be used with the encoder of FIG. 2.

5 is a flowchart illustrating a speech coding determination process.

6 is a block diagram of a closed loop, multimode, MDLP voice coder.

7 is a block diagram of a spectral coder that may be used in the coder of FIG. 6 or the encoder of FIG.

8 is an amplitude versus frequency graph showing the amplitude of sinusoids in a harmonic coder.

9 is a flowchart showing a mode determination process of a multimode, MDLP voice coder.

10A is a speech signal amplitude versus time graph, and FIG. 10B is a linear prediction (LP) residual component amplitude versus time graph.

11A is a rate / mode versus frame index graph in closed loop encoding determination, FIG. 11B is a perceptual signal-to-noise ratio (PSNR) versus frame index graph in closed loop determination, and FIG. 11C is a rate / mode when there is no closed loop encoding determination. And PSNR vs. frame index graph.

In FIG. 1, the first encoder 10 receives a digital voice sample s (n) and samples s (n) for transmission to the first decoder 14 over the transmission medium 12 or the communication channel 12. Encode. Decoder 14 decodes the encoded speech samples and synthesizes the output speech signal s _SYNTH (n). For transmission in the reverse direction, the second encoder 16 encodes the digital speech sample s (n) transmitted over the communication channel 18. Second decoder 20 receives and decodes the encoded speech sample to produce a synthesized output speech signal s _SYNTH (n).

For example, the negative sample s (n) may be used in various methods known in the art, including pulse code modulation (PCM), companded μ-law or A-law, and the like. Thus representing speech signals that are digitized and quantized. As is known in the art, the speech sample s (n) consists of an input data frame, where each frame comprises a predetermined number of digital speech samples s (n). In a preferred embodiment, each 20 ms frame uses a sampling rate of 8 kHz that includes 160 samples. In the preferred embodiment described below, the data transfer rate is preferably changed from 8kbps (full rate) to 4kbps (half rate) to 2kbps (1/4 rate) to 1kbps (1/8 rate) on a frame-by-frame basis. . Also, other data rates may be used. As used herein, the term “full rate” or “high rate” refers to a data rate that is generally 8 kbps or more, and “half rate” or “low rate” generally refers to a data rate that is 4 kbps or less. It is desirable to change the data rate since low bit rates can be selectively used for frames containing relatively less voice information. As will be appreciated by those skilled in the art, other sampling rates, frame sizes, and data transfer rates may be used.

The first encoder 10 and the second decoder 20 are jointly equipped with a first voice coder or language codec. Similarly, second encoder 16 and first decoder 14 jointly have a second voice coder. Those skilled in the art will appreciate that voice coders may be implemented in a digital signal processor (DSP), application specific integrated circuit (ASIC), discrete gate logic, firmware, or any conventional programmable software module and microprocessor. . The software module may be RAM memory, flash memory, registers, or any other type of recordable storage medium known in the art. Alternatively, the microprocessor can be replaced with any conventional processor, controller, or state machine. Representative ASICs specially designed for speech coding are entitled “VOCODER ASIC”, filed February 16, 1994, and are assigned to U.S. Patent Application No. 08 / 197,417, assigned to the assignee of the present invention and referenced herein. It is explained.

According to one embodiment, as shown in FIG. 2, a multimode, mixed region linear prediction (MDLP) encoder 100 that can be used in a voice coder includes a mode determination module 102, a pitch estimation module 104, a linear prediction. (LP) analysis module 106, LP analysis filter 108, LP quantization module 110, and MDLP residual encoder 112. The input speech frame s (n) is provided to the mode determination module 102, the pitch estimation module 104, the linear prediction (LP) analysis module 106, and the LP analysis filter 108. The mode determination module 102 is based on the periodicity, the mode index I _M and mode M, and the energy, spectral tilt, zero crossing rate, etc. of each input language frame s (n) Create another extraction parameter. Various methods for classifying speech frames according to periodicity, entitled “METHOD AND APPARATUS FOR PERFORMING REDUCED RATE VARIABLE RATE VOCODING”, filed on March 11, 1997, are assigned to the assignee of the present invention, see here. US patent application Ser. No. 08 / 815,354. These methods are also included in the TIA / EIA IS-127 and TIA / EIA IS-733 Provisional Standards.

를 생성한다. LP 분석 필터 (108) 은 입력 음성 프레임 s(n) 이외에 양자화된 LP 파라미터

를 수신한다. LP 분석 필터 (108) 는 LP 잔여 신호 R[n] 을 생성하며, 이는 양자화된 선형예측 파라미터

에 기초하여 입력 음성 프레임 s(n) 과 재구성된 음성 사이의 에러를 나타낸다. LP 잔여성분 R[n], 모드 M, 및 양자화된 LP 파라미터

는 MDLP 잔여 인코더 (112) 에 제공된다. 이들 값에 기초하여, MDLP 잔여 인코더 (112) 는 잔여 인덱스 I_R 과 양자화된 잔여 신호

을 도 4 의 플로우차트를 참조하여 이하 설명하는 방법에 따라서 생성한다. Pitch estimation module 104 generates based on pitch index I _p and lag value P _o based on each input speech frame s (n). LP analysis module 106 performs linear predictive analysis on each input speech frame s (n) to generate LP parameter a. LP parameter a is provided to LP quantization module 110. In addition, the LP quantization module 110 receives the mode M and performs a quantization process according to the mode. The LP quantization module 110 may be configured to convert LP index I _LP and quantized LP parameters.

Create LP analysis filter 108 may be configured to perform quantized LP parameters in addition to the input speech frame s (n).

Receive LP analysis filter 108 generates an LP residual signal R [n], which is a quantized linear prediction parameter.

Represents an error between the input speech frame s (n) and the reconstructed speech. LP residual R [n], mode M, and quantized LP parameters

Is provided to MDLP residual encoder 112. Based on these values, MDLP Residual Encoder 112 determines the residual index I _R and the quantized residual signal.

Is generated according to the method described below with reference to the flowchart of FIG.

를 생성한다. 잔여 디코딩 모듈 (204) 는 잔여 인덱스 I_R, 피치 인덱스 I_P, 및 모드 인덱스 I_M 을 수신한다. 잔여 디코딩 모듈 (204) 는 수신한 값을 디코딩하여 양자화된 잔여 신호

을 생성한다. 양자화된 잔여 신호

및 양자화된 LP 파라미터

는 LP 합성 필터 (208) 에 제공되며, LP 합성필터 (208) 은 그로부터 디코딩된 출력 음성 신호

을 합성한다. In FIG. 3, a decoder 200 that can be used in a speech coder includes an LP parameter decoding module 202, a residual decoding module 204, a mode decoding module 206, and an LP synthesis filter 208. Mode decoding module 206 produces a mode M from it receives the mode index I _M. LP parameter decoding module 202 receives mode M and LP index I _LP . LP parameter decoding module 202 decodes the received value to quantized LP parameters.

Create Residual decoding module 204 receives the residual index I _R , the pitch index I _P , and the mode index I _M. The residual decoding module 204 decodes the received value to quantize the residual signal

Create Quantized Residual Signal

And quantized LP parameters

Is provided to the LP synthesis filter 208, where the LP synthesis filter 208 decodes the output speech signal decoded therefrom.

Synthesize

Except for the MDLP residual encoder 112, the operation and implementation of the various modules of the encoder 100 of FIG. 2 and the encoder 200 of FIG. 3 are described in U.S. Patent Nos. 5,414,796 and L.B. Rabiner & R.W. Schafer, Digital Processing of Speech Signals 396-453 (1978).

According to one embodiment, an MDLP encoder (not shown) performs the steps shown in the flowchart of FIG. 4. The MDLP encoder can be the MDLP residual encoder 112 of FIG. 2. In step 300, the MDLP encoder checks whether mode M is full rate (FR), quarter rate (QR), or eighth rate (ER). If mode M is FR, QR, or ER, the MDLP encoder proceeds to step 302. In step 302, the MDLP encoder applies FR, QR, or ER to the residual index I _R depending on the value of the corresponding rate M. In the FR mode, time-domain coding, which is high accuracy, high-rate coding, and preferably CELP coding, is applied to the LP residual frame or the speech frame. The frame is then transmitted (after further signal processing including digital-to-analog conversion and modulation). In one embodiment, the frame is an LP residual frame indicating a prediction error. In another embodiment, the frame is a speech frame representing speech samples.

On the other hand, in step 300, if mode M is not FR, QR, or ER (ie, mode M is half rate (HR)), the MDLP encoder proceeds to step 304. In step 304, spectral coding, preferably harmonic coding, is used as the half rate for the LP residual or speech signal. The MDLP encoder then proceeds to step 306. Step 306 obtains a distortion measure D by decoding the encoded speech and comparing it with the original input frame. The MDLP encoder then proceeds to step 308. In step 308, the distortion measure D is compared with a predetermined threshold T. In step 308, the distortion measure D is compared with a predetermined threshold T. If the distortion measure D is greater than the threshold T, modulate and transmit the corresponding quantized parameters for the half rate, spectral encoded frame. On the other hand, if the distortion measure D is not greater than the threshold T, the MDLP encoder proceeds to step 310. In step 310, the decoded frame is re-encoded at full rate in the time domain. Preferably, any conventional high rate, high accuracy coding algorithm, such as CELP coding, can be used. Then modulate and transmit the FR mode quantized parameters associated with the frame.

As shown in the flowchart of FIG. 5, a closed loop, multimode, MDLP voice coder, in accordance with an embodiment of the present invention, follows a series of steps to process a sample for transmission. In step 400, the voice coder receives digital samples of the voice signal that are consecutive frames. Upon receiving a given frame, the voice coder proceeds to step 402. In step 402, the coder detects the energy of the frame. Energy is a measure of the voice activity of a frame. Speech detection is performed by summing the square of the magnitude of the digital speech samples and comparing the combined energy with a threshold. In one embodiment, the threshold is adopted based on the level of change of background noise. Representative variable threshold voice activity detectors are disclosed in US Pat. No. 5,414,796, described above. The sound of some unvoiced speech may be extremely low energy samples, and may be mistakenly encoded as background noise. To prevent this, as disclosed in US Pat. No. 5,414,796 described above, spectral tilt of low energy samples can be used to distinguish the unvoiced language from background noise.

After detecting the energy of the frame, the voice coder proceeds to step 404. In step 404, the speech coder determines whether the detected frame energy is sufficient to classify the frame as containing speech information. If the detected frame energy is lower than the predetermined threshold level, the voice coder proceeds to step 406. In step 406, the voice coder encodes the frame as background noise (ie, non-voice or silence). In one embodiment, the background noise frame is time-domain encoded at 1/8 rate or 1 kbps. In step 404, if the detected frame energy matches or exceeds a predetermined threshold level, the frame is classified as voice and the voice coder proceeds to step 408.

In step 408, the voice coder determines if the frame is periodic. Various known methods for determining periodicity include, for example, the use of zero crossings and the use of a normalized autocorrelation function (NACF). In particular, the use of sign transform points and NACF to detect periodicity is assigned to the assignee of the present invention and is referred to herein, filed March 11, 1997, and entitled "METHOD AND APPARATUS FOR PERFORMING REDUCED RATE VARIABLE RATE VOCODING". US patent application Ser. No. 08 / 815,354. In addition, the methods used to distinguish voice voices from unvoiced voices are included in the TIA / EIA IS-127 and TIA / EIA IS-733 Provisional Standards. In step 410, if the frame is not determined to be periodic, the voice coder proceeds to step 410. In step 410, the voice coder encodes the frame into an unvoiced voice. In one embodiment, unvoiced speech frames are time-domain encoded at quarter rate or 2 kbps. If it is determined in step 408 that the frame is periodic, the voice coder proceeds to step 412.

In step 412, the voice coder determines whether the frame is sufficiently periodic using a period detection method known in the art, for example, as described in US Patent Application No. 08 / 815,354, supra. If it is determined that the frame is not periodic enough, the voice coder proceeds to step 414. In step 414, the frame is time domain encoded with transition speech (i.e., transition from unvoiced voice to voice voice). In one embodiment, the transition speech frame is time-domain encoded at full rate or 8 kbps.

In step 412, if the voice coder determines that the frame is sufficiently periodic, the voice coder proceeds to step 416. In step 416, the voice coder encodes the frame as voice voice. In one embodiment, the voice speech frame is spectral encoded at half rate or 4 kbps. Preferably, as described below with reference to FIG. 7, the vocal speech frame is spectrally encoded with a harmonic coder. Alternatively, other spectral coders, such as, for example, sinusoidal transform coders, multiband excitation coders, and the like, may be used. The voice coder then proceeds to step 418. In step 418, the speech coder decodes the encoded voice speech frame. The voice coder then proceeds to step 420. In step 420, the decoded voice speech frame is compared with the corresponding input speech sample for that frame to measure synthesized speech distortion and determine whether the half rate, voice speech, spectral coding model operates within tolerances. The voice coder then proceeds to step 422.

In step 422, the speech coder determines whether the error between the decoded voice speech frame and the input speech samples corresponding to the frame is below a predetermined threshold. According to one embodiment, this determination is performed by the method described below with reference to FIG. If the encoding distortion is lower than the predetermined threshold, the voice coder proceeds to step 424. In step 424, the voice coder transmits the frame as voice voice using the parameters of step 416. If the encoding distortion in step 422 matches or exceeds the predetermined threshold, the voice coder proceeds to step 414 to time-domain encode the frame of the digital voice sample received in step 400 as a full voice as transition voice.

Steps 400 through 410 include an open loop encoding determination mode. On the other hand, steps 412 to 426 include a closed loop, encoding determination mode.

In one embodiment, as shown in FIG. 6, a closed loop, multimode, MDLP voice coder includes an analog-to-digital converter (A / D) coupled to a frame buffer 502 coupled to a control processor 504; 500). An energy calculator 506, voice speech detector 508, background noise encoder 510, high rate, time domain encoder 512, low rate spectrum encoder 514 are coupled to the control processor 504. The spectrum decoder 516 is coupled with the spectrum encoder 514, and the error calculator 518 is coupled to the spectrum decoder 516 and the control processor 504. Threshold comparator 520 is coupled to error calculator 518 and control processor 504. The buffer 522 is coupled to the spectral encoder 514, the spectral decoder 516, and the threshold comparator 520.

In the embodiment of FIG. 6, the voice coder components are preferably implemented as firmware or other software driven module in the voice coder, which is itself installed in a DSP or ASIC. Voice coder parts can likewise be implemented in many other known ways. Preferably, control processor 504 may be a microprocessor, but alternatively, may be implemented in a controller, state machine, or separate logic.

In the multimode coder of FIG. 6, a speech signal is provided to the A / D 500. A / D 500 converts the analog signal into a frame of digital speech samples S (n). The digital voice sample is provided to the frame buffer 502. The control processor 504 takes the digital voice samples from the frame buffer 502 and provides them to the energy calculator 506. Energy calculator 506,

Calculate the energy E of the speech samples, where the frame is 20 ms long and the sampling rate is 8 kHz.

The control processor 504 compares the calculated speech energy with the speech activity threshold. If the calculated energy is lower than the voice activity threshold, the control processor 504 sends the digital voice sample from the frame buffer 502 to the background noise encoder 510. Background noise encoder 510 encodes the frame using the minimum number of bits needed to maintain an estimate of the background noise.

If the calculated energy is greater than or equal to the voice activity threshold, the control processor 504 sends the digital voice sample from the frame buffer 502 to the voice voice detector 508. Voice speech detector 508 determines whether the periodicity level of the speech frame allows for efficient coding using low rate spectral encoding. Methods of determining the periodicity level of speech frames are known in the art and include, for example, the use of normalized autocorrelation functions (NACFs) and code-conversion points. These and other methods are disclosed in the aforementioned US application Ser. No. 08 / 815,354.

The voice speech detector 508 provides a control processor 504 with a signal indicating whether the speech frame contains speech of sufficient periodicity that is effectively encoded by the spectral encoder 514. If the voice speech detector 508 determines that the speech frame does not have sufficient periodicity, the control processor 504 sends a digital speech sample to the high rate encoder 512, and the high rate encoder 512 sends the speech to a predetermined maximum. Time-domain encoding at the data rate. In one embodiment, the predetermined maximum data rate is 8 kbps and high rate encoder 512 is a CELP coder.

Initially, when the voice speech detector 508 determines that the speech signal has sufficient periodicity to be effectively encoded by the spectrum encoder 514, the control processor 504 may extract the digital speech samples from the frame buffer 502 (the spectrum encoder (502). 514). An exemplary spectral encoder is described below with reference to FIG. 7.

The spectral encoder 514 extracts the estimated pitch frequency F ₀ , the amplitude A _I of the harmonics of the pitch frequency, and the voice information V _c . Spectrum encoder 514 provides these parameters to buffer 522 and spectral decoder 516. Preferably, the spectral decoder 516 is similar to the in-encoder decoder of a conventional CELP encoder. Spectrum decoder 516 is a synchronous speech sample, in accordance with a spectral decoding format (described below with reference to FIG. 7).

And synthesized speech samples are provided to the error calculator 518. The control processor 504 sends the voice sample S (n) to the error calculator 518.

Error calculator 518

In accordance with it, the mean square error (MSE) between each speech sample, S (n) and each corresponding synchronous speech sample is calculated. The calculated MSE is provided to the threshold comparator 520, and the threshold comparator 520 determines whether the distortion level is within an acceptable boundary, i.e., the level of distortion is lower than a predetermined threshold.

If the calculated MSE is within an acceptable limit, the threshold comparator 520 provides a signal to the buffer 502, and the spectral encoded data is output from the speech coder. On the other hand, if the MSE is not within the acceptable limits, the threshold comparator 520 provides a signal to the control processor 504, which in turn controls the digital sample from the frame buffer 502, high rate, time. To region encoder 512. Time domain encoder 512 encodes the frame at a predetermined maximum rate, and discards the contents of buffer 522.

In the embodiment of FIG. 6, the type of spectral coding used is harmonic coding described below with reference to FIG. 7, but in another way, for example, any type such as sinusoidal transform coding or multiband excitation coding, and the like. May be spectral coding. The use of multiband excitation coding is described, for example, in US Pat. No. 5,195,166, and the use of sinusoidal transform coding is described, for example, in US Pat. No. 4,865,068.

For transition frames and for voice frames where the phase distortion threshold is less than or equal to the periodicity parameter, the multimode coder of FIG. 6 is preferably used by the high rate, time domain encoder 512 to achieve a full rate or 8 kbps. Use CELP coding. Alternatively, any other known high rate, time domain coding may be used for these frames. Thus, transition frames (and voice frames that are not sufficiently periodic) can be coded with high precision to match waveforms at the input and output well while maintaining phase information well. In one embodiment, after processing a predetermined number of consecutive voice frames whose threshold exceeds periodicity, regardless of the determination of threshold comparator 520, the multicoder is full-rate CELP in half spectral coding for one frame. Switch to coding.

The energy calculator 506 and voice speech detector 508 in connection with the control processor 504 include an open loop encoding decision. In contrast, with respect to control processor 504, spectral encoder 514, spectral decoder 516, error calculator 518, threshold comparator 520, and buffer 522 include closed loop encoding decisions.

In one embodiment described in connection with FIG. 7, spectral coding, and preferably harmonic coding, is used to encode voice frames with sufficiently periodicity at low bit rates. In general, a spectral coder is defined as an algorithm that maintains time evolution of speech spectral characteristics in a perceptually meaningful way by modeling and encoding each speech frame in the frequency domain. Essential parts of these algorithms include: (1) spectral analysis or parameter estimation; (2) parameter quantization, and (3) synthesis of the output speech waveform with the decoded parameters. Therefore, the purpose is to maintain important characteristics of the short-term speech spectrum with the spectral parameter set, encode the parameters, and synthesize the output speech using the decoded spectral parameters. In general, the output speech is synthesized as a weighted sum of sinusoids. The amplitude, frequency, and phase of the sine wave are the spectral parameters estimated during analysis.

"Analysis by Synthesis" is a known technique in CELP coding, but this technique is not used in spectral coding. The main reason for not applying the synthesis analysis to the spectral coder is that the mean square energy (MSE) of the synthesized speech is due to the loss of initial phase information, even though the speech model functions properly from a perceptible perspective. ) Can be `. Thus, another advantage of accurately generating the initial phase is the ability to directly compare the speech samples with the reconstructed speech to determine whether the speech model correctly encodes the speech frame.

In spectral coding, the output speech frame is

S [n] = S _v [n] + S _uv [n], n = 1, 2, ..., N

N is the number of samples per frame, and S _v and S _uv are the voice and unvoiced components, respectively. The sum-of-sinusoid synthesis process consists of a voice component,

Where L is the total number of sinusoidal functions, f _k is the frequency of interest in the short-term spectrum, A (k, n) is the amplitude of the sinusoid, and θ (k, n) is the phase of the sinusoid. Amplitude, frequency, and phase parameters are estimated from the short term spectrum of the input frame by a spectral analysis process. The unvoiced component can be generated with the voice component in a single sinusoidal synthesis or separately calculated by a dedicated unvoiced process and added back to S _v .

In the embodiment of FIG. 7, a particular type of spectral coder called a harmonic coder is used to spectral encode voice frames that are sufficiently periodic at low bit rates. Harmonic coders analyze small segments of a frame by characterizing the frame as a sinusoidal sum. Each sinusoid in the sinusoidal wave has a frequency that is an integer multiple of the pitch F ₀ of the frame. In other embodiments where the particular type of spectral coder used is not a harmonic coder, the sinusoidal frequency for each frame is taken from a real set between 0 and 2π. In the embodiment of Fig. 7, the amplitude and phase of each sinusoidal wave in the sum state are selected, and as shown in the graph of Fig. 8, the signal is best matched for one period. Generally, harmonic coders use an external classification that labels each input voice frame as "voiced" or "unvoiced". For voice frames, the frequency of the sinusoids is limited to harmonics of estimated pitch F ₀ , ie f _k = kF ₀ . For unvoiced speech, short-term peaks are used to determine sinusoids. The amplitude and phase are similar to their evolution over the frame.

A (k, n) = C ₁ (k) * n + C ₂ (k)

θ (k, n) = B ₁ (k) * n ² + B ₂ (k) * n + B ₃ (k)

Where the coefficients [Ci (k), Bi (k)] are obtained from the short-term Fourier transform (STFT) of the windowed input speech frame, at amplitude, frequency, and at a particular frequency position f _k (= kf ₀ ). It is estimated from the instantaneous value of the phase. The parameters to transmit for each sine wave are amplitude and frequency. This phase does not transmit, but instead is modeled according to one of several known techniques, including, for example, a quadratic phase model.

As shown in FIG. 7, the harmonic coder includes a pitch extractor 600 and a harmonic analysis logic 604 coupled to a window processing logic 602 and a discrete Fourier transform (DFT). Also, a pitch extractor 600 that receives speech sample S (n) as input is coupled to the DFT and harmonic analysis logic 604. DFT and harmonic analysis logic 604 is coupled to residual encoder 606. Pitch extractor 600, DFT and harmonic analysis logic 604, and residual encoder 606 are each coupled to parameter quantizer 608. Parametric quantizer 608 is coupled to channel encoder 610, which is coupled to transmitter 612. Transmitter 612 is coupled to receiver 614 by a standard radio frequency (RF) interface, such as, for example, a CDMA over-the-air interface. Receiver 614 is coupled to channel decoder 616, and channel decoder 616 is coupled to inverse quantizer 618. Inverse quantizer 618 is coupled to sinusoidal speech synthesizer 620. Also, a phase estimator 622 that receives previous frame information as input is coupled to a sinusoidal speech synthesizer 620. The sinusoidal speech synthesizer 620 is configured to generate a synthesized speech output, s _SYNTH (n).

Pitch extractor 600, windowing logic 602, DFT and harmonic analysis logic 604, residual encoder 606, parametric quantizer 608, channel encoder 606, sinusoidal speech synthesizer 620, and Phase estimator 622 may be implemented in a variety of ways known in the art, including, for example, firmware or software air modules. Transmitter 612 and receiver 614 may be implemented with any equivalent standard RF component known in the art.

In the harmonic coder of FIG. 7, the input sample S (n) is received by the pitch extractor 600 which extracts the pitch frequency information F ₀ . The speech frame then enables analysis of small segments by multiplying the sample by the appropriate window function by windowing logic 602. Using the pitch of the information supplied by the pitch extractor 608, the DFT and harmonic analysis logic 604 calculates the DFT of the samples so that the complex spectral point from which harmonic amplitude A ₁ is extracted as shown in the graph of FIG. 8. A complex spectrum point is generated, and L in FIG. 8 represents the total number of harmonics. The DFT is provided to a residual encoder 606 which extracts speech information V _c .

As shown in Fig. 8, the V _c parameter represents a point on the frequency axis, and above this point, the spectrum is characteristic of the unvoiced speech signal and is not a harmonic number. In contrast, below point V _c , the spectrum is harmonic and characterizes voice speech.

The A ₁ , F ₀ , and V _c components are provided to a parametric quantizer 608 that quantizes the information. The quantized information is provided in a packet form to the channel encoder 610, which quantizes the packet at low bit rates such as half rate, 4 kbps, and the like. The packet is provided to a transmitter 612, which modulates the packet and wirelessly transmits the modulated signal to the receiver 614. Receiver 614 receives and demodulates the signal and forwards the encoded packet to channel decoder 616. Channel decoder 616 decodes the packet and provides it to dequantizer 618 as a decoded packet. Inverse quantizer 618 dequantizes the information. The information is provided to the sinusoidal speech synthesizer 620.

The sinusoidal speech synthesizer 620 is configured to synthesize a plurality of sinusoids modeling the short term speech spectrum according to the above equation for S [n]. The frequency of the sine wave, f _k, is a multiple or harmonic of the fundamental frequency F ₀ , which is the frequency with pitch periodicity for the pseudo period (ie, transition) voice speech segment.

Sinusoidal speech synthesizer 620 also receives phase information from phase estimator 622. The phase estimator 622 also receives previous frame information, i.e., the A ₁ , F ₀ , and V _c parameters for the immediately preceding frame. In addition, phase estimator 622 receives the reproduced N samples of the previous frame, where N is the frame length (ie, N is the number of samples per frame). Phase estimator 622 determines the initial phase for the frame based on the information for the previous frame. Initial phase determination is provided to a sinusoidal speech synthesizer 620. Based on the information about the current frame and the initial phase calculation performed by the phase estimator 622 based on the past frame information, the sinusoidal speech synthesizer 620 generates a synthesized speech frame as described below.

As described above, the harmonic coder synthesizes or plays back a speech frame by using previous frame information and predicting that the phase changes linearly from frame to frame. In the synthesis model described above, generally called the secondary phase model, the coefficient B ₃ (k) represents the initial phase for the current voice frame being synthesized. When determining the phase, conventional harmonic coders set the initial phase to zero or generate the initial phase value randomly or in some pseudorandom generation method. In order to predict the phase more accurately, the phase estimator 622 determines the initial phase according to whether the immediately preceding frame is a voice speech frame (ie, a frame with sufficient periodicity) or a transition speech frame. Use one of If the previous frame is a voice speech frame, the final estimated phase value of that frame is used as the initial phase value of the current frame. On the other hand, when the previous frame is classified as a transition frame, the initial phase value for the current frame is obtained from the spectrum of the previous frame obtained by performing DFT of the decoder output for the previous frame. Thus, phase estimator 622 uses the exact phase information already available (since the previous frame, which is a transition frame, was processed at full rate).

In one embodiment, the closed loop, multimode, MDLP voice coder follows the voice processing steps shown in the flow chart of FIG. The speech coder encodes the LP residual of each input speech frame by selecting the optimal encoding mode. Some modes encode LP residual or speech residual in the time domain, while other modes represent LP residual or speech residual in the frequency domain. The set of modes includes the full rate, time domain for the transition frame (T mode); Half-rate, frequency domain for voice frames (V mode); Quarter rate, time domain for unvoiced frame (U mode); And 1/8 rate, time domain (N mode) for noise frames.

The steps shown in FIG. 9 can be followed to encode a speech signal or a corresponding LP residual. The waveform characteristics of noise, unvoiced, transitional and voiced speech are shown as time functions in the graph of FIG. 10A. Noise, unvoice, transition, and voice LP residuals are shown as a function of time in the graph of FIG. 10B.

In step 700, the open loop mode determination is made for one of four modes (T, V, U or N) that apply to the input speech residual component S (n). When the T mode is applied, the negative residual component is processed in the time domain with the T mode, that is, the full rate, in step 702. When the U mode is applied, the speech residual component is processed in the time domain at U mode, that is, at 1/4 rate, in step 704. When the N mode is applied, the speech residual component is processed in N mode, that is, 1/8 rate, in the time domain in step 706. When applying the V mode, the negative residual is processed in step 708 in the V domain, namely half rate.

In step 710, the performance measure D is calculated by decoding the speech encoded in step 708 and comparing it to the input speech residual S (n). In step 712, the performance measure D is compared with a predetermined threshold T. If the performance measure D is greater than or equal to the threshold T, the spectral encoded speech residual component of step 708 is approved for transmission in step 714. In contrast, if the performance measure D is less than the threshold T, the input speech residual S (n) is processed in T mode in step 716. In other embodiments, no performance measure is calculated and no threshold is defined. Instead, after processing a predetermined number of speech residual component frames in the V mode, the next frame is processed in the T mode.

Preferably, the decision steps shown in FIG. 9 allow the high bit rate T mode to be used only when necessary, thereby switching to full rate when not performing the V mode properly, thereby preventing lower bit quality while preventing degradation of quality. The V mode of rate makes it possible to exploit the periodicity of the voice speech segment. Thus, very good high sound quality reaching the sound quality of the full rate can be produced at a significantly lower average rate than the full rate. In addition, the target voice quality can be controlled by the selected performance measure and the selected threshold.

 In addition, “update” to the T mode improves the performance of subsequent application of the V mode by keeping the model phase track close to the phase track of the input speech. When the performance in the V mode is inadequate, the closed loop performance check in steps 710 and 712 switches to the T mode to "refresh" the initial phase value, thereby improving the performance of subsequent V mode processing, which is a model phase track. This allows close proximity to the initial input voice phase track. For example, as shown in the graphs of FIGS. 11A-C, the fifth frame from the beginning does not operate properly in V mode, as evidenced by the PSNR distortion measurement used. As a result, there is no closed loop determination and update, so that the phase track modeled as shown in FIG. 11C is significantly deviated from the original input speech phase track and severely degraded in the PSNR. In addition, the performance of subsequent frames processed in the V mode is also degraded. However, under the closed loop determination, the fifth frame is switched to T mode processing as shown in Fig. 11A. As shown in FIG. 11B, as evidenced by the improvement in PSNR, the performance of subsequent frames processed under V mode is also improved. In addition, the performance of subsequent frames processed in the V mode is also improved.

The decision step shown in FIG. 9 provides a very accurate initial phase estimate to improve the quality of the V mode representation, so that the enhanced V mode synthesized speech residual signal is precisely time aligned with the original input speech residual, S (n). In the following manner, the initial phase for the first V mode processed speech residual segment is derived from the immediately decoded frame. For each harmonic, when processing the previous frame under V mode, the initial phase is set equal to the last estimated phase of the previous frame. For each harmonic, when processing the previous frame in T mode, the initial phase is set equal to the actual harmonic phase of the previous frame. The actual harmonic phase of the previous frame can be derived by taking the DFT of the residual components previously decoded using the previous full frame. Alternatively, it is possible to derive the actual harmonic phase of the previous frame by taking the DFT of past decoded frames in a pitch sync method by processing the previous frame of various pitch periods.

The novel closed loop, multimode, mixed region linear prediction (MDLP) speech coder has been described. The various exemplary logic blocks and algorithm steps described in connection with this embodiment may include digital signal processors (DSPs), application specific integrated circuits (ASICs), separate gate or transistor logic, such as separate hardware components such as registers and FIFOs, It may be implemented or performed by a processor that executes a series of firmware instructions, or by any conventional programmable software module and processor. Preferably, the processor may be a microprocessor, but in other ways, the processor may be any conventional processor, controller, microcontroller, or state machine. The software module may be a RAM memory, a flash memory, a register, or any type of recordable storage medium known in the art. Preferably, data, instructions, commands, information, signals, bits, symbols, and chips may be represented throughout the specification as voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof. .

In the above, preferred embodiment of this invention was shown and described. However, various changes can be made to the embodiments disclosed herein without departing from the spirit and scope of the invention. Accordingly, the invention is not limited except as to the case according to the following claims.

Claims (26)

As a multimode, mixed domain, voice processor, A coder having at least one time domain coding mode and at least one frequency domain coding mode; And A closed loop mode selection device coupled to the coder and configured to implement the one or more time domain coding modes if the output of the frequency domain coding mode is distorted outside an acceptable range A voice processor comprising: a. The method of claim 1, The coder encodes speech frames. The method of claim 1, The coder encodes a linear prediction residual component of a speech frame. The method of claim 1, The one or more time domain coding modes include a coding mode that codes the frames at a first coding rate, Wherein said at least one frequency domain coding mode comprises a coding mode for coding frames at a second coding rate, The second coding rate is smaller than the first coding rate. The method of claim 1, And said at least one frequency domain coding mode comprises a harmonic coding mode. The method of claim 1, A comparison circuit coupled to the coder, the comparison circuit for comparing an uncoded frame with a frame coded in the one or more frequency domain coding modes and generating a performance measure based on the comparison, The coder applies the one or more time domain coding modes only if the performance measure is lower than a predetermined threshold, and otherwise applies the one or more frequency domain coding modes. The method of claim 1, The coder applies the one or more time domain coding modes to respective frames immediately following a predetermined number of successive frames coded in the one or more frequency domain coding modes. As a multimode, mixed domain, voice processor, A coder having at least one time domain coding mode and at least one frequency domain coding mode, wherein the at least one frequency domain coding mode converts the short-term spectrum of each frame into a plurality of sinusoids having a parameter set comprising frequency, phase, and amplitude. Wherein the phase is modeled with a polynomial representation and an initial phase value, the initial phase value being (1) the final estimated phase value of the previous frame if the previous frame is coded in one or more frequency domain coding modes, or (2) The coder, if the previous frame was coded in one or more time-domain coding modes, one of the phase values derived from the short-term spectrum of the previous frame; And A closed loop mode selection device coupled to the coder and configured to select a coding mode for the coder based on content of a frame processed by the speech processor A voice processor comprising: a. The method of claim 8, Sine wave frequency for each frame is an integer multiple of the pitch frequency of the frame. The method of claim 8, And the sinusoidal frequency for each frame is taken from a real set between 0 and 2π. Applying an open loop coding mode selection process to each successive input frame to select one of a time domain coding mode or a frequency domain coding mode based on speech content of the input frame; Frequency domain coding the input frame if the voice content of the input frame represents a steady state voice voice; Time-domain coding the input frame if the voice content of the input frame indicates something other than a steady state voice voice; Comparing a frequency domain coded frame with the input frame to obtain a performance measure; And Time-domain coding the input frame if the performance measure is lower than a predetermined threshold Frame processing method comprising a. The method of claim 11, wherein And the frames are linear predictive residual frames. The method of claim 11, wherein And the frames are voice frames. The method of claim 11, wherein Said time domain coding step comprises coding a frame at a first coding rate, The frequency domain coding step includes coding the frame at a second coding rate, And wherein the second coding rate is smaller than the first coding rate. The method of claim 11, wherein The frequency domain coding step includes harmonic coding. The method of claim 11, wherein Said frequency domain coding step comprises representing the short-term spectrum of each frame as a plurality of sinusoids having a parameter set comprising frequency, phase, and amplitude, The phase is modeled with a polynomial representation and an initial phase value, The initial phase value may be (1) a final estimated phase value of the previous frame if the previous frame is frequency-domain coded, or (2) a phase value derived from the short-term spectrum of the previous frame if the previous frame is time-domain coded. Frame processing method, characterized in that. The method of claim 16, The sine wave frequency for each frame is an integer multiple of the pitch frequency of the frame. The method of claim 16, And the sinusoidal frequency for each frame is taken from a real set between 0 and 2π. As a multimode, mixed domain, voice processor, Means for applying an open loop coding mode selection process to an input frame to select one of a time domain coding mode or a frequency domain coding mode based on speech content of the input frame; Means for frequency domain coding the input frame if the speech content of the input frame represents a steady state voice speech; Means for time-domain coding the input frame if the voice content of the input frame indicates something other than a steady state voice voice; Means for comparing a frequency domain coded frame with the input frame to obtain a performance measure; And Means for time-domain coding the input frame when the performance measure is lower than a predetermined threshold A voice processor comprising: a. The method of claim 19, And the input frame is a linear predictive residual frame. The method of claim 19, And the input frame is a voice frame. The method of claim 19, The time domain coding means comprises means for coding a frame at a first coding rate, The frequency domain coding means comprises means for coding a frame at a second coding rate, The second coding rate is smaller than the first coding rate. The method of claim 19, Said frequency domain coding means comprising a harmonic coder. The method of claim 19, Said frequency domain coding means comprising means for representing the short-term spectrum of each frame as a plurality of sinusoids having a parameter set comprising frequency, phase and amplitude, The phase is modeled with a polynomial representation and an initial phase value, The initial phase value may be (1) a final estimated phase value of the previous frame if the previous frame is frequency-domain coded, or (2) a phase value derived from the short-term spectrum of the immediately preceding frame if the previous frame is time-domain coded. Voice processor, characterized in that one of. The method of claim 24, Sine wave frequency for each frame is an integer multiple of the pitch frequency of the frame. The method of claim 24, And the sinusoidal frequency for each frame is taken from a real set between 0 and 2π.

Images