JPH11504733A - Multi-stage speech coder by transform coding of prediction residual signal with quantization by auditory model - Google Patents

Abstract

(57) [Summary] A speech compression system called "Transform Predictive Coding" or TPC is a 7 kHz band of speech at 16 kHz sampling at a standard bit rate of one or two 16 or 32 kb / s bits per sample. Is performed. The system uses short-term and long-term predictions to remove redundancy. The prediction residual is transformed and encoded in the frequency domain as shown in the figure by (110) after accepting the time domain data from (60) and the parameter input from (100), and thus the spectrum for auditory perception Is corrected. The TPC encoder uses only open-loop quantization as indicated by (70) and thus has low complexity. Voice quality is transparent at 32 kb / s, very good at 24 kb / s and acceptable at 16 kb / s.

Description

DETAILED DESCRIPTION OF THE INVENTION Multi-stage speech coder by transform coding of prediction residual signal with quantization by auditory modelField of the invention   The present invention uses a predictive coding system for audio signals, for example audio signals. Compression (encoding).Background of the Invention   As taught in the signal compression literature, audio and music waveforms are very Are encoded by different encoding techniques. Telephone encoding at 16 kb / s or less Speech coding, such as bandwidth (3.4 kHz) speech, is a time domain prediction code This has been done mainly by vessels. These encoders predict the speech waveform to be encoded. Use a speech production model to At this time, to reduce redundancy in the original signal The predicted waveform is subtracted from the actual (original) (to be coded) waveform. signal The reduction in redundancy results in coding gain. As an example of such a predictive speech coder Is adaptive predictive coding, all well known in the art of audio signal compression, There are multi-pulse linear prediction coding and code excitation linear prediction (CELP) coding.   On the other hand, the wideband (0 to 20 kHz) music coding of 64 kb / sec or more uses the frequency domain. It has mainly been performed by band transforms or subband encoders. These music encoders Is fundamentally very different from the speech coder described above. The difference is that the music source , Unlike audio sources, are too volatile to be immediately predictable Due to the fact that there is. As a result, models of music sources are generally music-coded Not used for Instead, the music encoder is the perceptually relevant part of the signal Use a sophisticated human auditory model to encode only. That is, general Unlike a speech coder that uses a speech generation model for music, a music coder uses Use the hearing (sound receiving) model to get it.   For speech encoders, the noise masking ability of the music to be encoded must be determined. For this purpose, a hearing model is used. The term "noise masking ability" How much quantization noise can be introduced into a music signal without noticing the noise To tell. This noise masking capability is then determined by the quantizer resolution (eg, quantizer resolution). Step size). In general, music ", The poorer the masking quantization noise, and therefore The required step size is smaller and vice versa. Smaller stepsa Is corresponding to a smaller coding gain, and vice versa. Such a music encoder Examples of AT & T's Perceptual Audio Code (PAC) and ISO MPEG Audio coding standards are included.   Between the telephone bandwidth speech coding and the wideband music coding, the speech signal is at 16 kHz. There is a wideband speech coding sampled with a bandwidth of 7 kHz. 7 kh The advantage of z wideband speech is that the resulting speech quality is And the bit rate required for encoding is 20 kHz. much lower than the audio signal of z. These previously proposed wideband speech Some encoders use time-domain predictive coding, while others use frequency-domain transforms. Or use sub-band coding, and furthermore in the time domain and frequency domain Some use a mixture of both techniques.   Include perceptual criteria in predictive speech coding, whether wideband or otherwise. Is to select the best synthesized speech signal from a plurality of synthesized speech signal candidates. The use of perceptual weighting filters has been limited. For example, against Atal et al See U.S. Patent No. Re. Such filters are Achieves some type of noise shaping that is useful for reducing noise in the process. One known encoder utilizes a perceptual model in forming a perceptual weighting filter. Attempts have been made to improve this technology.Summary of the Invention   Despite the above efforts, none of the known speech or audio encoders Hearing model to set quantizer resolution according to analysis of noise masking ability It does not make use of both audio and sound generation models for signal prediction purposes.   On the other hand, the present invention provides a noise mask determined by a human auditory sensitivity model for noise. A prediction process and a quantization process that quantizes the signal based on the I'm matching. The output of the predictive coding system thus becomes an audio perception model Therefore, a resolution (eg, uniform scan) that is a function of the determined noise masking signal. The step size in the La quantizer or the vector in the Vector quantizer. (The number of bits used for discrimination).   According to the invention, one signal representing the estimation (or prediction) of a signal representing speech information Issue is generated. The term "original signal representing speech information" refers not only to the speech itself but also to the speech itself. Audio signal derivatives commonly found in audio coding systems (eg, linear prediction and This is a broad enough meaning to also mean the “switch prediction residual signal”. At this time, The signal is compared to the original signal to form a signal that represents the difference between these compared signals. This signal, which represents the difference between the compared signals, It is quantized according to the perceptual noise masking signal generated by the perceptual model.   An embodiment of the present invention called "Transform Predictive Coding" or TPC is 16-32k Encode wideband speech as high as 7 kHz at a target bit rate of b / s. That As the name implies, TPC is a technique for transform coding and predictive coding within a single encoder. Are combined. More specifically, the encoder extracts redundancy from the input speech waveform. To use linear prediction to remove and then encode the resulting prediction residual Use transform coding techniques. The transformed prediction residuals are audible and encoded In human auditory perception expressed in the form of an auditory perception model to discard what is heard Is quantized based on knowledge.   One important feature of embodiments is the perceptual noise masking capability of the signal (eg, Perceptual threshold of "recognizable distortion") and subsequent bit allocation The manner in which the resizing is performed. As is done in traditional music encoders This embodiment, instead of using the unquantized input signal to determine the perceptual threshold, The noise masking threshold and bit allocation of the Is determined based on the frequency response of the quantized LPC synthesis filter). Is done. This feature allows the decoder to decode the received and encoded wideband speech information Encoder to duplicate the perceptual threshold and bit allocation process required for System does not need to transmit the bit allocation signal from the To provide. Instead, the synthesis filter coefficients being transmitted for other purposes Are developed and used to save bitrate.   Another important feature of the embodiment is that the TPC encoder is between encoder frequencies. Or the decoder is quantized based on the allocated bits It is about how to generate the output signal. Under certain circumstances, T The PC encoder allocates bits to only a part of the audio band (for example, 0 to 4). Bits can only be allocated for coefficients between kHz.) 4kHz and 7k No bits are allocated to represent the coefficients between the Hz, and thus the decoder Do not obtain any coefficients in the frequency range of An example of such a situation is the TPC encoder. Must operate at very low bit rates, for example 16 kb / s Occurs when the Signals encoded in the 4 kHz and 7 kHz frequency ranges Despite having no bits to represent, the decoder still has a wideband response If it must be provided, signals within this range must be combined. Implementation form In accordance with this feature of the state, the decoder may use other available information, namely (LPC Estimation of the signal spectrum (obtained from the parameters) and noise maps at frequencies within that range. A coefficient signal within this frequency range is generated (e.g., Composite). With this technique, the decoder calculates the audio signal coefficients for all bands. A broadband response can be provided without having to communicate.   ISDN videoconferencing or audio Audio conferencing, multimedia audio, "hi-fi" phone technology and 28.8k b / s or more, simultaneous voice & data (S VD).BRIEF DESCRIPTION OF THE FIGURES   FIG. 1 shows a diagram of an encoder embodiment of the present invention.   FIG. 2 shows a diagram of a decoder embodiment of the present invention.   FIG. 3 is a detailed block diagram of the LPC parameter processor of FIG. Is shown.Detailed description A. Overview of the embodiment   For the sake of clarity, the embodiments of the invention shown in the figures are represented by individual functional blocks ( (Including functional blocks labeled "processor"). Are shown. The functions represented by these blocks execute software Shared (including, but not limited to) hardware that can Alternatively, it can be provided through the use of dedicated hardware. For example, FIG. 4 is provided by a single shared processor. You may. (The use of the word "processor" is Software should not be considered exclusive).   The illustrated embodiment includes an AT & T DSP16 or DSP32C, for example. Digital signal processor (DSP) hardware, the operations discussed below Read-only memory (ROM) for storing software for executing the A random access memory (RAM) for storing DSP results may be included. No. Similarly, very large scale integrated circuit (VLSI) hardware embodiments and general A custom VLSI circuit combined with a custom DSP circuit.   According to the present invention, a sequence of digital input audio samples is called a frame. Are divided into successive 20 ms blocks, and each frame is further divided into 4 blocks each. ms into 5 equal subframes. Common for wideband audio signals Assuming a sampling rate of 16 kHz so that Compatible with frame size of sample and subframe size of 64 samples I do. The TPC speech coder buffers and processes the input speech signal frame by frame. Within each frame, several coding operations are performed per subframe. Is performed.   FIG. 1 shows an example of an embodiment of the TPC speech encoder of the present invention. Shown in FIG. See the embodiment. Once every 20 ms frame, the LPC parameter The processor 10 derives a line spectrum pair (LSP) parameter from the input audio signal S. Then, such LSP parameters are quantized, and for each 4 ms subframe, And interpolate them, then add LPC predictor coefficient array a for each subframe. Convert. The short-term redundancy determines whether the input speech signal s is Removed. The resulting LPC prediction residual signal d is still within the voiced speech Have some long-term redundancy due to the pitch periodicity of Shaping filter coefficient processor The processor 30 converts the shaped filter coefficient awc from the quantized LPC filter coefficient a. Derive. The shaping filter 40 filters the LPC prediction residual signal d and weights it perceptually. To generate the audio signal sw. The zero input response processor 50 includes a Calculate the zero input response zir of the data. Subtraction unit 60 subtracts zir from sw Thus, a target signal tp for pitch prediction is obtained.   The open loop pitch extractor and interpolator 70 is used for each 20 ms frame. Use the LPC prediction residual d to extract the pitch period, and then use each 4 ms sub The interpolated pitch period kpi is calculated for the frame. Closed loop pitcher The quantizer and pitch estimator 80 uses this interpolated pitch period kpi. To calculate three pitch predictor taps from the codebook of the pitch tap candidate set. Select one set. This selection corresponds to the previously quantized LPC residual signal dt. Filtered by a 3-tap pitch synthesis filter, and then has zero initial memory. When the output signal hd is filtered by the shaping filter, the mean square error (M SE) in such a way that it is closest to the target signal tp in the sense. A subtraction unit 90 subtracts hd from tp to obtain the target signal t t for transform coding. Get t.   The shaping filter absolute response processor 100 calculates the absolute value of the frequency response of the shaping filter. Calculate the value of the signal mag. The conversion processor 110 calculates, for the signal tt, Perform a linear transformation such as a fast Fourier transform (FFT). Next, this The processor is a quantized version of the gain values calculated for three different frequency bands. The transformation coefficient is normalized using the function and mag. The result is the normalized transform coefficient This is the signal tc. At this time, the transform coefficient quantizer 120 performs the transform at a different frequency. Auditory model quantizer control processor according to time-varying perceptual magnitude of coefficients Using the adaptive bit allocation signal ba determined by 130, the signal tc is quantized. Become   At low bit rates, such as 16 kb / s, the processor 130 Bits are allocated only to the lower half (0 to 4 kHz). In this case, the high frequency synthesis The processor 140 synthesizes the transform coefficients in the high frequency band (4-8 kHz), Is combined with the quantized low-frequency transform coefficient signal dtc to obtain the final quantized The generated all-band transform coefficient signal qtc is generated. Higher such as 24 or 32 kb / s At higher bit rates, each transform coefficient within the entire frequency band is Process is allowed to accept bits, but in some cases, It is possible that no bits will be accepted due to lack of available bits. this In that case, the high frequency synthesis processor 140 simply accepts no bits at all, Frequency within the Hz band, and the "vortex" which is typically found in adaptive transform encoders. In order to avoid distortion of the “swing” type, such “spectral homing” with low level noise ”.   The inverse transform processor 150 takes the quantized transform coefficient signal qtc and transforms it. Linear transformation, which is the inverse operation of the linear transformation used in (Inverse FFT in our particular embodiment here). As a result, the conversion Time domain signal qt, which is a quantized version of tt, the target signal for encoding t is provided. At this time, the inverse shaping filter 160 filters the qtt, Obtain the excitated excitation signal et. Adder 170 calculates the pitch prediction in block 80. Dh (this is the pitch prediction version of the LPC prediction residual d) ) Is added. The resulting signal dt is the LPC prediction This is a quantized version of the residual d. This is the zero input response processor 50 Note on filter memory of internal shaping filter and pitch predictor in block 80 Update the file. Thus, the signal loop is completed.   LPC predictor parameters (IL), pitch predictor parameters (IP and IT) , A codebook representing the transform gain level (IG) and the quantized transform coefficient (IC) The targets are multiplexed into one bit stream by multiplexer 180, It is transmitted on the channel to the decoder. Channels can be wireless channels, All suitable communication channels, including computer and data networks, telephone networks And solid state memory (eg, semiconductor memory), optical memory System (eg, CD-ROM), magnetic memory (eg, disk memory), etc. Including You can go out.   FIG. 2 shows a TPC speech decoder embodiment of the present invention. Demultifluxa 200 Separates the codebook indices IL, IP, IT, IG and IC. Pitch decoding The interpolator and interpolation circuit 205 decodes the IP and calculates an interpolated pitch period kpi You. The pitch tap decoder and pitch estimator 210 comprises a pitch estimator tap array. b to decode the IT and obtain the pitch prediction of the signal dh or LPC prediction residual d. Calculated version. The LPC parameter decoder and interpolation circuit 215 is And then calculate the interpolated LPC filter coefficient array a. Block 2 From 20 to 255, in order to generate a quantized LPC residual signal dt, FIG. Perform exactly the same operations as its corresponding configuration in. Long-term post filter 2 60 enhances the pitch periodicity within the dt and the filtered version as its output Generate fdt. This signal is passed through the LPC synthesis filter 265 and The resulting signal st is further filtered by a short-term post-filter 270. , This filter 270 produces the final filtered output audio signal fst.   To keep complexity low, TPC utilizes open-loop quantization whenever possible. Open loop quantization means that the quantizer has no effect on the output speech quality Minimizes the difference between the unquantized parameter and its quantized version Means to try. This is where the pitch estimator, gain and excitation In contrast to a normally closed loop quantized CELP coder. Of encoder parameters In closed-loop quantization, the quantizer codebook search is the last reconstructed Try to minimize distortion in the output audio. Not surprisingly, this is generally This leads to better output speech quality, but at the cost of higher codebook search complexity. Miscellaneous.   In the present invention, the TPC encoder has a closed loop amount only for the three pitch predictor taps. Use childization. The quantization operation that leads to the quantized excitation signal et Basically similar to open-loop quantization, but the effect on output speech is It is close to the child. The spirit of this approach is described in Lefebvre et al., High Quality Coding of Broadband Audio Signals Using Ki (TCX) ",Proc.IEEE Inte rnational Conf . Acoustics, Speech. Signal Processing , 1994, p. I Similar to the approach used in the TCX encoder by -193 to I-196. You. For example, a feature of the present invention not found in the TCX encoder is that the shaping filter absolute Normalization of transform coefficients by value response, adaptive bit allocation controlled by hearing model And there are high frequency synthesis and noise filling procedures. B. Encoder embodiment 1. LPC analysis and prediction   A detailed block diagram of the LPC parameter processor 10 is shown in FIG. Have been. Processor 10 is a windowing and autocorrelation processor 310; Smoothing and white noise correction processor 315; Levinson-Durbin cyclic type processor Processor 320; bandwidth extension processor 325; LPC-LSP conversion processor 3 30 and LPC power spectrum processor 335; LSP quantizer 340; L SP sorting processor 345; LSP interpolation processor 350 and LSP- An LPC conversion processor 355 is included.   Windowing and autocorrelation processor 310 initiates the LPC coefficient generation process. . Processor 310 determines the basis on which the LPC coefficients are subsequently calculated, as discussed below. The autocorrelation coefficient r is generated once every 20 ms in the conventional manner. Rabiner L.R . et al.,Digital processing of audio signals, Prentice-Hall, Inc. Englewood Cliffs, See New Jersey, 1978 (Rabiner et al.). LPC frame size Is 20 ms (or 320 audio samples at a 16 kHz sampling rate) It is. Each 20 ms frame is also 4 ms (or 64 samples) each. It is divided into five subframes of length. LPC analysis processor is the traditional way 24 ms centered on the last 4 ms subframe of the current frame Use a humming window.   To mitigate potential fault conditioning, several conventional signal conditioning techniques have Used. Spectral smoothing technology (SST) and white noise correction technology use LPC solution Applied by spectral smoothing and white noise correction processor 315 prior to analysis . SST (Tohkura, Y. et al., "PARC", which is well known in the art. OR Speech Analysis-Spectrum Smoothing Technology in Synthesis "IEEE Trans, Acoust, S peech , Signal Processing , ASSP-26: 589-596, December 1978 The moon (Tohkura et al.) Has a Gaussian probability density with a standard deviation of 40 Hz. Array of autocorrelation coefficients corresponding to the function (pdf), but in the Gaussian window (From processor 310). Same as before (Chen, JH, "Robust low-delay CELP at 16 kbit / sec. Speech encoder ", Proc IEEE, Global Comm. Conf.pp. 1237-11241 , Dallas, TX, November 1989) White noise correction is based on the zero-lag autocorrelation coefficient ( That is, the energy term) is increased by 0.001%.   At this time, the coefficients generated by the processor 315 are based on Levinson-Darby Provided to a recursive processor 320, which may be provided in a conventional manner. 16 LPC coefficients α for i = 1, 2,..., 16₁(LPC prediction error fill (The order of the data 20 is 16).   The bandwidth extension processor 325 determines each α for further signal conditioning._iTo g_i (Where g = 0.994). This is 30Hz Support bandwidth expansion (Tohkura et al.)   After such a bandwidth extension, the LPC predictor coefficients are calculated as LPC-LS in a conventional manner. It is converted to a line spectrum pair (LSP) coefficient by the P conversion processor 330. So ong, F.K. et al., "Line spectrum pair (LSP) and audio data compression" Proc.I EEE Int. Conf. Acoust. Speech, Signal Processing , Pp1.10.1-1-1. 10.4, March 1984 (Soong et al). Please refer to. This document is here It is not included as a reference as if it were completely described.   Next, an LSP quantizer 34 for quantizing the resulting LSP coefficients. 0 provides vector quantization (VQ). Used by processor 240 The specific VQ technology to be referred to is as if fully described here. Paliwal, K. K. et al., “24-bit / frame LPC Efficient vector quantization of parameters ”,Proc . IEEE Int. Conf. Acoust . Speech, Signal Processing   pp 661-664, Toronto, Canada, 199 Similar to the split VQ proposed in May 1st (Paliwal et al). 16 The dimensional LSP vector is 2,2,2,2,2,3,3 counted from the low frequency end. Into seven smaller vectors of dimension Of the seven subvectors Each is quantized to 7 bits (ie, VQ code of 128 code vectors). Using a book). Thus, seven codebook indices I each of length 7 bits L (1) to IL (7) exist, and one frame in LPC parameter quantization in total. 49 bits are used per frame. These 49 bits provide side information and And provided to multiplexer 180 for transmission to the decoder.   Processor 340 uses conventional weighting as described in Paliwal et al. Through the VQ codebook using the calculated mean square error (WMSE) strain measurement To perform the search. The LPC power spectrum processor 335 uses this WM Used to calculate weights in SE strain measurements. In processor 340 The codebook used is a conventional codebook generator known in the art. It is designed using technology. Traditional MSE distortion measurements also have Substitute for WMSE measurement to reduce encoder complexity without significant degradation in Can also be used.   Usually, the LSP coefficient monotonically increases. However, the quantization is Breakage can result. As a result of this break, the LPC signal in the decoder The synthesis filter becomes unstable. To avoid this problem, LSP Sorting Processor 345 sorts the quantized LSP coefficients and increases monotonically Restore order and ensure stability.   The quantized LSP coefficients are used in the last subframe of the current frame . These LSP coefficients and the LSP coefficients from the last subframe of the previous frame are The linear interpolation between them is performed by the LSP interpolation processor 350 in the conventional manner. Performed to provide LSP coefficients for subframes. At this time, The quantized LSP coefficients are converted to an LSP-LPC conversion processor 35 in a conventional manner. 5 converted back to LPC predictor coefficients for use in each subframe. You. This is done at both the encoder and the decoder. LSP interpolation is output This is important for maintaining smooth playback of audio. LSP interpolation is a subframe (4ms) LPC predictor coefficients can be updated smoothly once every time I do. The resulting LPC predictor coefficient array a predicts the encoder input signal. Measure It is used in the LPC prediction error filter 20 for this purpose. The input signal and its predicted The difference between the versions is the LPC prediction residual d. 2. Shaping filter   The shaping filter coefficient processor 30 calculates the first three values of the LPC predictor coefficient array a. , And then for the corresponding optimal second-order all-pole predictor, c_j, J = 0, 1, 2 and the coefficient C_jUse the Levinson-Durbin circuit to solve To use. These predictor coefficients are then bandwidth expanded by a factor of 0.7 ( That is, the j-th coefficient C_jIs C_j(0.7)^jIs replaced by Next, the processor 30 also performs bandwidth extension of the 16th order all-pole LPC predictor coefficient array a, , In this case the coefficient is 0.8. These two bandwidth extended all-pole filters By cascading (2nd and 16th order), the desired 18th order shaping field A filter 40 is obtained. The shaping filter coefficient array awc is a direct 18th order filter. In order to obtain the data, the above two bandwidth-extended coefficient arrays (2nd and 16th) are Calculated by convolution.   The shaping filter 40 is controlled by an LPC prediction error filter as shown in FIG. Once scaled, the two filters are actually the desired coding noise spectrum. Form a perceptual weighting filter with a frequency response that is approximately the inverse of the torque. Scratch The output of the shaping filter 40 is called a perceptually weighted audio signal sw.   Zero input response processor 50 has a shaping filter therein. 4m each At the beginning of the s subframe, the processor sends a 4 ms zero equivalent input signal. Performs shaped filtration by feeding the filter. Generally, the corresponding output signal level The vector zir is based on the fact that filters generally have non-zero memory (encoder initial Encoder input during the very first subframe after encoding or because the encoder has started Non-zero, except when the signal is exactly zero). The processor 60 calculates the weight The zir is subtracted from the assigned speech vector sw. The resulting vector Tp is a target vector for closed loop pitch prediction. 3. Closed loop pitch prediction   In pitch prediction, the parameters that need to be quantized and transmitted to the decoder There are two types. That is, the pitch corresponding to the period of the substantially periodic waveform of the voiced voice Switch cycle, and three prediction coefficients (tap). a. Pitch period   The pitch period of the LPC prediction residual is as if fully described here. "How to Use Voiced Message Encoder / Decoder", which is included as a reference in Efficient two-stage search technique discussed in US Pat. No. 5,327,620 entitled Using a modified version of the technique, the open loop pitch extractor and interpolator 70 ,It is determined. Processor 70 controls the cubic ellipse to limit the bandwidth to about 700 Hz. Pass the LPC residue through a circular low pass filter, then 8: 1 of the low pass filter output. Perform decimation. Corresponds to the last three subframes of the current frame Using the pitch analysis window, the correlation coefficient of the decimated signal is 3 to 34 tags corresponding to a time lag of 24 to 272 samples Calculated for imlag. Thus, the acceptable range for the pitch period is 1 . 5 ms to 17 ms or 59 to 667 Hz in terms of pitch frequency. You. This is the normal pitch for most speakers, including low pitch men and high pitch children. Is sufficient to cover the switch range.   Has the lowest time lag after the correlation coefficient of the decimated signal is calculated The first large peak of the correlation coefficient is identified. This is the first stage search. result Let t be the time lag obtained as Do not decimate by multiplying this value t by 8. To obtain a time lag in a large signal area. The resulting time lag 8t is true It refers to the neighborhood where the pitch period is most likely to be. Not decimated In order to keep the original time resolution in the signal area, the time range from t-4 to t + 4 A two-stage pitch search is performed. Original undecimated LPC residual phase The relation number d is calculated for the time lag from t−4 to t + 4 (the lower bound is 24 suns). Pull, upper bound 272 samples). The timeline corresponding to the maximum correlation coefficient within this range Is then identified as the last pitch period. This pitch period is 8 bits And the 8-bit index IP is transmitted to the decoder as side information. To the multiplexer 180. Thought it could be chosen as pitch period Since there are only 272-24 + 1 = 249, the pitch period represents the pitch period. Eight bits are sufficient for   One such 8-bit pitch indicator for each 20 ms frame Only one is transmitted. The processor 70 performs processing for each subframe in the following manner. To determine the pitch period kpi. The extracted pitch period of the current frame and the last If the difference between the frame's is greater than or equal to 20%, the extracted pitch period as described above Is used for all subframes in the current frame. On the other hand, this relative If the pitch change is less than 20%, the extracted pitch period will be the maximum of the current frame. Used for the last three subframes, while the first two subframes The pitch period between the extracted pitch period of the last frame and that of the current frame. Obtained by linear interpolation. b. Pitch predictor tap   The closed loop pitch tap quantizer and pitch estimator 80 performs the following operations: Is performed for each subframe: (1) closed-loop quantization of three pitch taps , (2) the pitch predicted version of the LPC prediction residual d in the current frame. dh and (3) the generation of hd which is the closest match to the target signal tp. .   Processor 80 may consider the quantized version of LPC prediction residual d Has an internal buffer for storing the previous sample of the signal dt. Each sub frame Processor 80 extracts three 64-dimensional vectors from the dt buffer Therefore, the pitch period kp1 is used. x₁, X_TwoAnd x_ThreeThese three called Are kpi-1, kpi, and kp respectively than the current frame of dt. Early by i + 1 samples. Then, these three vectors are zero initial Separately filtered by shaping filter with filter memory (with coefficient array awc) Is done. The resulting three 64-dimensional output vectors are represented by y₁, Y_TwoAnd y_ThreeWhen Let's call. Next, the processor 80 determines the 64 candidate sets b of the three pitch predictor taps._1j , B_2j, B_3j(J = 1, 2,..., 64) Optimal set b to minimize measurement_1k, B_2k, B_3kNeed to find out.   This type of problem has been previously studied and is disclosed in US Pat. No. 5,327,520. You can find an efficient search method in. The details of this technology are here Although not introduced, the basic idea is as follows.   Minimizing this strain measurement maximizes the inner product of the two 9-dimensional vectors. Can be shown to be equivalent to Of these 9-dimensional vectors One is y₁, Y_TwoAnd y_ThreeOnly the correlation coefficient is included. Another 9-dimensional vector Torr is only the product term derived from the set of three pitch predictor taps being evaluated Contains. Such vectors do not depend on the signal, and Because it depends only on the (One for each pitch tap code vector), these are pre-calculated, It can be stored in one table, the VQ codebook. Real In the codebook search, y₁, Y_Two, And y_ThreeIs the first 9-dimensional correlation vector Is calculated. Next, with each of the 64 pre-computed and stored 9-dimensional vectors, The inner product of the resulting vector is calculated. Remember to give maximum inner product The vector in the specified table is the winner and the three quantized pitch estimator Is derived therefrom. There are 64 vectors in the stored table Thus, to represent the three quantized pitch predictor taps, the mth sub A 6-bit index IT (m) for the frame is sufficient. 5 in each frame Since there are three subframes, three subframes used for all subframes A total of 30 bits per frame are used to represent pitch taps . These 30 bits are used for transmission to the decoder as side information. It is supplied to the chipplexer 180.   For each sub-frame, the three codebook search methods described above Little tap b_1k, B_2k, B_3kAfter the optimal set of is selected, the pitch prediction bar of d John is calculated as follows:   The output signal vector hd is calculated as follows.   This vector hd is subtracted from the vector tp by the subtraction unit 90. The result is the target vector tt for transform coding. 4. Transform coding of target vector a. Shaping filter absolute value response for normalization   The target vector tt is calculated using blocks 100-15 using a transform coding approach. 0 is encoded for each subframe. Shaping filter absolute value response processor 1 00 calculates the signal mag in the following manner. First of all, this processor Take the shaping filter coefficient array awc of the last subframe of the current frame, and Is zero padded to 64 samples, and then the resulting 64 dimensional vector Then a 64-point FFT is performed. Then this is the frequency range of 0-8kHz Is calculated, the absolute value of 33 FFT coefficients corresponding to. The resulting vector mag Is the absolute value response of the shaping filter for the last subframe. Save on calculations The first four subframes are the last subframes of the last frame Between the mag vector of the current frame and the mag vector of the last subframe of the current frame Obtained by shape interpolation. b. Conversion and gain normalization   Transform processor 110 performs a number of operations as described below. Execute. First, it uses the 64-point FFT to convert the current sub Transform a 64-dimensional vector in a frame. 64 samples (or 4 ms) This transform size is known in the audio coding art, so-called Avoid "pre-echo" distortion. As if it were completely described here Jayant, specifically included for reference. N. et al., “Based on human perception model. Signal compression "Proc. IEEE pp. 1385-1422, October 1993 That. Each of the first 33 composite FFT coefficients then corresponds to the corresponding one in the mag vector. Divided by The resulting normalized FFT coefficient vector Are the three frequency bands: (1) the first six normalized FFT coefficients (2) the next 10 normal bands Frequency bands (1500-3750 Hz) composed of normalized FFT coefficients and (3) ) The frequency band (4000-800) consisting of the remaining 17 normalized FFT coefficients 0 Hz).   The total energy in each of the three bands is calculated and then the log gain and Is converted to a called dB value. Logarithmic gain in the low frequency band is Using a 5-bit scalar quantizer designed using the well-known Lloyd algorithm And quantized. The quantized low-frequency logarithmic gain is the mid- and high-frequency band pair. It is subtracted from the number gain. The resulting level adjusted medium and high frequency The logarithmic gains are concatenated to form a two-dimensional vector, which in turn, Code designed by generalized Lloyd algorithm well known in the art It is quantized by a 7-bit vector quantizer with a book. Quantized low The frequency log gain is then quantized for the level adjusted medium and high frequency log gain. To the quantized logarithmic gain in the middle and high frequency bands . Next, all three quantized logarithmic gains change from the logarithmic (dB) domain to the linear domain. Is replaced. At this time, 33 normalized FFT coefficients (m ag) normalized to the frequency band where the FFT coefficients are Divided by the corresponding quantized linear gain of the region. After this second stage of normalization , The result is a final normal containing 33 complex numbers representing frequencies from 0 to 8000 Hz. This is a transformed vector tc.   During quantization of the log gain during the mth subframe, transform processor 110 may A 5-bit gain codebook index IG (m, 1) for frequency logarithmic gain, and Generate 7-bit codebook index IG (m, 2) for medium and high frequency logarithmic gain To achieve. Thus, the three logarithmic gains are 12 bits per subframe or frame. Encoded at a bit rate of 60 bits per frame. These 60 bits are It is provided to multiplexer 180 for transmission to the decoder as side information. This These 60 gain bits are 49 bits for the LSP and 8 bits for the pitch period. Together with the 30 bits for the bit and pitch taps, the side information is formed and the The total is 49 + 8 + 30 + 60 = 147 bits per frame. c. Bit stream   As described above, 49 bits / frame are allocated to encode LPC parameters. 8+ (6 × 5) = 38 bits / frame for a 3-tap pitch predictor (5 + 7) × 5 = 60 bits / frame for gain Was. Therefore, the total number of side information bits is 49 + 38 per 20 ms frame. + 60 = 147 bits, or about 30 bits per 4 ms subframe You. The encoder operates at one of three different rates: 16, 24 and 32 kb / s. Think it could be used. With a sampling route of 16 kHz, these three targets Typical rates are 1, 1.5 and 2 bits / sample or 64, 96 and 128 bits. And subframes, respectively. 30 bits used for side information Used in encoding main information (encoding of FFT coefficients) in the frame / subframe The number of bits remaining to be stored are three levels of 16, 24 and 32 kb / s, respectively. 34, 66 and 98 bits / subframe for each frame. d. Adaptive bit allocation   According to the principles of the present invention, enhance the perceptual quality of output speech in a TPC decoder Because of these residuals in different parts of the frequency spectrum with different quantization accuracy Adaptive bit allocation is performed to allocate bits. This is an audio signal This is done by using a model of the human sensitivity to noise in the signal. this Such models are well known in perceptual audio coding technology. An example For example, Tobias J.V. ed., "Basics of Model Auditory TheoryAcademic Press New York And London, 1970. Similarly, here is the complete description Schroeder, M.R. et al. "The human ear Optimization of Digital Speech Coder by Developing and Using Masking Characteristics. "J.Ac oust . Soc. Amer , 66: 1647-1652, December 1979 (Schroeder et al.  See also al).   The hearing model and quantization control processor 130 performs an adaptive bit allocation and t to quantize each of the 33 normalized transform coefficients contained in c Output vector that tells transform coefficient quantizer 120 whether to use the bits of generate ba. Adaptive bit allocation can be performed once per subframe. However, in the embodiment of the present invention, one frame is used to reduce computational complexity. Bit allocation is performed once each time.   As is done in conventional music encoders, noise masking thresholds and bit Rather than using an unquantized input signal to derive the allocation, The noise masking threshold and bit allocation of the embodiment may be quantized LPC The frequency response of the synthesis filter (often called the "LPC spectrum") Determined from the answer. The LPC spectrum is the input signal within the 24 ms LPC analysis window. It can be regarded as an approximation of the spectral envelope of the signal. The LPC spectrum is quantum Is determined based on the converted LPC coefficients. The quantized LPC coefficient is LPC Hearing model and quantizer control processor 13 that determines the spectrum as follows: 0 is provided by the LPC parameter processor 10. Quantized The LPC filter coefficient a is first transformed by a 64-point FFT . The power of each of the first 33 FET coefficients is determined, and then the reciprocal is calculated. Conclusion The result is an LPC power spectrum with a 64-point FFT frequency resolution. .   After the LPC power spectrum has been determined, it can be completely described here. No. 5,314,457, which is hereby incorporated by reference in its entirety. Using a modified version of the described method, the estimated noise masking threshold TM is calculated. Is calculated. Processor 130 was empirically determined from a subjective listening experiment Based on 33 samples of LPC power spectrum by frequency dependent attenuation function Become The decay function starts at 12 dB for the DC term of the LPC power spectrum. That is, it increases to about 15 dB between 700 and 800 Hz, and then to higher frequencies. Monotonically and eventually to 6 dB at 8000 Hz.   Each of the 33 attenuated LPC power spectrum samples was then "Basal plate spread" derived for that particular frequency to calculate the threshold Used to scale functions. The spread function for any frequency is The shape of the masking threshold in response to a single tone masker signal at different frequencies Corresponding. Equation (5) of Schroeder et al. Describes the "bulk" frequency scale or critical band. It describes such a spread function in the form of a band frequency, which, as if It is included as a reference as if fully described. The scaling process is For the "bulk" frequency scale, the first 33 frequencies (ie, 0 Hz, 250 Hz, 500 Hz, ... 8000 Hz) . Next, for each of the 33 resulting Bark values, Schroeder et al. Using these 33 bark values using equation (5), the corresponding spread function is summed Ringed. The resulting 33 spread functions are in one table Stored, which may be performed as part of an offline process. Estimated maski To calculate the thresholding threshold, each of the 33 spread functions has an attenuated LPC The corresponding sample values of the power spectrum were multiplied and the resulting 33 The scaled spread functions are summed. The result is an estimated masking threshold function is there. This technique for estimating the masking threshold is the only available technique It should be noted that this is not the case.   To keep complexity low, the processor 130 needs to perform adaptive bit allocation. Use the "greedy" algorithm. This technology provides a potential for future bit allocation. One at a time for the most "poor" frequency components, independent of local effects Is "greedy" in the sense of allocating bits. Any bit At the beginning, when no audio is assigned, the corresponding output audio is zero, The error signal is the input speech itself. Therefore, the initial LPC power spectrum is It is assumed to be the power spectrum of the quantization noise. Then the maski calculated above Simplification of the calculation method for noise threshold in Schroeder et al. Using each version, at each of the 33 frequencies of the 64-point FFT The magnitude of the noise is estimated.   The simplified noise magnitude at each of the 33 frequencies is calculated as follows: Is done. First, the critical bands listed in Table 1 of the book chapter of Scharf in Tobias Using linear interpolation of the width, the critical bandwidth B at the ith frequency₁Is calculated. Conclusion The result is an approximation of the term df / dx in equation (3) of Schroeder et al. 33 Is calculated in advance and stored in a table. Next, the i-th frequency For a number, the noise power N_iTo the masking threshold M_iCompare with N_i≤M_iIn The noise level L_iIs set to zero. N_i≧ M_iIf, the noise The size is calculated as follows.     L_i= B_i((N_i-M_i) / (1+ (S_i/ N_i)^Two))^0.25   In the equation, S_iIs a sample of the LPC power spectrum at the ith frequency Value.   Once the noise magnitude has been calculated for all 33 frequencies, the maximum noise A frequency having a magnitude is identified and one bit is assigned to this frequency. You. The noise power at this frequency is used to quantize the normalized FFT coefficients. Determined empirically from the signal-to-noise ratio (SNR) obtained during VQ codebook design (An example of the value of the reduction coefficient is 4 to 5 dB) ). The magnitude of the noise at this frequency is then updated using the reduced noise power. Be renewed. Next, a maximum value is identified from the updated noise magnitude array and one Bits are assigned to corresponding frequencies. This process repeats all available bits. Until they run out.   For 32 and 24 kb / sec TPC encoders, each of the 33 frequencies is suitable. Bits can be accepted during response bit allocation. On the other hand, T of 16 kb / s For PC encoders, the encoder is only able to use bits for the frequency range 0-4 kHz. (Ie, the first 16 FET coefficients) and the high frequency synthesis processor 140 Combining residual FFT coefficients in a higher frequency band of 4 to 8 kHz using In addition, better voice quality can be achieved.   Since the quantized LPC coefficient a can be used in a TPC decoder, Note that there is no need to transmit the packet allocation information. This The allocation information is a replica of the hearing model quantization control processor 50 in the decoder. Is determined by Thus, the TPC decoder converts such bit allocation information The adaptive bit allocation operation of the encoder can be replicated locally to obtain Wear. e. Transform coefficient quantization   The transform coefficient quantizer 120 uses the bit allocation signal ba to be included in tc. Quantize the transform coefficients. The DC term of the FFT is a real number and does not If such a bit is received, it is scalar-quantized. Of the bits it can accept The maximum number is four. For the 2nd to 16th FFT coefficients, the real part Uses conventional two-dimensional vector quantizer to simultaneously quantize and imaginary parts Is done. The maximum number of bits for this two-dimensional VQ is 6 bits. Remaining FF For the T coefficient, quantize the real and imaginary parts of two adjacent FFT coefficients simultaneously For this purpose, a conventional four-dimensional vector quantizer is used. After quantization of transform coefficients , The resulting VQ codebook index array IC is a key feature of the TPC encoder. Information. This index array IC is supplied to the multiplexer 180, where the index Combined with the site information bit. The result is transmitted through a communication channel to a TPC decoder. Is the last bit frame transmitted.   Transform coefficient quantizer 120 similarly computes the quantized value of the normalized transform coefficient. Decrypt. This is then the quantization of the corresponding mag element and the corresponding frequency band. By multiplying each of these coefficients by the calculated linear gain, To restore the gain level. The result is the output vector dtc. f. High frequency synthesis and noise filling   For a 16 kb / s encoder, the adaptive bit allocation should be in the 0-4 kHz band. Limited, processor 140 synthesizes a band of 4-8 kHz. Before doing it First, the hearing model quantizer control processor 130 firstly sets the frequency band of 4 to 7 kHz. For frequencies in the band, the ratio between the LPC power spectrum and the masking threshold Calculate the rate or signal-masking threshold ratio (SMR). 17th to 29th FFT coefficients (4-7kHz) are controlled by random phase and SMR It is synthesized using the absolute value that has been set. For frequencies where SMR> 5 dB , FFT coefficients are set to the quantized linear gain of the high frequency band . For frequencies where SMR ≦ 5 dB, the absolute value is the quantized value of the high frequency band. 2 dB below the obtained linear gain. About the 30th to 33rd FET coefficients Is between 2 dB and 30 dB higher than the quantized linear gain in the high frequency band. And the phase is again random.   For the 32 kb / s and 24 kb / s encoders, as described, Bit allocation is performed for the entire wavenumber band. However, 4-8 kHz Some frequencies in the z band may still not accept any bits. In this case, the high frequency synthesis and noise filling procedure described above does not accept any bits Applies to frequency only.   After applying such high frequency synthesis and noise filling to the vector dtc , The resulting output vector qtc is the quantization factor of the transform coefficients before normalization. Version. g. Inverse transformation and filter memory update   The inverse transform processor 150 is represented by a half-sized 33-element vector qtc. Performs an inverse FFT on the resulting 64-element composite vector. As a result, Quantized version of time-domain target vector tt for transcoding Is obtained as the output vector qtt.   In the zero initial filter state (filter memory), awc Inverse shaping filter 160, which is an all-zero filter with Filter the vector qtt to do so. At this time, the adder 170 adds dh to et. To obtain the quantized LPC prediction residual dt. This dt vector is then closed Update internal storage buffer in loop pitch tap quantizer and pitch estimator 80 Used to renew. This is also the zero input response generator for the next subframe. Of the zero input response processor 50 to establish a correction filter memory in preparation for Also used to excite the internal shaping filter of the unit. C. Embodiment of decryption   A decoding embodiment of the present invention is shown in FIG. For each frame, Multiplexer 200 converts all the main and side signals from the received bit stream. Separate information components. The main information, that is, the transform coefficient index array IC is used for transform coefficient decoding. Is supplied to the container 235. To decode this main information, a number of main information bits Adaptive bits to determine if is associated with each quantized transform coefficient Allocation must be performed.   The first step in adaptive bit allocation is to quantize LPC coefficients (allocation That affect the operation). The demultiplexer 200 is an LPC parameter. LSP codebook indices IL (1) to IL (7) for the data decoder 215 And the decoder provides 7 LS to obtain 16 quantized LSP coefficients. Perform a table lookup from the PVQ codebook. At this time, LPC Parameter decoder 215 has the same software as blocks 345, 350 and 355 in FIG. Performing the switching, interpolation and LSP-LPC coefficient conversion operations.   With the calculated LPC coefficient array a, the hearing model quantizer control processor 220 Is for each FFT coefficient in the same manner as the processor 130 in the TPC encoder. To determine the bit allocation (based on the quantized LPC parameters) (FIG. 1). ). Similarly, the shaping filter coefficient processor 225 and the shaping filter absolute value The answer processor 230 also has a corresponding processor 30 and 30 in the TPC encoder. And 100 replicas. Processor 230 is used by transform coefficient decoder 235 To generate the absolute value response mag of the shaping filter.   Once the bit allocation information is derived, the transform coefficient decoder 235 next It is possible to correctly decode the main information and obtain the quantized information of the normalized transform coefficients. it can. The decoder 235 similarly decodes the gain using the gain index array IG. For each subframe, there are two gain indicators (5 and 7 bits) These are the quantized logarithmic gain in the low frequency band and the mid and high frequency logarithms. The gain level is decoded into a quantized version of the logarithmic gain. At this time The quantized low frequency logarithmic gain is the quantized logarithmic gain of the middle and high frequency bands. For gain, leveled medium and high frequency log gain quantized versions It is added back. Next, all three quantized logarithmic gains are in the logarithmic (dB) domain. To the linear domain. Each of the three quantized linear gains has a corresponding Used to multiply the quantized version of the normalized transform coefficients in the waveband Can be. Each of the resulting 33 gain scaled quantized transform coefficients is , Then further multiply by the corresponding element in the shaping filter absolute value response array mag Is done. After these two scaling stages, the result is the decoded transform coefficient array dt c.   High frequency synthesis processor 240, inverse transform processor 245, and inverse shaping filter 250 is again the corresponding block (140, 150 and 1) in the TPC encoder 60) is an exact replica. Together they combine high frequency synthesis and noise Balance, inverse transform and inverse shaping filtering to produce the quantized excitation vector et. To achieve.   The pitch decoder and interpolator 205 provides the pitch for the last three subframes. Decode the 8-bit pitch index IP to obtain the first period, and then Of the first two subframes in the same manner as performed in block 70 To interpolate the pitch period. The pitch tap decoder and pitch predictor 210 has 3 One quantized pitch predictor tap b_1K, B_2K, B_3KTo get each sub Decode the pitch tap index IT for the frame. Then this was interpolated Using the pitch period kpi, the same three vectors x as described in the encoder section₁ , X_Two, And x_ThreeIs extracted. (These three vectors are the current Earlier than the frame by kpi-1, kpi and kpi + 1 samples). Then this Computes the pitch predicted version of the LPC residual as follows:   The adder 255 adds dh to et to obtain a quantized version of the LPC prediction residual d. Dt. This dt vector updates its internal storage buffer for dt To perform the pitch prediction in block 210 Feedback to the vessel.   The long-term post-filter 260 is a standard 16 kb / sec ITU-T G.728 standard. Basically similar to the long-term postfilter used in low-delay CELP encoders You. The main difference is that the three quantized pitch taps as voicing indicators Is the sum of And the scaling factor for the long-term post-filter coefficients is in G728. Therefore, it is 0.4 instead of 0.15. This voicing indicator If less than 0.5, the post-filtration operation is omitted and the output vector fdt Is the same as the input vector dt. If the indicator is greater than 0.5, A filtration operation is performed.   The LPC synthesis filter 265 is a standard LPC filter, ie, a quantized All-pole direct filter with an LPC coefficient array a. This translates the signal fdt into Filter and generate a long-term post-filtered quantized speech vector st. This st baek The tor is passed through a short-term post-filter 270 to output the final TPC decoder. A force audio signal fst is generated. Again, this short-term post-filter 270 Very similar to the short-term post-filter used in .728. The only difference Is in the following points. First, the polar control coefficient, the zero control coefficient, and the spectral tilt The dynamic control coefficients are 0.75, 0.65 and 0.15 in G.728, respectively. Instead of the corresponding values, they are 0.7, 0.55 and 0.4. Second, primary spectrum tilt The coefficients of the compensation filter are linearly interpolated between frames between samples. Thus, Possibly audible clicks due to discontinuities at frame boundaries Sounds can be avoided.   The long-term and short-term post-filters are the perceived coding noise in the output signal fst This has the effect of lowering the sound level and thus increasing the voice quality.

Claims (1)

[Claims] 1. In a method for encoding a frame of an audio signal,   Use the linear prediction filter to generate a short-term phase from the speech signal to generate a prediction residual signal. Removing the relationship;   Determining an open loop pitch period estimate of the audio signal based on the predicted residual signal When,   Two or more sub-frames of a frame based on a quantized version of the prediction residual signal Determining the pitch filter tap weights for the frame;   Open loop pitch period estimation, pitch filter for two or more subframes Based on the tap weights and the prediction residual signal, a pitch prediction residual signal is formed. Tep,   Quantizing the pitch prediction residual signal; A method comprising: