RU2256293C2 - Improving initial coding using duplicating band - Google Patents

Abstract

FIELD: radio engineering; initial coding systems.

SUBSTANCE: band width is reduced in front of coder or within coder followed by duplicating spectrum band in decoder. This is made by using new methods of transposition jointly with spectrum envelope. Proposed invention can be implemented in hardware or software codec, or can be used as self-contained processor in combination with codec. Improvement is ensured irrespective of type of codec and state-of-the-art.

EFFECT: reduced bit transfer speed at desired perception quality or enhanced perception quality at desired bit transfer speed.

20 cl, 34 dwg

Description

Technical field

In source coding systems, digital data is compressed before transmission or recording in order to reduce the required data rate or memory. The present invention relates to a new method and apparatus for improving source coding systems by duplicating a spectral band (DSP). A significant reduction in the data transfer rate is achieved without deterioration in the quality of perception, or, conversely, an improvement in the quality of perception at a given transmission rate is achieved. This is achieved by reducing the spectrum bandwidth on the encoding side and then duplicating the spectrum band in the decoder, i.e. the invention uses new concepts of signal redundancy in the spectral domain.

State of the art

The methods of source audio coding can be divided into two classes: natural audio coding and speech coding. Natural audio coding is widely used for music or arbitrary signals at medium data rates and, in principle, provides for a wide range of audio frequencies. Speech encoders are generally limited to reproducing the speech signal, but, on the other hand, can be used at very low transmission rates, albeit with a narrow band of audio frequencies. A broadband speech signal provides a very significant improvement in quality compared to a narrowband speech signal. Expanding the frequency band not only enhances speech intelligibility and naturalness, but also facilitates speaker recognition. Broadband speech coding is an important challenge facing next-generation telephone systems. In addition, due to the growth of multimedia applications, the transmission of music and other non-speech signals in telephone systems is a desirable quality.

A linear signal with pulse-code modulation (PCM), characterized by high reliability, is inefficient in transmission rate depending on the entropy of perception. The CD-ROM standard prescribes a sampling frequency of 44.1 kHz, a resolution of 16 bits per sample and stereo mode. This corresponds to a transfer rate of 1411 kbit / s. To significantly reduce the transmission speed, the original encoding can be performed using perceptual codecs of the audio signal with spectrum splitting. These natural audio codecs use irrelevant perception and statistical redundancy in the signal. By using the best encoding / decoding technology, a reduction of approximately 90% in the data volume for a standard CD format signal can be achieved without any degradation in legibility. Thus, a very high sound quality in stereo mode is possible at a speed of approximately 96 Kbps, i.e. the compression ratio is approximately 15: 1. Some perceptual codecs provide even higher compression rates. To achieve this, in the general case, it is necessary to reduce the sampling frequency and, thereby, the bandwidth of the audio frequencies. It is generally accepted to reduce the number of quantization levels, which allows for random distortion of the sound due to quantization, as well as the use of stereo region degradation due to intensive coding. The widespread use of such methods leads to poor perception. The existing encoding-decoding technology has almost exhausted itself and further progress in obtaining a gain from encoding is not expected. To further improve coding performance, a new approach is needed.

Human speech and most musical instruments form quasistationary signals received at the output of generation systems. According to Fourier theory, any periodic signal can be expressed as the sum of sinusoidal signals with frequencies f, 2f, 3f, 4f, 5f, etc., where f is the fundamental frequency. These frequencies form a sequence of harmonics. The limitation of the frequency band of such a signal is equivalent to truncation f of the sequence of harmonics. This trimming changes the perceived timbre, the color of the tone of a musical instrument or voice, and results in an audio signal that will sound “muffled” or “monotonous,” and intelligibility may be reduced. High frequencies are thus important for the subjective sense of sound quality.

Methods known from the prior art, mainly intended to improve the characteristics of the codec, and, in particular, were intended for the regeneration of high frequencies (MHF), which is a problem when encoding a speech signal. Such methods employ broadband linear frequency shifts, nonlinearities, or spectral overlays (US Pat. No. 5,127,054), resulting in the generation of intermodulation products or other non-harmonic frequency components that create a strong dissonance when applied to music signals. Such dissonance is described in the literature on speech coding as “sharp” and “rough” sound. Other methods for synthesizing a speech signal generate sinusoidal harmonics that are based on an assessment of the fundamental tone, and thus are limited to tonal, stationary sound signals (US Pat. No. 4,771,465). Such methods, known from the prior art, while being useful for low-quality speech applications, are not applicable for high-quality speech signal or music signals. A number of methods are aimed at improving the characteristics of codecs of high-quality audio sources. One of them uses synthesized noise signals generated in a decoder to replace noise-like signals in speech or music that were previously excluded by the encoder (see "Improving Audio Codecs by Noise Substitution" D. Schultz, JAES, Vol.44, No. 7/8, 1996). This is done within the high-frequency band, which is otherwise transmitted normally, on an intermittent basis in the presence of noise. Another method recreates some of the lost high-frequency harmonics that were lost during the encoding process (see Audio Spectral Coder by AJS Ferreira, AES Preprint 4201, 100 ^th Convention, May 11-14 1996, Copenhagen) and also depends on tones and pitch detection. Both methods work on the basis of a low duty cycle, providing a relatively limited gain from coding or efficiency.

SUMMARY OF THE INVENTION

The present invention provides a new method and apparatus for substantially improving digital source coding systems and, more specifically, for improving audio codecs. The invention allows to reduce the data transfer rate or to improve the quality of perception, or to implement a combination of these properties. The invention is based on new ways of using harmonics redundancy, providing the ability to discard the signal frequency bands before transmission or recording. There is no perception deterioration if the decoder performs high-quality repetition (duplication) of the spectrum according to the invention. The discarded bits represent a gain from coding with a fixed quality of perception. Alternatively, a larger number of bits may be allocated to encode low-frequency band information at a fixed transmission rate, thereby achieving higher quality perception.

The present invention postulates that a truncated sequence of harmonics can be extended based on the direct relationship between the spectral components of the low-frequency band and the high-frequency band. This extended sequence is similar to the original in the sense of perception, if certain rules are followed. First, the extrapolated spectral components must be harmonically coupled and truncated by a sequence of harmonics to avoid dissonance distortions. The present invention uses transposition as a means for a spectral duplication procedure that ensures that this criterion is met. However, for successful operation there is no need for the spectral components of the low-frequency band to form a sequence of harmonics, since the new duplicated components, harmoniously connected with the components of the low-frequency band, will not change the noise-like or non-stationary nature of the signal. A transposition is defined as the transfer of partial tones from one position on a musical scale to another while maintaining frequency relationships for these partial tones. Secondly, the spectral envelope, i.e. rough distribution of the spectrum of the duplicated high-frequency band, should repeat this distribution of the initial signal well enough. The present invention provides two operating modes, DSP-1 and DSP-2, which differ in the way the spectrum envelope is adjusted.

The first spectral band duplication mode (DSP-1), designed to improve medium-quality codec applications, is a single-channel process that uses only the information contained in the received low-frequency band signal in the decoder. The spectral envelope of this signal is determined and extrapolated, for example, using polynomials in conjunction with a set of rules or a code book. This information is used to continuously adjust and align the duplicated high frequency band. The DSP-1 method provides the advantage of post-processing, i.e. no modifications are required on the encoding side. The owner of the radio transmitting station will gain in the use of channels, or will be able to improve the quality of perception, or provide a combination of these qualities. Existing syntax and standard data flow can be used without modification.

The DSP-2 mode, designed to improve the applications of the high-quality codec, is a two-channel process in which, in addition to the transmitted low-frequency signal, according to the DSP-1 mode, the envelope of the high-frequency band spectrum is encoded and transmitted. Since changes in the spectrum envelope are much slower than changes in the components of the highband signal, only a limited amount of information is required to successfully represent the spectrum envelope. The DSP-2 mode can be used to increase the efficiency of existing encoding-decoding technologies with minimal or no change to existing syntaxes or protocols and as a very valuable tool for developing future codecs.

The DSP-1 and DSP-2 modes can be used to duplicate smaller pass-bands of the low-frequency band when such bands are excluded by the encoder, as determined by the psycho-acoustic model in the conditions of bit failure. This leads to improved perception by spectral duplication in the low frequency band in addition to spectral duplication outside the low frequency band. In addition, the DSP-1 and DSP-2 modes can also be used in codecs that use transmission rate scaling, where the signal perception quality in the receiver varies depending on the conditions in the transmission channel. This usually includes changes in the bandwidth of the audio signal of the receiver. Under these conditions, DSP modes can be successfully used to maintain a constant high-frequency band, which further improves the quality of perception.

The present invention operates on a continuous basis by duplicating the contents of signals of any type, i.e. tonal or non-tonal (noise-like and transient signals). In addition, the present spectrum duplication method creates a perceptually accurate copy of the discarded bands from the available frequency bands in the decoder.

Therefore, the DSP method provides a significantly higher level of gain from coding or improved perception quality compared to methods known from the prior art. This invention can be used in conjunction with methods for improving a codec known in the art; however, no such increase in effectiveness should be expected from such combinations.

Particleboard method includes the following steps:

- encoding a signal obtained from the original signal, where the frequency bands of the signal are removed, and this deletion is performed before or during encoding, at which the first signal is generated

- transposition of the frequency bands of the first signal during or after decoding, with the formation of the second signal,

- performing tuning of the envelope of the spectrum and

- combining the decoded signal and the second signal to form the output signal.

The bandwidth of the second signal can be set so as not to overlap or partially overlap with the frequency band of the first signal, and can be set depending on the time characteristics of the original signal and / or the first signal, or conditions in the transmission channel. The tuning of the spectral envelope is performed based on the evaluation of the initial spectral envelope of the first signal or the transmitted envelope information of the original signal.

The present invention contains two main types of transposition devices: multiband transposition devices and prediction transposition devices with a time-varying search pattern having various properties. The main multiband transposition can be performed according to the present invention as follows:

- filtering the signal to be transposed by means of a set of N≥ 2 bandpass filters with passbands containing frequencies (f1, ..., fn), respectively, for generating signals of N passbands,

a frequency shift of the passband signals in the region containing frequencies M (f1, ..., fn), where M ≠ 1 represents the transposition coefficient, and

- combining shifted bandwidth signals with the formation of a transposed signal.

Alternatively, this basic multiband transposition can be performed in accordance with the invention as follows:

- band-pass filtering of the signal to be transposed using a set of analyzing filters or a converter to generate low-frequency signals of real or complex subbands,

- an arbitrary number of channels k from the mentioned set of analyzing filters or a converter are connected to channels Mk, where M ≠ 1, into a set of synthesizing filters or a converter and

- a transposed signal is generated using a set of synthesizing filters or a converter.

The improved multiband transposition according to the present invention includes phase adjustments that improve the performance of the base multiband transposition.

A prediction transposition with a time-varying search pattern according to the present invention can be performed as follows:

- detection of a transient in the first signal,

- determining which segment of the first signal should be used when duplicating parts of the first signal, depending on the result of transient detection,

- fine-tuning the properties of the state vector and set of codes depending on the result of the detection of the transient and

- search for synchronization points in the selected segment of the first signal based on the synchronization point found during a previous search for the synchronization point.

Particleboard methods and devices according to the present invention provide the following qualities:

1. These methods and devices use new concepts of signal redundancy in the spectral region.

2. These methods and signals are applicable to arbitrary signals.

3. Each set of harmonics is individually created and adjusted.

4. All duplicated harmonics are generated in such a way as to form a continuation of the existing sequence of harmonics.

5. The process of spectrum duplication is based on transposition and does not cause any interference or creates slight interference.

6. Duplication of the spectrum can provide overlapping multiple smaller bands and / or a wide range of frequencies.

7. In the DSP-1 method, processing is performed only on the side of the decoder, i.e. All standards and protocols are subject to change.

8. The DSP-2 method can be used in accordance with most standards and protocols without changes or with minimal changes.

9. The DSP-2 method provides the codec designer with a new powerful compression tool.

10. Coding provides significant gains. The most effective application relates to the improvement of various types of low-speed codecs, such as MPEG 1/2 Layer I / II / III (US patent 5.040.217), MPEG 2/4 AAC, Dolby AC-2/3, NTT Twin VQ (US patent 5.684.920), AT & T / Lucent RAS, etc. This invention is also useful for high-quality speech codecs such as Broadband CELP and SB-ADPCM G.722, etc. to improve the quality of perception. The above codecs are widely used in multimedia, in the telephone industry, on the Internet, as well as in professional systems. T-DAB (Terrestrial Digital Audio Broadcasting System) systems use low-speed protocols that benefit in the use of channels when using this method or in improving the quality of FM and AM digital broadcasting. S-DAB satellite systems can greatly benefit from the high system costs of using the present invention to increase the number of channels in a multiplexed digital audio broadcasting system. In addition, for the first time, a full-range stream of real-time audio signals over the Internet is available using low-speed telephone modems.

Brief Description of the Drawings

The present invention is explained below with examples of its implementation, not limiting the scope or essence of the invention, with reference to the drawings, which show the following:

Figure 1 is a schematic representation of a chipboard in a coding system according to the present invention;

Figure 2 - representation of the duplication of the spectrum of the upper harmonics according to the present invention;

Figure 3 - representation of the duplication of the spectrum of the middle harmonics according to the present invention;

4 is a block diagram of an embodiment in the time domain of a transposition device according to the present invention;

5 is a flowchart in a duty cycle of a transposition device with prediction of a search pattern;

6 is a flowchart for searching for a synchronization point according to the present invention;

Figa-7b - positioning of the code sets during transients according to the present invention;

Fig. 8 is a block diagram illustrating the use of several transposition devices in the time domain in conjunction with a suitable set of filters, for a chipboard operation according to the present invention;

Figa-9c is a block diagram representing a device for analysis and synthesis using the Fourier transform for a short time interval PFKV made to generate 2nd-order harmonics according to the present invention;

Figa-10b is a block diagram for one subband with a linear frequency shift in the device PFKV according to the present invention;

11 is a diagram for one subband using a phase multiplier according to the present invention;

12 is an illustration of generating 3rd order harmonics according to the present invention;

Fig. 13 is an illustration of the generation of 2nd and 3rd order harmonics according to the present invention;

Fig. 14 is an illustration of generating a non-overlapping combination of several harmonic rows according to the present invention;

FIG. 15 is an illustration of generating a combination with alternating several harmonic rows according to the present invention; FIG.

16 is an illustration of generating wideband linear frequency shifts;

17 is an illustration of generating subharmonics according to the present invention;

Figa-18b is a block diagram of a perceptual codec;

Fig - the basic structure of a set of filters with maximum decimation;

FIG. 20 is an illustration of the generation of 2nd order harmonics in a filter set with maximum decimation according to the present invention; FIG.

21 is a block diagram of an improved multi-band transposition in a filter set with maximum decimation for subband signals according to the present invention;

FIG. 22 is a flowchart showing an improved multi-band transposition in a filter set with maximum decimation for subband signals according to the present invention; FIG.

23 is a representation of subbands and scaling factors for a typical codec;

24 is a representation of subbands and envelope information for a DSP-2 mode according to the present invention;

Fig - illustration of a hidden transmission of envelope information in the mode of the chipboard-2 according to the present invention;

FIG. 26 is an illustration of redundant coding in DSP-2 mode according to the present invention; FIG.

Fig. 27 is an embodiment of a codec using the DSP-1 method according to the present invention;

Fig. 28 is an embodiment of a codec using the DSP-2 method according to the present invention;

FIG. 29 is a block diagram of a “pseudo stereo” generator according to the present invention.

Description of Preferred Embodiments

In describing the embodiments, particular emphasis is placed on the problems of source coding of a natural audio signal. However, it should be borne in mind that the present invention is applicable to a whole range of source coding tasks, different from the tasks of encoding and decoding audio signals.

Transposition Basics

Transposition, as defined according to the present invention, is an ideal method of spectral duplication and has several important advantages compared with the prior art, including the need for detection of the fundamental tone, the same high-quality characteristic is achieved for monophonic and polyphonic program material, and the transposition is implemented equally well for tonal and non-tonal signals. In contrast to other methods, the transposition according to the invention can be used in source coding systems of arbitrary audio signals for signals of an arbitrary type.

The coefficient M of the exact transposition of the time-discrete signal x (n) in the form of the sum of cosines with time-varying amplitudes is determined by the relation

where N is the number of sinusoids, hereinafter defined as partial tones, f _i , e _i (n), α _i are individual input frequencies, temporal envelopes and phase constants, respectively, β _i are arbitrary output phase constants, and f _s are sampling rate, and O≤ Mf _i ≤ f _s / 2.

Figure 2 illustrates the generation of Mth order harmonics, where M is an integer ≥ 2. The term "Mth order harmonics" is used to simplify, although this process generates Mth order harmonics for all signals in a certain frequency range, which in most cases they themselves are harmonics of an unknown order. The input signal presented in the frequency domain X (f) is limited to band 201 to a range from 0 to f _max . The content of the signals in the range from f _max / M to Qf _max / M, where Q is the desired coefficient of expansion of the bandwidth 1 <Q≤ M, is extracted by a band-pass filter with the formation of a band-pass signal 203 with a spectrum of X _BP (f). This bandpass signal is transposed with a factor of M, forming a second bandpass signal 205 with a spectrum of X _t (f), covering the range from f _max to Qf _max . The spectrum envelope of this signal is adjusted using a programmable equalizer, forming a signal 207 with a spectrum of X _E (f). This signal is then combined with a delayed version of the input signal to compensate for the delay caused by the bandpass filter and the transpose device, whereby an output signal 209 with a spectrum Y (f) is generated, covering a range from 0 to Qf _max . Alternatively, the allocation of the frequency band can be performed after transposition M using cutoff frequencies f _max and Оf _max . When using multiple transposition devices, of course, the simultaneous generation of various harmonic series is possible. The above circuit can also be used to “fill” the obstacle bands in the input signal, as shown in FIG. 3, where the input signal has an obstacle band 301 from f ₀ to Qf ₀ . The frequency band [f ₀ / M, Qf ₀ / M], then is highlighted (303), transposed with a factor of M to [f ₀ , Qf ₀ ] (305), adjusts along the envelope (307) and combines with the delayed input signal, forming output signal 309 with a spectrum of Y (f).

An approximation of precise transposition can be used. According to the present invention, the quality of such approximations is determined using the theory of dissonance. The criterion for dissonance is presented in the work "Tonal Consonance and Critical Bandwidth" R.Plomp, W.J.M. Levelt JASA, Vol. 38, 1965 and consists in the fact that two partial tones are considered to be dissonant if the frequency difference is in the range of about 5 to 50% of the bandwidth of the critical frequency band in which these partial tones are located. The critical bandwidth for a given frequency can be approximately determined by the relation

with f and cb in hertz. In addition, the aforementioned work claims that the human hearing organs cannot separate two partial tones if they differ in frequency by less than about 5 percent of the critical bandwidth in which they are located. The exact transposition in equation (2) is approximated by

where f is the deviation from the exact transposition. If the input partial tones form a harmonic series, the hypothesis of the present invention claims that deviations from the harmonic series of transposed partial tones should not exceed five percent of the critical bandwidth in which they are located. This could explain why the methods known from the prior art give unsatisfactory “rough” results, since wideband linear frequency shifts create a much larger deviation than is permissible. When methods known from the prior art generate more than one partial tone for only one input partial tone, these partial tones must nevertheless be within the set deviation limit in order to be perceived as one partial tone. This once again explains the unsatisfactory results obtained in methods known from the prior art using non-linearities and the like, since they form intermodulation partial tones that are not within the deviation limits.

Using the above transposition-based spectrum duplication method according to the present invention, the following important properties are achieved.

- There is no overlap in the frequency domain between duplicated harmonics and existing partial tones.

- Duplicate partial tones are harmonics of the partial tones of the input signal and do not lead to an increase in dissonance or distortion.

- The envelope of the spectrum of the duplicated harmonics forms a smooth continuation of the envelope of the spectrum of the input signal, corresponding to the perception of the original envelope.

Prediction-based transposition with a time-varying search pattern

There are various ways to create the required transposition devices. Typical implementations in the time domain extend the signal over time by duplicating signal segments based on the pitch period. This signal is sequentially read at different speeds. Unfortunately, such methods are highly dependent on the detection of the fundamental tone and require accurate temporal conjugation of the signal segments. In addition, the need to work with signal segments based on the period of the fundamental tone makes them sensitive to transients. Since the detected pitch period can be much longer than the actual transient, the obvious risk of duplication of the full transient instead of simply expanding it in time. Another type of time-domain algorithm implements temporal expansion / contraction of a speech signal using prediction of the output signal search scheme (see "Pattern Search Prediction of Speech" R. Bogner, T. Li, Proc.ICASSP'89, Vol.1, May 1989 , "Time-Scale Modification of Speech based on a nonlinear Oscillator Model" (G. Kubin, WBKleijn, IEEE, 1994). This is a form of granular synthesis in which the input signal is divided into small parts, granules used to synthesize the output signal. This synthesis is usually done by correlating signal segments to determine the best joining points. This means that the segments used to form the output signal are independent of the period of the fundamental tone and thus, it is not necessary to solve the non-trivial task of detecting the fundamental pitch. However, in these methods, problems remain with rapidly changing signal amplitudes, and, if necessary, to ensure high-quality transposition, computing requirements increase. The invention provides an improved pitch shifting and transposition device in the time domain, where the use of transient detection and system dynamic parameters create more accurate transposition for high transposition coefficients for both stationary (tonal and non-tonal) and transition sounds at low computational costs.

Figure 4 shows the following modules: transient detector 401, window position controller 403, set generator 405, clock signal selector 407, clock position memory 409, minimum difference estimator 411, output segment memory 413, mixing unit 415, and sampling device with reduced frequency 417. The input signal is supplied to both the set generator 405 and the transient detector 401. If a transient is detected, information about its position is sent to the window position module 403. This module is installed Puts the size and position of the window, which is multiplied by the input signal when creating a set of codes. A code set generator 495 presses the synchronization position data from the synchronization data extraction module 407, provided that it is connected to another transposition device. If these synchronization positions are in the code set, then they are used and the output segment is generated. Otherwise, the set of codes is sent to the minimum difference estimator 411, which provides a new synchronization position. A new output segment is attached to the window along with the previous output segment in mixing module 415 and then sampled in module 417.

For clarification, a representation of the state region is introduced. Here, the state vectors or granules represent the input and output signals. The input signal is represented by the state vector x (n):

which is obtained from N delayed samples of the input signal, where N is the dimension of the state vector, and D is the delay between the input samples used to construct the vector. Granular reflection gives a sample of x (n), respectively, to each state vector x (n-1). As a result, we obtain equation (6), where a (*) is the map:

In the present method, granular mapping is used to determine the next output result based on the previous output result using a set of state transition codes. A set of codes of length L is constantly being rebuilt, including state vectors and the next sample following each state vector. Each state vector is separated from the neighboring K by samples; this allows the system to adjust the temporal resolution depending on the characteristics of the current signal being processed, where K, equal to one, represents the best resolution. The input signal segment used to construct the code set is selected based on the position of a possible transient and the synchronization position in the previous code set.

This means that the mapping a (*) is theoretically evaluated for all transitions included in the set of codes

With this set of transition codes, the new output result y (n) is calculated by searching for the state vector in the set of codes most similar to the current state vector y (n-1). This nearest neighbor search is performed by calculating the minimum difference and gives a new output sample

However, the system is not limited to working on the basis of samples; it preferably works on the basis of segments. A new output segment is entered into the window and summed, mixed with the previous output segment and then sampled. The transposition step coefficient is determined by the ratio of the length of the input segment represented by the set of codes and the length of the output segment read from the set of codes.

5 and 6 are flow charts showing the operation cycle of the transpose device. Step 501 represents data entry; at step 503, a transient is detected on the input signal segment; transient search is performed on a segment length equal to the length of the output segment. If a transient is found in step 505, then in step 507 the position of the transient is recorded and the parameters L (representing the length of the set of codes), K (representing the distance between the state vectors in quanta) and D (representing the delay between the quanta in each state vector) are set to step 509. The position of the transient is compared with the position of the previous output segment in step 511 to determine if this transient has been processed. If the verification result is positive at step 513, the position of the set of codes (window L) and the parameters K, L and D are set at step 515. After setting the necessary parameters, based on the result of detecting the transient, a new synchronization or interface point is searched (step 517). This procedure is shown in Fig.6. First, in step 601, a new synchronization point is calculated based on the previous one according to the relation

и

- есть новое и старое положения синхронизации, соответственно, S - длина обрабатываемого входного сегмента и М - коэффициент транспозиции. Точка синхронизации используется для сравнения точности новой точки сопряжения с точностью старой точки сопряжения на этапе 603. Если на этапе 605 установлено, что соответствие такое же или лучше, чем предыдущее, то эта новая точка синхронизации выдается на этапе 607 при условии, что она находится внутри набора кодов. Если нет, то осуществляется поиск новой точки синхронизации в цикле 609. Это выполняется аналогичным образом, в данном случае с функцией минимальной разности (611), однако, возможно также использовать корреляцию во временной или в частотной области. Если на этапе 613 определено, что это положение дает лучшее соответствие, чем предыдущее найденное положение, то положение синхронизации запоминается на этапе 615. Когда все положения проверены (этап 617), система возвращается (619) к процедуре согласно блок-схеме на фиг.5. Новая полученная точка синхронизации запоминается на этапе 519, и новый сегмент считывается из набора кодов на этапе 521, начинающегося с данной точки синхронизации. Этот сегмент добавляется к окну и прибавляется к предыдущему на этапе 523, квантуется с коэффициентом транспозиции на этапе 525 и запоминается в выходном буфере на этапе 527.Where

and

- there is a new and old synchronization position, respectively, S is the length of the processed input segment and M is the transposition coefficient. The synchronization point is used to compare the accuracy of the new interface point with the accuracy of the old interface point at step 603. If at step 605 it is determined that the match is the same or better than the previous one, then this new synchronization point is issued at step 607, provided that it is inside set of codes. If not, then a new synchronization point is searched in cycle 609. This is done in a similar way, in this case with the minimum difference function (611), however, it is also possible to use correlation in the time or in the frequency domain. If it is determined in step 613 that this position gives a better match than the previous found position, then the synchronization position is stored in step 615. When all positions are checked (step 617), the system returns (619) to the procedure according to the flowchart in FIG. 5 . The newly acquired synchronization point is stored in step 519, and the new segment is read from the code set in step 521 starting from that synchronization point. This segment is added to the window and added to the previous one at step 523, quantized with the transposition coefficient at step 525, and stored in the output buffer at step 527.

7 illustrates the mode of operation of the system in a transient process, taking into account the position of the set of codes. Before the transient, the set of codes 1 representing the input segment 1 is set to the “left” of the segment 1. Correlation segment 1 represents part of the previous output and is executed to find the synchronization point 1 in the set of codes 1. When the transient is detected and the transition point is processed, the set of codes moves according to Fig. 7 and remains stationary until the current processed input segment again becomes “right” in the set of codes. This makes duplication of the transient impossible because the system is not allowed to look for synchronization points before the transient.

Most pitch transposers or time-extension devices based on the prediction of the search pattern give satisfactory results for speech and solid signals. However, their characteristics quickly deteriorate for signals of high complexity, such as music, especially at high transposition coefficients. The present invention provides several solutions for enhanced performance giving good results for any type of signal. In contrast to other solutions, this system changes over time and the system parameters are based on the properties of the input signal and the parameters used on the previous work cycle. Using a transient detector that controls not only the size of the code set and position, but also the properties of the included state vectors is a reliable and computationally efficient way to avoid sound degradation for rapidly changing signal segments. In addition, it does not require a change in the length of the processed signal segment, which would require additional calculations. The present invention uses an advanced code set search based on the results of a previous search. This means that in contrast to the usual correlation of two segments, as is usually done in time-domain processing systems based on the prediction of the search scheme, the most suitable synchronization positions are checked first instead of checking all the positions in sequence. This new way to reduce the search for a set of codes significantly reduces the computational complexity of the system. In addition, when using multiple transposition devices, synchronization position information can be shared by these transposition devices to further reduce computational complexity, as shown in subsequent applications.

и

соответственно. Это дает следующее соотношение:Transposition devices in the time domain, as explained above, are used to implement the DSP-1 and DSP-2 systems according to the following example, illustrative, but not limiting. In Fig. 8, three time expansion modules are used to generate second, third and fourth order harmonics. Since in this example, each time-domain extension / transposition device operates using a broadband signal, it is advantageous to adjust the spectrum envelope of the original frequency range before transposition, given that there will be no means to do this after transpositions without adding a separate equalizer system. Spectrum envelope controls 801, 803, and 805 each operate on multiple channels of a filter set. The gain of each channel in the envelope controls must be set so that the sum, 813, 815, 817 at the output, after transposition, gives the desired envelope of the spectrum. Transpose devices 807, 809, and 811 are mutually connected to share position information of synchronization data. This is based on the fact that under certain conditions there will be a high correlation between the synchronization positions found in the code set during correlation in individual transposition blocks. We will offer, as an example, without any limitation on the scope of this invention, that a fourth-order harmonic transposition device operates on the basis of a time interval equal to half the interval of a second-order harmonic transposition device, but with a duty cycle of two times greater. Suppose further that the code sets used for these two expansion devices are the same and that the synchronization positions of these two expansion devices in the time domain are denoted as

and

respectively. This gives the following relationship:

Where

и

вычисляются индивидуально. Подобные результаты получаются для устройства транспозиции гармоник третьего порядка при присоединении к устройству транспозиции гармоник четвертого порядка.a S is the length of the input segment represented by a set of codes. This is valid until none of the synchronization position indicators reaches the end of the code set. During normal operation, n increases by one for each time frame processed by a second-order harmonic transposition device, and when the end of the set of codes by any of the pointers is inevitable, the counter n is set to n = 0, and

and

are calculated individually. Similar results are obtained for a third-order harmonics transposition device when fourth-order harmonics are connected to a transposition device.

The above use of several mutually connected transposition devices in the time domain to create higher-order harmonics leads to a significant reduction in the amount of computation. In addition, the proposed use of transposition devices in the time domain in conjunction with an appropriate set of filters makes it possible to adjust the envelope of the created spectrum while ensuring simplicity and low computational costs of transposition devices in the time domain, since these devices can more or less be performed using arithmetic with fixed point and exclusively addition / subtraction operations.

Other illustrative, but not limiting, examples of the present invention are as follows:

- the use of a transposition device in the time domain in each subband in the set of subband filters, thereby reducing signal complexity for each transposition device;

- the use of a transposition device in the time domain in conjunction with a transposition device in the frequency domain, thus allowing the system to use different methods for transposition, depending on the characteristics of the input signal being processed;

- the use of a transposition device in the time domain in a broadband speech codec operating, for example, on a residual signal obtained after linear extrapolation.

It should be borne in mind that the method described above in general terms can be advantageously used only for time-scale modifications by the simple exception of the conversion of the gating speed. In addition, it is understood that although this general outline of the method focuses on transposing the pitch to a higher pitch, i.e. on time extension, the same principles apply when transposing toward a lower pitch, i.e. time compression, as is apparent to those skilled in the art.

Filter Set Transposition

Various new transposition methods based on a set of filters are described below. The signal to be transposed is divided into a series of band signals or subbands. Subband signals are then transposed, accurately or approximately, which is realized by combining the analysis and synthesis subbands, hereinafter referred to as “Gluing”. This method is first demonstrated using the Multiple Time Fourier Transform (FFT).

FFT for N points of a time-discrete signal x (n) is determined by the equality

where k = 0, 1, ..., N-1 and ω _k = 2π k / N and h (n) is a window. If this window satisfies the following conditions:

there is an inverse transformation, and it is given by the equality

The direct conversion can be interpreted as an analyzer, see Fig. 9a, consisting of a set of N bandpass filters with pulse output signals h (n) exp (jω _k n) 901 followed by a set of N multipliers with carriers exp (-jω _k n) 903 with a shift of the band signals in the region around 0 Hz, forming N signals X _{to the} (n) analysis. This window acts like a low pass filter. X _k (n) have a small bandwidth and are sampled at a reduced frequency (block 905). Equation (12) is thus estimated only for n = rR, where R is the decimation coefficient and r is the new time variable. X _k (n) can be recovered from X _k (rR) by upsampling, see fig. 9b, i.e. by entering zeros (block 907) after filtering with a low-pass filter 909. The inverse transform can be interpreted as a synthesizer consisting of a set of N 911 multipliers with carriers (1 / N exp (jω _to n), which shifts the signals X _to (n) up by initial frequencies, followed by costs 913 (FIG. 9c), which add the Y components _to (n) from all channels. The FFT and inverse FFT (FFT) can be reordered to use discrete Fourier transform (DFT) and inverse DFT (FFT) ), which allows you to use the fast conversion algorithm Fourier (FFT) (see "Implementation of the Phase Vocoder using the Fast Fourier Transform" MRPortnoff, IEEE ASSP, Vol.24, No.3, 1976).

Fig. 9c shows a connection 915 for generating second harmonics, M = 2, at N = 32. For simplicity, only channels from 0 to 16 are shown. The center frequency of band 16 is equal to the Nyquist frequency, channels 17 to 31 correspond to negative frequencies. The blocks indicated by P 917 and amplification blocks 919 will be described later, and now should be considered as abbreviated. The input signal in this example is limited in band, so only channels 0 through 7 contain signals. The analyzer channels 8 to 16 are therefore empty and do not require mapping to the synthesizer. The analyzer channels from 0 to 7 are connected to the synthesizer channels from 0 to 7, corresponding to the input signal delay path. The analysis channels k, where 4≤ k≤ 7 are connected to the synthesis channels Mk, M = 2, which shift these signals in the frequency domain with center frequencies doubled relative to the bandpass filters k. Thus, the signals are shifted up to their original ranges, and are also transposed one octave up. In order to investigate the generation of harmonics in the sense of the real output responses of filters and modulators, negative frequencies must also be considered, see the lower branch in figa. Therefore, the combined output of the inverse transform corresponds to the mapping k → Mk 1001 and N-k → N-Mk 1003, where 4≤ k≤ 7.

This gives

where M = 2. Equation (15) can be interpreted as bandpass filtering of the input signal followed by a linear frequency shift or modulation of the Upper Sideband, i.e. modulation with one sideband using the upper sideband (see Fig. 10b), where 1005 and 1007 form a Hilbert transducer, 1009 and 1011 are multipliers with cosine and sinusoidal carriers, and 1013 is a differentiation cascade that selects the upper sideband. It is clear that this method of multi-band pass filtering of one side band can be applied explicitly, i.e. without linking a set of filters, in the time or in the frequency domain, which allows arbitrary sampling of individual passband and generator frequencies.

According to equation (15), a sinusoid with a frequency ω _i in the frequency band of the analysis channel k gives a harmonic at a frequency Mω _k + (ω _i -ω _k ). Hence, this method, called basic multiband transposition, generates only exact harmonics for input signals with frequencies ω _i = ω _k , where 4≤ k≤ 7. However, if the number of filters is large enough, the deviation from the exact transposition is insignificant (see equation (4 )). In addition, the transposition is performed exactly for quasi-stationary tones of arbitrary frequencies by entering the blocks indicated by P 917 (Fig. 9c), provided that each analysis channel contains no more than one partial tone. In this case, X _k (rR) are complex exponentials with frequencies equal to the differences between the frequencies of the partial tones ω _i and the center frequencies ω _k of the analysis filters. To obtain an accurate transposition, these frequencies must be increased by a factor M, modifying the above ratio of frequencies to the form ω _i → Mω _k + M (ω _i -ω _k ) = Mω _i . The frequencies X _k (rR) are equal to the time derivatives of their respective unfolded phase angles and can be estimated using first-order differences of consecutive phase angles. Frequency estimates are multiplied by M, and the phase angles of synthesis are calculated using these new frequencies. However, the same result, with the exception of the phase constant, is obtained in a simplified way, by multiplying the analysis arguments by M directly, eliminating the need for frequency estimation. This is described in FIG. 11, representing blocks 917. Thus, X _k (rR), where 4≤ k≤ 7 in this example, is converted from rectangular to polar coordinates, as shown by R → P blocks (block 1101). The arguments are multiplied by M = 2 (block 1103), and the amplitudes are not changed. The signals are then converted back to rectangular coordinates (P → R) in block 1105, generating signals Y _Mk (rR), and fed to the synthesizer channels according to FIG. 9c. This improved method of multiband transposition, therefore, has two steps: binding provides coarse transposition, as in the basic method, and phase multipliers provide accurate frequency adjustments. The above methods of multiband transposition are different from traditional pitch shifting technology using FFTs, where generators are used for synthesis based on conversion tables, or when FFTs are used to synthesize a signal that is time-stretched and thinned, i.e. binding is not used.

The harmonic binding of FIG. 9c is easily modified for other transposition coefficients other than 2. FIG. 12 illustrates the coupling of 1203 to generate 3rd order harmonics, where 1201 are the analysis channels and 1205 are the synthesis channels. Various orders of harmonics can be created simultaneously, as shown in FIG. 13, where 2nd and 3rd order harmonics are used. Fig. 14 illustrates a non-overlapping combination of 2nd, 3rd and 4th order harmonics. The lowest possible harmonic number is used as the highest possible frequency. Above the upper limit of the assigned harmonic range M, harmonic M + 1 is used. Fig. 15 illustrates a method for displaying all synthesizer channels (N = 64, O-32 channels shown). All high-frequency channels with non-prime index numbers are mapped according to the following relationship between source channel numbers and destination channel numbers: k = _desig = Mk _source , where M is the smallest integer ≥ 2, which satisfies the condition that k _source lies in the low frequency range, and k _desig - in the high frequency range. Therefore, no synthesizer channel receives a signal from more than one analysis channel. High-frequency channels with numbers in the form of prime numbers can be mapped to k _source = 1 or to the low-frequency channels k _source > 1, which gives good approximations of the above ratio (only connections with numbers not in the form of prime numbers with M = 2, 3, 4 5 are shown in FIG. 15).

It is also possible to combine amplitude and phase information from various analyzer channels. The amplitude signals [X _to (rR)] can be connected according to FIG. 16, while the phase signals arg {X _to (rR)} are connected according to the rule of FIG. 16. Thus, the lower frequencies will still be transported, whereby a periodic repetition of the envelope of the original region is generated, instead of the expanded envelope, which is the result of transposition according to equation (2). Gating or other means may be used to avoid amplification of the "empty" source channels. 17 illustrates another application, the generation of subharmonics with respect to filtered high frequencies or a bass-limited signal using connections from the upper to lower subbands. When using the aforementioned transpositions, it may be advantageous to apply controlled link switching based on signal characteristics.

In the above description, it is assumed that the highest frequency contained in the input signal is significantly lower than the Nyquist frequency. Thus, it is possible to perform bandwidth expansion without increasing the sampling frequency. This, however, is not always the case, as a result of which it may be necessary to first increase the sampling rate. When using methods based on a set of filters for transposition, it is possible to include sampling with increased frequency in the processing procedure.

Most perceptual codecs use filtering sets with maximum decimation when mapping time to frequency ["Introduction to Perceptual Coding" K. Brandenburg, AES, Collected Papers on Digital Audio Bitrate Reduction, 1996]. Fig. 18a shows the basic structure of a perceptual coding system. The 1801 analysis filterbank splits the input signal into multiple subband signals. These subband samples are individually quantized (1803) using a reduced number of bits, where the number of quantization levels is determined from the perceptual model (1807), which estimates the minimum masking threshold. These subbands are normalized, encoded with optional redundancy coding methods, and combined with additional information consisting of normalization coefficients, bit allocation information, and other specific codec data (1805) to form a sequential bit stream. This bitstream is then stored or transmitted. At the decoder (FIG. 18b), the encoded bitstream is demultiplexed (1809), decoded, and subband samples are re-quantized to an equal number of bits (1811). A set of synthesis filters combines subband samples to restore the original signal (1813). Implementation options using filter sets with maximum decimation significantly reduce computational costs. In the following description, emphasis is placed on cosine modulated filter sets. However, it should be borne in mind that the present invention can be implemented using other types of filter sets or converters, including interpretations of a filter set with low-intensity waveform conversion, other filter sets or converters with unequal bandwidths, and multidimensional filter sets or converters.

In an illustrative, but non-limiting description, it is assumed below that the L-channel cosine modulated filterbank splits the input signal x (n) into L subbands. The general structure of a filter set with maximum decimation is shown in FIG. Analysis filters are designated H _to (z) 1901, where k = 0, 1, ..., L-1. The signals of the subbands ν _to (n) are thinned out as much as possible (1903) by each of the sampling frequencies f _s / L, where f _s is the sampling frequency x (n). The synthesis unit reconnects the subband signals after interpolation (1905) and filtering (1907) to form x (n). Synthesis filters are designated F _to (z). In addition, the present invention performs spectrum duplication on x (n), resulting in a signal y (n).

возрастает на коэффициент Q, формируя у(n), это является версией набора фильтров с максимальным прореживанием многополосного устройства транспозиции согласно изобретению. Используя эту схему, процесс дискретизации с повышением частоты дискретизации интегрируется в процесс фильтрования процесса синтеза, как объяснялось ранее. Следует отметить, что может быть использован набор фильтров синтеза любого размера, что приводит к различным частотам дискретизации выходного сигнала, и, следовательно, к разным коэффициентам расширения полосы частот. Выполнение дублирования спектра на

согласно настоящему изобретению, соответствующему базовому многополосному способу транспозиции с целочисленным коэффициентом транспозиции М, выполняется связыванием сигналов поддиапазонов какSynthesizing the subband signals using a QL-channel filter set, where only L low-frequency channels are used, and the band expansion coefficient Q is chosen so that QL is an integer, resulting in an output bit stream with a sampling frequency of Qf _s . Therefore, the expanded filter set will act as if it were an L-channel filter set followed by a higher frequency sampling device. Since in this case L (Q-1) high-pass filters are not used (zeros are fed to them), the audio signal bandwidth will not change - the set of filters will simply reproduce the version of x (n) with increased frequency sampling. If, however, subband signals are associated with high-pass filters, then the bandwidth

increases by a factor of Q, forming y (n), this is a version of a set of filters with maximum thinning multiband transposition device according to the invention. Using this scheme, the sampling process with increasing sampling frequency is integrated into the filtering process of the synthesis process, as explained earlier. It should be noted that a set of synthesis filters of any size can be used, which leads to different sampling frequencies of the output signal, and, therefore, to different expansion coefficients of the frequency band. Perform spectrum duplication on

according to the present invention, the corresponding basic multiband transposition method with an integer transposition coefficient M, is performed by coupling the signals of the subbands as

where k∈ [0, L-1] and is chosen so that Mk∈ [L, QL-1], e _mk (n) is the envelope correction, and (-1) ^{(m-1) kn} is the correction coefficient for spectrally inverted subbands. Spectral inversion results from decimation of subband signals, and inverted signals can be re-inverted by changing the sign of every second sample in these channels. With reference to FIG. 20, consider a 16-channel synthesis filterbank associated (2009) for the transposition coefficient M = 2, with Q = 2. Blocks 2001 and 2003 denote analysis filters H _K (z) and decimation blocks of FIG. 19, respectively. Similarly, 2005 and 2007 are interpolators and synthesis filters F _k (z). Equation (16) is then simplified according to the coupling of the signals of the four upper frequency subbands of the received data into each second group of the eight highest channels in the set of synthesis filters. Due to spectral inversion, every second connected subband signal must be frequency inverted before synthesis. In addition, the amplitudes of the connected signals must be adjusted (2011) according to the rules of chipboard-1 or chipboard-2.

When using the basic multiband transposition method according to the present invention, the generated harmonics are generally not exactly multiples of the fundamental frequencies. All frequencies except the lowest in each subband differ to some extent from the exact transposition. In addition, the duplicated spectrum contains zeros, since the interval of the resulting range covers a wider range of frequencies than the interval of the original range. Moreover, the suppression properties of the spurious signal of the cosine modulated filter set disappears, since the subband signals are frequency-separated in the output interval. That is, the signals of adjacent subbands do not overlap in the high frequency region. However, methods for reducing spurious signals known to those skilled in the art can be used to reduce this type of interference. The advantages of this transposition method are its ease of implementation and very low computational cost.

To ensure high accuracy of sinusoidal transposition, a solution is presented based on a set of filters for efficient maximum decimation of an improved method of multiband transposition. The system uses an additional modified set of analysis filters, while the synthesis filter set is cosine modulated, as described in Multi-rate Systems and Filter Banks, PPVaidyanathan, Prentice Hall, Englewood Cliffs, New Jersey, 1993, ISBN 0 -13-605718-7. The steps of the multi-band transposition method according to the present invention based on filter sets with maximum decimation are shown schematically in FIG. 21 and in a flow chart of FIG. 22 and are as follows:

1. L received subband signals are synthesized using a QL channel filter bank 2101, 2201, 2203, where zeros are applied to the L (Q-1) of the upper channels to form an x (n) signal, which is thus oversampled with a spreading factor of width stripes Q.

2. x ₁ (n) is sampled at a reduced frequency with a factor of Q to form a signal x ₂ (n ') 2103, 2205, i.e. x ₂ (n ') = x ₁ (Qn').

3. An integer value K is selected as the size of the synthesis filter set, so limited that T = KM / Q is an integer, where T is the size of the modified analysis filter set, and M is the transposition coefficient 2207, 2209, 2211. K should preferably be selected large for stationary (tonal) signals and smaller for dynamic (transient) signals.

4. x (n ') is filtered by T-channel filters of the modified analysis filter set 2107, 2213, where T analysis filters are exponentially modulated to form a set of complex subband signals. These subband signals are sampled at a reduced frequency with a T / M coefficient, giving subband signals ν _k ^(M) (n ''), k = 0, 1, ... T-1. From here, the filter set will be excessively sampled with coefficient M.

5. The signals ν _k ^(M) (n ″) are converted to the polar representation (amplitude and phase angle). The phase angles are multiplied by the coefficient M, and these signals are converted back into a representation in rectangular coordinates according to the scheme of Fig.11. The real components of the complex signals are taken, as a result of which the signals s _k ^(M) (n ″) 2109, 2215 are obtained. After this operation, the signals s _k ^(M) (n ″) are critically sampled.

6. Signal amplification factors s _k ^(M) (n ") are regulated according to the rules of chipboard-1 or chipboard-2 (2111, 2217).

7. The signals of the subbands s _to ^(M) (n ''), where k∈ [T / M, min (К, Т) -1], are synthesized using a conventional cosine-modulated K-channel filter set, where the channels from 0 to T / M-1 zeros are given. As a result, a signal x ₃ ^(M) (n) is generated.

8. x ₃ ^(M) (n) is finally added to x ₁ (n) to give y (n) 2223, which is the desired signal of the duplicated spectrum.

Steps 3 through 6 can be repeated for different values of the transposition coefficient M, thus adding a lot of harmonics to x ₁ (n). This mode of operation is illustrated by dashed lines in FIG. 21 and FIG. 22, repeating a cycle including blocks 2211-2219. In this case, K is chosen to make T integer for all values of the selected M - for integer M: s; preferably, K is selected so that K / Q is a positive integer. All signals of the subbands s _k ^(Mi) (m ''), where i = 1, 2, ..., m, am are the number of transposition coefficients, are summed according to the equality

for all used k. In the first iteration of the loop of FIG. 22, the signals s _to (n '') can only be considered as zero signals of the subbands, where k = 0, 1, ..., K-1. In each cycle, new signals are added (2219) to s _to (n '') as follows:

where k = K / Q, K / Q + 1, ..., min (K, T _i ) -1. The subband signals s _k (n ″) are synthesized once using the K-channel filter set according to step (7).

The modified analysis filter set for step (4) is obtained according to the theory of cosine modulated filter sets, where the modulated transform with overlap (see "Lapped Transforms for Efficient Transform / Subband Coding" HSMalvar, IEEE Trans ASSP, vol. 38, no.6, 1990) is a special case. The impulse responses of h _k (n) filters in a T-channel cosine modulated filter set can be written as

where k = 0, 1, ..., T-1, N is the prototype length of the low-pass filter p _o (n), C is a constant, and Ф _к is the phase angle, which ensures the exclusion of interference between adjacent channels. The restrictions on f _{to the} following:

which can be simplified to express in closed form

With such a choice of Ф _к using a set of synthesis filters, systems of exact reconstruction or systems of approximate reconstruction (pseudoQMF system) with impulse responses in the form

Consider filters

where h _k '(n) is a sinusoidally modulated version of the filter p _o (n). Filters H _k '(z) H _k (z) have identical bandwidth supports, but differing phase responses. The filter passbands are actually Hilbert transforms of each other (this is not true for frequencies close to w = 0 and w = π). Combining equations (19) and (23) according to

gives filters that have the same shape of amplitude responses as H _to (z) for positive frequencies, but are zeros for negative frequencies, using a set of filters with impulse responses as in equation (24) gives a set of subband signals that can be interpreted as signals analysis (complex) corresponding to subband signals obtained from a set of filters with impulse responses, as in equation (19). Analysis signals are suitable for manipulation, since samples in the form of complex values can be recorded in polar form, i.e.

z (n) = r (n) + ji (n) = | z (n) | exp {j arg (z (n))}.

However, when using a set of complex filters for transposition, the constraints for Φ _k must be generalized in order to maintain the property of eliminating interference. A new restriction for Ф _к to ensure the elimination of interference together with a set of synthesis filters with impulse responses as in equation (22) has the form:

which simplifies to equation (21) when M = 1. With this choice, the transposed partial tones will have the same relative phases as they would have at M = 1 (no transposition).

The combination of equality 24 and equality 25 gives

what are the filters used in the modified filter set for step (4) according to the present invention.

Below are some explanations regarding step (5). Discretization with a reduced frequency of complex subband signals leads to oversampling by M, which is an essential criterion when subsequently the phase angles are multiplied by the transposition coefficient M. Oversampling results in the number of subband samples per bandwidth after being transposed to the destination range becomes the number of subband samples of the original range. The individual passband of the transposed subband signals is M times greater than the passband of the original range due to the action of the phase multiplier. This leads to the fact that the signals of the subbands become critically sampled after step (5), and in addition, there will be no zeros in the spectrum during the transposition of tonal signals.

To avoid trigonometric calculations, i.e. if necessary, calculate new subband signals as

where | ν _K ^(M) (n``) | absolute value ν <sub>K</sub> <sup>(M)</sup> (n``), the following trigonometric relation is used:

On condition

and

the calculations for step (5) can be performed without trigonometric calculations, reducing the complexity of the calculations.

When using transposition with even M, problems may arise for the phase multiplier, depending on the characteristics of the low-pass filter p _o (n). All applicable filters have zeros on the unit circle in the Z plane. Zero on the unit circle creates a 180 degree shift in the phase response of the filter. For even M, the phase multiplier translates these shifts into 360-degree shifts, i.e. phase shifts disappear. Partial tones so arranged in frequency that such phase shifts disappear will lead to interference in the synthesized signal. The worst case in this situation occurs when the partial tone corresponds to a point in frequency corresponding to the top of the first side lobe of the analysis filter characteristic. Depending on the attenuation of this lobe in the amplitude response, these interferences will be more or less audible. As an example, the first side lobe of the filter used for levels 1 and 2 of the ISO / MPEG standard is attenuated by 96 dB, while the attenuation of the first side lobe of the sine window used in the ISO / MPEG level 3 MDCT scheme is only 23 dB. It is clear that this type of interference, when using a sinusoidal window, will be tapped. The following is a solution to this problem, defined as relative phase synchronization.

Filters h ^a _to (n) have linear phase responses. Phase angles Φ _k introduce the relative phase differences between adjacent channels, and zeros on the unit circle introduce a phase shift of 180 degrees in frequency positions that may differ for different channels. By controlling the phase difference between the signals of adjacent subbands, before starting the phase multiplier, it is easy to isolate channels that contain information with inverted phase. For tonal signals, the phase difference is approximately π / 2M according to equation (25) for non-inverted signals and, accordingly, is approximately π (1-1 / 2M) for signals if any of the signals is inverted. The selection of inverted signals can be performed by calculating the scalar product of the signals in adjacent subbands in the form

If the product in equation (32) is negative, the phase difference is greater than 90 degrees, and the condition of phase inversion is present. The phase angles of the complex signals of the subbands are multiplied by M according to the scheme for step (5) and, finally, the signals marked as inverse are subtracted. The method of relative phase synchronization, thus, causes the subband signals shifted by 180 degrees to maintain this shift after phase multiplication and thereby maintain the property of suppressing interference.

Spectrum envelope adjustment.

Most sounds, such as speech and music, are characterized by products of slowly varying envelopes and rapidly changing carrier frequencies with constant amplitude, as described in "The Application of Generalized Linearity to Automatic Gain Control" by TGStockham, Jr, IEEE Tans on Audio and Electroacoustics, Vol.AU-16, No.2, June 1968 and in equation (1).

In split-band perceptual audio encoders, the audio signal is segmented into blocks and split into a plurality of frequency bands using subband filters or time-domain transform frequency domain. In most types of codecs, the signal is sequentially divided into two main signal components for transmission or storage, presentation of the envelope of the spectrum and normalized samples of subbands or coefficients. In the following description, the term “subband samples” or “coefficients” refers to sample values obtained from subband filters, as well as coefficients obtained for conversion from the time domain to the frequency domain. The term “spectral envelope” or “scaling factors” are values for subbands based on a time frame, such as the average or maximum amplitude in each subband, used to normalize samples of subbands. However, the spectral envelope can also be obtained using linear prediction (US patent 5684920). In a typical codec, normalized subband samples require coding at high bit rates (using about 90% of the available bit rate) compared to envelopes that slowly change over time, and thus spectrum envelopes that can be encoded at much lower speeds (using about 10 % available baud rate).

The exact spectral envelope of the duplicated bandwidth is important if the quality of the timbre of the original signal must be preserved. The perceived timbre of a musical instrument or voice is determined mainly by the spectral distribution below the frequency f _lim (boundary), located in the highest octaves of the audible range. Parts of the spectrum above f _lim are thus less important and, accordingly, the fine structures of the upper band obtained by the above transposition methods do not require adjustment, while the coarse structures generally require. To provide such adjustment, it is useful to filter out the spectral representation of the signal to separate the coarse envelope structure from the fine structure.

In the embodiment using DSP-1 according to the present invention, the coarse spectral envelope of the upper band is estimated from the information of the lower band available in the decoder. This assessment is performed by continuously monitoring the envelope of the lower band and adjusting the spectral envelope of the upper band in accordance with special rules. A new way of calculating the envelope uses asymptotes in the logarithmic frequency-amplitude region, which is equivalent to approximating the curve using variable-order polynomials in the linear region. The level and slope of the upper part of the spectrum of the lower band is calculated, and estimates are used to determine the level and slope of one or more segments representing the new envelope of the upper band. The intersection points of the asymptotes are fixed in frequency and act as reference points. However, it is not always necessary, although beneficial, to establish restrictions in order to keep the deviations of the envelope of the upper band at real boundaries. An alternative approach to estimating the spectral envelope is to use vector quantization, VQ, more characteristic spectral envelopes and store them in a conversion table or a set of codes. Vector quantization is performed by training the desired number of vectors on a large amount of training data in this case, the envelopes of the spectrum of the audio signal. Learning is usually done using the Generalized Lloyd Algorithm (see "Vector Quantization and Signal Compression" by A. Gersho, RMGray, Kluwer Academic Publishers, USA 1992, ISBN 0-7923-9181-0) and provides vectors that optimally capture content training data. Considering a set of VQ codes consisting of A envelopes of spectra, trained on B envelopes (B >> A), then A envelopes represent A of the most probable transitions from the envelope of the lower band to the envelope of the upper band based on B observations of a wide variety of sound signals. This, in theory, represents A rules for predicting an envelope based on B observations. When evaluating the new envelope of the upper band spectrum, the original envelope of the lower band is used to search for a set of codes, and part of the upper band of the most exactly matching code set record is used to create a new spectrum of the upper band.

23, the normalization of the subband sample is indicated by 2301, and the spectral envelopes are represented by scaling factors 2305. For purposes of illustration, the transmission to decoder 2303 is shown in parallel form. In the DSP-2 method (FIG. 24), spectrum envelope information is generated and transmitted according to FIG. 23, and only samples of subbands of the lower band are transmitted. The transmitted scaling factors thus cover the full frequency range, while subband samples cover only a limited frequency range excluding the upper band. At the decoder, the samples of the lowband subbands 2401 are transposed (2403) and combined with the received envelope information of the highband spectrum 2405. Thus, the synthesized envelope of the upper band spectrum is identical to the original envelope, while supporting a significant decrease in the data rate.

In some codecs, it is possible to transmit scale factors for the full spectral envelope, while omitting the samples of the upper band subbands, as shown in FIG. Other codec standards establish that scale factors and subband samples should cover the same frequency range, i.e. scale factors cannot be transmitted if subband samples are omitted. In such cases, there are several solutions: information about the spectrum envelope of the upper band can be transmitted in separate frames, and these frames have their own headers and additional protection against errors, followed by data.

Conventional decoders that do not take advantage of the present invention will not recognize these headers and will therefore discard these additional frames. According to the second decision, the information on the spectral envelope of the upper band is transmitted as additional data in the encoded bit stream. However, the available additional data field should be large enough to contain envelope information. In cases where none of the first two solutions is applicable, a third solution can be used, according to which the information of the spectrum envelope of the upper band is hidden in the form of subband samples. Large-scale subband selection factors cover a large dynamic range, usually in excess of 100 dB. Thus, it is possible to set an arbitrary number of scaling factors of subband samples (2505 in FIG. 25) to very small values and transmit scaling factors of the upper band “masked” as subband samples 2501. This method of transmitting scaling factors of the upper band to decoder 2503 provides compatibility with bitstream syntax. Therefore, in this mode arbitrary data can be transmitted. A similar method is known in which information is encoded in a stream of subband samples (US Pat. No. 5,687,191). A fourth solution (FIG. 26) can be applied when the coding system uses Huffman coding or other coding with redundancy (2603). The subband sample for the upper band is then set to zero (2601) or a constant value to achieve high redundancy.

Transient Response Improvement

Transient distortions are a common problem in audio codecs, and similar distortions exist for the present invention. In principle, coupling creates spectral “zeros” or dips corresponding to preliminary and subsequent echoes in the time domain, i.e. false transient before and after “true” transients. Although the P-blocks “fill in zeros” for slowly changing tones, the preliminary and subsequent echoes remain. An improved multiband method is designed to operate on discrete sinusoids, and the number of sinusoids is limited to one per subband. Transients or noise in a subband can be considered as a large number of discrete sinusoids in such a subband. This creates intermodulation distortion. Such distortions are considered as additional sources of quantization noise associated with duplicated channels of the upper band at transient intervals. To improve the subjective quality of the improved multiband method, traditional methods of eliminating preliminary and subsequent echo signals in perceptual audio encoders, for example, adaptive window switching, can be used. Using transient detection provided by a codec or a separate detector and reducing the number of channels under transient conditions leads to the fact that the quantization noise does not exceed the time-dependent masking threshold. Fewer channels are used for transients, while more are used for tones. Such adaptive window switching is typically used in codecs to exploit the trade-off between frequency resolution and time resolution. Various methods can be used in tasks where the size of the filter set is fixed. One approach is to transform the quantization noise over time by linearly predicting in the spectral region. The transposition is then performed on the residual signal, which is the output of the linear prediction filter. After that, the inverse prediction filter is applied simultaneously to the original and spectrally duplicated channels. Another approach uses a compander system (compressor + expander), i.e. dynamic compression of the transition signal before transposition or coding, and additional expansion after transposition. It is also possible to alternately use transposition methods depending on the signal, for example, a transposition method with a set of high-resolution filters is used for stationary signals, and a method for predicting a time-varying search scheme is used for transition signals.

Practical implementation

When using standard signal processors or personal computers with great computing capabilities, it is possible to operate a codec based on chipboard in real time. Such a codec can be made in the form of hardware on a conventional chip. It can also be implemented in various types of systems for storing or transmitting signals, analog or digital, using arbitrary codecs (Fig. 27 and Fig. 28). The DSP-1 method can be integrated into a decoder or provided as an additional post-processing module, hardware or software. The DSP-2 method requires additional modification of the encoder. As shown in FIG. 21, an analog input signal is supplied to an analog-to-digital converter 2701, which generates a digital signal, which is supplied to an arbitrary encoder 2703, on which the source encoding is performed. The signal supplied to the system may be a low-frequency signal of the type whose spectral bands have already been discarded within the audio range or whose spectral bands are discarded in an arbitrary encoder. The resulting lower-band signals are fed to a multiplexer 2705, forming a sequential bit stream that is transmitted or stored (2707). The demultiplexer 2709 reconstructs these signals and feeds them to an arbitrary decoder 2711. The information of the envelope of the spectrum 2715 is evaluated in the decoder 2711 and fed to the DSP-1 unit 2713, which transports the signal of the lower band to the signal of the upper band and creates a broadband signal with a tunable envelope of the spectrum. Finally, the digital broadband signal is converted (2717) into an analog output signal.

The DSP-2 method requires additional modification of the encoder. According to FIG. 28, an analog input signal is supplied to an analog-to-digital converter 2801, generating a digital signal that is supplied to an arbitrary encoder 2803 in which the source encoding is performed. Spectral envelope information is extracted 2805. The resulting signals, lower band subband samples, or coefficients and broadband envelope information are supplied to multiplexer 2807, forming a sequential bit stream that is transmitted or stored (2809). Demultiplexer 2811 reconstructs these signals, samples of subbands of the lower band or coefficients and information of the broadband envelope and feeds them to an arbitrary decoder 2815. Information from envelope of spectrum 2813 is supplied from demultiplexer 2811 to block DSP-2 2817, which transposes the signal of the lower band into the signal of the upper band and Broadband signal With adjustable spectrum envelope. Finally, the digital broadband signal is converted (2819) to an analog output signal.

When only very low transfer rates (the Internet and low-speed telephone modems, AM broadcasting, etc.) are available, monocoding of audio program material is inevitable. To improve the quality of perception and make the sound of the programs more pleasant, a simple “pseudo stereo” generator is created (Fig. 29) by introducing a delay line with taps 2901. This allows you to send signals with a delay of 10 ms and 15 ms with a level of about 06 dB (2903) to each output channel in addition to the original mono signal 2905. This pseudo-stereo generator provides a significant improvement in perception at low computational cost.

The above embodiments illustrate the principles of the present invention, aimed at improving the coding of audio sources. It is clear that modifications and variations of the described devices and details of their implementation will be obvious to specialists in this field of technology. Therefore, the scope of the invention is limited only by the following claims, and not by the specific details presented in the description of its embodiments.

Claims (22)

1. A method for decoding an encoded signal (201, 301), wherein the encoded signal is obtained from the original signal and represents only part of the frequency bands included in the original signal, the frequency content of the encoded signal (201, 301) is represented by samples of subbands for multiple subbands or represented by multiple spectral coefficients, namely, that allocate the first bandpass signal (203, 303), and the first bandpass signal has samples of subbands from a predetermined number of analyzed subbands or has a predetermined number of analyzed spectral coefficients, while the first band signal has a frequency band smaller than the frequency band of the encoded signal (201, 301), transpose (205, 305) samples of subbands from analysis subbands or spectral analysis coefficients included in the band signal (203, 303), into a second band signal (205, 305) having frequency content that is included in the original signal and which is not included in the encoded signal, the second band signal having synthesis subbands or spectral synthesis coefficients, and during the transposition, add analysis subbands to synthesis subbands or add analysis spectral coefficients to synthesis spectral coefficients, while the selected samples of subbands or spectral coefficients included in the second band signal (203, 303) are tuned before or after transposition along the spectrum envelope using information from the spectrum envelope obtained from the original signal or the encoded signal to obtain a dividing spectrum of transposed samples of subbands or tuned spectral coefficients of the spectrum envelope, and combining samples of subbands and adjusted transposed samples of subbands or spectral coefficients and adjusted transposed spectral coefficients to obtain an output signal (209, 309), the output signal having a frequency content including themselves the frequency content of the encoded signal and the frequency content of the second band signal. 2. The method according to claim 1, characterized in that the passband (s) of the tuned transposed samples of subbands or tuned transposed spectral coefficients is set so as not to overlap or only partially overlap the passband (s) of the samples of subbands or spectral coefficients representing the frequency content encoded signal. 3. The method according to claim 1, characterized in that it further evaluates the information of the spectrum envelope of the second strip signal using the encoded signal. 4. The method according to claim 1, characterized in that the encoded signal further includes transmitted information of the envelope of the spectrum in the reconstructed frequency band, which was discarded during the encoding of the original signal, further demultiplexing (2811) the transmitted information of the envelope of the discarded band (s) frequencies of the original signal from the encoded signal. 5. The method according to claim 4, characterized in that the information of the envelope of the spectrum is transmitted in the form of samples of subbands in an arbitrary number of channels of the subbands of the encoded signal, and the amplifications of the mentioned channels of the subbands are set to a low level. 6. The method according to claim 4, characterized in that the information of the envelope of the spectrum is transmitted in the form of scale factors without transmitting the corresponding samples of the subbands in the encoded signal. 7. The method according to claim 4, characterized in that the information of the envelope of the spectrum is transmitted in the form of scale factors and the corresponding samples of the subbands are set to zero or a constant value in the encoded signal. 8. The method according to claim 1, characterized in that the decoded output signal is a monaural audio signal, while further splitting the decoded output signal into two signals, each of which consists of said output signal and its delayed versions to obtain a pseudo stereo signal. 9. The method according to claim 1, characterized in that the signal is filtered by a set of N≥2 bandpass filters, said filters having passbands containing frequencies [f ₁ , ..., f _N ], respectively, for generating N signals sub-bands, while transposition shift the signals of the sub-bands in frequency in the region containing the frequencies M [f ₁ , ..., f _N ], where M ≠ 1 is the transposition coefficient. 10. The method according to claim 9, characterized in that the frequency shift is obtained by modulating the upper sideband (GDP). 11. The method according to claim 1, characterized in that the signal is band-filtered using a set of analysis or conversion filters of such a nature that generate real or complex analysis subband signals of the low-frequency type, and with said transposition, an arbitrary number of channels k of the said set are linked analysis or conversion filters with channels Mk, M ≠ 1, in the set of synthesis or conversion filters, where M is the transposition coefficient, while the set of filters or transformation Do not use when filtering or performing inverse conversion. 12. The method according to claim 11, characterized in that the set of filters is characterized by maximum decimation, while the binding is performed in accordance with the ratio: V _Mk (n) = (- 1) ^{(M-1) kn} V _k (n), where (-1) ^{(M-1) kn} is the correction factor; V _k (n) is the signal of the subband of channel k; V _Mk (n) is the signal of the subband of the channel Mk, while providing compensation for the spectrally inverted signals of the subbands. 13. The method according to claim 1, characterized in that during transposition, the phases of the analysis subband samples or the analysis frequency coefficients from the channels k of the analysis or conversion filter set are connected as the phases of the subband samples associated with the synthesis channels Mk, where M is the transposition coefficient, which is an integer not equal to 1, and k is the channel number, and the amplitudes of the samples of the analysis sub-bands or spectral coefficients of the analysis are linked from the serial channels l of the analysis or conversion filter set as an amp and thereafter, samples of subbands or spectral coefficients associated with successive synthesis channels l + S, where S is an integer not equal to 1, and l is the channel number. 14. The method according to p. 13, characterized in that when the transposition phase of the samples of the sub-analysis channels k is multiplied by a coefficient M before using the said set of synthesis or conversion filters. 15. The method according to item 13, wherein M = K ± ¹ , where K is an integer greater than 1. 16. The method according to claim 11, characterized in that with the said binding, a plurality of values of the transposition coefficient M are used. 17. A method of providing a transposed signal, which is transposed with a coefficient M, from an input signal from which at least one frequency band is dropped, which consists in filtering the input signal using a parallel set of L filters with impulse responses of the form

where k = 0,1, ..., L-1, K is a constant, p ₀ (n) is a model of a low-pass filter of length N, forming a set of L complex signals, sampling with a reduced frequency of the said set of L signals with coefficient L / M to form a set of L complex signals of subbands, multiply the phase angles of the set of L complex signals of subbands by M to generate a new set of signals of subbands, extract the real parts of the said new set of signals of subbands to form a set of L Valid The subband signals, perform oversampling said set of L real subband signals with coefficient L 'to generate the set of valid signals carried filtering said set of valid signals via parallel sets of L' filters with impulse responses of the form

where k = 0,1, ..., L'-1, K 'is a constant, p' ₀ (n) is a low-pass filter model of length N ', forming a set of L' filtered signals, and summing up the said set of L 'filtered signals and input signal to form a transposed signal. 18. The method according to 17, characterized in that the multiplication of the mentioned phase angles and the allocation of the real part to obtain a set of L valid subband signals is performed by calculations when determining the said set of complex subband signals in the form Z _k (n) = R _k (n) + jI _k (n), where R _k (n) and I _k (n) are the real and imaginary parts of Z _k (n), respectively, computing said set of valid subband signals W _k (n) in the form

Where

and M is a positive integer transposition coefficient using the following trigonometric identity: cos (Ma) = cos ^M (a) - (M / 2) sin ² (a) cos ^M-2 (a) + (M / 4) sin ⁴ (a) cos ^M-4 (a) ..., where a = arctan {I _k (n) / R _k (n)}, and the following relations:

and

19. A decoder for decoding an encoded signal (201, 301), wherein the encoded signal is obtained from the original signal or encoded signal and represents only part of the frequency bands included in the original signal, the frequency content of the encoded signal (201, 301) is represented by samples of subbands for multiple subbands or represented by a plurality of spectral coefficients, comprising a selection device for extracting a first band signal (203, 303), wherein the first band signal has subband samples of predetermined the number of analysis subbands or has a predetermined number of spectral analysis coefficients, the first bandpass signal having a frequency band smaller than the frequency content of the encoded signal (201, 301), a transposition device for transposing (205, 305) selected subband samples from the analysis subbands or spectral analysis coefficients included in the first strip signal (203, 303), in the second strip signal (205, 305) having a frequency content that is included in the original signal and which is not included in the a doped signal, wherein the second bandpass signal has synthesis subbands or spectral synthesis coefficients, and the transposition includes attaching analysis subbands to synthesis subbands or attaching analysis spectral coefficients to synthesis spectral coefficients, wherein selected samples of subbands or spectral coefficients included in the second bandpass signal (203, 303), before or after performing the transposition, adjust (207, 307) according to the spectral envelope using the information of the spectral envelope obtained one of the original signal or the encoded signal, to obtain transposed samples of the subband adapted to the spectrum envelope, or transposed spectral coefficients adjusted to the spectral envelope, and a combiner for combining the subband samples and the adjusted transposed samples of the subbands or spectral coefficients and the adjusted transposed spectral coefficients to obtain the output signal ( 209, 309), wherein the output signal has a frequency content including frequencies th content of the encoded signal and the second frequency band signal contents. 20. Device for providing a transposed signal, which is transposed with a coefficient M, from an input signal from which at least one frequency band is dropped, containing a filter for filtering the input signal using a parallel set of L filters with impulse responses of the form

where k = 0,1, ..., L-1, K is a constant; p ₀ (n) is a low-pass filter model of length N; M is a coefficient forming a set of L complex signals, a sampling device with a reduced frequency for sampling with a reduced frequency of said set of L signals with an L / M coefficient for generating a set of L complex signals of subbands, a multiplier for multiplying phase angles of said set of complex signals of subbands on M to form a new set of subband signals, a selection device for extracting the real parts of said new set of subband signals to form a set of L actions itelnyh subband signals, the sampling device with high frequency for oversampling said set of L real subband signals with coefficient L 'to generate the set of valid signals, a filter for filtering said set of valid signals via parallel sets of L' filters with impulse responses of the form

where k = 0,1, ..., L'-1, K 'is a constant, p' ₀ (n) is a low-pass filter model of length N ', forming a set of L' filtered signals, and an adder to summarize the said set from L 'filtered signals and an input signal to form a transposed signal. Priorities: 06/10/1997 - paragraphs 1-16 and 19; 01/30/1998 - pp. 17, 18 and 20.

Images