ES2818562T3 - Audio decoder and decoding procedure - Google Patents

Abstract

A method for representing a second presentation of audio channels or objects as a data stream, the method comprising the steps of: (a) providing base signals, said base signals representing a first presentation of the audio channels or objects; (b) providing transformation parameters, said transformation parameters being provided to transform the base signals of said first presentation into output signals of said second presentation, said transformation parameters including at least specified high-frequency transformation parameters for a upper frequency band and low-frequency transform parameters specified for a lower frequency band, the low-frequency transform parameters including a set of multi-tap convolution matrix parameters to convolve low-frequency components of the base signals with the low-frequency transformation parameters to produce convoluted low-frequency components and the high-frequency transformation parameters including a set of parameters from a stateless matrix to multiply high-frequency components of the base signals with the high-frequency transformation parameters to produce multiplied high-frequency components; the first presentation being provided for loudspeaker playback and the second presentation being provided for headphone playback, or vice versa; and (c) combining said base signals and said transformation parameters to form said data stream.

Description

DESCRIPTION

Audio decoder and decoding procedure

Related request reference

This application claims the benefit of US provisional application number 62 / 209,742, filed August 25, 2015, and European patent application number 15189008.4, filed October 8, 2015. Technical sector of the invention

The present invention relates to the technical field of signal processing and, in particular, discloses a system for the efficient transmission of audio signals having spatialization components.

Background of the invention

No discussion of the background art throughout the specification should be construed as an admission that such technique is widely known or is part of common general knowledge in the technical field.

The creation, encoding, distribution and playback of audio content are typically carried out in a channel-based format, that is, a specific target playback system is contemplated for content across the entire content ecosystem. Examples of such audio formats of target playback systems are mono, stereo, 5.1,7.1, and the like.

If content is to be played on a different playback system than intended, a downmix or upmix process can be applied. For example, 5.1 content can be played back on a stereo playback system using specific downmix equations. Another example is the playback of stereo encoded content over the 7.1 speaker system, which may comprise a so-called upmix process, which may or may not be guided by information present in the stereo signal. A suitable system for upmixing is Dolby Pro Logic from Dolby Laboratories Inc (Roger Dressler, "Dolby Pro Logic Surround Decoder, Principles of Operation", www.Dolby.com).

When playing stereo or multichannel content over headphones, it is often desirable to simulate a multichannel speaker setup using head-related impulse responses (HRIR) or binaural room impulse responses (BRIR). impulse responses), which simulate the acoustic path from each speaker to the eardrums, in an anechoic or echoic (simulated) environment, respectively. In particular, audio signals can be convolved with HRIR or BRIR to restore inter-aural level differences (ILD), inter-aural time differences (ITD), and spectral inputs that allow the listener to determine the position of each individual channel. Simulating an acoustic environment (reverberation) also helps to achieve a certain perceived distance.

Sound source localization and virtual speaker simulation

When stereo, multichannel, or object-based content is to be played back on headphones, it is often desirable to simulate a multichannel speaker configuration or set of discrete virtual acoustic objects, using head-related impulse response (HRIR) convolution. ), or Binaural Room Impulse Responses (BRIR), which simulate the acoustic path from each speaker to the eardrums, in an anechoic or echoic (simulated) environment, respectively.

In particular, audio signals can be convolved with HRIR or BRIR to restore inter-aural level differences (ILD), inter-aural time differences (ITD), and spectral inputs that allow the listener to determine the position of each individual object or channel. The simulation of an acoustic environment (early reflections and after reverberation) helps to achieve a certain perceived distance.

Turning to Figure 1, a schematic overview of the processing flow for delivering two channel or object signals xi 13, 11 is shown 10, which are read from a content storage 12 for processing by 4 HRIR, for example, 14 The output of the HRIRs are then added 15, 16, for each channel signal, in order to produce headphone speaker outputs to reproduce for a listener through the headphones 18. The basic principle of the HRIR, is explained, for example, in the document by Wightman et al (1989). The HRIR / BRIR convolution approach has several drawbacks, one of these being the substantial amount of processing that is required for playback on headphones. The HRIR or BRIR convolution has to be applied for each input channel or object separately, and therefore usually the complexity grows linearly with the number of channels or objects. Since headphones are commonly used in conjunction with battery-powered portable devices, high computational complexity is undesirable as it will substantially reduce battery life. Also, with the introduction of audio content based on objects, which can consist of more than 100 simultaneously active objects, the complexity of the HRIR convolution can be substantially greater than for traditional channel-based content.

Parametric coding techniques

Computational complexity is not the only problem for the provision of content based on channels or objects, within an ecosystem that involves creation, distribution and reproduction of content. In many practical situations, and especially for mobile applications, the data rate available for content delivery is severely limited. Consumers, broadcasters and content providers have been delivering stereo (two channel) audio content, using lossy perceptual audio codecs, with typical bit rates between 48 and 192 kbits / s. These conventional channel-based audio codecs, such as MPEG-1 layer 3 (Brandenberg et al., 1994), MPEG AAC (Bosi et al., 1997), and Dolby Digital (Andersen et al., 2004) have a rate of bits that scale roughly linearly with the number of channels. As a result, supplying tens or even hundreds of objects results in bit rates that are impractical or even infeasible for consumer distribution purposes.

To enable delivery of complex, object-based content at bit rates that are compatible with the bit rates required for delivery of stereo content using conventional perceptual audio codecs, the last of each has undergone study and development. called parametric procedures. These parametric procedures allow the reconstruction of a large number of channels or objects from a small number of base signals. These base signals can be transported from the sender to the receiver using conventional audio codecs, augmented with additional (parametric) information to allow reconstruction of the original channels or objects. Examples of such techniques are Parametric Stereo (Schuijers et al., 2004), MPEG Surround (Herre et al., 2008), and MPEG spatial audio object encoding (Herre et al., 2012).

An important aspect of techniques such as parametric stereo and MPEG Surround is that these procedures are aimed at a parametric reconstruction of a single predetermined presentation (eg, stereo speakers in parametric stereo, and 5.1 speakers in MPEG Surround). In the case of MPEG Surround, a headphone virtualization element may be integrated into the decoder which generates a virtual 5.1 speaker configuration for headphones, where the virtual 5.1 speakers correspond to the 5.1 speaker configuration for loudspeaker playback. Consequently, these displays are not independent because the headphone display represents the same (virtual) speaker layout as the speaker display. On the other hand, MPEG spatial audio object encoding is aimed at the reconstruction of objects that require subsequent delivery.

Turning now to FIG. 2, a parametric system 20 supporting channels and objects will be described in an overview. The system is divided into parts of encoder 21 and decoder 22. Encoder 21 receives channels and objects 23 as inputs, and generates a downmix 24 with a limited number of base signals. Additionally, a number of object / channel reconstruction parameters 25 are calculated. A signal encoder 26 encodes the base signals from the down mixer 24, and includes the calculated parameters 25, as well as object metadata 27 that indicates how they should be deliver the objects in the resulting bitstream.

The decoder 22 first decodes 29 the base signals, followed by the reconstruction of channels and / or objects 30 with the help of the transmitted reconstruction parameters 31. The resulting signals can be reproduced directly (if they are channels) or can be delivered 32 (if they are objects). For the second case, each reconstructed object signal is delivered based on its associated object metadata 33. An example of such metadata is a position vector (for example, an x, y, and z coordinate of the object in a three-dimensional coordinate system) .

Decoder Matrix

Reconstruction of objects and / or channels 30 can be achieved by time and frequency variation matrix operations. If the decoded base signals 35 are called z ^s [n], where s is the base signal index, and n the sample index, the first stage usually comprises the transformation of the base signals by means of a transform or bank of filters.

A wide range of transforms and filter banks can be used, such as a Discrete Fourier Transform (DFT), a Modified Discrete Cosine Transform (MDCT), or a quadrature mirror filter bank (QMF, Quadrature Mirror Filter). The output of such a transform or of the filter bank is called Z ^s [k, b], where p is the sub-band or spectral index, and k is the time of the frame, interval or sub-band or the index of the sample.

In most cases, the subbands or spectral indices are mapped to a smaller set of parametric bands p that share common object / channel reconstruction parameters. This can be stated as that be B (p). In other words, B (p) represents a set of consecutive subbands b belonging to a parametric band index p. Conversely, p (b) refers to the index of the parametric band to which sub-band b is mapped. The objects or channels reconstructed in the transform domain or sub-band Y ^j are obtained at continuation by matrix signals Z ⁱ with matrices M [p (b)]:

Yi [k, b] Zx [k, b] - = M [p (b)]

Yj [k, b] Zs [k, b].

The reconstructed object and / or channel signals in the time domain yj [n] are then obtained by an inverse transform or synthesis filter bank.

The above process is typically applied to a certain limited range of sub-band samples, intervals, or k-frames. In other words, matrices M [p (b)] are usually updated / modified over time. For simplicity of notation, these updates are not denoted here. However, it is considered that the processing of a set of samples k associated with a matrix M [p (b)] can be a variable process in time.

In some cases, where the number of reconstructed signals J is significantly greater than the number of base signals S, it is often useful to use optional decorrelator outputs D ^m [k, b] that operate on one or more base signals that can be included in the reconstructed output signals.

Zi [k, b] "

Yi [k, b]

Zs [k, b]

= M [p (b)]

Say [k, b]

Yj [k, b]

DM [k, b] _

Figure 3 shows schematically in greater detail one kind of channel or object reconstruction unit 30 of Figure 2. Input signals 35 are processed first by analysis filter banks 41, followed by optional decorrelation (D1, D2) 44 and matrixization 42, and a synthesis filter bank 43. The manipulation of the matrix M [p (£>)] is controlled by reconstruction of parameters 31.

Minimum mean square error (MMSE) prediction for object / channel reconstruction

Although there are different strategies and procedures for reconstructing objects or channels from a set of base signals Z ^s [k, b], a particular procedure is often called a least mean square error (MMSE) predictor, which uses correlations and covariance matrices to obtain matrix coefficients M that minimize the L2 norm between a desired and reconstructed signal. For this method, it is assumed that the base signals z ^s [n] are generated in the encoder down mixer 24 as a linear combination of channel signals or input object x ⁱ [n]:

For channel-based input content, the amplitude panning gains g ⁱ , ^s are typically constant, whereas for object-based content, where the predicted position of an object is provided by time-varying object metadata, the gains g ⁱ , ^s can therefore be variable with time. This equation can also be formulated in the sub-band or transformed domain, in which case a set of gains g ⁱ , ^s [k] is used for each band / frequency band k and, thus, the gains g ⁱ , ^s [k] can be made variable in frequency:

Zs [k, b] = ^ g i .J k j X j ^ b]

i

Decoder matrix 42, ignoring the decorrelators for now, produces:

or, in matrix formulation, omitting the sub-band index b and the parametric band index p for clarity:

Y = ZM

Z = XG

The criterion for calculating the matrix coefficients M using the encoder is to minimize the root mean square error E which represents the squared error between the decoder outputs Yj and the original input channels / objects Xj:

The matrix coefficients that minimize E are given below in matrix notation by:

M = (Z * Z d) -1Z * X

being epsilon a regularization constant, and (*) the complex conjugate transpose operator. This operation can be performed for each parametric band p independently, producing a matrix M [p (b)].

Least Root Mean Square Error (MMSE) Prediction for Rendering Transformation

In addition to the reconstruction of objects and / or channels, parametric techniques can be used to transform one representation into another representation. An example of such a rendering transformation is converting a stereo mix intended for loudspeaker reproduction to a binaural representation for headphones, or vice versa.

Figure 4 shows the flow of control for a procedure 50 for such a representation transformation. Channel or object audio is first processed in an encoder 52 by a hybrid quadrature mirror filter analysis bank 54. A speaker delivery matrix C is calculated and applied 55 to the object signals X ⁱ stored in the storage medium 51, based on the object metadata using amplitude pan techniques, resulting in a Z ^s stereo speaker display. This speaker presentation can be encoded with an audio encoder 57.

Additionally, a binaural delivery matrix H is generated and applied 58 using an HRTF database 59. This matrix H is used to calculate binaural signals Y ^j that allow the reconstruction of a binaural mix using the stereo speaker mix as input. The matrix coefficients M are encoded by the audio encoder 57.

The transmitted information is transmitted from the encoder 52 to the decoder 53, where 61 is unpacked to include components M and Z ^s. If loudspeakers are used as the playback system, the loudspeaker presentation is reproduced using Z ^s channel information, thereby discarding the M matrix coefficients. For headphone playback, on the other hand, the loudspeaker presentation is transformed 62 first in a binaural presentation applying the time and frequency variable matrix M before hybrid QMF synthesis and playback 60.

If the desired binaural output from array element 62 is described in matrix notation as:

Y = XH

then the matrix coefficients M can be obtained at encoder 52 as:

In this application, the H encoder matrix coefficients applied at 58 usually have complex values, for example, having a delay or phase shift element, to allow the resetting of interaural time differences, which are perceptually highly relevant to location from the sound source with headphones. In other words, the binaural delivery matrix H has complex values, and therefore the transformation matrix M has complex values. For perceptually transparent resetting of sound source localization inputs, a frequency resolution that reproduces the frequency resolution of the human auditory system has been shown to be desirable (Breebaart 2010).

In the previous sections, a least root mean square error criterion is used to determine the matrix coefficients M. Without loss of generality, other well-known criteria or procedures can be used analogously, to calculate the matrix coefficients, in order to substitute or complement the principle of the least mean square error. For example, matrix coefficients M can be calculated using higher order error terms, or by minimizing an L1 norm (eg, minimum absolute deviation criterion). In addition, various procedures may be used including non-negative factoring or optimization techniques, non-parametric estimators, maximum probability estimators, and the like. Additionally, matrix coefficients can be calculated using iterative or gradient-down processes, interpolation procedures, heuristics procedures, dynamic programming, machine learning, fuzzy optimization, simulated annealing or closed-form solutions, and analysis-by-synthesis techniques. Last but not least, matrix coefficient estimation can be restricted in various ways, for example by limiting the range of values, regularization terms, overlapping energy conservation requirements, and the like.

Transform and filter bank requirements

Depending on the application, and whether objects or channels are to be reconstructed, certain requirements may be superimposed on the filter bank or transform frequency resolution for the filter bank unit 41 of Figure 3. For the most part For practical applications, the frequency resolution is matched with the assumed resolution of the human auditory system, to provide the best perceived audio quality for a given bit rate (determined by the number of parameters) and a given complexity. It is known that the human auditory system can be considered as a filter bank with a non-linear frequency resolution. These filters are called critical bands (Zwicker, 1961) and are approximately logarithmic in nature. At low frequencies, the critical bands are less than 100 Hz wide, while at high frequencies, the bands can be found to be greater than 1 kHz.

This non-linear behavior can pose challenges in filter bank design. Transforms and filter banks can be implemented very efficiently using symmetries in their processing structure, as long as the resolution in frequency is constant across the frequency.

This implies that the length of the transform, or the number of sub-bands, will be determined by the critical bandwidth at low frequencies, and mapping of DFT sections on so-called parametric bands can be used to reproduce a non-linear frequency resolution. . This mapping process is explained, for example, in Breebaart et al., (2005) and Breebaart et al., (2010). A drawback of this approach is that a very long transform is required to satisfy the critical low frequency bandwidth limitation, while the transform is relatively long (or inefficient) at high frequencies. An alternative solution to improve frequency resolution at low frequencies is to use a hybrid filter bank structure. In this structure, a cascade of two filter banks is used, in which the second filter bank improves the resolution of the first, but only in a few of the lower sub-bands (Schuijers et al., 2004).

Figure 5 shows a kind of hybrid filter bank structure 41, similar to that established in the documents of Schuijers et al. The input signal z [n] is first processed by a complex-valued Quadrature Mirror Filter (CQMF) analysis bank 71. The signals are then downsampled by a Q factor. , for example, 72, with the result of sub-band signals Z [k, b] with k being the sub-band sample index and b the sub-band frequency index. Furthermore, at least one of the resulting sub-band signals is processed by a second filter bank (Nyquist) 74, while the remaining sub-band signals are delayed 75 to compensate for the delay introduced by the filter bank of Nyquist. In this particular example, the filter bank cascade results in 8 subbands (b = 1, ..., 8) that are mapped onto 6 parametric bands p = (1, ..., 6) with a resolution of non-linear frequency. Bands 76 merging together to form a single parametric band (p = 6).

The benefit of this approach is less complexity compared to using a single filter bank with many more (narrower) subbands. However, the drawback is that the overall system delay increases significantly, and consequently the memory utilization is also significantly higher, which causes an increase in power consumption.

Limitations of the prior art

Returning to Figure 4, it is suggested that the prior art uses the concept of matrixing 62, possibly augmented with the use of decorrelators, to reconstruct the channels, objects or display signals Yj from a set of base signals Zs. This leads to the following matrix formulation to describe the prior art in a generic way:

The matrix coefficients M are transmitted directly from the encoder to the decoder, or are obtained from localization parameters of the sound sources, for example as described in the document by Breebaart et al 2005 for parametric or Herre stereo coding. et al., (2008) for multichannel decoding. Furthermore, this approach can also be used to restore phase differences between channels using complex value matrix coefficients (see Breebaart at al., 2010, and Breebaart., 2005, for example).

As shown in Figure 6, in practice, using complex value matrix coefficients implies that a desired delay 80 is represented by a piecewise constant phase approximation 81. Assuming that the desired phase response is a pure delay 80 with a phase decreasing linearly with frequency (dashed line), the complex value matrix operation of the prior art results in a constant piecewise approximation 81 (solid line). The approximation can be improved by increasing the resolution of the matrix M. However, this has two major disadvantages. It requires an increase in filter bank resolution, resulting in higher memory utilization, higher computational complexity, longer latency, and therefore higher power consumption. It also requires more parameters to be sent, causing a higher bit rate.

All of these disadvantages are especially troublesome for mobile and battery-powered devices. It would be advantageous if a more optimal solution were available.

Compendium of the invention

An object of the invention, in its preferred form, is to provide an improved way of encoding and decoding audio signals for reproduction in different presentations.

In accordance with a first aspect of the present invention, there is provided a method for representing a second presentation of audio channels or objects as a data stream, as defined in claim 1. In some embodiments, the parameters of Transformation associated with higher frequencies do not modify the phase of the signal, while for lower frequencies, the transformation parameters modify the phase of the signal. The filter coefficient set can function to process a multi-tap convolution matrix. The set of filter coefficients is used to process a low frequency band.

The set of base signals and the set of transformation parameters are preferably combined to form the data stream. The transform parameters may include high-frequency audio matrix coefficients for matrix manipulation of a high-frequency portion of the base signal set. In some embodiments, for an intermediate frequency part of the high frequency part of the base signal set, the matrix manipulation may preferably include complex value transformation parameters.

In accordance with another aspect of the present invention, there is provided a decoder for decoding an encoded audio signal, as defined in independent claim 8.

The matrix multiplication unit can modify the phase of the low-frequency components of the audio base signals. In some embodiments, the multi-tap convolution matrix transformation parameters are preferably complex values. The high-frequency audio transformation parameters also preferably have complex values. The transformation parameter set may further comprise real valued higher frequency audio transformation parameters. In some embodiments, the decoder may further include filters to separate the audio base signals into the low-frequency components and the high-frequency components.

In accordance with another aspect of the present invention, a method of decoding an encoded audio signal is disclosed, as defined in independent claim 10.

In some embodiments, the encoded signal may comprise multiple time slots, the method preferably further including the steps of: interpolating transformation parameters from multiple time slots of the encoded signal, to produce interpolated transformation parameters, including audio transformation parameters of low frequency interpolated; and convolving multiple time slots of the low-frequency components of the audio base signals with the interpolated low-frequency audio transform parameters, to produce multiple time slots of the convolved low-frequency components.

The set of encoded audio signal transformation parameters may preferably be time variable, and the method may further preferably include the steps of: convolving the low-frequency components with the low-frequency transformation parameters for multiple time slots to produce multiple sets of intermediate convoluted low-frequency components; interpolate the multiple sets of intermediate convolved low-frequency components to produce the convolved low-frequency components.

Interpolation can use an overlap and add procedure of the multiple sets of intermediate convolved low-frequency components.

Brief description of the drawings

Embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

Figure 1 shows a schematic overview of the HRIR convolution process for two source objects, each channel or object being processed by an HRIR / BRIR pair;

Figure 2 schematically shows a generic parametric coding system supporting channels and objects;

Figure 3 schematically shows one kind of channel or object reconstruction unit 30 of Figure 2, in greater detail;

Figure 4 shows the data flow of a method for transforming a stereo speaker display into a binaural headphone display;

Figure 5 schematically shows the structure of the hybrid analysis filter bank, according to the prior art; Figure 6 shows a comparison of the desired (dashed line) and actual (solid line) phase response obtained with the prior art;

Figure 7 schematically shows an exemplary filter bank and encoder parameter mapping system in accordance with one embodiment of the invention;

Figure 8 schematically shows the filter bank and parameter mapping of the decoder, according to one embodiment; Y

Figure 9 shows an encoder for transformation of stereo to binaural presentations.

Figure 10 schematically shows a decoder for transforming stereo to binaural presentations.

References

Wightman, F. L., and Kistler, D. J. (1989). "Headphone simulation of free-field listening. I. Stimulus synthesis," J. Acoust. Soc. Am. 85, 858-867.

Schuijers, Erik, et al. (2004). "Low complexity parametric stereo coding." Audio Engineering Society Convention 116. Audio Engineering Society.

Herre, J., Kjorling, K., Breebaart, J., Faller, C., Disch, S., Purnhagen, H., ... & Chong, K. S. (2008). "MPEG surroundthe ISO / MPEG standard for efficient and compatible multichannel audio coding". Journal of the Audio Engineering Society, 56 (11), 932-955.

Herre, J., Purnhagen, H., Koppens, J., Hellmuth, O., Engdegárd, J., Hilpert, J., & Oh, H. O. (2012). "MPEG Spatial Audio Object Coding-the ISO / MPEG standard for efficient coding of interactive audio scenes." Journal of the Audio Engineering Society, 60 (9), 655-673.

Brandenburg, K., & Stoll, G. (1994). "ISO / MPEG-1 audio: A generic standard for coding of high-quality digital audio". Journal of the Audio Engineering Society, 42 (10), 780-792.

Bosi, M., Brandenburg, K., Quackenbush, S., Fielder, L., Akagiri, K., Fuchs, H., & Dietz, M. (1997). "ISO / IEC MPEG-2 advanced audio coding". Journal of the Audio engineering society, 45 (10), 789-814.

Andersen, R. L., Crockett, B. G., Davidson, G. A., Davis, M. F., Fielder, L. D., Turner, S. C., ... & Williams, P. A. (2004, October). "Introduction to Dolby digital plus, an enhancement to the Dolby digital coding system". In Audio Engineering Society Convention 117. Audio Engineering Society.

Zwicker, E. (1961). "Subdivision of the audible frequency range into critical bands (Frequenzgruppen)". The Journal of the Acoustical Society of America, (33 (2)), 248.

Breebaart, J., van de Par, S., Kohlrausch, A., & Schuijers, E. (2005). "Parametric coding of stereo audio". EURASIP Journal on Applied Signal Processing, 2005, 1305-1322.

Breebaart, J., Nater, F., & Kohlrausch, A. (2010). "Spectral and spatial parameter resolution requirements for parametric, filter-bank-based HRTF processing". Journal of the Audio Engineering Society, 58 (3), 126-140.

Breebaart, J., van de Par, S., Kohlrausch, A., & Schuijers, E. (2005). "Parametric coding of stereo audio". EURASIP Journal on Applied Signal Processing, 2005, 1305-1322.

Detailed description

This embodiment provides a method for reconstructing objects, channels or 'displays' from a set of base signals that can be applied to filter banks with low frequency resolution. An example is the transformation of a stereo presentation to a binaural presentation intended for headphone playback, which can be applied without a Nyquist (hybrid) filter bank. The reduced decoder frequency resolution is compensated for by a convolution, multi-shot matrix. This convolution matrix it requires only a few taps (eg two) and, in practical cases, is required only at low frequencies. This procedure (1) reduces the computational complexity of a decoder, (2) reduces the memory utilization of a decoder, and (3) reduces the parametric bit rate.

In the preferred embodiment, a system and method are disclosed to overcome undesirable computational complexity and memory requirements on the decoder side. This is implemented by arranging a high frequency resolution in an encoder, using a limited (lower) frequency resolution in the decoder (for example, using a frequency resolution that is significantly worse than that used in the corresponding encoder), and using a matrix Multi-take (convolution) to compensate for the decoder's reduced frequency resolution.

According to the invention, since a high-frequency matrix resolution is required only at low frequencies, a multi-tap (convolution) matrix is used at low frequencies, while a conventional (stateless) matrix is used for frequencies. greater. In other words, at low frequencies, the matrix represents a set of FIR filters operating on each input and output combination, while at high frequencies, a stateless matrix is used.

Filter bank and encoder parameter mapping

FIG. 7 shows an exemplary encoder parameter mapping and filter bank system in accordance with one embodiment. In this example embodiment 90, 8 subbands (b = 1, ..., 8), for example 91, are initially generated by means of a hybrid (cascaded) filter bank 92 and a Nyquist filter bank 93. Next, the first four sub-bands are mapped 94 onto the same parametric band (p = 1) to calculate a convolution matrix M [k, p = 1], for example, the matrix now has an additional index k . The remaining sub-bands (b = 5, ..., 8) are mapped onto parametric bands (p = 2.3) using stateless matrices M [p (b)] 95, 96.

Filter bank and decoder parameter mapping

Figure 8 shows the corresponding decoder parameter mapping and filter bank system 100, by way of example. In contrast to the encoder, no Nyquist filter bank is present, nor is there any delay to compensate for the Nyquist filter bank delay. The decoder analysis filter bank 101 generates only 5 sub-bands (b = 1, ..., 5), for example 102, which are downsampled by a Q factor. The first sub-band is processed by a convolution matrix M [k, p = 1] 103, while the remaining bands are processed by stateless matrices 104, 105, according to the prior art.

Although the above example applies a Nyquist filter bank in encoder 90 and a corresponding convolution matrix only for the first CQMF sub-band in decoder 100, the same process can be applied to a multitude of sub-bands, without necessarily limiting to only the lowest sub-band or sub-bands.

Encoder realization

One embodiment that is especially useful is in transforming the loudspeaker presentation to a binaural presentation. Figure 9 shows an encoder 110 using the proposed method for presentation transformation. A set of input channels or objects x ⁱ [n] is first transformed using filter bank 111. Filter bank 111 is a Hybrid Complex Quadrature Mirror Filter (HCQMF) bank, but is they can also use other filter bank structures. The resulting subband representations X; [k, b] are processed twice 112, 113.

First 113, to generate a set of base signals Z ^s [k, b] 113 destined for the encoder output. This output can be generated, for example, using amplitude pan techniques, such that the resulting signals are intended for loudspeaker reproduction.

Secondly 112, to generate a set of desired transformed signals Y ^j [k, b] 112. This output can be generated, for example, using HRIR processing, such that the resulting signals are intended for playback on headphones. Such HRIR processing can be used in the filter bank domain, but it can also work in the time domain by means of HRIR convolution. HRIRs are obtained from a database 114.

The convolution matrix Z ^s [k, b] is then obtained by feeding the base signals Z ^s [k, b] through a derived delay line 116. Each of the taps of these delay lines serves as input additional to an MMSE predictor stage 115. The MMSE predictor stage calculates the convolution matrix M [k, p] that minimizes the error between the desired transformed signals Y ^j [k, b] and the output of the decoder 100 of the figure 8, applying convolution matrices. It follows then that the matrix coefficients M [k, p] are given by:

In this formulation, the matrix Z contains all the inputs of the derived delay lines.

Taking initially the case for the reconstruction of a signal Y [k] for a given sub-band b, where there are A inputs from the derived delay lines, we have:

The resulting convolution matrix coefficients M [k, p] are quantized, encoded, and transmitted along with the base signals zs [n]. The decoder can then use a convolution process to reconstruct Y [k, b] from input signals Zs [k, b].

Y [k, b] = ^ Zs [k, b] * ms [., B]

S

or written another way, using a convolution expression:

^A —1

Y [k, b] = ^ ^ Zs [k - a, b] ms [a, b]

s a = 0

The convolution approach can be mixed with a linear array process (stateless).

Another distinction can be made between stateless arrays of complex values and real values. At low frequencies (usually below 1 kHz), the convolution process (A> 1) is preferable to allow reconstruction of interchannel properties in line with a perceptual frequency scale. At medium frequencies, up to 2 or 3 kHz, the human auditory system is sensitive to interchannel phase differences, but does not require high frequency resolution for interchannel phase reconstruction. This implies that a single complex-valued, single-shot (stateless) array suffices. For higher frequencies, the human auditory system is virtually insensitive to the fine-structure phase of the waveform, and a stateless, real-value matrix is sufficient. With increasing frequencies, the number of filter bank outputs mapped onto a parametric band is typically increased to reflect the non-linear frequency resolution of the human auditory system.

In another embodiment, the first and second displays at the encoder are swapped, for example, the first display is intended for headphone playback, and the second display is intended for speaker playback. In this embodiment, the loudspeaker presentation (second presentation) is generated by applying time-dependent transformation parameters in at least two frequency bands to the first presentation, where it is further specified that the transformation parameters include a set of coefficients filter for at least one of the frequency bands.

In some embodiments, the first display can be temporarily divided into a series of segments, with a separate set of transformation parameters for each segment. In further refinement, when no transformation parameters are available, the parameters can be interpolated from previous coefficients.

Realization of the decoder

Figure 10 shows an embodiment of the decoder 120. The input bit stream 121 is divided into a base signal bit stream 131 and transform parameter data 124. Next, a base signal decoder 123 decodes the signals. of base z [n], which are then processed by a bank of analysis filters 125. The signals resulting from the frequency domain Z [k, b] with sub-band b = 1, ..., 5 are processed by matrix multiplication units 126, 129 and 130. In particular, matrix multiplication unit 126 applies a convolution matrix of complex values M [k, p = 1] to the frequency domain signal Z [k, b = 1]. In addition, the matrix multiplier unit 129 applies one-shot, complex-valued matrix coefficients, M [p = 2], to the signal Z [k, b = 2]. Finally, the matrix multiplication unit 130 applies real value matrix coefficients M [p = 3] to signals in the frequency domain Z [k, b = 3 ... 5]. The output signals from the matrix multiplication unit are converted into time domain output 128 by means of a synthesis filter bank 127. References az [n], Z [k], etc., refer to to the set of base signals, not to a specific base signal. Therefore, z [n], Z [k], etc., can be interpreted as z ^s [n], Z ^s [k], etc., where 0 <s < N and N is the number of base signals.

In other words, the matrix multiplication unit 126 determines subband b = 1 output samples of an output signal Y [k] from weighted combinations of current subband b = 1 samples of signals. base Z [k] and previous samples of sub-band b = 1 of base signals Z [k], (for example Z [ka], where 0 <a <A, and A is greater than 1). The weights used to determine the output samples of the subband b = 1 of the output signal Y [k] correspond to the convolution matrix of complex values M [k, p = 1] for the signal.

In addition, the array multiplier unit 129 determines subband b = 2 output samples of the output signal Yj [k] from weighted combinations of current subband b = 2 samples of the base signals. Z [k]. The weights used to determine the output samples of the sub-band b = 2 of the output signal V, [k] correspond to the one-shot matrix coefficients of complex values, M [p = 2].

Finally, the matrix multiplier unit 130 determines output samples of the sub-bands b = 3 ... 5 of the output signal Yj [k] from weighted combinations of current samples of sub-bands b = 3. ..5 of the base signals Z [k]. The weights used to determine output samples of the sub-bands b = 3 ... 5 of the output signal Y [k] correspond to the matrix coefficients of real values M [p = 3].

In some cases, the base signal decoder 123 may operate on signals at the same resolution in frequency as that provided by the analysis filter bank 125. In such cases, the base signal decoder 125 may be configured to deliver signals. in the frequency domain Z [k] rather than signals in the time domain z [n], in which case the analysis filter bank 125 can be omitted. Also, in some cases, it may be preferable to apply matrix coefficients single tap, from complex values, rather than real-value matrix coefficients, to signals in the Z frequency domain [k, b = 3 .... 5].

In practice, the matrix coefficients M can be updated over time; for example, associating individual frames of the base signals with matrix coefficients M. Alternatively or additionally, the matrix coefficients M are complemented with timestamps, which indicate in which time or interval of base signals z [n] they should be apply the matrices. To reduce the transmission bit rate associated with matrix updates, the number of updates is ideally limited, resulting in a time-sparse distribution of matrix updates. Such infrequent array updates require dedicated processing to ensure smooth transitions from one instance of the array to the next. The M matrices can be provided associated with specific time slots (frames) and / or frequency zones of the base Z signals. The decoder can use various interpolation procedures to ensure a smooth transition from subsequent instances of the M matrix with the weather. An example of such an interpolation procedure is to compute overlapping, windowed frames of the Z signals and to compute a corresponding set of Y output signals for each of said frames using the matrix coefficients M associated with that particular frame. Subsequent screens can then be added using an overlap-add technique that provides a smooth crossfade transition. Alternatively, the decoder may receive timestamps associated with matrices M, which describe the desired matrix coefficients at specific time instances. For audio samples between timestamps, the matrix coefficients of the M matrix can be interpolated using linear, cubic, band-limited, or other means, so that the interpolation ensures smooth transitions. Along with interpolation over time, similar techniques can be used to interpolate matrix coefficients across frequency.

Therefore, the present document describes a method (and a corresponding encoder 90) for representing a second presentation of X audio channels or objects; as a data stream to be transmitted or provided to a corresponding decoder 100. The method comprises the step of providing base signals Z ^s , said base signals representing a first presentation of the audio channels or objects X ⁱ . As outlined above, the base signals Z ^s can be determined from channels or audio objects X ⁱ using first delivery parameters G (i.e. notably using a first gain matrix, for example for pan panning). amplitude). The first presentation may be intended for loudspeaker playback or headphone playback. On the other hand, the second presentation may be intended for headphone playback or loudspeaker playback. Thus, a transformation from loudspeaker playback to headphone playback (or vice versa) can be carried out.

The method further comprises providing transformation parameters M (notably, one or more transformation matrices), said transformation parameters M being provided to transform the base signals Z ^s of said first presentation into output signals Y ^j of said second presentation. The transformation parameters can be determined as outlined herein. In particular, the desired output signals Y ^j for the second presentation can be determined from the channels or audio objects X ⁱ using second delivery parameters C (as outlined herein). The transform parameters M can be determined by minimizing the deviation of the output signals Y ^j from the desired output signals Y ^j (eg, using a least root mean square error criterion).

Even more particularly, the M transform parameters can be determined in the sub-band domain (ie i.e. for different frequency bands). For this purpose, the base signals in the sub-band domain Z [k, b] can be determined for P frequency bands using a filter bank of the encoder 92, 93. The number B of frequency bands is greater than one, for example B is equal to or greater than 4, 6, 8, 10. In the examples described herein, B = 8 or B = 5. As outlined above, the encoder filter bank 92, 93 may comprise a hybrid filter bank providing low-frequency bands B, the B frequency bands having a higher frequency resolution than the high-frequency bands of the B frequency bands. Furthermore, desired output signals in the Y sub-band domain [k, b] can be determined for the B frequency bands. The transform parameters M for one or more frequency bands can be determined by minimizing the deviation of the output signals Y ^j from the desired output signals Y ^j within one or more frequency bands (for example, using a criterion of least mean square error).

Therefore, each of the M transform parameters can be specified for at least two frequency bands (notably, for P frequency bands). In addition, the transform parameters may include a set of multi-tap convolution matrix parameters for at least one of the frequency bands.

Therefore, a method (and a corresponding decoder) is described for determining output signals of a second presentation of audio channels / objects from base signals of a first presentation of the audio objects / channels. The first presentation can be used for speaker playback and the second presentation can be used for headphones playback (or vice versa). The output signals are determined using transform parameters for different frequency bands, where the transform parameters for at least one of the frequency bands comprise multi-tap convolution matrix parameters. As a result of using multi-tap convolution matrix parameters for at least one of the frequency bands, the computational complexity of a decoder 100 can be reduced, notably by reducing the frequency resolution of a filter bank used by the decoder.

For example, determining an output signal for a first frequency band using multi-tap convolution matrix parameters may comprise determining a current sample from the first frequency band of the output signal as a weighted combination of current samples, and a or several above, of the first frequency band of the base signals, where the weights used to determine the weighted combination correspond to the multi-tap convolution matrix parameters for the first frequency band. Typically, one or more of the multi-tap convolution matrix parameters for the first frequency band have complex values.

Furthermore, determining an output signal for a second frequency band may comprise determining a current sample from the second frequency band of the output signal as a weighted combination of current samples from the second frequency band of the base signals (and not based on previous samples of the second frequency band of the base signal), where the weights used to determine the weighted combination correspond to transformation parameters for the second frequency band. The transformation parameters for the second frequency band can have complex values, or alternatively they can have real values.

In particular, the same set of multi-tap convolution matrix parameters can be determined for at least two adjacent frequency bands of the B frequency bands. As shown in Figure 7, a single set of multi-tap convolution matrix parameters can be determined for the frequency bands provided by the Nyquist filter bank (that is, for frequency bands that have a relatively high frequency resolution). By doing this, the use of a Nyquist filter bank within decoder 100 can be omitted, thereby reducing the computational complexity of decoder 100 (while maintaining the quality of the output signals for the second presentation).

Furthermore, the same real-value transform parameter can be determined for at least two adjacent high-frequency bands (as shown in the context of Figure 7). Thus, the computational complexity of the decoder 100 can be further reduced (while maintaining the quality of the output signals for the second presentation).

Interpretation

Reference throughout this specification to "one embodiment" or "some embodiments" means that a particular aspect, structure, or feature described in connection with the embodiment is included in at least one embodiment of the present invention. Therefore, the occurrences of the terms "in one embodiment" or "in some embodiments" in various places throughout this specification do not necessarily all refer to the same embodiment, but may. Furthermore, the particular aspects, structures or characteristics can be combined in any suitable way, as will be apparent to one skilled in the art of this invention, in one or more embodiments.

As used herein, unless otherwise specified, the use of the ordinal adjectives "first", "second", "third", etc., to describe a common object, indicates only that it is being alluded to. to different instances of similar objects, and is not intended to imply that the objects thus described have to be found in a certain sequence, whether temporally, spatially, in order or in any other way. In the following claims and the description herein, any of the terms "comprise," or "comprising" is an open term meaning that it includes at least the following elements / features, but not excluding others. Therefore, the term comprising, when used in the claims, should not be construed as being limited to the means or elements or steps listed below. For example, the scope of the term a device comprising A and B should not be limited to devices consisting only of the elements A and B. Any of the terms includes, or includes, as used herein, it is also an open expression which also means that it includes at least the elements / characteristics that follow the expression, but without excluding others. Therefore, including is synonymous with a medium that understands.

As used herein, the term "exemplary" is used in the sense of providing examples, not indicating quality. That is, an "exemplary embodiment" is an embodiment provided as an example, without necessarily being an embodiment of exemplary quality.

It will be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of simplifying the description and assisting in the understanding one or more of the various inventive aspects. However, this method of the invention should not be construed as reflecting the intention that the claimed invention requires more features than are expressly enumerated in each claim. Rather, as reflected in the following claims, the inventive aspects reside in less than all the features of a single embodiment disclosed in the foregoing. Therefore, the claims that follow the detailed description are hereby expressly incorporated into this detailed description, each claim functioning by itself as an independent embodiment of this invention.

Furthermore, while some embodiments described herein include part, but not all of the features included in other embodiments, it is understood that combinations of features from different embodiments are within the scope of the invention, and form different embodiments, as will be understood by the subject matter experts. For example, in the following claims, any of the claimed embodiments can be used in any combination.

Furthermore, some of the embodiments are described herein as a method or combination of elements of a method that can be implemented by a processor of a computer system, or by other means to carry out the function. Therefore, a processor with the instructions necessary to carry out said procedure or element of a procedure, forms a means for carrying out the procedure or element of a procedure. Furthermore, an element described herein of an apparatus embodiment is an example of a means for carrying out the function performed by the element for the purpose of carrying out the invention.

In the description disclosed herein, numerous specific details are set forth. However, it is understood that embodiments of the invention can be practiced without these specific details. In other cases, well known procedures, structures and techniques have not been shown in detail so as not to obscure an understanding of this disclosure.

Similarly, it should be noted that the term coupled, when used in the claims, should not be construed as being limited to direct connections only. The expressions "docked" and "connected" can be used, together with their derivatives. It should be understood that these expressions are not considered synonymous with each other. Therefore, the scope of the expression a device A coupled to a device B should not be limited to devices or systems in which an output of device A is directly connected to an input of device B. This means that there is a path between an output from A and an input from B which can be a path that includes other devices or media. "Coupled" can mean that two or more elements are in direct physical or electrical contact, or that two or more elements are not in direct contact with each other but are nonetheless cooperating or interacting with each other.

Therefore, although what are considered preferred embodiments of the invention have been described, those skilled in the art will recognize that those other modifications can be made without departing from the scope of the invention, as defined in the appended claims, and are it is intended to claim that all such changes and modifications are within the scope of the invention. For example, any formulas provided above are only representative of procedures that can be used. Functionality can be added or removed from block diagrams, and operations can be exchanged between function blocks. Steps may be added or removed to the procedures described, within the scope of the present invention.

Claims (14)

1. A procedure for representing a second presentation of audio channels or objects as a data stream, the procedure comprising the steps of: (a) providing base signals, said base signals representing a first presentation of the audio channels or objects; (b) providing transformation parameters, said transformation parameters being provided to transform the base signals of said first presentation into output signals of said second presentation, said transformation parameters including at least specified high-frequency transformation parameters for a upper frequency band and low-frequency transform parameters specified for a lower frequency band, the low-frequency transform parameters including a set of multi-tap convolution matrix parameters to convolve low-frequency components of the base signals with the low-frequency transformation parameters to produce convoluted low-frequency components and the high-frequency transformation parameters including a set of parameters from a stateless matrix to multiply high-frequency components of the base signals with the high-frequency transformation parameters to produce multiplied high-frequency components; the first presentation being provided for loudspeaker playback and the second presentation being provided for headphone playback, or vice versa; Y (c) combining said base signals and said transformation parameters to form said data stream. 2. The method according to claim 1, wherein said multi-tap convolution matrix parameters are indicative of a finite impulse response (FIR) filter; me wherein said multi-tap convolution matrix parameters include at least one coefficient having complex values. The method according to any preceding claim, wherein said base signals are divided into a series of time slots, and transformation parameters are provided for each time slot. 4. The method according to any preceding claim, wherein providing the base signals comprises determining the base signals from audio channels or objects using first delivery parameters; the method comprises determining desired output signals for the second presentation from the audio channels or objects using second delivery parameters; Y providing the transformation parameters comprises determining the transformation parameters by minimizing a deviation of the output signals from the desired output signals. The method of claim 4, wherein determining the transformation parameters comprises determining subband domain base signals for a number B of frequency bands using an encoder filter bank; determining desired subband domain output signals for the frequency bands using the encoder filter bank; Y determining a same set of multi-tap convolution matrix parameters for at least two adjacent frequency bands of the B frequency bands. 6. The method according to claim 5, wherein the filter bank of the encoder comprises a hybrid filter bank providing low-frequency bands of the B frequency bands having a higher frequency resolution than the high-frequency bands of the B frequency bands; Y the at least two adjacent frequency bands are low frequency bands. 7. A decoder for decoding an encoded audio signal, the encoded audio signal including: a first display including base audio signals intended for reproduction of the encoded audio signal in a first audio display format; Y transformation parameters, to transform said audio base signals into said first audio format presentation, in output signals of a second presentation format; said transformation parameters comprising specified high-frequency transformation parameters for an upper frequency band and specified low-frequency transformation parameters for a lower frequency band, said low-frequency transformation parameters including multi-tap convolution matrix parameters and the high-frequency transformation parameters including a set of parameters of a stateless matrix, the first presentation format being provided for playback on loudspeakers and the second presentation format being provided for playback on headphones, or vice versa, including decoder: a first separation unit for separating the audio base signals, and the transformation parameters, a matrix multiplication unit for applying said multi-tap convolution matrix parameters to low-frequency components of the audio base signals, to apply a convolution to the low-frequency components, producing convolved low-frequency components; a scalar multiplication unit for applying said high-frequency transformation parameters to high-frequency components of the audio base signals to produce scalar high-frequency components; Y an output filter bank for combining said convoluted low frequency components and said scalar high frequency components to produce a time domain output signal of said second presentation format. The decoder according to claim 7, further comprising filters for separating the audio base signals into said low frequency components and said high frequency components. 9. A method for decoding an encoded audio signal, the encoded audio signal including: a first display including base audio signals intended for reproduction of the encoded audio signal in a first audio display format; Y transformation parameters, for transforming said audio base signals in said first presentation format, into output signals of a second presentation format; said transformation parameters comprising specified high-frequency transformation parameters for an upper frequency band and specified low-frequency transformation parameters for a lower frequency band, said low-frequency transformation parameters including multi-tap convolution matrix parameters and the high-frequency transformation parameters including a set of parameters of a stateless matrix, the first presentation format being provided for playback on loudspeakers and the second presentation format being provided for playback on headphones, or vice versa, The procedure includes the stages of: convolving low-frequency components of the audio base signals with the low-frequency transform parameters to produce convoluted low-frequency components; multiplying high-frequency components of the audio base signals with the high-frequency transform parameters to produce multiplied high-frequency components; combining said convoluted low-frequency components and said multiplied high-frequency components to produce output audio signal frequency components for the second presentation format. The method of claim 9, wherein said encoded audio signal comprises multiple time slots, and said convolution of low-frequency components of the audio base signals includes the steps of: interpolating transformation parameters of multiple time segments of the encoded audio signal to produce interpolated transformation parameters, including interpolated low-frequency transformation parameters; Y convolving multiple time slots of the low-frequency components of the audio base signals with the interpolated low-frequency transform parameters, to produce multiple time slots of said convolved low-frequency components. The method according to claim 9, wherein the transformation parameters of said encoded audio signal are variable in time, and said convolution of the low-frequency components of the audio base signals includes the steps of: convolving the low-frequency components of the audio base signals with the low-frequency transform parameters for multiple time slots to produce multiple sets of intermediate convolved low-frequency components; and interpolating the multiple sets of intermediate convolved low-frequency components, to produce said convolved low-frequency components. The method according to claim 10 or 11, wherein said interpolation uses a method of overlapping and adding, of the multiple sets of intermediate convoluted low-frequency components. The method according to any one of claims 9 to 12, further comprising filtering the audio base signals at said low frequency components and said high frequency components. 14. A non-computer-readable transient storage medium, including program instructions for operating a computer, according to the method of any of claims 1 to 6, or 9 to 13.