KR101893410B1 - Multi-channel decorrelator, multi-channel audio decoder, multi-channel audio encoder, methods and computer program using a premix of decorrelator input signals - Google Patents

Abstract

A multi-channel decorrelator for providing a plurality of decorrelated signals based on the plurality of decorrelator input signals is configured to premix a first set of N decorrelator input signals into a second set of K decorrelator input signals , Where K < N. The multi-channel decorrelator is configured to provide a first set of K 'decorrelator output signals based on the first set of K decorrelator input signals. The multi-channel decorrelator is further configured to upmix a first set of K 'decorrelator output signals into a second set of N' decorrelator output signals, where N '> K'. The multi-channel decorrelator can be used in a multi-channel audio decoder. A multi-channel audio encoder provides complexity control information for a multi-channel decorrelators.

Description

MULTI-CHANNEL AUDIO DECODER, MULTI-CHANNEL AUDIO ENCODER, METHODS AND COMPUTER PROGRAM USING MULTI-CHANNEL DECORRELATOR, MULTI-CHANNEL AUDIO DECODER, MULTI-CHANNEL AUDIO ENCODER, METHODS AND COMPUTER PROGRAM USING A PREMIX OF DECORRELATOR INPUT SIGNALS}

Embodiments in accordance with the present invention are directed to a multi-channel decorrelator for providing a plurality of decorrelated signals based on a plurality of decorrelator input signals.

Yet another embodiment according to the present invention relates to a multi-channel audio decoder for providing at least two output audio signals based on an encoded representation.

Yet another embodiment according to the present invention relates to a multi-channel audio encoder for providing an encoded representation based on at least two input audio signals.

Yet another embodiment in accordance with the present invention is directed to a method for providing a plurality of decorrelated signals based on a plurality of decorrelator input signals.

Some embodiments in accordance with the present invention are directed to a method for providing at least two output audio signals based on an encoded representation.

Some embodiments in accordance with the present invention are directed to a method for providing an encoded representation based on at least two input audio signals.

Some examples according to the present invention relate to a computer program for carrying out any of the above methods.

Some embodiments in accordance with the present invention are directed to encoded audio representations.

Generally speaking, some embodiments in accordance with the present invention relate to decorrelation concepts for multi-channel downmix / upmix parameter audio object coding systems.

Recently, there has been a steady increase in demand for the storage and transmission of audio content. In addition, quality requirements for the storage and transmission of audio content are also steadily increasing. Thus, the concept for encoding and decoding audio content has improved.

For example, so-called " Advanced Audio Coding " (AAC), described in the International Standard ISO / IEC 13818-7: 2003, has been developed. In addition, some space extensions, such as the so-called " MPEG surround " concept, described for example in International Standard ISO / IEC 23003-1: In addition, additional enhancements for encoding and decoding spatial information of audio signals are described in the International Standard ISO / IEC 23003-2: 2010, which is related to so-called " spatial audio object coding ".

In addition, the concept of switchable audio encoding / decoding, which encodes both common audio signals and speech signals with excellent coding efficiency and provides the possibility to process multi-channel audio signals, is called " Integrated Voice and Audio Coding ISO / IEC 23003-3: 2012, which describes the concept of "

In addition, other conventional concepts are set forth in the references, referred to at the end of the description of the present invention.

However, there is a desire to provide a much more advanced concept for efficient encoding and decoding of 3D audio scenes.

One embodiment in accordance with the present invention generates a multi-channel decorrelator for providing a plurality of decorrelated signals based on a plurality of decorrelator input signals. The multi-channel decorrelator is configured to premix a first set of N decorrelator input signals into a second set of K decorrelator input signals, where K < N. The multi-channel decorrelator is configured to provide a first set of K 'decorrelator output signals based on the first set of K decorrelator input signals. The multi-channel decorrelator is configured to upmix a first set of K 'decorrelator output signals into a second set of N' decorrelator output signals, where N '> K.

One embodiment in accordance with the present invention is based on the concept that the complexity of the decorrelation can be reduced by premixing the first set of N decorrelator input signals into the second set of K decorrelator input signals, The second set of signals includes fewer signals than the first set of N de-correlator input signals. Thus, the basic decorrelator function is, for example, a set of K signals (a second set of K decorrelator input signals < RTI ID = 0.0 > In addition, to provide N 'decorrelators output signals, an upmix is performed and a first set of K' decorrelators output signals are upmixed into a second set of N 'decorrelators output signals A relatively large number of decorrelated signals (i. E., The decorrelator output < RTI ID = 0.0 > Signals of the second set of signals), and the core inverse correlation function is performed based only on the K signals (e.g., using only K individual inverse correlators). Thus, Significant benefits in correlation efficiency Is achieved, which helps to save processing power and resources (e. G., Energy).

In a preferred embodiment, the number K of signals of the second set of decorrelator input signals is equal to the number K 'of signals of the first set of decorrelator output signals. Thus, for example, it is possible to receive one de-correlator input signal (of the second set of de-correlator input signals) from each of the premixing, and each de-mix with one de-correlated output signal ), ≪ / RTI > may be present. Thus, simple individual decorrelators can be used, each providing one output signal based on one input signal.

In another preferred embodiment, the number of signals (N) of the first set of decorrelator input signals may be equal to the number of signals (N ') of the second set of decorrelator output signals. Thus, The number of signals received by the multi-channel decorrelator such that the decorrelator appears from the outside, such as a bank of N independent decorrelators, is equal to the number of signals provided by the multi-channel decorrelator (however, But may include some defects due to the use of only K input signals for the core inverse correlators). Thus, a multi-channel decorrelator can be used as an immediate replacement for conventional decorrelators having the same number of input and output signals. In addition, it should be noted that upmixing can be derived, for example, from premixing in such a configuration with reasonable effort.

In a preferred embodiment, the number of signals (N) in the first set of decorrelator input signals may be greater than or equal to three, and the number of signals in the second set of decorrelator output signals (N ' May be greater or equal. In such a case, the multi-channel decorrelator can provide a certain efficiency.

In a preferred embodiment, the multi-channel decorrelator is configured to pre-mix a first set of N decorrelator input signals into a second set of K decorrelator input signals using a premixing matrix (i.e., using a linear pre-mixing function) Lt; / RTI > In this case, the multi-channel decorrelator may be configured to obtain a first set of K 'decorrelator output signals (e.g., using individual decorrel correlators) based on the second set of K decorrelator input signals . The multi-channel decorrelator may also be configured to upmix the first set of K 'decorrelator output signals into the second set of N' decorrelator output signals using a postmixing matrix, i. E., Using a linear postmixing function . Thus, the distortions can be kept small. Also, premixing and postmixing (also designated as upmixing) can be performed in a computationally efficient manner.

In a preferred embodiment, the multi-channel historical analyzer can be configured to select the premixing matrix in dependence on the associated spatial locations of the first set of channel signals of the N de-correlator input signals. Thus, spatial dependencies (or correlations) can be considered in the pre-mixing process, which helps to prevent excessive degradation due to the premixing process being performed in a multi-channel decorrelator.

In a preferred embodiment, the multi-channel decorrelator may be configured to select the premixing matrix depending on the correlation features or covariance characteristics of the first set of N decorrelator input signals. Such a function also helps prevent excessive degradation due to the premixing process performed by the multi-channel decorrelator. For example, the decorrelator input signals (of the first set of decorrelator input signals) that are closely related (i. E., Include a high cross-correlation or a high cross covariance) May be coupled into a correlator input signal (of the second set of decorrelator input signals) and, as a result, may be processed by, for example, a common individual decorrelator (of the decorrelator core). Thus, it is preferred that substantially different decorrelator input signals (of the first set of decorrelator input signals) are premixed into a single decorrelator input signal (of the second set of decorrelator input signals), which is input into the decorrelator core May be avoided because it generally causes inadequate decorrelator output signals (e.g., the audio signals may interfere with the spatial perception when used to cross the desired cross-correlation features or cross crossover features) I will do it. Thus, the multi-channel decorrelator can determine in an intelligent manner which signals should be combined in the premixing (or downmixing) process to allow for an excellent tradeoff between the decorrelation efficiency and the audio quality.

In a preferred embodiment, the multi-channel decorrelator is configured to determine the premixing matrix such that the matrix product between the premixing matrix and its Hermitian matrix is well maintained with respect to the inversion operation. Thus, the pre-mixing matrix can be determined such that the post-mixing matrix can be selected without numerical problems.

In a preferred embodiment, the multi-channel decorrelator is configured to obtain a post-mixing matrix based on a pre-mix matrix using some matrix multiplication and matrix-turn operations. In this way, the post-mixing matrix can be efficiently obtained so that the post-mixing matrix is well suited to the pre-mixing process.

In a preferred embodiment, the multi-channel decorrelator is configured to receive information about the rendering configuration associated with the first set of channel signals of the N decorrelator input signals. In this case, the multi-channel decorrelator includes information about the rendering configuration Lt; RTI ID = 0.0 > a < / RTI > pre-mixing matrix. Thus, the pre-mixing matrix may be selected in a manner that is well-suited to the rendering configuration, such that excellent audio quality is obtained.

In a preferred embodiment, the multi-channel decorrelator is configured to combine the channel signals of the first set of non-correlator input signals with spatially adjacent positions of the audio scene when performing premixing. Thus, the fact that the channel signals associated with spatially contiguous locations of the audio scene are generally similar is used when setting up premixing. As a result, similar audio signals can be combined and processed in premixing using the same individual decorrelators in the decorrelator core. Thus, unacceptable degradation of the audio content can be prevented.

In a preferred embodiment, the multi-channel decorrelator is configured to combine a first set of channel signals of N decorrelator input signals associated with vertically spatially adjacent positions of the audio scene when performing premixing. This concept is based on the fact that the audio signals from vertically spatially adjacent positions of the audio scene are generally similar. In addition, the human perception is not particularly sensitive to differences between signals associated with vertically and spatially adjacent positions of the audio scene. Thus, it has been found that the combination of audio signals associated with vertically spatially adjacent positions does not cause a substantial degradation of the listening impression obtained on the basis of decorrelated audio signals.

In a preferred embodiment, the multi-channel decorrelator can be configured to combine the first set of channel signals of N decorrelator input signals with the horizontal pair of spatial positions including the left side position and the right side position. It has been found that the channel signals associated with the horizontal pair of spatial positions including the left side position and the right side position are generally also somewhat related because the channel signals associated with the horizontal pair of spatial positions generally have a spatial impression gain . Thus, for example, if it was not sufficient to combine channel signals associated with vertically spatially adjacent locations of an audio scene, it was found to be a reasonable solution to combine the channel signals associated with the horizontal pair of spatial locations, The reason is that the combination of the channel signals associated with the horizontal pair of spatial locations generally does not cause excessive degradation of the listening impression.

In a preferred embodiment, the multi-channel decorrelator is configured to combine at least four channel signals of a first set of N decorrelator input signals, at least two of the at least four channel signals being associated with a spatial location And at least two of the at least four channel signals are associated with spatial locations on the right side of the audio scene. Thus, four or more channel signals are combined so that effective decorrelation can be obtained without significantly including the listening impression.

In a preferred embodiment, the multi-channel decorrelator includes at least two left-side channel signals (i. E., Channel signals associated with spatial locations on the left side of the audio scene) associated with the center plane of the audio scene, Is associated with spatial locations that are symmetric to spatial positions associated with the two right-hand side channel signals (i. E., Channel signals associated with spatial positions on the right side of the audio scene). It has been found that the combination of channel signals associated with " symmetric " spatial positions generally yields excellent results because the signals associated with such " symmetric " spatial positions are generally somewhat related, ) Reverse correlation.

In a preferred embodiment, the multi-channel decorrelator is configured to receive complexity information describing the number (K) of decorrelator input signals of the second set of decorrelator input signals. In this case, the multi-channel decorrelator is configured to select the pre-mixing matrix depending on the complexity information. Thus, a multi-channel decorrelator can be flexibly applied to different complexity requirements. Thus, it is possible to change the trade-off between audio quality and complexity.

In a preferred embodiment, the multi-channel decorrelator includes a first multi-channel decorrelator with a reduced value of complexity information, a first multi-channel decorrelator coupled to obtain the decorrelator input signals of the second set of decorrelator input signals, And to incrementally increase the number of decorrelator input signals in the set. Thus, if it is desired to reduce the complexity, it may be desirable to combine more decorrelator input signals of the first set of decorrelator input signals (e.g., into a single set of decorrelator input signals of the decorrelator input signals) , Which allows to change the complexity with little effort.

In a preferred embodiment, the multi-channel decorrelator includes only first-order channel signals of decorrelator input signals associated with vertically spatially adjacent positions on the left side of the audio scene when performing pre-mixing for the first value of complexity information . However, the multi-channel decorrelator may also include vertically spatially adjacent positions on the left side of the audio scene to obtain a given signal of the second set of decorrelator input signals when performing premixing for the second value of complexity information At least two of the first set of related decorrelator input signals and at least two of the first set of decorrelator input signals associated with vertically spatially adjacent positions on the right side of the audio scene . In other words, for the first value of the complexity information, no combination from the different sides of the audio scene can be performed, which means that the audio signals (and of the listening impression, which can be obtained based on the decorrelated audio signals) Especially high quality. In contrast, if less complexity is required, horizontal coupling in addition to vertical coupling can also be performed. This rational concept for step-by-step adjustment of complexity has been found, and a somewhat higher drop in the listening impression is found for reduced complexity.

In a preferred embodiment, the multi-channel decorrelators are configured to receive at least four of the first set of decorrelator input signals to obtain a given signal of the second set of decorrelator input signals when performing premixing for the second value of complexity information Wherein at least two of the at least four channel signals are associated with spatial positions on the left side of the audio scene and at least two of the at least four channel signals are associated with spatial positions on the right side of the audio scene ≪ / RTI > This concept is applicable to at least two channel signals associated with spatial positions on the left side of the audio scene and spatial positions on the right side of the audio scene, even though the signal channels are not vertically adjacent (or at least not perfectly perpendicularly adjacent) Based on the finding that relatively low computational complexity can be obtained by combining at least two channel signals associated with the < RTI ID = 0.0 >

In a preferred embodiment, the multi-channel decorrelator includes a decorrelator input signal associated with vertically spatially adjacent locations on the left side of the audio scene to obtain a first decorrelator input signal of the second set of decorrelator input signals And to obtain a second set of decorrelator input signals for a first value of complexity information, wherein the second set of decorrelator input signals for the first value of complexity information comprises a first set of decorrelator input signals, And to combine at least two of the first set of decorrelator input signals associated with spatially adjacent positions. In addition, the multi-channel decorrelator is associated with vertically spatially adjacent positions on the left side of the audio scene to obtain a second decorrelator input signal of a second set of decorrelator input signals for a second value of complexity information And to combine the at least two of the first set of decorrelator input signals and the at least two of the first set of decorrelator input signals associated with vertically spatially adjacent positions on the right side of the audio scene do. In this case, the number of decorrelator input signals of the second set of decorrelator input signals is greater for the first value of complexity information than for the second value of complexity information. In other words, the four channel signals, which are used to obtain the two decorrelator input signals of the second set of decorrelator input signals for the first value of complexity reduction, Can be used to obtain a second set of single decorrelator input signals. Thus, the signals acting as input signals for the two individual decorrelators for the first value of complexity reduction are combined to act as input signals to a single individual decorrelator for the second value of complexity reduction. Thus, the number of individual decorrelators (or the number of decorrelator input signals of the second set of decorrelator input signals) may be obtained for a reduced value of the complexity information.

One embodiment in accordance with the present invention creates a multi-channel audio decoder for providing at least two output audio signals based on the encoded representation. The multi-channel audio decoder includes a multi-channel decorrelators, such as those described herein.

This embodiment is based on the fact that a multi-channel audio decorrelator is well suited for application in a multi-channel audio decoder.

In a preferred embodiment, a multi-channel audio decoder is configured to render a plurality of decoded audio signals, which are obtained based on the encoded representation, in dependence on one or more rendering parameters, to obtain a plurality of rendered audio signals . The multi-channel audio decoder is configured to derive one or more decorrelated audio signals from the rendered audio signals using a multi-channel decorrelator, wherein the rendered audio signals constitute a first set of decorrelator input signals, The second set of signals constitute the decorrelated audio signals. The multi-channel audio decoder is configured to combine the rendered audio signals or a scaled version thereof with one or more decorrelated audio signals (of a second set of decorrelator output signals) to obtain output audio signals. This embodiment in accordance with the present invention is based on the discovery that the multi-channel decorrelator described herein is well suited for post-rendering, wherein a relatively large number of rendered audio signals are input into the multi-channel decorrelator, The numbered correlated signals are then combined with the rendered audio signals. In addition, it has been found that defects due to the use of a relatively small number of individual decorrel correlators (reduced complexity of the multi-channel decorrelator) do not generally cause a serious degradation of the quality of the output audio signals output by the multi-channel decoder .

In a preferred embodiment, the multi-channel audio decoder is configured to select a premixing matrix for use by a multi-channel decorrelator in dependence on the control information contained in the encoded representation. Thus, it is possible for the audio encoder to even control the quality of the decorrelation so that the quality of the decorrelation is well adapted to the particular audio content, which results in an excellent balance between audio quality and inverse correlation complexity.

In a preferred embodiment, the multi-channel audio decoder is configured to select a premixing matrix for use by the multi-channel decorrelator depending on an output configuration that describes the assignment of output audio signals to spatial locations of the audio scene. Thus, the multi-channel decorrelators can be adapted to specific rendering scenarios, which helps prevent substantial degradation of audio quality by efficient decorrelation.

In a preferred embodiment, the multi-channel audio decoder is configured to select between three or more different pre-mixing matrices ( M _pre ) for use by a multi-channel decorrelator depending on the control information contained in the encoded representation for a given output configuration . In this case, each of the three or more different pre-mixing matrices is associated with a different number of signals of the second set of decorrelator input signals. Thus, the complexity of the decorrelation can be adjusted over a wide range.

In a preferred embodiment, the multi-channel audio encoder rely on a mixing matrix (Dconv, Drender) used by a format converter or renderer that receives at least two output audio signals to provide a pre-mixing matrix M _pre ).

In yet another embodiment, a multi-channel audio decoder is configured for use by a multi-channel decorrelator to be the same as a mixing matrix (Dconv, Drender) used by a format converter or renderer that receives at least two output audio signals. ( M _pre ).

One embodiment in accordance with the present invention creates a multi-channel audio encoder to provide an encoded representation 814 based on at least two input audio signals. The multi-channel audio encoder is configured to provide one or more downmix signals based on at least two input audio signals. The multi-channel audio encoder is also configured to provide one or more parameters that describe the relationship between at least two input audio signals. In addition, a multi-channel audio encoder is configured to provide an inverse correlation complexity parameter that describes the complexity of the decorrelator to be used on the side of the audio decoder. Thus, the multi-channel audio encoder can control the multi-channel audio decoder described above so that the complexity of the decorrelation can be adjusted to the requirements of the audio content encoded by the multi-channel audio encoder.

Another embodiment in accordance with the present invention creates a method for providing a plurality of decorrelated signals based on a plurality of decorrelator input signals. The method includes premixing a first set of N de-correlator input signals into a second set of K de-correlator input signals, where K < N. The method also includes providing a first set of K 'decorrelator output signals based on the first set of K decorrelator input signals. In addition, the method includes upmixing a first set of K 'decorrelators output signals into a second set of N' decorrelator output signals, where N '> K'. This method is based on the same concept as the multi-channel decorrelators described above.

Yet another embodiment according to the present invention creates a method for providing at least two output audio signals based on an encoded representation. The method includes providing a plurality of decorrelated signals based on a plurality of decorrelator input signals, such as those described above. This method is based on the same findings as the multi-channel audio decoder described above.

Yet another embodiment creates a method for providing an encoded representation based on at least two input audio signals. The method includes providing at least one downmix signal based on at least two input audio signals. The method also includes providing at least one parameter describing a relationship between at least two input audio signals. The method also includes providing an inverse correlation complexity parameter that describes the complexity of the decorrelator to be used on the side of the audio decoder. This method is based on the same concept as the audio encoder described above.

In addition, embodiments according to the present invention create a computer program for carrying out the methods.

Yet another embodiment in accordance with the present invention produces an encoded audio representation. The encoded audio representation includes an encoded representation of the downmix signal and an encoded representation of one or more parameters describing the relationship between the at least two input audio signals. In addition, the encoded audio representation includes an encoded decorrelation method parameter that describes which of the plurality of decorrelation modes should be used on the side of the audio decoder. The encoded audio representation thus permits signaling the appropriate decorrelation mode and thus helps to realize the advantages described with respect to the multi-channel audio encoder and the multi-channel audio decoder.

In addition, it should be noted that the above-described methods may be added by any of the features and functions described in connection with an apparatus such as those mentioned above.

Embodiments according to the present invention will be described below with reference to the accompanying drawings.
1 shows a schematic block diagram of a multi-channel audio decoder according to an embodiment of the present invention.
2 shows a schematic block diagram of a multi-channel audio encoder according to an embodiment of the present invention.
Figure 3 shows a flowchart of a method for providing at least two output audio signals based on an encoded representation, in accordance with an embodiment of the present invention.
Figure 4 illustrates a flowchart of a method for providing an encoded representation based on at least two input audio signals, in accordance with an embodiment of the present invention.
Figure 5 shows a schematic representation of an encoded audio representation, in accordance with an embodiment of the invention.
Figure 6 shows a schematic block diagram of a multi-channel decorrelator, in accordance with an embodiment of the present invention.
Figure 7 shows a schematic block diagram of a multi-channel audio decoder, in accordance with an embodiment of the present invention.
Figure 8 shows a schematic block diagram of a multi-channel audio encoder, in accordance with an embodiment of the present invention.
9 shows a flowchart of a method for providing a plurality of decorrelated signals based on a plurality of decorrelator input signals, in accordance with an embodiment of the present invention.
10 shows a flowchart of a method for providing at least two output audio signals based on an encoded representation, in accordance with an embodiment of the present invention.
11 shows a flowchart of a method for providing an encoded representation based on at least two input audio signals, in accordance with an embodiment of the present invention.
Figure 12 shows a schematic representation of an encoded representation, in accordance with one embodiment of the present invention.
Figure 13 shows a schematic representation that provides an overview of a minimum mean square error (MMSE) based parameter downmix / upmix, in accordance with an embodiment of the present invention.
Figure 14 shows a geometric representation for the orthogonality principle in three-dimensional space.
Figure 15 shows a schematic block diagram of a parameter reconstruction system with decorrelation applied on the rendered output, in accordance with an embodiment of the present invention.
Figure 16 shows a schematic block diagram of an decorrelation unit.
17 shows a schematic block diagram of a reduced complexity decorrelation unit, in accordance with an embodiment of the present invention.
Figure 18 shows a table representation of loudspeaker positions according to one embodiment of the present invention.
Figures 19a through 19g illustrate table representations of premixing coefficients for K equal to N = 22 and between 5 and 11.
Figures 20a through 20d show table representations of pre-mixing coefficients for N = 10 and K between 2 and 5;
Figures 21A through 19C show table representations of pre-mixing coefficients for N = 8 and K between 2 and 4;
Figures 21d through 21f show table representations of the pre-mixing coefficients for N = 7 and K between 2 and 4;
Figures 22A and 22B show table representations of pre-mixing coefficients for N = 5 and K = 2 or K = 3.
Figure 23 shows a table representation of the pre-mixing coefficients for N = 2 and K = 1.
24 shows a table representation of groups of channel signals.
Figure 25 shows a syntax representation of additional parameters that may be included in the SAOCSpecifigConfig (), or even the SAOC3DSpecifigConfig () syntax.
Figure 26 shows a table representation of the different values for the bitstream variable bsDecorrelationMethod.
Figure 27 shows a table representation of a number of decorrelators for different decorrelation levels and output configurations, as indicated by the bitstream variable bsDecorrelationLevel.
Figure 28 shows an overview of a three-dimensional audio encoder in the form of a schematic block diagram.
Figure 29 shows an overview of a three-dimensional audio decoder in the form of a schematic block diagram.
Figure 30 shows a schematic block diagram of the structure of a format converter.
Figure 31 shows a schematic block diagram for a downmix processor, in accordance with an embodiment of the present invention.
Figure 32 shows a table representing decoding modes for different numbers of spatial audio object coding (SAOC) downmix objects.
33 shows a syntax expression of the bit stream element " SAOC3DSpecificConfig ".

1. A multi-channel audio decoder

Figure 1 shows a schematic block diagram of a multi-channel audio decoder 100, in accordance with an embodiment of the present invention.

The multi-channel audio decoder 100 is configured to receive the encoded representation 110 and to provide at least two output audio signals 112, 114 based thereon.

The multi-channel audio decoder 100 preferably includes a decoder 120 configured to provide decoded audio signals 122 based on the encoded representation 110. In addition, the multi-channel audio decoder 100 may generate an encoded representation 110 (e.g., decoder 120) in dependence on one or more rendering parameters 132 to obtain a plurality of rendered audio signals 134,136. And a renderer 130, which is configured to render a plurality of decoded audio signals 122, which are obtained on the basis of a plurality of decoded audio signals 122 (e.g. In addition, the multi-channel audio decoder 100 includes an decorrelator 140 configured to derive one or more decorrelated audio signals 142, 144 from the rendered audio signals 134, 136. In addition, the multi-channel audio decoder 100 may convert the rendered audio signals 134,136 or their scaled version into one or more decorrelated audio signals 142,144 to obtain output audio signals 112,114, And 144, respectively.

However, it should be noted that different hardware structures of the multi-channel audio decoder 100 may be possible as long as the functions described above are given.

With respect to the functionality of the multi-channel audio decoder 100, the decorrelated audio signals 142 and 144 are derived from the rendered audio signals 134 and 136 and the decorrelated audio signals 142 and 144 are Is coupled with the rendered audio signals 134,136 to obtain the output audio signals 112,114. By deriving the decorrelated audio signals 142,144 from the rendered audio signals 134,136 a particularly efficient processing can be achieved because the number of rendered audio signals 134,136 Since it is generally independent of the number of decoded audio signals 122 that are input into the renderer 130. Thus, the inverse correlation effect is generally independent of the number of decoded audio signals 122, which improves implementation efficiency. In addition, the application of decorrelation after rendering prevents the introduction of artifacts, which may be caused by the renderer when combining a plurality of decorrelated signals when decorrelation is applied prior to rendering. In addition, features of the rendered audio signals may be considered in decorrelation performed by decorrelator 140, which generally results in output audio signals of excellent quality.

Besides. It should be noted that the multi-channel audio decoder 100 may be added by any of the features and functions described herein. In particular, it should be noted that the individual enhancements as described herein may be introduced into the multi-channel audio decoder 100 to thereby greatly improve the processing efficiency and / or quality of the output audio signals.

2. The multi-channel audio encoder

2 shows a schematic block diagram of a multi-channel audio encoder 200 according to an embodiment of the present invention. The multi-channel audio encoder 200 is configured to receive two or more input audio signals 210, 212 and to provide an encoded representation 214 based thereon. The multi-channel audio encoder 200 includes a downmix signal provider 220 configured to provide one or more downmix signals 222 based on at least two input audio signals 210, In addition, the multi-channel audio encoder 200 provides one or more parameters 232 (e.g., correlation, cross-correlation, crosstalk crossover, level difference, etc.) relationships between at least two input audio signals 210 and 212 And a parameter provider 230,

In addition, the multi-channel audio encoder 200 is also configured to provide an anticoronance method parameter 242 that is configured to provide an inverse correlation method parameter 242 that describes which of the plurality of inverse correlation modes should be used on the side of the audio decoder (240). One or more downmix signals 222, one or more parameters 232 and an anticoronality method parameter 242 are included in the encoded representation 214, for example, in encoded form.

However, it should be noted that the hardware structure of the multi-channel audio encoder 200 may be different so long as the functions described above are satisfied. In other words, to the individual blocks of functions of the multi-channel audio encoder 200 (e.g., to the downmix signal provider 220, to the parameter provider 230 and to the de-correlation method parameter provider 240) ) Should only be considered as an example.

With respect to the functionality of the multi-channel audio encoder 200, one or more of the downmix signals 222 and the one or more parameters 232 may be encoded in a conventional manner such as, for example, in a spatial audio object coding multi-channel audio encoder or USAC . ≪ / RTI > However, the decorrelation method parameters, also provided by the multi-channel audio encoder 200 and included in the encoded representation 214, may be used to apply the decorrelation mode to the input audio signals 210, 212 or to the required playback quality Can be used. Thus, the decorrelation mode can be applied to different types of audio content. For example, the types of audio content for which the input audio signals 212 and 214 are strongly correlated and the different decorrelation modes for the types of independent audio content may be selected for the input audio signals 212 and 214 . In addition, the different decorrelation modes may be used for different types of audio content such as, for example, the types of audio content for which spatial perception is particularly important, and the types of audio content for which space effects are less important or of secondary importance (e.g., And may be signaled by the inverse correlation mode parameter 242. [ Thus, a multi-channel audio decoder receiving the encoded representation 214 may be controlled by the multi-channel audio encoder 200 and may be sent in a decoding mode that yields the best possible trade-off between decoding complexity and playback quality .

In addition, it should be noted that the multi-channel audio encoder 200 may be added by any of the features and functions described herein. In particular, possible additional features and enhancements, such as those described herein, may be added to the multi-channel audio encoder 200, individually or in combination, thereby improving (or improving) the multi-channel audio encoder 200 .

A method for providing at least two audio signals according to Fig. 3

FIG. 3 shows a flowchart of a method 300 for providing at least two output audio signals based on an encoded representation. The method includes rendering (310) a plurality of decoded audio signals, which are obtained based on the encoded representation (312), in dependence on one or more rendering parameters, to obtain a plurality of rendered audio signals . The method 300 also includes deriving 320 one or more decorrelated audio signals from the rendered audio signals. The method 300 also includes combining (330) the rendered audio signals, or a scaled version thereof, with one or more decorrelated audio signals to obtain output audio signals 332.

It should be noted that the method is based on the same considerations as the multi-channel audio decoder 100 according to FIG. In addition, it should be noted that the method 300 may be added (either individually or in combination) by any of the features and functions described herein. For example, the method 300 may be added by any of the features and functions described in connection with the multi-channel audio decoder described herein.

4. A method for providing an encoded representation according to FIG. 4

4 shows a flowchart of a method 400 for providing an encoded representation based on at least two input audio signals. The method 400 includes providing 410 one or more downmix signals based on at least two input audio signals 412. The method 400 includes providing 420 one or more parameters describing the relationship between at least two input audio signals 412 and determining which of the plurality of decorrelation modes should be used on the side of the audio decoder (Step 430). Thus, an encoded representation 432 is provided that preferably includes an encoded representation of one or more downmix signals, one or more parameters describing a relationship between at least two input audio signals, and an anti-correlation method parameter. It should be noted that the method 400 is also based on the same considerations as the multi-channel audio encoder 200 according to FIG. 2 so that the above description applies.

In addition, the order of steps 410, 420 and 430 may be changed flexibly, and steps 410, 420 and 430 may be performed in parallel as well as possible in an execution environment for method 400 Be careful. In addition, it should be noted that the method 400 may be added either individually or in combination, by any of the features and functions described herein. For example, the method 400 may be added by any of the features and functions described in connection with the multi-channel audio encoder described herein. However, it is also possible to introduce features and functions corresponding to the features and functions of the multi-channel audio decoders described herein that receive the encoded representation 432.

5. Encoded audio representation according to FIG. 5

FIG. 5 shows a schematic representation of an encoded audio representation 500 in accordance with an embodiment of the present invention.

The encoded audio representation 500 includes an encoded representation 510 of the downmix signal, an encoded representation 520 of one or more parameters describing a relationship between at least two audio signals. In addition, the encoded audio representation 500 also includes an encoded decorrelation method parameter 530 that describes which of the plurality of decorrelation modes should be used on the side of the audio decoder. Thus, the encoded audio representation allows signaling of the decorrelation mode from the audio encoder to the audio decoder. Thus, audio content (e.g., by an encoded representation 510 of one or more downmix signals, and by downmixing into at least two audio signals (e.g., an encoded representation 510 of one or more downmix signals) Which is described by an encoded representation of one or more parameters describing the relationship between the audio signal (e.g., at least two audio signals that have been encoded). Thus, the encoded audio representation 500 allows for the rendering of audio content represented by an encoded audio representation 500 that has a particularly good balance between decoding complexity and auditory spatial effects and / or auditory spatial effects in particular.

In addition, it should be noted that the encoded representation 500 may be added individually or in combination by any of the features and functions described in connection with multi-channel audio encoders and multi-channel audio decoders.

6. The multi-channel Inverse correlator

FIG. 6 shows a schematic block diagram of a multi-channel decorrelator 600 according to an embodiment of the present invention.

The multi-channel decorrelator 600 receives a first set of N decorrelator input signals 610a through 610n and provides a second set of N 'decorrelator output signals 612a through 612n' based thereon . In other words, the multi-channel decorrelator 600 is configured to provide a plurality of (at least approximately) decorrelated signals 612a through 612n 'based on the decorrelator input signals 610a through 610n.

The multi-channel decorrelator 600 is configured to pre-mix a first set of N decorrelator input signals 610a through 610n into a second set of K decorrelator input signals 622a through 622k, , 620), where K is less than N (K and N are integers). The multi-channel decorrelator 600 is also configured to provide a first set of K 'decorrelator output signals 632a through 632k' based on K decorator input signals 622a through 622k, Or an inverse correlation core, 630). In addition, the multi-channel decorrelator is configured to upmix the first set of K 'decorrelator output signals 632a through 632k' into the second set of N 'decorrelator output signals 612a through 612n' Mixer 640, where N 'is greater than K' (N 'and K' are integers).

However, the structure of a given multi-channel decorrelator 600 should only be considered as an example, and the multi-channel decorrelator 600 may be provided as functional blocks (e.g., pre-mixer 620) , An inverse correlated or non-correlated core 630, and a post mixer 640).

Related to the functionality of the multi-channel decorrelator 600, the concept of performing pre-mixing to derive a second set of K decorrelator input signals from the first set of N decorrelator input signals, Downmixed ") K decorrelator input signals, the concept of performing the decorrelation based on the second set of decorrelator input signals is a reduction in complexity as compared to the concept that the actual decorrelation is applied, for example, directly to the N decorrelator input signals Lt; / RTI > In addition, a second (upmixed) set of N 'decorrelator output signals is obtained based on the first (original) set of decorrelator output signals that can be executed by the upmixer 640. Thus, the multi-channel decorrelator 600 efficiently receives (as viewed externally) the N decorrelator input signals and provides N 'decorrelator output signals based thereon, and only the actual decorrelator core 630 (I.e., a second set of K downmixed decorrelator input signals 622a through 622k of K decorator input signals). Thus, by downmixing or " premixing (which may be preferably linear premixing without any decorrelation function) " at the input of the decorrelation (or decorrelator core 630) By performing upmixing or " postmixing " (e.g., linear upmixing without any additional decorrelation function) based on the (original) output signals 632a through 632k 'of the correlator core 630 The complexity of the channel decorrelator 600 can be substantially reduced when compared to conventional decorrelators.

In addition, it should be noted that the multi-channel decorrelator 600 may be added by any of the features and functions described herein with respect to multi-channel decorrelation and also with multi-channel audio decoders. It should be noted that the features described herein may be added to the multi-channel decorrelator 600 individually or in combination, thereby improving or improving the multi-channel decorrelator 600. [

It should be noted that a multi-channel decorrelator without complexity reduction can be derived from the multi-channel decorrelator described above for K = N (and possibly K '= N' or even K = N = K '= N' .

7. Multichannel audio decoder < RTI ID = 0.0 >

7 illustrates a schematic block diagram of a multi-channel audio decoder 700 in accordance with an embodiment of the present invention.

The multi-channel audio decoder 700 is configured to receive the encoded representation 710 and to provide at least two output signals 712, 714 based thereon. The multi-channel audio decoder 700 includes a multi-channel decorrelator 720, which may be substantially the same as the multi-channel decorrelator 600 according to FIG. In addition, the multi-channel audio decoder 700 may include any of the features and functions of a multi-channel audio decoder as described herein or otherwise described herein in connection with other multi-channel audio decoders have.

In addition, it should be noted that the multi-channel audio decoder 700 includes a particularly high efficiency when compared to conventional multi-channel audio decoders because the multi-channel audio decoder 700 has a high efficiency multi- 720).

8. Multichannel audio encoder < RTI ID = 0.0 >

FIG. 8 illustrates a schematic block diagram of a multi-channel audio encoder 800 in accordance with an embodiment of the present invention. The multi-channel audio encoder 800 receives at least two input audio signals 810 and 812 and generates an encoded representation 814 of the audio content represented by the input audio signals 810 and 812 .

The multi-channel audio encoder 800 includes a downmix signal provider 820 configured to provide one or more downmix signals 822 based on at least two input audio signals 810, 812. The multi-channel audio encoder 800 also includes one or more parameters 832, e.g., cross-correlation parameters or cross-covariance parameters, or inter-object correlation parameters and / Or object level difference parameters). ≪ / RTI > In addition, the multi-channel audio encoder 800 is configured to provide an inverse correlation complexity parameter 842 that describes the complexity of the inverse correlation to be used on the side of the audio encoder (which receives the encoded representation 814) And a parameter provider 840. One or more downmix signals 822, one or more parameters 832 and an inverse correlation parameter 842 are preferably included in the encoded representation 814 in an encoded form.

However, the internal structure of the multi-channel audio encoder 800 (e.g., presence of downmix signal provider 820, parameter provider 830, and inverse correlation complexity parameter provider 840) Should be understood. Different structures are possible as long as the functionality described herein is achieved.

Channel audio encoder 800 provides an encoded representation 814 and one or more downmix signals 822 and one or more parameters 832 are provided to conventional audio encoders (E.g., conventional spatial audio object coding audio encoders or unified voice and audio coding audio decoders) that are similar to or similar to downmix signals and parameters. However, the multi-channel audio encoder 800 is also configured to provide an inverse correlation complexity parameter 842 that allows it to determine the inverse correlation complexity that is applied on the side of the audio decoder. Thus, the inverse correlation complexity can be applied to the currently encoded audio content. For example, it is possible to signal the required inverse correlation complexity, which corresponds to the achievable audio quality, depending on the knowledge of the encoder side of the input audio signals. For example, if spatial features are found to be important for an audio signal, a high decorrelation complexity can be signaled using the decorrelation complexity parameter 842, as compared to the case where spatial features are not so important. Alternatively, if the path of the audio content or the entire audio content is found to require a high complexity decorrelation on the side of the audio decoder for other reasons, then the use of the high decorrelation complexity using the decorrelation complexity parameter 842 Lt; / RTI >

Channel audio encoder 800 may be configured to control the multi-channel audio decoder 800 in order to use the inverse correlation complexity applied to the signal features or required playback features that may be set by the multi- Lt; / RTI >

In addition, it should be noted that the multi-channel audio encoder 800 may be added, either individually or in combination, by any of the features and functions described herein in connection with a multi-channel audio encoder. For example, some or all of the features described herein in connection with multi-channel audio encoders may be added to a multi-channel audio encoder 800. In addition, the multi-channel audio encoder 800 may be adapted to cooperate with the multi-channel audio decoders described herein.

9. According to Fig. 9, Multiple reverse correlation  Based on the input signal A plurality of decorrelated  Method for providing a signal

FIG. 9 shows a flowchart of a method 900 for providing a plurality of decorrelated signals based on a plurality of decorrelated input signals.

The method 900 includes pre-mixing (910) a first set of N de-correlator input signals into a second set of K de-correlator input signals, where K is less than N. The method 900 also includes providing 920 a first set of K 'decorrelator output signals based on the second set of K decorator input signals. For example, a first set of K 'decorrelator output signals may be generated based on a second set of K decorrelator input signals using decorrelation, which may be performed using, for example, an decorrelator core or an decorrelation algorithm Can be provided. The method 900 further includes postmixing 930 a first set of K 'decorrelator output signals into a second set of N' decorrelator output signals, where N 'is greater than K' (N ' The second set of N 'decorrelator output signals, which are the outputs of the method 900, are input to the method 900 based on the first set of N' decorrelator input signals .

It should be noted that the method is based on the same considerations as the multi-channel decorrelators described above. In addition, the method 900 may be implemented separately or in combination with one or more of the features and functions described herein in connection with a multi-channel decorrelator (and, where applicable, also with respect to a multi-channel audio encoder) . ≪ / RTI >

10. A method for providing at least two output audio signals based on an encoded representation,

FIG. 10 shows a flowchart of a method 1000 for providing at least two output audio signals based on an encoded representation.

The method 1000 includes providing 1010 at least two output audio signals 1014 and 1016 based on an encoded representation 1012. [ The method 1000 includes step 1020 of providing a plurality of decorrelated signals based on a plurality of decorrelator input signals, depending on the method 900 according to FIG.

It should be noted that the method 1000 is based on the same considerations as the multi-channel audio decoder 700 according to FIG.

It should also be appreciated that the method 1000 may be added individually or in combination by any of the features and functions described herein in connection with multi-channel decoders.

11. A method for providing an encoded representation based on at least two input audio signals,

11 shows a flowchart of a method 1100 for providing an encoded representation based on at least two input audio signals.

The method 1100 includes providing (1110) at least one downmix signal based on at least two input audio signals (1112, 1113). The method 1100 also includes providing 1220 one or more parameters describing the relationship between the at least two input audio signals 1112, 1114. In addition, the method 1100 includes a step 1130 of providing an inverse correlation complexity parameter that describes the complexity of the decorrelation to be used on the side of the audio decoder. Thus, the encoded representation 1132 is provided based on at least two input audio signals 1112, 1114, and the encoded representation generally describes the relationship between one or more downmix signals, at least two input audio signals Lt; / RTI > and one or more parameters and an encoded complex correlation parameter.

It should be noted that steps 1110, 1120, 1130 may be performed in parallel or in a different order in some embodiments in accordance with the present invention. In addition, the method 1110 is based on the same considerations as the multi-channel audio encoder 800 according to FIG. 8, and the method 1100, separately or in combination, It is to be understood that the present invention can be added by any one of the functions. In addition, it should be noted that the method 1100 may be applied to match a multi-channel audio decoder and a method for providing at least two output audio signals as described herein.

12. Encoded audio representation according to FIG.

Figure 12 shows a schematic representation of an encoded audio representation, in accordance with an embodiment of the present invention. The encoded audio representation 1200 includes an encoded representation 1210 of the downmix signal, an encoded representation 1220 of one or more parameters describing a relationship between at least two input audio signals, And an encoded inverse correlation parameter 1230 that describes the complexity of the inverse correlation. Thus, the encoded audio representation 1200 allows to adjust the inverse correlation complexity used by the multi-channel audio decoder, which improves the decoding efficiency, and possibly an improved audio quality, or an improved balance between coding efficiency and audio quality In addition, it should be noted that the encoded audio representation 1200 may be provided by a multi-channel audio encoder as described herein, and may be used by a multi-channel audio decoder such as the one described herein. Thus, the encoded audio representation 1200 may be added by any of the features and functions described in connection with the multi-channel audio encoders and the multi-channel audio decoders.

13. Symbols and Basic Considerations

Recently, parameter descriptions for bit rate efficient transmission / storage of audio scenes containing a large number of audio objects have been applied to audio coding (e.g., BCC, JSC, SAOC, SAOC1, (ISS1), [ISS2], [ISS3], [ISS4], [ISS5], and [ISS6]). These techniques are aimed at reconstructing the required output audio scene or audio source object based on additional side information describing source objects in the transmitted / stored audio scene and / or audio scene. This reconstruction occurs at the decoder using a parameter notification source separation strategy. In addition, reference is also made to the so-called " MPEG surround " concept, for example described in the international standard ISO / IEC 23003-1: 2007. In addition, reference is also made to the so-called " spatial audio object coding " described in the international standard ISO / IEC 23003-2: 2010. In addition, reference is made to the so-called " unified voice and audio coding " concept described in the international standard ISO / IEC 23003-3: 2012. The concepts from these standards can be used in embodiments according to the invention, for example in the multi-channel audio encoders mentioned here and in the multi-channel audio decoders mentioned here, and some applications may be required.

Below, some background information is described. In particular, an overview of a parameter separation strategy using MPEG spatial audio object coding (SAOC) techniques (see, for example, Reference [SAOC]) will be provided. The mathematical properties of these methods are considered.

13.1. Symbols and Definitions

The following mathematical definitions apply to the present invention:

N _{Objects Number of} audio object signals

N _{DmxCh Number of} downmixed (processed) channels

N _{UpmicCh Number of} upmix (output) channels

N _{Samples Number of} processed data samples

D _Downmix Matrix, Size N _DmxCh N _Objects

X Input Audio object signal, size N _Objects × N _Samples

E _x Object Covariance Matrix, Size N _Objects × N N _Objects

Defined as E _X = XX ^H.

Y _Downmix audio signal, size N _DmxCh × N _Samples

Defined as Y = DX

E _gamma downmix signals, the size N _DmxCh N _DmxCh

Defined as E _γ = YY ^H

G parameter source estimation matrix, size N _Objects x N _DmxCh

E _X D ^H ( DE _X D ^H ) ^-1 and approximate

파라미터로 재구성된 오브젝트 신호, 크기 N _DmxCh ×N _Samples

Object signal reconstructed by parameters, size N _DmxCh × N _Samples

로서 정의됨Approximate to X

Defined as

R Rendering Matrix (specified on the decoder side), size N _UpmixCh N N _Objects

Z Ideally rendered output scene signal, size N _UpmixCh × N _Samples

Defined as Z = RX

렌더링된 파라미터 출력, 크기 N _UpmixCh ×N _Samples

Rendered parameter output, size N _UpmixCh × N _Samples

로서 정의됨

Defined as

C Covariance matrix of ideal output, size N _UpmixCh × N _UpmixCh

Defined as C = RE _X R ^H

W inverse correlation outputs, size N _UpmixCh N N _Samples

, 크기 2N _UpmixCh ×N _Samples S combined signal

, Size 2 N _UpmixCh × N _Samples

E _S Combined signal covariance matrix, size 2 N _UpmixCh × 2 N _UpmixCh

E Defined as _S = SS ^H

최종 출력, 크기 N _UpmixCh ×N _Samples

Final output, size N _UpmixCh × N _Samples

(·) ^H self-adjoint (Hermite) operator.

Express complex conjugate transpose of (·). The symbol (*) ^* may also be used.

F _decorr (·) Inverse correlation function

Additional constants to prevent division by ε 0

H = matdiag ( M ) A matrix containing the values from the main diagonal of the matrix ( M ) on the main diagonal and zero values on the non-diagonal positions

In order to improve the readability of the equations without loss of generality, exponents representing time and frequency dependencies for all introduced variables are omitted herein.

13.2 Parameter separation system

A general parameter separation system may extract multiple audio (s) from a signal mixture (downmix) using supplemental parameter information (e.g., interchannel correlation values, interchannel level difference values, interobserver correlation values, and / The goal is to estimate the sources. A common solution to this task is based on the application of minimum mean square error (MMSE) estimation algorithms. The spatial audio object coding technique is an example of such parameter audio encoding / decoding systems.

13 shows a general principle of a spatial audio object coding encoder / decoder structure. In other words, Fig. 13 shows an outline of the minimum mean square error-based parameter downmix / upmix concept, in schematic block diagram form.

The encoder 1310 receives a plurality of object signals 1312a, 1312b through 1312n. In addition, the encoder 1310 also receives mixing parameters D 1314, which may be, for example, downmix parameters. The encoder 1310 provides one or more downmix signals 1316a, 1316b, etc. based thereon. In addition, the encoder provides additional information 1318. One or more downmix signals and side information may be provided, for example, in encoded form.

Encoder 1310 is generally configured to receive object signals 1312a through 1312n and to couple object signals 1312a through 1312n into one or more downmix signals 1316a through 1316n in dependence on mixing parameters 1314. [ And a mixer 1320, as shown in FIG. In addition, the encoder includes an additional information estimator 1330 configured to derive additional information 1318 from the object signals 1312a through 1312n. For example, the additional information estimator 1330 may determine that the additional information is related to the relationship between the object signals, e.g., cross correlation between object signals (which may be designated as " inter-object correlation (IOC) In order to describe information (which may be designated as " object level difference information OLD ") describing level differences between signals (which may be designated as " object level difference information OLD "Lt; RTI ID = 0.0 > 1318 < / RTI >

One or more downmix signals 1316a and 1316b and side information 1318 may be stored in decoder 1350 or may be transmitted,

Decoder 1350 receives a plurality of output audio signals 1352a through 1352n on the basis of one or more downmix signals 1316a and 1316b and additional information 1318 (e.g., in encoded form) to provide. Decoder 1350 may also receive user interaction information 1354, which may include one or more rendering parameters ( R , which may define a rendering matrix). Decoder 1350 includes a parameter object separator 1360, a side information processor 1370, and a renderer 1380. The side information processor 1370 receives the side information 1318 and provides control information 1372 to the parameter object separator 1360 based on the side information 1318. The parameter object separator 1360 generates a plurality of object signals 1362a to 1360b based on the downmix signals 1360a and 1360b and the control information 1372, which are derived from the side information 1318 by the side information processor 1370. [ 1362n. For example, the object separator can perform decoding of the encoded downmix signals and object separation. The renderer 1380 renders the reconstructed object signals 1362a through 1362n and thereby obtains the output audio signals 1352a through 1352n.

Below, the functionality of the minimum average error-based parameter downmix / upmix concept will be described.

General parameter downmix / upmix processing is performed in a time / frequency selective manner and can be described as a result of the following steps:

• Input "Audio Objects X " and "Mixing Parameters D " are provided in "Encoder 1310". Mixer 1320 downconverts " audio objects X " into a plurality of " downmix signals Y " using " mixing parameters D " (e.g., downmix gains) Mix. The additional information estimator extracts additional information 1318 that describes the characteristics (e.g., covariance characteristics) of the input " audio objects X ".

&Quot; Downmix signals ( Y ) " and additional information are transmitted or stored. These downmix audio signals may be further compressed using audio coders (such as MPEG-1/2 Layer II or III, MPEG-2/4 advanced audio coding, MPEG integrated voice and audio coding, etc.). The side information is also efficiently represented (e.g., as lossless coded relations of object powers and object correlation coefficients) and encoded.

)에 의해 표현되는, (다채널) 표적 장면으로 렌더링된다.Decoder 1350 uses the transmitted side information 1318 to recover the original " audio objects " from the decoded " downmix signals ". Estimates the mixing coefficients (1372) - "side information processor 1370" is applied to up to "the down-mix signal,""Parameter object splitter 1360" in phase to obtain a reconstruction of the X parameter object. The reconstructed " audio objects 1362a through 1362n " are generated by applying " rendering parameters R "

(Multi-channel) target scene, which is represented by a multi-channel target scene.

In addition, it should be noted that the functions described with respect to encoder 1310 and decoder 1350 can also be used in other audio encoders and decoders described herein.

13.3 The minimum mean square error Orthogonality ( orthogonality ) principle

)를 발견하고 평균 제곱 오차를 최소화기를 원하면, 오차 벡터는 벡터들(y₁)에 의해 스패닝되는 공간 상에 직각이 될 것이다:The orthogonality principle is one key characteristic of least mean square error estimators. V is considered to be vectors spanning by the set of (y ₁₎ (spanned) vectors x∈ W is, two Hermitian space (W and V). If a linear combination of vectors (y1 < _{RTI ID} = 0.0 _> W ) <

) And want to minimize the mean square error, the error vector will be perpendicular to the space spanned by the vectors y ₁ :

As a result, the estimation error and the estimate itself are orthogonal:

Geometrically this can be visualized by the embodiments shown in FIG.

) 및 차이 벡터(또는 오차 벡터, e)의 합계와 동일하다. 도시된 것과 같이, 오차 벡터(e)는 벡터들(y₁, y₂)에 의해 스패닝되는 벡터 공간(또는 평면)(V)에 직각이다. 따라서, 벡터(

)는 벡터 공간(V) 내의 x의 최상의 근사치로서 고려될 수 있다.Figure 14 shows a geometric representation for the orthogonality principle in three-dimensional space. As shown, the vector space is spanned by vectors y ₁ , y ₂ . The vector (x) is a vector (

) And a difference vector (or an error vector, e). As shown, the error vector e is orthogonal to the vector space (or plane) V spanned by the vectors y ₁ , y ₂ . Therefore,

) Can be considered as the best approximation of x in vector space (V).

13.4. Parameter reconstruction error

) 및 재구성 오차(X _Error )의 합계로서 표현될 수 있다:With the definition of a matrix containing N signals: x and X _Error , the following properties can be formulated: The original signal is the parameter reconstruction (

) And a reconstruction error ( X _Error ): < EMI ID =

) 및 추정 오차들의 공분산 매트릭스(

)의 합계로서 공식화될 수 있다:Due to the orthogonality principle, the covariance matrix ( E _X = XX ^H ) of the original signals is transformed into a covariance matrix of reconstructed signals

) And the covariance matrix of the estimation errors (

): ≪ RTI ID = 0.0 >

)을 도입한다.If the input objects X are not in space spanned by the downmix channels (e.g., the number of downmix channels is less than the number of input channels) and the input objects are represented as linear combinations of downmix channels When not available, the minimum mean square error-based algorithms use reconstruction inaccuracy

).

13.5. Between Objects  relation

In auditory systems, cross covariance (interference / correlation) is closely related to the perception of the envelopment surrounded by the sound and the perceived width of the sound source. For example, inter-object correlation (IOC) parameters in spatial audio object coding based systems are used for characterization of these characteristics:

One embodiment of the reproduction of a sound source using two audio signals is contemplated. If the correlation value between objects is close to 1, the sound is perceived as a well-localized point source. If the correlation value between objects is close to zero, the perceived sound source width increases and for extreme cases this can be perceived as two distinct sources [Blauert, Chapter 3].

13.6 Reconstruction Inaccuracy  Compensation for

In the case of a defective parameter reconstruction, the output signal may exhibit lower energy compared to the original objects. The error within the diagonal elements of the covariance matrix may cause audible level differences and errors in the non-diagonal elements within the distorted spatial acoustic image (as compared to the ideal reference output). The proposed method has a purpose to solve such a problem.

In MPEG Surround (MPS), for example, this problem is solved only for some specific channel based processing scenarios, namely mono / stereo downmix and limited fixed output configurations (e.g., mono, stereo, 5.1, Lt; / RTI > In object-oriented techniques such as spatial audio object coding using a mono / stereo downmix, this problem is handled only by the application of MPEG surround post-processing rendering for a 5.1 output configuration.

Existing solutions are limited to standard output configurations and a fixed number of input / output channels. That is, they are realized as a subsequent application of some blocks, implementing only "mono-to-three" (or "stereo-to-three") channel decorrelation methods.

 Thus, a common solution for parameter reconstruction inaccuracy compensation (e.g., energy level and correlation property correction methods) is desirable, which can be applied for a flexible number of downmix / upmix channels and arbitrary output configuration settings.

13.7. conclusion

In conclusion, an overview of the codes has been provided. In addition, a parameter separation system on which the embodiments according to the present invention are based has been described. In addition, it has been described that the orthogonality principle is applied to the Chi square mean square error estimation. In addition, equations have been provided for the calculation of the applied covariance matrix ( E _X ) in the presence of a reconstruction error ( X _Error ). The relationship between so-called inter-object correlation values and the elements of the covariance matrix ( E _X ) has also been described, for example from inter-object correlation values (which may be included in the parameter side information) Can be applied to embodiments according to the present invention to derive the required communicative features (or correlation features) from the differences. In addition, it has been described that the features of the reconstructed object signals may differ from those required due to faulty reconstruction. In addition, it has been described that existing solutions for handling problems are limited to some specific output configurations and rely on certain combinations of standard blocks, making conventional solutions inflexible.

14. Embodiment according to Fig. 15

14.1. Concept overview

Embodiments in accordance with the present invention extend to minimum mean square error parameter reconstruction methods used in parameter audio separation strategies as an inverse correlation solution for any number of downmix / upmix channels. Embodiments in accordance with the present invention, such as the apparatus of the present invention and the method of the present invention, can compensate for energy loss during parameter reconstruction and recover the correlation properties of the estimated objects.

Figure 15 provides an overview of a parameter downmix / upmix concept with an integrated decorrelation path. In other words, FIG. 15 shows a parameter reconstruction system with a decorrelation applied on the rendered output in the form of a schematic block diagram.

The system according to FIG. 15 includes an encoder 1510, which is substantially the same as encoder 1310 according to FIG. The encoder 1510 receives the plurality of object signals 1512a through 1512n and provides additional information 1518 as well as one or more downmix signals 1516a and 1516b based thereon. The downmix signals 1516a and 1516b may be substantially the same as the downmix signals 1316a and 1316b and may be designated by Y. [ The additional information 1518 may be substantially the same as the additional information 1318. [ However, the side information may include, for example, an inverse correlation mode parameter or an inverse correlation method parameter, or an inverse correlation complexity parameter. In addition, the encoder 1510 may receive mixing parameters 1514. [

The parameter reconstruction system also includes the transmission and / or storage of one or more downmix signals 1516a and 1516b and additional information 1518, wherein the transmission and / or storage is designated as 1540 and the one or more downmix signals 1516a, 1516b and additional information 1518, which may include parameter side information, may be encoded.

15 further includes one or more (possibly encoded) downmix signals 1516a, 1516b that are transmitted or stored and additional information 1518 that is transmitted or stored (possibly encoded) And based thereon, to provide output audio signals 1552a through 1552n. Decoder (1550, which may be considered as a multi-channel audio decoder) includes a parameter object separator (1560) and a side information processor (1570). In addition, the decoder 1550 includes a renderer 1580, an decorrelator 1590, and a mixer 1598.

로서 지정되고 디코딩된 오디오 신호들로서 고려될 수 있는, 오브젝트 신호들(1562a 내지 1562b)을 제공하도록 구성된다. 제어 정보(1572)는 예를 들면, 재구성된 오브젝트 신호들(예를 들면, 디코딩된 오디오 신호들들(1562a 내지 1562b))을 획득하기 위하여 파라미터 오브젝트 분리기 내의 다운믹스 신호들 상에 적용되려는(예를 들면, 인코딩된 다운믹스 신호들(1516a, 1516b)로부터 유도되는 디코딩된 다운믹스 신호들에 대한) 비-믹싱 계수들을 포함할 수 있다. 렌더러(1580)는 디코딩된 오디오 신호들(1562a 내지 1562n, 재구성된 오디오 신호들일 수 있고, 예를 들면 입력 오브젝트 신호들(1512a 내지 1512n)과 상응할 수 있는)을 렌더링하고, 이에 의해 복수의 렌더링된 오디오 신호(1582a 내지 1582n)를 획득한다. 예를 들면, 렌더러(1580)는 예를 들면 사용자 상호작용에 의해 제공될 수 있고, 예를 들면 렌더링 매트릭스를 정의할 수 있는, 렌더링 파라미터들(R)을 고려할 수 있다. 그러나, 대안으로서, 렌더링 파라미터들은 인코딩된 표현(인코딩된 다운믹스 신호들(1516a, 1516b)과 인코딩된 부가 정보(1518)를 포함할 수 있는)로부터 얻을 수 있다.The parameter object separator 1560 receives the control information 1572 provided by the side information processor 1570 based on the one or more downmix signals 1516a and 1516b and the side information 1518, Also

And can be considered as decoded audio signals. Control information 1572 may include information that is to be applied on the downmix signals in the parameter object separator to obtain reconstructed object signals (e.g., decoded audio signals 1562a through 1562b) (E.g., for decoded downmix signals derived from encoded downmix signals 1516a, 1516b). The renderer 1580 may be adapted to render the decoded audio signals 1562a through 1562n, which may be reconstructed audio signals and may correspond to, for example, input object signals 1512a through 1512n, And acquires the audio signals 1582a to 1582n. For example, the renderer 1580 may consider rendering parameters R, which may be provided by, for example, user interaction and may, for example, define a render matrix. However, as an alternative, rendering parameters may be obtained from an encoded representation (which may include encoded downmix signals 1516a, 1516b and encoded side information 1518).

The decorrelator 1590 receives the rendered audio signals 1582a through 1582n and is configured to provide the decorrelated audio signals 1592a through 1592n, also designated as W , based thereon. The mixer 1598 receives the rendered audio signals 1582a through 1582n and the decoded correlated audio signals 1592a through 1592n and thereby generates the rendered audio signal 1552a through 1552n to obtain the output audio signals 1552a through 1552n. 1582a through 1582n and the decorrelated audio signals 1592a through 1592n. The mixer 1598 also uses control information 1574 derived by the side information processor 1570 from the encoded side information 1518, as will be described below.

14.2. Inverse correlator  function

In the following, some details will be described with respect to the inverse correlator 1590. However, it should be noted that different decorrelator concepts may be used, some of which will be described below.

)는 입력 신호(

)에 직각인 출력 신호(w)를 제공한다. 출력 신호(w)는 동일한(입력신호(

)와) 스펙트럼 및 시간적 엔벨로프 특성들(또는 적어도 유사한 특성들)을 갖는다. 게다가, 신호(w)는 유사하게 지각되고 입력 신호(

)와 동일한(또는 유사한) 주관적 품질을 갖는다(예를 들면, [SAOC] 참조).In one embodiment, the decorrelator function (

) ≪ / RTI >

Lt; RTI ID = 0.0 > w < / RTI > The output signal w has the same (input signal

) And spectral and temporal envelope properties (or at least similar properties). In addition, the signal w is similarly delayed and the input signal < RTI ID = 0.0 >

) (See, for example, [SAOC]).

이고 i≠j에 대하여

인 것과 같이,

) 다중 출력을 생산하면 이는 바람직하다.In the case of multiple input signals, if the decorrelation function is orthogonal to each other (e.g., for all i and j

And for i ≠ j

As is the case,

) It is desirable to produce multiple outputs.

The exact specification for the implementation of the inverse correlation function is beyond the scope of this description. For example, a bank of some infinite impulse response (IIR) based decorrelators specified in the MPEG Surround standard can be used for correlation purposes [MPS].

)을 갖는 입력(

) 및 출력(

)을 위하여 아래의 공분산 매트릭스의 특성들이 유지된다:The general decorrelators described in this description are assumed to be ideal. This indicates that the output of each decorrelator (in addition to the perceptual requirements) is perpendicular to its input and to the output of all other decorrelators. Therefore,

) ≪ / RTI >

) And output (

) The following properties of the covariance matrix are retained:

From these relationships, the following is followed:

The decorrelator output W can be used to compensate for the prediction inaccuracies in the minimum mean square error estimator by using predicted signals as inputs (see that the prediction error is orthogonal to the predicted signals).

It should also be noted that the prediction errors are not orthogonal between them in the general case. Thus, one object of the inventive concept (e.g., method) is to provide a covariance matrix of the resulting mixture (e. G., Output audio signals 1552a through 1552n) (E. G., Decorrelator input) signals (e. G., Rendered audio signals 1582a through 1582n) and " wet " Correlated audio signals 1592a through 1592n), for example.

In addition, it should be noted that the complexity reduction for the decorrelation unit, which may be acceptable or acceptable, may be used to bring some of the defects of the decorrelation signal, which will be described in detail below.

14.3. Reverse correlation  Output covariance correction using signals

The concept for adjusting the covariance characteristics of the output audio signals 1552a through 1552n will be described below to obtain a reasonably good auditory effect.

, 예를 들면 렌더링된 오디오 신호들(1582a 내지 1582n)) 및 그것의 역상관된 부분(W)의 가중 합계로서 출력 신호(

, 예를 들면 출력 오디오 신호들(1552a 내지 1552n))을 포함한다. 이러한 합계는 다음과 같이 표현될 수 있다:The proposed method for output covariance error correction is based on the reconstructed signal (

(E.g., rendered audio signals 1582a through 1582n) and its decorrelated portion W ,

, E.g., output audio signals 1552a through 1552n). This sum can be expressed as: < RTI ID = 0.0 >

)에 적용되는 믹싱 매트릭스들(P) 및 역상관된 신호(W)에 적용되는 M은 다음의 구조를 갖는다(여기서 N=N _UpmixCh 이고, N _UpmixCh 은 출력 오디오 신호들의 수와 동일할 수 있는, 렌더링된 오디오 신호들의 수를 지정한다):Direct signal (

(Where N = N _UpmixCh and N _UpmixCh is the number of output audio signals, which may be equal to the number of output audio signals), and M applied to the mixing matrices P and the decorrelated signal W, Specifies the number of rendered audio signals):

)에 대한 기호를 적용하여, 아래와 같이 생성된다:The combined matrix ( F = [ PM ]) and signal (

), It is generated as follows:

)의 공분산 매트릭스(

)는 다음과 같이 정의된다:Using this expression, the output signal (

) Covariance matrix (

) Is defined as: < RTI ID = 0.0 >

The target covariance ( C ) of the ideally generated rendered output scene is defined as:

)가 표적 공산에 근사치이거나 또는 동일하도록 계산된다:The mixing matrix ( F ) can be expressed as the covariance matrix of the final output

) Is calculated to be approximate or equal to the target conjugate:

Mixing the matrix (F) is, for example, is calculated as the following known amount of functions (F = F such as (E _s, E _X, R):

Where the matrices U , T and V , Q may be determined using a single value decomposition (SVD) of the covariance matrices E _S and C , for example,

C = UTU ^H , E _S = VQV ^H

The prototype matrix H may be selected according to the required weights for the direct and the decorrelated signal paths.

For example, the possible prototype matrix H can be determined as follows:

, 여기서

이다.

, here

to be.

Below, a mathematical derivation of the general matrix F will be provided.

In other words, the derivation of the mixing matrix F for a general solution will be described below.

The covariance matrices E _S and C may be expressed using, for example, single value decomposition (SVD) as follows:

E _S = VQV ^H , C = UTU ^H.

Where T and Q are diagonal matrices with single values of E _S and C , respectively, and U and V are unitary matrices containing corresponding single vectors.

If the application of Schur triangulation or eigenvalue decomposition (instead of single valued decomposition) yields similar results (or even if the diagonal matrices Q and T are limited to positive values, ).

C)에 적용하여, 다음을 생성한다(적어도 근사치로):This decomposition is called a requirement ( E _Z

C ) to generate (at least approximate):

인 특성을 갖는 크기 N _UpmixCh ×2N _UpmixCh 의 프로토타입 매트릭스(H)가 적용될 수 있다:In order to note the dimensionality of covariance matrices, in some cases a rule is needed. For example,

A prototype matrix ( H ) of size N _UpmixCh × 2 N _{UpmixCh with} the in- _feature can be applied:

The mixing matrix F can then be determined as follows:

The prototype matrix H is selected according to the required weights for the direct and inverse correlated signal paths. For example, a possible prototype matrix ( H ) can be determined as follows:

, 여기서

이다.

, here

to be.

Depending on the condition of the covariance matrix ( E _S ) of the combined signals, the last equation may be needed to include some regularization, but otherwise it must be numerically stable.

)에 의해, 또는 동등하게 벡터()에 의해 표현되는) 및 역상관된 오디오 신호들(매트릭스(W)에 의해, 또는 동등하게 벡터(w)에 의해 표현되는)을 기초로 하여 출력 오디오 신호들(매트릭스(

)에 의해, 또는 동등하게 벡터(w)에 의해 표현되는)을 유도하기 위한 개념이 설명되었다. 알 수 있는 것과 같이, 일반 매트릭스 구조의 두 개의 믹싱 매트릭스(P 및 M)가 공통으로 결정된다. 예를 들면, 위에 정의된 것과 같이, 결합된 매트릭스(F)는 출력 오디오 신호들(1552a 내지 1552n)의 공분산 매트릭스(

)가 요구되는 공분산(또한 표적 공분산으로서 지정되는, C)과 근사치이거나 또는 동등하도록 결정될 수 있다. 요구되는 공분산 매트릭스(C)는 예를 들면, 렌더링 매트릭스(R, 예를 들면, 사용자 상호작용에 의해 제공될 수 있는)의 지식을 기초로 하고 오브젝트 공분산 매트릭스(E _X )의 지식을 기초로 하여 유도될 수 있고, 이는 예를 들면 인코딩된 부가 정보(1518)를 기초로 하여 유도될 수 있다. 예를 들면, 오브젝트 공분산 매트릭스(E _X )는 위에서 설명되고 인코딩된 부가 정보(1518) 내에 포함될 수 있는, 오브젝트간 상관 값들(IOC)을 사용하여 유도될 수 있다. 따라서, 표적 공분산 매트릭스(C)는 예를 들면, 정보(1574)로서 또는 정보(1574)의 일부분으로서 부가 정보 프로세서(1570)에 의해 제공될 수 있다.In conclusion, the rendered audio signals (matrix

) Or on the basis of the decorrelated audio signals (represented by the matrix W , or equivalently, by the vector w) (matrix(

), Or equivalently, by the vector w) has been described. As can be seen, the two mixing matrices P and M of the general matrix structure are commonly determined. For example, as defined above, the combined matrix F may be a covariance matrix of output audio signals 1552a through 1552n

) May be approximated or equal to the required covariance (also designated as the target covariance, C ). The required covariance matrix C is based on knowledge of, for example, a rendering matrix R (e.g., which may be provided by user interaction) and based on knowledge of the object covariance matrix E _X Which may be derived, for example, based on the encoded side information 1518. [ For example, the object covariance matrix E _X may be derived using inter-object correlation values (IOC), which may be included within the encoded side information 1518 as described above. Thus, the target covariance matrix C may be provided by the side information processor 1570, for example, as information 1574 or as part of information 1574. [

However, as an alternative, the supplementary information processor 1570 may also provide the mixing matrix F as information 1574 directly to the mixer 1598.

In addition, calculation rules for the mixing matrix F , using single value decomposition, have been described. However, there are some degrees of freedom, because the entries ( a _{i, i} , b _{i, i} ) of the prototype matrix H can be selected. Preferably, the entries of the prototype matrix H may be selected to be somewhere between zero and one. If the values ( a _{i, i} ) are chosen to be close to 1, there will be a significant mixing of the rendered audio signals and the effect of the decorrelated audio signals is relatively small, which may be desirable in some situations. However, in some other situations it may be preferable to have only a weak mixing between the rendered audio signals with a relatively large impact of the decoded audio signals. In this case, the values ( b _{i, i} ) are generally chosen to be greater than a _{i, i} . Thus, the decoder 1550 can be applied to requirements by appropriately selecting the entries of the prototype matrix H.

14.4. Simplified methods for output covariance correction

In this section, two alternative schemes for the above-mentioned mixing matrix F are described with the preferred algorithms for determining its values. Two strategies are designed for different input content (e.g., audio content):

- covariance adjustment methods for highly correlated content (e.g., channel based input with high correlation between different channel portions)

Energy compensation methods for independent input signals (e.g., object-based inputs, which are generally estimated independently).

14.4.1. Covariance Adjustment Method (A)

, 예를 들면 렌더링된 오디오 신호들(1582a 내지 1582n))가 최소 평균 제곱 오차 의미에서 이미 최적인 것으로 고려하면, 출력(

)의 공분산 특성들을 향상시키기 위하여 파라미터 재구성들(

, 예를 들면 출력 오디오 신호들(1552a 내지 1552n))을 변형하는 것은 일반적으로 바람직하지 않은데 그 이유는 이것이 분리 품질에 영향을 미칠 수 있기 때문이다.signal(

, E.g., rendered audio signals 1582a through 1582n) are already considered optimal in the minimum mean square error sense, the output

Lt; RTI ID = 0.0 > (c) < / RTI &

, E.g., output audio signals 1552a through 1552n) is generally undesirable because it can affect the quality of the separation.

If only a mixture of decorrelated signals W is adjusted, the mixing matrix P may be reduced to a unit matrix (or a plurality of it). Thus, a simplified method can be described by setting:

The final output of the system can be expressed as:

As a result, the final output covariance of the system can be expressed as:

) 사이의 차이(△ _E )는 다음에 의해 주어진다:The ideal (or required) output covariance matrix ( C ) and covariance matrix (covariance matrix) of the rendered parameter reconstruction (e.g., rendered audio signals)

) ≪ / RTI & _gt ; is given by: < RTI ID = 0.0 >

Thus, the mixing matrix M is determined to be:

)에 근사치일 것이다:The mixing matrix M is equal to the covariance difference between the covariance of the required covariance and dry signals (e.g., the rendered audio signals) and the covariance matrix of the mixed and deconvolved signals MW Or approximate. As a result, the covariance of the final output is the target covariance (

):

The matrices (U, T and V, Q) can be determined using the singular value decomposition (SVD) of the covariance matrix, for example to generate, as shown below (△ _E and E _W):

_E = UTU ^H , E _W = VQV ^H

This approach ensures excellent cross-correlation reconstruction that maximizes the use of pure output (e.g., using the rendered audio signals 1582a through 1582n) and uses the degree of freedom of mixing only the decorrelated signals. In other words, There is no mixing allowed between different rendered audio signals when combining the audio signals (or their scaled version) with one or more decorrelated audio signals. However, the cross-correlation features of the output audio signals Or a scaled version of a plurality of rendered audio signals or a scaled version thereof, with the same or different scaling, to adjust cross covariance characteristics. Is defined by the same matrix ( M ).

Below, a mathematical derivation for the constrained matrix F will be provided.

In other words, the derivation of the mixing matrix M for the simplified method " A " will be described.

The covariance matrices? _E and E _W can be expressed using, for example, single valued decomposition (SVD) as follows:

_E = UTU ^H , E _W = VQV ^H

Where T and Q are respectively _ΔE and E _W And U and V are unit matrices containing corresponding single vectors.

It should be noted that the application of Schur triangulation or eigenvalue decomposition (instead of single value decomposition) leads to similar results (or even if the diagonal matrices Q and T are limited to positive values, even the same results).

C)에 적용하여, 다음을 생성한다(적어도 근사치로):This decomposition is called a requirement ( E _Z

C ) to generate (at least approximate):

It should be noted that both of the equations express the square of the matrix and the inventors of the present invention drop the square and solve for the complete matrix M.

The mixing matrix M may then be determined as follows:

This method can be derived from the general method by setting the prototype matrix H as follows:

Depending on the condition of the covariance matrix ( E _W ) of the applied signals, the last equation may be required to include some regularization, but otherwise it must be numerically stable.

14.4.2. Energy compensation method (B)

Sometimes it is possible to mix parameter reconstructs (e.g., rendered audio signals) or decorrelated signals (depending on the application scenario), but each reconstructed signal (e.g., a rendered audio signal) It is not desirable to mix separately with its inherent decoded signal.

In order to achieve this requirement, additional constraints must be introduced in the simplified method " A ". Now, the mixing matrix M of the applied signals (decorrelated signals) is required to have a diagonal shape:

The main purpose of this approach is to use the decorrelated signals to compensate for the loss of energy in the parameter reconstruction (e. G., The rendered audio signal) and diagonal deformation of the covariance matrix of the output signal is ignored, There is no direct treatment of correlations. Thus, no cross leakage is introduced between the output objects / channels (e.g. between the rendered audio signals) in the application of the decorrelated signals.

As a result, only the main diagonal of the target covariance matrix (or the required covariance matrix) can be reached, and the non-diagonals are based on the parameter reconstruction and the added decorrelated signals. This method is most appropriate for object-based applications where signals can be considered as uncorrelated.

)의 에너지들과 상응하는 공분산 매트릭스 엔트리들이 요구되는 에너지들과 동일하도록 계산된 대각선 매트릭스(M)를 갖는

에 의해 주어진다:The final output (e.g., output audio signals) of the method may be reconstructed signals

With the diagonal matrices M calculated so that the corresponding covariance matrix entries correspond to the required energies

Lt; / RTI >

C can be determined as described above for the general case.

For example, the mixing matrix M may include compensation signals (which may be described by diagonal elements of the crossover crossover matrix C ) and the energies of the parameter reconstructions (which may be determined by the audio decoder ) Into the energies of the decorrelated signals (which may be determined by the audio decoder):

Where [lambda] _Dec is the non-negative threshold used to limit the amount of the decorrelated component added to the output signals (e.g., [lambda] _Dec = 4).

The energies can be reconstructed with parameters (e.g., using object level difference information, inter-object correlations and rendering coefficients), or actually computed by a decoder (typically computationally expensive) I must understand.

This method can be derived from the general method by setting the prototype matrix H as follows:

This method explicitly maximizes the use of purely rendered outputs. The method is equivalent to the simplification " A " when the covariance matrices have no non-diagonal entries.

This method has a reduced computational complexity.

It should be understood, however, that the energy compensation method does not necessarily indicate that the cross-correlation terms are not modified. This is valid if the inventors of the present invention use ideal decorrelators and there is no reduction in complexity for the decorrelating unit. The idea of this method is to restore energy and ignore the transformations in the cross terms (a change in cross terms will not substantially modify the correlation properties and will not affect the overall spatial effect).

14.5. Mixing  matrix( F ) Requirements

It will be described below that the mixing matrix F , whose derivation is described in Sections 14.3 and 14.4, satisfies the requirements to prevent degradations.

To avoid degradation of output, some method for compensating for parameter reconstruction should produce results with the following characteristics: if the rendering matrix is the same as the downmix matrix, then the output channels must be the same as the downmix channels (or At least approximate). The proposed mode satisfies these characteristics. If the rendering matrix is the same as the downmix matrix ( R = D ), the parameter reconstruction is given by:

The required covariance matrix would be:

_{^{C = RE X R H = DE}} X D H = E Y.

Therefore, the equation to be solved for obtaining the mixing matrix F is as follows:

는 0들의 크기 N _UpmixCh ×N _UpmixCh 의 정방 매트릭스(square matrix)이다. F에 대한 이전의 방정식을 해결하여, 아래와 같이 획득될 수 있다:here

_Is a square matrix of the size of 0 N _UpmixCh N _UpmixCh . By solving the previous equation for F , it can be obtained as follows:

This means that the decorrelated signals will have a zero-weighting in the add, and the final output will be given by the same pure signals as the downmix signals:

As a result, in such a rendering scenario, the given requirements for the system output are satisfied to be the same as the downmix signal.

14.6. Covariance matrix ( E _S )

Knowledge of the covariance matrix E _S of the combined signals S to obtain the mixing matrix F is necessary or at least desirable.

) 및 역상관기 출력(W)으로부터) 공분산 매트릭스(E _S )를 직접적으로 추정하는 것이 가능하다. 비록 접근법이 더 정확한 결과들에 이르게 할 수 있더라도, 이는 관련 계산 복잡도 때문에 실용적이지 않을 수 있다. 제안된 방법들은 공분산 매트릭스(E _S )의 파라미터 근사치를 사용한다.In principle, from available signals (i.e., parameter reconstruction (

) And the decorrelator matrix ( E _S ) directly from the decorrelator output ( W ). Although the approach may lead to more accurate results, this may not be practical due to the associated computational complexity. The proposed methods use parameter approximations of the covariance matrix ( E _S ).

The general structure of the covariance matrix ( E _S ) can be expressed as:

)는 직접적인 신호들(

) 및 역상관된 신호들(W) 사이의 교차 공분산이다.Here,

) ≪ / RTI >

≪ / RTI > and the decorrelated signals W. < RTI ID = 0.0 >

Assuming that the decorrelators are ideal (i. E. , Energy conserving, the outputs are orthogonal to the inputs and all outputs are mutually orthogonal), the covariance matrix E _S can be expressed using the simplified form :

)의 공분산 매트릭스(

)는 다음과 같이 파라미터로 결정된다:Signal reconstructed with parameters (

) Covariance matrix (

) Is determined by the following parameters:

의 대각선 요소들만을 포함하는 것으로 추정된다:The covariance matrix E _W of the decorrelated signal W satisfies the mutual orthogonality characteristic and is expressed as

It is assumed that only diagonal elements of < RTI ID = 0.0 >

If the assumption of mutual orthogonality and / or energy conservation is violated (e. G. When the number of available decorrelators is less than the number of signals to be decorrelated), the covariance matrix ( E _W ) is estimated as Can:

15. Reverse correlation  Reduced complexity for units

Below, how the complexity of the decorrelators used in embodiments according to the present invention can be reduced will be described.

It should be understood that the implementation of decorrelator functionality is sometimes computationally complex. In some applications (e.g., small decoder solutions), a restriction on the number of decorrelators to be introduced due to limited computational resources may be required. This section provides a description of the means for reducing the decorrelator unit complexity by controlling the number of applied decorrelators (or decorrelators). The decorrelator unit interface is shown in Figures 16 and 17.

) 같은, N 역상관기 입력 신호들(1610a 내지 1610n)을 수신하도록 구성된다. 게다가, 역상관 유닛(1600)은 N 역상관 출력 신호들(1612a 내지 1612n)을 제공한다. 역상관 유닛(1600)은 예를 들면, N 개별 역상관기들(또는 역상관 함수들, 1620n 내지 1620n)을 포함할 수 있다. 예를 들면, 각각의 개별 역상관기들(1620a 내지 1620n)은 역상관기 입력 신호들(1610a 내지 1610n) 중 관련된 하나를 기초로 하여 역상관기 출력 신호들(1612a 내지 1612n) 중 하나를 제공할 수 있다. 따라서, N 개별 역상관기들, 또는 역상관 함수들(1620a 내지 1620n)은 역상관기 입력 신호들(1610a 내지 1610n)을 기초로 하여 N 역상관된 신호들(1612a 내지 1612n)을 제공하기 위하여 필요할 수 있다.Figure 16 shows a schematic block diagram of a simple (conventional) decorrelation unit. The decorrelation unit 1600 according to FIG. 16 may include, for example, rendering rendered audio signals

), ≪ / RTI > N decorator correlator input signals 1610a through 1610n. In addition, the decorrelation unit 1600 provides N de-correlated output signals 1612a through 1612n. The decorrelation unit 1600 may include, for example, N individual decorrelators (or decorrelation functions, 1620n through 1620n). For example, each of the individual decorrelators 1620a through 1620n may provide one of the decorrelator output signals 1612a through 1612n based on a related one of the decorrelator input signals 1610a through 1610n . Thus, N individual decorrelators, or decorrelation functions 1620a through 1620n, may be needed to provide N decorrelated signals 1612a through 1612n based on decorrelator input signals 1610a through 1610n have.

)일 수 있고, N 역상관기 출력 신호들(1712a 내지 1712n)은 역상관된 오디오 신호들(W)일 수 있다.However, FIG. 17 shows a schematic block diagram of a reduced complexity decorrelation unit 1700. Decreased complexity decorrelation unit 1700 is configured to receive N decorrelator input signals 1710a through 1710n and to provide N decorrelator output signals 1712a through 1712n based thereon. For example, the N de-correlator input signals 1710a through 1710n may be generated from the rendered audio signals

), And the N de-correlator output signals 1712a through 1712n may be decorrelated audio signals W.

)를 사용하여 표현될 수 있다. 역상관 유닛(또는 동등하게는, 다채널 역상관기, 1700)는 또한 역상관기 입력 신호들(1722a 내지 1722k)의 제 1 세트의 K 신호들을 수신하고, 이를 기초로 하여, 역상관기 출력 신호들(1732a 내지 1732k)의 제 1 세트를 구성하는 K 역상관기 출력 신호들을 제공하도록 구성되는, 역상관기 코더(1730)를 포함한다. 예를 들면, 역상관기 코더(1730)는 K 개별 역상관기들(또는 역상관 함수들)을 포함할 수 있고, 각각의 개별 역상관기들(또는 역상관 함수들)은 K 역상관기 입력 신호들(1722a 내지 1722k)의 제 2 세트의 상응하는 역상관기 입력 신호를 기초로 하여 K 역상관기 출력 신호들(1732a 내지 1732k)의 제 1 세트의 역상관기 출력 신호들 중 하나를 제공한다. 대안으로서, 주어진 역상관기, 또는 역상관 함수는 K 역상관기 출력 신호들(1732a 내지 1732k)의 제 1 세트의 각각의 역상관기 출력 신호들이 K 역상관기 입력 신호들(1722a 내지 1722k)의 제 2 세트의 역상관기 입력 신호들 중 단일의 하나를 기초로 하도록 K번 적용될 수 있다.The decorrelator 1700 receives a first set of N decorrelator input signals 1710a through 1710n and provides a second set of K decorrelator input signals 1722a through 1722k based on the first set of N decorrelator input signals 1710a through 1710n, (Or equivalently, premixing functionality, 1720). For example, the pre-mixer 1720 may be a so-called " pre-mixer " to derive a second set of K decorrelator input signals 1722a through 1722k based on the first set of N decorrelator input signals 1710a through 1710n, Pre-mixing " or " down-mixing " For example, a second set of K signals of K decorrelator input signals 1722a through 1722k are applied to a matrix

). ≪ / RTI > The decorrelator unit (or equivalently, a multi-channel decorrelator, 1700) also receives the first set of K signals of decorrelator input signals 1722a through 1722k and, based thereon, decorrelator output signals Correlator coder 1730 that is configured to provide K decorator output signals that make up the first set of symbols 1732a through 1732k. For example, the decorrelator coder 1730 may include K individual decorrelators (or decorrelation functions), and each individual decorrelator (or decorrelation functions) may include K decorrelator input signals Correlator output signals 1732a through 1732k based on the corresponding decorrelator input signals of the first set of K correlator output signals 1732a through 1722k. Alternatively, a given decorrelator, or decorrelation function, may be used such that each of the decorrelator output signals of the first set of K decorrelator output signals 1732a through 1732k is associated with a second set of K decorrelator input signals 1722a through 1722k Lt; RTI ID = 0.0 > K < / RTI >

The decorrelator unit 1700 also receives the K-decorrelator output signals 1732a-1732k of the first set of decorrelator output signals and, based thereon, a decorrelator (which constitutes " outer " decorrelator output signals) And a post mixer 1740 configured to provide a second set of N signals 1712a through 1712n of output signals.

It should be noted that pre-mixer 1720 may preferably perform a linear mixing operation, which may be described by a _pre- mixing matrix ( M _pre ). In addition, the post mixer 1740 preferably outputs the decorrelator output signals from the first set of K decorrelator output signals 1732a through 1732k (i.e., from the output signals of the decorrelator core 1730) (Or upmixing) operation, which may be described by a post-mixing matrix ( M _post ) to derive N inverse correlator output signals 1712a through 1712n.

The main concept of the proposed method and apparatus is to reduce the number of input signals for N to K decorrelators (or decorrelator cores) by:

Pre-mixing of signals (e.g., rendered audio signals) to a low number of channels such as:

The application of the decorrelation using the available K correlation correlators (for example of the decorrelator core) as follows:

Upmixing of the correlated signals back to the N channels as follows:

.

.

)이 잘 조절되도록(도치(inversion) 운영과 관련하여) 다운믹스/렌더링/상관/등등의 정보를 기초로 하여 구성될 수 있다. 포스트믹싱 매트릭스는 다음과 같이 계산될 수 있다:The pre-mixing matrix ( M _pre )

(In relation to the inversion operation ) so that they are well-controlled (for example, in the case of an inversion operation). The post-mixing matrix can be calculated as: < RTI ID = 0.0 >

또는

)의 공분산 매트릭스가 대각선이더라도(이상적인 역상관기들을 가정하여), 최종 역상관된 신호들의 공분산 매트릭스(W)는 이러한 종류의 처리를 사용할 때 더 이상 상당히 대각선 같지는 않을 것이다. 따라서, 공분산 매트릭스는 다음과 같이 믹싱 매트릭스들을 사용하여 추정될 수 있다:Although the intermediate decorrelated signals (

or

(Assuming ideal decorrelators), the covariance matrix W of the final decorrelated signals will no longer be significantly diagonal when using this kind of processing. Thus, the covariance matrix can be estimated using mixing matrices as follows:

.

.

The number of inverse correlators (or individual inverse correlators) used, K is not specified and depends on the required computational complexity and the available decorrelators. Its value can vary from N (highest computational complexity) to 1 (lowest computational complexity).

The number of input signals to the decorrelator unit, N, is arbitrary and the proposed method supports any number of input signals, independent of the rendering configuration of the system.

For example, with an output channel of a high number of which depends on the output channel, in the application of using the 3-D audio content, one possible expression for the pre-mixing matrix _(pre M) is described below.

Below, how the pre-mixing (and hence the post-mixing performed by the post mixer 1740) to be performed by the pre-mixer 1720 is adjusted if the decorrelation unit 1700 is used in a multi-channel audio decoder And the first set of decorrelator input signals 1710a through 1710n of the decorrelator input signals are associated with different spatial locations of the audio scene.

For this purpose, Figure 18 shows a table representation of the loudspeaker locations used for different output formats.

 In table 1800 of FIG. 18, first row 1810 describes the loudspeaker exponent number. A second row 1820 describes the loudspeaker level. The third row 1830 describes the azimuth position of each loudspeaker, and the fourth row 1832 describes the azimuth error of the location of the loudspeaker. A fifth row 1840 describes the altitude of the position of each loudspeaker, and a sixth row 1842 describes the corresponding altitude error. The seventh row 1850 indicates which loudspeakers are used for the output format (O-2,0). The eighth row 1860 indicates which loudspeakers are used for the output format (O-5.1). Ninth line 1864 shows which loudspeakers are used for the output format (O-7.1). The tenth row 1870 indicates which loudspeakers are used for the output format O-8.1, the eleventh row 1880 indicates which loudspeakers are used for the output format O-10.1, Line 1890 shows which loudspeakers are used for output format (O-22.2). As shown, two loudspeakers are used for the output format (O-2.0), six loudspeakers are used for the output format (O-5.1), eight loudspeakers are used for the output format (O-7.1) Nine loudspeakers are used for the output format (O-8.1), 11 loudspeakers are used for the output format (O-10.1), and 24 loudspeakers are used for the output format (O-22.2).

However, one low frequency effect loudspeaker is used for the output formats (O-5.1, O-7.1, O-8,1 and O-10.1) and two low frequency effect loudspeakers (LFE1, LFE2) is used. Further, in the preferred embodiment, one rendered audio signal (e.g., one of the rendered audio signals 1582a through 1582n) is associated with each loudspeaker except one or more low-frequency effect loudspeakers Be careful. Thus, the two rendered audio signals are associated with two loudspeakers used in accordance with the O-2.0 format, the five rendered audio signals are associated with five non-low frequency effect loudspeakers if 5.1 format is used, The rendered audio signal is associated with seven non-low frequency effect loudspeakers if the O-7.1 format is used, eight rendered audio signals associated with eight non-low frequency effect loudspeakers if the 8.1 format is used, The rendered audio signal is associated with 10 non-low frequency effect loudspeakers if the O-10.1 format is used and the 22 rendered audio signals are associated with 22 non-low frequency effect loudspeakers if the O-22.2 format is used.

) 또는 매트릭스(

)에 의해 표현될 수 있는)가 존재하도록, 역상관기들의 수가 어떻게 유연하게 감소하는지가 설명될 것이다.However, it is sometimes desirable to use fewer (individual) decorrelators (decorrelator cores), as mentioned above. Below, when the O-22.2 output format is used by the multi-channel audio decoder, the 22 rendered audio signals 1582a through 1582n,

) Or matrix (

) Can be expressed by the number of decorrelators), there will be explained how the number of decorrelators decreases flexibly.

Figs. 19A-19G depict different choices for premixing the rendered audio signals 1582a-1582n under the assumption that a rendered audio signal of N = 22 is present. For example, FIG. 19A shows a table representation of the entries of the premixing matrix M _pre . Labeled 1 to 11 in Fig. 19a, the columns represent the heat of the pre-mixing matrix _(pre M) and the row labeled 1 to 22 are associated with the rows of the pre-mixing matrix _(pre M). In addition, each column of the _pre- mixing matrix M _pre is associated with one of the second set of K decorrelator input signals 1722a through 1722k of the decorrelator input signals (i.e., with the input signals of the decorrelator core) . In addition, each row of the premixing matrix M _pre is associated with one of the first set of N decorrelator input signals 1710a through 1710n of the decorrelator input signals, such that the rendered audio signals 1582a through < RTI ID = 0.0 > 1582n since the decorrelator input signals 1710a-1710n of the first set of decorrelator input signals in one embodiment are identical to the generally rendered audio signals 1582a-1582n Because). Thus, pre-mixing each row of the matrix (M _pre) is associated with a particular loudspeaker, so that a loudspeaker are related since the connection with the spatial position, with a specific spatial location. Column 1910 indicates which rows of premixing matrix M _pre are associated with which loudspeaker (and, consequently, which spatial location) (loudspeakers are defined in row 1820 of table 1800).

Below, the functionality defined by the _pre- mix ( M _pre ) of Figure 19a will be described in more detail. As shown, a second set of decorrelator input signals (denoted by " 1 " -values in the first and second rows of the first column of the _pre- mixing matrix M _pre ) CH_M_000 " and " CH_L_000 " associated with the loudspeakers (or equally speaker positions) are combined to obtain the first downmixed decorrelator input signal. Similarly, the rendered audio signals " CH_U_000 " and " CH_T_000 " associated with the speakers (or equally speaker positions) are converted into a second downmixed decorrelator input signal A second decorrelator input signal). In addition, it can be seen that the premixing matrix ( M _pre ) of Figure 19a defines 11 combinations of two rendered audio signals, each such that 11 downmixed decorrelator input signals are derived from 22 rendered audio signals have. It can also be seen that to obtain the two downmixed decorrelator input signals, the four center signals are combined (see rows 1 to 4 and columns 1 and 2 of the premixing matrix). In addition, it can be seen that the remaining downmixed decorrelator input signals are obtained by combining two audio signals, each associated with the same side of the audio scene. For example, a combination of, as represented by the third column of the pre-mixing matrix, the third downmixing the decorrelator input signal is rendered audio signal related to the azimuth position of ^{+135 o ( "CH_M_L135", "} CH_U_L135") Lt; / RTI > In addition, the fourth decorrelator input signal (represented by the fourth column of the premix matrix) is obtained by combining the rendered audio signals (" CH_M_R135 "," CH_U_R135 ") associated with an azimuthal position of -135 ^o . Thus, each downmixed decorrelator input signal is obtained by combining two rendered audio signals associated with the same (or similar) azimuthal position (or equally horizontal position) and is typically at a different altitude ≪ / RTI > vertical position).

Referring now to FIG. 19B, which shows N = 22 and K = 10 for premixing coefficients (entries of the _pre- mix matrix M _pre ), the structure of the table of FIG. 19B is the same as the structure of the table of FIG. 19A Do. As shown, however, the premixing matrix M _pre according to FIG. 19b is similar to the pre-mixing matrix M _pre according to FIG. 19b except that the first column contains four rendered audio signals CH_M_000 ", CH_L_000", CH_U_000 "and CH_T_000" ( M _pre ) in FIG. 19A in that it describes the pre-mixing matrix M _pre . In other words, four rendered audio signals associated with vertically adjacent positions are included in premixing to reduce the number of required decorrelators (ten decorrelators instead of eleven decorrelators for the matrix according to Figure 19a) do.

Now, N = 22 and K = the pre-mixing coefficient with respect to 9 when representing a (pre-mixing matrix (entries of M _pre)), see Fig. 19c, pre-mixing the matrix (M _pre) according to Figure 19c is of 9 columns . ≪ / RTI > Furthermore, the rendered audio signals (CH_M_L135 ", CH_U_L135", CH_M_R135 "and CH_U_R135") associated with the channel IDs (or positions) from the second column of the _pre- mixing matrix M _{pre of} FIG. (In a pre-mixer configured according to the pre-mix matrix of FIG. 19C) to obtain the de-correlator input signal (the de-correlator input signal of the second set of de-correlator input signals). As shown, the rendered audio signals combined into the individually downmixed decorrelator input signals by the premixing matrices according to Figures 19a and 19b are downmixed to a common downmixed decorrelator input signal according to Figure 19c . In addition, the rendered audio signals (CH_M_L135 ", CH_U_L135") having channel IDs are associated with the same horizontal positions (or azimuth positions) and spatially adjacent vertical positions (or altitudes) on the same side of the audio scene , The rendered audio signals (CH_M_R135 " and CH_U_R135 ") with channel IDs are associated with the same horizontal positions (or azimuth positions) and spatially adjacent vertical positions (or altitudes) on the second side of the audio scene . Furthermore, the rendered audio signals (CH_M_L135 ", CH_U_L135", CH_M_R135 "and CH_U_R135") with channel IDs may be associated with a horizontal pair (or even four horizontal pairs) of spatial positions including the left and right positions have. In other words, two of the four rendered audio signals, which are combined to be de-correlated using a single given decorrelator in the second column of the _pre- mixing matrix M _pre of Figure 19c, , And that two of the four rendered audio signals, which are combined to be deconvolved using a single given decorrelator, are associated with spatial locations on the right side of the audio scene. In addition, the left-hand rendered audio signals (of the four rendered audio signals) are arranged so that the four pairs of "symmetrical" are combined by premixing to be inverse-correlated using a single And spatial positions are associated with the rendered audio signals on the right (of the four rendered audio signals).

Referring to Figures 19d, 19e, 19f, and 19g, it can be seen that more rendered audio signals are combined with reduced (i.e., reduced to K) number of (individual) decorrelators. As shown in FIGS. 19A through 19G, the rendered audio signals, typically downmixed to two separate downmixed decorrelator input signals, are combined when the number of decorrelators is one as probing. In addition, such rendered audio signals associated with " symmetrical quadruple pairs " of spatial positions are combined and associated with a relatively high number of decorrelators with respect to the same or at least similar horizontal positions (or azimuthal positions) It can be seen that for a relatively low number of decorrelators, the rendered audio signals associated with spatial locations on opposite sides of the audio scene are also combined.

Referring now to Figures 20a-20d, 21a-21c, 22a-22b and 23, it should be noted that similar concepts may be applied for different numbers of rendered audio signals.

For example, Figs. 20a to 20d describe N = 10 and K entries of the premixing matrix M _pre between 2 and 5.

Similarly, FIGS. 21A through 21C describe N = 8 and K entries for the premixing matrix ( M _pre ) between 2 and 4.

Similarly, Figures 21d through 21f describe N = 7 and K entries for the premixing matrix ( M _pre ) between 2 and 4.

Figures 22A and 22B describe the entries of the premixing matrix for N = 5 and K = 2 and K = 3.

FIG. 23 describes entries of the premixing matrix for N = 2 and K = 1.

In summary, the premixing matrices according to Figs. 19-23 can be used, for example, in a switchable manner in a multi-channel decorrelator that is part of a multi-channel audio decoder. The switching between the premixing matrices depends, for example, on the required output configuration (which generally determines the number N of rendered audio signals) and also on the required complexity of the decorrelation And may be adjusted depending on, for example, the complexity information contained within the encoded representation of the audio content).

Referring to FIG. 24, the complexity reduction for the 22.2 output format will be described in greater detail. As already explained above, one solution for constructing the pre-mixing matrix and post-mixing matrix is to use the spatial information of the reproduction layout to select the channels to be mixed together and to calculate the mixing coefficients. Based on their location, the geometrically related loudspeakers (and, for example, the rendered audio signals associated with them) are grouped together taking vertical and horizontal pairs, as described in the table of FIG. In other words, Figure 24 shows the grouping of loudspeaker positions, which may be associated with rendered audio signals, in the form of a table. For example, the first column 2410 describes a first group of loudspeaker positions located at the center of the audio scene. Second column 2412 represents a second group of loudspeaker positions spatially related. The loudspeaker positions ("CH_M_L135" and "CH_U_L135") are associated with the same azimuth positions (or equally, horizontal positions) and adjacent altitude positions (or equivalently, vertically adjacent positions). Similarly, positions ("CH_M_R135" and "CH_U_R135") include the same orientation ((or equivalently, the same horizontal position) and similar height (or equivalently, vertically adjacent position) CH_M_L135 "," CH_U_L135 "," CH_M_R135 "and" CH_U_R135 "form four pairs of positions and positions (" CH_M_L135 "and" CH_U_L135 " (Or equivalently, the same horizontal position) and a similar altitude (or, equivalently, an adjacent vertical position), as shown in Fig. .

Third column 2414 represents a third group of locations. It should be noted that positions (CH_N_L030 " and CH_L_L045 ") are spatially adjacent positions and include similar azimuthal angles (or equivalently, similar horizontal positions) and similar altitudes (or equivalently, similar vertical positions). The positions of the third group of positions form four pairs of positions, and positions CH_N_L030 " and CH_L_L045 ") are also applied to positions (" CH_M_R030 " and CH_L_L045 " CH_L_R045 "), which are symmetric with respect to the center plane of the audio scene.

Fourth column 2416 represents four additional positions having four symmetrical positions with similar features when compared to the first four positions of the second row.

The fifth column 2418 represents another four pairs of symmetric positions ("CH_M_L060", "CH_U_L060", "CH_M_R060" and "CH_U_R060").

In addition, it should be noted that the rendered audio signals associated with the positions of different groups of positions may be combined more with a reduction in the number of decorrelators. For example, in the presence of eleven individual decorrelators in a multi-channel decorrelator, rendered audio signals associated with locations within the first and second rows may be combined for each group. In addition, the rendered audio signals associated with the positions represented in the third and fourth rows may be combined for each group. In addition, the rendered audio signals associated with the positions shown in the fifth and sixth rows can be combined for the second group. Thus, eleven downmixed decorrelator input signals (which may be input into individual decorrelators) can be obtained. However, if it is desired to have fewer individual inverse correlators, the rendered audio signals associated with the locations shown in rows 1 to 4 may be combined for one or more groups. Also, if it is desired to further reduce the number of individual decorrelators, the rendered audio signals associated with all positions in the second group may be combined.

In summary, the signals provided to the output layout (e.g., loudspeakers) have horizontal and vertical dependencies that must be preserved during the decorrelation process. Thus, the mixing coefficients are calculated such that the channels corresponding to the different loudspeaker groups and the corresponding channels are not mixed together.

Depending on the number of available decorrelators or the level of required decorrelators, the vertical pairs (between the middle and upper layers or between the middle and lower layers) are first mixed together in each group. Second, horizontal pairs (between left and right) or remaining vertical pairs are mixed together. For example, in group 3, the channels in the left vertical pair ("CH_M_L030" and "CH_L_L045") and the right vertical pair ("CH_M_R030" and "CH_L_R045") are mixed together, Decrease the number of inverse correlators from four to two. If it is desired to reduce a much larger number of decorrelators, the obtained horizontal pairs are downmixed to only one channel, and the number of decorrelators required for this group is reduced from four to one.

Based on the presented mixing rules, the above-mentioned tables (e.g., shown in Figures 19-23) can be used to derive different levels of required down correlation (or different levels of required inverse correlation complexity) do.

16. Second outside Renderer / Compatibility with format converters

When a spatial audio object coding decoder (or more generally, a multi-channel audio decoder) is used with an external secondary renderer / format converter, the following changes to the proposed concept (method or apparatus) can be used:

)로 설정되거나(외부 렌더러가 사용될 때) 또는 중간 렌더링 구성으로부터 유도되는 믹싱 계수들로 초기화된다(외부 포맷 컨버터가 사용될 때).The internal rendering matrix (R, for example, of the renderer) is an identity (identity,

) (When an external renderer is used) or with mixing coefficients derived from an intermediate rendering configuration (when an external format converter is used).

The number of inverse correlators is reduced using the method described in Section 15 and the number of premixing matrices M _pre is computed based on feedback information received from the renderer / format converter (e.g., M _pre = D _convert where D _convert is the downmix matrix used internally by the format _converter ). Channels to be mixed together outside the spatial audio object coding decoder are premixed and provided to the same decorrelators inside the spatial audio object coding decoder.

Using an external format converter, the spatial audio object coding inner renderer will pre-render to an intermediate configuration (e.g., the configuration with the highest number of loudspeakers).

In conclusion, in some embodiments, the information in which the output audio signals are mixed together in an external renderer or format converter allows the combination of such decorrelator input signals (in the first set of decorrelator input signals) Is used to determine the _pre- mixing matrix ( M _pre ) to define the pre-mixing matrix ( M _pre ). Thus, the information received from the external renderer / format converter (which receives the output audio signals of the multi-channel decoder) is used to select or adjust the premixing matrix (the internal rendering matrix of the multi-channel audio decoder is set to identity, The external renderer / format converter is coupled to receive the output audio signals as described above in connection with the multi-channel audio decoder.

17. Bit stream

Below, what additional signaling information may be used in the bitstream (or equivalently, the encoded representation of the audio content) will be described. In embodiments according to the present invention, the decorrelation method may be signaled into the bitstream to ensure the required quality level. In this way, the user (or audio encoder) has more flexibility in selecting the method based on the content. For this purpose, the MPEG spatial audio object coding bitstream syntax may be extended to two bits, for example, to specify two bits and / or a configuration (complexity) for specifying the method of de-correlation to be used.

Figure 25 shows the syntax representation of the bitstream elements ("bsDecorrelationMethod" and "bsDecorrelationLevel"), which may be added, for example, to the bitstream portion ("SAOCSpecificConfig ()" or "SAOC3DSpecificConfig ()". As can be seen in Fig. 25, two bits can be used for the bit stream element ("bsDecorrelationMethod"), and two bits can be used for the bit stream element ("bsDecorrelationLevel").

Figure 26 shows the relationship between the values of the bitstream variable " bsDecorrelationNethod " and the different decorrelation methods in the form of a table. For example, three different decorrelation methods may be signaled by different values of the bitstream variable. For example, an output covariance correction using inverse correlations, such as that described in section 14.3, may be signaled as one of the choices. As another option, a covariance adjustment method, for example, as described in Section 14.4.1, may be signaled. As yet another option, an energy compensation method, for example, as described in Section 14.4.2, may be signaled. Thus, three different methods for reconstructing the signal characteristics of the output audio signals based on the decoded audio signals and the decorrelated audio signals may be selected depending on the bitstream parameters.

The energy compensation mode uses the method described in Section 14.4.2. The restricted covariance adjustment mode uses the method described in Section 14.4.1, and the general covariance adjustment mode uses the method described in Section 14.3.

Referring now to Fig. 27, which illustrates how different decorrelation levels can be signaled by the bitstream variable " bsDecorrelationLevel " in the form of a table representation, a method for selecting the decorrelation complexity will now be described. In other words, the variable can be evaluated by a multi-channel audio decoder including the multi-channel decorrelators described above to determine what inverse correlation complexity is used. For example, the bitstream parameter may signal different decorrelation " levels " that may be assigned values (0, 1, 2, and 3).

One example of the decorrelation configurations (which may be specified, for example, as the de-correlation levels) is given in the table of FIG. Figure 27 shows a table representation of a number of " levels " (e.g., decorrelation levels) and a plurality of decorrelators for output configurations. In other words, Figure 27 shows the number (K) of decorrelator input signals (of the second set of decorrelator input signals) used by the multi-channel decorrelator. As can be seen in Figure 27, the number of multiple (individual) decorrelators in a multi-channel decorrelator depends on which "bitstream level" is signaled by the bitstream parameter "bsDecorrelationLevel" 11, 9, 7 and 5, respectively. Depending on the " decorrelator level " signaled by the bitstream parameter, it is selected between 10, 5, 3 and 2 individual decorrelators for a 10.1 output configuration and 8, 4, 3 or 2 Are selected among the individual inverse correlators, and are selected among 7, 4, 3, or 2 individual inverse correlators for the 7.1 configuration. In the 5.1 output configuration, there are only three valid choices, 5, 3 or 2, for the number of individual decorrelators. For the 2.1 output configuration, there is only a choice between two individual decorrelators (de-correlation level 0) and one individual de-correlator (de-correlation level 1).

In summary, the inverse correlation method can be determined at the decoder side based on the computational power and available number of decorrelators. In addition, the choice of the number of inverse correlators can be made and signaled on the encoder side using bitstream parameters.

Thus, in order to obtain output audio signals, both how the decorrelated audio signals are applied, and the complexity for the decorrelated signals, is determined by the bitstream parameters < RTI ID = 0.0 > Can be controlled from the side of the audio encoder.

18. Fields of Application for the Treatment of the Present Invention

It should be noted that the methods in which it is necessary to repeat audio signals, which is very important for human perception of an audio scene, is one of the purposes. Embodiments in accordance with the present invention improve the reconstruction accuracy of the energy level and correlation characteristics and thus increase the perceptual output quality of the final output signal. Embodiments according to the present invention may be applied for any number of downmix / upmix channels. In addition, the methods and apparatuses described herein can be combined with existing parameter source separation algorithms. Embodiments in accordance with the present invention allow to control the computational complexity of the system by setting limits on the number of decorrelator functions applied. Embodiments in accordance with the present invention can lead to simplification of object-based parameter construction algorithms such as spatial audio object coding by eliminating the MPS transcoding step.

19. Encoding / decoding environment

Hereinafter, an audio encoding / decoding environment to which the concepts according to the present invention can be applied will be described.

A three-dimensional audio codec system, in which the concepts according to the present invention may be used, is based on the MPEG-D USAC codec for coding of channel and object signals to increase the efficiency for coding a large amount of objects. MPEG-space audio object coding techniques have been applied. Three types of renderers perform rendering of objects to channels, rendering channels to headphones, or rendering channels to the above loudspeaker settings. When the object signals are explicitly transmitted or encoded with parameters using spatial audio object coding, the corresponding object metadata information is compressed and multiplexed into the three-dimensional audio stream.

Figures 28, 29 and 30 illustrate different algorithm blocks of a three-dimensional audio system.

Fig. 28 shows a schematic block diagram of such an audio encoder, and Fig. 29 shows a schematic block diagram of such an audio decoder. In other words, Figures 28 and 29 show different algorithm blocks of a three-dimensional audio system.

Some details will be described with reference to FIG. 28, which shows a schematic block diagram of a three-dimensional audio encoder 2900. The encoder 2900 receives one or more channel signals 2912 and one or more object signals 2914 and provides one or more channel signals 2916 as well as one or more object signals 2918 and 2920 based thereon. And an optional pre-renderer / mixer 2910. The audio encoder also includes a USAC encoder 2930 and optionally a spatial audio object coding encoder 2940. The spatial audio object coding encoder 2940 provides one or more spatial audio object coding transmission channels 2942 and spatial audio object coding side information 2944 based on one or more objects 2920 provided to the spatial audio object coding encoder . In addition, the USAC encoder 2930 receives channel signals 2916 including channels and pre-rendered objects from the pre-renderer / mixer 2910 and provides one or more object signals Receive spatial audio object coding transmission information 2918 and receive one or more spatial audio object coding transmission channels 2942 and spatial audio object coding side information 2944 and to provide an encoded representation 2932 based thereon. In addition, audio encoder 2900 also receives object metadata 2952 (which may be evaluated by pre-renderer / mixer 2910) to obtain encoded object metadata 2954 and encodes the object metadata And an object meta data encoder 2950 that is configured to do so. The encoded audio signal is also received by the USAC encoder 2930 and used to provide an encoded representation 2932.

Some details regarding the individual components of the audio encoder 2900 will be described below.

29, an audio decoder 3000 will be described. The audio decoder 3000 receives the encoded representation 3010 and generates, based thereon, the multi-channel loudspeaker signals 3012, the headphone signals 3014 and / or the multi-channel loudspeaker signals 3012 in an alternative format (e.g., 5.1 format) And to provide loudspeaker signals 3016. Audio decoder 3000 may generate one or more channel signal 3022, one or more pre-rendered object signals 3024, one or more object signals 3025, one or more spatial audio object coding A USAC decoder 3020 that provides a channel 3028, spatial audio object coding side information 3030 and compressed object metadata information 3032. [ Audio decoder 3000 also includes an object renderer 3040 configured to provide one or more rendered object signals 3042 based on one or more object signals 3026 and object metadata information 3044, The object meta data information 3044 is provided by the object meta data decoder 3050 based on the compressed object meta data information 3062. [ Audio decoder 3000 also receives channel signals 3022, pre-rendered object signals 3024, rendered object signals 3042 and rendered object signals 3062 and, based thereon, And a mixer 3070 configured to provide a plurality of mixed channel signals 3072, which may constitute multi-channel loudspeaker signals 3012. For example, The audio decoder 3000 also includes a binaural renderer 3080 that is configured to receive the mixed channel signals 3072 and to provide the headphone signals 3014 based thereon, . The audio decoder 3000 is configured to receive the mixed channel signals 3072 and playback layout information 3092 and to provide a loudspeaker signal 3016 for alternative loudspeaker setup 3090 < / RTI >

Some details of the components of the audio encoder 2900 and the audio decoder 3000 will be described below.

19.1. free - Renderer /mixer

The pre-renderer / mixer 2910 may optionally be used to convert the channel and object input scenes into channel scenes prior to encoding. Functionally, this may be the same as, for example, the object renderer / mixer described below.

Pre-rendering of objects can ensure, for example, deterministic signal entropy at an encoder input that is independent of the number of object signals that are essentially active at the same time.

With pre-rendering of objects, no object meta data is required.

The discrete object signals are rendered in a channel layout configured for use by the encoder, and the weights of the objects for each channel are obtained from the associated object metadata (OAM, 1952).

19.2. USAC  core coder

The core codecs 2930 and 3020 for the loudspeaker channel signals, discrete object signals, object downmix signals and pre-rendered signals are based on the MPEG-D USAC technology. It handles the decoding of multiple signals by generating channel-and object-mapping information based on the geometry and semantic information of the input channel and object allocation. This mapping information describes how input channels and objects are mapped to USAC channel elements (CPEs, SCEs, LFEs) and corresponding information is sent to the decoder.

Additional payloads such as spatial audio object coding data or object metadata have passed through the expansion elements and are considered in encoder rate control. The decoding of objects is possible in different ways, depending on the ratio / distortion requirements and the interaction requirements for the renderer. The following object coding variants are possible:

Pre-rendered objects: Object signals are pre-rendered and mixed in 22.2 channel signals prior to encoding. The following coding chain will see 22.2 channel signals.

Discrete object waveforms: objects such as those applied to the decoder as monophonic waveforms. The encoder uses single channel elements (SCEs) to transmit objects in addition to channel signals. The decoded objects are rendered and mixed on the receiver side. The compressed object metadata information is sent to the receiver / renderer together.

● Parameter object waveforms:

The object properties and their relationship to each other are described by spatial audio object coding parameters. The downmix of the object signals is coded in USAC. Parameter information is studied together. The number of downmix channels is selected depending on the number of objects and the total data rate. The compressed object metadata information is transmitted to the spatial audio object coding renderer.

19.3. Space audio Object  Coding

The spatial audio object coding encoder 2940 and spatial audio object coding decoder 3060 for object signals are based on the MPEG spatial audio object coding technique. The system regenerates multiple audio objects based on a smaller number of transmitted channels and additional parameter data (object level differences (OLDs), inter-object correlations (IOCs), downmix gains (DMGs) , Transform, and render. The additional parameter data represents a significantly lower data rate than is required for all transmitted objects, which makes decoding very efficient. The spatial audio object coding encoder takes as input the object / channel signals such as monophonic waveforms and uses parameter information (packed into three-dimensional audio bitstreams 2932 and 3010) and spatial audio object coding transport channels And encoded and transmitted). The spatial audio object coding decoder 3060 reconstructs the object / channel signals from the decoded spatial audio object coding transmission channels 3028 and parameter information 3030 and generates a spatial audio object based on the playback layout, And generates an output audio scene based on user interaction information.

19.4. Object  Metadata codec

For each object, the associated metadata specifying the geometric location and volume of the object in the three-dimensional space is efficiently coded by quantization of object properties in time and space. The compressed metadata cOAMs 2954 and 3032 are transmitted to the receiver as additional information.

19.5. Object Renderer /mixer

The object renderer uses compressed object metadata (OAM) 3044 to generate object waveforms in accordance with a given playback format. Each object is rendered on certain output channels according to its metadata. The output of this block results from the sum of the partial results.

If both the channel-based content as well as the discrete / parameter objects are decoded, the channel-based waveforms and the rendered object waveforms may be processed prior to the output of the resulting waveforms (or in a post- processor module such as a binaural renderer, Before being provided to the module.

19.6. Binaural Renderer

The binaural renderer module 3080 produces a binaural downmix of multi-channel audio data such that each input channel is represented by a virtual sound source. The processing is performed in a frame-wise manner within a right-angled symmetric filter (QMF) domain. Binauralization is based on measured binaural room impulse responses.

19.7 Loudspeakers Renderer / Format conversion

The loudspeaker renderer 3090 switches between the transmitted channel configuration and the required playback format. This is therefore referred to below as a " format converter ". The format converter performs conversions to a lower number of output channels, i. E., Generates downmixes. The system generates optimized downmix matrices for a given combination of input and output formats and applies these matrices in a downmixing process. The format converter allows for arbitrary configurations with non-standard loudspeaker locations as well as standard loudspeaker configurations.

Figure 30 shows a schematic block diagram of a format converter. In other words, Fig. 30 shows the structure of the format converter.

As shown, the format converter 3100 receives mixer output signals 3110, e.g., mixed channel signals 3072, and outputs loudspeaker signals 3112, e.g., speaker signals 3016, . The format converter includes a downmix process 3120 and a downmix constructor 3130 in a rectangular symmetric filter domain and the downmix configurator includes mixer output layout information 3032 and playout layout information 3034, And provides configuration information for the process 3020.

19.8. General Introduction

In addition, the concepts described herein, such as audio decoder 100, audio encoder 200, multi-channel decorrelator 600, multi-channel audio decoder 700, audio encoder 800 or audio decoder 1550, May be used within the audio encoder 2900 and / or the audio decoder 3000. For example, the above-mentioned audio encoders / decoders may be used as part of the spatial audio object coding encoder 2940 and / or as part of the spatial audio object coding decoder 3060. However, the above-mentioned concepts may also be used in other positions of the three-dimensional audio decoder 3000 and / or the audio encoder 2900.

Naturally, the above-mentioned methods can also be used in the concepts for the encoding and decoding of audio information according to FIGS. 28 and 29.

20. Additional embodiments

20.1 Introduction

Hereinafter, another embodiment according to the present invention will be described.

Figure 31 shows a schematic block diagram of a downmix processor, in accordance with an embodiment of the present invention.

The downmix processor 3100 includes an upmixer 3110, a renderer 3120, a combiner 3130 and a multi-channel decorrelator 3140. The renderer provides the rendered audio signals (Y _dry ) to the combiner 3130 and the multi-channel decorrelator 3140. The multi-channel decorrelator receives the rendered audio signals (which may be considered as a first set of decorrelator input signals) and based thereon a pre-mixed second set of decorrelator input signals to decorrelator core 3160 And a pre-mixer 3150 that provides the pre-mixer 3150 with a pre-mixer. The decorrelator core provides a first set of decorrelator output signals based on the second set of decorrelator input signals for use by the postmixer 3170 and the postmixer provides a decorrelator output signal (Upmix) the decorrelator output signals provided by the decorrelator core 3160 to obtain a second set of signals.

The renderer 3130 may apply, for example, a matrix R for rendering and the pre-mixer may, for example, apply a matrix M _pre for pre-mixing, and the post- , A matrix for post mixing ( M _post ), and the combiner may provide, for example, a matrix P for combining.

It should be noted that the downmix processor 3100, or the individual components or their functions, may be used within the forwarded audio decoders. In addition, it should be noted that the downmix processor may be added by any of the features and functions described herein.

20.2 Spatial audio Object  Coded 3D Processing

The hybrid filter bank described in ISO / IEC 23003-1: 2007 applies. Inverse quantization of downmix gain (DMG), object level difference information (OLD), and inter-object correlation (IOC) paramaters follows the same rules as defined in ISO / IEC 23003-2: 2010, 7.1.2.

 20.2.1 Signals and Parameters

Audio signals are defined for all time slots ( n ) and all hybrid subbands ( k ). Corresponding spatial audio object coding three-dimensional parameters are defined for all parameter timeslot ( t ) and processing band ( m ). The following mapping between the hybrid and parameter domains is specified by ISO / IEC 23003-1: 2007, Table A.31. Thus, all calculations are performed with respect to specific time / band indices and corresponding dimension numbers are displayed for each introduced variable.

The data available in the spatial audio object coding three-dimensional decoder consists of a multi-channel downmix signal X , a covariance matrix E , a rendering matrix R and a downmix matrix D.

20.2.1.1 Object Parameters

)를 표현하고 다음과 같이 오브젝트 레벨 차이 정보 및 오브젝트간 상관 파라미터들로부터 획득된다:The (N x N) covariance matrix E with elements e _{i, j} is an approximation of the original signal covariance matrix (

) And is obtained from object level difference information and inter-object correlation parameters as follows:

.

.

Thus, the dequantized object parameters are obtained as follows:

OLD _i = D _OLD ( i , l , m ), IOC _{i, j} = D _IOC ( i , j , l , m ).

20.2.1.3 Downmix Matrix

The downmix matrix D applied to the input audio signals S determines the downmix signal as X = DS . The downmix matrix D of size ( N _dmx x N ) is obtained as follows:

D = D _dmx D _premix .

The matrix ( D _dmx ) and the matrix ( D _premix ) have different sizes depending on the processing mode. The matrix ( D _dmx ) is obtained from the downmix gain parameters as follows:

Thus, the dequantized downmix parameters are obtained as follows:

DMG _{i, j} = D _DMG ( i , j , l )

20.2.1.3.1 Direct mode

In the case of direct mode, no pre-mixing is used. The matrix ( D _premix ) has a size ( N × N ) and is given by D _premix = I. The matrix ( D _dmx ) has a size ( N _dmx x N ) and is obtained from the downmix parameters according to 20.2.1.3.

20.2.1.3.2 Pre-Mixing Mode

In the case of the premixing mode, the matrix D _premix has a magnitude ( N _ch + N _premix ) and is given by:

Where the size ( N _premix x N _obj ) of the _premixing matrix A is received as input from the object renderer into a spatial audio object coding three-dimensional decoder.

The matrix ( D _dmx ) has a magnitude ( N _dmx × ( N _ch + N _premix )) and is obtained from the downmix gain parameters according to 20.2.1.3.

2.2.1.2 Rendering Matrix

The rendering matrix R applied on the input output signals S determines the target rendered output, such as Y = RS . Next, a rendering matrix R of size ( N _out x N ) is given:

R = ( R _ch R _obj )

Wherein R _ch of the size (N × N _out) is R _obj expression of the rendering matrix associated with the input channel and the size (N × N _{obj obj)} represents a rendering matrix associated with the input object.

20.2.1.4 Target Output Covariance Matrix

)를 표현하고 공분산 매트릭스(E) 및 렌더링 매트릭스(R)로부터 획득된다:The covariance matrix C of size ( N _out x N _out ) with the elements ( c _{i, j} ) is an approximation of the target output signal covariance matrix (

) And is obtained from the covariance matrix ( E ) and the rendering matrix ( R ): < EMI ID =

C = RER ^* .

20.2.2 Decoding

Space Audio Object Coding A method for obtaining an output signal using three-dimensional parameters and rendering information is described. The spatial audio object coding three-dimensional decoder is composed of, for example, a spatial audio object coding three-dimensional parameter parameterizer and a spatial audio object coding three-dimensional downmix processor.

20.2.2.1 Downmix Processor

The output signal of the downmix processor (represented within the quadrature symmetric filter domain) is provided in a corresponding synthesis filter bank as described in ISO / IEC 23003-1: 2007, which produces the final output of the spatial audio object coding three-dimensional decoder . The detailed structure of the downmix processor is shown in Fig.

)는 다음과 같이 다운믹스 신호(X) 및 역상관된 다채널 신호(X _d)로부터 계산되는데:Output signal (

Is calculated from the downmix signal X and the decorrelated multi-channel signal X _d as follows:

,

,

Where U represents the parameter non-mixing matrix and is defined in 20.2.2.1.1 and 20.2.2.1.2.

The de-correlated multi-channel signal ( X _d ) is calculated according to 20.2.3.

.

.

The mixing matrix ( P = P _dry P _wet ) is described in 20.2.3. Matrices M _pre for different output configurations are given in Figures 19-23 and matrices M _post are obtained using the following equation:

The decoding mode is controlled by the bitstream element (bsNumSaocDmxObjects) as shown in Fig.

20.2.2.1.1 Combined decoding mode

In the case of the combined decoding mode, the parameter non-mixing matrix U is given by:

U = ED ^* J.

에 의해 주어지고 여기서

이다.The matrix J of size ( N _dmx x N _dmx )

Lt; RTI ID = 0.0 >

to be.

20.2.2.1.2 Independent decoding mode

In the case of the independent decoding mode, the non-mixing matrix U is given by:

이고

이다.here

ego

to be.

Size from the channel-based covariance matrix (E _ch) and the size of the object based on the covariance matrix (E _obj) is the covariance matrix (E), by selecting only the corresponding diagonal block of (N _obj × N _obj) of the (N _ch × N _ch) Obtained:

Where the matrix ( E _{obj, ch} = ( E _{obj, ch} ) ^* ) is not required to represent and calculate the cross covariance matrix between input channels and input objects.

)의 채널 기반 다운믹스 매트릭스(D _ch) 및 크기(

)의 오브젝트 기반 다운믹스 매트릭스(D _obj)는 상응하는 대각선 블록들만을 선택함으로써 다운믹스 매트릭스(D)로부터 획득된다:size(

Channel downmix matrix ( _Dch ) and size (< RTI ID = 0.0 >

) Object-based down-mix matrix (D _obj) is obtained from the down-mix matrix (D) by selecting only the corresponding diagonal blocks:

.

.

)의 매트릭스(

)는

를 위하여 20.2.2.1.4에 따라 유도된다.size(

) Matrix

)

In accordance with 20.2.2.1.4.

)의 매트릭스(

)는

을 위하여 20.2.2.1.4에 따라 유도된다.size(

) Matrix

)

Shall be derived in accordance with 20.2.2.1.4.

20.2.2.1.4 Calculation of Matrix ( J )

)는 다음의 방정식을 사용하여 계산된다:matrix(

) Is calculated using the following equation:

Where a single vector (V) of the matrix (?) Is obtained using the following characteristic equation:

.

.

)은 다음과 같이 계산된다:The inverse of the diagonal single valued matrix [Delta]

) Is calculated as follows:

.

.

The relative regularization scalar () is determined using the absolute value of the absolute threshold ( T _reg ) and Δ as follows:

.82-3

.82-3

20.2.3 Reverse correlation

The decorrelated signals X _d are generated from the decorrelator described in 6.6.2 of ISO / IEC 23003-1: 2007, where bsDecorrConfig == 0 and the decorrelator index ( X ) It is generated according to the tables. Thus, decorrfunc () denotes the decorrelation process:

X _d = decorrfunc ( M _pre Y _dry ).

20.2.4 Mixing Matrix ( P )

The calculation of the mixing matrix (P = P _dry P _wet ) is controlled by a bit stream element (bsDecorrelationMethod). The matrix P has a size ( N _out 2 N _out ) and P _dry and P _wet both have a size N _out N _out ).

20.2.4.1 Energy Compensation Mode

The energy compensation mode uses decorrelated signals to compensate for the loss of energy in the parameter reconstruction. The mixing matrices P _dry and P _wet are given by:

P _dry = I ,

Where [lambda] _Dec = 4 is a constant used to limit the amount of the decorrelated component added to the output signals.

20.2.4.2 Restricted Covariance Adjustment Mode

믹싱 매트릭스들()은 다음이 방정식을 사용하여 정의되는데:The limited covariance adjustment mode ensures that the covariance matrix (PwetYdry) of the mixed and deconvolved signals approximates the differential covariance matrix:

The mixing matrices () are defined using the following equation:

P _dry = I ,

)이 다음과 같이 계산된다:Where the ordered inverse of the diagonal single valued matrix (Q ₂ )

) Is calculated as follows:

.

.

)는 다음과 같이 절대 임계(T _reg ) 및

의 최대 값을 사용하여 결정된다:Relative rule scalar (

) Is expressed as absolute threshold ( T _reg ) and

Is determined using the maximum value of:

The matrix (? _E ) is decomposed using single valued decomposition as follows:

)가 또한 단일 값 분해를 사용하여 표현된다:The covariance matrix of the decorrelated signals (

) Is also expressed using single valued decomposition:

20.2.4.3 General covariance adjustment mode

)가 표적 공분산 매트릭스와 근사치가 되도록 보장한다:

. 믹싱 매트릭스(P)는 다음의 방정식을 사용하여 정의되는데:A common covariance adjustment mode is a covariance matrix of the final output signals

) Approximates the target covariance matrix:

. The mixing matrix ( P ) is defined using the following equation:

)이 다음과 같이 계산된다:Where the ordered inverse of the diagonal single valued matrix (Q ₂ )

) Is calculated as follows:

.

.

)는 다음과 같이 절대 임계(T _reg ) 및

의 최대 값을 사용하여 결정된다:Relative rule scalar (

) Is expressed as absolute threshold ( T _reg ) and

Is determined using the maximum value of:

The target covariance matrix ( C ) is decomposed using single value decomposition as follows:

)가 또한 단일 값 분해를 사용하여 표현된다:The covariance matrix of the combined signals (

) Is also expressed using single valued decomposition:

The matrix H represents a prototype weighted matrix of size ( N _out × 2 N _out ) and is given by the following equation:

20.2.4.4 Introduced covariance matrices

) 사이의 차이를 표현하고 다음에 의해 주어진다:Matrix (△ _E) is the covariance matrix of the reconstructed signal to the target output covariance matrix (C) and a parameter (

) And is given by: < RTI ID = 0.0 >

)는 파라미터로 추정된 신호들의 공분산 매트릭스(

)를 표현하고 다음의 방정식을 사용하여 정의된다:matrix(

) Is the covariance matrix of the signals estimated by the parameters (

) And is defined using the following equation:

)는 역상관된 신호들의 공분산 매트릭스(

)를 표현하고 다음의 방정식을 사용하여 정의된다:matrix(

) Is the covariance matrix of the decorrelated signals < RTI ID = 0.0 > (

) And is defined using the following equation:

Considering a signal ( Y _com ) consisting of a combination of parameterally-estimated and decorrelated signals as follows:

The covariance matrix of Y _com is defined by the following equation:

21 Implementation alternatives

While some aspects have been described in the context of an apparatus, it is to be understood that these aspects also illustrate the corresponding method of the method, or block, corresponding to the features of the method steps. Similarly, the aspects described in the context of the method steps also represent the corresponding block or item or feature of the device. Some or all method steps may be performed (by use) by a hardware device, such as, for example, a microprocessor. In some embodiments, some or more of some of the most important method steps for such devices may be performed.

The encoded audio signals of the present invention may be stored on a digital storage medium or transmitted on a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.

Depending on the specific implementation requirements, embodiments of the invention may be implemented in hardware or software. An implementation may be implemented using a digital storage medium, such as a floppy disk, DVD, Blu-ray, CD, RON, PROM, EPROM, EEPROM or flash memory, having electronically readable control signals stored therein , Which cooperate (or cooperate) with the programmable computer system so that each method is executed. Thus, the digital storage medium may be computer readable.

Some embodiments in accordance with the present invention include a data carrier having electronically readable control signals that can cooperate with a programmable computer system such that any one of the methods described herein is executed.

In general, embodiments of the present invention may be implemented as a computer program product having program code, wherein the program code is operable to execute any of the methods when the computer program product is running on the computer. The program code may, for example, be stored on a machine readable carrier.

Other embodiments include a computer program for executing any of the methods described herein, stored on a machine readable carrier.

In other words, one embodiment of the method of the present invention is therefore a computer program having program code for executing any of the methods described herein when the computer program runs on a computer.

Another embodiment of the methods of the present invention is thus a data carrier (or digital storage medium, or computer readable medium) recorded therein, including a computer program for carrying out any of the methods described herein. Data carriers, digital storage media or recorded media are typically type and / or non-transient.

Another embodiment of the method of the present invention is thus a sequence of data streams or signals representing a computer program for carrying out any of the methods described herein. The data stream or sequence of signals may be configured to be transmitted, for example, over a data communication connection, e.g., the Internet.

Yet another embodiment includes processing means, e.g., a computer, or a programmable logic device, configured or adapted to execute any of the methods described herein.

Yet another embodiment includes a computer in which a computer program for executing any of the methods described herein is installed.

Yet another embodiment according to the present invention includes an apparatus or system configured to transmit (e.g., electrically or optically) a computer program to a receiver to perform one of the methods described herein. The receiver may be, for example, a computer, a mobile device, a memory device, or the like. A device or system may include, for example, a file server for delivering a computer program to a receiver.

In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to implement some or all of the methods described herein. In some embodiments, the field programmable gate array may cooperate with the microprocessor to perform any of the methods described herein. Generally, the methods are preferably executed by any hardware device.

The embodiments described herein are merely illustrative for the principles of the present invention. It will be appreciated that variations and modifications of the arrangements and details described herein will be apparent to those of ordinary skill in the art. Accordingly, it is intended that the invention not be limited to the specific details presented by way of description of the embodiments described herein, but only by the scope of the patent claims.

references

[BCC] C. Faller and F. Baumgarte, "Binaural Cue Coding - Part II: Schemes and applications," IEEE Trans. on Speech and Audio Proc., vol. 11, no. 6, Nov. 2003.

[Blauert] J. Blauert, " Spatial Hearing - The Psychophysics of Human Sound Localization ", Revised Edition, The MIT Press, London,

[JSC] C. Faller, " Parametric Joint-Coding of Audio Sources ", 120th AES Convention, Paris, 2006.

[ISS1] M. Parvaix and L. Girin: "Informed Source Separation of Underdetermined Instantaneous Stereo Mixtures Using Source Index Embedding", IEEE ICASSP, 2010.

[ISS2] M. Parvaix, L. Girin, J.-M. Brossier: " A watermarking-based method for informed source separation of audio signals with a single sensor ", IEEE Transactions on Audio, Speech and Language Processing, 2010.

[ISS3] A. Liutkus and J. Pinel and R. Badeau and L. Girin and G. Richard: "Informed source separation through spectrogram coding and data embedding", Signal Processing Journal, 2011.

[ISS4] A. Ozerov, A. Liutkus, R. Badeau, G. Richard: "Informed source separation: source coding meets source separation", IEEE Workshop on Applications of Signal Processing to Audio and Acoustics,

[ISS5] S. Zhang and L. Girin: "An Informed Source Separation System for Speech Signals", INTERSPEECH, 2011.

[ISS6] L. Girin and J. Pinel: "Informed Audio Source Separation from Compressed Linear Stereo Mixtures", AES 42nd International Conference: Semantic Audio, 2011.

[MPS] ISO / IEC, "Information technology - MPEG audio technologies - Part 1: MPEG Surround," ISO / IEC JTC1 / SC29 / WG11 (MPEG) international Standard 23003-1: 2006.

[OCD] J. Vilkamo, T. B ㅁ ckstrom, and A. Kuntz. "Optimized covariance domain framework for time-frequency processing of spatial audio", Journal of the Audio Engineering Society, 2013. in press.

[SAOC1] J. Herre, S. Disch, J. Hilpert, and O. Hellmuth: "From SAC To SAOC - Recent Developments in Parametric Coding of Spatial Audio", 22nd Regional UK AES Conference, Cambridge, UK, April 2007.

J. Engdeg? Rd, B. Resch, C. Falch, O. Hellmuth, J. Hilpert, A. H? Lzer, L. Terentiev, J. Breebaart, J. Koppens, E. Schuijers and W. Oomen : &Quot; Spatial Audio Object Coding (SAOC) - The Upcoming MPEG Standard on Parametric Object Based Audio Coding ", 124th AES Convention, Amsterdam 2008.

[SAOC] ISO / IEC, "MPEG audio technologies - Part 2: Spatial Audio Object Coding (SAOC)," ISO / IEC JTC1 / SC29 / WG11 (MPEG) International Standard 23003-2.

International Patent No. WO / 2006/026452, "MULTICHANNEL DECORRELATION IN SPATIAL AUDIO CODING" issued on 9 March 2006.

100: Multi-channel audio decoder
110: Encoded representation
112, 114: output audio signal
120: decoder
122: decoded audio signal
130: Renderer
132: Rendering parameters
134, 136: Rendered audio signal
140:
142, 144: decoded audio signal
150: coupler
200: Multi-channel audio encoder
210, 212: input audio signal
214: Encoded representation
220: Downmix signal provider
222: Downmix signal
230: Parameter provider
232: Parameter
240: an inverse correlation method parameter provider
242: Reverse correlation method parameters
500: Encoded audio representation
510: Encoded representation of the downmix signal
520: Encoded representation of the parameter
530: Encoded anticorrosion method parameter
600: Multichannel correlator
610a-610n: N-inverse correlator input signal
612a-612n ': N' decorrelator output signal
620: pre-mixer
622a-622k: K correlator input signal
630: Inverse Correlator Core
640: Post mixer
700: Multi-channel audio decoder
710: Encoded representation
712, 714: output signal
720: Multi-channel decorrelator
800: Multi-channel audio encoder
810, 812: input audio signal
814: Encoded representation of audio content
820: Downmix signal provider
822: Downmix signal
830: Parameter provider
832: Parameter
840: Inverse correlation complexity parameter provider
842: Inverse correlation complexity parameter
1012: Encoded representation
1014, 1016: output audio signal
1112, 1114: Input audio signal
1200: Encoded audio representation
1210: Encoded representation of the downmix signal
1220: Encoded representation of the parameter
1230: Encoded inverse correlation complexity parameter
1310: Encoder
1312a-1312n: an object signal
1314: Mixing parameters
1316a, 1316b: Downmix signal
1318: Additional information
1320: Mixer
1330: additional information estimator
1340: decoder
1352a-1352n: Output audio signal
1354: About user interaction
1360: Parameter object separator
1360a, 1360b: Downmix signal
1362a-1362n: Object signal
1370: Additional information processor
1372: Control information
1380: The renderer
1510: Encoder
1512a-1512n: Object signal
1514: Mixing parameter
1516a, 1516b: Downmix signal
1518: Additional information
1550: decoder
1552a-1552n: Output audio signal
1560: Parameter object separator
1570: Additional information processor
1580: The renderer
1582a-1582n: Rendered audio signal
1590: Inverse correlator
1592a-1592n: decoded audio signal
1598: Mixer
1600: Inverse correlation unit
1610a-1610n: N inverse correlator input signal
1612a-1612n: N backplane tube output signal
1620n-1620n: N individual inverse correlators
1700: Inverse correlation unit
1710a-1710n: N inverse correlator input signal
1712a-1712n: N inverse correlator output signal
1720: pre-mixer
1722a-1722k: K correlator input signal
1730: Inverse Correlator Coder
1732a to 1732k: the decorrelator output signal
1740: Post mixer
2900: 3D Audio Encoder
2910: Free-Renderer / Mixer
2912: Channel signal
2914: Object signal
2916: Channel signal
2918, 2920: Object signal
2930: USAC encoder
2932: Encoded representation
2940: Spatial audio object coding encoder
2942: Spatial audio object coding transmission channel
2944: Spatial audio object coding additional information
2950: Object Metadata Encoder
2952: Object Metadata
2954: Encoded object metadata
3000: Audio decoder
3010: Encoded representation
3012: Multi-channel loudspeaker signal
3014: Headphone signal
3016: Loudspeaker signal
3020: USAC decoder
3022: Channel signal
3024: Pre-rendered object signal
3025: Object signal
3026: Object signal
3028: Spatial audio object coding transmission channel
3030: Space audio object coding additional information
3032: compressed object metadata information
3040: Renderer
3042: rendered object signal
3044: Object meta data information
3050: object meta data decoder
3062: Compressed object metadata information
3070: Mixer
3072: Mixed channel signal
3080: Binaural Renderer
3092: Playback layout information
3090: Format conversion
3100: Downmix processor
3110: Upmixer
3120: Renderer
3130: Coupler
3140: Multi-channel decorrelator
3150: Pre-Mixer
3160: Correlator core
3170: Post mixer

Claims (52)

A plurality of decorrelated signals (142, 144; 612a-612n ';1592a-1592n; 1712a-1712n) based on a plurality of decorrelator input signals (134, 136; 610a-610n; 1582a-1582n; 1710a-1710n) A multi-channel decorrelator (140; 600; 1590; 1700) for providing a multi-
The multi-channel decorrelator includes a first set of N decorrelator input signals 134, 136, 610a-610n, 1582a-1582n, 1710a-1710n,

) To a second set of K decorrelator input signals (622a-622k; 1722a-1722k;

), Where K < N,
The multi-channel decorrelator provides a first set of K 'decorrelator output signals (632a-632k'; 1732a-1732k) based on a first set of K decorrelator input signals,
The multi-channel decorrelator upmixes the first set of K 'decorrelator output signals into a second set of N' decorrelator output signals (142, 144; 612a-612n ';1592a-1592n; 1712a-1792n) Where N '>K'
The multi-channel decorrelators are as follows:

A pre-mixing matrix ( M _pre ) is used to generate a first set of N decorrelator input signals

) To a second set of K decorrelator input signals (

), ≪ / RTI >
The multi-channel decorrelator includes a second set of K decorrelator input signals (

A first set of K 'decorrelator output signals (< RTI ID = 0.0 >

),
The multi-channel decorrelators are as follows:

A first set of K 'decorrelator output signals (" M _post "

) Into a second set ( W ) of N 'decorrelator output signals,
The multi-channel decorrelator includes a first set of N decorrelator input signals (

Channel signals are selected in dependence on the spatial positions associated with the pre-mixing matrix ( M _pre ).
The multi-channel decorrelator of claim 1, wherein K = K '.
The multi-channel decorrelator of claim 1, wherein N = N '.
2. The multi-channel decorrelator of claim 1, wherein N > = 3 and N >
The apparatus of claim 1, wherein the multi-channel decorrelator comprises a first set of N decorrelator input signals (

( M _pre ) depending on the correlation features or covariance characteristics of the multi-channel decorrelator.
The apparatus of claim 1, wherein the multi-channel decorrelator is a matrix product,

Wherein the pre-mixing matrix is determined to be in good condition with respect to the inversion operation.
2. The apparatus of claim 1, wherein the multi-channel decorrelator comprises:

And obtains the post-mixing matrix ( M _post ).
The apparatus of claim 1, wherein the multi-channel decorrelator comprises a first set of N decorrelator input signals (

Receiving information on a rendering configuration associated with the channel signal) and the multi-channel decorrelator multichannel decorrelator characterized in that in dependence on information about the rendered configuration select pre-mixing matrix (M _pre).
The apparatus of claim 1, wherein the multi-channel decorrelator comprises a first set of N decorrelator input signals associated with spatially adjacent positions of the audio scene when performing the premixing

≪ / RTI > of the plurality of channel decorrelators.
10. The apparatus of claim 9, wherein the multi-channel decorrelator includes a first set of N decorrelator input signals associated with vertically spatially adjacent locations of the audio scene when performing the premixing

≪ / RTI > of the plurality of channel decorrelators.
2. The apparatus of claim 1, wherein the multi-channel decorrelator comprises a first set of N decorrelator input signals associated with a horizontal pair of spatial positions including a left side position and a right side position

≪ / RTI > of the plurality of channel decorrelators.
2. The apparatus of claim 1, wherein the multi-channel decorrelator comprises a first set of N decorrelator input signals associated with vertically spatially contiguous positions of the audio scene when performing the premixing

≪ / RTI > of the plurality of channel decorrelators.
13. The method of claim 12 wherein at least two left side channel signals to be combined are associated with spatial positions symmetrical to spatial positions associated with at least two right side channel signals to be combined, Channel decorrelator.
The apparatus of claim 1, wherein the multi-channel decorrelator receives complexity information describing a number (K) of decorrelator input signals of a second set of decorrelator input signals, and wherein the multi-channel decorrelator dependent and is characterized in that for selecting the pre-mixing matrix _(pre M) channel decorrelator.
15. The apparatus of claim 14, wherein the multi-channel decorrelator is operative to reduce the value of the complexity information to a first value of the decorrelator input signals to obtain the decorrelator input signals of the second set of decorrelator input signals. set(

) Incrementally increases the number of decorrelator input signals of the multi-channel decorrelator.
15. The apparatus of claim 14, wherein the multi-channel decorrelator comprises a first one of the decorrelator input signals associated with vertically spatially adjacent positions on the left side of the audio scene when performing the premixing for a first value of the complexity information, set(

≪ / RTI > only the channel signals of < RTI ID =
The multi-channel decorrelator is operable to perform a pre-mixing operation on a vertically spatially adjacent location on the left side of the audio scene to obtain a given signal of the second set of decorrelator input signals when performing the premixing for a second value of the complexity information A first set of decorrelator input signals associated with < RTI ID = 0.0 >

A first set of decorrelator input signals associated with at least two of the channel signals and vertically spatially adjacent positions on the right side of the audio scene

And a channel correlation unit for combining the at least two channel signals.
15. The apparatus of claim 14, wherein the multi-channel decorrelator is operable to obtain a given signal of the second set of decorrelator input signals when performing the premixing for a second value of the complexity information, The first set (

Wherein at least two of the at least four channel signals are associated with spatial positions on the left side of the audio scene, and at least two of the at least four channel signals are associated with at least two of the at least four channel signals, And the spatial positions on the right side are related to spatial positions on the right side.
15. The apparatus of claim 14, wherein the multi-channel decorrelator is further adapted to obtain a first decorrelator input signal of a second set of decorrelator input signals, A first set of correlator input signals (

, And to obtain a second set of decorrelator input signals for a first value of the complexity information, a second set of decorrelator input signals for a first value of the complexity information, A first set of decorrelator input signals associated with spatially adjacent locations

, ≪ / RTI >
The multi-channel decorrelator is operable to obtain a second decorrelator input signal of a second set of decorrelator input signals for a second value of the complexity information, the plurality of vertically spatially adjacent positions A first set of decorrelator input signals associated with the decorrelator < RTI ID = 0.0 >

A first set of decorrelator input signals associated with the at least two channel signals and vertically spatially adjacent locations on the right side of the audio scene

) Of the at least two channel signals,
Wherein the number of decorrelator input signals of the second set of decorrelator input signals is greater for a first value of the complexity information than for a second value of the complexity information.
A multi-channel audio decoder (100; 1550) for providing at least two output audio signals (112,114; 1552a-1552n) based on an encoded representation (110; 1516a, 1516b, 1518)
Characterized in that the multi-channel audio decoder comprises a multi-channel decorrelator (140; 600; 1590; 1700) according to any one of claims 1 to 18.
20. The method of claim 19,
Wherein the multi-channel audio decoder is operative to generate a plurality of rendered audio signals (134, 136; 1582a-1582n) based on the encoded representation, To render the decoded audio signal 122 (1562a-1562n)
The multi-channel audio decoder derives one or more decorrelated audio signals (142, 144; 1592a-1592n) from the rendered audio signals using the multi-channel decorrelator, A second set of decorrelator output signals constituting the decorrelated audio signals,
Wherein the multi-channel audio decoder combines the rendered audio signals or a scaled version thereof with the one or more decorrelated audio signals to obtain the output audio signals.
20. The method of claim 19 wherein the multi-channel audio decoder is a channel, characterized in that for selecting the pre-mixing matrix (M _pre) for use by a said multi-channel decorrelator, depending on the control information included in the encoded representation Audio decoder.
20. The method of claim 19 wherein the multi-channel audio decoder premixing matrix for use by the multi-channel decorrelator, depending on the output configuration for describing the allocation of a spatial location of the audio scene by the output audio signal (M _pre Channel audio decoder.
20. The method of claim 19 wherein the multi-channel audio decoder has three or more different pre-mixing matrix for use by the control information, the multi-channel decorrelator depending on included in the encoded representation for a given output configuration (M _pre ), Wherein each of the three or more different pre-mixing matrices is associated with a different number of signals of the second set of decorrelator input signals.
20. The apparatus of claim 19, wherein the multi-channel audio decoder is configured for use by the multi-channel decorrelator in dependence on a mixing matrix (Dconv, Drender) used by a format converter or renderer that receives the at least two output audio signals ( M _pre ) is selected based on the _pre- mixing matrix ( M _pre ).
25. The apparatus of claim 24, wherein the multi-channel audio decoder is configured for use by the multi-channel decorrelator to be the same as the mixing matrix (Dconv, Drender) used by the format converter or renderer receiving the at least two output audio signals ( M _pre ) is selected based on the _pre- mixing matrix ( M _pre ).
A multi-channel audio encoder (800) for providing an encoded representation (814) based on at least two input audio signals (810; 812)
The multi-channel audio encoder provides one or more downmix signals (822) based on the at least two input audio signals,
The multi-channel audio encoder provides one or more parameters (832) describing a relationship between the at least two input audio signals,
The multi-channel audio encoder provides an inverse correlation complexity parameter 842 that describes the complexity of the decorrelator to be used on the side of the audio decoder,
Wherein the decorrelated complexity parameter determines a number K of decorrelators used in a multi-channel decorrelator that premixes a first set of N de-correlated input signals to a second set of K de-correlated input signals,
Wherein the decorrelation complexity parameter determines the selection of a premixing matrix used to premix a first set of N de-correlated input signals to a second set of K de-correlated input signals in a multi-channel decorrelator. Encoder.
A method (900) for providing a plurality of decorrelated signals based on a plurality of decorrelator input signals,
Pre-mixing (910) a first set of N de-correlator input signals into a second set of K de-correlator input signals, wherein K < N -;
Providing (920) a first set of K 'decorrelator output signals based on the first set of K decorrelator input signals; And
Upmixing the first set of K 'decorrelator output signals into a second set of N' decorrelator output signals (930), wherein N '>K'-;
A first set of N decorrelator input signals ( ) Is as follows:

A pre-mixing matrix ( M _pre ) is used to generate a second set of K decorrelator input signals

Lt; / RTI >
The first set of K 'decorrelator output signals (

≪ / RTI > is a second set of K decorrelator input signals (

), ≪ / RTI >
The first set of K 'decorrelator output signals (

) Is as follows:

Is upmixed into a second set ( W ) of N 'decorrelator output signals using a postmixing matrix ( M _post )
The pre-mix matrix ( M _pre ) may comprise a first set of N decorrelator input signals (

≪ / RTI > are selected in dependence on the associated spatial positions. ≪ Desc / Clms Page number 13 >
A method (1000) for providing at least two output audio signals based on an encoded representation,
The method includes providing (1020) a plurality of decorrelated signals based on a plurality of decorrelator input signals according to claim 27.
A method (1100) for providing an encoded representation based on at least two input audio signals,
Providing (1110) at least one downmix signal based on the at least two input audio signals;
Providing (1120) at least one parameter describing a relationship between at least two input audio signals; And
And providing (1130) an inverse correlation complexity parameter (842) that describes the complexity of the decorrelator to be used on the side of the audio decoder,
Wherein the decorrelated complexity parameter determines a number K of decorrelators used in a multi-channel decorrelator that premixes a first set of N de-correlated input signals to a second set of K de-correlated input signals,
Wherein the decorrelated complexity parameter determines the selection of a premixing matrix used to premix a first set of N de-correlated input signals to a second set of K de-correlated input signals in a multi-channel decorrelator. / RTI >
28. A computer program stored on a computer readable recording medium for executing the method of claim 27, 28 or 29 when the computer program is run on a program.
13. A digital storage medium storing an encoded audio representation (1200)
An encoded representation (1210) of the downmix signal;
An encoded representation (1220) of one or more parameters describing a relationship between at least two input audio signals; And
An encoded decorrelation complexity parameter describing the complexity of the decorrelation to be used on the side of the audio decoder,
Wherein the decorrelated complexity parameter determines a number K of decorrelators used in a multi-channel decorrelator that premixes a first set of N de-correlated input signals to a second set of K de-correlated input signals,
Wherein the decorrelated complexity parameter determines the selection of a premixing matrix used to premix a first set of N de-correlated input signals to a second set of K de-correlated input signals in a multi-channel decorrelator. A digital storage medium storing representations.
A plurality of decorrelated signals (142, 144; 612a-612n ';1592a-1592n; 1712a-1712n) based on a plurality of decorrelator input signals (134, 136; 610a-610n; 1582a-1582n; 1710a-1710n) A multi-channel decorrelator (140; 600; 1590; 1700) for providing a multi-
The multi-channel decorrelator includes a first set of N decorrelator input signals 134, 136, 610a-610n, 1582a-1582n, 1710a-1710n,

) To a second set of K decorrelator input signals (622a-622k; 1722a-1722k;

), Where K < N,
The multi-channel decorrelator provides a first set of K 'decorrelator output signals (632a-632k'; 1732a-1732k) based on a first set of K decorrelator input signals,
The multi-channel decorrelator upmixes the first set of K 'decorrelator output signals into a second set of N' decorrelator output signals (142, 144; 612a-612n ';1592a-1592n; 1712a-1792n) Where N '>K'
The multi-channel decorrelators are as follows:

A pre-mixing matrix ( M _pre ) is used to generate a first set of N decorrelator input signals

) To a second set of K decorrelator input signals (

), ≪ / RTI >
The multi-channel decorrelator includes a second set of K decorrelator input signals (

A first set of K 'decorrelator output signals (< RTI ID = 0.0 >

),
The multi-channel decorrelators are as follows:

A first set of K 'decorrelator output signals (" M _post "

) Into a second set ( W ) of N 'decorrelator output signals,
The multi-channel decorrelator includes a first set of N decorrelator input signals (

Channel pre-mixer matrix ( M _pre ) depending on the correlation features or the covariance characteristics of the channel signals of the multi-channel decorrelator.
A plurality of decorrelated signals (142, 144; 612a-612n ';1592a-1592n; 1712a-1712n) based on a plurality of decorrelator input signals (134, 136; 610a-610n; 1582a-1582n; 1710a-1710n) A multi-channel decorrelator (140; 600; 1590; 1700) for providing a multi-
The multi-channel decorrelator includes a first set of N decorrelator input signals 134, 136, 610a-610n, 1582a-1582n, 1710a-1710n,

) To a second set of K decorrelator input signals (622a-622k; 1722a-1722k;

), Where K < N,
The multi-channel decorrelator provides a first set of K 'decorrelator output signals (632a-632k'; 1732a-1732k) based on a first set of K decorrelator input signals,
The multi-channel decorrelator upmixes the first set of K 'decorrelator output signals into a second set of N' decorrelator output signals (142, 144; 612a-612n ';1592a-1592n; 1712a-1792n) Where N '>K'
The multi-channel decorrelators are as follows:

A pre-mixing matrix ( M _pre ) is used to generate a first set of N decorrelator input signals

) To a second set of K decorrelator input signals (

), ≪ / RTI >
The multi-channel decorrelator includes a second set of K decorrelator input signals (

A first set of K 'decorrelator output signals (< RTI ID = 0.0 >

),
The multi-channel decorrelators are as follows:

A first set of K 'decorrelator output signals (" M _post "

) Into a second set ( W ) of N 'decorrelator output signals,
The multi-channel decorrelators are as follows:

And obtains the post-mixing matrix ( M _post ).
A plurality of decorrelated signals (142, 144; 612a-612n ';1592a-1592n; 1712a-1712n) based on a plurality of decorrelator input signals (134, 136; 610a-610n; 1582a-1582n; 1710a-1710n) A multi-channel decorrelator (140; 600; 1590; 1700) for providing a multi-
The multi-channel decorrelator includes a first set of N decorrelator input signals 134, 136, 610a-610n, 1582a-1582n, 1710a-1710n,

) To a second set of K decorrelator input signals (622a-622k; 1722a-1722k;

), Where K < N,
The multi-channel decorrelator provides a first set of K 'decorrelator output signals (632a-632k'; 1732a-1732k) based on a first set of K decorrelator input signals,
The multi-channel decorrelator upmixes the first set of K 'decorrelator output signals into a second set of N' decorrelator output signals (142, 144; 612a-612n ';1592a-1592n; 1712a-1792n) Where N '>K'
The multi-channel decorrelator includes a first set of N decorrelator input signals (

) To receive information for rendering the configuration related to the channel signal, the channel decorrelator is a multi-channel station, characterized in that, depending on the basis of the information on the rendering configuration select pre-mixing matrix (M _pre) Correlator.
delete delete A plurality of decorrelated signals (142, 144; 612a-612n ';1592a-1592n; 1712a-1712n) based on a plurality of decorrelator input signals (134, 136; 610a-610n; 1582a-1582n; 1710a-1710n) A multi-channel decorrelator (140; 600; 1590; 1700) for providing a multi-
The multi-channel decorrelator includes a first set of N decorrelator input signals 134, 136, 610a-610n, 1582a-1582n, 1710a-1710n,

) To a second set of K decorrelator input signals (622a-622k; 1722a-1722k;

), Where K < N,
The multi-channel decorrelator provides a first set of K 'decorrelator output signals (632a-632k'; 1732a-1732k) based on a first set of K decorrelator input signals,
The multi-channel decorrelator upmixes the first set of K 'decorrelator output signals into a second set of N' decorrelator output signals (142, 144; 612a-612n ';1592a-1592n; 1712a-1792n) Where N '>K'
The multi-channel decorrelator includes a first set of N decorrelator input signals (

), Wherein at least two of the at least four channel signals are associated with spatial locations on the left side of the audio scene, and at least two of the at least four channel signals are associated with at least two of the at least four channel signals, Channel spatial correlators are associated with spatial locations on the right-hand side.
A plurality of decorrelated signals (142, 144; 612a-612n ';1592a-1592n; 1712a-1712n) based on a plurality of decorrelator input signals (134, 136; 610a-610n; 1582a-1582n; 1710a-1710n) A multi-channel decorrelator (140; 600; 1590; 1700) for providing a multi-
The multi-channel decorrelator includes a first set of N decorrelator input signals 134, 136, 610a-610n, 1582a-1582n, 1710a-1710n,

) To a second set of K decorrelator input signals (622a-622k; 1722a-1722k;

), Where K < N,
The multi-channel decorrelator provides a first set of K 'decorrelator output signals (632a-632k'; 1732a-1732k) based on a first set of K decorrelator input signals,
The multi-channel decorrelator upmixes the first set of K 'decorrelator output signals into a second set of N' decorrelator output signals (142, 144; 612a-612n ';1592a-1592n; 1712a-1792n) Where N '>K'
The multi-channel decorrelator receives complexity information describing a number K of decorrelator input signals of the second set of decorrelator input signals, wherein the multi-channel decorrelator is operable to generate a premixing matrix M _pre ) is selected.
A multi-channel audio decoder (100; 1550) for providing at least two output audio signals (112,114; 1552a-1552n) based on an encoded representation (110; 1516a, 1516b, 1518)
The multi-channel audio decoder includes a plurality of decorrelated signals (142, 144; 612a-612n '; 1592a) based on a plurality of decorrelator input signals (134, 136; 610a-610n; 1582a-1582n; 1710a-1710n) A multi-channel decorrelator (140; 600; 1590; 1700) for providing a multi-channel decorrelator (1712a-1712n;
The multi-channel decorrelator includes a first set of N decorrelator input signals 134, 136, 610a-610n, 1582a-1582n, 1710a-1710n,

) To a second set of K decorrelator input signals (622a-622k; 1722a-1722k;

), Where K < N,
The multi-channel decorrelator provides a first set of K 'decorrelator output signals (632a-632k'; 1732a-1732k) based on a first set of K decorrelator input signals,
The multi-channel decorrelator upmixes the first set of K 'decorrelator output signals into a second set of N' decorrelator output signals (142, 144; 612a-612n ';1592a-1592n; 1712a-1792n) Where N '>K'
Channel audio decoder selects a pre-mix matrix ( M _pre ) for use by the multi-channel decorrelator depending on an output configuration that describes the assignment of the output audio signals to spatial positions of the audio scene Channel audio decoder.
A multi-channel audio decoder (100; 1550) for providing at least two output audio signals (112,114; 1552a-1552n) based on an encoded representation (110; 1516a, 1516b, 1518)
The multi-channel audio decoder includes a plurality of decorrelated signals (142, 144; 612a-612n '; 1592a) based on a plurality of decorrelator input signals (134, 136; 610a-610n; 1582a-1582n; 1710a-1710n) A multi-channel decorrelator (140; 600; 1590; 1700) for providing a multi-channel decorrelator (1712a-1712n;
The multi-channel decorrelator includes a first set of N decorrelator input signals 134, 136, 610a-610n, 1582a-1582n, 1710a-1710n,

) To a second set of K decorrelator input signals (622a-622k; 1722a-1722k;

), Where K < N,
The multi-channel decorrelator provides a first set of K 'decorrelator output signals (632a-632k'; 1732a-1732k) based on a first set of K decorrelator input signals,
The multi-channel decorrelator upmixes the first set of K 'decorrelator output signals into a second set of N' decorrelator output signals (142, 144; 612a-612n ';1592a-1592n; 1712a-1792n) Where N '>K'
The multi-channel audio decoder selects between three or more different pre-mixing matrices ( M _pre ) for use by the multi-channel decorrelator depending on the control information contained in the encoded representation for a given output configuration, Wherein each of the three or more different pre-mixing matrices is associated with a different number of signals of the second set of decorrelator input signals.
A multi-channel audio decoder (100; 1550) for providing at least two output audio signals (112,114; 1552a-1552n) based on an encoded representation (110; 1516a, 1516b, 1518)
The multi-channel audio decoder includes a plurality of decorrelated signals (142, 144; 612a-612n '; 1592a) based on a plurality of decorrelator input signals (134, 136; 610a-610n; 1582a-1582n; 1710a-1710n) A multi-channel decorrelator (140; 600; 1590; 1700) for providing a multi-channel decorrelator (1712a-1712n;
The multi-channel decorrelator includes a first set of N decorrelator input signals 134, 136, 610a-610n, 1582a-1582n, 1710a-1710n,

) To a second set of K decorrelator input signals (622a-622k; 1722a-1722k;

), Where K < N,
The multi-channel decorrelator provides a first set of K 'decorrelator output signals (632a-632k'; 1732a-1732k) based on a first set of K decorrelator input signals,
The multi-channel decorrelator upmixes the first set of K 'decorrelator output signals into a second set of N' decorrelator output signals (142, 144; 612a-612n ';1592a-1592n; 1712a-1792n) Where N '>K'
The multi-channel audio decoder renders a pre-mixing matrix ( M _{pre) for} use by the multi-channel decorrelator depending on a mixing matrix (Dconv, Drender) used by a format converter or renderer that receives the at least two output audio signals. Channel audio decoder.
A method (900) for providing a plurality of decorrelated signals based on a plurality of decorrelator input signals,
Pre-mixing (910) a first set of N de-correlator input signals into a second set of K de-correlator input signals, wherein K < N -;
Providing (920) a first set of K 'decorrelator output signals based on the first set of K decorrelator input signals; And
Upmixing the first set of K 'decorrelator output signals into a second set of N' decorrelator output signals (930), wherein N '>K'-;
A first set of N decorrelator input signals (

) Is as follows:

A pre-mixing matrix ( M _pre ) is used to generate a second set of K decorrelator input signals

Lt; / RTI >
The first set of K 'decorrelator output signals (

≪ / RTI > is a second set of K decorrelator input signals (

), ≪ / RTI >
The first set of K 'decorrelator output signals (

) Is as follows:

Is upmixed into a second set ( W ) of N 'decorrelator output signals using a postmixing matrix ( M _post )
The pre-mix matrix ( M _pre ) may comprise a first set of N decorrelator input signals (

≪ / RTI > is selected depending on the crosstalk characteristics or crossover characteristics of the channel signals of the channel.
A method (900) for providing a plurality of decorrelated signals based on a plurality of decorrelator input signals,
Pre-mixing (910) a first set of N de-correlator input signals into a second set of K de-correlator input signals, wherein K < N -;
Providing (920) a first set of K 'decorrelator output signals based on the first set of K decorrelator input signals; And
Upmixing the first set of K 'decorrelator output signals into a second set of N' decorrelator output signals (930), wherein N '>K'-;
A first set of N decorrelator input signals (

) Is as follows:

A pre-mixing matrix ( M _pre ) is used to generate a second set of K decorrelator input signals

Lt; / RTI >
The first set of K 'decorrelator output signals (

≪ / RTI > is a second set of K decorrelator input signals (

), ≪ / RTI >
The first set of K 'decorrelator output signals (

) Is as follows:

Is upmixed into a second set ( W ) of N 'decorrelator output signals using a postmixing matrix ( M _post )
The post-mixing matrix ( M _post ) is as follows:

≪ / RTI > wherein the first and second signals are obtained.
A method (900) for providing a plurality of decorrelated signals based on a plurality of decorrelator input signals,
Pre-mixing (910) a first set of N de-correlator input signals into a second set of K de-correlator input signals, wherein K < N -;
Providing (920) a first set of K 'decorrelator output signals based on the first set of K decorrelator input signals; And
Upmixing the first set of K 'decorrelator output signals into a second set of N' decorrelator output signals (930), wherein N '>K'-;
The method includes receiving a first set of N decorrelator input signals (

Receiving a plurality of de-correlated signals, characterized in that the _pre- mixing matrix ( M _pre ) is selected depending on the information about the rendering configuration, Methods for providing.
delete delete A method (900) for providing a plurality of decorrelated signals based on a plurality of decorrelator input signals,
Pre-mixing (910) a first set of N de-correlator input signals into a second set of K de-correlator input signals, wherein K < N -;
Providing (920) a first set of K 'decorrelator output signals based on the first set of K decorrelator input signals; And
Upmixing the first set of K 'decorrelator output signals into a second set of N' decorrelator output signals (930), wherein N '>K'-;
A first set of N decorrelator input signals (

Wherein at least two of the at least four channel signals are associated with spatial locations on the left side of the audio scene and at least two of the at least four channel signals are associated with at least two of the at least four channel signals, Wherein the spatial positions of the right-side spatial locations are related to spatial positions on the right-hand side.
A method (900) for providing a plurality of decorrelated signals based on a plurality of decorrelator input signals,
Pre-mixing (910) a first set of N de-correlator input signals into a second set of K de-correlator input signals, wherein K < N -;
Providing (920) a first set of K 'decorrelator output signals based on the first set of K decorrelator input signals; And
Upmixing the first set of K 'decorrelator output signals into a second set of N' decorrelator output signals (930), wherein N '>K'-;
The method includes the decorrelator input claim comprising the step of receiving the complexity information describing the number (K) of the decorrelator input signal of the second set, and depending on the complexity of the information pre-mixing matrix (M _pre) is selected of the signals ≪ / RTI > wherein the plurality of decorrelated signals comprise a plurality of decorrelated signals.
A method (1000) for providing at least two output audio signals based on an encoded representation,
The method includes providing (1020) a plurality of decorrelated signals based on a plurality of decorrelator input signals,
Wherein providing a plurality of decorrelated signals based on the plurality of decorrelator input signals comprises:
Pre-mixing (910) a first set of N de-correlator input signals into a second set of K de-correlator input signals, wherein K < N -;
Providing (920) a first set of K 'decorrelator output signals based on the first set of K decorrelator input signals; And
Upmixing the first set of K 'decorrelator output signals into a second set of N' decorrelator output signals (930), wherein N '>K'-;
Characterized in that a _pre- mix matrix ( M _pre ) is selected for use by a multi-channel decorrelator depending on an output configuration that describes the assignment of the output audio signals to spatial positions of an audio scene. / RTI >
A method (1000) for providing at least two output audio signals based on an encoded representation,
The method includes providing (1020) a plurality of decorrelated signals based on a plurality of decorrelator input signals,
Wherein providing a plurality of decorrelated signals based on the plurality of decorrelator input signals comprises:
Pre-mixing (910) a first set of N de-correlator input signals into a second set of K de-correlator input signals, wherein K < N -;
Providing (920) a first set of K 'decorrelator output signals based on the first set of K decorrelator input signals; And
Upmixing the first set of K 'decorrelator output signals into a second set of N' decorrelator output signals (930), wherein N '>K'-;
The method includes selecting between three or more different pre-mixing matrices ( M _pre ) for use by a multi-channel decorrelator depending on control information contained in the encoded representation for a given output configuration, Wherein each of the at least three different pre-mixing matrices is associated with a different number of signals of the second set of decorrelator input signals.
A method (1000) for providing at least two output audio signals based on an encoded representation,
The method includes providing (1020) a plurality of decorrelated signals based on a plurality of decorrelator input signals,
Wherein providing a plurality of decorrelated signals based on the plurality of decorrelator input signals comprises:
Pre-mixing (910) a first set of N de-correlator input signals into a second set of K de-correlator input signals, wherein K < N -;
Providing (920) a first set of K 'decorrelator output signals based on the first set of K decorrelator input signals; And
Upmixing the first set of K 'decorrelator output signals into a second set of N' decorrelator output signals (930), wherein N '>K'-;
Characterized in that a pre-mixing matrix ( M _pre ) is selected for use by a multi-channel decorrelator depending on a mixing matrix (Dconv, Drender) used by a format converter or renderer receiving said at least two output audio signals / RTI > of at least two output audio signals.
A computer program stored on a computer readable recording medium for executing the method of any one of claims 42 to 44 and 47 to 51 when the computer program is run on a program.

Images