BR112017015930B1 - APPARATUS AND METHOD FOR PROCESSING A CODED AUDIO SIGNAL - Google Patents

Abstract

APPARATUS AND METHOD FOR PROCESSING A CODED AUDIO SIGNAL. It is an apparatus (1) for processing a coded audio signal (100) comprising a plurality of downmix signals (101) associated with a plurality of input audio objects (111) and object parameters ( AND). The apparatus (1) comprises a grouper (2) configured to group the reduction mixing signals (101) into groups of reduction mixing signals (102) associated with a set of input audio objects (111). The apparatus (1) comprises a processor (3) configured to perform at least one individual processing step on the object parameters (Ek) of each set of input audio objects (111) in order to provide group results (103, 104). Furthermore, there is a combiner (4) configured to combine said group results (103, 104) or processed group results to provide a decoded audio signal (110). The grouper (2) is configured to group the reduction mix signals (101) so that each input audio object (111) belongs to only one set of input audio objects (111). The invention also relates to a corresponding method.

Description

[001] DESCRIPTIVE REPORT

[002] The invention relates to an apparatus and a method for processing a coded audio signal.

[003] Recently, parametric techniques for effective bitrate transmission/storage of audio situations containing multiple audio objects have been proposed in the field of audio code conversion (see references [BCC, JSC, SAOC, SAOC1, SAOC2] below) and informed source separation (see, for example, references [ISS1, ISS2, ISS3, ISS4, ISS5, ISS6] below).

[004] These techniques aim to reconstruct a desired output audio situation or desired audio source objects based on additional side information describing the signals and/or source objects transmitted/stored in the audio situation. This reconstruction occurs at the decoder using a parametric informed source separation scheme.

[005] Unfortunately, it has been found that in some cases parametric separation schemes can result in severe audible articles that cause an unsatisfactory listening experience.

[006] Therefore, an object of the invention is to improve the audio quality of decoded audio signals using parametric code conversion techniques.

[007] The objective is achieved by an apparatus according to claim 1 and a corresponding method according to claim 22.

[008] The objective is achieved by an apparatus for processing a coded audio signal. The encoded audio signal comprises a plurality of downmix signals associated with a plurality of input audio objects and object parameters (E). The apparatus comprises a grouper, a processor and a combiner.

[009] The grouper is configured to group the plurality of reduction mix signals into a plurality of groups of reduction mix signals. Each group of downmix signals is associated with a set of input audio objects (or input audio signals) among the plurality of input audio objects. In other words: groups cover subsets of the set of input audio signals represented by the encoded audio signal. Each group of down mix signals is also associated with some of the E object parameters that describe the input audio objects. In the following, the individual groups Gk are identified with an index k with 1 < k < K with K as the number of groups of downmix signals.

[010] Furthermore, the processor - after grouping - is configured to perform at least one individual processing step on the object parameters of each set of input audio objects. In this way, at least one processing step is not carried out simultaneously on all object parameters, but individually on the object parameters that belong to the respective group of downmix signals. In one embodiment, only one step is performed individually. In a different embodiment, more than one step is performed, while in an alternative embodiment, the entire processing is performed individually on the groups in the reduction mix signals. The processor provides batch results for the individual groups.

[011] In a different embodiment, the processor - after grouping - is configured to perform at least one individual processing step on each group among the plurality of groups of downmix signals. In this way, at least one processing step is not carried out simultaneously on all reduction mix signals, but individually on the respective groups of reduction mix signals.

[012] Eventually, the combiner is configured to combine the group results or processed group results to provide a decoded audio signal. In this way, the group results or the results of additional processing steps performed on the group results are combined to provide a decoded audio signal. The decoded audio signal corresponds to the plurality of input audio objects that are encoded by the encoded audio signal.

[013] The grouping performed by the grouper is performed at least under the restriction that each input audio object among the plurality of input audio objects belongs to only or exactly one set of input audio objects. This implies that each input audio object belongs to only one group of downmix signals. This also implies that each reduction mix signal belongs to only one reduction mix signal group.

[014] According to one embodiment, the grouper is configured to group the plurality of reduction mix signals into the plurality of groups of reduction mix signals such that each input audio object of each set of input audio objects input is free from a signed relationship in the encoded audio signal with other input audio objects or has a signed relationship in the encoded audio signal only with at least one input audio object that belongs to the same set of input audio objects . This implies that no input audio object has a signed relationship to an input audio object that belongs to a different downmix signal group. Such a signaled relationship is, in one embodiment, that two input audio objects are stereo signals arising from a single source.

[015] The apparatus of the invention processes a coded audio signal comprising downmix signals. Down mixing is a part of the process of encoding a given number of individual audio signals and implies that a given number of input audio objects are combined into a down mixing signal. The number of input audio objects is thereby reduced to a smaller number of downmix signals. Because of this, the downmix signals are associated with a plurality of input audio objects.

[016] The reduction mix signals are grouped into reduction mix signal groups and are subjected individually - that is, as single groups - to at least one processing step. In this way, the device performs at least one processing step not jointly on all downmix signals, but individually on the individual groups of downmix signals. In a different embodiment, the object parameters of the groups are treated separately in order to obtain the matrices to be applied to the encoded audio signal.

[017] In one embodiment, the apparatus is a decoder of coded audio signals. The apparatus is, in an alternative embodiment, a part of a decoder.

[018] In one embodiment, each reduction mixing signal is assigned to a group of reduction mixing signals and is consequently processed individually with respect to at least one processing step. In this embodiment, the number of groups of down mix signals is equal to the number of down mix signals. This implies that grouping and individual processing coincide.

[019] In one embodiment, combining is one of the final steps of processing the encoded audio signal. In a different embodiment, the group results are further subjected to different processing steps that are performed individually or together with the group results.

[020] Grouping (or detection of groups) and individual treatment of groups have been shown to result in an improvement in audio quality. This especially concerns, for example, parametric code conversion techniques.

[021] According to one embodiment, the grouper of the apparatus is configured to group the plurality of downmix signals into the plurality of groups of downmix signals while minimizing a diversity of downmix signals within each group of signals. reduction mixer. In this mode, the device tries to reduce the number of reduction mix signals that belong to each group. In one case, only one reduction mix signal belongs to at least one group of reduction mix signals.

[022] According to one embodiment, the grouper is configured to group said plurality of down mix signals into said plurality of groups of down mix signals such that only a single down mix signal belongs to a group of reduction mix signals. In other words: Grouping results in several groups of reduction mix signals in which at least one reduction mix signal group is determined, to which only one reduction mix signal belongs. In this way, at least one group of reduction mix signals refers to only a single reduction mix signal. In a further embodiment, the number of groups of downmix signals to which only one downmix signal belongs is maximized.

[023] In one embodiment, the grouper of the apparatus is configured to group the plurality of downmix signals into the plurality of groups of downmix signals based on information within the encoded audio signal. In a further embodiment, the apparatus uses only information within the encoded audio signal to group the downmix signals. With the use of information within the bitstream of the encoded audio signal comprises - in one embodiment - taking correlation or covariance information into account. The grouper, in particular, extracts information about the relationship between different input audio objects from the encoded audio signal.

[024] In one embodiment, the grouper is configured to group said plurality of downmix signals into said plurality of groups of downmix signals based on bsRelatedTo values within said encoded audio signal. Regarding these values, see, for example, the document in WO 2011 /039195 A1.

[025] According to one embodiment, the grouper is configured to group the plurality of down mix signals into the plurality of groups of down mix signals by applying at least the following steps (to each group of down mix signals) reduction):

[026] • detect the possibility of a reduction mix signal being assigned to an existing group of reduction mix signals;

[027] • detect the possibility that at least one input audio object among the plurality of input audio objects associated with the downmix signal is part of a set of input audio objects associated with an existing group of signals reduction mixing;

[028] • assign the reduction mix signal to a new group of reduction mix signals in the event that the reduction mix signal is exempt from an assignment to an existing group of reduction mix signals (thus the down mix signal is not readily assigned to a group) and in the event that all input audio objects among the plurality of input audio objects associated with the down mix signal are exempt from an association with an existing group of reduction mix signals (in this way, the input audio objects of the reduction mix signal are not readily - via a different reduction mix signal - assigned to a group); It is

[029] • combine the reduction mix signal with an existing group of reduction mix signals in the case where the reduction mix signal is assigned to the existing group of reduction mix signal or, in the case of at least one object of input audio objects from among the plurality of input audio objects associated with the down mixing signal be associated with the existing group of down mixing signals.

[030] If a signaled relationship in the encoded audio signal is also taken into consideration, then another detect step will be added, resulting in an additional requirement for assigning and combining the reduction mix signals.

[031] According to one embodiment, the processor is configured to perform several individual processing steps on the object parameters (Ek) of each set of input audio objects (or each group of downmix signals) in order to of providing individual matrices as group results. The combiner is configured to combine the individual matrices to provide said decoded audio signal. The object parameters (Ek) belong to the input audio objects of the respective group of reduction mix signals with index k and are processed to obtain individual matrices for this group having index k.

[032] According to the different embodiment, the processor is configured to perform several individual processing steps on each group of said plurality of groups of downmix signals in order to provide output audio signals as group results. The combiner is configured to combine the output audio signals to provide said decoded audio signal.

[033] In this embodiment, groups of reduction mixing signals are processed so that output audio signals are obtained, which correspond to input audio objects that belong to the respective group of reduction mixing signals. Therefore, combining the output audio signals with the decoded audio signals is close to the final steps of the decoding processes performed on the encoded audio signal. In this embodiment, each group of downmix signals is individually subjected to all processing steps after detecting the groups of downmix signals.

[034] In a different embodiment, the processor is configured to perform at least one individual processing step on each group of said plurality of groups of downmix signals in order to provide processed signals as group results. The apparatus further comprises a post-processor configured to jointly process said processed signals in order to provide output audio signals. The combiner is configured to combine the output audio signals as processed group results in order to provide said decoded audio signal.

[035] In this embodiment, the reduction mixing signal groups are subjected to at least one individual processing step and at least one processing step together with other groups. Individual processing results in processed signals that - in one embodiment - are processed together.

[036] Referring to matrices, in one embodiment, the processor is configured to perform at least one individual processing step on the object parameters (Ek) of each set of input audio objects in order to provide individual matrices. A post-processor comprised by the apparatus is configured to jointly process object parameters to provide at least one general matrix. The combiner is configured to combine said individual matrices and said at least one general matrix. In one embodiment, the post-processors perform at least one processing step together on the individual matrices in order to obtain at least one overall matrix.

[037] The following embodiments refer to processing steps performed by the processor. Some of these steps are also suitable for the postprocessor mentioned in the previous embodiment.

[038] In one embodiment, the processor comprises a demixer configured to demix the downmix signals of respective groups of said plurality of downmix signal groups. By demixing the downmix signals the processor obtains representations of the original input audio objects that were subjected to downmixing in the downmix signal.

[039] According to one embodiment, the demixer is configured to demix the downmix signals of respective groups of said plurality of downmix signal groups based on a Minimum Mean Square Error (MMSE) algorithm. This algorithm will be explained in the following description.

[040] In a different embodiment in which the processor comprises a demixer configured to process the object parameters of each set of input audio objects individually to provide individual demix matrices.

[041] In one embodiment, the processor comprises a calculator configured to compute, individually, for each group of downmix signals, matrices with sizes that depend on at least one of a variety of input audio objects from the set of objects of input audio associated with the respective down mix signal group and a variety of down mix signals that belong to the respective down mix signal group. Since the reduction mix signal groups are smaller than the entire set of reduction mix signals, and since the reduction mix signal groups refer to smaller numbers of input audio signals, the matrices used for the processing of the reduction mixing signal groups are smaller than those used in the prior art. This makes computing easier.

[042] According to one embodiment, the calculator is configured to compute for the individual demixing matrices an individual threshold based on a maximum energy value within the respective group of downmixing signals.

[043] According to one embodiment, the processor is configured to compute an individual threshold based on a maximum energy value within the respective group of downmix signals for each group of downmix signals, individually.

[044] In one embodiment, the calculator is configured to compute for a regularization step to demix the downmix signals from each group of downmix signals an individual threshold based on a maximum energy value within the respective group of reduction mix signals. The thresholds for the groups of downmix signals are computed in a different modality by the demixer itself.

[045] The following discussion will show the interesting effect of computing the threshold for groups (one threshold for each group) and not for all downmix signals.

[046] According to one embodiment, the processor comprises a renderer configured to render the demixed downmix signals of the respective groups to an output situation of said decoded audio signal in order to provide rendered signals. Rendering is based on input provided by the listener or based on data about the actual output situation.

[047] In one embodiment, the processor comprises a renderer configured to process object parameters to provide at least one rendering matrix.

[048] The processor comprises in one embodiment a post-mixer configured to process object parameters to provide at least one decorrelation matrix.

[049] According to one embodiment, the processor comprises a post-mixer configured to perform at least one decorrelation step on said rendered signals and configured to combine results (Ycompressed(wet)) of the decorrelation step performed with said respective signals. rendered (Yuncompressed(dry)).

[050] According to one embodiment, the processor is configured to determine an individual reduction mixing matrix (Dk) for each group of reduction mixing signals (where k is the index of the respective group), the processor is configured To determine an individual group covariance matrix (Ek) for each group of downmix signals, the processor is configured to determine an individual group downmix covariance matrix (Δk) for each group of downmix signals. reduction based on the individual reduction mixing matrix (Dk) and the individual group covariance matrix (Ek), and the processor is configured to determine an individual regularized inverse group matrix (Jk) for each group of mixing signals. reduction.

[051] According to one embodiment, the combiner is configured to combine the individual regularized inverse group matrix (Jk) to obtain an overall regularized inverse group matrix (J).

[052] According to one embodiment, the processor is configured to determine an individual group parametric demix matrix (Uk) for each group of down mix signals based on the individual down mix matrix (Dk) in the matrix of individual group covariance (Ek) and the individual regularized inverse group matrix (Jk), and the combiner is configured to combine an individual group parametric unmixing matrix (Uk) to obtain an overall group parametric unmixing matrix ( U).

[053] According to one embodiment, the processor is configured to determine an individual group parametric demix matrix (Uk) for each group of down mix signals based on the individual down mix matrix (Dk) in the matrix of individual group covariance (Ek) and the individual regularized inverse group matrix (Jk), and the combiner is configured to combine the individual group parametric unmixing matrix (Uk) to obtain an overall group parametric unmixing matrix (U ).

[054] According to one embodiment, the processor is configured to determine an individual group rendering matrix (Rk) for each group of downmix signals.

[055] According to one embodiment, the processor is configured to determine an individual upmix matrix (RkUk) for each group of downmix signals based on the individual group rendering matrix (Rk) and the individual group parametric demixing (Uk), and the combiner is configured to combine the individual boost mixing matrices (RkUk) to obtain an overall boost mixing matrix (RU).

[056] According to one embodiment, the processor is configured to determine an individual group covariance matrix (Ck) for each group of downmix signals based on the individual group rendering matrix (Rk) and the individual group covariance (Ek), and the combiner is configured to combine the individual group covariance matrices (Ck) to obtain an overall group covariance matrix (C).

[057] According to one embodiment, the processor is configured to determine an individual group covariance matrix of the parametrically estimated signal (Eyuncompressed(dry))k based on the individual group rendering matrix (Rk), in the individual group parametric unmixing (Uk), into the individual downmixing matrix (Dk) and the individual group covariance matrix (Ek), and the combiner is configured to combine the individual group covariance matrices of the parametrically estimated signal ( Eyuncompressed(dry))k to obtain a general parametrically estimated signal Eyuncompressed(dry) .

[058] According to one embodiment, the processor is configured to determine the regularized inverse matrix (J) based on a singular value decomposition of a reduction mixing covariance matrix (EDMX).

[059] According to one embodiment, the processor is configured to determine a submatrix (Δk) for a determination of a parametric demixing matrix (U), selecting elements (Δ (m, n)) that correspond to the signals of reduction mix (m, n) assigned to the respective group (which has index k) of reduction mix signals. Each group of reduction mix signals covers a specific number of reduction mix signals and a set of associated input audio objects and is denoted here by an index k.

[060] According to this embodiment, individual submatrices (Δk) are obtained by selecting or collecting the elements of the reduction mixing covariance matrix Δ that belong to the respective group k.

[061] In one embodiment, individual submatrices (Δk) are individually inverted and the results are combined into the regularized inverse matrix (J).

[062] In a different embodiment, the submatrix (Δk) is obtained using its definition as Δk = DkEkDk* with the individual reduction mixing matrix (Dk)

[063] According to one embodiment, the combiner is configured to determine the post-mix matrix (P) based on the matrices determined individually for each group of reduction mix signals and the combiner is configured to apply the post matrix -mixing (P) to reduce the plurality of mixing signals in order to obtain the decoded audio signal. In this embodiment, from object parameters a post-mix matrix is computed, which is applied to the encoded audio signal in order to obtain the decoded audio signal.

[064] According to one embodiment, the apparatus and its respective components are configured to perform for each group of reduction mixing signals, individually, at least one of the following computations:

[065] • computation of group covariance matrix Ek of size Nk times Nk with the elements:

[066] • computing group reduction mixing covariance matrix Δk of size Mk times

[067] • computation of singular value decomposition of group reduction mixing covariance matrix

[068] • computation of regularized inverse group matrix approximation including computing the individual matrix Ainv k (details will be given below),

[069] • computation of the parametric unmixing matrix group Uk of size Nk times Mk:

[070] • multiplication of the group rendering matrix Rk of size NK times increase mixing with the unmixing matrix Uk of size Nk times

[071] • computation of the group covariance matrix Ck of size Noutput times Noutput:

[072] • computation of the group covariance of the parametrically estimated signal (Eynot compressed(dry))k of size Noutput times Noutput:

[073] In this regard, k denotes a group index of the respective group of downmix signals, Nk denotes the number of input audio objects of the associated set of input audio objects, Mk denotes the number of signals downmix signal that belongs to the respective downmix signal group, and Noutput denotes the number of output channels upmixed or rendered.

[074] The computed matrices are smaller in size than those used in the prior art. Consequently, in one embodiment as many processing steps as possible are performed individually on the reduction mix signal groups.

[075] The object of the invention is also achieved by a corresponding method for processing a coded audio signal. The encoded audio signal comprises a plurality of downmix signals associated with a plurality of input audio objects and object parameters. The method comprises the following steps:

[076] • grouping the reduction mixing signals into a plurality of groups of reduction mixing signals associated with a set of input audio objects among the plurality of input audio objects,

[077] • perform at least one individual processing step on the object parameters of each set of input audio objects in order to provide group results, and

[078] • combining said results into a group in order to provide a decoded audio signal.

[079] Grouping is performed at least with the restriction that each input audio object among the plurality of input audio objects belongs to only one set of input audio objects.

[080] The above-mentioned modalities of the apparatus can also be carried out by the method steps and corresponding modalities of the method. Therefore, the explanations given for the modalities of the device are also maintained for the method.

[081] The invention will be explained below in relation to the attached drawings and the embodiments depicted in the attached drawings, in which:

[082] Figure 1 shows an overview of a parametric down mixing/up mixing concept based on MMSE,

[083] Figure 2 shows a parametric reconstruction system with decorrelation applied to rendered output,

[084] Figure 3 shows a structure of a reduction mixing processor,

[085] Figure 4 shows spectrograms of five input audio objects (left-hand column) and spectrograms of the corresponding reduction mix channels (right-hand column),

[086] Figure 5 shows spectrograms of reference output signals (left side column) and spectrograms of the corresponding decoded and 3D rendered SAOC output signals (right side column),

[087] Figure 6 shows spectrograms of the 3D SAOC output signals using the invention,

[088] Figure 7 shows a frame parameter processing according to the prior art,

[089] Figure 8 shows a frame parameter processing according to the invention,

[090] Figure 9 shows an example of an implementation of a group detection function,

[091] Figure 10 schematically shows an apparatus for encoding input audio objects,

[092] Figure 11 schematically shows an example of an apparatus of the invention for processing a coded audio signal,

[093] Figure 12 schematically shows a different example of an apparatus of the invention for processing a coded audio signal,

[094] Figure 13 shows a sequence of steps of an embodiment of the method of the invention,

[095] Figure 14 schematically shows an example of an apparatus of the invention,

[096] Figure 15 schematically shows an additional example of an apparatus,

[097] Figure 16 schematically shows a processor of a device of the invention, and

[098] Figure 17 schematically shows the application of an apparatus of the invention.

[099] In the following, an overview of parametric separation schemes will be provided, using the example of MPEG's Spatial Audio Object-Oriented Coding (SAOC) technology ([SAOC]) and SAOC's 3D processing part of MPEG-H 3D Audio ([SAOC3D, SAOC3D2]). The mathematical properties of these methods are considered.

[100] The following mathematical notation is used:

[101] N number of input audio objects (alternatively: input objects)

[102] Ndmx number of downmix (transport) channels

[103] Noutput number of boost mix channels (rendered)

[104] Samples number of samples per audio signal

[105] D reduction mixing matrix, size Ndmx times N

[106] S input audio object signal, size N times NaSamples

[107] And object covariance matrix, size N times N, approximation of

[108] X down mix audio signals, size Ndmx times Na samples, defined as

[109] EDMX covariance matrix of reduction mixing signals, size Ndmx times Ndmx,

[110] defined as

[111] U parametric font estimation matrix, size N times Ndmx,

[112] which approaches

[113] R render matrix (specified on the decoder side), size Noutput times N^.,

[114] S parametrically reconstructed object signals, size N times Na samples,

[115] which approaches S and is defined as

[116] Uncompressed (dry) object signals reconstructed and rendered parametrically,

[117] size Noutput times Nasamples, defined as Yuncompressed(dry) = RUX

[118] Ycompressed(wet) decorrelator outputs, size Noutput times Nasamples

[119] Y final output, size Noutput times Nasamples

[120] (•)* self-adjoint operator (Hermitian),

[121] which represents the conjugate transposition of (•)

[122] F decorrelator function (• ) decorrelator

[123] Without loss of generality, in order to improve the readability of the equations, for all introduced variables the indices denoting time and frequency dependence are omitted.

[124] Parametric object separation systems:

[125] Parametric separation schemes aim to estimate a number of audio sources from the signal mix (reduction mix) with the use of auxiliary parametric information. The typical solution for this task is based on the application of Minimum Mean Square Error (MMSE) estimation algorithms. SAOC technology is an example of such parametric audio code conversion systems.

[126] Figure 1 depicts the general principle of the SAOC encoder/decoder architecture.

[127] General parametric downmix/boost mix processing is performed in a time/frequency selective manner and can be described as a sequence of the following steps:

[128] • The “encoder” is provided with input “audio objects” S and “mixing parameters” D. The “mixer” subjects the “audio objects” S to downmixing into a number of “mixing signals reduction”

[129] • The “side information estimator” extracts side information that describes characteristics of the input “audio objects” S (e.g., covariance properties).

[130] • “Reduction mixing signals” X and side information are transmitted or stored. These downmix audio signals can be further compressed using converters into audio code (such as MPEG-1/2 Layer II or III, MPEG-2/4 Advanced Audio Coding (AAC), MPEG Unified Speech and Audio (USAC), etc.). Side information can also be represented and encoded effectively (for example, as coded relations of object powers and object correlation coefficients).

[131] The “decoder” restores the original “audio objects” from the “reduction mix signals” decoded using the transmitted side information (this information provides the object parameters). The “secondary information processor” estimates the demixing coefficients to be applied to the “reduction mix signals” within the “parametric object separator” to obtain the parametric object reconstruction of S. The reconstructed “audio objects” are rendered for a target situation (multiple channels), represented by output channels Y, applying “rendering parameters” R.

[132] The same principle and sequence steps are applied in 3D SAOC processing, which incorporates an additional decorrelation trajectory.

[133] Figure 2 provides an overview of the concept of parametric step-down mixing/boost mixing with integrated decorrelation trajectory.

[134] Using the example of the SAOC 3D technique, part of MPEG-H 3D Audio, the main processing steps of such a parametric separation system can be summarized as follows:

[135] The SAOC 3D decoder produces the modified rendered output Y as a mixture of the reconstructed and parametrically rendered signal (uncompressed (dry) signal) Yuncompressed (dry) and its uncorrelated version (compressed (wet) signal) Ycompressed ( wet).

[136] For the discussion of the relevant invention - the processing steps can be differentiated as illustrated in Figure 3:

[137] • Unmix, which parametrically reconstructs input audio objects using U-matrix,

[138] • Render using rendering information (R matrix),

[139] • Uncorrelate,

[140] • Post-mix using matrix P, computed based on information contained in the bit stream.

[141] Parametric object separation is achieved from the reduction mix signal X using the demix matrix U based on ê additional side information:

[142] The R rendering information is used to obtain the dry compressed signal as:

[143] The final output signal Y is computed from the signals Yuncompressed(dry) and Ycompressed(wet) as

[144] The mixing matrix P is computed, for example, based on rendering information, correlation information, energy information, covariance information, etc.

[145] In the invention, this will be the post-mix matrix applied to the encoded audio signal in order to obtain the decoded audio signal.

[146] In the following, the common parametric object separation operation using MMSE will be explained.

[147] The demixing matrix U is obtained based on information derived from variables contained in the bitstream (e.g., the demixing matrix D and the covariance information E), which use the Quadratic Error estimation algorithm Minimum Average (MMSE): U = ED*J.

[148] The matrix J of size Ndmx times Ndmx represents an approximation of the pseudoinverse of the reduction mixing covariance matrix to the

[149] The computation of matrix J is derived according to:

[150] in which the matrices V and A are determined using the singular value decomposition (SVD) of the EDMX matrix as:

[151] It should be noted that similar results can be obtained using different decomposition methods such as: eigenvalue decomposition, Schur decomposition, etc.

[152] The regularized inverse operation (• )inv, used for the diagonal singular value matrix A, can be determined, for example, as done in SAOC 3D, with the use of a truncation of the singular values relative to the largest value singular:

[153] In a different embodiment, the following formula is used:

[154] The relative regularization scalar is determined using absolute threshold Treg and maximum value of Λ as: with Treg = 10-2, for example.

[155] Depending on the singular values, Ai,i can be restricted to positive values only (if λi,i < 0, then λi,i = abs(λi,i) and the sign (λi,i) is multiplied with the corresponding left or right singular vector) or negative values may be allowed.

[156] In the second case with negative values of Ai, relative regularization scalar is computed as:

[157] For simplicity, in the following the second definition of Treg will be used.

[158] Similar results can be obtained using truncation of singular values relative to an absolute value or other regularization methods used for matrix inversion.

[159] Inverting very small singular values can result in very high demixing coefficients and consequently high amplifications of the corresponding downmix channels. In such a case, channels with very low energy levels can be amplified using high gains and this can result in audible articles. In order to reduce this effect.... . TA ~ . undesirable, singular values smaller than the relative threshold reg are truncated to zero.

[160] Now, the disadvantages found in the prior art parametric object separation technique are explained.

[161] The described prior art parametric object separation methods specify the use of regularized inversion of the reduction mixing covariance matrix in order to avoid separation articles. However, for some real use cases in mixing situations, harmful articles caused by too aggressive smoothing have been identified in the system output.

[162] In the following, an example of such a situation is constructed and analyzed.

[163] An N = 5 number of input audio objects (S) are encoded using the described technique (more precisely, the MPEG-H 3D Audio SAOC 3D processing part method) into a number Ndmx = 3 reduction mix channels (X).

[164] The example input audio objects might consist of:

[165] • a group of two correlated audio objects that contain musical accompaniment signals (Left Side and Right Side of a stereo pair),

[166] • a group of an independent audio object that contains a speech signal, and

[167] • a group of two correlated audio objects that contain a piano recording (Left Side and Right Side of a stereo pair).

[168] Input signals are downmixed into three groups of transport channels:

[169] • group G1 with M1 = 1 down mix channels, which contains the first group of objects,

[170] • group G2 with M2 = 1 down mix channels, which contains the second group of objects, and

[171] • group G3 with M3 = 1 reduction mix channels, which contains the third group of objects,

[172] so that Ndmx = M1 + M2 + M3.

[173] The reduction mixing matrices Dk corresponding to each group Gk, for k = 1, 2, 3, are constructed using unity mixing gains, and the full reduction mixing matrix D is given by:

[174] One can observe the absence of cross-mixing between the group of first two object signals, the third object signal, and the group of last two object signals. It is also noted that the third object signal containing speech is mixed alone in a down mixing channel. Therefore, a good reconstruction of this object is expected and, consequently, also a good rendering. The spectrograms of the input signals and the reduction mixing signal obtained are illustrated in Figure 4.

[175] The possible downmix signal core code conversion used in a real system is omitted here to better highlight the undesired effect. On the decoder side, SAOC 3D parametric decoding is used to reconstruct and render the audio object signals to a 3-channel configuration (Noutput = 3): The Left (L), Center (C), and Right channels ( R).

[176] A simple remix of the example input audio objects is used in the following:

[177] • the first two audio objects (the musical accompaniment) are muted (that is, rendered with a gain of 0),

[178] • the third input object (speech) is rendered to the center channel, and

[179] • object 4 is rendered to the left channel and object 5 to the right channel.

[180] Consequently, the rendering matrix used is given by:

[181] The reference output can be computed by applying the specified render matrix directly to the input signals: Yref = RS.

[182] Spectrograms of the reference output and the SAOC 3D decoding and rendering output signals are illustrated by the two columns of Figure 5.

[183] From the spectrograms shown of the SAOC 3D decoder output, the following observations can be noted:

[184] • The central channel containing only the speech signal is severely damaged compared to the reference signal. Large spectral holes can be observed. These spectral holes (which are regions of time and frequency with missing energy) result in bass audible articles.

[185] • Small spectral gaps are also present in the left and right channels, especially in the low-frequency regions where most of the signal energy is concentrated. These spectral gaps also result in audible articles.

[186] • There is no cross-mixing of groups of objects on the down mix channels, that is, objects downmixed on one down mix channel are not present on any other down mix channel. The second down mix channel contains only one object (speech); therefore, spectral gaps in the system output may be generated simply due to the fact that it is processed in conjunction with the other down mix channels.

[187] Based on the aforementioned observations, it can be concluded that:

[188] • The SAOC 3D system is not a “through” system, that is, if an input signal is mixed alone on a down mix channel, the audio quality of that input signal must be preserved upon decoding and rendering.

[189] • The 3D system of SAOC can introduce audible articles due to multi-channel downmix signal processing. The output quality of objects contained in a group of down mix channels depends on the processing of the rest of the down mix channels.

[190] Spectral gaps, especially those in the center channel, indicate that some useful information contained in the down mix channels is discarded by processing. This loss of information can be traced back to the step of separating the parametric object, more precisely, to the regularization step of the reduction mixing covariance matrix inversion.

[191] By definition, the reduction mixing matrix in the example has a block-diagonal structure:

[192] Furthermore, due to the specified relationship between input objects (e.g. signaling parametric correlations) also the input object signal covariance matrix available in the decoder has a block-diagonal structure:

[193] As a consequence, the reduction mixing covariance matrix can be represented in a block-diagonal form:

[194] In this case, the EDMX matrix is already in the block-diagonal, however, for the general case, its block-diagonal form can be obtained after permuting rows/columns using the permutation operator

[195] A permutation operator Φ is defined as a matrix obtained by permuting the rows of an identity matrix. If a symmetric matrix A can be represented in block-diagonal form by permuting rows and columns, the permutation operator can be used to express the resulting matrix à as:

[196] If Φ is a permutation operator, then the following properties are true:

[197] • first, if V is a unitary matrix, then, is also a unitary matrix, and

[198] • second, with the identity matrix I.

[199] As a consequence, permutation operators are transparent to singular value decomposition algorithms. This means that the original matrix A and the permuted matrix à share the same singular values and permuted singular vectors:

[200] Due to the block-diagonal representation, the singular values of the EDMX matrix can be computed by applying the SVD to the EDMX matrix or by applying the SVD to the EDMXk block diagonal submatrices and combining the results:

[201] Since the singular values of the downmix covariance matrix are directly related to the power levels of the downmix channels (which are described by the main diagonal of the EDMX matrix):

[202] and objects contained in one channel are not contained in any other down mix channel, it can be concluded that each singular value corresponds to a down mix channel.

[203] Therefore, if one of the reduction mix channels has a much lower energy level than the rest of the reduction mix channels, the singular value corresponding to that channel will be much lower than the rest of the singular values.

[204] The truncate step used in inverting the matrix containing the singular values of the EDMX matrix:

[205] can result in truncation of singular values corresponding to the reduction mix channel with the small energy level (relative to the reduction mix channel with the highest energy). Due to this, the information present in this relatively low energy reduction mix channel is discarded and the spectral gaps observed in the Spectrogram and Audio Output Figures are generated.

[206] For a better understanding, it was considered that the downmixing of input audio objects occurs for each sample and for each frequency band separately. Especially the separation into different bands helps to understand why gaps can be seen in the spectrograms of output signals at different frequencies.

[207] The identified problem can be isolated to the fact that the relative regularization threshold is computed for singular values without considering that the matrix to be inverted is block-diagonal:

[208] Each block-diagonal matrix corresponds to an independent group of downmix channels. Truncation is performed with respect to the largest singular value, however, this value describes only a group of channels. In this way, the reconstruction of objects contained in all independent groups of downmix channels becomes dependent on the group containing the largest singular value.

[209] In the following, the invention will be explained based on the embodiment discussed above with respect to the prior art:

[210] Considering the example described above, the three covariance matrices can be associated with the three different groups of Gk reduction mixing channels with 1 < k < 3. The audio objects or input audio objects contained in the mixing channels of each group are not contained in any other group. Additionally, no relationship (e.g., correlation) is noted between objects contained in reduction mix channels of different groups.

[211] In order to solve the identified problem of the parametric reconstruction system, the method of the invention proposes to apply the regularization step independently for each group. This implies that three different thresholds are computed for the inversion of the three independent Teg = max (abs (Ai)) T: reduction mixing covariance matrices: , where 1 ≤ k ≤ 3. Thus, in the invention, in one embodiment such threshold is computed for each group separately and not, as in the prior art, a general threshold for the respective bands and frequency samples.

[212] The inversion of singular values is, consequently, obtained by applying regularization independently to the EDMXk submatrices, with 1 ≤ k ≤ 3:

[213] In a different embodiment, the following formula is used:

[214] With the use of the method of the proposed invention in an otherwise identical 3D SAOC system to the example discussed in the previous section, the audio output quality of the decoded and rendered output is improved. The resulting signals are illustrated in Figure 6.

[215] Comparing the spectrograms in the right column of Figure 5 and Figure 6, it can be seen that the method of the invention solves the problems identified in the existing prior art parametric separation system. The method of the invention guarantees the “through” capability of the system, and, more importantly, spectral gaps are removed.

[216] The described solution for processing three independent groups of downmix channels can be easily generalized to any number of groups.

[217] The method of the invention proposes to modify the parametric object separation technique by using the grouping information in the inversion of the reduction mixing signal covariance matrix. This results in significant improvement in audio output quality.

[218] Grouping can be obtained, for example, from mixing and/or correlation information already available in the decoder without additional signaling.

[219] More precisely, a group is defined in an embodiment by the smallest set of downmix signals with the following two properties in this example:

[220] • First, the input audio objects contained in these down mix channels are not contained in any other down mix channel.

[221] • Second, any input signals contained in the down mix channels of a group are unrelated (e.g., no intercorrelation is signaled within the encoded audio signal) to any input signals contained in the down mix channels. reduction mix from any other group. Such an intercorrelation implies a combined treatment of the respective audio objects during decoding.

[222] Based on the introduced group definition, a number of groups K (1 < K < Ndmx) can be defined: Gk (1 < k < K) and the EDMX reduction mixing covariance matrix can be expressed with the use of a block-diagonal form applying a permutation operator Φ:

[223] The EDMXk submatrices are constructed by selecting elements from the reduction mixing covariance matrix corresponding to the independent groups Gk. For each group Gk, the EDMXk matrix of size Mk times Mk is expressed using SVD as: EDMXk = V k A k V k*

[224] The pseudoinverse of the matrix EDMXk is computed as (EDMXk)-1 = V k Ainv k V k* where the regularized inverse matrix Ainv k is given in one embodiment by:

[225] and in a different modality by:

[226] The relative regularization scalar is determined using absolute threshold Treg and maximum value of Λ as: with Treg = 10-2, for example.

[227] The inverse of the EDMX reduction mixing covariance permuted matrix is obtained as:

[228] and the inverse of the reduction mixing covariance matrix is computed by applying the inverse permutation operation:

[229] Additionally, the method of the invention proposes, in one embodiment, to determine the groups entirely based on the information contained in the bit stream. For example, this information may be provided by reduction mixing information and correlation information.

[230] More precisely, a Gk group is defined by the smallest set of downmix channels with the following properties:

[231] • Input audio objects contained in Gk group down mix channels are not contained in any other down mix channels. An input audio object is not contained in a down mix channel, for example, if the corresponding down mix gain is provided by the lowest quantization index or if it is equal to zero.

[232] • All input signals i contained in the reduction mix channels of group Gk are unrelated to any input signal j contained in any reduction mix channels of any other group. For example (compare e.g. document in WO 2011/039195 A1), the bitstream variable bsRelatedTo[i][j] can be used to signal whether two objects are related (bsRelatedTo[i][j] == 1) or if they are not related (bsRelatedTo[i][j] == 0). Furthermore, different methods of signaling that two objects are related can be used based on correlation or covariance information, for example.

[233] Groups may be determined once per frame or once per parameter set for all processing bands, or once per frame or once per parameter set for each processing band.

[234] The method of the invention also allows, in one embodiment, to significantly reduce the computational complexity of the parametric separation system (e.g., SAOC 3D decoder) by using the grouping information in the parametric processing components with greater computational expense .

[235] Therefore, the method of the invention proposes to remove computations that do not result in any contribution to the final output audio quality. These computations can be selected based on the clustering information.

[236] More precisely, the method of the invention proposes to compute all parametric processing steps independently for each predetermined group and combine the results at the end.

[237] Using the example 3D processing part of MPEG-H 3D Audio SAOC, computationally complex operations are provided by:

[238] • computation of covariance matrix E of size N times N with the elements:

[239] • computation of reduction mixing signal covariance matrix Δ of size Ndmx times Ndmx:

[240] • matrix singular value decomposition computation

[241] • computation of regularized inverse matrix approximation

[242] • computation of parametric demixing matrix U of size N times Ndmx:

[243] • multiplication of the rendering matrix R of size Noutput times N with the demixing matrix U of size N times Ndmx: RU,

[244] computation of covariance matrix C of size Nout times Nout:

[245] • computation of the covariance of the parametrically estimated signal Eyuncompressed(dry) of size Noutput times Noutput:

[246] Object Level Differences (OLD) refers to the relative energy of an object to the object with the most energy for a given time and frequency band and Inter-Object Cross Coherence (IOC) describes the amount of similarity, or cross-correlation , for two objects in a given time and frequency band.

[247] The method of the invention proposes to reduce computational complexity by computing all parametric processing steps for all predetermined K groups Gk with 1 < k < K, independently, and combining the results at the end of parameter processing.

[248] A group Gk contains Mk reduction mix channels and Nk input audio objects such that:

[249] For each Gk group, a reduction mixing matrix group is defined as Dk by selecting reduction mixing matrix elements D corresponding to reduction mixing channels and input audio objects contained by the Gk group.

[250] Similarly, a rendering matrix group Rk is obtained from the rendering matrix R by selecting the rows corresponding to input audio objects contained by the group Gk.

[251] Similarly, an OLDk group vector and an IOCk group matrix are obtained from the OLD vector and the IOC matrix by selecting the elements corresponding to input audio objects contained by the Gk group.

[252] For each Gk group, the described processing steps are replaced by less computational processing steps as follows:

[253] • computing group covariance matrix Ek of size Nk times Nk with the elements:

[254] • computation of group reduction mixing covariance matrix Δk of size Mk times Mk:

[255] • computing singular value decomposition of group reduction mixing covariance matrix

[256] • computation of regularized inverse group matrix approximation

[257] • computation of the parametric unmixing matrix group Uk of size Nk times Mk:

[258] • multiplication of the group rendering matrix Rk of size NMixing boost times Nk with the demixing matrix Uk of size Nk times Mk: RkUk,

[259] • computation of the group covariance matrix Ck of size Nout times Nout:

[260] • computation of the group covariance of the parametrically estimated signal (Eyuncompressed(dry))k of size Noutput times Noutput:

[261] And the results of individual group processing steps are combined at the end:

[262] • the RU boost mixing matrix of size Nout times Ndmx is obtained by merging the RkUk group matrices:

[263] • the covariance matrix C of size Noutput times Noutput is obtained by summing the group matrices Ck:

[264] • the covariance of the parametrically estimated signal Eyuncompressed(dry) of size Noutput times Noutput is obtained by summing group matrices (Eyuncompressed(dry))k:

[265] Summarizing the processing steps, according to the structure of the reduction mixing processor illustrated in Figure 3, while omitting the decorrelation step, the frame parameter processing of the existing prior art can be depicted as in Figure 7.

[266] With the use of the proposed invention method, the computation complexity is reduced using group detection as illustrated in Figure 8.

[267] An example of an implementation of a group detection function, called: [K,Gk]= groupDetect(D,RelatedTo) , is provided in Figure 9 using ANSI C code and the static function “getSaocCoreGroups( )”.

[268] The proposed method of the invention proves, computationally significantly, much more effective than performing the operations without grouping. It also allows better allocation and use of memory, supports computational parallelization, reduces the accumulation of error numbers, etc.

[269] The proposed method of the invention and the proposed apparatus of the invention solve an existing problem of prior art parametric object separation systems and provide significantly higher output audio quality.

[270] The proposed method of the invention describes a group detection method that is performed entirely based on existing bit stream information.

[271] The proposed inventive clustering solution results in a significant reduction in computational complexity. In general, singular value decomposition is computationally expensive and its complexity grows exponentially with the size of the matrix to be inverted:

[272] For a large number of downmix channels, computer K times an SVD operation for a smaller matrix size is computationally much more efficient:

[273] Using the same considerations, all parametric processing steps in the decoder can be implemented effectively by computing all matrix multiplications described in the system for the independent groups only and combining the results.

[274] An estimate of the reduction in complexity for different number of input audio objects, i.e., input audio objects, reduction mix channels, and a fixed number of 24 output channels is provided in the following table:

[275] The invention presents the following additional advantages:

[276] • For situations where only one group can be created, the output is bit identical with the current state of the art system.

[277] • Grouping presents the “through” feature of the system. This implies that if an input audio object is mixed alone in a downmix channel, the decoder has the ability to reconstruct it perfectly.

[278] The invention results in the following proposed exemplary modifications to the standard text.

[279] Added in “9.5.4.2.4 Regularized inverse operation”:

[280] The regularized inverse matrix approximation is calculated as

[281] The matrices V and A are determined as the singular value decomposition of the matrix Δ as: ∆ = V Λ V*

[282] The regularized inverse Ainv of the diagonal singular value matrix A is computed in accordance with 9.5.4.2.5.

[283] In case the matrix Δ is used in the calculation of the parametric unmixing matrix U, the described operations are applied to all submatrices Δk. A submatrix Δk is obtained by selecting the elements Δ (m, n) that correspond to the reduction mix channels m and n assigned to group k.

[284] Group k is defined by the smallest set of down mix channels with the following properties:

[285] • Input signals contained in the k-group reduction mix channels are not contained in any other reduction mix channels. An input signal is not contained in a down mix channel if the corresponding down mix gain is provided by the lowest quantization index (Table 49 of ISO/IEC 23003-2:2010).

[286] • All input signals i contained in the reduction mix channels of group k are unrelated to any input signal contained in any reduction mix channel of any other group (i.e., bsRelatedTo[i][j ] == 0).

[287] The results of the independent regularized inversion operations Jk ~Δk are combined to obtain the matrix J.

[288] The invention also results in the following proposed exemplary modifications to the standard text.

[289] 9.5.4.2.5 Regularized inverse operation

[290] The regularized inverse matrix approximation JJ ~ Δ1 is calculated as:

[291] The matrices V and A are determined as the singular value decomposition of the matrix Δ as:

[292] The regularized inverse Ainv of the diagonal singular value matrix A is computed in accordance with 9.5.4.2.6.

[293] In case the matrix Δ is used in the calculation of the parametric unmixing matrix U , the described operations are applied to all sub-matrices Δd. A submatrix Δd of size Ng ( idx1)=ch1 and gq(idx2)=ch2).

[294] The GQ group of size 1X Ngq is defined by the smallest set of down mix channels with the following properties:

[295] • Input signals contained in the GQ Group Reduction Mix channels are not contained in any other Reduction Mix channels. An input signal is not contained in a down mix channel if the corresponding down mix gain is provided by the lowest quantization index (Table 49 of ISO/IEC 23003-2:2010).

[296] • All input signals i contained in the reduction mix channels of group gq are unrelated to any input signal j contained in any reduction mix channel of any other group (i.e. bsRelatedTo[i][ j] == 0).

[297] The results of the independent regularized inversion operations Jq Δq are combined to obtain the matrix J as:

[298] 9.5.4.2.6 Regularization of singular values

[299] The regularized inverse operation (.)inv used for the diagonal singular value matrix A is determined as:

[300] The relative regularization scalar is determined using absolute threshold reg and maximum value of A as follows:

[301] In some of the following Figures, individual signals are shown as obtained from processing steps. This is done for a better understanding of the invention and is a possibility for carrying out the invention, i.e. extracting individual signals and carrying out processing steps on these signals or processed signals.

[302] The other embodiment is to calculate all necessary matrices and apply them as a last step to the encoded audio signal in order to obtain the decoded audio signal. This includes the calculation of different matrices and their respective combinations.

[303] One modality combines both forms.

[304] Figure 10 schematically shows an apparatus 10 for processing a plurality (here, in this example, five) of input audio objects 111 to provide a representation of the input audio objects 111 by a coded audio signal 100 .

[305] Input audio objects 111 are allocated or downmixed into downmix signals 101. In the shown embodiment, four out of five input audio objects 111 are assigned to two downmix signals 101 A single input audio object 111 is assigned to a third reduction mix signal 101. Thus, five input audio objects 111 are represented by three reduction mix signals 101.

[306] These reduction mix signals 101 are then - possibly after some of the processing steps not shown - combined with the encoded audio signal 100.

[307] Such coded audio signal 100 is fed to an apparatus of invention 1, for which an embodiment is shown in Figure 11.

[308] From the encoded audio signal 100, the three reduction mix signals 101 (compare to Figure 10) are extracted.

[309] The reduction mix signals 101 are grouped - in the example shown - into two groups of reduction mix signals 102.

[310] Since each reduction mix signal 101 is associated with a certain number of input audio objects, each group of reduction mix signals 102 refers to a certain number of input audio objects (a corresponding expression is the input object). In this way, each group of downmix signals 102 is associated with a set of input audio objects among the plurality of input audio objects that are encoded by the encoded audio signal 100 (compare to Figure 10).

[311] Grouping occurs in the embodiment shown under the following restrictions:

[312] 1. Each input audio object 111 belongs to only one set of input audio objects and thus to one group of downmix signals 102.

[313] 2. Each input audio object 111 has no signed relationship in the encoded audio signal to an input audio object 111 that belongs to a different set associated with a different downmix signal group. This means that the encoded audio signal does not have such information that due to the standard would result in a combined computation of the respective input audio objects.

[314] 3. The number of downmix signals 101 within respective groups 102 is minimized.

[315] The (here: two) groups of reduction mix signals 102 are processed individually in the following to obtain five output audio signals 103 corresponding to the five input audio objects 111.

[316] A group of reduction mixing signals 102 that is associated with the two reduction mixing signals 101 that covers two pairs of input audio objects 111 (compare to Figure 10) allows obtaining four output audio signals 103.

[317] The other group of reduction mix signals 102 results in an output signal 103 as the only reduction mix signal 101 or this group of reduction mix signals 102 (or more precisely: group of a signal signal mixdown) refers to an input audio object 111 (compare to Figure 10).

[318] The five output audio signals 103 are combined into a decoded audio signal 110 as the output of the device 1.

[319] In the embodiment of Figure 11, all processing steps are performed individually on the reduction mix signal groups 102.

[320] The embodiment of the device 1 shown in Figure 12 can receive, here, the same coded audio signal 100 as the device 1 shown in Figure 11 and obtained by a device 10, as in Figure 10.

[321] From the encoded audio signal 100, the three reduction mixing signals 101 (for three transport channels) are obtained and grouped into two groups of reduction mixing signals 102. These groups 102 are processed individually to obtain five processed signals 104 that correspond to the five input audio objects shown in Figure 10.

[322] In the following steps, from the five processed signals 104 together, eight output audio signals 103 are obtained, for example, rendered to be used for eight output channels. The output audio signals 103 are combined into the decoded audio signal 110 which is the output of the apparatus 1. In this embodiment, individual processing as well as joint processing is performed on the groups of the downmix signals 102.

[323] Figure 13 shows some steps of an embodiment of the method of the invention in which a coded audio signal is decoded.

[324] In step 200, the downmix signals are extracted from the encoded audio signal. In the following step 201, the reduction mix signals are allocated to reduction mix signal groups.

[325] In step 202, each group of reduction mix signals is processed individually to provide individual group results. The individual processing of the groups comprises at least the demixer to obtain representations of the audio signals that have been combined through downmixing of the input audio objects in the encoding process. In one embodiment - not shown here - individual processing is followed by joint processing.

[326] In step 203, these group results are combined into a decoded audio signal to be the output.

[327] Figure 14 again shows an embodiment of the apparatus 1, in which all processing steps after grouping the reduction mixing signals 101 of the encoded audio signal 100 into groups of reduction mixing signals 102 are performed individually. The apparatus 1, which receives the encoded audio signal 100 with the down mixing signals 101, comprises a grouper 2 that groups the down mixing signals 101 to provide the groups of down mixing signals 102. The groups of reduction mix signals 102 are processed by a processor 3 that performs all necessary steps individually on each group of reduction mix signals 102. The individual group results of processing the groups of reduction mix signals 102 are output audio 103 which are combined by combiner 4 in order to obtain the decoded audio signal 110 to be output by device 1.

[328] The apparatus 1 shown in Figure 15 differs from the embodiment shown in Figure 14 after grouping the reduction mix signals 101. In the example, not all processing steps are performed individually on the groups of reduction mix signals 102, however, some steps are performed together, thus taking more than one group of reduction mix signals 102 into consideration.

[329] Because of this, the processor 3 in this embodiment is configured to perform only part or at least one individual processing step. The result of the processing are processed signals 104 which are processed together by post-processor 5. The obtained output audio signals 103 are finally combined by combiner 4 which results in decoded audio signal 110.

[330] In Figure 16, a processor 3 is schematically shown that receives the groups of reduction mixing signals 102 and provides the output audio signals 103.

[331] Processor 3 comprises a demixer 300 configured to demix the down mix signals 101 from respective groups of down mix signals 102. The demixer 300 thereby reconstructs the individual input audio objects that have been combined by the encoder into the respective reduction mix signals 101.

[332] The reconstructed or separated input audio objects are submitted to a renderer 302. The renderer 302 is configured to render the demixed downmix signals of the respective groups to an output situation of said decoded audio signal 110 in order to of providing rendered signals 112. The rendered signals 112 are thus adapted to the type of repetition situation of the decoded audio signal. The rendering depends, for example, on the number of speakers to be used, their arrangement or the type of effects to be obtained when reproducing the decoded audio signal.

[333] The rendered signals 112, Yuncompressed(dry), furthermore, are submitted to a post-mixer 303 configured to perform at least one decorrelation step on said rendered signals 112 and configured to combine the Ycompressed(wet) results of the step decorrelation performed with said respective rendered signals 112, Yuncompressed(dry). The post mixer 303 thereby takes steps to decorrelate the signals that have been combined into a down mix signal.

[334] The resulting output audio signals 103 are finally submitted to a combiner as shown above.

[335] For the steps, the processor 3 depends on a calculator 301 which here is separate from the different units of the processor 3, however, it is in an alternative embodiment - not shown - a resource of the grouper 300, the renderer 302 and the post- mixer 303, respectively.

[336] It is relevant that matrices, values, etc. required are calculated individually for the respective groups of reduction mixing signals 102. This implies, for example, that the matrices to be computed are smaller than the matrices used in the prior art. The matrices have sizes that depend on a number of input audio objects from the respective set of input audio objects associated with the down mix signal groups and/or a number of down mix signals that belong to the respective group of reduction mix signals.

[337] In the prior art, the matrix to be used for demixing has a size of the number of input audio objects or input audio signals times that number. The invention allows computing a smaller matrix with a size that depends on the number of input audio signals that belong to the respective group of downmix signals.

[338] In Figure 17, the purpose of rendering is explained.

[339] Apparatus 1 receives a coded audio signal 100 and decodes it providing a decoded audio signal 110.

[340] This decoded audio signal 110 is played back in a specific output situation or output situation 400. The decoded audio signal 110 should, in the example, be output from five speakers 401: Left, Right, Center, Surround Left and Surround right. The listener 402 is in the middle of this situation 400 facing the center speaker.

[341] The renderer in apparatus 1 distributes the reconstructed audio signals to be delivered to individual speakers 401 and thereby distributes a reconstructed representation of the original audio objects as sources of the audio signals in the given output situation 400.

[342] Rendering thus depends on the type of output situation 400 and the preferences of the listener 402.

[343] Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of a method step also represent a description of a corresponding block or item or a resource of a corresponding apparatus. Part or all of the steps of the method may be performed by (or with the use of) a hardware device, such as, for example, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, one or more of the most important method steps may be performed by such an apparatus.

[344] Depending on certain implementation requirements, embodiments of the invention may be implemented in hardware or in software or at least partially in hardware or at least partially in software. Implantation can be performed using a digital storage medium, for example, a floppy disk, a DVD, a Blu-Ray, a CD, a ROM, a PROM, an EPROM, an EEPROM or a Flash memory, which has electronically readable control signals stored therein, which cooperate (or have the ability to cooperate) with a programmable computer system, so that the respective method is carried out. Therefore, the digital storage medium can be computer readable.

[345] Some embodiments, in accordance with the invention, comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, so that one of the methods described herein is performed .

[346] Generally, embodiments of the present invention may be implemented as a computer program product with a program code, the program code being operative to perform one of the methods when the computer program product is run on a computer. Program code can, for example, be stored on a machine-readable carrier.

[347] Other embodiments comprise the computer program for carrying out one of the methods described herein, stored on a machine-readable carrier.

[348] In other words, an embodiment of the method of the invention is therefore a computer program that has a program code to perform one of the methods described herein, when the computer program is executed on a computer.

[349] A further embodiment of the methods of the invention is therefore a data carrier (or a digital storage medium, or a computer readable medium) comprising, recorded thereon, the computer program for carrying out one of the methods described in this document. The data carrier, digital storage medium, or recorded medium is typically tangible and/or non-transitory.

[350] A further embodiment of the method of the invention is, therefore, a data stream or a sequence of signals that represents the computer program for carrying out one of the methods described herein. The data stream or signal sequence may, for example, be configured to be transferred via a data communication connection, for example via the Internet.

[351] An additional embodiment comprises processing means, for example, a computer or a programmable logic device, configured or adapted to perform one of the methods described herein.

[352] An additional embodiment comprises a computer that has installed on it the computer program to perform one of the methods described in this document.

[353] A further embodiment according to the invention comprises an apparatus or system configured to transfer (e.g., electronically or optically) a computer program for carrying out one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may comprise, for example, a file server for transferring the computer program to the receiver.

[354] In some embodiments, a programmable logic device (e.g., a field-programmable gate array) may be used to realize some or all of the functionality of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably carried out by any hardware device.

[355] The apparatus described herein can be implanted with the use of a hardware apparatus, or with the use of a computer, or with the use of a combination of a hardware apparatus and a computer.

[356] The methods described herein can be carried out with the use of a hardware apparatus, or with the use of a computer, or with the use of a combination of a hardware apparatus and a computer.

[357] REFERENCES

[358] [BCC] C. Faller and F. Baumgarte, “Binaural Cue Coding - Part II: Schemes and applications,” IEEE Trans. on Speech and Audio Proc., volume 11, no 6, November 2003.

[359] [ISS1] M. Parvaix and L. Girin: “Informed Source Separation of underdetermined instantaneous Stereo Mixtures using Source Index Embedding”, IEEE ICASSP, 2010.

[360] [ISS2] M. Parvaix, L. Girin, J.-M. Brossier: “A watermarking-based method for informed source separation of audio signals with a single sensor”, IEEE Transactions on Audio, Speech and Language Processing, 2010.

[361] [ISS3] A. Liutkus, J. Pinel, R. Badeau, L. Girin, G. Richard: “Informed source separation through spectrogram coding and data embedding”, Signal Processing Journal, 2011.

[362] [ISS4] A. Ozerov, A. Liutkus, R. Badeau, G. Richard: “Informed source separation: source coding meets source separation”, IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, 2011.

[363] [ISS5] S. Zhang and L. Girin: “An Informed Source Separation System for Speech Signals”, INTERSPEECH, 2011.

[364] [ISS6] L. Girin and J. Pinel: “Informed Audio Source Separation from Compressed Linear Stereo Mixtures”, AES 42nd International Conference: Semantic Audio, 2011.

[365] [JSC] C. Faller, “Parametric Joint-Coding of Audio Sources”, 120th AES Convention, Paris, 2006.

[366] [SAOC] ISO/IEC, “MPEG audio technologies - Part 2: Spatial Audio Object Coding (SAOC)”, ISO/IEC JTC1/SC29/WG11 (MPEG) International Standard 23003-2.

[367] [SAOC1] J. Herre, S. Disch, J. Hilpert, O. Hellmuth: “From SAC To SAOC - Recent Developments in Parametric Coding of Spatial Audio”, 22nd Regional UK AES Conference, Cambridge, GB, April 2007.

[368] [SAOC2] J. Engdegârd, B. Resch, C. Falch, O. Hellmuth, J. Hilpert, A. Holzer, L. Terentiev, J. Breebaart, J. Koppens, E. Schuijers and W. Oomen: “Spatial Audio Object Coding (SAOC) - The Upcoming MPEG Standard on Parametric Object Based Audio Coding”, 124th AES Convention, Amsterdam, 2008.

[369] [SAOC3D] ISO/IEC, JTC1/SC29/WG11 N14747, Text of ISO/MPEG 23008-3/DIS 3D Audio, Sapporo, July 2014.

[370] [SAOC3D2] J. Herre, J. Hilpert, A. Kuntz, and J. Plogsties, “MPEG-H Audio - The new standard for universal spatial / 3D audio coding”, 137th AES Convention, Los Angeles, 2011.

Claims (22)

1. Apparatus (1) for processing a coded audio signal (100) comprising a plurality of reduction mixing signals (101) associated with a plurality of input audio objects (111) and characterized object parameters (E) comprising: a grouper (2) configured to group said plurality of reduction mixing signals (101) into a plurality of groups of reduction mixing signals (102) associated with a set of input audio objects (111) of said plurality of input audio objects (111), a processor (3) configured to perform at least one individual processing step on the object parameters (Ek) of each set of input audio objects (111) in order to providing group results (103, 104), and a combiner (4) configured to combine said group results (103, 104) or processed group results to provide a decoded audio signal (110), wherein the said grouper (2) is configured to group said plurality of down mixing signals (101) into said plurality of groups of down mixing signals (102) so that each input audio object (111) among said plurality of input audio objects (111) belong to only one set of input audio objects (111). 2. Apparatus (1) according to claim 1, characterized in that said grouper (2) is configured to group said plurality of downmixing signals (101) into said plurality of groups of downmixing signals ( 102) such that each input audio object (111) of each set of input audio objects is free from a signal relationship in the encoded audio signal (100) with other input audio objects (111) or has a signaled relationship in the encoded audio signal (100) only with at least one input audio object (111) that belongs to the same set of input audio objects (111). 3. Apparatus (1) according to claim 1 or 2, characterized in that said grouper (2) is configured to group said plurality of reduction mixing signals (101) into said plurality of groups of reduction mixing signals. reduction (102) while minimizing a diversity of reduction mix signals (101) within each group of reduction mix signals (102). 4. Apparatus (1) according to any one of claims 1 to 3, characterized in that said grouper (2) is configured to group said plurality of downmix signals (101) into said plurality of groups of downmix signals. reduction mix (102) so that only a single reduction mix signal (101) belongs to a group of reduction mix signals (102). 5. Apparatus (1) according to any one of claims 1 to 4, characterized in that said grouper (2) is configured to group said plurality of downmix signals (101) into said plurality of groups of downmix signals. downmixing (102) based on information within said encoded audio signal (100). 6. Apparatus (1) according to any one of claims 1 to 5, characterized in that said grouper (2) is configured to group said plurality of downmix signals (101) into said plurality of groups of downmix signals. reduction mixing (102) by applying at least the following steps: detecting the possibility of a reduction mixing signal (101) being assigned to an existing group of reduction mixing signals (102); detect the possibility that at least one input audio object (111) among the plurality of input audio objects (111) associated with the downmix signal (101) is part of a set of input audio objects (111 ) associated with an existing group of reduction mix signals (102); assign the reduction mix signal (101) to a new group of reduction mix signals (102) in case the reduction mix signal (101) is exempt from an assignment to an existing group of reduction mix signals (102) and in the event that all input audio objects (111) among the plurality of input audio objects associated with the down mixing signal (101) are exempt from an association with an existing group of down mixing signals (101) reduction (102); and combining the reduction mix signal (101) with an existing group of reduction mix signals (102) in case the reduction mix signal (101) is assigned to the existing group of reduction mix signals (102) or in the event that at least one input audio object (111) among the plurality of input audio objects associated with the down mixing signal (101) is associated with the existing group of down mixing signals (102). 7. Apparatus (1) according to any one of claims 1 to 6, characterized in that said processor (3) is configured to perform several individual processing steps on the object parameters (Ek) of each set of audio objects of input (111) in order to provide individual matrices as group results, and wherein said combiner (4) is configured to combine said individual matrices. 8. Apparatus (1) according to any one of claims 1 to 7, characterized in that said processor (3) is configured to perform at least one individual processing step on the object parameters (Ek) of each set of processing objects. input audio (111) in order to provide individual matrices, wherein said apparatus (1) comprises a post-processor (5) configured to jointly process object parameters in order to provide at least one general matrix, and wherein said combiner (4) is configured to combine said individual matrices and said at least one general matrix. 9. Apparatus (1), according to any one of claims 1 to 8, characterized in that said processor (3) comprises a calculator (301) configured to compute, individually, for each group of reduction mixing signals (102) , matrices with sizes that depend on at least one of a plurality of input audio objects from the set of input audio objects associated with the respective group of downmix signals (102) and a plurality of downmix signals ( 101) that belong to the respective group of reduction mix signals (102). 10. Apparatus (1), according to any one of claims 1 to 9, characterized in that the processor (3) is configured to compute, for each group of reduction mixing signals, an individual threshold based on a maximum value of energy within the respective group of reduction mix signals (102). 11. Apparatus (1) according to any one of claims 1 to 10, characterized in that said processor (3) is configured to determine an individual reduction mixing matrix (Dk) for each group of reduction mixing signals ( 102), wherein said processor (3) is configured to determine an individual group covariance matrix (Ek) for each group of downmix signals (102), wherein said processor (3) is configured to determine an individual group reduction mixing covariance matrix (Δk) for each group of reduction mixing signals (102) based on the individual reduction mixing matrix (Dk) and the individual group covariance matrix (Ek), and wherein said processor (3) is configured to determine an individual regularized inverse group matrix (Jk) for each group of downmix signals (102). 12. Apparatus (1) according to claim 11, characterized in that said combiner (4) is configured to combine individual regularized inverse group matrices (Jk) to obtain an overall regularized inverse group matrix (J). 13. Apparatus (1) according to claim 11 or 12, characterized in that said processor (3) is configured to determine an individual group parametric demix matrix (Uk) for each group of downmix signals (102 ) based on the individual reduction mixing matrix (Dk), the individual group covariance matrix (Ek) and the individual regularized inverse group matrix (Jk), and wherein said combiner (4) is configured to combine the an individual group parametric unmixing matrix (Uk) to obtain an overall group parametric unmixing matrix (U). 14. Apparatus (1) according to claim 13, characterized in that said processor (3) is configured to determine an individual group parametric demix matrix (Uk) for each group of downmix signals (102) with based on the individual reduction mixing matrix (Dk), the individual group covariance matrix (Ek) and the individual regularized inverse group matrix (Jk), and wherein said combiner (4) is configured to combine the individual group parametric unmixing (Uk) to obtain an overall group parametric unmixing matrix (U). 15. Apparatus (1) according to any one of claims 1 to 14, characterized in that said processor (3) is configured to determine an individual group rendering matrix (Rk) for each group of downmix signals ( 102). 16. Apparatus (1) according to claim 15, characterized in that said processor (3) is configured to determine an individual upmixing matrix (RkUk) for each group of downmixing signals (102) based in the individual group render matrix (Rk) and the individual group parametric demix matrix (Uk), and wherein said combiner (4) is configured to combine the individual boost mixing matrices (RkUk) to obtain a matrix of general boost mixing (RU). 17. Apparatus (1) according to claim 15 or 16, characterized in that said processor (3) is configured to determine an individual group covariance matrix (Ck) for each group of downmix signals (102) based on the individual group rendering matrix (Rk) and the individual group covariance matrix (Ek), and wherein said combiner (4) is configured to combine the individual group covariance matrices (Ck) to obtain a overall group covariance matrix (C). 18. Apparatus (1) according to any one of claims 15 to 17, characterized in that said processor (3) is configured to determine an individual group covariance matrix of the parametrically estimated signal (Eyuncompressed(dry))k with based on the individual group rendering matrix (Rk), the individual group parametric unmixing matrix (Uk), the individual reduction mixing matrix (Dk), and the individual group covariance matrix (Ek), and where the said combiner (4) is configured to combine the individual group covariance matrices of the parametrically estimated signal (Eyuncompressed(dry))k to obtain a general parametrically estimated signal Eyuncompressed(dry) . 19. Apparatus (1), according to any one of claims 1 to 18, characterized in that said processor (3) is configured to determine a regularized inverse matrix (J) based on a singular value decomposition of a covariance matrix reduction mixer (EDMX). 20. Apparatus (1), according to any one of claims 1 to 19, characterized in that said processor (3) is configured to determine, for a determination of a parametric demixing matrix (U), the submatrix (Δk), selecting elements (Δ (m, n)) corresponding to the reduction mixing signals (m, n) assigned to the respective group (k) of reduction mixing signals (102). 21. Apparatus (1) according to any one of claims 1 to 20, characterized in that said combiner (4) is configured to determine a post-mix matrix (P) based on matrices determined individually for each group of input signals. down mixing (102), and wherein said combiner (4) is configured to apply the post-mix matrix (P) to the plurality of down mixing signals (101) in order to obtain the decoded audio signal ( 110). 22. Method (1) for processing a coded audio signal (100) comprising a plurality of downmix signals (101) associated with a plurality of input audio objects (111) and object parameters (E), said method being characterized by comprising: grouping reduction mixing signals (101) into a plurality of groups of reduction mixing signals (102) associated with a set of input audio objects (111) among said plurality of input audio objects (111), perform at least one individual processing step on the object parameters (Ek) of each set of input audio objects (111) in order to provide group results (103, 104), and combining said group results (103, 104) to provide a decoded audio signal (110), wherein grouping said plurality of down mixing signals (101) into said plurality of groups of down mixing signals (101) reduction (102) so that each input audio object (111) among said plurality of input audio objects (111) belongs to only one set of input audio objects (111).