RU2772423C1 - Device, method and computer program for encoding, decoding, scene processing and other procedures related to spatial audio coding based on dirac using low-order, medium-order and high-order component generators - Google Patents

Abstract

FIELD: computer technology.

SUBSTANCE: invention relates to the field of computer technology for processing audio data. The expected result is achieved by generating a description of a low-order sound field from the input signal up to a predetermined order and mode, while the low-order generator is designed to output a description of a low-order sound field by copying the input signal or performing a weighted combination of input signal channels; generating a description of a medium-order sound field above the said predefined order or with the said predefined order and above the predefined mode and below the high order, using the synthesis of at least one direct part and at least one diffuse part, using direction data and diffusivity data, so that the description of a medium-order sound field contains a direct contribution and diffuse contribution; and generating a description of a high-order sound field having a high-order component or higher, using the synthesis of at least one direct part without the synthesis of any diffuse component, so that the description of a high-order sound field contains only a direct contribution.

EFFECT: reducing the delay to enable conversational services in mobile communication networks.

22 cl, 15 dwg

Description

The present invention is directed to audio coding and, in particular, to generating a sound field description from an input signal using one or more audio component generators.

The Directional Audio Coding (DirAC) technique [1] is an efficient approach to spatial audio analysis and reproduction. DirAC uses a perceptually motivated sound field representation based on direction of arrival (DOA) and diffuseness measured for each frequency band. It is based on the assumption that at one point in time and in one critical range the spatial resolution of the auditory system is limited by decoding one pointer for direction and another pointer for interural consistency. Spatial audio is then represented in the frequency domain by smoothly mixing two streams: a non-directional diffuse stream and a directional non-diffuse stream.

DirAC was originally intended for B-format recorded audio, but it can also be extended to microphone signals conforming to some particular speaker layout such as 5.1 [2] or any configuration of microphone arrays [5]. In the latter case, more flexibility can be achieved by not recording signals for some particular speaker circuit, but instead recording intermediate format signals.

Such an intermediate format, which has proven itself in practice, is represented by Ambisonics / Ambisonics (higher order) [3]. From the ambisonic signal it is possible to generate signals of any desired loudspeaker scheme, including binaural signals for playback through headphones. This requires a special renderer that is applied to the ambisonic signal using either a linear ambisonic renderer [3] or a parametric renderer such as directional audio coding (DirAC).

An ambisonic signal can be represented as a multi-channel signal in which each channel (referred to as the ambisonic component) is equivalent to a coefficient of the so-called spatial basis function. Using the weighted sum of these spatial basis functions (with weights corresponding to the coefficients), it is possible to recreate the original sound field at the recording location [3]. Therefore, the coefficients of the spatial basis function (ie, the ambisonic components) are a compact description of the sound field at the recording location. There are different types of spatial basis functions, such as spherical harmonics (SH) [3] or cylindrical harmonics (CH) [3]. CH can be used to describe the sound field in 2D space (for example, to reproduce 2D sound), while SH can be used to describe the sound field in 2D and 3D space (for example, to reproduce 2D and 3D sound). ).

, который приходит с определенного направления

, приводит в результате к пространственному аудиосигналу

, который может быть представлен в амбисоническом формате путем расширения сферических гармоник вплоть до порядка отсечения H:For example, an audio signal

that comes from a certain direction

, results in a spatial audio signal

, which can be represented in ambisonic format by extending the spherical harmonics up to the cutoff order H :

представляет собой сферические гармоники порядка l и моды (mode) m, а

представляют собой коэффициенты расширения. С увеличением порядка отсечения H такое расширение приводит к более точному пространственному представлению. Сферические гармоники вплоть до порядка H=4 с индексом нумерации амбисонических каналов (ACN) проиллюстрированы на Фиг. 1a для порядка n и моды m.where

are spherical harmonics of order l and mode (mode) m , and

are the expansion coefficients. As the cutoff order H increases, this extension leads to a more accurate spatial representation. Spherical harmonics up to order H=4 with ambisonic channel numbering index (ACN) are illustrated in FIG. 1a for order n and mode m .

DirAC has already been extended to deliver higher order ambisonic signals from a first order ambisonic signal (FOA, also referred to as B-format) or from various microphone arrays [5]. This document focuses on a more efficient way to synthesize higher order ambisonic signals from DirAC parameters and a reference signal. In this document, a reference signal, also referred to as a downmix signal, is considered to be a subset of a higher order ambisonic signal, or a linear combination of a subset of the ambisonic components.

In addition, the present invention considers a case in which DirAC is used to parametrically transmit an audio scene. In this case, the downmix signal is encoded with a conventional basic audio encoder while the DirAC parameters are transmitted in compressed form as ancillary information. The advantage of the present method is that it takes into account the quantization error that occurs during audio coding.

The following is an overview of a DirAC-based spatial audio coding system developed for immersive voice and audio services (IVAS). This represents one of various contexts, such as an overview of the DirAC spatial audio encoder system. The goal of such a system is to be able to process various spatial audio formats representing an audio scene and encode them at low bit rates, and to reproduce the original audio scene after transmission as faithfully as possible.

The system can take various representations of audio scenes as input. The input audio scene may be captured by multi-channel signals designed to be reproduced at various speaker positions, by audible objects along with metadata describing the positions of those objects over time, or by a first order or higher order ambisonic format representing the sound field at the listening position or reference position.

Preferably, this system is based on 3GPP Enhanced Voice Services (EVS), as such a solution is expected to operate with low latency to enable conversational services in mobile communication networks.

As shown in FIG. 1b, an encoder (IVAS encoder) may support various audio formats transmitted to the system separately or at the same time. Audio signals may be acoustic in nature, picked up by microphones, or electrical in nature, to be transmitted to loudspeakers. Supported audio formats can be multi-channel signal, first order and higher order ambisonic components, and audio objects. A complex audio scene can also be described by combining different input formats. All audio formats are then passed to the DirAC parser, which extracts a parametric representation of the entire audio scene. The direction of arrival and the diffuseness measured for each time-frequency unit form the parameters. The DirAC parser is followed by a spatial metadata encoder that quantizes and encodes the DirAC parameters to obtain a low bit rate parametric representation.

Along with these parameters, the downmix signal output from various audio sources or input signals is encoded for transmission using a conventional basic audio encoder. In this case, an EVS-based audio encoder is applied to encode the downmix signal. The downmix signal consists of different channels, called transport channels: the signal can be, for example, four coefficient signals constituting a B-format signal, a stereo pair, or a mono downmix, depending on the target bitrate. The encoded spatial parameters and the encoded audio bitstream are multiplexed before transmission over the communication channel.

A DirAC-based spatial audio encoder side supporting various audio formats is illustrated in FIG. 1b. Acoustic/electrical input (input data) 1000 is input to the encoder interface 1010, where the encoder interface has specific functionality for the first order ambisonic (FOA) or high order ambisonic (HOA) illustrated at 1013. In addition, the encoder interface has functionality for multichannel ( MC) data such as stereo data, 5.1 data, or data having more than two or five channels. In addition, the encoder interface 1010 has functionality for object encoding, such as SAOC (Spatial Audio Object Coding) illustrated 1011. The IVAS encoder comprises a DirAC stage 1020 with a DirAC analysis unit 1021 and a downmix (DMX) unit 1022. The signal output by block 1022 is encoded by an IVAS core encoder 1040, such as an AAC or EVS encoder, and the metadata generated by block 1021 is encoded using a DirAC metadata encoder 1030.

In the decoder shown in FIG. 2, the transport channels are decoded by the core decoder, while the DirAC metadata is first decoded before being transmitted with the decoded transport channels to the DirAC synthesis block. At this stage, various options may be considered. It may be desirable to play a certain audio scene directly on some speaker or headphone configurations, as is usually possible in a traditional DirAC system (MC in Fig. 2).

The decoder may also deliver individual objects as they were presented at the encoder side (Objects in FIG. 2).

Alternatively, it may also be necessary to render the scene to ambisonic format for other further manipulations such as rotating, flipping or moving the scene (FOA/HOA in Fig. 2) or to use an external renderer not defined in the source system.

A DirAC spatial audio coding decoder delivering various audio formats is illustrated in FIG. 2 and contains an IVAS decoder 1045 and a further connected decoder interface 1046. The IVAS decoder 1045 includes a base IVAS decoder 1060 that is configured to perform an operation of decoding the content encoded by the base IVAS encoder 1040 of FIG. 1b. In addition, a DirAC metadata decoder 1050 is provided that provides decoding functionality for decoding content encoded by the DirAC metadata encoder 1030. Synthesizer 1070 receives data from block 1050 and 1060 and with or without some user interactivity, output (output data) is input to decoder interface 1046 which generates FOA/HOA data, as illustrated in 1083, multi-channel data (MC data), as illustrated at block 1082, or object data, as illustrated at block 1081.

компонент. Опционально он также может выводить амбисонический сигнал, рендеринг которого выполняется с конкретной схемой громкоговорителей. Далее мы подробно рассмотрим, как получить

компонент из сигнала понижающего микширования, сопровождаемого в некоторых случаях входными пространственными параметрами.Conventional HOA synthesis using the DirAC paradigm is depicted in FIG. 3. An input signal, referred to as a downmix signal, is a time-frequency unit analyzed by a frequency filter bank. The frequency filter bank 2000 may be a complex valued filter bank such as a complex QMF, or a block transform such as an STFT. Synthesizing HOA generates an output ambisonic signal of the order H containing

component. Optionally, it can also output an ambisonic signal, which is rendered with a particular loudspeaker layout. Next, we'll take a closer look at how to get

component from the downmix signal, followed in some cases by input spatial parameters.

The downmix signal may be the original microphone signals or a mixture of the original signals representing the original audio scene. For example, if an audio scene is captured by a sound field microphone, the downmix signal can be an omnidirectional stage component (W), a stereo downmix (L/R), or a first order ambisonic signal (FOA).

For each time-frequency slice, the direction of sound, also referred to as the direction of arrival (DOA), and the diffuseness coefficient are estimated by the direction estimator 2020 and diffuseness estimator 2010, respectively, if the downmix signal contains sufficient information to determine such DirAC parameters. This is the case, for example, if the downmix signal is a first order ambisonic (FOA) signal. Alternatively, or if the downmix signal is insufficient to determine such parameters, the parameters may be passed directly to the DirAC synthesis via an input bitstream containing these spatial parameters. The bitstream may consist, for example, of quantized and encoded parameters taken as ancillary information in the case of audio data transmission applications. In this case, the parameters are derived outside the DirAC synthesis module from the original microphone signals or input audio formats passed to the encoder side DirAC analysis module, as illustrated by switch 2030 or 2040.

направленных усилений

где H представляет порядок синтезируемого амбисонического сигнала.The sound directions are used by the directional gain estimator 2050 to estimate, for each time-frequency slice from a plurality of time-frequency slices, one or more sets of

directional gains

where H represents the order of the synthesized ambisonic signal.

единичной нормы или через азимутальный угол

и/или угол

возвышения, которые связаны, например, как:Directional gains can be obtained by evaluating the spatial basis function for each estimated sound direction with the desired order (level) l and mode m of the ambisonic signal for synthesis. The direction of the sound can be expressed, for example, in terms of the vector

unit norm or through the azimuth angle

and/or angle

elevations that are related, for example, as:

After estimating or obtaining the sound direction, the response of the spatial basis function of the desired order (level) l and mode m can be determined, for example, by considering real-valued spherical harmonics with SN3D normalization as the spatial basis function:

представляют собой функции Лежандра, а

представляет собой член нормализации как для функций Лежандра, так и для тригонометрических функций, который принимает следующую форму для SN3D:with ranges 0≤l≤H and - l≤m≤l .

are the Legendre functions, and

is a normalization term for both Legendre and trigonometric functions, which takes the following form for SN3D:

равна единице для m=0 и нулю в ином случае. Затем направленные усиления выводятся непосредственно для каждого частотно-временного фрагмента с индексами (k, n) как:where is the Kronecker delta

equals one for m=0 and zero otherwise. The directional gains are then derived directly for each time-frequency slice with indices (k, n) as:

прямого звука вычисляются путем выведения опорного сигнала

из сигнала понижающего микширования и умножаются на направленные усиления и коэффициентную функцию диффузности

:Ambisonic components

direct sound are calculated by deriving the reference signal

from the downmix signal and multiplied by the directional gains and diffuseness coefficient function

:

может быть всенаправленной компонентой сигнала понижающего микширования или линейной комбинацией K каналов сигнала понижающего микширования.For example, the reference signal

may be an omnidirectional component of the downmix signal or a linear combination of the K channels of the downmix signal.

посредством рассмотрения интеграла квадрата абсолютной величины пространственной базисной функции

по всем возможным углам

и

:The ambisonic component of a diffuse sound can be modeled by using the spatial basis function response for sounds coming from all possible directions. One example is the definition of the mean response

by considering the integral of the square of the absolute value of the spatial basis function

in all possible angles

and

:

диффузного звука вычисляются из сигнала

умноженного на средний ответ и коэффициентную функцию диффузности

:Ambisonic components

diffuse sound calculated from signal

multiplied by the mean response and diffuseness coefficient function

:

может быть получен с помощью различных декорреляторов, применяемых к опорному сигналу

.Signal

can be obtained with various decorrelators applied to the reference signal

.

желаемого порядка (уровня) l и моды m для частотно-временного фрагмента (k, n), т.е.Finally, the ambisonic component of the direct sound and the ambisonic component of the diffuse sound are combined 2060, for example, through a summation operation, to obtain the final ambisonic component

desired order (level) l and mode m for the time-frequency fragment (k, n) , i.e.

The resulting ambisonic components can be converted back to the time domain using the inverse filter block 2080 or inverse STFT, stored, transmitted, or used, for example, for spatial audio applications. Alternatively, a linear ambisonic renderer 2070 may be applied per frequency band to obtain signals to be reproduced with a particular speaker circuit or via headphones before converting the speaker or binaural signals to the time domain.

диффузного звука могут быть синтезированы только вплоть до порядка L, где L<H. Это снижает вычислительную сложность, одновременно с этим избегая синтетических артефактов из-за интенсивного использования декорреляторов.It should be noted that [5] also indicates the possibility that the components

diffuse sound can only be synthesized up to the order of L , where L<H . This reduces computational complexity while avoiding synthetic artifacts due to heavy use of decorrelators.

The purpose of the present invention is to provide an improved circuit for generating a sound field description from an input signal.

This object is achieved by the device for generating a sound field description of item 1, the method for generating a sound field description of item 20, or the computer program of item 21.

The present invention according to the first aspect is based on the discovery that it is not necessary to perform the synthesis of the sound field components, including the calculation of the diffuse part, for all generated components. It is sufficient to perform the synthesis of diffuse components only up to a certain order. However, in order to avoid energy fluctuations or energy errors, energy compensation is performed by generating the sound field components of the first group of sound field components that have a diffuse and a direct component, such energy compensation being dependent on the diffuseness data and at least one of the number of sound field components in the second group, the maximum order of the first group sound field components, and the maximum order of the second group sound field components. In particular, according to the first aspect of the present invention, an apparatus for generating a sound field description from an input signal containing one or more channels comprises an input signal analyzer for obtaining diffuseness data from the input signal and an audio component generator for generating one or more audio components from the input signal. a sound field component of a first group of sound field components having a direct component and a diffuse component for each sound field component, and for generating from the input signal a second group of sound field components having only a direct component. In particular, the sound component generator performs energy compensation when generating the first group of sound field components, wherein the energy compensation depends on the diffuseness data and at least one of the number of sound field components in the second group, the number of diffuse components in the first group, the maximum order of the sound field components. fields of the first group and the maximum order of the sound field components of the second group.

The first group of sound field components may contain low order sound field components and medium order sound field components, and the second group contains high order sound field components.

An apparatus for generating a sound field description from an input signal containing at least two channels, according to the second aspect of the invention, comprises an input signal analyzer for obtaining direction data and diffuseness data from the input signal. The apparatus further comprises an estimator for estimating a first energy or amplitude related metric for the omnidirectional component derived from the input signal and for estimating a second energy or amplitude related metric for the directional component derived from the input signal. In addition, the apparatus comprises an audio component generator for generating sound field components for the sound field, the audio component generator being configured to perform directional component energy compensation using a first energy or amplitude related metric, a second energy or amplitude related metric, direction data. and diffuseness data.

In particular, the second aspect of the present invention is based on the discovery of the fact that in a situation where a directional component is received by a device for generating a sound field description and at the same time direction data and diffuseness data are also received, the direction and diffuseness data may be used to compensate for any errors likely introduced due to quantization or any other processing of the directional or omnidirectional component in the encoder. Thus, the direction and diffuseness data are not simply used to generate the description of the sound field as such, but these data are used "second time" to correct the directional component in order to eliminate or at least partially eliminate and therefore compensate for energy loss. directed component.

Preferably, this energy compensation is performed on low order components that are received at the interface of the decoder or that are generated from data received from the audio encoder generating the input signal.

According to a third aspect of the present invention, an apparatus for generating a sound field description using an input signal comprising a mono signal or a multi-channel signal comprises an input signal analyzer, a low audio component generator, a medium order component generator, and a high order component generator. In particular, the different "sub" generators are configured to generate the sound field components in an appropriate order based on a specific processing procedure that is different for each of the low, medium, or high order component generators. This ensures that an optimal compromise is maintained between processing requirements on the one hand, audio quality requirements on the other hand, and procedural usability on the other hand. Through this procedure, the use of decorrelators, for example, is limited to only generating medium-order components, but for generating low-order components and generating high-order components, any artifact-prone decorrelators are avoided. On the other hand, energy compensation is preferably performed for the energy loss of the diffuse components, and this energy compensation is performed only within the low order sound field components, or only within the medium order sound field components, or both in the low order sound field components and in the sound field of medium order. Preferably, the energy compensation for the directional component generated in the low order component generator is also performed using the transmitted directional diffuseness data.

Preferred embodiments relate to an apparatus, method or computer program for synthesizing an ambisonic signal (higher order) using the Directional Audio Coding (DirAC) paradigm, a perceptually motivated technique for spatial audio processing.

Embodiments refer to an efficient method for synthesizing an ambisonic representation of an audio scene from spatial parameters and a downmix signal. Using the method, but not limited to, the audio scene is transmitted and therefore encoded to reduce the amount of data transmitted. The downmix signal is then severely limited in the number of channels and quality by the bit rate that is available for transmission. The embodiments relate to an efficient way of using the information contained in the transmitted downmix signal to reduce synthesis complexity while improving quality.

Another embodiment of the invention relates to the diffuse component of the sound field, which can be limited to simulation only up to a predetermined order of synthesized components to avoid synthesizing artifacts. An embodiment provides a method for compensating for the resulting energy loss by amplifying the downmix signal.

Another embodiment concerns a directional sound field component whose characteristics can be changed within the downmix signal. The downmix signal may be further normalized in energy to keep the energy relationship dictated by the transmitted direction parameter but corrupted during transmission due to quantization or other introduced errors.

Further preferred embodiments of the present invention are described with reference to the accompanying drawings, in which:

Fig. 1a illustrates spherical harmonics with ambisonic channel/component numbering;

Fig. 1b illustrates the encoder side of a DirAC-based spatial audio coding processor;

Fig. 2 illustrates a DirAC-based spatial audio coding processor decoder;

Fig. 3 illustrates a high order ambisonic synthesis processor known in the art;

Fig. 4 illustrates a preferred embodiment of the present invention using the first aspect, the second aspect and the third aspect;

Fig. 5 illustrates a general processing scheme for energy compensation;

Fig. 6 illustrates an apparatus for generating a sound field description in accordance with the first aspect of the present invention;

Fig. 7 illustrates an apparatus for generating a sound field description in accordance with a second aspect of the present invention;

Fig. 8 illustrates an apparatus for generating a sound field description in accordance with a third aspect of the present invention;

Fig. 9 illustrates a preferred implementation of the low order component generator of FIG. eight;

Fig. 10 illustrates a preferred implementation of the mean order component generator of FIG. eight;

Fig. 11 illustrates a preferred implementation of the high-order component generator of FIG. eight;

Fig. 12a illustrates a preferred implementation of the compensation gain calculation according to the first aspect;

Fig. 12b illustrates an implementation of the compensation gain calculation according to the second aspect; and

Fig. 12c illustrates a preferred implementation of energy compensation combining the first aspect and the second aspect.

Fig. 6 illustrates an apparatus for generating a sound field description in accordance with the first aspect of the invention. The device includes an input signal analyzer 600 for obtaining diffuseness data from the input signal illustrated on the left side of FIG. 6. In addition, the apparatus includes a sound component generator 650 for generating from an input signal one or more sound field components of a first group of sound field components having a direct component and a diffuse component for each sound field component. In addition, the audio component generator generates from the input signal a second group of sound field components having only a direct component.

In particular, the sound component generator 650 is configured to perform energy compensation when generating the first group of sound field components. The energy compensation depends on the diffuseness data and the number of sound field components in the second group, or on the maximum order of the first group sound field components or the maximum order of the second group sound field components. In particular, according to the first aspect of the invention, energy compensation is performed to compensate for energy loss due to the fact that only direct components are generated for the second group of sound field components and no diffuse components are generated.

In contrast, in the first group of sound field components, the direct and diffuse parts are included in the sound field components. Thus, the sound component generator 650 generates, as illustrated by the upper grating, sound field components that have only a direct part and not a diffuse part, as illustrated in other figures, by reference 830, and the sound component generator generates sound field components, which have a straight part and a diffuse part, as illustrated by numerals 810, 820, which are explained below with reference to other figures.

Fig. 7 illustrates an apparatus for generating a sound field description from an input signal containing at least two channels, in accordance with a second aspect of the invention. The apparatus includes an input signal analyzer 600 for obtaining direction data and diffuseness data from an input signal. In addition, an estimator 720 is provided for estimating a first energy or amplitude related metric for the omnidirectional component derived from the input signal and for estimating a second energy or amplitude related metric for the directional component derived from the input signal.

In addition, the sound field description generating apparatus comprises a sound component generator 750 for generating sound field components for the sound field, wherein the sound component generator 750 is configured to perform directional component energy compensation using a first amplitude related metric, a second energy or amplitude related metric. indicator, direction data, and diffuseness data. Thus, the audio component generator generates, in accordance with the second aspect of the present invention, corrected/compensated directional (direct) components and, if appropriately implemented, other components of the same order as the input signal, such as omnidirectional components, which are preferably not undergo energy compensation or undergo energy compensation only for the purpose of compensating for diffuse energy, as discussed in the context of FIG. 6. It should be noted that the amplitude-related index can also be the norm or magnitude or absolute value of a directional or omnidirectional component such as B ₀ and B ₁ . Preferably, the power or energy output by a power of 2 is preferred as indicated in the equation, but other powers applied to a rate or magnitude or absolute value can also be used to obtain an energy or amplitude related metric.

In an implementation, an apparatus for generating a sound field description according to the second aspect performs energy compensation of a directional component of a signal included in an input signal comprising at least two channels, such that the directional component is included in the input signal or can be calculated from the input signal, for example, by calculating the difference between the two channels. This device can only perform correction without generating any higher order data or the like. However, in other embodiments, the audio component generator is configured to also generate other sound field components from other orders, as illustrated at 820, 830, described below, but for these (higher order) audio components for which there was no If any add-on parts are included, directional component energy compensation is optional.

Fig. 8 illustrates a preferred implementation of an apparatus for generating a sound field description using an input signal containing a mono signal or a multi-channel signal, in accordance with a third aspect of the present invention. The apparatus includes an input signal analyzer 600 for analyzing the input signal to derive direction data and diffuseness data. In addition, the apparatus comprises a low order component generator 810 for generating a low order sound field description from an input signal up to a predetermined order and a predetermined mode, wherein the low order component generator 810 is configured to derive a low order sound field description by copying or taking the input signal. or part of the input signal as is, or by performing a weighted combination of channels of the input signal when the input signal is a multi-channel signal. In addition, the apparatus comprises a mean order component generator 820 for generating a mean order sound field description above said predetermined order or with said predetermined order and above a predetermined mode and below or with a first cutoff order using synthesis of at least one direct part and at least one diffuse part, using direction data and diffuseness data, so that the average order sound field description contains a direct contribution and a diffuse contribution.

The apparatus for generating a sound field description further comprises a high order component generator 830 for generating a high order sound field description having a component of the above mentioned first cutoff order using synthesis of at least one direct part, wherein the high order sound field description contains only a direct contribution. Thus, in an embodiment, the synthesis of at least one direct part is performed without synthesizing any diffuse components, so that the high-order sound field description contains only the direct contribution.

Thus, low order component generator 810 generates a low order sound field description, medium order component generator 820 generates a medium order sound field description, and high order component generator generates a high order sound field description. The description of the low order sound field is extended up to a specific order and mode, such as in the context of high order ambisonic spherical components, as illustrated in FIG. 1. However, any other sound field description, such as a sound field description with cylindrical features or a sound field description with any other components other than any ambisonic representation, can also be generated in accordance with the first, second and/or third aspect of the present invention. .

The mean order component generator 820 generates sound field components above the aforementioned predetermined order or mode and up to a certain cutoff order, which is also referred to as L in the following description. Finally, the high order component generator 830 is configured to apply generation of sound field components from the cutoff order L up to the maximum order, referred to as H in the following description.

Depending on the implementation, the energy compensation provided by the audio component generator 650 of FIG. 6 cannot be applied to a low order component generator 810 or a medium order component generator 820 as illustrated by the respective reference numerals in FIG. 6, for the direct/diffuse sound component. In addition, the second group of sound field components generated by the sound field component generated by the sound field component generator 650 corresponds to the output of the high-order component generator 830 of FIG. 8 illustrated at 830 under direct/non-diffuse in FIG. 6.

With reference to FIG. 7, it is shown that the directional component energy compensation is preferably performed in the low order component generator 810 illustrated in FIG. 8, i.e. is performed for some or all of the sound field components up to a predetermined order and a predetermined mode, as illustrated at 810 above the top arrow exiting block 750. . 7 as illustrated by reference numerals 820, 830 indicated below this upper arrow. Thus, the low order component generator 810 of FIG. 8 can apply diffuse energy compensation according to the first aspect and directional (direct) signal compensation according to the second aspect, while the mean order component generator 820 can only perform diffuse component compensation because this mean order component generator generates output data having diffuse parts that can be improved in terms of their energy to have a higher energy budget for the diffuse components in the output signal.

The following is a reference to FIG. 4 illustrating the implementation of the first aspect, the second aspect and the third aspect of the present invention in one apparatus for generating a sound field description.

Fig. 4 illustrates an input analyzer 600. The input analyzer 600 includes a direction estimator 610, a diffuseness estimator 620, and switches 630, 640. The input signal analyzer 600 is configured to analyze the input signal, typically following the analysis filter bank 400, to look for time for each fragment. /frequency direction information, referred to as DOA, and/or diffuseness information. Direction information, DOA, and/or diffuseness information may also originate from the bitstream. Thus, in situations where this data cannot be extracted from the input signal, i.e. when the input signal has only an omnidirectional W component, then the input signal analyzer extracts direction data and/or diffuseness data from the bitstream. When, for example, the input signal is a two-channel signal having a left channel L and a right channel R, then analysis can be performed to obtain direction and/or diffuseness data. When the input signal is a first order ambisonic (FOA) signal or any other signal with more than two channels, such as an A-format signal or a B-format signal, then the actual signal analysis performed by block 610 or 620 can be performed. However, when the bitstream is parsed to extract direction data and/or diffuseness data from the bitstream, this is also the analysis performed by the input signal analyzer 600, but without actually analyzing the signal as otherwise. In the latter case, the analysis is performed on the bitstream and the input signal consists of both the downmix signal and the bitstream data.

In addition, the device for generating the sound field description illustrated in FIG. 4 includes a directional gain calculator 410, a splitter 420, a combiner 430, a decoder 440, and a synthesis filter bank 450. Synthesizing filter bank 450 receives data for a high-order ambisonic representation or signal to be reproduced by headphones, i.e. a binaural signal, or a signal to be reproduced by loudspeakers located in a specific loudspeaker configuration representing a multi-channel signal adapted to that particular loudspeaker configuration, from a sound field description that is usually independent of the specific loudspeaker configuration.

In addition, the device for generating the sound field description includes an audio component generator, typically consisting of a low order component generator 810, comprising a "low order component generation" block and a "low order component mixing" block. In addition, a medium order component generator 820 is provided, consisting of a generated reference signal block 821, decorrelators 823, 824, and a medium order component mixing block 825. And a high-order component generator 830 is also provided and represented in FIG. 4 containing a high order mixing unit 822. In addition, a (diffuse) compensation gain calculation block is provided, illustrated by reference numerals 910, 920, 930, 940. Reference numerals 910 to 940 are further explained with reference to Figures 12a to 12c.

Although not illustrated in FIG. 4, at least the diffuse signal energy compensation is performed not only in the low order audio component generator, which is clearly illustrated in FIG. 4, but this energy compensation can also be performed in the middle order component mixer 825.

In addition, FIG. 4 illustrates the situation where all processing is performed on the individual time/frequency slices generated by the analysis filter bank 400. Thus, a certain DOA value, a certain diffuseness value, and a certain processing for applying these values, as well as for applying different compensations, is provided for each time/frequency tile. In addition, the sound field components are also generated/synthesized for the individual time/frequency chunks, and the combining performed by the combiner 430 also occurs in the time/frequency domain for each individual time/frequency chunk, and additionally, the procedure of the HOA decoder 440 is performed in the domain time/frequency, and synthesis filter bank 450 then generates full band time domain signals with full range HOA components, full range binaural headphone signals, or full range speaker signals for specific speaker configurations.

Embodiments of the present invention use two basic principles:

диффузного звука могут быть синтезированы с ограничением только для компонент низкого порядка синтезируемого амбисонического сигнала вплоть до порядка L<H.Ambisonic components

diffuse sound can be synthesized with a limitation only for the low-order components of the synthesized ambisonic signal up to the order of L<H .

From the downmix signal, K low-order ambisonic components can usually be extracted, for which full synthesis is not required.

In the case of a mono downmix, the downmix is typically the omnidirectional W component of the ambisonic signal.

In the case of stereo downmixing, the left (L) and right (R) channels can easily be converted to ambisonic W and Y components.

In the case of FOA downmix, order 1 ambisonic components are already available. Alternatively, FOA can be recovered from a linear combination of a 4-channel downmix signal, DMX, which is, for example, A-format:

where

and

Two improvements can also be applied to these two principles.

The energy loss due to the lack of modeling of the ambisonic components of the diffuse sound up to the order of H can be compensated by amplifying the K low order ambisonic components extracted from the downmix signal.

In transmission applications in which the downmix signal is lossy encoded, the transmitted downmix signal is corrupted by quantization errors, which can be eliminated by limiting the energy ratio K of the low order ambisonic components extracted from the downmix signal.

Fig. 4 illustrates an embodiment of the new method. One of the differences from the state depicted in FIG. 3 is the differentiation of the mixing process, which differs depending on the order of the ambisonic component to be synthesized. The low order components are mainly determined from the low order components extracted directly from the downmix signal. Mixing low-order components can be as simple as directly copying the extracted components to the output.

However, in the preferred embodiment, the extracted components are further processed by applying energy compensation, diffuse function and L and H cutoff orders, or by applying energy normalization, diffuse function and sound directions, or both.

амбисонических компонент вплоть до порядка

отсечения, но только для прямого звука, игнорируя диффузный звук. Далее подробно описывается микширование или генерирование компонент низкого порядка.Mixing the mid-order components is effectively the same as the modern method (except for the optional diffuseness compensation) and generates and combines the ambisonic components of both direct and diffuse sounds up to the L clipping order, but ignoring the K low-order components already synthesized by mixing the low-order components. Mixing the high-order components consists of generating the remaining

ambisonic components up to order

clipping, but only for direct sound, ignoring diffuse sound. The following describes in detail the mixing or generation of low-order components.

без потери общности.The first aspect relates to energy compensation, generally illustrated in FIG. 6 showing the outline of the first aspect. The principle is explained for a specific case for

without loss of generality.

представляет собой физически корректный амбисонический сигнал с порядком

отсечения. Он содержит

коэффициентов, обозначенных

где

представляет собой порядок коэффициента, а

представляет собой моду. Обычно амбисонический сигнал

представляется в частотно-временной области.Fig. 5 shows a general processing scheme. Input vector

is a physically correct ambisonic signal with order

clipping. He contains

coefficients indicated

where

is the order of the coefficient, and

represents fashion. Usually ambisonic signal

represented in the time-frequency domain.

вплоть до максимального порядка

, где

. Результирующий вектор

содержит синтезированные коэффициенты порядка

, обозначаемые

. Синтезирование HOA обычно зависит от диффузности

(или аналогичного показателя), которая описывает, насколько диффузным является звуковое поле для текущей частотно-временной точки. Обычно коэффициенты в

синтезируются только в том случае, если звуковое поле становится недиффузным, тогда как в диффузных ситуациях коэффициенты становятся равными нулю. Это предотвращает появление артефактов в диффузных ситуациях, но также приводит к потере энергии. Подробности синтезирования HOA поясняются ниже.In the HOA synthesis block 820, 830, the ambisonic coefficients are synthesized from

up to the maximum order

, where

. Result vector

contains synthesized order coefficients

, denoted

. Synthesis of HOA usually depends on diffuseness

(or similar) that describes how diffuse the sound field is for the current time-frequency point. Usually the coefficients in

synthesized only if the sound field becomes non-diffuse, while in diffuse situations the coefficients become zero. This prevents artifacts in diffuse situations, but also results in wasted energy. Details of the synthesis of HOA are explained below.

в блоке 650, 750 компенсации энергии. Результирующий сигнал обозначается как

и имеет тот же максимальный порядок

, что и

. Компенсация энергии зависит от диффузности (или аналогичного показателя) и увеличивает энергию коэффициентов в диффузных ситуациях, так что потеря энергии коэффициентов в

компенсируется. Подробности поясняются далее.To compensate for energy loss in the diffuse situations mentioned above, we apply energy compensation to

in block 650, 750 energy compensation. The resulting signal is denoted as

and has the same maximum order

, which is

. Energy compensation depends on diffuseness (or similar) and increases the energy of the coefficients in diffuse situations, so that the energy loss of the coefficients in

compensated. Details are explained below.

объединяются 430 с синтезированными коэффициентами в

для получения выходного амбисонического сигнала

содержащего все

коэффициенты, т.е.In the combining block, the coefficients with compensated energy in

combined 430 with the synthesized coefficients in

to get the output ambisonic signal

containing everything

coefficients, i.e.

, например, рендеринг на основе ковариации или прямой рендеринг с использованием направленного аудиокодирования (DirAC). В простейшем случае коэффициенты в

синтезируются из всенаправленной компоненты

в

с использованиемNext, the synthesis of HOA is explained as an embodiment. There are several modern approaches to synthesizing HOA coefficients in

, such as covariance based rendering or direct rendering using Directional Audio Coding (DirAC). In the simplest case, the coefficients in

synthesized from an omnidirectional component

in

using

.

.

является направлением прихода (DOA) звука, а

представляет собой соответствующее усиление амбисонического коэффициента порядка l и моды m. Обычно

соответствует действительнозначной диаграмме направленности хорошо известной сферической гармонической функции порядка l и моды m, оцениваемой в DOA

. Диффузность Ψ становится 0, если звуковое поле является недиффузным, и 1, если звуковое поле является диффузным. Следовательно, коэффициенты

, вычисленные выше порядка L, становятся равными нулю в ситуациях диффузной записи. Обратите внимание, что параметры

,

и

могут быть оценены из амбисонического сигнала

первого порядка на основе вектора активной интенсивности звука, как поясняется в исходных документах DirAC.Here

is the direction of arrival (DOA) of the sound, and

is the corresponding gain of the ambisonic coefficient of order l and mode m . Usually

corresponds to the real-valued radiation pattern of the well-known spherical harmonic function of order l and mode m , estimated in DOA

. The diffuseness Ψ becomes 0 if the sound field is non-diffuse and 1 if the sound field is diffuse. Therefore, the coefficients

, computed above the order of L , become zero in diffuse write situations. Please note that the parameters

,

and

can be estimated from the ambisonic signal

first order based on the active sound intensity vector as explained in the original DirAC documents.

The energy compensation of the diffuse sound components is discussed next. To obtain energy compensation, we consider a typical sound field model in which the sound field consists of a direct sound component and a diffuse sound component, i.e. omnidirectional signal can be written as

является прямым звуком (например, плоской волной), а

является диффузным звуком. Предполагая эту модель звукового поля и SN3D нормализацию амбисонических коэффициентов, ожидаемая мощность физически корректных коэффициентов

определяется какwhere

is a direct sound (for example, a plane wave), and

is a diffuse sound. Assuming this sound field model and SN3D normalization of the ambisonic coefficients, the expected power of the physically correct coefficients is

defined as

представляет собой мощность прямого звука, а

представляет собой мощность диффузного звука. Кроме того,

представляет собой фактор направленности коэффициентов

го порядка, который определяется как

, где

представляет собой число коэффициентов на порядок l. Чтобы вычислить компенсацию энергии, мы можем либо рассмотреть DOA

(более точная компенсация энергии), либо предположить, что

является равномерно распределенной случайной величиной (более практичный подход). В последнем случае ожидаемая мощность

равнаHere,

is the direct sound power, and

is the diffuse sound power. Besides,

is the direction factor of the coefficients

th order, which is defined as

, where

is the number of coefficients per order of l . To calculate energy compensation, we can either consider DOA

(more accurate energy compensation), or assume that

is a uniformly distributed random variable (a more practical approach). In the latter case, the expected power

is equal to

обозначает физически корректный амбисонический сигнал максимального порядка

. Используя приведенные выше уравнения, общая ожидаемая мощность

определяется какFurther let

denotes a physically correct ambisonic signal of maximum order

. Using the equations above, the total expected power

defined as

полная ожидаемая мощность синтезируемого амбисонического сигнала

определяется какSimilarly, using the general definition of diffuseness

total expected power of the synthesized ambisonic signal

defined as

на

т.е.Energy compensation is carried out by multiplying the coefficient

on the

those.

теперь определяется какTotal expected ambisonic output power

is now defined as

должна соответствовать общей ожидаемой мощности

. Следовательно, квадрат коэффициента компенсации вычисляется какTotal Expected Power

must match the total expected power

. Therefore, the square of the compensation factor is calculated as

This can be simplified to

представляет собой диффузность,

представляет собой максимальный порядок входного амбисонического сигнала, а

представляет собой максимальный порядок выходного амбисонического сигнала.where

is diffuse,

represents the maximum order of the input ambisonic signal, and

represents the maximum order of the output ambisonic signal.

, где

амбисонических компонент диффузного звука синтезируются с использованием декорреляторов и среднего диффузного ответа.The same principle can be applied to

, where

The ambisonic components of a diffuse sound are synthesized using decorrelators and an average diffuse response.

и никакие компоненты диффузного звука не синтезируются. Это особенно верно для высоких частот, где абсолютные фазы не слышны, а использование декорреляторов неуместно. Затем компоненты диффузного звука могут быть смоделированы с помощью компенсации энергии путем вычисления порядка Lk и числа мод mk, соответствующих K компонентам низкого порядка, при этом K представляет число диффузных компонент в первой группе:In certain cases

and no diffuse sound components are synthesized. This is especially true at high frequencies, where absolute phases are not audible and the use of decorrelators is inappropriate. The diffuse sound components can then be modeled with energy compensation by calculating the order Lk and the number of modes mk corresponding to the K low order components, with K representing the number of diffuse components in the first group:

The compensating gain then becomes:

являлся физически корректным амбисоническим сигналом максимального порядка L. Однако на входной сигнал понижающего микширования могут оказывать влияние ошибки квантования, которые могут нарушать соотношение энергии. Это соотношение может быть восстановлено путем нормализации входного сигнала понижающего микширования:Next, embodiments of energy normalization of the direct sound components corresponding to the second aspect generally illustrated in FIG. 7. It was assumed above that the input vector

was a physically correct ambisonic signal of maximum order L . However, the downmix input signal may be affected by quantization errors, which may disturb the energy relationship. This relationship can be restored by normalizing the downmix input signal:

Given the direction of the sound and the diffuseness parameters, the direct and diffuse components can be expressed as:

как:The expected power according to the model can then be expressed for each of the components

as:

The compensating gain then becomes:

и

where 0

and

как:Alternatively, the expected power according to the model can then be expressed for each of the components

as:

The compensating gain then becomes:

и

where 0

and

и

являются комплексными значениями, и для вычисления

берется норма или величина или абсолютное значение или представление комплексного значения в полярных координатах и возводится в квадрат, чтобы получить ожидаемую мощность или энергию в качестве связанного с энергией или амплитудой показателя.

and

are complex values, and to calculate

a norm or magnitude or an absolute value or a complex value representation in polar coordinates is taken and squared to obtain the expected power or energy as an energy or amplitude related metric.

Compensating the energy of the diffuse sound components and normalizing the energy of the direct sound components can be achieved jointly by applying amplification of the following form:

In a real implementation, the resulting normalization gain, compensation gain, or a combination thereof may be limited to avoid large gains resulting in severe flattening of the frequency response, which can lead to audio artifacts. For example, gains can be limited to -6 to +6 dB. In addition, the gains can be smoothed in time and/or frequency (using a moving average or recursive average) to prevent abrupt changes and for the subsequent stabilization process.

The following will summarize some of the benefits and advantages of the preferred embodiments over the prior art.

Simplified (less complex) synthesis of HOA in DirAC.

More direct synthesis without full synthesis of all ambisonic components.

Reducing the number of required decorrelators and their impact on the final quality.

Reduce coding artifacts introduced into the downmix signal during transmission.

Separation of processing into three different orders to achieve the optimal compromise between quality and processing efficiency.

The following summarizes some aspects of the invention, partly or wholly included in the above description, which can be used independently of each other or in combination with each other, or only in a certain combination, combining only two arbitrarily selected aspects from the three mentioned aspects.

First Aspect: Energy Compensation for Diffuse Sound Components

The present invention is based on the fact that when a sound field description is generated from an input signal containing one or more signal components, the input signal can be analyzed to obtain at least diffuseness data for the sound field represented by the input signal. The analysis of the input signal may be the extraction of diffuseness data associated as metadata with one or more signal components, or the analysis of the input signal may be the analysis of a real signal when, for example, the input signal has two, three, or even more signal components, such as a complete a first-order representation, such as a B-format representation or an A-format representation.

There is now a sound component generator that generates one or more first group sound field components that have a direct component and a diffuse component. And, additionally, one or more second group sound field components are generated, wherein for such a second group, the sound field component has only direct components.

Unlike generating a full sound field, this will result in an error in energy provided that the diffuseness value for the current frame or current time/frequency slice under consideration has a value other than zero.

To compensate for this energy error, energy compensation is performed when generating the first group of sound field components. This energy compensation depends on the diffuseness data and the number of sound field components in the second group, representing the energy loss due to the lack of synthesis of diffuse components for this second group.

In one embodiment, the audio component generator for the first group may be the low order branch of FIG. 4 which extracts the sound field components of the first group by copying or performing a weighted addition, i. e. without estimating the complex spatial basis function. Thus, the sound field component of the first group is not available separately as a direct part and a diffuse part. However, an increase in the entire sound field component of the first group with respect to its energy automatically increases the energy of the diffuse part.

Alternatively, the sound component generator for one or more first group sound field components may also be a medium order branch of FIG. 4, relying on separate synthesis of the direct part and synthesis of the diffuse part. Here we have the diffuse portion separately available, and in one embodiment the diffuse portion of the sound field component is increased, but not the direct portion, to compensate for the energy loss due to the second group. Alternatively, however, in this case it would be possible to increase the energy of the resulting sound field component after combining the direct part and the diffuse part.

Alternatively, the audio component generator for one or more first group sound field components can also be branches of the low and medium order components of FIG. 4. Energy compensation can then only be applied to low order components or to both low and medium order components.

Second aspect: Energy normalization of the direct sound component

This invention is based on the assumption that the generation of an input signal that has two or more audio components was accompanied by some kind of quantization. Typically, when considering two or more audio components, one audio component of the input signal may be an omnidirectional signal, such as omnidirectional microphone signals W in B-format representation, and other audio components may be separate directional signals, such as X signals, Y, Z of a figure-eight microphone in B-format representation, that is, in first-order ambisonic representation.

When a signal encoder is in a situation where the bitrate requirements are too high for an ideal encoding operation, then the typical procedure is that the encoder encodes the omnidirectional signal as accurately as possible, but the encoder only spends fewer bits on the directional components, which may even be as low as low that one or more directional components are completely nullified. It represents such a loss or mismatch of energy in direction information.

Now, however, there may be a requirement, which, for example, is obtained by having explicit parametric ancillary information indicating that a certain frame or time/frequency bin has a certain diffuseness less than one and sound direction. Thus, a situation may arise where, according to the parametric data, there is some certain non-diffuse component with some certain direction, while, on the other hand, the transmitted omnidirectional signal and the directional signals do not reflect this direction. For example, an omnidirectional signal could be transmitted without any significant loss of information, while a directional signal, Y, responsible for the left and right directions, could be set to zero due to missing bits. In this scenario, even if the direct audio component originates from the left in the original audio scene, the transmitted signals will reflect the audio scene without any left-right directional characteristic.

Thus, in accordance with the second invention, energy normalization is performed for the direct sound components to compensate for energy imbalance with direction/diffusion data that is either explicitly included in the input signal or derived from the input signal itself.

This energy normalization can be applied in the context of all individual processing branches of FIG. 4 either completely or only individually.

This invention allows the use of additional parametric data, either received from the input signal or derived from the undisturbed portions of the input signal, and therefore coding errors included in the input signal for whatever reason can be reduced using the additional direction data and data about diffuseness derived from the input signal.

In this invention, an energy or amplitude related metric for the omnidirectional component output from the input signal and an additional energy or amplitude related metric for the directional component output from the input signal are estimated and used for energy compensation along with direction data and diffuseness data. . Such an energy- or amplitude-related metric could be the amplitude or power itself, i. e. squared and summed amplitudes, or may be energy, such as power multiplied by a certain period of time, or may be any other exponent derived from the amplitude with an exponent for the amplitude other than one, and then summed. Thus, an additional energy or amplitude related metric could also be loudness with a power of three versus power with a power of two.

Third aspect: System implementation with different processing procedures for different orders

In the third invention, which is illustrated in FIG. 4, the sound field is generated using an input signal containing a mono signal or a multi-component signal having two or more signal components. The signal analyzer derives direction data and diffuseness data from the input signal, either by explicitly analyzing the signal in case the input signal has two or more signal components, or by analyzing the input signal to extract the direction data and diffuseness data included in the input signal as metadata.

The low order component generator generates a low order audio description from the input signal up to a predefined order and performs this task for the available modes that can be extracted from the input signal by copying the signal component from the input signal or by performing a weighted combination of the components in the input signal.

The medium order component generator generates a medium order audio description having components with an order above a predefined order or with a predefined order and above a predefined mode and below or with a first cutoff order using at least one direct component synthesis and at least one diffuse component synthesis, using the direction data and the diffuseness data obtained from the analyzer, so that the sound description of the average order contains a direct contribution and a diffuse contribution.

In addition, the high order component generator generates a high order audio description having components with orders above the first cutoff and below or equal to the second cutoff order using the synthesis of at least one direct component without synthesizing any diffuse component, so that the high order audio description has only a direct contribution.

The invention of this system has significant advantages in that the generation of the most accurate low order sound field by using the information included in the input signal is performed as best as possible, while the processing operations for performing the low order sound description are low cost due to that only copy operations or weighted combination operations such as weighted additions are required. Thus, high-quality, low-order audio description is performed with a minimum amount of processing power required.

The mean order audio description requires more processing power, but allows a very accurate mean order audio description with direct and diffuse contributions to be generated using the analyzed direction data and diffuseness data, typically up to some order, i.e. high order, below which the diffuse contribution to the sound field description is still required from a perceptual point of view.

Finally, the high order component generator generates a high order audio description only by performing direct synthesis without performing diffuse synthesis. This, again, reduces the amount of processing power required due to the fact that only direct components are generated, while at the same time eliminating diffuse synthesis is not so perceptually problematic.

Naturally, the third invention can be combined with the first and/or the second invention, but even when for some reason compensation for not performing diffuse synthesis by the high-order component generator is not applied, the procedure nevertheless results in an optimal compromise between processing power, with on the one hand, and sound quality on the other. The same is true for performing low order energy normalization compensating for the coding used to generate the input signal. In an embodiment, this compensation is performed additionally, but even without this compensation, significant non-trivial advantages are obtained.

Fig. 4, as a symbolic illustration of parallel transmission, shows the number of components processed by each component generator. The low order component generator 810 illustrated in FIG. 4 generates a low order sound field description from an input signal up to a predetermined order and a predetermined mode, wherein the low order component generator 810 is configured to derive the low order sound field description by copying or taking the input signal as is or by performing a weighted combination of input channels. signal. As illustrated between the low order component generator block and the low order component mixing block, the K individual components are processed by this low order component generator 810 . The average order component generator 820 generates a reference signal, and as an exemplary situation, it is indicated that an omnidirectional signal is used included in the downmix signal at the input or output of the filter bank 400. However, when the input signal has a left channel and a right channel, the mono signal obtained by adding the left and right channels is calculated by the reference signal generator 821 . In addition, the number of (L+1) ² - K components are generated by the average order component generator. In addition, the high-order component generator generates a number of (H+1) ² - (L+1) ² components, so that at the end, at the output of the combiner, there are (H+1) ² components from one or more (small number) components at the entrance to the block 400 filters. The splitter is configured to provide separate direction/diffusion data to the respective component generators 810, 820, 830. Thus, the low order component generator takes in K data elements. This is indicated by a line connecting separator 420 and the low order mixing unit.

In addition, the middle order component mixer 825 receives (L+1) ² - K data elements, and the high order component mixer receives (H+1) ² - (L+1) ² data elements. Accordingly, the individual component mixers provide a certain number of sound field components to the combiner 430.

Further, a preferred implementation of the low order component generator 810 of FIG. 4 is illustrated with reference to FIG. 9. An input signal is input to an input signal examining block 811, and this input signal examining block 811 passes the received information to a processing mode selection block 812. The processing mode selection block 812 is configured to select a plurality of different processing modes, which are schematically illustrated as a copy block 813, indicated by 1, a take (as is) block 814, indicated by 2, a linear combination (first mode) block, indicated by 3, and 815, and a linear combination (second mode) block 816, denoted by 4. For example, when the input signal examining block 811 determines a certain kind of input signal, then the processing mode selection block 812 selects one of the aforementioned plurality of different processing modes, as shown in table in Fig. 9. For example, when the input signal is an omnidirectional signal W or a mono signal, then copy 813 or take 814 is selected. However, when the input signal is a stereo signal with a left channel or a right channel, or a multi-channel signal with 5.1 or 7.1 channels, then the linear combination block 815 is selected to output an omnidirectional signal W from the input signal by adding the left and right, and by calculating the directional component by calculating the difference between left and right.

However, when the input signal is a combined stereo signal, i.e., a mid/side ("mid/side") representation, then either block 813 or block 814 is selected, since the middle signal already represents the omnidirectional signal, and the side signal already represents the directional component.

Similarly, when it is determined that the input signal is a first order ambisonic (FOA) signal, then the mode select block 812 selects either block 813 or block 814. However, when it is determined that the input signal is an A-format signal, then linear block 816 combination (second mode) is selected to perform a linear transformation on the A-format signal to obtain a first-order ambisonic signal having an omnidirectional component and tri-directional components representing the low-order K component blocks generated by block 810 in FIG. 8 or Fig. 6. In addition, in FIG. 9 illustrates an energy compensator 900 that is configured to perform energy compensation for output from one of blocks 813 to 816 to perform diffuse compensation and/or direct compensation with appropriate gain values g and g _s .

Therefore, the implementation of the energy compensator 900 corresponds to the procedure of the audio component generator 650 or the audio component generator 750 of FIG. 6 and FIG. 7, respectively.

Fig. 10 illustrates a preferred implementation of the midrange component generator 820 of FIG. 8 or parts of the generator 650 sound components for the down arrow with direct/diffuse (components) from block 650 belonging to the first group. Specifically, the mid-range component generator 820 includes a reference signal generator 821 that receives an input signal and generates a reference signal by copying or taking it as is when the input signal is a mono signal, or by deriving the reference signal from the input signal by performing the calculation as described above, or to that presented in the application WO 2017/157803 A1, the contents of which are incorporated herein by this link in their entirety.

In addition, FIG. 10 illustrates a directional gain calculator 410 that is configured to calculate a directional gain G _l ^m from certain DOA information (Φ,θ) and from a certain mode number m and a certain order number l . In a preferred embodiment, when processing is performed in the time/frequency domain for each individual chunk, which is denoted by k, n, a directional gain is computed for each such time/frequency chunk. The weight processor 820 receives the reference signal and the diffuseness data for a specific time/frequency slice, and the result of the weight processor 820 is a forward portion. The diffuse part is generated by the processing performed by the decorrelation filter 823 and the subsequent weight processing block 824 taking the diffuseness value Ψ for a certain time frame and frequency bin and specifically taking the average response for a certain mode m and order l indicated as D _l and generated by block 826 to provide an average answer, which takes as input the desired mode m and the required order l .

The result of weight processing 824 is a diffuse part, and this diffuse part is added to the direct part by adder 825 in order to obtain a specific average order sound field component for a specific mode m and a specific order l . Preferably, the diffuse compensation gain described with reference to FIG. 6, only to the diffuse part generated by block 823. This can advantageously be done within the procedure performed by the weight processing (diffuse components) block. Thus, only the diffuse portion in the signal is amplified to compensate for the loss of diffuse energy incurred by higher components that do not receive full synthesis, as illustrated in FIG. ten.

The generation of only the straight part is illustrated in FIG. 11 for the high order component generator. Basically, the high order component generator is implemented in the same way as the medium order component generator with respect to the direct branch, but does not contain blocks 823, 824, 825, and 826. Thus, the high order component generator contains only a weight processing block 822 (direct component) receiving input from the directional gain calculator 410 and receiving a reference signal from the reference signal generator 821. Preferably, only one reference signal is generated for the high order component generator and the medium order component generator. However, both units can also have individual reference generators, depending on the circumstances. However, it is preferable to have only one reference signal generator. Thus, the processing performed by the high-order component generator is extremely efficient, since only one weighting operation with a specific directional gain G _l ^m with specific diffuseness information Ψ needs to be performed for a time/frequency slice. In this way, high order sound field components can be generated extremely efficiently and quickly, and any error associated with not generating diffuse components or not using diffuse components in the output signal is easily compensated for by amplifying the low order sound field components, or preferably only the diffuse part of the components. sound field of medium order.

Normally the diffuse part will not be available separately in the low order sound field components generated by copying or performing a (weighted) linear combination. However, an increase in the energy of such components automatically increases the energy of the diffuse part. As the inventors found out, the simultaneous increase in the energy of the direct part is not a problem.

Further reference is made to Figures 12a to 12c in order to further illustrate the calculation of individual compensation gains.

Fig. 12a illustrates a preferred implementation of the audio component generator 650 of FIG. 6. Compensation gain (diffuse) is calculated in one embodiment using diffuseness value, maximum order H, and cutoff order L. In another embodiment, the diffuse compensation gain is calculated using the parameter L _k derived from the number of components in the low order processing branch 810. In addition, the parameter mk is used depending on the parameter lk and the number K of components actually generated by the low-order component generator. In addition, the value of N, depending on L _k , is also used. Both H, L in the first embodiment or H, Lk, mk generally represent the number of sound field components in the second group (related to the number of sound components in the first group). Thus, the more components for which the diffuse component is not synthesized, the higher the compensatory energy gain will be. On the other hand, the greater the number of low-order sound field components that can be compensated, i.e. multiplied by the gain, the lower this gain may be. Typically, the gain g will always be greater than 1.

Fig. 12a illustrates the calculation of the gain factor g by the (diffuse) equalization gain calculator 910 and then applying this gain to the (low order) component to be "adjusted", as done by the equalization gain application block 900. In the case of linear numbers, the compensation gain block will be a multiplier, and in the case of logarithmic numbers, the compensation gain block will be an adder. However, other implementations of applying the compensation gain may be implemented depending on the specific nature and manner in which the compensation gain is calculated by block 910. Thus, the gain need not be a multiplicative gain, but may also be any other gain.

Fig. 12b illustrates a third implementation of (direct) equalization gain processing. The (forward) compensation gain calculator 920 takes as input an energy or amplitude related metric for the omnidirectional component, referred to as "omnidirectional power" in FIG. 12b. In addition, a second energy or amplitude related metric for the directional component is also entered in block 920 as "directional power". In addition, direct gain compensation calculator 920 further receives QL information or, alternatively, N information. N is (2l+1), which is the number of coefficients per order of _l , and Ql is 1/N. In addition, a directional gain G _l ^m for some time/frequency slice (k, n) is also required to calculate the (direct) compensation gain. The directional gain is the same data that was output by the directional gain calculator 410 in FIG. 4. The (direct) equalization gain g _s is passed from block 920 to the equalization gain application block 900, which can be implemented similarly to block 900, i. takes the component(s) to be "corrected" and outputs the corrected component(s).

Fig. 12c illustrates a preferred implementation of a combination of diffuse sound component energy compensation and direct sound component compensation energy normalization to be performed together. To this end, a (diffuse) equalization gain g and a (direct) equalization gain g _s are introduced into a gain combiner 930 . The result of the gain combiner (i.e., the combined gain) is input to a gain keyer 940, which is implemented as a post-processor and performs a minimum or maximum limit, or which applies a compression function to perform some kind of softer limiting, or performs smoothing of temporal or frequency fragments. The manipulated gain that is clipped, compressed, or smoothed or processed in other post-processing techniques is then applied by the gain applicator to the low order component(s) to obtain the corrected low order component(s).

In the case of linear gains g, g _s , the gain combiner 930 is implemented as a multiplier. In the case of logarithmic gains, the gain combiner is implemented as an adder. In addition, with regard to the implementation of the estimator of FIG. 7 indicated at 620, it is shown that estimator 620 can provide any energy or amplitude related metrics for the omnidirectional and directional components as long as the power applied to the amplitude is greater than 1. In the case of power as energy related or amplitude of the exponent, the exponent is 2. However, exponents of 1.5 to 2.5 can also be useful. In addition, even higher exponents or exponents are useful, such as a power of 3 applied to the amplitude corresponding to the loudness value rather than the power value. Thus, powers of 2 or 3 are generally preferred for energy or amplitude related performance, but powers of 1.5 to 4 are also generally preferred.

The following is a summary of several examples of aspects of the invention.

Basic example 1a for the first aspect (energy compensation for diffuse sound components)

1a. A device for generating a sound field description from an input signal containing one or more channels, the device comprising:

an input signal analyzer for obtaining diffuseness data from the input signal;

an audio component generator for generating from the input signal one or more sound field components of the first group of sound field components having a direct component and a diffuse component for each sound field component, and for generating from the input signal of the second group of sound field components having only a direct component,

wherein the sound component generator is configured to perform energy compensation when generating the first group of sound field components, wherein the energy compensation depends on diffuseness data and the number of sound field components in the second group.

Basic example 1b for the second aspect (energy normalization for direct signal components)

1b. A device for generating a sound field description from an input signal containing at least two channels, the device comprising:

an input signal analyzer for obtaining direction data and diffuseness data from the input signal;

an estimator for estimating a first amplitude-related metric for the omnidirectional component derived from the input signal and for estimating a second amplitude-related metric for the directional component derived from the input signal, and

an audio component generator for generating sound field components for the sound field, the audio component generator being configured to perform directional component energy compensation using the first amplitude related metric, the second amplitude related metric, direction data, and diffuseness data.

Basic example 1c for the third aspect: Implementation of a system with branches of different generators

1c. A device for generating a sound field description using an input signal containing a mono signal or a multi-channel signal, the device comprising:

an input signal analyzer for analyzing the input signal for deriving direction data and diffuseness data;

a low order component generator for generating a low order audio description from the input signal up to a predetermined order and mode, the low order component generator being configured to derive the low order audio description by copying the input signal or by performing a weighted channel combination of the input signal;

a mean order component generator for generating a mean order audio description that is above a predetermined order or with a predetermined order and above a predetermined mode and below or with a first cutoff order using synthesis of at least one direct component and at least one diffuse component using data about the direction and diffuseness data, so that the sound description of the average order contains a direct contribution and a diffuse contribution; and

a high order component generator for generating a high order audio description having a component that is above the first cutoff order using synthesis of at least one direct component without synthesizing any diffuse component such that the high order audio description has only a direct contribution.

2. Device according to examples 1a, 1b, 1c,

in which the low order audio description, the medium order audio description, or the high order audio description contain sound field components of the output sound field that are orthogonal such that any two audio descriptions do not contain the same sound field components, or

wherein the medium order component generator generates components below or with a first cutoff order not used by the low order component generator.

3. The device according to one of the previous examples, containing:

receiving a downmix input signal having one or more audio channels that represent the sound field

receiving or determining one or more sound directions that represent the sound field;

evaluating one or more spatial basis functions using one or more sound directions;

deriving a first set of one or more sound field components from the first weighted channel combination of the downmix input signal.

deriving a second set of one or more direct sound field components from a second weighted combination of downmix input signal channels and one or more estimated spatial basis functions.

combining the first set of one or more sound field components and the second set of one or more sound field components.

4. The device according to one of the preceding examples, wherein the first and second sets of sound field components are orthogonal.

5. The device according to one of the preceding examples, wherein the sound field components are coefficients of orthogonal basis functions.

6. The device according to one of the preceding examples, wherein the sound field components are the coefficients of the spatial basis functions.

7. An arrangement according to one of the preceding examples, wherein the sound field components are spherical or circular harmonic coefficients.

8. The device according to one of the preceding examples, wherein the sound field components are ambisonic coefficients.

9. Apparatus as in one of the preceding examples, wherein the downmix input signal has less than three audio channels.

10. The device according to one of the preceding examples, further comprising:

receiving or determining the value of diffuseness;

generating one or more diffuse sound components depending on the diffuse value; and

combining one or more diffuse sound components with a second set of one or more direct sound field components;

11. The apparatus of one of the preceding examples, wherein the diffuse component generator further comprises a decorrelator for decorrelating diffuse sound information.

12. The apparatus of one of the preceding examples, wherein a first set of one or more sound field components are derived from a diffuseness value.

13. The apparatus of one of the preceding examples, wherein a first set of one or more sound field components are output from one or more directions of sound.

14. An apparatus according to one of the preceding examples, which derives frequency and time dependent sound directions.

15. An apparatus according to one of the preceding examples which derives frequency and time dependent diffuseness values.

16. The apparatus of one of the preceding examples, further comprising: decomposing a plurality of time domain downmix signal channels into a frequency representation having a plurality of time-frequency slices.

17. A method for generating a sound field description from an input signal containing one or more channels, comprising:

obtaining diffuseness data from the input signal;

generating from the input signal one or more sound field components of the first group of sound field components having for each sound field component a direct component and a diffuse component, and generating from the input signal a second group of sound field components having only a direct component,

wherein the generation comprises performing an energy compensation when generating the first group of sound field components, wherein the energy compensation depends on the diffuseness data and the number of sound field components in the second group.

18. A method for generating a sound field description from an input signal containing at least two channels, comprising:

obtaining direction data and diffuseness data from the input signal;

estimating a first amplitude-related metric for the omnidirectional component derived from the input signal and estimating a second amplitude-related metric for the directional component derived from the input signal, and

generating sound field components for the sound field, wherein the sound component generator is configured to perform directional component energy compensation using the first amplitude related metric, the second amplitude related metric, direction data, and diffuseness data.

19. A method for generating a sound field description using an input signal containing a mono signal or a multi-channel signal, the method comprising:

analyzing the input signal to derive direction data and diffuseness data;

generating a low order audio description from the input signal up to a predetermined order and mode, wherein the low order generator is configured to derive the low order audio description by copying the input signal or performing a weighted channel combination of the input signal;

generating a medium order audio description that is above a predetermined order or with a predetermined order and above a predetermined mode and below a high order using synthesis of at least one direct part and at least one diffuse part using direction data and diffuseness data, so that the sound description of the mean order contains a direct contribution and a diffuse contribution; and

generating a high order audio description having a component that is at or above a high order using synthesis of at least one direct part without synthesizing any diffuse component such that the high order audio description contains only the direct contribution.

20. Computer program for execution when running on a computer or processor of the method according to one of examples 17, 18 or 19.

It should be mentioned here that all alternatives or aspects that are described above, as well as all aspects defined by independent claims in the following claims, can be used individually, that is, without any other alternative or subject, other than the intended alternative, subject or independent claim . However, in other embodiments, two or more alternatives or aspects or independent claims may be combined with each other, and in other embodiments, all aspects or alternatives and all independent claims may be combined with each other.

The audio signal encoded according to the invention may be stored on a digital storage medium or a non-volatile storage medium, or may be transmitted over a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.

Although some aspects have been described in the context of a device, it is clear that these aspects also represent a description of the corresponding method, with the block or device corresponding to a method step or a feature of a method step. Likewise, aspects described in the context of a method step are also descriptions of the respective block, element, or feature of the respective device.

Depending on certain implementation requirements, embodiments of the invention may be implemented in hardware or software. The implementation may be carried out using a digital storage medium, such as a floppy disk, DVD, CD, ROM, PROM, EPROM, EEPROM, or FLASH memory, which stores electronically readable control signals that interact with (or are capable of interacting with) a programmable computer. system to execute the appropriate method.

Some embodiments of the invention comprise a storage medium having electronically readable control signals that are capable of interfacing with a programmable computer system such that one of the methods described herein is performed.

Typically, embodiments of the present invention may be implemented as a computer program product with program code, the program code being operable to perform one of the methods when the computer program product is running on the computer. The program code may, for example, be stored on a computer-readable medium.

Other embodiments include a computer program for performing one of the methods described herein, stored on a computer-readable medium or non-volatile storage medium.

In other words, an embodiment of the method of the present invention is a computer program having program code for performing one of the methods described herein when the computer program is executed on a computer.

Therefore, a further embodiment of the methods of the invention is a storage medium (either a digital storage medium or a computer-readable medium) containing a computer program recorded thereon for performing one of the methods described herein.

Thus, a further embodiment of the method of the present invention is a data stream or sequence of signals representing a computer program for performing one of the methods described herein. The data stream or sequence of signals may, for example, be configured to be transmitted over a data connection, such as the Internet.

An additional embodiment comprises a processing means, such as a computer or a programmable logic device, configured or adapted to perform one of the methods described herein.

Yet another embodiment includes a computer on which a computer program is installed to perform one of the methods described herein.

In some embodiments, a programmable logic device (eg, a field programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, a user-programmable gate array may communicate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed using any hardware device.

The above embodiments are merely illustrative of the principles of the present invention. It is understood that modifications and changes to the arrangements and details described herein will be apparent to others skilled in the art. Thus, it is intended to be limited only by the scope of the appended claims, and not by the specific details provided as a description and explanation of the embodiments herein.

Information sources

[1] V. Pulkki, M-V Laitinen, J Vilkamo, J Ahonen, T Lokki and T Pihlajamäki, Directional audio coding - perception-based reproduction of spatial sound, International Workshop on the Principles and Application on Spatial Hearing, Nov. 2009, Zao; Miyagi Japan.

[2] M.V. Laitinen and V. Pulkki, Converting 5.1 audio recordings to B-format for directional audio coding reproduction, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Prague, 2011, pp. 61-64.

[3] R.K. Furness, Ambisonics - An overview, in AES 8th International Conference, April 1990, pp. 181-189.

[4] C. Nachbar, F. Zotter, E. Deleflie, and A. Sontacchi, AMBIX - A Suggested Ambisonics Format, Proceedings of the Ambisonics Symposium 2011.

[5] APPARATUS, METHOD OR COMPUTER PROGRAM FOR GENERATING A SOUND FIELD DESCRIPTION (corresponding to WO 2017/157803 A1). As in the case of diffuse compensation.

Claims (54)

1. A device for generating a sound field description using an input signal containing a mono signal or a multi-channel signal, the device comprising: an input signal analyzer (600) for analyzing the input signal to derive direction data and diffuseness data; a low order component generator (810) for generating a low order sound field description from an input signal up to a predetermined order and mode, wherein the low order component generator is configured to derive a low order sound field description by copying or taking the input signal or performing a weighted combination of channels input signal; a medium order component generator (820) for generating a medium order sound field description above said predetermined order or with said predetermined order and above a predetermined mode and below or with a first cutoff order using synthesis of at least one direct part and at least one diffuse part , using direction data and diffuseness data, so that the mean-order sound field description contains a direct contribution and a diffuse contribution; and a high order component generator (830) for generating a high order sound field description having a component of the above mentioned first cutoff order using synthesis of at least one direct part, wherein the high order sound field description contains only a direct contribution. 2. The apparatus of claim 1, wherein the input signal analyzer (600) is configured to extract diffuseness data from metadata associated with the input signal or to extract diffuseness data from the input signal by analyzing (610, 620) the signal itself. an input signal having two or more channels or component. 3. The device according to claim 1, wherein the low order component generator (810) is configured to generate K low order sound field components, wherein K is 1 or greater than 1 and depends on a predetermined order and a predetermined mode, wherein the mean order component generator (820) is configured to generate (L+1)2 - K mean order sound field components, L is greater than K and depends on the cutoff order and wherein the high order component generator (830) is configured to generate (H+1)2 - (L+1)2 high order sound field components, wherein H is an integer greater than L, and wherein H is the maximum order of the generated sound field description. 4. The apparatus of claim 1, further comprising a sound field combiner (430) for combining low order sound field components, a medium order sound field component, and a high order sound field component to produce a combined sound field having an (H+1)2 sound field component. field, with H being the maximum order of the generated sound field description. 5. The apparatus of claim 1, further comprising a directional gain calculator (410) for calculating directional gains for medium order and high order sound field components from direction data and/or diffuseness data and for supplying directional gains for medium order sound field components to the medium order component generator (820); and to provide directional gains for the high order sound field components to the high order component generator (830). 6. The apparatus of claim 5, wherein the directional gain calculator is configured to receiving, for each time-frequency slice of a plurality of time-frequency slices, direction related data and/or diffuseness data, and estimating the spatial basis function for a specific order and mode using direction-related data to obtain a directional gain for that specific order and mode. 7. The apparatus of claim 1, wherein the average order component generator (820) comprises: a reference signal generator (821) for providing a reference signal for the sound field component of the medium order sound field description and for modifying (822) this reference signal using direction related data; a decorrelator (823, 824) for decorrelating the reference signal or a signal derived from the reference signal to obtain a decorrelated signal; and mixer (824, 825) for mixing the modified reference signal and the decorrelated signal using diffuseness data. 8. Device according to claim 1, wherein the input signal contains a mono signal, and wherein the low order component generator (810) is configured to generate an ambisonic zero order signal by taking or copying that mono signal (813, 814), or wherein the input signal contains at least two channels, and in which the low-order component generator (810) is configured to generate a zero-order ambisonic signal by adding these two channels and generating a first-order ambisonic signal based on the difference of these two channels (815), or wherein the input signal contains a first order ambisonic signal with three or four channels, and in which the low order component generator (810) is configured to generate a first order ambisonic signal by taking or copying these three or four channels of the input signal (813, 814), or wherein the input signal comprises an A-format signal having four channels, and wherein the low order component generator (810) is configured to compute a first order ambisonic signal by performing a weighted linear combination of the four channels (816). 9. The apparatus of claim 1, wherein the high order component generator (830) is configured to only synthesize the direct portion for the high order sound field description component using direction data. 10. The apparatus of claim 9, further comprising a directional gain calculator (410) for the sound field component of the high order sound field description, configured to estimate a spatial basis function for the order and mode of said high order sound field description component, and the generator (830) the high order component is configured to derive the sound field component from the reference signal and the directional gain. 11. The apparatus of claim 10, wherein the high order component generator (830) is configured to multiply (822) the reference signal by the directional gain for the order and mode of the sound field component and multiply (822) by a factor derived from diffuseness data, to get the sound field component to describe the high order sound field. 12. Device according to claim 7, wherein the decorrelator (823, 824) comprises a decorrelation filter (823) for receiving the reference signal and for outputting the filtered reference signal, and wherein the decorrelator is configured to weight (824) the filtered signal or the signal input to the decorrelator filter (823) with a weight value derived from the diffuseness data and with an additional weight value derived from the order and mode of the sound field component in question, to get the diffuse part, and wherein the mixer is configured to add (825) the direct part and the diffuse part of the same order and mode to obtain a medium order sound field component. 13. Device according to claim 1, wherein the low order component generator (810) and/or the medium order component generator (820) is configured to perform energy compensation for one or more sound field components from the low order sound field description or at least one diffuse portion of the medium order sound field description or for the sound field component from the medium order sound field description to compensate for the energy loss due to the high order sound field description having only a direct component, or wherein the sound component generator is configured to perform energy compensation for one or more sound field components from a low order sound field description or at least one diffuse portion of a medium order sound field description or for a sound field component from a medium order sound field description, wherein the compensation energy depends on the diffuseness data and at least one of the number of sound field components generated by the high order component generator (830), the number of diffuse components generated by the low order component generator (810) and/or the medium order component generator (820), maximum the order of the sound field components generated by the low order component generator (810) and/or the medium order component generator (820), and the maximum order of the sound field components generated by the high order component generator (830). 14. The apparatus of claim 1, wherein the low order component generator (810) is configured to perform energy compensation for a directional component included in or removed from the input signal by copying, taking, or performing a weighted combination. 15. The apparatus of claim 14, wherein the low order component generator comprises: an estimator (620) for estimating a first energy or amplitude related metric for the omnidirectional component derived from the input signal and for estimating a second energy or amplitude related metric for the directional component derived from the input signal; and wherein the low order component generator (810) is configured to perform directional component energy compensation using the first energy or amplitude related metric, the second energy or amplitude related metric, direction data, and diffuseness data. 16. Device according to claim 14, wherein the low-order component generator (810) comprises an energy compensator (910, 900) for performing energy compensation, wherein the energy compensator comprises a compensation gain calculator (910) for calculating the compensation gain using diffuseness data, the maximum order of the sound field components of the first group, and the maximum order of the sound field components of the second group, while the maximum order of the sound field components of the second group depends on the number of sound field components in the second group. 17. The apparatus of claim 16, wherein the high order component generator (830) is configured to multiply (822) the reference signal by the directional gain for the order and mode of the sound field component and multiply (822) by a factor derived from diffuseness data, to get the sound field component to describe the high order sound field. 18. Device according to claim 16, wherein the energy compensator (910, 920, 930, 940) comprises a compensation gain application unit (900) for applying compensation gain to at least one sound field component. 19. Device according to claim 1, wherein the first group of sound field components and the second group of sound field components are orthogonal to each other, or wherein the sound field components are at least one of orthogonal basis function coefficients, spatial basis function coefficients, spherical or circular harmonic coefficients, and ambisonic coefficients. 20. The device according to claim 1, further comprising: block (400) analysis filters for generating one or more components of the sound field of the first group and the second group for a variety of different time-frequency fragments, wherein the input signal analyzer (600) is configured to obtain a direction data element and a diffuseness data element for each time-frequency slice, and wherein the low order component generator (810), the medium order component generator (820) or the high order component generator (830), the audio component generator (650) is configured to operate separately for each time-frequency slice. 21. A method for generating a sound field description using an input signal containing a mono signal or a multi-channel signal, the method comprising: analyzing the input signal to derive direction data and diffuseness data; generating a low order sound field description from the input signal up to a predetermined order and mode, the low order generator being configured to derive a low order sound field description by copying the input signal or performing a weighted channel combination of the input signal; generating a medium order sound field description above said predetermined order or with said predetermined order and above a predetermined mode and below a high order using synthesis of at least one direct part and at least one diffuse part using direction data and diffuseness data, so that the description of the sound field of the average order contains a direct contribution and a diffuse contribution; and generating a high order sound field description having a high order component or higher using synthesis of at least one direct part without synthesizing any diffuse component such that the high order sound field description contains only the direct contribution. 22. A physical digital storage medium having a computer program stored on it for execution when running on a computer or processor of the method according to claim 21.

Images