Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04S—STEREOPHONIC SYSTEMS 
  • H04S5/00—Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation 
• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
  • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04S—STEREOPHONIC SYSTEMS 
  • H04S3/00—Systems employing more than two channels, e.g. quadraphonic
  • H04S3/002—Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
  • H04R2205/00—Details of stereophonic arrangements covered by H04R5/00 but not provided for in any of its subgroups
  • H04R2205/024—Positioning of loudspeaker enclosures for spatial sound reproduction
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04S—STEREOPHONIC SYSTEMS 
  • H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
  • H04S2400/11—Positioning of individual sound objects, e.g. moving airplane, within a sound field
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04S—STEREOPHONIC SYSTEMS 
  • H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
  • H04S2420/11—Application of ambisonics in stereophonic audio systems

Definitions

• the invention relates to a method and to an apparatus for changing the relative positions of sound objects contained within a two-dimensional or a three-dimensional Higher-Order Ambisonics representation of an audio scene.
• HOA Higher-order Ambisonics
• the disadvantage is that sophisticated and error- prone scene decomposition is mandatory.
• the content of the HOA representation can be modified via linear transformation of HOA vectors.
• rotation, mirroring, and emphasis of front/back directions have been proposed. All of these known, transformation- based modification techniques keep fixed the relative po ⁇ sitioning of objects within a scene.
• space warping For manipulating or modifying a scene's contents, space warping has been proposed, including rotation and mirroring of HOA sound fields, and modifying the dominance of specific directions :
• a problem to be solved by the invention is to facilitate the change of relative positions of sound objects contained within a HOA-based audio scene, without the need for analys ⁇ ing the composition of the scene.
• This problem is solved by the method disclosed in claim 1.
• An apparatus that utilises this method is disclosed in claim 2.
• the invention uses space warping for modifying the spatial content and/or the reproduction of sound-field information that has been captured or produced as a higher-order Ambi ⁇ sonics representation.
• Spatial warping in HOA domain represents both, a multi-step approach or, more computationally efficient, a single-step linear matrix multiplication. Different warping characteristics are feasible for 2D and 3D sound fields.
• the warping is performed in space domain without performing scene analysis or decomposition.
• Input HOA coefficients with a given order are decoded to the weights or input signals of regularly positioned (virtual) loudspeakers.
• the inventive space warping processing has several advan ⁇ tages :
• the inventive method is suited for changing the relative positions of sound objects contained within a two-dimensional or a three-dimensional Higher-Order Ambison- ics HOA representation of an audio scene, wherein an input vector A; n with dimension 0[ n determines the coefficients of a Fourier series of the input signal and an output vector A out with dimension O out determines the coefficients of a Fourier series of the correspondingly changed output signal, said method including the steps :
• the inventive apparatus is suited for changing the relative positions of sound objects contained within a two-dimensional or a three-dimensional Higher-Order Ambison- ics HOA representation of an audio scene, wherein an input vector A; n with dimension 0[ n determines the coefficients of a Fourier series of the input signal and an output vector A out with dimension O out determines the coefficients of a Fourier series of the correspondingly changed output signal, said apparatus including:
• Fig. 1 principle of warping in space domain
• ATM CTM j n (kr) .
• N (N + l) 2 .
• the HOA 'signal' comprises a vector A of Ambisonics coeffi ⁇ cients for each time instant.
• a of Ambisonics coeffi ⁇ cients for each time instant.
• ⁇ 3D ( ⁇ , ⁇ ⁇ ⁇ , ⁇ , ⁇ , ⁇ 2 2 ,..; ⁇ / ) ' ⁇ . (3)
• HOA representations behaves in a linear way and therefore the HOA coefficients for multiple, separate sound objects can be summed up in order to derive the HOA coefficients of the resulting sound field.
• Plain encoding of multiple sound objects from several direc- tions can be accomplished straight-forwardly in vector alge ⁇ bra.
• the i-th column of ⁇ contains the mode vector according to the direc ⁇ tion ⁇ of the i-th sound object
• encoding of a HOA representation can be interpreted as a space-frequency transformation because the input signals (sound objects) are spatially distributed.
• the conditions for re ⁇ versibility are that the mode matrix ⁇ must be square (Ox 0) and invertible.
• the driver signals of real or virtual loud ⁇ speakers are derived that have to be applied in order to precisely play back the desired sound field as described by the input HOA coefficients.
• Such decoding depends on the number M and positions of loudspeakers.
• the three following important cases have to be distinguished (remark: these cases are simplified in the sense that they are defined via the 'number of loudspeakers', assuming that these are set up in a geometrically reasonable manner. More precisely, the definition should be done via the rank of the mode matrix of the targeted loudspeaker setup) .
• the mode matching decoding principle is applied, but other decoding principles can be utilised which may lead to different decoding rules for the three strigr ⁇ ios.
• the number of loudspeakers is higher than the number of HOA coefficients, i.e. M> 0.
• M the number of HOA coefficients
• no unique solution to the decoding problem exists, but a range of admissible solutions exist that are lo ⁇ cated in an M— O-dimensional sub-space of the M- dimensional space of all potential solutions.
• This solution delivers the loudspeaker signals with the minimal gross playback power s T s (see e.g. L.L.Scharf, "Statistical Signal Processing.
• the mathematical problem of decoding the sound field is un ⁇ derdetermined and no unique, precise solution exists.
• numerical optimisation has to be used for deter ⁇ mining loudspeaker signals that best possibly match the desired sound field. Regularisation can be applied in order to derive a stable solution, for example by the formula
• ⁇ ⁇ ⁇ ( ⁇ ⁇ ⁇ + AI) _1 A , (8) wherein I denotes the identity matrix and the scalar fac- tor ⁇ defines the amount of regularisation .
• ⁇ can be set to the average of the eigenvalues of ⁇ ⁇ ⁇ .
• the resulting beam patterns may be sub-optimal because in general the beam patterns obtained with this approach are overly directional, and a lot of sound information will be underrepresented .
• Fig. la The principle of the inventive space warping is illustrated in Fig. la.
• the warping is performed in space domain.
• ⁇ fore, first the input HOA coefficients A; n with order N; n and dimension 0[ n are decoded in step/stage 12 to the weights or input signals Sj n for regularly positioned (virtual) loud ⁇ speakers.
• a determined decoder i.e. one for which the number O warp of virtual loudspeakers is equal to or larger than the number of HOA coefficients 0[ n .
• the order or dimension of the vector A; n of HOA coefficients can easily be extended by add ⁇ ing in step/stage 11 zero coefficients for higher orders.
• the dimension of the target vector Sj n will be denoted by
• the positions of the virtual loudspeakers are modified in the 'warp' processing according to the desired warping characteristics. That warp processing is in step/stage 14 combined with encoding the target vector S jn (or s out , respectively) using mode matrix ⁇ 2 , resulting in vector Ao Ut of warped HOA coefficients with dimension O warp or, following a further processing step described below, with dimension O 0 ut -
• the aforementioned modification of the loudspeaker density can be countered by applying a gain function g((p) to the virtual loudspeaker output signals Sj n in weighting step/ stage 13, resulting in signal s out .
• any weight- ing function g((p) can be specified.
• One particular advanta ⁇ geous variant has been determined empirically to be propor ⁇ tional to the derivative of the warping function " ( ⁇ ) :
• weighting function can be used, e.g. in order to obtain an equal power per opening angle.
• step/stage 14 the weighted virtual loudspeaker signals are warped and encoded again with the mode matrix ⁇ 2 by performing ⁇ 2 ⁇ ⁇ 1; . ⁇ 2 comprises different mode vectors than ⁇ ⁇ according to the warping function ( ⁇ ) .
• the result is an 0 warp -dimension HOA representation of the warped sound field .
• this stripping operation can be described by a windowing operation: the encoded vector ⁇ 2 s out is multiplied with a window vector w which comprises zero coefficients for the highest orders that shall be removed, which multiplication can be considered as representing a further weighting.
• a rectangular window can be applied, however, more sophisticated windows can be used as described in section 3 of M.A. Poletti, "A
• Space warping has its maximum impact for sound objects on the equator, while it has the lowest impact to sound objects at the poles of the sphere.
• the angular distance c of two points A and B can be deter ⁇ mined by the cosine rule of spherical geometry, cf .
• the weighting function is the product of the two weighting functions in ⁇ -direction and in ⁇ -direction
• this sequence of operations can be replaced by multiplication of the input HOA coefficients with a single matrix in step/stage 16 as depicted in Fig. lb.
• the full O warp x O warp transformation matrix T is determined as
• T diag(w) ⁇ 2 diag(g) ⁇ 1 , ( 2 4 )
• diag( -) denotes a diagonal matrix which has the values of its vector argument as components of the main diagonal
• g is the weighting function
• w is the window vector for preparing the stripping described above, i.e., from the two functions of weighting for preparing the stripping and the coefficients-stripping itself carried out in step/stage 15
• window vector w in equation ( 2 4 ) serves only for the weighting .
• the two adaptions of orders within the multi-step approach i.e. the extension of the order preceding the decoder and the stripping of HOA coefficients after encoding, can also be integrated into the transformation matrix T by removing the corresponding columns and/or lines.
• a matrix of the size O out x 0[ n is derived which directly can be applied to the input HOA vectors.
• Rotations and mirroring of a sound field can be considered as 'simple' sub-categories of space warping.
• the special characteristic of these transforms is that the relative po ⁇ sition of sound objects with respect to each other is not modified. This means, a sound object that has been located e.g. 30° to the right of another sound object in the original sound scene will stay 30° to right of the same sound object in the rotated sound scene. For mirroring, only the sign changes but the angular distances remain the same.
• all warping matrices for rotation and/or mirroring operations have the special characteristics that only coefficients of the same order n are affecting each other. Therefore these warping matrices are very sparsely populated, and the output N out can be equal to the input or ⁇ der Nj n without loosing any spatial information.
• Fig. 2 illustrates an example of space warping in the two- dimensional (circular) case.
• the warping function has been chosen to ( ⁇ ) (27)
• the warping function is shown in Fig. 2a. This particular warping function " ( ⁇ ) has been selected because it guarantees a 2n:-periodic warping function while it allows to modify the amount of spatial distortion with a single parameter a.
• Fig. 2c depicts the 7x25 single-step transformation warping matrix T.
• the logarithmic absolute values of individual co ⁇ efficients of the matrix are indicated by the gray scale or shading types according to the attached gray scale or shad- ing bar.
• a very useful characteristic of this particular warping ma ⁇ trix is that large portions of it are zero. This allows to save a lot of computational power when implementing this op- eration, but it is not a general rule that certain portions of a single-step transformation matrix are zero.
• Fig. 2d and Fig. 2e illustrate the warping characteristics at the example of beam patterns produced by some plane waves. Both figures result from the same seven input plane waves at ⁇ positions 0 , 2/ 7 ⁇ , 4/ 7 ⁇ , 6/ 7 ⁇ , 8/ 7 ⁇ , 10/ 7 ⁇ and 12/ 7 ⁇ , all with identical amplitude of one, and show the seven angular amplitude distributions, i.e. the result vec ⁇ tor s of the following overdetermined, regular decoding operation
• HOA vector A is either the original or the warped variant of the set of plane waves.
• the numbers outside the circle represent the angle ⁇ .
• the number (e.g. 360) of vir ⁇ tual loudspeakers is considerably higher than the number of HOA parameters.
• Fig. 2e shows the amplitude distributions for the same sound objects, but after the warping operation has been performed.
• the beam patterns have become asymetric due to the large gradi ⁇ ent of the Fig. 2b weighting function g((p) for these angles.
• the warping steps introduced above are rather generic and very flexible. At least the following basic operations can be accomplished: rotation and/or mirroring along arbitrary axes and/or planes, spatial distortion with a continuous warping function, and weighting of specific directions (spa ⁇ tial beamforming) .
• the space warping transformation is not space-invariant. This means that the operation be ⁇ haves differently for sound objects that are originally lo ⁇ cated at different positions on the hemisphere. In mathe- matical terms, this property is the result of the non-line ⁇ arity of the warping function f(0), i.e. f(0 + a) ⁇ f(0) + a (30) for at least some arbitrary angles ⁇ £]0...2 ⁇ [ .
• the transformation matrix T cannot be simply reversed by mathematical inversion.
• T normally is not square. Even a square space warping matrix will not be reversible because information that is typically spread from lower-order coefficients to higher-order coeffi ⁇ cients will be lost (compare section How to set the HOA or ⁇ ders and the example in section Example) , and loosing infor ⁇ mation in an operation means that the operation cannot be reversed.
• the reverse warping transformation T rev can be designed via the reverse function rev (") of the warping function " ( ⁇ ) for which
• HOA orders An important aspect to be taken into account when designing a space warping transformation are HOA orders. While, normally, the order N; n of the input vectors A; n are predefined by external constraints, both the order N 0 ut °f the output vectors A out and the 'inner' order N war p of the actual non- linear warping operation can be assigned more or less arbitrarily. However, that both orders Ni n and N warp have to be chosen with care as explained below.
• the 'inner' order N warp defines the precision of the actual decoding, warping and encoding steps in the multi-step space warping processing described above.
• the order N warp defines the precision of the actual decoding, warping and encoding steps in the multi-step space warping processing described above.
• FIG. 3 shows an example of the full warping matrix for the same warping function as used for the example from Fig. 2.
• Figures 3a, 3c and 3e depict the warp ⁇ ing functions ⁇ ) , ⁇ 2 ( ⁇ ) and f 3 (0), respectively.
• Figures 3b, 3d and 3f depict the warping matrices T ⁇ dB), T 2 (dB) and
• FIG. 3d Another scenario is shown in Fig. 3d.
• the inner order has been specified to be equal to the output order, i.e.
• the output order has to be larger than the input order N; n in order to retain all information that is spread to coefficients of different orders.
• the actual required size depends as well on the characteristics of the warping function. As a rule of thumb, the less
• the warping function ( ⁇ ) the smaller the re ⁇ quired output order. It appears that in some cases the warping function can be low-pass filtered in order to limit the required output order N 0 ut ⁇
• the output HOA coefficients will be used for a processing or a device which are capable of han- dling a limited order only.
• the target may be a loudspeaker setup with limited number of speakers.
• the output order should be specified according to the capabilities of the target system.
• the reduction of the inner order N warp to the output order N out can be done by mere dropping of higher-order coeffi- cients. This corresponds to applying a rectangular window to the HOA output vectors.
• more sophisticated bandwidth reduction techniques can be applied like those discussed in the above-mentioned M.A. Poletti article or in the above-mentioned J. Daniel article. Thereby, even more information is likely to be lost than with rectangular windowing, but superior directivity patterns can be accom ⁇ plished .
• the invention can be used in different parts of an audio processing chain, e.g. recording, post production, transmission, playback.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• Acoustics & Sound (AREA)
• Signal Processing (AREA)
• Computational Linguistics (AREA)
• Quality & Reliability (AREA)
• Health & Medical Sciences (AREA)
• Audiology, Speech & Language Pathology (AREA)
• Human Computer Interaction (AREA)
• Multimedia (AREA)
• Stereophonic System (AREA)

Priority Applications (8)

Applications Claiming Priority (2)

Publications (1)

Family

ID=46354265

Family Applications (1)

Country Status (11)

Cited By (11)

Families Citing this family (39)

Family Cites Families (17)

• 2011
  • 2011-06-30 EP EP11305845A patent/EP2541547A1/en not_active Withdrawn
• 2012
  • 2012-06-15 CN CN201280032460.1A patent/CN103635964B/zh active Active
  • 2012-06-15 EP EP12729512.9A patent/EP2727109B1/en active Active
  • 2012-06-15 JP JP2014517583A patent/JP5921678B2/ja active Active
  • 2012-06-15 DK DK12729512.9T patent/DK2727109T3/da active
  • 2012-06-15 HU HUE12729512A patent/HUE051678T2/hu unknown
  • 2012-06-15 US US14/130,074 patent/US9338574B2/en active Active
  • 2012-06-15 WO PCT/EP2012/061477 patent/WO2013000740A1/en not_active Ceased
  • 2012-06-15 AU AU2012278094A patent/AU2012278094B2/en active Active
  • 2012-06-15 BR BR112013032878-9A patent/BR112013032878B1/pt active IP Right Grant
  • 2012-06-15 KR KR1020147002760A patent/KR102012988B1/ko active Active
  • 2012-06-21 TW TW101122126A patent/TWI526088B/zh active

Non-Patent Citations (12)

Also Published As

Similar Documents

Legal Events