DE202017102729U1 - Signal processing systems for reproducing audio data on virtual speaker arrays - Google Patents

Abstract

An electronic device configured to reproduce sound fields in a left and a right ear of a human listener, the sound fields being generated by a plurality of virtual speakers, the electronic device comprising: memory; and control circuits connected to the memory, wherein the control circuits are configured to: receive, by processor circuits of a sound reproduction computer configured to play the sound fields in the left and right ears of the head of the human listener, a plurality of outer ear impulse responses (HRIRs), wherein each of the plurality of HRIRs is associated with a virtual speaker of the plurality of virtual speakers and an ear of the human listener, each of the plurality of HRIRs including samples of a sound field that at a specific sampling rate in a left or right ear generated in response to an audio pulse generated by this virtual speaker; Creating a first state space representation of each of the plurality of HRIRs, wherein the first state space representation includes a matrix, a column vector, and a row vector, wherein both the matrix and the column vector and the row vector of the first state space representation have a first size; Performing a state space reduction operation to generate a second state space representation of each of the plurality of HRIRs, the second state space representation including a matrix, a column vector, and a row vector, wherein both the matrix and the column vector and the row vector of the second state space representation are a second one Having a size smaller than a first size; and generating a plurality of outer ear transfer functions (HRTFs) based on the second state space representation, wherein each of the plurality of HRTFs corresponds to a respective HRIR of the plurality of HRIRs, where an HRTF corresponds to a respective HRIR that, when combined with a frequency domain sound field multiplied generated by the virtual speaker to which the corresponding HRIR is associated generates a component of a sound field reproduced in an ear of the human listener.

Description

Under protection and subject to the utility model are, according to the provisions of the utility model law, only devices as defined in the appended claims, but no method. Wherever in the description, if appropriate, reference is made to methods, these references are merely illustrative of the device or devices set forth in the appended claims

BACKGROUND

A virtual speaker array surrounding a listener is generally used to create a virtual spatial acoustic environment to play audio data over headphones. The sound field generated by this speaker array can be altered to provide the effect of moving audio sources that move relative to the user or to stabilize the source in a fixed spatial location as the user moves his head. These operations are very important for the playback of audio data via headphones in virtual reality (VR) systems.

The multi-channel audio data processed for playback via the virtual speakers are combined to provide a pair of signals to the left and right headphone speakers. This combination process of multi-channel audio data is known as binaural playback. The most generally considered way to implement this rendition is to use a multi-channel filtering system that implements Head Related Transfer Functions (HRTFs). For example, in a system based on a number M (where M is an arbitrary number) of virtual speakers, the binaural renderer must have 2M HRTF filters, as one pair is used per speaker to control the transfer function between the speaker and the left and model the right ear of the user.

SUMMARY

Conventional approaches to performing binaural playback require large amounts of computational resources. In this sense, when an HRTF is represented as a finite impulse response (FIR) order of n order, each binaural output requires 2M n multiplication and addition operations per channel. Such operations may burden the limited resources allocated for binaural rendering, for example in VR applications.

In contrast to the conventional approaches to performing binaural rendering requiring large amounts of computational resources, improved techniques involve applying a state space model with balanced implementation to each HRTF to determine the order of an effective FIR or even infinite impulse response ("Infinite Impulse Response - IIR "). In this sense, each HRTF G (z) is derived from an outer-ear impulse-response filter (HRIR) by, for example, a Z-transform. The data of the HRIR can be used to construct a first state space representation [A, B, C, D] of the HRTF by the relation G (z) = C (zI-A) ^-1 B + D. This first state space representation is ambiguous and thus for a FIR filter A and B simple arrays of binary values can be set while C and D contain the HRIR data. This representation leads to a simple form of Gram matrix Q whose eigenvectors provide system states that maximize the system gain measured by a Hankel standard. Furthermore, a factorization of Q offers a transformation to a balanced state space in which the Gram matrix corresponds to a diagonal matrix of the eigenvalues of Q. By taking into account only the states associated with an eigenvalue that is above a threshold, the balanced state space representation of the HRTF can be shortened to provide an approximate HRTF that closely approximates the original HRTF while satisfying the required Quantity of calculations reduced by up to 90%.

A general aspect of the improved techniques involves a method of rendering sound fields in a left and a right ear of a human listener, wherein the sound fields are generated by a plurality of virtual speakers. The method may include obtaining a plurality of outer ear impulse responses (HRIRs) by processor circuits of a sound reproduction computer configured to render the sound fields in the left and right ears of the human listener's head, each having a plurality of HRIRs virtual speaker is associated from the multitude of virtual speakers and one ear of the human listener, with each variety of HRIRs generated at a specific sampling rate in a left or right ear generates samples of a sound field generated in response to an audio pulse generated by the virtual speaker. The method may further include creating a first state space representation of each plurality of HRIRs, wherein the first state space representation includes a matrix, a column vector, and a row vector, wherein each of the matrix, column, and row vectors of the first state space representation has a first size , The method may further include performing a state space reduction operation to generate a second state space representation from each plurality of HRIRs, the second state space representation including a matrix, a column vector, and a row vector, wherein both the matrix and column and Line vector of the second state space representation each have a second size that is smaller than the first size. The method may further include generating a plurality of outer ear transfer functions (HRTFs) based on the second state representation, wherein each plurality of HRTFs corresponds to a respective HRIR of the plurality of HRIRs, an HRTF corresponding to a respective HRIR as they become is multiplied by a frequency domain sound field generated by the virtual speaker to which the corresponding HRIR is associated, generates a component of a sound field reproduced in an ear of the human listener.

Performing the state space reduction process may include, for each HRIR of the plurality of HRIRs, constructing a corresponding Gram matrix based on the first state space representation of that HRIR, the Gram matrix having a plurality of eigenvalues, in descending order of size and may include constructing the second state space representation of that HRIR based on the Gram matrix and the plurality of eigenvalues, the second size corresponding to a number of eigenvalues of the plurality of eigenvalues greater than a specified threshold.

Creating the second state space representation of each HRIR of the plurality of HRIRs may include forming a transformation matrix that, when applied to the Gram matrix based on the first state space representation of that HRIR, generates a diagonal matrix, each diagonal element the diagonal matrix is equal to a corresponding eigenvalue of the plurality of eigenvalues.

The method may further include creating a cepstrum of each HRIR for each plurality of HRIRs, the cepstrum having causal samples taken at positive times and non-causal samples taken at negative times, wherein for each of the plurality of HRIRs noncausal samples of the cepstrum, a phase minimization process is performed by adding this noncausal sample, taken at a negative time, to a causal sample of the cepstrum taken at the opposite point of that negative time, and generating a minimum phasic HRIR; setting each of the non-causal samples of the cepstrum to zero after performing the phase minimization procedure for each of the non-causal samples of the cepstrum.

The method may further include creating a multiple input / multiple output (MIMO) state space representation, wherein the MIMO state space representation includes a composite matrix, a column vector matrix, and a row vector matrix, wherein the composite The matrix of the MIMO state space representation is the matrix of the first representation of each plurality of HRIRs, the column vector matrix of the MIMO state space representation, including the column vector of the first representation of each plurality of HRIRs, and the row vector matrix of the MIMO state space representation, including the row vector of the first representation, each Variety of HRIRs, includes. In this case, the vector matrix and row vector matrix, performing the state space reduction process involves creating a reduced composite matrix, a reduced column vector matrix, and a reduced row vector matrix, wherein both the reduced composite matrix and the reduced column vector matrix and reduced row vector matrix have a size that is smaller than the size of the composite matrix, the column

Creating the MIMO state space representation may include forming a first block matrix as a composite matrix of the MIMO state space representation, wherein the first block matrix comprises a matrix of the first state space representation of HRIR that is a diagonal of the first block matrix with a virtual speaker of the plurality of virtual Loudspeakers, the matrices of the first state space representation of the HRIRs associated with the same virtual loudspeaker being in adjacent diagonal elements of the first block matrix. The creation of the MIMO state space representation may also involve the formation of a second block matrix as the column vector matrix of MIMO state space representation comprising, as a diagonal element of the second block matrix, a column vector of the first state space representation of an HRIR associated with a virtual speaker of the plurality of virtual speakers, column vectors of the first state space representation of the HRIRs associated with the same virtual speaker , lie in adjacent diagonal elements of the second block matrix. The creation of the MIMO state space representation may also include forming a third block matrix as a row vector matrix of the MIMO state space representation comprising as an element of the third block matrix a row vector of the first state space representation of an HRIR associated with a virtual speaker of the plurality of virtual speakers where row vectors of the first state space representation of the HRIRs representing sounds in the left ear are in odd-numbered elements of the first row of the third block matrix and row vectors of the first state space representation of the HRIRs representing sounds in the right ear are in even-numbered elements of the second row of the third Block matrix lie.

The method may further include, prior to creating the MIMO state space representation for each HRIR of the plurality of HRIRs, performing a single input single issue (SISO) state space reduction operation to generate a SISO state space representation of that HRIR as the first state space representation of that HRIR.

With regard to the method, for each of the plurality of virtual speakers, there are left and right HRIRs of the plurality of HRIRs associated with that virtual speaker, the left HRIR, once multiplied by the frequency domain sound field, generated by this virtual speaker, which generates the component of the sound field reproduced in the left ear of the human listener, and the right HRIR, as multiplied by the frequency domain sound field generated by this virtual speaker, the component of the Sound field generated, which is reproduced in the right ear of the human listener. Further, for each of the plurality of virtual speakers, there is an Interaural Time Delay (ITD) between the left HRIR associated with this virtual speaker and the right HRIR associated with that virtual speaker. wherein the ITD in the left HRIR and in the right HRIR is manifested by a difference between a number of original samples of the sound field of the left HRIR having zero values and a number of original samples of the sound field of the right HRIR having zero values. In this case, the method may further include creating an ITD unit subsystem matrix based on the ITD between the left HRIR and the right HRIR associated with each of the plurality of virtual speakers, and multiplying the plurality of HRTFs with the ITD Unit Subsystem Matrix to generate a variety of delayed HRTFs.

With regard to the method, each of the plurality of HRTFs may be represented by Finite Impulse Response (FIR) filters. In this case, the method may further include performing a conversion operation on each of the plurality of HRTFs to generate a further plurality of HRTFs, each represented by Infinite Impulse Response (IIRs) filters.

With regard to the method, for each of the plurality of virtual speakers, there is an HRIR associated with this virtual speaker that corresponds to the ear on the side of the head closest to the speaker, this is referred to as ipsilateral HRIR. The other HRIR associated with this virtual speaker is referred to as contralateral HRIR. The multitude of HRTFs can be divided into two groups. One group contains all ipsilateral HRTFs and the other group contains all contralateral HRTFs. In this case, the method can be applied independently to each group, thereby producing a reasonable approximation for that group.

BRIEF DESCRIPTION OF THE DRAWINGS

1 FIG. 12 is a block diagram illustrating an exemplary system for head-tracking, ambisonic-coded, virtual-speaker-based binaural audio data, according to one or more embodiments described herein.

2 FIG. 12 is a graphical representation of an exemplary state space system having Hankel singular values according to one or more embodiments described herein.

3 FIG. 10 is a graph illustrating impulse responses of a 25th order finite impulse response approximation and a 6th order infinite impulse response approximation for an exemplary state space system according to one or more embodiments described herein. FIG.

4 FIG. 12 is a graph illustrating impulse responses of a 25th order finite impulse response approximation and a 3rd order infinite impulse response approximation for an exemplary state space system according to one or more embodiments described herein. FIG.

5 FIG. 12 is a block diagram illustrating an exemplary arrangement of speakers relative to a user. FIG.

6 Figure 12 is a block diagram illustrating an exemplary binaural display system.

7 FIG. 12 is a block diagram illustrating an exemplary MIMO binaural display system according to one or more embodiments described herein.

8th FIG. 12 is a block diagram illustrating an exemplary binaural display system in accordance with one or more embodiments described herein.

9 FIG. 12 is a block diagram illustrating an exemplary computing device provided in accordance with one or more binaural rendering embodiments described herein.

10 FIG. 10 is a graphical representation illustrating exemplary results of a single input single output (SISO) IIR approach using balanced implementation for a first left node according to one or more embodiments described herein. FIG.

11 FIG. 12 is a graphical representation illustrating exemplary results of a single input single output (SISO) IIR approach using balanced implementation for a first right node according to one or more embodiments described herein. FIG.

12 FIG. 10 is a graphical representation illustrating exemplary results of a single input single output (SISO) IIR approach using balanced implementation for a second left node according to one or more embodiments described herein. FIG.

13 FIG. 12 is a graphical representation illustrating exemplary results of a single input single output (SISO) IIR approach using balanced implementation for a second right node according to one or more embodiments described herein. FIG.

14 FIG. 12 is a graphical representation illustrating exemplary results of a single-input single-issue (SISO) IIR approximation using balanced implementation for a third left node in accordance with one or more embodiments described herein. FIG.

15 FIG. 12 is a graphical representation illustrating exemplary results of a single-input single-issue (SISO) IIR approximation using balanced realization for a third right node according to one or more embodiments described herein. FIG.

16 FIG. 12 is a graphical representation illustrating exemplary results of a single input single output (SISO) IIR approach using balanced implementation for a fourth left node according to one or more embodiments described herein. FIG.

17 FIG. 12 is a graphical representation illustrating exemplary results of a single input single output (SISO) IIR approximation using balanced implementation for a fourth right node according to one or more embodiments described herein. FIG.

18 FIG. 12 is a flow chart illustrating an exemplary method of performing the improved techniques described herein. FIG.

The headings herein are for convenience only and do not necessarily affect the extent or meaning of the information claimed in this present disclosure.

In the drawings, the same reference numerals and any acronyms identify elements or tasks having similar or similar structure or functionality to facilitate understanding facilitate and clarify the context. The drawings will be described in detail in the course of the following detailed description.

DETAILED DESCRIPTION

Various examples and embodiments of the methods and systems of the present disclosure will now be described. The following description provides specific details for a thorough understanding and allows the description of the examples given. However, it will be understood by those skilled in the art that one or more embodiments described herein may be practiced without many of these details. Likewise, it will be understood by those skilled in the art that one or more embodiments of the present disclosure may include other features that are not described in detail herein. In addition, some well-known structures or functions may not be illustrated or described in detail below to avoid unnecessary confusion in the description.

The methods and systems of the present disclosure address the computational complexities of the binaural rendering process mentioned above. For example, one or more embodiments of the present disclosure relate to a method and system for reducing the number of arithmetic operations required to implement the 2M filtering functions.

introduction

1 shows an exemplary system 100 depicting how the final stage of a spatial audio player (for the purposes of the present example without regard to any processing of environmental effects) channels multichannel feeds to a virtual speaker array and encodes them into a headphone pair of signals. As illustrated, the final M-channel to 2-channel conversion is done using M individual 1-to-2 encoders, each encoder corresponding to a pair of left / right ear outer ear transmission functions (HRTFs). Thus, in the system description, the operator G (z) corresponds to a matrix

Each subsystem is usually the transfer function associated with the impulse response measured from a speaker location to the left / right ear. As described in more detail below, the methods and systems of the present disclosure provide a way to reduce the order of each subsystem through the use of a finite impulse response (FIR) to infinite impulse response (IIR) process. A conventional approach to this challenge is to consider each subsystem as a single-input single-issue (SISO) system in isolation and to simplify its structure. The following section examines this conventional approach and further examines how greater efficiency can be achieved by acting on the entire system as an M-input and 2-output multiple-input multiple output (MIMO) system.

While some existing techniques deal with MIMO models of HRTF systems, none is concerned with their use in Ambisonic-based virtual speaker systems, such as: In the present disclosure. The basis of the system order reduction described in the present disclosure is based on a metric known as the Hankel standard. Since this metric is not widely known or well understood, the following section tries to explain what the metric measures and why it has practical significance for acoustic system responses.

HRIR / HRTF structure

The impulse responses between a sound source and the left and right ears of a listener are referred to as outer ear impulse responses (HRIRs) and, once transformed to the frequency plane, as HRTFs. These answering features provide basic directional guidance on how the listener perceives the location of the sound source. Signal processing for the creation of virtual auditory representations uses these functions as filters in the synthesis of spatially accurate sound sources. In VR applications, user gaze tracking requires that audio synthesis be done as efficiently as possible For example, because (i) processing resources are limited and (ii) low latency is often required.

Z-Transformation befolgend Y(z) = G(z)X(z) (2) G(z) = [g₀ + g₁z^–1 + g₂z^–2 + ... + g_N–1z^N–1] (3) The signal transmission by the HRIR / HRTF, g, can be written for input x [k] and output y [k] as follows: (for simplicity, editions for k> N will be discussed below)
with g = [g ₀ , g ₁ , g ₂ , ..., g _N-1 ],

Following Z transformation Y (z) = G (z) X (z) (2) G (z) = [g ₀ + g ₁ z ^-1 + g ₂ z ^-2 + ... + g _N-1 z ^N-1 ] (3)

Here, an N-point HRIR for the left (L) or right (R) ear is represented as a transfer function of a Z domain. The first n _{L / R} sample values of HRIR are approximately zero due to the transport delay from the source location to the L / R ear. The difference n _L - n _R contributes to the interaural time delay (ITD), giving a significant binaural indication of the direction of the source. From here G (z) refers to any of the HRTFs and the subscripts L and R are only used when differential properties are described.

Approaching an FIR through a low-order IIR structure

Introduction to the Hankel standard

The following description tries G (z) through an alternative system G ^ (z) which offers an advantage, such as lower computational load, as well as a "good" approximation to G (z), such as through a metric || G (z) - G ^ (z) || measured, where y = Gx and y ^ = G ^ x a useful metric of the difference of the H _∞ norm of the error system resulting from corresponding

This energy ratio as a norm gives the maximum energy in the difference for the minimum energy in the signal driving the systems. Therefore, in order for the approximation error to be slight, it is obvious to delete those modes that transfer the least energy from the input x to the output y. It is helpful to see that the H _∞ norm of the error has the practical relevance to match

This shows that the H _∞ norm corresponds to the peak of the Bode Magnitude Diagram of the error.

The challenge and difficulty, however, is to characterize the relationship between this standard and the system modes. Instead, the following section will examine the use of Hankel-norm of the error, as this advantageous conditions into system characteristics will become apparent and can be easily shown that by doing an upper limit for the H _∞ norm is provided.

The Hankel norm of a system is the induced gain of a system for an operator, the Hankel operator φ _G , which is defined by the convolution-like ratio

It should be noted that by assuming k = 0 as the "current" time, this operator determines φ _G , such as an input sequence x [k] applied from -∞ to k = -1, below the output of the system appears.

The Hankel norm induced by φ _G is defined as

It should also be apparent that the Hankel standard represents maximizing the future energy that can be recovered at the system output while minimizing the historical input energy into the system. Or, in other words, the future output energy resulting from any input is at most Hankel's norm times the input energy, assuming the future input is zero.

State space system representation and the Hankel standard

kann somit für w[k]εR^n–1 und mit AεR^{(n–1)x(n–1)}, BεR^(n–1)x1, CεR^1x(n–1), und DεR, dieses System durch das Zustandsraum-Modell S: [A, B, C, D] beschrieben werden: w[k + 1] = Aw[k] + Bx[k] y[k] = Cw[k] + Dx[k] (9) From the above description, it can be seen that the Hankel standard provides a useful level of energy transfer through a system. However, to understand how the norm relates to the system order and its reduction, it is necessary to characterize the internal dynamics of the system as modeled by its state space representation. The representative connection between the state space model of a linearly shifting invariant (LSI) system and its transfer function is well known. With a single-input single-output (SISO) system of n order, which is described by the transfer function,

Thus, for w [k] εR ^n-1 and with AεR ^{(n-1) x (n-1)} , BεR ^{(n-1) x1} , CεR ^{1x (n-1)} , and DεR, this system can pass through the state space Model S: [A, B, C, D] are described: w [k + 1] = Aw [k] + Bx [k] y [k] = Cw [k] + Dx [k] (9)

The Z transformation of this system corresponds zW (z) = AW (z) + BX (z) Y (z) = CW (z) + DX (z) yielding Y (z) = [C (zI-A) ^-1 B + D] X (z) = G (z) X (z) (10)

It should be noted that the system matrices [A, B, C, D] are not unique, and an alternative state space model can be obtained, for example v [k] by the following similarity transformation for an invertible matrix TεR ^{(n-1) x (n-1)} , Tv = w A ^ = T- ¹ AT, B ^ = T- ¹ B, C ^ = CT, and D ^ = D. The state space model S: [A ^, B ^ C ^ D ^] has the same transfer function G (z).

It should be understood that, for purposes of the present example, it is assumed that G (z) is a stable system which, accordingly, is S stable, which means that the eigenvalues of A = λ. (A) all lie on the disk unit [λ] <1.

The Henkel norm of G (z) can now be described in terms of the energy stored in w [0] as a result of an input sequence x [k] for -∞ <k ≤ -1, and then how much of that energy to the output y [k] is supplied for k ≥ 0.

• (i) Die Gram'sche Zugänglichkeits-(Steuerbarkeits)Matrix
  
  und
• (ii) Die Gram'sche Beobachtbarkeits-Matrix
  

To describe the internal energy of S, it is necessary to introduce two system properties:

• (i) The Gram Accessibility (controllability) matrix
  
  and
• (ii) Gram's observability matrix
  

Since A is stable, the two above additions coincide, and it is easy to show that P is symmetric and positive definite if, and only if, pair (A, B) is controllable (meaning that starting at w [0], from a sequence x [k], k> 0 can be determined that this can lead the system into any state w *). Q is also symmetric and positive definite if, and only if, the pair (A, C) is observable (meaning that the state of the system at each instant j is determinable from the system outputs y [k] for k> j) ,

It can easily be shown that P and Q can be obtained as solutions of the Lyapunov equations, APA ^T + BB ^T - P = 0 and that A ^T QA + C ^T C - Q = 0 the observed energy of the state of energy in the course y [k] for k ≥ 0 with w [0] = w0 and x [k] = 0 for k ≥ 0. It's easy to show that

The minimum control energy problem is defined as what corresponds to the minimum energy:

This is a standard problem of optimal control, which can be solved as follows x _opt [k] = B ^T (A ^T ) ^{- (1 + k)} P ^-1 w ₀ f or k <0 yielding J (x _opt ) = w T / 0P ^-1 w ₀

In view of the above, it is now possible to associate the Hankel norm of a system G (z), or equivalently S: [A, B, C, D], with Q and P Gram matrices as

Balanced state space system representations

It should now be apparent that it is possible for HRTF systems to compute an appropriate similarity transformation, T, to system implementation S: [A ^, B ^, C ^, D ^] to obtain the same Gram accessibility and observability matrices corresponding to the shape of a diagonal matrix Σ Q = P = Σ = diag '(σ ₁ , ..., σ _n-1 ) with σ ₁ ≥ σ ₂ ≥ ... ≥ σ _n-1 > 0

• (i) Beginnend mit G(z) wird diese als Zustandsraumsystem S: [A, B, C, D] bestimmt (z. B. erkannt).
• (ii) Für S werden die Gram'schen Matrizen gelöst, um P und Q zu erhalten.
• (iii) Dabei wird Lineare Algebra verwendet, um Folgendes zu ergeben Σ = diag(σ₁, ..., σ_n–1) = √λ(PQ)
• (iv) Faktorisierung P = M^T M und MQM^T = W^TΣ²W, wobei W einheitlich ist, ergibt M und W, sodass T = M^TWΣ – ½ wofür P ^ = T^TPT = Σ = Q ^ = T^–1Q(T^–1)^T
• (v) Das T aus (iv) kann verwendet werden, um eine neue Darstellung des Systems zu erhalten als A ^ = T^–1AT, B ^ = T^–1B, C ^ = CT, und D ^ = D
• (vi) In der Darstellung, die in (v) erhalten wird, gibt es ausgeglichene Zustände. Mit anderen Worten beträgt die minimale Energie, um das System in den Zustand (0.0....1.0...0)^T mit a 1 in Position i zu versetzend σ –1 / i, während, wenn das System in diesem Zustand freigegeben wird, die an dem Ausgang wiedererlangte Energie σ_i beträgt.
• (vii) In diesem ausgeglichenen Modell werden die Zustände hinsichtlich ihrer Bedeutung für die Übertragung von Energie vom Signaleingang zum -ausgang geordnet. Somit führen ein Kürzen der Zustände in dieser Struktur und gleichermaßen eine Reduktion der Ordnung von G(z) dazu, dass die Zustände hinsichtlich ihrer Bedeutung für die Energieübertragung entfallen.

According to at least one embodiment of the present disclosure, obtaining a balanced state space system representation may include:

• (i) Starting with G (z), it is determined (eg recognized) as state space system S: [A, B, C, D].
• (ii) For S, the Gramian matrices are solved to obtain P and Q.
• (iii) Linear algebra is used to give the following Σ = diag (σ ₁ , ..., σ _n-1 ) = √ λ (PQ)
• (iv) Factorization P = M ^T M and MQM ^T = W ^T Σ ² W, where W is uniform, yields M and W, so T = M ^T WΣ - ½ for which P ^ = T ^T PT = Σ = Q ^ = T ^-1 Q (T ^-1 ) ^T
• (v) The T of (iv) can be used to obtain a new representation of the system as A ^ = T ^-1 AT, B ^ = T ^-1 B, C ^ = CT, and D ^ = D
• (vi) In the representation obtained in (v), there are balanced states. In other words, the minimum energy is to put the system in the state (0.0 .... 1.0 ... 0) ^T with a 1 in position i σ -1 / i, while, when the system is released in this state, the energy recovered at the output is σ _i .
• (vii) In this balanced model, the states are ordered in terms of their importance for the transmission of energy from the signal input to the output. Thus, truncating the states in this structure, as well as reducing the order of G (z), eliminates the states of importance for energy transfer.

Example of an order reduction based on a balanced state space system

The following section examines creating a state space model of an FIR structure and its order reduction using the balanced system representation described above.

mit Übertragungsfunktion G(z) = [g₀ + g₁z^–1 + ... g₂₅z^–25] The present example proceeds by examining a 26-point FIR filter g [k]

with transfer function G (z) = [g ₀ + g ₁ z ^-1 + ... g ₂₅ z ^-25 ]

A state space model of the 25th order is created with

As in 2 The system S has: [A, B, C, D] Hankel singular values (SVs).

S becomes too S: [A ^ = T ^-1 AT converted. B ^ = T ^-1 B, C ^ = CT, and D ^ = D]. From the profile of Hankel SVs (eg as in 2 illustrated), a 6th order approximation can be obtained. The system is thus partitioned as follows:

The order-reduced system corresponds S ₆ : [A ^ _{6 × 6} , B ^ _{6 × 1} , C ^ _{1 × 6} , D ^], resulting in the order-reduced transfer function

For comparison are in 3 illustrates the impulse responses of the original FIR G (z) and the 6th order IIR approximation. This in 3 The diagram shown discloses an almost lossless match.

In addition, in 4 For comparison, the impulse responses of the original FIR G (z) and the 3rd order IIR approximation are illustrated.

Balanced approach to HRIRs

Virtual speaker array and HRIR set

The following section describes an exemplary scenario based on a simple square array of speakers, as in 5 illustrated in which the outputs are down-converted to binaural using the HRIRs of subject 15 of the CIPIC set. These are 200-point HRIRs that are sampled at 44.1kHz and the set contains a set of associated data that includes interaural time difference (ITD) measurements between each pair of HRIRs. The transfer function G (z) of an HRIR (eg, equation (3) above) has a number of leading coefficients [g ₀ , ..., g _m ] that are zero, and give a response delay for each response, and as shown in equation (12) below, give G (z). The difference between the deployment times of the left and right pair of HRIRs largely determines their contribution to the ITD. The shape of a typical left HRTF is given in equation (12), while the right HRTF has a similar form:

The ITD results from ITD = | m _L = m _R |, which is provided for each HRIR pair in the CIPIC database. The excess phase associated with the insertion delay means that each G (z) is not minimally phasic, and it has also been demonstrated that the bulk of the HRTF G `(z) also is not minimal phasic. However, it has also been demonstrated that listeners do not distinguish between the filter effect of G `(z) and distinguish its minimal phasic version, which is indicated as H (z). Thus, in the present example, the FIR-IIR approximation equates the original FIRs G (z) by their minimum phasic H (z) to an action that removes the insertion delay from each HRIR.

Single Input Single Output IIR Approach Using Balanced Realization

• (i) Auslesen der HRIR(1/r, 1:200) für jeden Knoten.
• (ii) Erhalten des minimalphasischen Äquivalents unter Verwendung eines Cepstrum, wodurch sich HHRIR(1/r, 1:200) ergibt.
• (iii) Erstellen einer SISO-Zustandsraumdarstellung von HHRIR(1/x, 1:200) als S: [A, B, C, D]. Dabei handelt es sich um einen 199-dimensionalen Zustandsraum.
• (iv) Verwenden des oben beschriebenen ausgeglichenen Reduktionsverfahrens, um eine ordnungsreduzierte Version von S in Dimension rr zu erhalten. Zum Beispiel, S_rr: [A_rr, B_rr, C_rr, D_rr].

In accordance with at least one embodiment, single-input single-issue (SISO) IIR approximation using balanced implementation is a simple process that includes, for example:

• (i) read the HRIR (1 / r, 1: 200) for each node.
• (ii) obtaining the minimum phase equivalent using a cepstrum to give HHRIR (1 / r, 1: 200).
• (iii) Creating a SISO state space representation of HHRIR (1 / x, 1: 200) as S: [A, B, C, D]. This is a 199-dimensional state space.
• (iv) using the balanced reduction method described above to obtain a reduced order version of S in dimension rr. For example, S _rr : [A _rr , B _rr , C _rr , D _rr ].

The cepstrum of this HRIR may include causal samples taken at positive times rather than causal samples taken at negative times. Thus, for each of the non-causal samples of the cepstrum, a phase minimization process can be performed by adding that non-causal sample taken at a negative time to a causal sample of the cepstrum taken at the opposite point of that negative time. A minimum phase HRIR may be established by setting each of the non-causal samples of the cepstrum to zero after performing the phase minimization procedure for each of the non-causal samples of the cepstrum.

Exemplary results of the approximation of the left and right HRIRs for each 12th-order node (eg, for rr = 12) are shown in the diagrams appearing in the 10 - 17 to be shown.

10 - 17 Figure 15 shows plots illustrating frequency responses of subject 15 CIPIC [+/- 45 degrees, +/- 135 degrees], Fs = 44100 Hz, original FIR 200 point, IIR 12th order approximation.

In the 10 - 17 Graphically presented results show that the 12th order IIR approximations yield very close matches, both in magnitude and in phase, with the frequency responses of the original HRTFs. This means that instead of implementing 8x200 Pkt FIRs, the HRIR calculation can be implemented as 8x [{6 biquad} IIR sections + ITD delay line].

Multiple Input Multiple Output IIR Approach Using Balanced Realization

• (i) Auslesen der HRIR(1/r, 1:200) für jeden Knoten.
• (ii) Erhalten des minimalphasischen Äquivalents unter Verwendung eines Cepstrum, wie oben beschrieben, wodurch sich für jeden Knoten HHRIR(1/r, 1:200) ergibt.
• (iii) Erstellen einer SISO-Zustandsraumdarstellung von jedem HHRIR(1/r, 1:200) als S_ij: [A_ij, B_ij, C_ij, D_ij] für i = 1, 2 = links/rechts und j = 1, 2, 3, 4 = Knoten 1, 2, 3, 4. Jedes S_ij entspricht einem 199-dimensionalen Zustandsraumsystem. Hier A_ij ∈ R^199×199, B_ij ∈ R^199×1, C_ij ∈ R^1×199, and D_ij ∈ R^1×1.
• (iv) Erstellen ein zusammengesetztes MIMO-System mit einem internen Zustandsraum mit Abmessungen von beispielsweise 4 × 199 = 796 und mit 4 Eingängen und 2 Ausgängen. Dieses System S: [A, B, C, D], wobei A, B, C, D strukturiert ist als:
  

In accordance with at least one embodiment, multiple input multiple output (MIMO) IIR approximation using balanced implementation corresponds to a process that may be initiated in the same manner as the SISO described above. The process may include, for example:

• (i) read the HRIR (1 / r, 1: 200) for each node.
• (ii) Obtaining the minimum phase equivalent using a cepstrum as described above, resulting in HHRIR (1 / r, 1: 200) for each node.
• (iii) constructing a SISO state space representation of each HHRIR (1 / r, 1: 200) as S _ij : [A _ij , B _ij , C _ij , D _ij ] for i = 1, 2 = left / right and j = 1, 2, 3, 4 = nodes 1, 2, 3, 4. Each S _ij corresponds to a 199-dimensional state space system. Here A _ij ∈ R ^{199 × 199} , B _ij ∈ R ^{199 × 1} , C _ij ∈ R ^{1 × 199} , and D _ij ∈ R ^{1 × 1} .
• (iv) Create a composite MIMO system with an internal state space having dimensions of, for example, 4 x 199 = 796 and 4 inputs and 2 outputs. This system S: [A, B, C, D] where A, B, C, D is structured as:
  

This 796-dimensional system may be reduced using the described balanced reduction method according to one or more embodiments of the present disclosure.

At least in the exemplary embodiment described above, each of the subsystems S _ij is reduced to a 30th order SISO system prior to creating S. This step makes S a system with dimensions of 4 x 30 = 120. This can then be applied to, for example, a system with n = 12, 4th order inputs and 2 outputs similar to those in 6 illustrated system to be reduced.

As described in more detail below, the methods and systems of the present disclosure address the computational complexities of the binaural rendering process. For example, one or more embodiments of the present disclosure relate to a method and system for reducing the number of arithmetic operations needed to implement the 2M filtering functions.

Existing binaural rendering systems incorporate HRTF filter functions. These are usually implemented using the finite impulse response (FIR) filter structure, with some implementations using the infinite impulse response (IIR) filter structure. The FIR approach uses a filter of length n and requires n multiply and add (MA) operations for each HRTF (eg, 400) to provide an output sample to each ear. This means that every binaural output needs n × 2M MA operations. For example, in a typical binaural display system n = 400 may be used. The IIR approach described in the present disclosure uses a recursive structure of order m, where m is typically on the order of, for example, 12-25 (e.g., 15).

It should be noted that in order to compare the computational load of the IIR with that of the FIR, one would need to consider numerator and denominator. For 2M-SISO-IIR, every order m would be almost 2m × 2M MA (that is, there would be 1 multiplication less). For a MIMO structure one would have [(m - 1) × 2M + 2m] MA, where {+ 2m} stands for the common recursive sections. Of course, m in MIMO is greater than m in SISO.

Unlike existing approaches, the methods and systems of the present disclosure have recursive parts that are common to, for example, all HRTFs of the left (or right) ear or other architectural devices, such as all HRTFs of the ipsilateral (or contralateral) ear.

The methods and systems of the present disclosure may be of particular importance for the reproduction of binaural audio data in Ambisonic audio systems. That's because Ambisonic delivers spatial sound in a way that activates all speakers in the virtual array. Thus, the growing Importance of saving computational steps by applying the present techniques as long as M is growing.

The final M-channel 2-channel binaural playback is usually done using m individual 1-to-2 encoders, with each encoder corresponding to a pair of left / right ear external ear transmission functions (HRTFs). So the system description corresponds to the HRTF operator Y (z) = G (z) X (z) G (z) resulting from the matrix here

With FIR filters, each subsystem has the following form (with the leading k ^ij coefficients equal to zero in the non-minimal phasic case {Z. G ij / 0: g ij / k-1 = 0}): G _ij (Z) = [g ij / 0 + g ij / 1 ^z-1 + g ij / 2 ^z-2 + ... + g ij / N-1z ^N-1 ]

In accordance with one or more embodiments of the present disclosure, an approximation to G (z) may be made by an n order MIMO state space system S: [A ^, B ^, C ^, D ^]. This results in the exemplary binaural MIMO rendering (eg mixer) system that is described in US Pat 7 (which may be used for 3D audio according to at least one embodiment).

In 7 For example, the ITD subsystem corresponds to a set of pairs of delay lines, where, per input channel, only one of the pairs corresponds to one delay and the other corresponds to one unit. Therefore, there is an input / output representation in the Z domain, such as

Each pair (δ _1k , δ _2k ) has the form (α, β) where α = 0, when the left ear is ipsilateral to the source, and conversely, β> 0 corresponds to the ITD delay when the right ear is ipsilateral.

The M Input 2 Output MIMO System S: [A ^, B ^, C ^, D ^], which has been reduced to order n using the balanced reduction method can be used to obtain a HRTF set that can be written as G ^ (z) = {C ^ [zI - A ^] ^-1 B ^ + D ^} · Δ (z)

Here the '.' the Hadamard product. This transfer function matrix is different from the above G (z) because each subsystem now has the same denominator. The subsystems are the IIR form of the HRTF to the left / right ear [i = 1 ≡ left i = 2 ≡ right] from the virtual speaker j and have the shape

Therefore, if the balanced reduction approach to MIMO (as described above) is used to approximate the original n-point FIR HRFTs of an n-order {eg, n = N / 10}, the binaural rendering may be the one in 8th illustrated system can be implemented.

It should be noted that according to at least one embodiment, the final IIR section, as in FIG 8th shown, can be combined with space effect filtering.

In addition, it should be noted that this factorization into individual angle-dependent FIR sections in cascade is consistent with a shared IIR section with experimental research results. These experiments have shown how HRIRs are susceptible to approximate factorization.

9 shows a high level block diagram of an exemplary computing device ( 900 ), which is designed for binaural reproduction, by reducing the number of arithmetic operations necessary to implement the (eg 2M) filtering functions according to one or more embodiments described herein. In a very basic configuration ( 901 ) contains the computer device ( 900 ) usually one or more processors ( 910 ) and system memory ( 920 ). A memory bus ( 930 ) can be used for data transfer between the processor ( 910 ) and the system memory ( 920 ) be used.

Depending on the desired configuration, any type of processor ( 910 ), including a microprocessor (μP), a microcontroller (μC), a digital signal processor (DSP) or the like or a combination thereof. The processor ( 910 ) can be single or multi-level caching, for example, a level one cache ( 911 ) and one of the stage two ( 912 ), a processor core ( 913 ), as well as registers ( 914 ). The processor core ( 913 ) may include an arithmetic logic unit (ALU), a floating point unit (FPU), a digital signal processing core (DSP Core), or the like, or any combination thereof include. A memory controller ( 915 ) can also be used with the processor ( 910 ) may be used in some implementations of the memory controller ( 915 ) an integral part of the processor ( 910 ) form.

Depending on the desired configuration, the system memory ( 920 ) may be of any kind, including volatile memory (such as RAM), nonvolatile memory (such as ROM, flash memory, etc.) or a combination thereof. A system memory ( 920 ) typically includes an operating system ( 921 ), one or more applications ( 922 ) and program data ( 924 ). The application ( 922 ), a system for binaural reproduction ( 923 ). According to at least one embodiment of the present disclosure, the system for binaural reproduction ( 923 ) designed to reduce the computational complexity of the binaural rendering process. For example, the system for binaural playback ( 923 ) are able to reduce the number of arithmetic operations needed to implement the 2M filtering functions described above.

Program data ( 924 ) may contain stored instructions which, when executed by the one or more processing devices, comprise a system ( 923 ) and methods for binaural reproduction. Additionally and in accordance with at least one embodiment, program data ( 924 ) Audio data ( 925 ), which may, for example, relate to multi-channel audio signal data from one or more virtual speakers. According to at least some embodiments, the application ( 922 ) be designed with program data ( 924 ) on an operating system ( 921 ) to work.

The computing device ( 900 ) may have additional features or functionalities, as well as additional interfaces to facilitate communications between the basic configuration ( 901 ) and all necessary equipment and interfaces.

A system memory ( 920 ) is an example of computer storage media. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other storage technology, CD-ROM, Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes and tapes, magnetic disk storage or other magnetic storage devices or any other Medium which can be used to store the desired information and that of the computing device 900 can be accessed. Each of these computer storage media may be part of the device ( 900 ) form.

Computer device ( 900 ) can be used as a small, portable (or mobile) electronic device such as a mobile phone, a smartphone, a personal data assistant (PDA), a personal media player device, a tablet computer (tablet), a wireless web Watch device, a personal headset device, an application-specific device or a hybrid device, which includes one of the above-mentioned functions to be implemented. In addition, the computer device ( 900 ) may also be implemented as a personal computer, including as both a laptop computer and non-laptop computer configuration, as one or more servers, Internet of Things systems, and the like.

18 illustrates an exemplary method 1800 to carry out the binaural reproduction. method 1800 can be done by software constructs that are related to 9 be described in the memory 920 of the computer device 900 reside and processor 910 be executed.

at 1802 receives the computer device 900 each of the plurality of HRIRs associated with a virtual speaker of the plurality of virtual speakers and an ear of the human listener. Each of the plurality of HRIRs includes samples of a sound field generated at a specific sampling rate in a left or right ear generated in response to an audio pulse generated by this virtual speaker.

at 1804 creates the computer device 900 a first state space representation of each of the plurality of HRIRs. The first state space representation includes a matrix, a column vector, and a row vector. Both the matrix and the column vector and the row vector of the first state space representation have a first size.

at 1806 leads the computer device 900 a state space reduction process to generate a second state space representation of each of the plurality of HRIRs. The second state space representation includes a matrix, a column vector, and a row vector. Both the matrix and the column vector and the row vector of the second state space representation have a second size less than the first size.

at 1808 creates the computer device 900 a plurality of outer ear transfer functions (HRTFs) based on the second state space representation. Each of the plurality of HRTFs corresponds to a respective HRIR of the plurality of HRIRs. An HRTF corresponding to a respective HRIR, when multiplied by a frequency domain sound field generated by the virtual speaker to which the corresponding HRIR is associated, generates a component of a sound field reproduced in an ear of the human listener becomes.

The foregoing detailed description sets forth various embodiments of the devices and / or processes through the use of block diagrams, flowcharts, and / or examples. If these block diagrams, flowcharts, and / or examples include one or more functions and / or operations, those skilled in the art will appreciate It will be appreciated by those in the art that any function and / or operation within the same block diagrams, flowcharts, or examples, individually and / or collectively, may be implemented by a wide range of hardware, software, firmware, or virtually any combination thereof. In accordance with at least one embodiment, several portions of the subject matter described herein may be implemented via application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (Digital Signal Processors - DSPs ") or other integrated formats. Nevertheless, those skilled in the art will recognize that some aspects of the embodiments set forth herein may be implemented, in whole or in part, also in integrated circuits, as one or more computer programs executing on one or more computers as one or more programs one or more processors are executed as firmware or any combination thereof; and that designing the circuitry and / or writing the code for the software and / or firmware with respect to the present disclosure is well within the purview of those skilled in the art.

Moreover, those skilled in the art will recognize that the mechanisms of the subject matter described herein are capable of being propagated as a program product in a variety of forms, and that an illustrative embodiment of the subject matter described herein, independent of the particular type of non-transitory signal bearing medium used to perform the dissemination is applicable. Examples of non-transitory signal carrying media include, but are not limited to, a writable type of media, such as: As a floppy disk, a hard disk, a CD, a DVD, a digital tape, a computer memory, etc., and a transitory type of media such. Digital and / or analog communication media (such as a fiber optic cable, a waveguide, a wired communication cable connection, a wireless communication connection, etc.).

As for the use of substantially all plural and / or singular terms herein, those skilled in the art may change the plural form to the singular form and vice versa, depending on the context and / or application. The various singular / plural permutations may be expressly set forth herein for clarity.

Accordingly, certain embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. Thus, in some cases, the actions indicated in the claims may be performed in a different order and still achieve desirable results. Moreover, for example, the processes illustrated in the attached figures do not necessarily require the particular order or sequence shown to achieve desired results. Under certain circumstances, multitasking and parallel processing can be beneficial.

Claims (20)

An electronic device configured to reproduce sound fields in a left and a right ear of a human listener, the sound fields being generated by a plurality of virtual speakers, the electronic device comprising: Storage; and control circuits connected to the memory, wherein the control circuits are configured for: Obtained by processor circuits of a sound reproduction computer configured to reproduce the sound fields in the left and right ears of the head of the human listener, a plurality of outer ear impulse responses (HRIRs), each of the plurality of HRIRs having a virtual speaker is associated with the plurality of virtual speakers and an ear of the human listener, each of the plurality of HRIRs including samples of a sound field generated at a specific sampling rate in a left or right ear generated in response to an audio pulse; which is generated by this virtual speaker; Creating a first state space representation of each of the plurality of HRIRs, wherein the first state space representation includes a matrix, a column vector, and a row vector, wherein both the matrix and the column vector and the row vector of the first state space representation have a first size; Performing a state space reduction operation to generate a second state space representation of each of the plurality of HRIRs, the second state space representation including a matrix, a column vector, and a row vector, wherein both the matrix and the column vector and the row vector of the second state space representation are a second one Having a size smaller than a first size; and Generating a plurality of outer ear transmission functions (HRTFs) based on the second state space representation, wherein each of the plurality of HRTFs corresponds to a respective HRIR of the plurality of HRIRs, where HRTF corresponds to a respective HRIR that multiplies as multiplied by a frequency domain sound field generated by the virtual speaker to which the corresponding HRIR is associated generates a component of a sound field that is reproduced in an ear of the human listener. The electronic device of claim 1, wherein performing the state space reduction operation for each HRIR of the plurality of HRIRs includes: Constructing a respective Gram matrix based on the first state space representation of this HRIR, the Gram matrix having a plurality of eigenvalues arranged in descending order; and Creating the second state space representation of this HRIR based on the Gram matrix and the plurality of eigenvalues, the second size being equal to a number of eigenvalues of the plurality of eigenvalues greater than a specific threshold. The electronic device of claim 2, wherein constructing the second state space representation of each HRIR of the plurality of HRIRs includes forming a transformation matrix that, when applied to the Gram matrix based on the first state space representation of that HRIR generates diagonal matrix, each diagonal element being equal to a respective eigenvalue of the plurality of eigenvalues. The electronic device of claim 1, wherein the control circuits for each of the plurality of HRIRs are further configured for: Creating a cepstrum of this HRIR, the cepstrum having causal samples taken at positive times and non-causal samples taken at negative times; Performing a phase minimization process for each of the non-causal samples of the cepstrum by adding that non-causal sample taken at a negative time to a causal sample of the cepstrum taken at the opposite point of that negative time; and generating a minimum phase HRIR by setting each of the non-causal samples of the cepstrum to zero after performing the phase minimization process for each of the non-causal samples of the cepstrum. The electronic device of claim 1, further comprising establishing a multiple input / multiple output (MIMO) state space representation, wherein the MIMO state space representation includes a composite matrix, a column vector matrix, and a row vector matrix, wherein the composite matrix is the MIMO state space representation the matrix of the first representation of each of the plurality of HRIRs, the column vector matrix of the MIMO state space representation includes the column vector of the first representation of each of the plurality of HRIRs, and the row vector matrix of the MIMO state space representation the row vector of the first representation of each of Variety of HRIRs; and wherein performing the state space reduction operation includes creating a reduced composite matrix, a reduced column vector matrix, and a reduced row vector matrix, wherein both the reduced composite matrix and the reduced column vector matrix and the reduced row vector matrix have a size, each less than a size of the composite matrix, the column vector matrix and the row vector matrix. The electronic device of claim 5, wherein creating the MIMO state space representation includes: Forming a first block matrix as a composite matrix of the MIMO state space representation comprising a matrix of the first state space representation of an HRIR associated with a virtual speaker of the plurality of virtual speakers as a diagonal element of the first block matrix, matrices of the first state space representation of HRIRs associated therewith virtual speakers are located in adjacent diagonal elements of the first block matrix; Forming a second block matrix as a column vector matrix of the MIMO state space representation comprising a column vector of the first state space representation of an HRIR associated with a virtual loudspeaker of the plurality of virtual loudspeakers as a diagonal element of the second block matrix, wherein column vectors of the first state space representation of HRIRs associated with the same virtual speaker are in adjacent diagonal elements of the second block matrix; and Forming a third block matrix as a row vector matrix of the MIMO state space representation comprising a row vector of the first state space representation of an HRIR associated with a virtual speaker of the plurality of virtual speakers as an element of the third block matrix, wherein row vectors of the first state space representation of HRIRs represent the sounds in the left Ear, are in odd elements of the first row of the third block matrix, and row vectors of the first state space representation of HRIRs representing sounds in the right ear are in even elements of the second row of the third block matrix. The electronic device of claim 5, further comprising performing, prior to creating the MIMO state space representation, a single input single issue (SISO) state space reduction process for each HRIR of the plurality of HRIRs, a SISO state space representation of that HRIR as the first one State space representation of this HRIR. The electronic device of claim 1, wherein for each of the plurality of virtual speakers there is a left HRIR and a right HRIR of the plurality of HRIRs associated with that virtual speaker, the left HRIR being equal to that of the virtual one Multiplied loudspeaker generated frequency domain sound field, which generates component of the sound field reproduced in the left ear of the human listener, and the right HRIR, when multiplied by the frequency domain sound field generated by this virtual speaker, generates the component of the sound field, which is reproduced in the right ear of the human listener; and wherein there is an interaural time delay (ITD) between the left HRIR associated with this virtual speaker and the right HRIR associated with that virtual speaker for each of the plurality of virtual speakers, the ITD being in the left and right HRIR by a difference between a number of original samples of the sound field of the left HRIR having zero values and a number of original samples of the sound field of the right HRIR having zero values. The electronic device of claim 8, wherein the control circuits are further configured to: Creating an ITD unit subsystem matrix based on the ITD between the left HRIR and the right HRIR associated with each of the plurality of virtual speakers; and Multiplying the plurality of HRTFs by the ITD unit subsystem matrix to generate a plurality of delayed HRTFs. An electronic device according to claim 1, wherein each of said plurality of HRTFs is represented by Finite Impulse Response (FIR) filters; and wherein the control circuits are further configured to perform a conversion operation on each of the plurality of HRTFs to generate a further plurality of HRTFs. each represented by Infinite Impulse Response (IIRs) filters. A computer program product comprising a non-transitory storage medium, the computer program product including code that, when executed by a processor circuit of a sound reproduction computer configured to reproduce sound fields in a left and a right ear of a human listener, the processor circuit caused to perform a method, the method comprising: Obtaining a plurality of outer ear impulse responses (HRIRs), wherein each of the plurality of HRIRs is associated with a virtual speaker of the plurality of virtual speakers and an ear of the human listener, each of the plurality of HRIRs including samples of a sound field at a specific sampling rate in a left or right ear generated in response to an audio pulse generated by this virtual speaker; Creating a first state space representation of each of the plurality of HRIRs, wherein the first state space representation includes a matrix, a column vector, and a row vector, wherein both the matrix and the column vector and the row vector of the first state space representation have a first size; Performing a state space reduction operation to generate a second state space representation of each of the plurality of HRIRs, the second state space representation including a matrix, a column vector, and a row vector, wherein both the matrix and the column vector and the row vector of the second state space representation are a second one Having a size less than the first size; and Generating a plurality of outer ear transfer functions (HRTFs) based on the second state space representation, wherein each of the plurality of HRTFs corresponds to a respective HRIR of the plurality of HRIRs, wherein an HRTF corresponding to a respective HRIR as multiplied by a frequency domain sound field generated by the virtual speaker to which the corresponding HRIR is associated generates a component of a sound field reproduced in an ear of the human listener. The computer program product of claim 11, wherein performing the state space reduction operation for each HRIR of the plurality of HRIRs includes: Constructing a respective Gram matrix based on the first state space representation of said HRIR, said Gram matrix having a plurality of eigenvalues arranged in descending order; and Creating the second state space representation of this HRIR based on the Gram matrix and the plurality of eigenvalues, the second size being equal to a number of eigenvalues of the plurality of eigenvalues greater than a specific threshold. The computer program product of claim 12, wherein constructing the second state space representation of each HRIR of the plurality of HRIRs includes forming a transformation matrix that, when applied to the Gram matrix based on the first state space representation of that HRIR, generates a diagonal matrix wherein each diagonal element of the diagonal matrix corresponds to a respective eigenvalue of the plurality of eigenvalues. The computer program product of claim 11, wherein the method further comprises for each of the plurality of HRIRs: Creating a cepstrum for this HRIR, the cepstrum having causal samples taken at positive times and non-causal samples taken at negative times; Performing a phase minimization procedure for each of the non-causal samples of the cepstrum by adding that non-causal sample taken at a negative time to a causal sample of the cepstrum taken at the opposite point of that negative time; and generating a minimum phase HRIR by zeroing each of the non-causal samples of the cepstrum after performing the phase minimization operation for each of the non-causal samples of the cepstrum. The computer program of claim 1, wherein the method further comprises constructing a multiple input / multiple output (MIMO) state space representation, wherein the MIMO state space representation includes a composite matrix, a column vector matrix, and a row vector matrix, wherein the composite matrix is the MIMO state space representation. State space representation includes the matrix of the first representation of each of the many HRIRs, the column vector matrix of the MIMO state space representation includes the column vector of the first representation of each of the many HRIRs, and the row vector matrix of the MIMO state space representation includes the row vector of the first representation of each of the many HRIRs; and wherein traversing the state space reduction process includes creating a reduced composite matrix, a reduced column vector matrix and a reduced row vector matrix, wherein both the reduced composite matrix and the reduced column vector matrix and the reduced row vector matrix size each smaller than a size of the composite matrix, the column vector matrix and the row vector matrix. The computer program product of claim 15, wherein creating the MIMO state space representation includes: Forming a first block matrix as a composite matrix of the MIMO state space representation comprising a matrix of the first state space representation of an HRIR associated with a virtual speaker of the plurality of virtual speakers as a diagonal element of the first block matrix, wherein matrices of the first state space representation of HRIRs, which are associated with the same virtual speaker, are located in adjacent diagonal elements of the first block matrix. Forming a second block matrix as a column vector matrix of the MIMO state space representation having a column vector of the first state space representation of an HRIR associated with a virtual speaker of the plurality of virtual speakers as a diagonal element of the second block matrix, where column vectors are the first state space representation of HRIRs that are associated with the same virtual speaker, are in adjacent diagonal elements of the second block matrix; and Forming a third block matrix as a row vector matrix of the MIMO state space representation comprising a row vector of the first state space representation of an HRIR associated with a virtual speaker of the plurality of virtual speakers as an element of the third block matrix, wherein row vectors of the first state space representation of HRIRs, represent the sounds in the left ear, are in odd elements of the first row of the third block matrix, and row vectors of the first state space representation of HRIRs that represent sounds in the right ear are in even elements of the second row of the third block matrix. The computer program product of claim 11, wherein for each of the plurality of virtual speakers, there are left and right HRIRs of the plurality of HRIRs associated with that virtual speaker, the left HRIRs when generated with the one from the virtual speaker Is multiplied by the frequency domain sound field that produces the component of the sound field reproduced in the left ear of the human listener, and the right HRIR, when multiplied by the frequency domain sound field generated by that virtual speaker, produces the component of the sound field that is produced in the sound field right ear of the human listener is reproduced; and wherein there is an interaural time delay (ITD) between the left HRIR associated with this virtual speaker and the right HRIR associated with that virtual speaker for each of the plurality of virtual speakers, the ITD being in the left and right HRIR are manifested by a difference between a number of original samples of the sound field of the left HRIR having zero values and a number of original samples of the sound field of the right HRIR having zero values. The computer program product of claim 17, the method further comprising: constructing an ITD unit subsystem matrix based on the ITD between the left and right HRIR associated with each of the plurality of virtual speakers: and Multiplying the plurality of HRTFs by the ITD unit subsystem matrix to generate a plurality of delayed HRTFs. The computer program product of claim 11, wherein each of said plurality of HRTFs is represented by finite impulse response (FIR) filters; and wherein the method further comprises performing a conversion operation on each of the plurality of HRTFs to generate a further plurality of HRTFs, each represented by Infinite Impulse Response (IIRs) filters. An electronic device configured to reproduce sound fields in a left and a right ear of a human listener, the electronic device comprising: Storage; and control circuits connected to the memory, wherein the control circuits are configured for: Obtaining a plurality of outer ear impulse responses (HRIRs), wherein each of the plurality of HRIRs is associated with a virtual speaker of the plurality of virtual speakers and an ear of the human listener, each of the plurality of HRIRs including samples of a sound field at a specific sample rate in a left or right ear, generated in response to an audio pulse generated by that virtual speaker; Creating a first state space representation of each of the plurality of HRIRs, the first state space representation including a matrix, a column vector, and a row vector, wherein both the matrix and the column vector and the row vector of the first state space representation have a first size; Performing a state space reduction operation to generate a second state space representation of each of the plurality of HRIRs, the second state space representation including a matrix, a column vector, and a row vector, wherein both the matrix and the column vector, and the row vector of the second state space representation second size smaller than the first size; and Generating a plurality of outer ear transfer functions (HRTFs) based on the second state space representation, wherein each of the plurality of HRTFs corresponds to a respective HRIR of the plurality of HRIRs, wherein an HRTF corresponding to a respective HRIR as multiplied by a frequency domain sound field generated by the virtual speaker to which the corresponding HRIR is associated generates a component of a sound field reproduced in an ear of the human listener.

Images