Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04S—STEREOPHONIC SYSTEMS 
  • H04S1/00—Two-channel systems
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04S—STEREOPHONIC SYSTEMS 
  • H04S1/00—Two-channel systems
  • H04S1/002—Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04S—STEREOPHONIC SYSTEMS 
  • H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
  • H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04S—STEREOPHONIC SYSTEMS 
  • H04S5/00—Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation 

Definitions

• the present invention aims at processing sound signals for their spatialization.
• Spatial sound reproduction allows a listener to perceive sound sources coming from a direction or from any position in space.
• HRTF Head Related Transfer Functions
• HRIR Head Related Impulse Response
• the term "binaural” aims at restitution on a stereophonic headphones with nevertheless effects of spatialization.
• the present invention is not limited to this technique and also applies in particular to techniques derived from the binaural such as rendering techniques called “transaural”, that is to say on remote speakers.
• Such techniques can then use what is called a “crosstalk cancellation” (or “cross-talk cancellation”) which consists in canceling the acoustic cross paths so that a sound, thus processed and then emitted by the loudspeakers. speakers, can be perceived only by one of the two ears of a listener.
• this decomposition makes it possible to do encoding and decoding called "binaural multichannel".
• the decoding functions which are actually filters
• a set of spatial encoding functions which are in fact encoding gains
• when they are optimal in rendering ensure a feeling of immersion perfect for the listener inside a sound stage, whereas it actually has, for the binaural restitution, only two loudspeakers (headphones of a headphone or distant loudspeakers).
• the encoding is generally inexpensive in memory and / or in calculations since the spatial functions are gains that depend solely on the effects of the effects. sources to encode and not the number of sources themselves. Decoding also has a cost independent of the number of sources to be spatialised.
• the decoding functions can be individualized for each of the listeners.
• the present invention aims in particular an improved obtaining of decoding filters and / or encoding gains in binaural multichannel technique.
• the context is as follows: sources are spatialised by multichannel encoding and the restitution of the spatially encoded content is done by applying appropriate decoding filters.
• the reference WO-00/19415 discloses a binaural multichannel processing which provides for the calculation of decoding filters. By denoting: g t ( ⁇ p , ⁇ p ) fixed encoding spatial functions where g is the gain corresponding to the channel i ⁇ ⁇ , .., Nei at the position /?
• e 1, .., P defined by its angles of incidence ⁇ (azimuth) and ⁇ (elevation), h [ ⁇ p , ⁇ p , /) and R ⁇ ⁇ p , ⁇ p , f ⁇ bases of HRTF functions obtained by measuring the acoustic transfer functions of each ear L and R of an individual for a number P of positions of the space (pel, ..., P) and for a given frequency / it is provided in this document WO- 00/19415 essentially two steps to obtain filters from these spatial functions.
• each HRTF The delays of each HRTF are extracted. Indeed, the shape of a head is usually such that, for a given position, a sound arrives at one ear a certain time before reaching the other ear (a sound to the left arriving of course to the ear left before reaching the right ear).
• the delay difference t between the two ears is an interaural location index called ITD (for "Interaural Time Difference").
• ITD Interaural Time Difference
• decoding filters Z, ( (/) and R 1 (/) of channel i are obtained which satisfy the equations:
• a second approach, proposed in US-5,500,900, for jointly calculating the decoding filters and the spatial encoding functions is to decompose the HRIR sets by performing a principal component analysis (PCA) and then selecting a reduced number of components (which corresponds to the number of channels).
• PCA principal component analysis
• HRIR is very good with a small number of components.
• the techniques of the prior art require the extraction of delays HRIR.
• the techniques of sound recording or multi-channel encoding at a point in space are widely used since it is then possible to make transformations to the encoded signals (for example rotations).
• the delay information is not extractable from the signal alone.
• the decoding filters must then be able to reproduce the delays for an optimal sound reproduction.
• the number of channels can be low and the techniques of the prior art do not allow good decoding with few channels without extracting delays.
• the multichannel signal acquired may consist of only four channels, typically.
• ambiophonic microphones means microphones composed of coinciding directional sensors. The interaural delays must then be reproduced at decoding.
• the extraction of delays has at least two other major drawbacks: - the delays must be taken into account (addition of a step) at the time of the encoding, which increases the resources necessary in computation,
• the signals must be encoded for each ear and the number of filtering necessary for decoding is double.
• the present invention improves the situation.
• a sound spatialization method with multichannel encoding and binaural reproduction on two loudspeakers comprising a spatial encoding defined by encoding functions associated with a plurality of encoding channels and an application decoding. filters for binaural playback on the two speakers.
• the method according to the invention comprises the steps of: a) obtaining an original set of acoustic transfer functions specific to an individual morphology (HRIR, HRTF), b) choosing spatial encoding functions and / or filters decoding, and c) by successive iterations, optimizing the filters associated with the chosen encoding functions or the encoding functions associated with the selected filters, or together the filters and the encoding functions chosen, minimizing an error calculated according to a comparison between:
• the invention proposes the optimization calculation of the filters associated with a set of chosen encoding gains or encoding gains associated with a set of selected decoding filters, or a joint optimization of the decoding filters. and encoding gains.
• These filters and / or these gains have for example been fixed or initially calculated by the techniques of the pseudo-inverse or the virtual loudspeakers, described in particular in the document WO-00/19415. Then, these filters and / or the associated gains are improved, within the meaning of the invention, by an iterative optimization which aims to reduce a predetermined error function.
• the invention thus proposes the determination of decoding filters and encoding gains which allow both a good reconstruction of the delay but also a good reconstruction of the HRTF amplitude (modulus of HRTF), and this, for a weak number of channels, as will be seen with reference to the detailed description below.
• FIG. 1 illustrates the general steps of a method in the sense of the invention
• FIG. 2 illustrates the amplitude (gray levels) of the time functions HRIR (on several successive samples Ech) which have been chosen for the implementation implementation of step EO of FIG. 1, as a function of the azimuth (in degrees denoted deg °),
• FIG. 3 illustrates the appearance of some first spherical harmonics in ambiophonic context, as spatial encoding functions in a first embodiment
• FIGS. 4A, 4B, 4C compare the performances of the processing according to the first embodiment, for a non-optimized solution (FIG. 4A), for a solution partially optimized by a few processing iterations (FIG. 4B) and for a completely optimized solution. by the treatment in the sense of the invention (FIG. 4C),
• FIG. 5 illustrates the encoding functions in the virtual loudspeaker technique used in a second embodiment
• FIG. 6 compares a real average HRTF function (represented in solid line) with the reconstructed average HRTF functions using the solution of the pseudo-inverse in the sense of the prior art (represented by dotted lines), the starting solution given by the virtual loudspeaker method (shown in long broken lines) and the convergent optimized solution, in the sense of the second embodiment of the invention (shown in phantom),
• FIG. 7 compares the variations of the original ITD interaural delay (solid lines) with that obtained by the solution optimized in the sense of the second embodiment of the invention (mixed lines), with that reconstructed from the technique of the virtual speakers (long broken lines) and the one reconstructed from the filters obtained by the solution of the pseudo-inverse in the sense of the prior art (dashed lines), - Figure 8 schematically represents a spatialization system that can be obtained by the implementation of the first embodiment, taking into account interaural delays in encoding,
• FIG. 9 schematically represents a spatialization system that can be obtained by implementing the second embodiment, without taking into account the interaural delays in the encoding but including these delays in the decoding filters.
• the method within the meaning of the invention can be broken down into three steps:
• b) set spatial encoding functions and / or base filters, the encoding functions being denoted by g ( ⁇ , ⁇ , n) (or also g ( ⁇ , ⁇ , «, /)), where: ⁇ , ⁇ are the angles of incidence in azimuth and elevation,
• n is the index of the encoding channel considered
• obtaining the HRTF of the second ear can be deduced from the measurement of the first ear by symmetry.
• the set of functions HRIR can for example be measured on a subject by positioning microphones at the entrance of his ear canal.
• this HRIR game can also be calculated by numerical simulation methods (modeling of the morphology of the subject or calculation by network of artificial neurons) or having been the subject of a chosen treatment (reduction of the number of samples, phase correction, or other).
• step a) it is possible in this step a) to extract the delays from the HRIRs, to store them and then to add them at the time of the spatial encoding, the steps b) and c) remaining unchanged.
• This first step a) has the reference EO in FIG.
• step b if one seeks to obtain optimized filters on the one hand, it is necessary to set the spatial encoding functions g ( ⁇ , ⁇ , n) (or g ( ⁇ , ⁇ , n, f)) and, to obtain optimized spatial functions, the decoding filters denoted F (t, n) must be fixed.
• the spatial encoding functions are fixed, they are then reproducible and universal and the individualization of the filters is simply decoding. Moreover, the spatial encoding functions, when they comprise a large number of zeros among n encoding channels as in the second embodiment described below, make it possible to limit the number of operations during encoding. Pan-intensity laws between two-dimensional virtual loudspeakers and their three-dimensional extensions can be represented by encoding functions with only two non-zero gains, at most, for two dimensions and three non-zero gains for three dimensions, for a single given source. The number of non-zero winnings is, of course, independent of the number of channels and, most importantly, the zero winnings make it possible to lighten the encoding calculations.
• Spherical harmonic space functions in ambiophonic context have mathematical qualities that make it possible to transform the encoded signals (for example rotations of the sound field).
• such functions provide compatibility between binaural decoding and surround sound recordings based on a decomposition of the sound field into spherical harmonics.
• the encoding functions may be real or simulated directivity functions of microphones to allow listening of binaural multichannel recordings.
• the encoding functions can be arbitrary (non-universal) and determined by any method, the rendering then having to be optimized during subsequent steps of the method within the meaning of the invention.
• Spatial functions may also be a function of time or frequency.
• the optimization will be done taking into account this dependence (for example by optimizing independently each time sample or frequency).
• these can be fixed so that the decoding can be universal.
• Decoding filters can also be chosen to reduce the resource cost of filtering. For example, the use of filters called “infinite impulse response” or "HR” is advantageous.
• the decoding filters can also be chosen according to a psychoacoustic criterion, for example constructed from standardized Bark bands.
• the decoding filters can be determined by any method.
• the rendering, in particular for an individual listener, can then be optimized during the next steps of the method relating to the encoding functions.
• This second step b) relating to the calculation of an initial solution SO bears the reference E1 in FIG. 1.
• it consists in choosing the decoding filters (referenced “F”) and / or the spatial encoding functions ( referenced “g") and determine an initial solution SO for the encoding functions or the decoding filters, by a method also chosen.
• the filters of the starting solution SO at step E1 may be directly the HRIR functions given to the corresponding positions of the virtual speakers.
• the starting solution SO being further determined by functions defining the pan-effect laws. as encoding functions and by the HRIR functions, themselves, given to the positions of the virtual loudspeakers, as decoding filters.
• the decoding filters in step E1 are calculated from the pseudo-inverse to determine the starting solution SO. More generally, the starting solution SO in step E1 can be calculated from the least squares solution:
• the elements F, HRIR and g are matrices.
• the starting solution SO can be arbitrary (random or fixed), the essential being that it leads to obtaining a converged solution SC in step E6 of FIG.
• FIG. 1 also illustrates the operations E2, E3, T4, E5, E6 of general step c), optimization within the meaning of the invention.
• this optimization is conducted by iterations.
• step E3 the calculation of an error function is an important point of the optimization method within the meaning of the invention.
• a proposed error function is to simply minimize the difference in modules between the HRTF * Fourier transform of the reconstructed HRIR function set and the HRTF Fourier transform of the original HRIR function set (given in step EO).
• the error function can also minimize the energy difference between the modules, ie:
• any error function calculated entirely or in part from the HRIR functions can be provided (module, phase, delay or estimated ITD, interaural differences, or other).
• the optimization iterations can be successively applied to each frequency sample, with the advantage of then reducing the number of simultaneous variables, to have an error function specific to each frequency / and to meet a stop criterion according to the convergence specific to each frequency.
• Step T4 is a test for stopping or not the iteration of the optimization according to a chosen stopping criterion. It may be a criterion characterizing the fact that:
• variable c has reached a minimum value ⁇ , and / or that
• the filters F (n, t) or the gains g ( ⁇ , ⁇ , n) or the calculated filter / gain pairs make it possible to obtain an optimal spatial rendering, as will be seen in particular with reference to Figure 4C or Figure 6 below.
• the treatment then stops by obtaining a converged solution (step E6).
• this embodiment illustrated in FIG. 1 applies just as well when it was chosen to fix the decoding filters in step E1, and then to optimize the spatial encoding functions during the steps E2, E3. , E5, E6. It also applies when has been chosen to iteratively optimize both the encoding functions and the decoding filters.
• a set of HRIR functions measured for the left ear in the deaf chamber and for 64 different azimuth angle values ranging from 0 to about 0 are used.
• the HRIRs of the right ear are the symmetries of the HRIRs of the left ear.
• the HRIR functions can be obtained from standard databases ("Kemar head") or by modeling the morphology of the individual, or the like.
• the starting solution SO for step E1 is given by calculating the pseudo-inverse (with linear resolution).
• This starting solution constitutes the decoding solution which was proposed as such in the document WO-00/19415 of the prior art described above.
• the optimization technique used in the sense of the invention is preferably that of the gradient described above.
• the error function c used corresponds to the least squares on the module of the Fourier transform of the HRIR functions, namely:
• FIGS. 4A, 4B, 4C show the time course (over a few tens of time samples) of the five decoding filters and the module reconstruction errors (in dB, illustrated by gray levels) and the phase ( in radians, illustrated by gray levels) of the Fourier transform of the HRIR functions for each position (ordinates indicated in azimuth) and for each frequency (abscissa located in frequencies), respectively:
• Panoramic laws are commonly used by sound technicians to produce audio content, including multichannel content in so-called “surround” formats that are used in sound reproduction 5.1, 6.1, or other.
• panning encoding by panning laws is achieved by mixing a sound environment in a "surround” format (tracks 5.1 of a digital recording for example). Optimized filters from the same panning laws then allow for optimal binaural decoding for the desired rendering with this "surround" effect.
• the present invention is advantageously applied to the case where the positions of the virtual speakers correspond to positions of a multichannel rendering system for the general public, with a "surround” effect.
• the optimized decoding filters then allow decoding of multimedia consumer content (typically multi-channel content with "surround” effect) for playback on two speakers, for example on binaural headphones.
• This binaural rendering of a content that is for example initially in 5.1 format is optimized thanks to the implementation of the invention.
• the HRIR functions are obtained at 64 positions around the listener, as described with reference to the first embodiment above.
• R is the gain of the right speaker
• ⁇ v is the angle for which it is desired calculate the gains (typically the angle between the plane of symmetry of the two speakers and the desired direction).
• the optimization method used in the second embodiment is still that of the gradient.
• the starting solution SO at step E1 is given by the ten decoding filters which correspond to the ten HRIR functions given to the positions of the loudspeakers. virtual speakers.
• the fixed spatial functions are the encoding functions representing the panning laws.
• the error function c is based on the module of the Fourier transform of the HRIR functions, namely:
• FIG. 6 compares a real HRTF function (shown in solid lines), averaged over a set of 64 measured positions (for azimuth angles ranging from 0 to about 350 °), to the average HRTF functions reconstructed using: - the pseudo-inverse starting solution, without optimization (represented in dotted lines),
• Figure 7 illustrates the variations of the interaural delay ITD as a function of the azimuth position of the HRIR functions.
• the optimized solution makes it possible to reconstruct a delay ITD (mixed lines) relatively close to the original ITD (solid lines), but just as close as that reconstructed from the initial solution, here obtained by the technique of the loudspeakers virtual (long broken lines).
• the reconstructed ITD delay from the filters obtained by linear (pseudo-inverse) resolution, represented by dashed lines in FIG. 7, is rather irregular and remote from the original ITD.
• the case treated in the example described here is that of two spatially distinct sources to encode multichannel and restore binaural.
• the two exemplary embodiments of FIGS. 8 and 9 use the symmetry properties of the HRIR functions.
• FIG. 9 corresponds to the case where the encoding gains are obtained by applying the virtual loudspeaker method according to the second embodiment described above.
• Figure 8 shows an implementation of multichannel encoding and decoding when delays are not included in the decoding filters but must be taken into account as soon as encoding. It may correspond to that of the prior art described hereinbefore WO-00/19415, provided that the decoding filters (and / or the encoding functions) have not been optimized within the meaning of the invention. .
• FIG. 8 consists, in generic terms, of extracting, from the transfer functions obtained in step a), interaural delay information, while that the optimization, within the meaning of the invention, encoding functions and / or decoding filters is conducted here from the transfer functions from which these delay information has been extracted. Then, these interaural delays can be stored and then applied later, in particular to the encoding.
• FIGS. 8 and 9 the same notations S 1 and S 2 have, of course, been adopted for the two sources to be encoded, each being placed at a given position in space.
• ⁇ ⁇ ⁇ D and ⁇ ⁇ 2 D denote the delays (ITD) corresponding to the positions of the sources S 1 and S 2 .
• ITD delays
• both sounds are supposed to arrive at the right ear before reaching the left ear.
• F ⁇ The decoding filter for channel j and F JL symmetrical filters filters F, L. It is indicated here that in the case of virtual loudspeakers, the symmetrical filter of a given virtual loudspeaker (a given channel) is the filter of the virtual symmetrical loudspeaker (considering the left / right plane of symmetry of the head).
• L and R are the left and right binaural channels.
• the ITD delay is introduced at the time of encoding, the multichannel signals for the left channel are different from those for the right channel.
• the consequences of the introduction of coding delays are therefore the doubling of the number of encoding operations and the doubling of the number of channels, compared to the second implementation illustrated in FIG. 9 and taking advantage of the advantages offered by the second embodiment of the invention.
• each signal coming from a source S 1 in the encoding block ENCOD is split in order to apply to one of them a delay (positive or negative) ⁇ I ⁇ TD , T ⁇ 2 0 , and each doubled signal is multiplied by each gain g ⁇ ' L , the multiplication results being then grouped by channel index y (n channels) and whether interaural delay has been applied or not (2 times n channels in total).
• the 2n signals obtained are conveyed through a network, stored, or otherwise, for restitution and, for this purpose, are applied to a DECOD decoding block comprising n filters F JL for a left channel L and n filters. symmetrical F JL for a straight line R.
• the symmetry of the filters results from the fact that we consider a symmetry of HRTF functions.
• the signals to which the filters are applied are grouped together in each channel and the signal resulting from this grouping is intended to supply one of the two speakers with playback on two distant loudspeakers (in which case it is necessary to add an operation of cross paths cancellation) or directly one of the two channels of a headset with auricles in binaural restitution.
• FIG. 9 shows an implementation of multichannel encoding and decoding when the delays are, on the contrary, included in the decoding filters in the sense of the second embodiment using the virtual loudspeaker method. and exploiting the observation resulting from Figures 6 and 7 above.
• each sum or each difference of filters is to be considered as a filter in itself. What is indicated here as being a sum or a difference of filters is to be considered in relation to the expressions of the filters F JL and F j L described above with reference to FIG. 8.
• the decoding processing of FIG. 9 continues with a grouping of the sums SS and a grouping of the differences SD supplying by their sum the channel L (module SL delivering the signal SS + SD) and by their difference the channel R (module DR delivering the SS-SD signal).
• the useful working memory (buffer) for the implementation of FIG. 8 requires more than twice that useful for the implementation of FIG. 9, since 2n channels transit between the encoding and the decoding and it is necessary to implement a delay line by source in the implementation of FIG. figure 8.
• the present invention thus aims at a sound spatialization system with multichannel encoding and for a two-channel reproduction comprising an ENCOD spatial encoding block defined by encoding functions associated with a plurality of encoding channels and a decoding block. DECOD by applying filters for binaural rendition.
• the spatial encoding functions and / or the decoding filters are determined by the implementation of the method described above.
• Such a system may correspond to that illustrated in FIG. 8, in one embodiment for which the delays are integrated at the time of encoding, which corresponds to the state of the art within the meaning of document WO-00/19415.
• Another advantageous embodiment consists in implementing the method according to the second embodiment to then build a spatialization system with a direct encoding block, without applying a delay, so as to reduce a number of encoding channels and a corresponding number of decoding filters, which directly include the ITD interaural delays, according to an advantage offered by the implementation of the invention, as illustrated in FIG. 9.
• This embodiment of FIG. 9 makes it possible to achieve a quality of spatial rendering that is at least as good, if not better, than the techniques of the prior art, and this, with a number of filters half as great and a lower calculation cost. . Indeed, as shown with reference to FIGS. 6 and 7, in the case where the decomposition is aimed at a set of HRIR functions, this embodiment allows a quality of reconstruction of the HRTF module and the interaural delay better than the techniques of the prior art with a reduced number of channels.
• the present invention also provides a computer program comprising instructions for implementing the method described above and whose algorithm can be illustrated by a general flowchart of the type shown in Figure 1.

Landscapes

• Physics & Mathematics (AREA)
• Engineering & Computer Science (AREA)
• Acoustics & Sound (AREA)
• Signal Processing (AREA)
• Stereophonic System (AREA)

Applications Claiming Priority (2)

Publications (2)

Family

ID=37452726

Family Applications (1)

Country Status (3)

Families Citing this family (20)

Family Cites Families (9)

• 2007
  • 2007-03-01 EP EP07731684.2A patent/EP1992198B1/de active Active
  • 2007-03-01 WO PCT/FR2007/050867 patent/WO2007101958A2/fr active Application Filing
  • 2007-03-01 US US12/224,840 patent/US9215544B2/en active Active

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events