EP3400599A1

EP3400599A1 - Improved ambisonic encoder for a sound source having a plurality of reflections

Info

Publication number: EP3400599A1
Application number: EP16808645.2A
Authority: EP
Inventors: Pierre Berthet
Original assignee: 3d Sound Labs
Current assignee: Mimi Hearing Technologies GmbH
Priority date: 2016-01-05
Filing date: 2016-12-08
Publication date: 2018-11-14
Anticipated expiration: 2036-12-08
Also published as: CN108701461B; FR3046489B1; WO2017118519A1; US20200058312A1; FR3046489A1; CN108701461A; US11062714B2; EP3400599B1; US10475458B2; US20190019520A1

Abstract

The present invention relates to an ambisonic encoder for a sound wave having a plurality of reflections. The claimed ambisonic encoder improves the sensation of immersion in a 3-D audio scene. The complexity of encoding of reflections of sound sources for an ambisonic encoder according to the invention is less than the complexity of encoding of the reflections of sound sources of an ambisonic encoder according to the prior art. The claimed ambisonic encoder can encode a greater number of reflections of a sound source in real time. The claimed ambisonic encoder can reduce the power consumption related to ambisonic encoding, and can increase the life of a battery of a mobile device used for said application.

Description

IMPROVED AMBASSIC ENCODER OF SOUND SOURCE A

PLURALITY OF REFLECTIONS

FIELD OF THE INVENTION

The present invention relates to ambisonic encoding of sound sources. It concerns more specifically the improvement of the efficiency of this coding, in the case where a sound source is affected by reflections in a sound scene.

STATE OF THE ART PREVIOUS

Spatial representations of sound include synthetic capture techniques and sound environment reproduction for a much greater immersion of the listener in a sound environment. In particular, they allow a user to discern a number of sound sources greater than the number of loudspeakers he has, and to precisely locate these sound sources in 3D, even when their direction is not that of a loudspeaker. The applications of spatialized representations of sound are numerous, and include the precise location of sound sources in 3 dimensions by a user from a sound coming from a stereo headset, or the location of 3-dimensional sound sources by users in a room, the sound being emitted by speakers, for example 5.1 speakers. In addition, the spatialized representations of sound allow the realization of new sound effects. For example, they allow the rotation of a sound stage or the application of reflection of a sound source to simulate the rendering of a given sound environment, for example a movie theater or a concert hall.

The spatialized representations are carried out in two main steps: an ambisonic encoding, and an ambisonic decoding. To benefit from a spatial representation of the sound, ambisonic decoding in real time is always necessary. Real-time sound production or processing may also involve real-time ambisonic encoding of the sound. Ambisonic encoding being a complex task, ambisonic encoding capabilities in real time may be limited. For example, a given computing capacity can only be able to encode in real time a limited number of sound sources.

Spatial representation techniques of sound are described by J. Daniel, Acoustic field representations, application to the transmission and reproduction of sound scenes in a multimedia context, INIST-CNRS, Cote INIST: T 1 39957. The ambisonic encoding of a sound field consists of the decomposition of the sound pressure field at a point, corresponding for example to the position of a user, in the form of spherical coordinates, expressed in the following form:

In which p (r, t) represents the sound pressure, at a time t, in the direction r with respect to the point at which the sound field is calculated. j ^m represents the Bessel spherical function of order m.

[0005] Y _m n (Q _> <P) represents the spherical harmonic of order mn in the directions (θ, φ, defined by the direction r) The symbol B _mn t) defines the ambisonic coefficients corresponding to the various spherical harmonics , at a moment t.

The ambison coefficients thus define, at each moment, the entire sound field surrounding a point. The treatment of sound fields in the ambisonic domain has particularly interesting properties. In particular, it is very easy to rotate the entire sound field. It is further possible to broadcast on loudspeakers, from a set of ambisonic coefficients, sound including direction information. For example, it is possible to broadcast sound on 5.1 type speakers. It is also possible to reproduce, in headphones having only a left speaker and a right speaker, sound with directional information, using transfer functions known as HRTF ( Head-Related Transfer Functions, or Head-Related Transfer Functions). These functions make it possible to reproduce a directional signal on two loudspeakers, by adding to at least one channel of a stereo signal a delay and / or attenuation, which will be interpreted by the brain as defining the direction of the sound source.

The so-called HOA (Higher Order Ambisonics) decomposition consists in truncating this infinite sum to an order M, greater than or equal to 1:

p (, t) = (t) Y _mn (e, cp)

In general, a sufficiently distant source is considered to propagate a sound wave in a spherical manner. It is then possible to consider that the value at an instant t of an ambisonic coefficient B _mn (t) linked to this source depends, on the one hand, on the sound pressure 5 (t) of the source at this instant t, and on the other hand the spherical harmonic linked to the orientation (0 _s , <p _s ) of this sound source. We can write, for a single sound source:

In the case of a set of N _s distant sound sources, the ambisonic coefficients describing the sound scene are calculated as the sum of the ambison coefficients of each of the sources, each source i having an orientation 0 _if , <p _si ) :

This calculation may also be represented in vector form:

The ambisonic coefficients keeping the form B _mn , with, at the order M, m ranging from 0 to M, and n ranging from -m to m.

An apparatus comprising an ambisonic encoding of at least one source can therefore define a complete sound field, by calculating the ambisonic coefficients at an order M. Depending on the order M, and the number of sources, this calculation can to be long and greedy in resource. Indeed, at an order M, (M + 1) ² ambisonic coefficients are calculated at each instant t. For each coefficient, the contribution = ^s (t) Y _mn (9 _s , (p _s ) of each of the N _s sources must be calculated If a source S is fixed, the spherical harmonic K _mn (0 _s , <p _s ) can be precalculated. Otherwise, it must be recalculated at every moment.

An increase in the order of the ambisonic coefficient allows a better quality of the auditory rendering. It can therefore be difficult to obtain good sound quality, while preserving a reasonable load, computing time, power consumption and battery usage. This is especially true since calculations of ambison coefficients are often performed in real-time on mobile devices. This is for example the case of a smartphone, for listening to music in real time, with directional information calculated using ambison coefficients.

This problem is even stronger when reflections are calculated in a sound scene.

The calculation of reflections can simulate a sound scene in a room, for example a movie theater or concert. Under these conditions, the sound is reflected on the walls of the room, giving a characteristic "atmosphere", the reflections being defined by the respective positions of the sound sources, the listener, but also by the materials on which the sound waves are diffuse, for example the material of the walls. The creation of room effects using ambisonic audio coding is described by J. Daniel, Acoustic field representations, application to the transmission and reproduction of sound scenes in a multimedia context, INIST-CNRS, INIST Rating: T 1 39957, pp. 283-287.

It is possible to simulate the effect of the reflections and to give a "mood" ambisonie, adding, for each sound source, a set of secondary sound sources, whose intensity and direction are calculated from the reflections of sound sources on the walls and obstacles of a sound stage. Some sound sources are needed, for each initial sound source, in order to satisfactorily simulate a sound stage. However, this makes the above-mentioned problem of computing capacity and battery even more critical, since the computational complexity of the ambison coefficients is further multiplied by the number of secondary sound sources. The complexity of the computation of the ambisonic coefficients for a satisfactory sound rendering can then make this solution impractical, for example because it becomes impossible to calculate the ambison coefficients in real time, because the calculation load of ambisonic coefficients becomes too important, or because the power consumption and / or battery on a mobile device becomes prohibitive.

N. Tsingos et al. Perceptional Audio Rendering of Complex Virtual Environment, ACM Transactions on Graphics (TOG) - Proceedings of ACM SIGGRAPH 2004, Volume 23 Issue 3, August 200, pp. 249-258 discloses a binaural processing method to overcome this problem. The solution proposed by Tsingos is to reduce the number of sound sources by:

- Evaluating the power of each sound source;

- classifying sound sources, from the most to the least powerful;

- Removing the least powerful sound sources;

- Grouping the remaining sound sources in clusters of sound sources close to each other, and merging them to obtain, for each cluster, a single virtual sound source.

The method disclosed by Tsingos reduces the number of sound sources, and therefore the complexity of the overall processing when reverberations are used. However, this technique has several disadvantages. It does not improve the complexity of the processing of the reverberations themselves. The problem encountered would therefore arise again, if, with a small number of sources, it is desired to increase the number of reverberations. In addition, the processing to determine the sound power of each source, and merging the sources in clusters themselves have a significant computing load. The experiments described are limited to cases where sound sources are known in advance, and their respective powers pre-calculated. In cases of sound scenes for which several sources of variable intensities are present, and whose powers must be recalculated, the associated calculation load would at least partially cancel the calculation gain obtained by limiting the number of sources.

Finally, the tests conducted by Tsingos give satisfactory results when the sound sources are similar to noise, for example in the case of a crowd in the subway. On other types of sound sources, such a method could be harmful. For example, when recording a concert given by an orchestra symphonic, it is common that several instruments, although having a low sound power, contribute significantly to the overall harmony. Simply removing the associated sound sources, because they are relatively weak, would then seriously affect the quality of the recording.

There is therefore a need for an apparatus and a method for calculating ambisonic coefficients, which makes it possible to calculate in real time a set of representative ambisonic coefficients of at least one sound source and one or more reflections of this one in a sound scene, while limiting the complexity of additional computation related to the sound or reflections of the sound source, without reducing a priori the number of sound sources.

SUMMARY OF THE INVENTION [0020] For this purpose, the invention relates to an ambisonic sound wave encoder with a plurality of reflections, comprising: a frequency conversion logic of the sound wave; a logic for calculating spherical harmonics of the sound wave and the plurality of reflections from a position of a source of the sound wave and of positions of obstacles to propagation of the sound wave; a plurality of frequency domain filtering logic receiving as input spherical harmonics of the plurality of reflections, each filtering logic being parameterized by acoustic coefficients and reflection delays; a logic of addition of spherical harmonics of the sound wave and the outputs of the filtering logic.

Advantageously, the calculation logic of spherical harmonics of the sound wave is configured to calculate the spherical harmonics of the sound wave and the plurality of reflections from a fixed position of the source of the wave. sound.

Advantageously, the calculation logic of spherical harmonics of the sound wave is configured to iteratively calculate the spherical harmonics of the sound wave and the plurality of reflections from successive positions of the source of the sound wave. sound wave.

[0023] Advantageously, each reflection is characterized by a single acoustic coefficient. [0024] Advantageously, each reflection is characterized by an acoustic coefficient for each frequency of said frequency sampling.

[0025] Advantageously, the reflections are represented by virtual sound sources.

Advantageously, the ambisonic encoder further comprises a logic for calculating the acoustic coefficients, delays and the position of virtual sound sources of the reflections, said calculation logic being configured to calculate the acoustic coefficients and the delays of the reflections. based on estimates of a distance difference traveled by the sound between the position of the source of the sound wave and an estimated position of a user on the one hand, and a distance traveled by the sound between the positions of the virtual sound sources reflections and the estimated position of the user on the other hand.

Advantageously, the logic for calculating the acoustic coefficients, the delays and the positions of the virtual sound sources of the reflections is furthermore configured to calculate the acoustic coefficients of the reflections as a function of at least one acoustic coefficient of at least one obstacle. to the propagation of sound waves, on which the sound is reflected.

Advantageously, the logic for calculating the acoustic coefficients, the delays and the positions of the virtual sound sources of the reflections is furthermore configured to calculate the acoustic coefficients of the reflections as a function of an acoustic coefficient of at least one obstacle to the propagation of sound waves, on which the sound is reflected.

Advantageously, the calculation logic of spherical harmonics of the sound wave and the plurality of reflections is further configured to calculate spherical harmonics of the sound wave and the plurality of reflections at each output frequency of the sound wave. frequency conversion circuit, said ambisonic encoder further comprising a logic for calculating binaural coefficients of the sound wave, configured to calculate binaural coefficients of the sound wave by multiplying at each output frequency of the frequency transformation circuit of the sound wave the signal of the sound wave by the spherical harmonics of the sound wave and the plurality of reflections at this frequency. Advantageously, the calculation logic of the acoustic coefficients, delays and positions of the virtual sound sources of the reflections is configured to calculate acoustic coefficients and delays of a plurality of late reflections.

The invention also relates to an ambisonic sound wave encoding method with a plurality of reflections, comprising: a frequency transformation of the sound wave; a calculation of spherical harmonics of the sound wave and the plurality of reflections from a position of a source of the sound wave and of positions of obstacles to propagation of sound waves; filtering, by a plurality of filtering logic in the frequency domain, spherical harmonics of the plurality of reflections, each filtering logic being parameterized by acoustic coefficients and reflection delays; an addition of spherical harmonics of the sound wave and the outputs of the filtering logic.

The invention also relates to a computer program for ambisonic sound wave encoding with a plurality of reflections, comprising: computer code instructions configured to perform a frequency transformation of the sound wave; computer code instructions configured to calculate spherical harmonics of the sound wave and the plurality of reflections from a position of a source of the sound wave and from positions of obstacles to a propagation of the sound wave 'sound wave ; computer code instructions configured to parameterize a plurality of frequency domain filtering logic receiving as input spherical harmonics of the plurality of reflections, each filter logic being parameterized by acoustic coefficients and reflection delays; computer code instructions configured to add spherical harmonics of the sound wave and outputs of the filtering logic.

The ambisonic encoder according to the invention makes it possible to improve the immersion sensation in a 3D audio scene.

The complexity of encoding the reflections of sound sources of an ambisonic encoder according to the invention is less than the complexity of encoding the reflections of sound sources of an ambisonic encoder according to the state of the art. The ambisonic encoder according to the invention makes it possible to encode a greater number of reflections of a sound source in real time.

The ambisonic encoder according to the invention reduces the power consumption related to ambisonic encoding, and increase the life of a battery of a mobile device used for this application.

LIST OF FIGURES

Other features will become apparent on reading the detailed description given by way of example and not limiting thereafter made with reference to appended drawings which represent:

FIGS. 1a and 1b, two examples of sound wave listening systems, according to two embodiments of the invention

FIG. 2, an example of a binauralization system comprising a sound source binauralization engine of an audio scene according to the state of the art;

FIGS. 3a and 3b, two examples of binauralization engines of a 3D scene, respectively in the time domain and the frequency domain according to the state of the art

FIG. 4, an example of an ambisonic encoder of a sound wave with a plurality of reflections, in a set of embodiments of the invention;

FIG. 5, an example of calculation of a secondary sound source, in one embodiment of the invention;

FIG. 6, an example of calculation of early reflections and late reflections, in one embodiment of the invention;

FIG. 7, a method of encoding a sound wave at a plurality of reflections in a set of modes of implementation of the invention.

DETAILED DESCRIPTION

Figures 1a and 1b show two examples of sound wave listening systems, according to two embodiments of the invention. Figure 1a shows an example of a sound wave listening system, according to one embodiment of the invention.

The system 100a includes a touch pad 1 10a, a headset 120a to allow a user 130a to listen to a sound wave. The system 100a includes, by way of example only, a touch pad. However, this example is also applicable to a smartphone, or any other mobile device with display and sound broadcasting capabilities. The sound wave can for example be derived from the reading of a film or a game. According to several embodiments of the invention, the system 100a can be configured to listen to several sound waves. For example, when the system 100a is configured for the playback of a film comprising a 5.1 multichannel sound track, 6 sound waves are listened to simultaneously. In the same way, when the system 100a is configured to play a game, many sound waves can be listened to simultaneously. For example, in the case of a game involving several characters, a sound wave can be created for each character.

Each of the sound waves is associated with a sound source whose position is known.

The touch pad 1 10a comprises an ambisonic encoder 1 1 1 a according to the invention, a transformation circuit 1 12a, and an ambisonic decoder 1 13a.

According to a set of embodiments of the invention, the ambisonic encoder 1 1 1 a, the transformation circuit 1 12a and the ambisonic decoder 1 13 a consist of computer code instructions executed on a processor. of the touch pad. For example, they may have been obtained by installing an application or specific software on the tablet. In other embodiments of the invention, at least one of the ambisonic encoder 1 1 1 a, the transformation circuit 1 12a and the ambisonic decoder 1 13 a is a specialized integrated circuit, for example an ASIC ( English acronym "Application-Specific Integrated Circuit," an FPGA (English acronym for Field Programmable Gate Array).

The ambisonic encoder 1 1 1 a is configured to calculate, in the frequency domain, a set of ambison coefficients representative of the whole of a sound scene, from at least one sound wave. It is further configured to apply reflections to at least one sound wave, to simulate a listening environment, for example a movie theater of a certain size, or a concert hall.

The transformation circuit 1 12a is configured to perform rotations of the sound scene by modifying the ambison coefficients, to simulate the rotation of the user's head, so that, regardless of the orientation of his face, the different sound waves appear to him from the same position. For example, if the user turns his head to the left of an angle, a rotation of the sound stage to the right of the same angle a continues to send the sound always the same direction. According to a set of embodiments of the invention, the helmet 120a is equipped with at least one motion sensor 121a, for example a gyrometer, making it possible to obtain an angle or a derivative of a rotation angle of the the head of the user 130a. A signal representative of an angle of rotation, or a derivative of a rotation angle, is then sent by the helmet 121a to the tablet 120a, so that the transformation circuit 1 12a rotates the sound stage corresponding.

The ambisonic decoder 1 13a is configured to reproduce the sound scene on the two stereo channels of the headphones 120a, converting the transformed ambisonic coefficients into two stereo signals, one for the left channel and the other for the right channel. . In one set of embodiments of the invention, ambisonic decoding is performed using functions called HRTF (Head Related Transfer Functions), allowing to render, on two stereo channels, the directions of the different sound sources. French Patent Application No. 1558279, filed by the applicant, describes a method for creating HRTF functions optimized for a user based on a HRTF function bank, and facial characteristics of said user.

The 100a system thus allows the user to benefit from a particularly immersive experience: during a game or a reading of a multimedia content, in addition to the image, this system allows him to benefit from an impression of immersion in a sound stage. This impression is amplified both by the follow-up of the orientations of the different sources sound when the user turns his head, and by the application of reflections giving an impression of immersion in a particular listening environment. This system makes it possible, for example, to watch a movie or a concert with an audio headset, having a feeling of immersion in a movie theater or a concert hall. All of these operations are performed in real time, which allows to constantly adapt the sound perceived by the user to the orientation of his head.

The ambisonic encoder 1 1 1 according to the invention makes it possible to encode a greater number of reflections of sound sources, with less complexity compared to an ambisonic encoder of the prior art. It thus makes it possible to carry out all the ambisonic calculations in real time, while increasing the number of reflections of the sound sources. This increase in the number of reflections allows to model more precisely the simulated listening environment (concert hall, cinema ...) and thus to improve the feeling of immersion in the sound stage. Reducing the complexity of the ambisonic encoding also makes it possible, considering an identical number of sound sources, to reduce the power consumption of the encoder with respect to a state-of-the-art encoder, and therefore to increase the unloading time of the battery of the touch pad 1 10a. This allows the user to enjoy multimedia content for a longer period of time.

Figure 1b shows a second example of a sound wave listening system, according to one embodiment of the invention.

The system 100b comprises a central unit 1 10b connected to a screen 1 14b, a mouse 1 15b and a keyboard 1 1 6b and a headset 120b and is used by a user 130b. The central unit comprises an ambisonic encoder 1 1 1 b according to the invention, a transformation circuit 1 12b, and an ambisonic decoder 1 13b, respectively similar to the ambisonic encoder 1 1 1 a, transformation circuit 1 12a, and ambisonic decoder 1 13a of the system 100a. In a similar way to the system 100a, the ambisonic encoder 11 1b is configured to encode at least one representative wave of a sound scene by adding reflections, the headset 120a comprises at least one motion sensor 120b, the 120b transformation is configured to perform rotations of the scene in order to follow the orientation of the user's head, and the ambisonic decoder 1 13b is configured to reproduce the sound on the two stereo channels of the headphone 120b, so that the user 130b has an impression of immersion in a sound stage.

The system 100b is suitable for viewing multimedia content, but also for the video game. Indeed, in a video game, very many sound waves from different sources can occur. This is for example the case in a strategy game or war, in which many characters can emit different sounds (footsteps, running, shooting ...) for various sound sources. An ambisonic encoder 1 1 1b can encode all these sources, while adding many reflections making the scene more realistic and immersive, in real time. Thus, the system 100b comprising an ambisonic encoder 1 1 1 b according to the invention allows an immersive experience in a video game, with a large number of sound sources and reflections.

FIG. 2 represents an example of a binauralization system comprising a sound source binauralization engine of an audio scene according to the state of the art.

The binauralization system 200 is configured to transform a set 210 of sound sources of a sound stage into a left channel 240 and a right channel 241 of a stereo listening system, and includes a set of binaural motors 220 , comprising a binaural motor by sound source.

The sources can be of any type of sound sources (mono, stereo, 5.1, multiple sound sources in the case of a video game for example). Each sound source is associated with an orientation in space, for example defined by angles {θ, φ) in a repository, and by a sound wave, itself represented by a set of temporal samples.

Each binauralization engine of the set 220 is configured for, for a sound source and at each instant t corresponding to a sample of the sound source:

- perform an HOA encoding of the sound source to an order M; perform a transformation on the binaural coefficients, for example a rotation;

calculating a loudness p (r, i) at times t for a set of output channels, in which r represents the orientation of the output channel.

The possible output channels correspond to different listening channels, one can for example have two output channels in a stereo listening system, 6 output channels in a 5.1 listening system, etc.

Each binauralization engine produces two outputs (a left output and a right output), and the system 200 comprises an addition circuit 230 of all the left outputs and an addition circuit 231 of all the outputs of the right. 220 set of binauralization engines. The outputs of the addition logic 230 and 231 are respectively the sound wave of the left channel 240 and the sound wave of the right channel 241 of a stereo listening system.

The system 200 makes it possible to transform the set of sound sources 210 into two stereo channels, while being able to apply all the transformations allowed by the ambisonie, such as rotations.

However, the system 200 has a major disadvantage in terms of computation time: it requires calculations to calculate the ambison coefficients of each sound source, calculations for the transformations of each sound source, and calculations for the associated outputs. to each sound source. The computing load for the processing of a sound source by the system 200 is therefore proportional to the number of sound sources, and can, for a large number of sound sources, become prohibitive.

Figures 3a and 3b show two examples of binauralization engines of a 3D scene, respectively in the time domain and the frequency domain according to the state of the art.

FIG. 3a represents an example of binauralization engine of a 3D scene, in the time domain according to the state of the art.

In order to limit the complexity of the binaural processing in the case of a large number of sources, the binauralization engine 300a comprises a unique HOA 320a encoder engine for all sources 310 of the sound stage. This encoding engine 320a is configured to calculate, at each time step, the binaural coefficients of each sound source as a function of the intensity and the position of the sound source at said time step, and then to sum the binaural coefficients of the different sound sources. This makes it possible to obtain a single set 321a of binaural coefficients representative of the whole of the sound scene.

The binauralization engine 320a then comprises a conversion circuit 330a of the coefficients, configured to transform the set of coefficients 321a representative of the sound scene into a set of transformed coefficients 331a representative of the entire sound stage. . This allows for example to rotate the entire sound stage.

The binauralization engine 300a finally comprises a binaural decoder 340a, configured to restore the transformed coefficients 331a into a set of output channels, for example a left channel 341a and a right channel 342a of a stereo system.

The binauralization engine 300a therefore makes it possible to reduce the computational complexity necessary for the binaural processing of a sound scene with respect to the system 200, by applying the transformation and decoding steps to the whole of the sound scene, rather than 'to each sound source individually.

FIG. 3b represents an example of a binauralization engine of a 3D scene, in the frequency domain according to the state of the art.

The binauralization engine 300b is quite similar to binauralization engine 300a. It comprises a set 31 1 b of frequency transformation logic, the set 31 1b comprising a frequency transformation logic for each sound source. Frequency transformation logic can for example be configured to apply a Fast Fourier Transform (Fast Fourier Transform), in order to obtain a set 312b of sources in the frequency domain. The application of frequency transforms is well known to those skilled in the art, and is for example described by A. Mertins, Signal Analysis: Wavelets, Filter Banks, Time-Frequency Transforms and Applications, English (revised edition). ISBN: 9780470841839. It consists, for example, of transforming, by time windows, the sound samples into frequency intensity, according to a frequency sampling. The inverse operation, or inverse frequency transformation (so-called FFT ^"1 or inverse fast Fourier transform in the case of a fast Fourier transform) makes it possible to reproduce, from a sampling of frequencies, intensities of sound samples. .

The binauralization engine 300b then comprises an encoder HOA 320b in the frequency domain. The encoder 320b is configured to calculate, for each source and at each frequency of the frequency sampling, the corresponding ambison coefficients, then to add the ambison coefficients of the different sources, in order to obtain a set 321 b of representative ambisonic samples. of the entire sound stage, at different frequencies. An ambisonic coefficient at a frequency f of frequency sampling is obtained, similarly to an ambisonic coefficient at time t, by the formula: B _mn f) =

The binauralization engine 300b then comprises a transformation circuit 330b, similar to the transformation circuit 330a, for obtaining a set 331 b of transformed ambisonic coefficients representative of the entire sound stage, and a binaural decoder 340b. , configured to output two stereo channels 341b and 342b. The binaural decoder 340b includes an inverse frequency transformation circuit, in order to reproduce the stereo channels in the time domain.

The properties of binauralization engine 300b are quite similar to those of binauralization engine 300a. It also makes it possible to perform a binaural processing of a sound scene, with a reduced complexity compared with the system 200. [0071] In the event of a significant increase in the number of sources, the complexity of the binaural processing of the binauralization engines 300a and 300b is mainly due to the calculation of the HOA coefficients by the encoders 320a and 320b. Indeed, the number of coefficients to calculate is proportional to the number of sources. On the contrary, the transformation circuits 330a and 330b, as well as the binaural decoders 340a and 340b deal with sets of binaural coefficients representative of the whole of the sound stage, the number of which does not vary according to the number of sources.

For the treatment of reflections, the complexity of the binaural encoders 320a and 320b can increase significantly. Indeed, the solution of the state of the art for processing reflections is to add a virtual sound source for each reflection. The complexity of the HOA encoding of these encoders according to the state of the art therefore increases proportionally as a function of the number of reflections per source, and can become problematic when the number of reflections becomes too great.

FIG. 4 represents an example of an ambisonic encoder of a sound wave with a plurality of reflections, in a set of modes of implementation of the invention.

The ambisonic encoder 400 is configured to encode a sound wave 410 with a plurality of reflections, in a set of ambison coefficients to an order M. To do this, the ambisonic encoder is configured to compute a set 460 of spherical harmonics representative of the sound wave and the plurality of reflections. The ambisonic encoder 400 will be described, by way of example, for encoding a single sound wave. However, an ambisonic encoder 400 according to the invention can also encode a plurality of sound waves, the elements of the ambisonic encoder being used in the same way for each additional sound wave. The sound wave 410 may correspond, for example, to a channel of an audio track, or to a dynamically created sound wave, for example a sound wave corresponding to an object of a video game. In one set of embodiments of the invention, the sound waves are defined by successive samples of loudness. According to various embodiments of the invention, the sound waves can for example be sampled at a frequency of 22500Hz, 12000Hz, 44100Hz, 48000Hz, 88200Hz, or 96000Hz, and each of the intensity samples encoded on 8, 12, 1 6, 24 or 32 bits. In case of plurality of sound waves, these can be sampled at different frequencies, and the samples can be encoded on different bit numbers. The ambisonic encoder 400 comprises a logic 420 of frequency transformation of the sound wave. This is similar to logic 31 1 b of frequency transformation of the sound waves of binauralization system 300b according to the state of the art. In embodiments at a plurality of sound waves, encoder 400 includes frequency transformation logic for each sound wave. At the output of the frequency transformation logic, a sound wave is defined 421, for a time window, by a set of intensities at different frequencies of a frequency sampling. In one set of embodiments of the invention, the frequency transformation logic 420 is a logic for applying an FFT.

The encoder 400a also comprises a logic 430 for calculating spherical harmonics of the sound wave and the plurality of reflections from a position of a source of the sound wave and of obstacle positions. to the propagation of the sound wave. In a set of embodiments of the invention, the position of the source of the sound wave is defined by angles (θ _5. , Φ ₅ ) and a distance from a listening position of the sound wave. user. The calculation of spherical harmonics Υοο (θ _5. , Φ ₅ ), Yi-i (e _Si , <P _Si ), Yi (e _Si , <P _Si ), Yu (e _Si , <p _Si ), K _MM (0 _s ., <P _s .), From the sound wave to the order M can be performed according to the known methods of the state of the art, from angles (θ _5. , Φ ₅ . ) defining the orientation of the source of the sound wave.

The logic 430 is also configured to calculate, from the position of the source of the sound wave, a set of spherical harmonics of the plurality of reflections. In a set of embodiments of the invention, the logic 430 is configured to calculate, from the position of the source of the sound wave, and from positions of obstacles to the propagation of the sound wave, a orientation of a virtual source of a reflection, defined by angles (0 _sr , <p _sr ), then, from these angles, spherical harmonics Υοο {β _{5 Τ} , φ _{5 Τ} ), Υχ-ι { _{5 Τ} β, φ _{5 Τ)} Υχο {β _{5 Τ,} _Τ φ _5), Yn {Ps, r><Psr) YMM i {Q _s, r><Ps, r) of the reflection of the sound wave . This makes it possible to obtain, for each reflection, the spherical harmonics corresponding to the direction of the wave reflected on the obstacles to the propagation of the sound wave. The ambisonic encoder 400 also comprises a plurality 440 of frequency domain filtering logic receiving as input spherical harmonics of the plurality of reflections, each filtering logic being parameterized by acoustic coefficients and delay of the reflections. In the following description, the term has coefficient _r an acoustic reflection and 5 _r a period of reflection. According to various embodiments of the invention, the acoustic coefficient may be a coefficient of _r reverberation representative of a ratio of the intensities of reflection on the intensities of the sound source and set between 0 and 1. According to other embodiments of the invention, the acoustic coefficient is a coefficient of _said attenuation or absorption, or a coefficient defined between 0 and 1 such that _a = _r - 1. These filtering logics make it possible to apply delay and attenuation to the ambisonic coefficients of a reflection. Thus, the combination of the orientation of the virtual source of the reflection, the delay and the attenuation of the reflection makes it possible to model each reflection as a replica of the sound source, coming from a different direction, affected by a delay and attenuated, following the course and reflections of the sound wave. This modeling allows, with several reflections, to simulate the propagation of a sound wave in a scene in a simple and effective way.

In a general manner, the filtering, at a frequency f, of a spherical harmonic of a reflection can be written: H _r (f) K _y (0 _sr , <p _sr ). In one embodiment of the invention a filtering logic 440 is configured to filter the spherical harmonics by applying: _r e ^{~ j2n} f ^Sr Y '_ί; · (0 _sr , <p _sr ). In this embodiment, the coefficient _r is treated as a reverberation coefficient. In other embodiments, a coefficient _a may be treated as an attenuation coefficient, and the filtering of spherical harmonics may for example be performed by applying: (1 - _a ) e ^{~ i2lt} f ^Sr Y _tj { e _sr , <p _sr ). In the rest of the description, unless otherwise stated, the coefficient _{r is considered to} be a reverberation coefficient. Those skilled in the art can however easily implement the various embodiments of the invention with an attenuation coefficient rather than a reverberation coefficient.

The ambisonic encoder 400 also comprises a logic 450 for adding the spherical harmonics of the sound wave and the outputs of the filtering logic. This logic makes it possible to obtain a set ΥΌο, ΥΊ-ι, Y 'io, Y'ii, ■■■ Y' MM of spherical harmonics of order M, representative of both the sound wave, and reflections of the sound wave, in the frequency domain. A spherical harmonic Y ' _/ j (with 0 <i <M, and -i <j <i) representative of both the sound wave, and reflections of the sound wave, is therefore equal, at the output of the addition logic 450, at the value Y _tj = ^Y ij { ^e if, <Psi) + ΣLo ^H r (f Yij {e _s , r, <Ps, r), in which Y _tj (e _Si , < p _Si ) is a spherical harmonic of the source of the sound wave, N _r is the number of reflections of the sound wave, Yij (O _sr , <p _sr ) are the spherical harmonics of the positions of the virtual sound sources of the reflections , and the terms H _r (f) are the spherical harmonics filtering logic for reflection r at a frequency f. In a set of embodiments of the invention, the filtering logics H _r (f) are such that H _r (f) = _r e ^{~) 2nfSr} , and the spherical harmonics Y _tj to the order M, representative to both of the sound wave, and reflections of the sound wave are equal, at the output of the addition logic 450, to: Y ' _tJ = Y _tJ (e _Si , <p _Si ) + Σ ^ ^ - ^^ γ ^ θ ^, φ ^).

According to various embodiments of the invention, the number N _r of reflections can be predefined. According to other embodiments of the invention, the reflections of the sound wave are preserved according to their acoustic coefficient, the number Nr of reflections then depending on the position of the sound source, the position of the user, and obstacles to the propagation of sound. In the example above, the acoustic coefficient is defined as a ratio of the intensity of the reflection on the intensity of the sound source, a reverberation coefficient. In one embodiment of the invention, the reflections of the sound wave having an acoustic coefficient greater than or equal to a predefined threshold are retained. In other embodiments, the acoustic coefficient is defined as an attenuation coefficient, a ratio between the sound intensity absorbed by the obstacles to the propagation of sound waves and the path in the air and the intensity. of the sound source. In this embodiment, the reflections of the sound wave having an acoustic coefficient lower than or equal to a predefined threshold are retained

Thus, the ambisonic encoder 400 calculates a set of spherical harmonics Yj representative of both the sound wave and its reflections. Once these spherical harmonics are calculated, the encoder can include a logic of multiplication of spherical harmonics by the values of sound intensities of the source at different frequencies, in order to obtain ambisonic coefficients representative of both the sound wave and the reflections. In embodiments with several sound sources, the encoder 400 comprises a logic for adding the ambisonic coefficients of the different sound sources and their reflections, making it possible to obtain at the output ambisonic coefficients representative of the whole of the sound scene. .

In one set of embodiments of the invention, the ambison coefficients of the order M representative of the sound scene are then equal, at the output of the logic of addition of the ambison coefficients of the different sound sources and their reflections, for Ns sound sources and for a frequency f, to:

The use of a single ambisonic coefficient Y) j representative of both the sound wave and its reflections significantly reduces the calculation operations to obtain the ambison coefficients, especially when the number of reflections is high. Indeed, this makes it possible to reduce the number of multiplications, since it is no longer necessary to multiply each of the intensities; (/) of a source for each frequency by each of the spherical harmonics Υΐ ₎ {θ ₅ , φ ₅ ), for each value of i such that 0 <i <M, each value of j such that -i <j <i, and each reflection. This reduction in the number of multiplications allows a significant reduction in computational complexity, particularly in the case of a high number of reflections. In a set of embodiments of the invention, the logic 430 for calculating spherical harmonics of the sound wave is configured to calculate the spherical harmonics of the sound wave and the plurality of reflections from a fixed position of the source of the sound wave. In this case, the orientations (θ _5. , Φ _5. ) Of the sound source, and the orientations {Q _s , r _> <Ps, r) of each of the harmonics are constant. The spherical harmonics of the sound wave and the plurality of reflections then also have a constant value, and can be calculated once for the sound wave.

In other embodiments of the invention, the logic 430 for calculating spherical harmonics of the sound wave is configured to iteratively calculate the spherical harmonics of the sound wave and the plurality of reflections. from successive positions of the source of the sound wave. According to various embodiments of the invention, different possibilities exist for defining the calculation iterations. In one embodiment of the invention, the logic 430 is configured to recalculate the spherical harmonic values of the sound wave and the plurality of reflections whenever a change in the position of the wave source sound or position of the user is detected. In another embodiment of the invention, the logic 430 is configured to recalculate the spherical harmonic values of the sound wave and the plurality of reflections at regular intervals, for example every 10 ms. In another embodiment of the invention, the logic 430 is configured to recalculate the values of the spherical harmonics of the sound wave and of the plurality of reflections to each of the time windows used by the logic 420 of frequency transformation of the sound wave to convert the temporal samples of the sound wave into frequency samples.

In one set of embodiments of the invention, each reflection is characterized by a single acoustic coefficient a _r .

In other embodiments of the invention, each reflection is characterized by an acoustic coefficient for each frequency of said frequency sampling. This makes it possible to obtain different acoustic coefficients for the different frequencies, and to improve the rendering of certain effects. For example, it is known that thick materials absorb more importantly low frequencies. Similarly some types of materials absorb and reflect the high frequencies differently. Thus, the definition of different acoustic coefficients for the same reflection and different frequencies makes it possible to characterize the materials encountered by the reflections, allowing better rendering of different types of rooms, depending on the materials of the walls thereof.

In a set of embodiments of the invention, a reflection at a frequency can be considered as zero, depending on a comparison between the acoustic coefficient _r for this frequency and a predefined threshold. For example, if the coefficient a _r represents a reverberation coefficient, the frequency is considered to be zero if it is lower than a predefined threshold. On the contrary, if it is an attenuation coefficient, the frequency is considered to be zero if it is greater than or equal to a predefined threshold. This makes it possible to further limit the number of multiplications, and therefore the complexity of ambisonic encoding, while having a minimal impact on the binaural rendering.

In a set of embodiments of the invention, the ambisonic encoder 400 comprises a logic for calculating the acoustic coefficients and delays, and the position of the virtual sound source of the reflections. This calculation logic can for example be configured to calculate the acoustic coefficients and the reflection times according to estimates of a difference in distance traveled by the sound between the position of the source of the sound wave and an estimated position of the sound. a user on the one hand, and the distance traveled by the sound between the positions of the virtual sound sources of the reflections and the estimated position of the user on the other hand. It is indeed easy, knowing the difference in distance traveled by the sound wave to reach the user, in a straight line from the sound source on the one hand, and through a reflection on the other hand, and knowing the celerity of the sound, to deduce the delay felt by the user between the sound from the sound source in a straight line on the one hand, and the sound having been affected by reflection on the other hand.

In the same way, it is known that the intensity of a sound wave decreases as it travels through the air. The logic of calculation of the acoustic coefficients and the delays, and the position of the source virtual sound reflections can be configured to calculate an acoustic coefficient of a reflection of the sound wave according to the difference in distance traveled between the sound from the sound source in a straight line on the one hand, and the sound having been affected by reflection on the other hand.

In other embodiments of the invention, the calculation logic acoustic coefficients and delays, and the position of the virtual sound source reflections is also configured to calculate the acoustic coefficients of the reflections as a function of an acoustic coefficient of at least one obstacle to the propagation of sound waves, on which the sound is reflected. This makes it possible to better model the absorption by the materials of a room, and the acoustic coefficient of the obstacle can be variable according to the different frequencies. The acoustic coefficient of the obstacle may be a reverberation coefficient or an attenuation coefficient.

FIG. 5 represents an example of calculation of a secondary sound source, in one embodiment of the invention.

In this example, a source of the sound wave has a position 520 in a room 510, and the user has a position 540. The room 510 consists of 4 walls 51 1, 512, 513 and 514.

In a set of embodiments of the invention, the calculation logic acoustic coefficients and delays, and the position of the virtual sound source reflections is configured to calculate the position, delay and attenuation of virtual sound sources of the reflections in the following manner: for each of the walls 51 1, 512, 513, 514, the logic is configured to calculate a position of a virtual sound source of a reflection as the symmetrical of the position of the source sound compared to a wall. The calculation logic is thus configured to calculate the positions 521, 522, 523 and 524 of four virtual sound sources of the reflections, respectively with respect to the walls 51 1, 512, 513 and 514.

For each of these virtual sound sources, the calculation logic is configured to calculate a path of travel of the sound wave, and deduce the corresponding acoustic coefficient and delay. For example, in the case of the virtual sound source 51 1, the sound wave follows the path 530 to the point 531 of the wall 512, then the path 532 to the position of user 540. The distance traveled by the sound according to the path 530, 532 makes it possible to calculate an acoustic coefficient and a delay of the reflection. In a set of embodiments of the invention the calculation logic is also configured to apply an acoustic coefficient corresponding to the absorption of the wall 512 at point 531. In one set of embodiments of the invention, this coefficient depends on the different frequencies, and can for example be determined, for each frequency, as a function of the material and / or the thickness of the wall 512.

In one set of embodiments of the invention, the virtual sound sources 521, 522, 523, 524 are used to calculate secondary virtual sound sources, corresponding to multiple reflections. For example, a secondary virtual source 533 may be calculated as the symmetry of the virtual source 521 with respect to the wall 514. The path of the corresponding sound wave then comprises the segments 530 to the point 531; 534 between points 531 and 535; 536 between the point 535 and the position 540 of the user. The acoustic coefficients and the delays can then be calculated from the distance traveled by the sound on the segments 531, 535 and 536, and the absorption of the walls at the points 531 and 535.

According to different embodiments of the invention, virtual sound sources corresponding to reflections can be calculated up to a predefined order n. Different embodiments are possible for determining the reflections to be preserved. In one embodiment of the invention, the calculation logic is configured to calculate, for each virtual sound source, a higher order virtual sound source for each of the walls, up to a predefined order n. In one embodiment, the ambisonic encoder is configured to process a predefined Nr number of reflections per sound source, and retains the Nr reflections with the lowest attenuation. In another embodiment of the invention, the virtual sound sources are preserved on the basis of a comparison of an acoustic coefficient with a predefined threshold.

FIG. 6 represents an exemplary calculation of early reflections and late reflections, in one embodiment of the invention.

Diagram 600 represents the intensity of several reflections of the sound wave, with respect to time. The axis 601 represents the intensity of a reflection, and the axis 602 the delay between the emission of the sound wave by the source of the sound wave, and the perception of a reflection by the user. In this example, reflections occurring before a predefined time 603 are considered as early reflections 610, and reflections occurring after time 603 as late reflections 620. In one embodiment of the invention, early reflections are calculated at using a virtual sound source, for example according to the principle described with reference to FIG.

According to various embodiments of the invention, the late reflections are calculated as follows: a set of Nt secondary sound sources is calculated, for example according to the principle described in FIG. 5. The calculation logic of the acoustic coefficients and delays, and the position of the virtual sound source reflections is configured to keep a number Nr of reflections less than Nt, according to various embodiments described above. In one set of embodiments of the invention, it is further configured to construct a list of (Nt - Nr) late reflections, including all non-conserved reflections. This list includes only, for each late reflection, an acoustic coefficient and a delay of the late reflection, but no position of a virtual source.

According to one embodiment of the invention, this list is transmitted by the ambisonic encoder to an ambisonic decoder. The ambisonic decoder is then configured to filter its outputs, for example its stereo output channels, with the acoustic coefficients and delays of late reflections, and then add these filtered signals to the output signals. This makes it possible to improve the feeling of immersion in a room or a listening environment, while still limiting the calculation complexity of the encoder.

According to another embodiment of the invention, the ambisonic encoder is configured to filter the sound wave with the acoustic coefficients and delays of late reflections, and add the signals obtained in a uniform manner to all the Ambisonic coefficients. This makes it possible to obtain, with a limited computational complexity, an effect representative of multiple reflections in a sound environment. In this embodiment of the invention, as in the previous one, the reflections late have a low intensity and have no directional information from a sound source. They will therefore be perceived by a user as an "echo" of the sound wave, homogeneously distributed in the sound stage, and representative of a listening environment.

Calculation of acoustic coefficients and delays of late reflections induces the calculation of many reflections. It is therefore a relatively expensive operation in terms of calculation complexity. According to one embodiment of the invention, this calculation is carried out only once, for example at the initialization of the sound stage, and the acoustic coefficients and delays of the late reflections are reused without modification by the ambisonic encoder. This allows for late reflections representative of the listening environment at lower cost. According to other embodiments of the invention, this calculation is performed iteratively. For example, these acoustic coefficients and delays of late reflections can be calculated at predefined time intervals, for example every 5 seconds. This makes it possible to permanently store acoustic coefficients and delay the late reflections representative of the sound scene, and the relative positions of a source of the sound wave and the user, while limiting the calculation complexity related to the determination. late reflections.

In other embodiments of the invention, the acoustic coefficients and delays of the late reflections are calculated when the position of a source of the sound wave or the user varies significantly, for example when the difference between the position of the user and a previous position of the user during a calculation of the acoustic coefficients and delays of the late reflections representative of the sound scene is greater than a predefined threshold. This makes it possible to compute the acoustic coefficients and delays of the late reflections representative of the sound scene only when the position of a source of the sound wave or of the user has varied enough to perceptibly modify the late reflections.

[00106] FIG. 7 represents a method of encoding a sound wave at a plurality of reflections in a set of modes of implementation of the invention. The method 700 comprises a step 710 of frequency transformation of the sound wave.

It then comprises a step 720 for calculating spherical harmonics of the sound wave and the plurality of reflections from a position of a source of the sound wave and of positions of obstacles to propagation. sound waves.

It then comprises a step 730 of filtering, by a plurality of filtering logic in the frequency domain, spherical harmonics of the plurality of reflections, each filtering logic being parameterized by acoustic coefficients and delays of the reflections.

It then comprises a step 740 for adding spherical harmonics of the sound wave and outputs of the filtering logic.

The above examples demonstrate the ability of an ambisonic encoder according to the invention to calculate ambison coefficients of a sound wave at a plurality of reflections. They are however given only by way of example and in no way limit the scope of the invention, defined in the claims below.

Claims

Ambisonic sound wave encoder (400) (410) with a plurality of reflections, comprising:

- A logic (420) of frequency transformation of the sound wave;

A logic (430) for calculating spherical harmonics of the sound wave and the plurality of reflections from a position of a source of the sound wave and of positions of obstacles to a propagation of the sound wave sound wave ;

a plurality (440) of frequency domain filtering logic receiving as input spherical harmonics of the plurality of reflections, each filtering logic being parameterized by acoustic coefficients and delays of the reflections;

A logic (450) for adding spherical harmonics of the sound wave and outputs of the filtering logic.

An ambisonic encoder according to claim 1, wherein the spherical harmonics calculation logic of the sound wave is configured to calculate the spherical harmonics of the sound wave and the plurality of reflections from a fixed position of the source of the sound wave.

An ambisonic encoder according to claim 1, wherein the calculation logic of spherical harmonics of the sound wave is configured to iteratively calculate the spherical harmonics of the sound wave and the plurality of reflections from successive positions of the sound wave. source of the sound wave.

An ambisonic encoder according to one of claims 1 to 3, wherein each reflection is characterized by a single acoustic coefficient.

Ambisonic encoder according to one of claims 1 to 3, wherein each reflection is characterized by an acoustic coefficient for each frequency of said frequency sampling.

Ambisonic encoder according to one of claims 1 to 5, wherein the reflections are represented by virtual sound sources.

Ambisonic encoder according to one of claims 1 to 5, further comprising a logic for calculating the acoustic coefficients, the delays and the position of the virtual sound sources of the reflections, said calculation logic being configured to calculate the acoustic coefficients. and the delays of the reflections based on estimates of a difference in distance traveled by the sound between the position of the source of the sound wave and an estimated position of a user on the one hand, and a distance traveled sound between positions of virtual sound sources reflections and the estimated position of the user on the other hand.

Ambisonic encoder according to claim 7, in which the logic for calculating the acoustic coefficients, the delays and the positions of the virtual sound sources of the reflections is furthermore configured to calculate the acoustic coefficients of the reflections as a function of at least one coefficient. acoustic of at least one obstacle to the propagation of sound waves, on which the sound is reflected.

Ambisonic encoder according to one of Claims 7 to 8, in which the logic for calculating the acoustic coefficients, the delays and the positions of the virtual sound sources of the reflections is configured to calculate positions of the virtual sound sources of the reflections as symmetrical. the position of the source of the sound wave relative to a plane tangent to an obstacle to the propagation of sound waves.

The ambisonic encoder according to one of claims 1 to 9, wherein the calculation logic of spherical harmonics of the sound wave and the plurality of reflections is further configured to calculate spherical harmonics of the sound wave and the plurality of reflections at each output frequency of the frequency transformation circuit, said ambisonic encoder further comprising a binaural coefficients calculation logic of the sound wave, configured to calculate binaural coefficients of the sound wave by multiplying at each output frequency of the frequency conversion circuit of the sound wave the signal of the sound wave by the spherical harmonics of the sound wave and the plurality of reflections at this frequency.

1 1. Ambisonic encoder according to one of Claims 7 to 9, in which the logic for calculating the acoustic coefficients, the delays and the positions of the virtual sound sources of the reflections is configured to calculate acoustic coefficients and delays of a plurality of late reflections. .

An ambisonic sound wave encoding method with a plurality of reflections, comprising:

a frequency transformation (710) of the sound wave;

a calculation (720) of spherical harmonics of the sound wave and of the plurality of reflections from a position of a source of the sound wave and of positions of obstacles to propagation of sound waves;

a filtering (730), by a plurality of frequency domain filtering logic, of the spherical harmonics of the plurality of reflections, each filtering logic being parameterized by acoustic coefficients and reflection delays;

an addition (740) of spherical harmonics of the sound wave and the outputs of the filtering logic.

A computer program product comprising program code instructions recorded on a computer readable medium for ambisonic sound wave encoding with a plurality of reflections, said program code instructions being configured to:

- perform a frequency transformation of the sound wave; calculating spherical harmonics of the sound wave and the plurality of reflections from a position of a source of the sound wave and of positions of obstacles to propagation of the sound wave;

parameterizing a plurality of filtering logic in the frequency domain receiving as input spherical harmonics of the plurality of reflections, each filtering logic being parameterized by acoustic coefficients and delays of the reflections;

adding spherical harmonics of the sound wave and outputs of the filter logics when said program is running on a computer.