EP3449643A1 - Method and system of broadcasting a 360° audio signal - Google Patents

Method and system of broadcasting a 360° audio signal

Info

Publication number
EP3449643A1
EP3449643A1 EP17725294.7A EP17725294A EP3449643A1 EP 3449643 A1 EP3449643 A1 EP 3449643A1 EP 17725294 A EP17725294 A EP 17725294A EP 3449643 A1 EP3449643 A1 EP 3449643A1
Authority
EP
European Patent Office
Prior art keywords
sound signal
sound
microphones
format
ambisonic
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP17725294.7A
Other languages
German (de)
French (fr)
Other versions
EP3449643B1 (en
Inventor
Delphine Devallez
Frédéric AMADU
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Arkamys SA
Original Assignee
Arkamys SA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Arkamys SA filed Critical Arkamys SA
Publication of EP3449643A1 publication Critical patent/EP3449643A1/en
Application granted granted Critical
Publication of EP3449643B1 publication Critical patent/EP3449643B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
    • H04S7/30—Control circuits for electronic adaptation of the sound field
    • H04S7/302—Electronic adaptation of stereophonic sound system to listener position or orientation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R1/00—Details of transducers, loudspeakers or microphones
    • H04R1/20—Arrangements for obtaining desired frequency or directional characteristics
    • H04R1/32—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only
    • H04R1/40—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers
    • H04R1/406—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers microphones
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00—Circuits for transducers, loudspeakers or microphones
    • H04R3/005—Circuits for transducers, loudspeakers or microphones for combining the signals of two or more microphones
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
    • H04S2400/15—Aspects of sound capture and related signal processing for recording or reproduction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
    • H04S2420/11—Application of ambisonics in stereophonic audio systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
    • H04S7/30—Control circuits for electronic adaptation of the sound field
    • H04S7/302—Electronic adaptation of stereophonic sound system to listener position or orientation
    • H04S7/303—Tracking of listener position or orientation
    • H04S7/304—For headphones

Definitions

  • the present invention relates to the field of sound signal processing.
  • the most compact solution is the use of a network of microphones, such as the eigenmike of mh acoustics, the Soundfield of TSL Products, and the TetraMic Core Sound. Equipped with four to thirty-two microphones, these products are expensive and therefore reserved for professional use. Recent research has reduced the number of microphones (Palacino, JD, & Nicol, R. (2013).) "Spatial sound pick-up with a low number of microphones.” ICA 2013. Montreal, Canada. Microphones of reduced size and cost can be used such as those available to mobile phones.
  • the shape of the microphone arrays remains standardized, however, from the dodecahedron for the EigenMike to the tetrahedron for the Soundfield and the TetraMic.
  • This geometric shape makes it possible to use simple formulas to convert microphone signals into a format ambisonic, and were developed by Gerzon in 1975 (Gerzon, M. (1975). "The design of precisely coincide microphone arrays for stereo and surround sound.” 50th Audio Engineering Society Conference.).
  • the ambisonic format is a set of audio channels that contains all the information necessary for the spatial reconstruction of the sound field.
  • a novelty provided by this patent is the possibility of using any form of microphone array.
  • the present invention intends to overcome the drawbacks of the prior art by proposing a method of processing the sound signal making it possible to capture the sound signal in all directions and then to restore the sound signal.
  • the present invention relates, in its most general sense, to a sound signal processing method, characterized in that it comprises the following steps:
  • the matrix calculation involves a matrix H calculated by the least squares method from the measured directivities of the N microphones and the ideal directivities of the ambison components.
  • said microphones are arranged in a circle on a plane, spaced at an angle equal to 360 ° / N or at each corner of a mobile phone.
  • said method implements four microphones spaced at an angle of 90 ° to the hoirzontal.
  • said method implements a filter band pass filter from 100 Hz to 6 kHz.
  • the order R of the ambisonic format is equal to one.
  • a capture of said information relating to the orientation of the head of a user listening to the sound signal is performed by a sensor within a mobile phone or by a sensor located in a headset or a virtual reality helmet.
  • the data in ambisonic format is transformed into data in binaural format.
  • the present invention also relates to a system for processing the sound signal, comprising means for:
  • FIGS 1 and 3 show the different steps of the process according to the present invention
  • FIG. 2 illustrates the treatments applied in the context of the second step of the process according to the present invention
  • ⁇ Figures 4a, 4b and 4c represent the ideal W, Y and X components of an ambisonic format of order 1 (on a horizontal plane);
  • FIGS. 5a, 5b and 5c illustrate the approximated W, Y and X components of an ambisonic format of order 1;
  • Figure 6 shows the placement of eight virtual speakers, each placed at 45 ° around a user.
  • the present invention relates to a sound signal processing method, comprising the following steps:
  • N being a natural integer greater than or equal to three
  • FIGS 1 and 3 illustrate the different steps of the method according to the present invention.
  • said microphones are arranged in a circle on a plane, spaced at an angle equal to 360 ° / N or at each corner of a mobile phone.
  • the method according to the present invention implements four microphones spaced at an angle of 90 ° to the horizontal.
  • the order R of the ambisonic format is one.
  • the first step of the method according to the present invention consists in the recording of the sound signal.
  • N microphones are used for this recording, N being a natural number greater than or equal to three, said microphones being arranged in a circle on a plane, spaced at an angle equal to 360 ° / N or at each corner of a mobile phone.
  • N is equal to four and the microphones are spaced 90 °.
  • These microphones are arranged in a circle on a plane.
  • the radius of said circle is two centimeters, and the microphones are omnidirectional.
  • the sound signal is picked up by said microphones, and digitized. This is a synchronous capture. At the end of this first step, four sampled digital signals are obtained.
  • the second step of the method according to the present invention consists in the encoding of said four sampled digital signals, in an ambisonic format of order R, R being a natural integer greater than or equal to one.
  • the ambisonic format is a standard format of multi-dimensional audio coding.
  • the order R is equal to one.
  • This order 1 makes it possible to represent the sound with the following notions: Forward - Back and Left - Right.
  • Figures 4a, 4b and 4c represent the ideal W, Y and X components of an ambisonic format of order 1 (on a horizontal plane).
  • Figures 5a, 5b and 5c illustrate the approximated W, Y and X components of an ambisonic format of order 1.
  • Figure 2 illustrates the treatments applied in the context of the second step of the process according to the present invention. It can be observed in FIG. 2 that the input data are in the time domain, pass into the frequency domain following a Fast Fourier Transform (FFT) operation, and then the data in FIG. output are in the time domain following an inverse fast Fourier transform (IFFT) operation.
  • FFT Fast Fourier Transform
  • IFFT inverse fast Fourier transform
  • Hanning windows with overlap are used by implementing an "overlap-add" type function.
  • the input frequency data are modified using matrix multiplication. This matrix comprises weighting coefficients for each microphone signal and each frequency.
  • filtering by means of a bandpass filter is performed on the data before the output.
  • the method according to the present invention implements a filter bandpass filter from 100 Hz to 6 kHz. This eliminates the lower part and the acute part.
  • impulse responses of the N microphones are measured, in this case four microphones, with a source positioned every 5 ° or every 10 ° around the microphone array.
  • N is the number of microphones (four in the present embodiment)
  • D is the number of measured source angular positions (108 in the present embodiment)
  • V is the number of ambison channels (three in the present example) embodiment
  • C DX N denotes the directivities of the microphones
  • H Nx v denotes the matrix which transforms the directivities of the microphones into the desired directivities
  • PDXV denotes the directivities prescribed by the ambisonic format (W, X and Y in the present example realization).
  • H Nx v PDXV / CDXN for each index of frequency k if CDXN is invertible.
  • the matrix H is defined once for future uses of the considered microphone array. Then, with each use, a matrix multiplication is performed in the frequency domain.
  • Said matrix H has as many lines as microphones, therefore four in the present embodiment, and as many columns as required by the order of the ambisonic format used, therefore three columns in the present embodiment in which the order 1 is implemented on the horizontal plane.
  • Out In x H, where H denotes the previously computed matrix, In denotes the input (audio channels coming from the microphones array, passed in the frequency domain) and Out denotes the output (Out being reconverted into the time domain for get the ambisonic format).
  • the method according to the present invention implements, during this second step, an algorithm called least squares algorithm for each frequency, with for example 512 frequency points.
  • data is obtained in the ambisonic format (in the present embodiment the signals W, X and Y).
  • the third step of the method according to the present invention consists in the restitution of the sound signal, thanks to a transformation of the data in ambisonic format into two binaural channels.
  • the information relating to the orientation of the head of the user listening to the sound signal is retrieved and used. This can be performed by a sensor in a mobile phone, a headset or a virtual reality headset.
  • This orientation information consists of a vector comprising three angle values, in the English terminology “pitch”, “yaw” and “millet”.
  • the angle value "yaw" is used on a plane.
  • the ambisonic format is transformed into eight audio channels corresponding to a virtual placement of eight loudspeakers, each placed at 45 ° around the user.
  • Figure 6 shows the placement of eight virtual speakers, each placed at 45 ° around a user.
  • HRTF left ear and right ear filters
  • HRTF left ear and right ear filters
  • a pair of HRTF (left ear and right ear) filters are paired with each virtual speaker, and then all ("left ear” channels and all "right ear” channels together) are added to form two output channels .
  • IIR Infinity Impulse Response
  • FIG. 3 shows the different steps of the method according to the present invention.
  • the present invention also relates to a system for processing the sound signal, comprising means for:
  • N being a natural integer greater than or equal to three
  • This sound signal processing system comprises at least one calculation unit and one memory unit.
  • the invention is described in the foregoing by way of example. It is understood that the skilled person is able to realize different variants of the invention without departing from the scope of the patent.

Landscapes

  • Health & Medical Sciences (AREA)
  • Otolaryngology (AREA)
  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • General Health & Medical Sciences (AREA)
  • Stereophonic System (AREA)
  • Circuit For Audible Band Transducer (AREA)

Abstract

This invention relates to a method of processing a sound signal that comprises the following steps: ·Synchronous reception of an input sound signal (Sinput) by means of N microphones, N being a natural number greater than or equal to three; ·Encoding of the said input sound signal (Sinput) in a data format (D) of sound, said encoding comprising a sub-step of transforming the input signal into an ambisonic format of order R, R being a natural number greater than or equal to one; the said sub-step of transformation into an ambisonic format is carried out by means of a Fast Fourier Transform, a matrix multiplication, an Inverse Fast Fourier Transform and by means of a band pass filter; and ·Return of an output sound signal (Soutput) by means of digital processing of the sound data (D). This invention also relates to a system of processing a sound signal.

Description

PROCEDE ET SYSTEME DE DIFFUSION D'UN SIGNAL AUDIO A 360'  METHOD AND SYSTEM FOR BROADCASTING A 360 'AUDIO SIGNAL
Domaine de l'invention La présente invention se rapporte au domaine du traitement du signal sonore. Field of the Invention The present invention relates to the field of sound signal processing.
Etat de la technique On connaît dans l'état de la technique des procédés et des systèmes permettant de diffuser des signaux vidéo à 360° . Il existe un besoin dans l'état de la technique de pouvoir associer des signaux audio à ces signaux vidéo à 360° . Jusqu'à maintenant, l'audio 3D était réservé aux professionnels du son et aux chercheurs. Cette technologie a pour objectif de capturer le maximum d'informations spatiales lors de l'enregistrement pour les restituer ensuite à l'auditeur et lui donner une sensation d'immersion dans la scène sonore. Dans le domaine de la vidéo, l'intérêt est croissant pour les vidéos filmées à 360° et reproduites via un casque de réalité virtuelle pour une immersion complète dans l'image : l'utilisateur peut tourner la tête et explorer la scène visuelle tout autour de lui. Pour obtenir la même fidélité dans le domaine du son, la solution la plus compacte est l'utilisation d'un réseau de microphones, comme par exemple l'Eigenmike de mh acoustics, le Soundfield de TSL Products, et le TetraMic de Core Sound. Equipés de quatre à trente-deux microphones, ces produits sont onéreux et donc réservés à des utilisations professionnelles. De récentes recherches ont permis de réduire le nombre de microphones (Palacino, J. D., & Nicol, R. (2013). "Spatial sound pick-up with a low number of microphones. " ICA 2013. Montréal, Canada.), et des microphones de taille et de coût réduits peuvent être utilisés tels que ceux dont disposent les téléphones portables. La forme des réseaux de microphones, un polyhèdre, reste cependant standardisée, du dodécaèdre pour l'EigenMike au tetrahèdre pour le Soundfield et le TetraMic. Cette forme géométrique permet d'utiliser des formules simples pour convertir les signaux des microphones en un format ambisonique, et ont été développées par Gerzon en 1 975 (Gerzon, M. (1975). « The design of precisely coïncident microphone arrays for stereo and surround sound. » 50th Audio Engineering Society Conférence.). Le format ambisonique est un ensemble de canaux audio qui contient toutes les informations nécessaires à la reconstruction spatiale du champ sonore. Une nouveauté apportée par le présent brevet est la possibilité d'utiliser une forme quelconque de réseau de microphones. Ainsi, il est tout à fait possible d'utiliser une forme déjà existante, telle une caméra 360° ou un téléphone portable, pour y inclure un certain nombre de microphones. On obtient alors un système complet et compact d'enregistrement du son et de l'image à 360° . STATE OF THE ART Processes and systems for broadcasting 360 ° video signals are known in the state of the art. There is a need in the state of the art to be able to associate audio signals with these 360 ° video signals. Until now, 3D audio was reserved for sound professionals and researchers. The purpose of this technology is to capture as much spatial information as you can during the recording, then return it to the listener and give it a sense of immersion in the soundscape. In the field of video, there is a growing interest in 360-degree video footage, which is reproduced via a virtual reality headset for complete immersion in the image: the user can turn his head and explore the visual scene all around from him. To obtain the same fidelity in the field of sound, the most compact solution is the use of a network of microphones, such as the eigenmike of mh acoustics, the Soundfield of TSL Products, and the TetraMic Core Sound. Equipped with four to thirty-two microphones, these products are expensive and therefore reserved for professional use. Recent research has reduced the number of microphones (Palacino, JD, & Nicol, R. (2013).) "Spatial sound pick-up with a low number of microphones." ICA 2013. Montreal, Canada. Microphones of reduced size and cost can be used such as those available to mobile phones. The shape of the microphone arrays, a polyhedron, remains standardized, however, from the dodecahedron for the EigenMike to the tetrahedron for the Soundfield and the TetraMic. This geometric shape makes it possible to use simple formulas to convert microphone signals into a format ambisonic, and were developed by Gerzon in 1975 (Gerzon, M. (1975). "The design of precisely coincide microphone arrays for stereo and surround sound." 50th Audio Engineering Society Conference.). The ambisonic format is a set of audio channels that contains all the information necessary for the spatial reconstruction of the sound field. A novelty provided by this patent is the possibility of using any form of microphone array. Thus, it is quite possible to use an already existing form, such as a 360 ° camera or a mobile phone, to include a certain number of microphones. This gives a complete and compact sound and image recording system at 360 °.
Exposé de l'invention Presentation of the invention
La présente invention entend remédier aux inconvénients de l'art antérieur en proposant un procédé de traitement du signal sonore permettant de réaliser une captation du signal sonore dans toutes les directions, puis de restituer ledit signal sonore. The present invention intends to overcome the drawbacks of the prior art by proposing a method of processing the sound signal making it possible to capture the sound signal in all directions and then to restore the sound signal.
A cet effet, la présente invention concerne, dans son acception la plus générale, un procédé de traitement du signal sonore, caractérisé en ce qu'il comporte les étapes suivantes : To this end, the present invention relates, in its most general sense, to a sound signal processing method, characterized in that it comprises the following steps:
• Captation de façon synchrone d'un signal sonore d'entrée (Sentrée) à l'aide de N microphones, N étant un entier naturel supérieur ou égal à trois ; · Encodage dudit signal sonore d'entrée (Sentrée) en un format de données (D) de son, ledit encodage comportant une sous-étape de transformation dudit signal d'entrée en un format de type ambisonique d'ordre R, R étant un entier naturel supérieur ou égal à un, ladite sous-étape de transformation en un format de type ambisonique étant mise en œuvre à l'aide d'une transformée de• Capture synchronously a sound input signal (S in tre e) using N microphones, N being a natural integer greater than or equal to three; Encoding said input sound signal (S en tré e ) into a sound data format (D), said encoding comprising a substep of transforming said input signal into an ambisonic format of R-order, R being a natural integer greater than or equal to one, said substep of transformation into an ambisonic format is implemented using a transform of
Fourier rapide, d'une multiplication matricielle, d'une transformée de Fourier rapide inverse et à l'aide d'un filtre passe-bande ; et • Restitution d'un signal sonore de sortie (SSOrtie) à l'aide d'un traitement numérique desdites données (D) de son. Fast Fourier, Matrix Multiplication, Fast Fourier Transform, and Bandpass Filter; and • Restoring an output sound signal (S SO rtie) using a digital processing of the sound data (D).
Ainsi, grâce au procédé selon la présente invention, il est possible de réaliser une captation du signal sonore dans toutes les directions, puis de restituer ledit signal sonore. Thus, thanks to the method according to the present invention, it is possible to capture the sound signal in all directions, then to restore said sound signal.
Avantageusement, le calcul matriciel fait intervenir une matrice H calculée par la méthode des moindres carrés à partir des directivités mesurées des N microphones et des directivités idéales des composantes ambisoniques. Advantageously, the matrix calculation involves a matrix H calculated by the least squares method from the measured directivities of the N microphones and the ideal directivities of the ambison components.
Selon un mode de réalisation, lesdits microphones sont disposés selon un cercle sur un plan, espacés suivant un angle égal à 360°/N ou à chaque coin d'un téléphone portable. Selon un mode de réalisation, ledit procédé met en œuvre quatre microphones espacés suivant un angle de 90° à l'hoirzontal. According to one embodiment, said microphones are arranged in a circle on a plane, spaced at an angle equal to 360 ° / N or at each corner of a mobile phone. According to one embodiment, said method implements four microphones spaced at an angle of 90 ° to the hoirzontal.
Selon un mode de réalisation, ledit procédé met en œuvre un filtre passe bande filtrant de 100 Hz à 6 kHz. According to one embodiment, said method implements a filter band pass filter from 100 Hz to 6 kHz.
Selon un mode de réalisation, l'ordre R du format de type ambisonique est égal à un. According to one embodiment, the order R of the ambisonic format is equal to one.
Avantageusement, au cours de ladite étape de restitution, une information relative à l'orientation de la tête d'un utilisateur écoutant le signal sonore, est exploitée. Advantageously, during said restitution step, information relating to the orientation of the head of a user listening to the sound signal, is exploited.
De préférence, une captation de ladite information relative à l'orientation de la tête d'un utilisateur écoutant le signal sonore, est réalisée par un capteur au sein d'un téléphone mobile ou bien par un capteur situé dans un casque d'écoute ou un casque de réalité virtuelle. Selon un mode de réalisation, au cours de ladite étape de restitution, les données sous format ambisonique sont transformées en données sous format binaural. Preferably, a capture of said information relating to the orientation of the head of a user listening to the sound signal, is performed by a sensor within a mobile phone or by a sensor located in a headset or a virtual reality helmet. According to one embodiment, during said restitution step, the data in ambisonic format is transformed into data in binaural format.
La présente invention se rapporte également à un système de traitement du signal sonore, comportant des moyens pour : The present invention also relates to a system for processing the sound signal, comprising means for:
• Capter de façon synchrone un signal sonore d'entrée (Sentrée) à l'aide de N microphones, N étant un entier naturel supérieur ou égal à trois ; • Synchronously capture an input sound signal (S in tré e ) using N microphones, N being a natural integer greater than or equal to three;
• Encoder ledit signal sonore d'entrée (Sentrée) en un format de données (D) de son, et des moyens pour transformer ledit signal d'entrée en un format de type ambisonique d'ordre R, R étant un entier naturel supérieur ou égal à un, lesdits moyens pour transformer en un format de type ambisonique étant mis en œuvre à l'aide d'une transformée de Fourier rapide, d'une multiplication matricielle, d'une transformée de Fourier rapide inverse et à l'aide d'un filtre passe-bande ; et • Encoding said input sound signal (S in tré e ) into a sound data format (D), and means for transforming said input signal into an ambisonic format of order R, R being an integer greater than or equal to one, said means for transforming into an ambisonic format being implemented using a fast Fourier transform, a matrix multiplication, a fast inverse Fourier transform and a using a bandpass filter; and
• Restituer un signal sonore de sortie (SSOrtie) à l'aide d'un traitement numérique desdites données (D) de son. • Restore an output sound signal (S SO rt) using a digital processing of the sound data (D).
Brève description des dessins Brief description of the drawings
On comprendra mieux l'invention à l'aide de la description, faite ci-après à titre purement explicatif, d'un mode de réalisation de l'invention, en référence aux Figures dans lesquelles : The invention will be better understood by means of the description, given below purely for explanatory purposes, of one embodiment of the invention, with reference to the figures in which:
• Les Figures 1 et 3 représentent les différentes étapes du procédé selon la présente invention ;  Figures 1 and 3 show the different steps of the process according to the present invention;
• La Figure 2 illustre les traitements appliqués dans le cadre de la seconde étape du procédé selon la présente invention ; · Les Figures 4a, 4b et 4c représentent les composantes W, Y et X idéales d'un format ambisonique d'ordre 1 (sur plan horizontal) ; • Les Figures 5a, 5b et 5c illustrent les composantes W, Y et X approximées d'un format ambisonique d'ordre 1 ; et FIG. 2 illustrates the treatments applied in the context of the second step of the process according to the present invention; · Figures 4a, 4b and 4c represent the ideal W, Y and X components of an ambisonic format of order 1 (on a horizontal plane); FIGS. 5a, 5b and 5c illustrate the approximated W, Y and X components of an ambisonic format of order 1; and
• La Figure 6 représente le placement de huit haut-parleurs virtuels, chacun placé à 45° autour d'un utilisateir.  • Figure 6 shows the placement of eight virtual speakers, each placed at 45 ° around a user.
Description détaillée des modes de réalisation de l'invention DETAILED DESCRIPTION OF THE EMBODIMENTS OF THE INVENTION
La présente invention se rapporte à un procédé de traitement du signal sonore, comportant les étapes suivantes : The present invention relates to a sound signal processing method, comprising the following steps:
• Captation de façon synchrone d'un signal sonore d'entrée Sentrée à l'aide de N microphones, N étant un entier naturel supérieur ou égal à trois; • Capture synchronously with an input sound signal S in tre e using N microphones, N being a natural integer greater than or equal to three;
• Encodage dudit signal sonore d'entrée Sentrée en un format de données D de son, ledit encodage comportant une sous-étape de transformation dudit signal d'entrée en un format de type ambisonique d'ordre R, R étant un entier naturel supérieur ou égal à un, ladite sous-étape de transformation en un format de type ambisonique étant mise en œuvre à l'aide d'une transformée de Fourier rapide, d'une multiplication matricielle, d'une transformée de Fourier rapide inverse et à l'aide d'un filtre passe-bande ; et • encoding said sound signal S in input tre e in a data format D thereof, said encoding comprising a sub-step of processing said input signal in a serial format type ambisonic R, R being an integer greater than or equal to one, said substep of transformation into an ambisonic format being implemented using a fast Fourier transform, a matrix multiplication, a fast inverse Fourier transform, and using a bandpass filter; and
• Restitution d'un signal sonore de sortie à l'aide d'un traitement numérique desdites données D de son. • Restitution of an exit signal using a digital processing of said data D sound.
Les Figures 1 et 3 illustrent les différentes étapes du procédé selon la présente invention. Figures 1 and 3 illustrate the different steps of the method according to the present invention.
Dans un mode de réalisation, lesdits microphones sont disposés selon un cercle sur un plan, espacés suivant un angle égal à 360°/N ou à chaque coin d'un téléphone portable. Dans un mode de réalisation, le procédé selon la présente invention met en œuvre quatre microphones espacés suivant un angle de 90° à l'horizontal. In one embodiment, said microphones are arranged in a circle on a plane, spaced at an angle equal to 360 ° / N or at each corner of a mobile phone. In one embodiment, the method according to the present invention implements four microphones spaced at an angle of 90 ° to the horizontal.
Dans un mode de réalisation, l'ordre R du format de type ambisonique est égal à un. In one embodiment, the order R of the ambisonic format is one.
La première étape du procédé selon la présente invention consiste en l'enregistrement du signal sonore. On utilise pour cet enregistrement N microphones, N étant un entier naturel supérieur ou égal à trois, lesdits microphones étant disposés selon un cercle sur un plan, espacés suivant un angle égal à 360°/N ou à chaque coin d'un téléphone portable. Dans l'exemple de réalisation décrit ci-après, N est égal à quatre et les microphones sont espacés de 90° . Ces microphones sont disposés selon un cercle sur un plan. Dans un exemple particulier de mise en œuvre, le rayon dudit cercle est de deux centimètres, et les microphones sont omnidirectionnels. The first step of the method according to the present invention consists in the recording of the sound signal. N microphones are used for this recording, N being a natural number greater than or equal to three, said microphones being arranged in a circle on a plane, spaced at an angle equal to 360 ° / N or at each corner of a mobile phone. In the embodiment described below, N is equal to four and the microphones are spaced 90 °. These microphones are arranged in a circle on a plane. In a particular example of implementation, the radius of said circle is two centimeters, and the microphones are omnidirectional.
Le signal sonore est capté par lesdits microphones, et numérisé. Il s'agit d'une captation synchrone. On obtient à l'issue de cette première étape quatre signaux numériques échantillonnés. The sound signal is picked up by said microphones, and digitized. This is a synchronous capture. At the end of this first step, four sampled digital signals are obtained.
La seconde étape du procédé selon la présente invention consiste en l'encodage desdits quatre signaux numériques échantillonnés, en un format de type ambisonique d'ordre R, R étant un entier naturel supérieur ou égal à un. The second step of the method according to the present invention consists in the encoding of said four sampled digital signals, in an ambisonic format of order R, R being a natural integer greater than or equal to one.
On rappelle ici que le format ambisonique est un format normalisé de codage audio en plusieurs dimensions. Dans l'exemple de réalisation décrit ci-après, l'ordre R est égal à un. Cet ordre 1 permet de représenter le son avec les notions suivantes : Avant - Arrière et Gauche - Droite. Les Figures 4a, 4b et 4c représentent les composantes W, Y et X idéales d'un format ambisonique d'ordre 1 (sur plan horizontal). It is recalled here that the ambisonic format is a standard format of multi-dimensional audio coding. In the embodiment described below, the order R is equal to one. This order 1 makes it possible to represent the sound with the following notions: Forward - Back and Left - Right. Figures 4a, 4b and 4c represent the ideal W, Y and X components of an ambisonic format of order 1 (on a horizontal plane).
Les Figures 5a, 5b et 5c illustrent les composantes W, Y et X approximées d'un format ambisonique d'ordre 1 . Figures 5a, 5b and 5c illustrate the approximated W, Y and X components of an ambisonic format of order 1.
La Figure 2 illustre les traitements appliqués dans le cadre de la seconde étape du procédé selon la présente invention. On observe sur la Figure 2 que les données en entrée sont dans le domaine temporel, passent dans le domaine fréquentiel suite à une opération de transformée de Fourier rapide (FFT ou « Fast Fourier Transform » en terminologie anglo-saxonne), puis les données en sortie sont dans le domaine temporel suite à une opération de transformée de Fourier rapide inverse (IFFT ou « Inverse Fast Fourier Transform » en terminologie anglo-saxonne). Figure 2 illustrates the treatments applied in the context of the second step of the process according to the present invention. It can be observed in FIG. 2 that the input data are in the time domain, pass into the frequency domain following a Fast Fourier Transform (FFT) operation, and then the data in FIG. output are in the time domain following an inverse fast Fourier transform (IFFT) operation.
De préférence, on utilise des fenêtres de Hanning avec un recouvrement en mettant en œuvre une fonction de type « overlap-add ». On observe également sur la Figure 2 que les données fréquentielles d'entrées sont modifiées à l'aide d'une multiplication matricielle. Cette matrice comporte des coefficients pondérateurs pour chaque signal de microphone et chaque fréquence. On observe également sur la Figure 2 qu'un filtrage au moyen d'un filtre passe-bande est réalisé sur les données avant la sortie. Preferably, Hanning windows with overlap are used by implementing an "overlap-add" type function. It can also be seen in FIG. 2 that the input frequency data are modified using matrix multiplication. This matrix comprises weighting coefficients for each microphone signal and each frequency. It can also be seen in FIG. 2 that filtering by means of a bandpass filter is performed on the data before the output.
Dans un mode de réalisation, le procédé selon la présente invention met en œuvre un filtre passe bande filtrant de 100 Hz à 6 kHz. On élimine ainsi la partie basse et la partie aiguë. In one embodiment, the method according to the present invention implements a filter bandpass filter from 100 Hz to 6 kHz. This eliminates the lower part and the acute part.
Afin de calculer les coefficients de la matrice de pondération, on mesure des réponses impulsionnelles des N microphones, dans le cas présent des quatre microphones, avec une source positionnée tous les 5° ou tous les 10° autour du réseau de microphones. In order to calculate the coefficients of the weighting matrix, impulse responses of the N microphones are measured, in this case four microphones, with a source positioned every 5 ° or every 10 ° around the microphone array.
A l'aide d'une transformée de Fourier rapide, on obtient les réponses en fréquence des N microphones en fonction des angles mesurés, ou autrement dit les directivités des N microphones en fonction de la fréquence. Using a fast Fourier transform, we obtain the frequency responses of the N microphones as a function of the measured angles, or in other words the directivities of the N microphones as a function of the frequency.
Il est possible d'utiliser à ce stade les principes du procédé décrit dans la demande internationale publiée sous le numéro WO 2015/128160 « Procédé et système d'égalisation acoustique automatisé » pour égaliser les réponses en fréquence sur l'axe de chacun des microphones. Les mêmes filtres d'égalisation sont appliqués à tous les microphones et pour toutes les positions angulaires de source. Les réponses des microphones sont ensuite placées dans une matriceIt is possible to use at this stage the principles of the method described in the international application published under the number WO 2015/128160 "Method and automated acoustic equalization system" for equalizing the frequency responses on the axis of each of the microphones . The same equalization filters are applied to all microphones and for all source angular positions. The responses of the microphones are then placed in a matrix
C. vs.
Dans le domaine fréquentiel, pour chaque index de fréquence k, on a où N est le nombre de microphones (quatre dans le présent exemple de réalisation), D est le nombre de positions angulaires de source mesurées (108 dans le présent exemple de réalisation) et V est le nombre de canaux ambisoniques (trois dans le présent exemple de réalisation), CDXN désigne les directivités des microphones, HNxv désigne la matrice qui transforme les directivités des microphones en les directivités désirées, et PDXV désigne les directivités prescrites par le format ambisonique (W, X et Y dans le présent exemple de réalisation). In the frequency domain, for each frequency index k, we have where N is the number of microphones (four in the present embodiment), D is the number of measured source angular positions (108 in the present embodiment) and V is the number of ambison channels (three in the present example) embodiment), C DX N denotes the directivities of the microphones, H Nx v denotes the matrix which transforms the directivities of the microphones into the desired directivities, and PDXV denotes the directivities prescribed by the ambisonic format (W, X and Y in the present example realization).
On a ainsi HNxv = PDXV / CDXN pour chaque index de fréquence k si CDXN est inversible. En pratique, CDXN n'est pas inversible. Dans un mode de réalisation, on met en œuvre une méthode des moindres carrés pour résoudre C-iosx4■ H4x3 =We thus have H Nx v = PDXV / CDXN for each index of frequency k if CDXN is invertible. In practice, CD X N is not invertible. In one embodiment, it implements a method of least squares to solve C-iosx4 ■ H 4x3 =
P 108x3 La matrice H est définie une fois pour les utilisations futures du réseau de microphones considéré. Ensuite, à chaque utilisation on réalise une multiplication matricielle dans le domaine fréquentiel. P 108x3 The matrix H is defined once for future uses of the considered microphone array. Then, with each use, a matrix multiplication is performed in the frequency domain.
Ladite matrice H possède autant de lignes que de microphones donc quatre dans le présent exemple de réalisation, et autant de colonnes que le requiert l'ordre du format ambisonique utilisé, donc trois colonnes dans le présent exemple de réalisation dans lequel l'ordre 1 est mis en œuvre sur le plan horizontal. On a Out = In x H , où H désigne la matrice précédemment calculée, In désigne l'entrée (canaux audio provenant du réseau de microphones, passés dans le domaine fréquentiel) et Out désigne la sortie (Out étant reconverti dans le domaine temporel pour obtenir le format ambisonique). Le procédé selon la présente invention met en œuvre, au cours de cette seconde étape, un algorithme dit algorithme des moindres carrés pour chaque fréquence, avec par exemple 512 points de fréquence. Said matrix H has as many lines as microphones, therefore four in the present embodiment, and as many columns as required by the order of the ambisonic format used, therefore three columns in the present embodiment in which the order 1 is implemented on the horizontal plane. We have Out = In x H, where H denotes the previously computed matrix, In denotes the input (audio channels coming from the microphones array, passed in the frequency domain) and Out denotes the output (Out being reconverted into the time domain for get the ambisonic format). The method according to the present invention implements, during this second step, an algorithm called least squares algorithm for each frequency, with for example 512 frequency points.
A l'issue de cette seconde étape, on obtient des données sous le format ambisonique (dans le présent exemple de réalisation les signaux W, X et Y). At the end of this second step, data is obtained in the ambisonic format (in the present embodiment the signals W, X and Y).
La troisième étape du procédé selon la présente invention consiste en la restitution du signal sonore, grâce à une transformation des données sous format ambisonique en deux canaux binauraux. The third step of the method according to the present invention consists in the restitution of the sound signal, thanks to a transformation of the data in ambisonic format into two binaural channels.
Au cours de cette troisième étape, l'information relative à l'orientation de la tête de l'utilisateur écoutant le signal sonore, est récupérée et exploitée. Ceci peut être réalisé par un capteur au sein d'un téléphone mobile, d'un casque d'écoute ou d'un casque de réalité virtuelle. During this third step, the information relating to the orientation of the head of the user listening to the sound signal is retrieved and used. This can be performed by a sensor in a mobile phone, a headset or a virtual reality headset.
Cette information sur l'orientation consiste en un vecteur comportant trois valeurs d'angles, en terminologie anglo-saxonne « pitch », « yaw » et « mil ». This orientation information consists of a vector comprising three angle values, in the English terminology "pitch", "yaw" and "millet".
Dans le présent exemple de réalisation, sur un plan, on utilise la valeur d'angle « yaw ». On transforme le format ambisonique en huit canaux audio correspondant à un placement virtuel de huit haut-parleurs, chacun placé à 45° autour de l'utilisateur. In the present exemplary embodiment, on a plane, the angle value "yaw" is used. The ambisonic format is transformed into eight audio channels corresponding to a virtual placement of eight loudspeakers, each placed at 45 ° around the user.
La Figure 6 représente le placement de huit haut-parleurs virtuels, chacun placé à 45° autour d'un utilisateur. Figure 6 shows the placement of eight virtual speakers, each placed at 45 ° around a user.
Chaque haut parleur virtuel restitue un signal audio issu des composantes ambisoniques selon la formule: Pn = W + XcosBn + YsinBn (1 ) où W, X et Y sont les données relatives au format ambisonique, et où θη représente l'angle horizontal du nième haut-parleur. Par exemple, dans le présent exemple de réalisation θ0 = 0° , θι = 45° , θ2 = 90° , etc. Each virtual speaker renders an audio signal from the ambisonic components according to the formula: P n = W + XcosBn + YsinBn (1) where W, X and Y are the data relating to the ambisonic format, and where θ η represents the horizontal angle of the nth speaker. For example, in the present embodiment θ 0 = 0 °, θι = 45 °, θ 2 = 90 °, etc.
Ensuite, on effectue une étape de filtrage avec une paire de HRTF par haut-parleur, HRTF signifiant « Head-related transfer function » en terminologie anglo-saxonne. On associe une paire de filtres HRTF (oreille gauche et oreille droite) à chaque haut-parleur virtuel, puis on additionne (tous les canaux « oreille gauche » et tous les canaux « oreille droite » ensemble), afin de former deux canaux de sortie. Next, a filtering step is carried out with a pair of HRTFs per loudspeaker, HRTF signifying "Head-related transfer function" in English terminology. A pair of HRTF (left ear and right ear) filters are paired with each virtual speaker, and then all ("left ear" channels and all "right ear" channels together) are added to form two output channels .
Des coefficients IIR (« Infinité Impulse Response ») sont mis en œuvre à ce stade, lesdits filtres HRTF étant modélisés sous formes de filtres IIR. Lorsque l'utilisateur tourne la tête, la position des haut-parleurs virtuels est modifiée. Par exemple pour une rotation de la tête d'un angle a, l'angle des haut-parleurs virtuels devient βη = θη-α. On remplace alors θη par (θη-α) dans la formule (1 ) pour calculer le signal restitué par le n'eme haut-parleur virtuel. IIR (Infinity Impulse Response) coefficients are implemented at this stage, said HRTF filters being modeled as IIR filters. When the user turns the head, the position of the virtual speakers is changed. For example, for a rotation of the head of an angle a, the angle of the virtual speakers becomes β η = θ η -α . Then θ η is replaced by (θ η -α) in the formula (1) for calculating the signal output from the n 'th virtual loudspeaker.
Ainsi, grâce au procédé selon la présente invention, il est possible de réaliser une captation du signal sonore dans toutes les directions, puis de restituer ledit signal sonore. Thus, thanks to the method according to the present invention, it is possible to capture the sound signal in all directions, then to restore said sound signal.
La Figure 3 représente les différentes étapes du procédé selon la présente invention. Figure 3 shows the different steps of the method according to the present invention.
La présente invention se rapporte également à un système de traitement du signal sonore, comportant des moyens pour : The present invention also relates to a system for processing the sound signal, comprising means for:
• Capter de façon synchrone un signal sonore d'entrée Sentrée à l'aide de N microphones, N étant un entier naturel supérieur ou égal à trois ; • Capture synchronously an input sound signal S in tre e using N microphones, N being a natural integer greater than or equal to three;
• Encoder ledit signal sonore d'entrée Sentrée en un format de données D de son, et des moyens pour transformer ledit signal d'entrée en un format de type ambisonique d'ordre R, R étant un entier naturel supérieur ou égal à un, lesdits moyens pour transformer en un format de type ambisonique étant mis en œuvre à l'aide d'une transformée de Fourier rapide, d'une multiplication matricielle, d'une transformée de Fourier rapide inverse et à l'aide d'un filtre passe-bande ; et • Encoder said sound S in input signal e tre D into sound data format, and means for converting said input signal into a command format type ambisonic R, R being greater than or equal integer to one, said means for transforming into an ambisonic-like format being implemented using a fast Fourier transform, a matrix multiplication, a fast inverse Fourier transform and with the aid of a bandpass filter; and
• Restituer un signal sonore de sortie Ssortie à l'aide d'un traitement numérique desdites données D de son. • Restore a sound output signal S outgoing part using digital processing said sound data D.
Ce système de traitement du signal sonore comprend au moins une unité de calcul et une unité de mémoire. L'invention est décrite dans ce qui précède à titre d'exemple. Il est entendu que l'homme du métier est à même de réaliser différentes variantes de l'invention sans pour autant sortir du cadre du brevet. This sound signal processing system comprises at least one calculation unit and one memory unit. The invention is described in the foregoing by way of example. It is understood that the skilled person is able to realize different variants of the invention without departing from the scope of the patent.

Claims

REVENDICATIONS
1 . Procédé de traitement du signal sonore, caractérisé en ce qu'il comporte les étapes suivantes : 1. Sound signal processing method, characterized in that it comprises the following steps:
• Captation de façon synchrone d'un signal sonore d'entrée (Sentrée) à l'aide de N microphones, N étant un entier naturel supérieur ou égal à trois; • Capture synchronously a sound input signal (S in tre e) using N microphones, N being a natural integer greater than or equal to three;
• Encodage dudit signal sonore d'entrée (Sentrée) en un format de données (D) de son, ledit encodage comportant une sous-étape de transformation dudit signal d'entrée en un format de type ambisonique d'ordre R, R étant un entier naturel supérieur ou égal à un, ladite sous-étape de transformation en un format de type ambisonique étant mise en œuvre à l'aide d'une transformée de Fourier rapide, d'une multiplication matricielle, d'une transformée de Fourier rapide inverse et à l'aide d'un filtre passe-bande ; et Encoding said input sound signal (S in tré e ) into a sound data format (D), said encoding comprising a substep of transforming said input signal into an ambisonic format of order R, R being a natural integer greater than or equal to one, said substep of transformation into an ambisonic format is implemented using a fast Fourier transform, a matrix multiplication, a transform of Fast Fourier inverse and using a bandpass filter; and
• Restitution d'un signal sonore de sortie (Ssortie) à l'aide d'un traitement numérique desdites données (D) de son ; et en ce que le calcul matriciel fait intervenir une matrice H calculée par la méthode des moindres carrés à partir des directivités mesurées des N microphones et des directivités idéales des composantes ambisoniques. • Return of a sound output signal (S sor t e) with the aid of digital processing of said data (D) thereof; and in that the matrix calculation involves a matrix H calculated by the least squares method from the measured directivities of the N microphones and the ideal directivities of the ambisonic components.
Procédé de traitement du signal sonore selon la revendication 1 , caractérisé en ce que lesdits microphones sont disposés selon un cercle sur un plan, espacés suivant un angle égal à 360 °/l\| ou à chaque coin d'un téléphone portable. Sound signal processing method according to claim 1, characterized in that said microphones are arranged in a circle on a plane, spaced at an angle equal to 360 ° / 1 | or at each corner of a cell phone.
Procédé de traitement du signal sonore selon la revendication 2, caractérisé en ce qu'il met en œuvre quatre microphones espacés suivant un angle de 90° à l'horizontal. Sound signal processing method according to claim 2, characterized in that it implements four microphones spaced at an angle of 90 ° to the horizontal.
Procédé de traitement du signal sonore selon l'une des revendications précédentes, caractérisé en ce qu'il met en œuvre un filtre passe bande filtrant de 1 00 Hz à 6 kHz. Procédé de traitement du signal sonore selon l'une des revendications précédentes, caractérisé en ce que l'ordre R du format de type ambisonique est égal à un. Sound signal processing method according to one of the preceding claims, characterized in that it implements a filter band pass filter from 100 Hz to 6 kHz. Sound signal processing method according to one of the preceding claims, characterized in that the order R of the ambisonic format is one.
Procédé de traitement du signal sonore selon l'une des revendications précédentes, caractérisé en ce qu'au cours de ladite étape de restitution, une information relative à l'orientation de la tête d'un utilisateur écoutant le signal sonore, est exploitée. Sound signal processing method according to one of the preceding claims, characterized in that during said restitution step, information relating to the orientation of the head of a user listening to the sound signal, is exploited.
Procédé de traitement du signal sonore selon la revendication 6, caractérisé en ce qu'une captation de ladite information relative à l'orientation de la tête d'un utilisateur écoutant le signal sonore, est réalisée par un capteur au sein d'un téléphone, d'un casque d'écoute ou d'un casque de réalité virtuelle. Sound signal processing method according to claim 6, characterized in that a capture of said information relating to the orientation of the head of a user listening to the sound signal is performed by a sensor within a telephone, headphones or virtual reality headphones.
Procédé de traitement du signal sonore selon l'une des revendications précédentes, caractérisé en ce qu'au cours de ladite étape de restitution, les données sous format ambisonique sont transformées en données sous format binaural. Sound signal processing method according to one of the preceding claims, characterized in that during said restitution step, the data in ambisonic format are transformed into data in binaural format.
Système de traitement du signal sonore, caractérisé en ce qu'il comporte des moyens pour : Sound signal processing system, characterized in that it comprises means for:
• Capter de façon synchrone un signal sonore d'entrée (Sentrée) à l'aide de N microphones, N étant un entier naturel supérieur ou égal à trois ; • Synchronously capture an input sound signal (S in tré e ) using N microphones, N being a natural integer greater than or equal to three;
• Encoder ledit signal sonore d'entrée (Sentrée) en un format de données (D) de son, et des moyens pour transformer ledit signal d'entrée en un format de type ambisonique d'ordre R, R étant un entier naturel supérieur ou égal à un, lesdits moyens pour transformer en un format de type ambisonique étant mis en œuvre à l'aide d'une transformée de Fourier rapide, d'une multiplication matricielle, d'une transformée de Fourier rapide inverse et à l'aide d'un filtre passe-bande ; et • Restituer un signal sonore de sortie (SSOrtie) à l'aide d'un traitement numérique desdites données (D) de son ; et en ce que le calcul matriciel fait intervenir une matrice H calculée par la méthode des moindres carrés à partir des directivités mesurées des N microphones et des directivités idéales des composantes ambisoniques. • Encoding said input sound signal (S in tré e ) into a sound data format (D), and means for transforming said input signal into an ambisonic format of order R, R being an integer greater than or equal to one, said means for transforming into an ambisonic format being implemented using a fast Fourier transform, a matrix multiplication, a fast inverse Fourier transform and a using a bandpass filter; and • Restore an output sound signal (S SO ) by digital processing of the sound data (D); and in that the matrix calculation involves a matrix H calculated by the least squares method from the measured directivities of the N microphones and the ideal directivities of the ambisonic components.
EP17725294.7A 2016-04-26 2017-04-20 Method and system of broadcasting a 360° audio signal Active EP3449643B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
FR1653684A FR3050601B1 (en) 2016-04-26 2016-04-26 METHOD AND SYSTEM FOR BROADCASTING A 360 ° AUDIO SIGNAL
PCT/FR2017/050935 WO2017187053A1 (en) 2016-04-26 2017-04-20 Method and system of broadcasting a 360° audio signal

Publications (2)

Publication Number Publication Date
EP3449643A1 true EP3449643A1 (en) 2019-03-06
EP3449643B1 EP3449643B1 (en) 2020-06-10

Family

ID=56943619

Family Applications (1)

Application Number Title Priority Date Filing Date
EP17725294.7A Active EP3449643B1 (en) 2016-04-26 2017-04-20 Method and system of broadcasting a 360° audio signal

Country Status (5)

Country Link
US (1) US10659902B2 (en)
EP (1) EP3449643B1 (en)
CN (1) CN109661824A (en)
FR (1) FR3050601B1 (en)
WO (1) WO2017187053A1 (en)

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AUPO099696A0 (en) * 1996-07-12 1996-08-08 Lake Dsp Pty Limited Methods and apparatus for processing spatialised audio
US6021206A (en) * 1996-10-02 2000-02-01 Lake Dsp Pty Ltd Methods and apparatus for processing spatialised audio
NZ502603A (en) * 2000-02-02 2002-09-27 Ind Res Ltd Multitransducer microphone arrays with signal processing for high resolution sound field recording
FR2858512A1 (en) * 2003-07-30 2005-02-04 France Telecom METHOD AND DEVICE FOR PROCESSING AUDIBLE DATA IN AN AMBIOPHONIC CONTEXT
GB0906269D0 (en) * 2009-04-09 2009-05-20 Ntnu Technology Transfer As Optimal modal beamformer for sensor arrays
US9986356B2 (en) * 2012-02-15 2018-05-29 Harman International Industries, Incorporated Audio surround processing system
US9736609B2 (en) * 2013-02-07 2017-08-15 Qualcomm Incorporated Determining renderers for spherical harmonic coefficients
US9685163B2 (en) * 2013-03-01 2017-06-20 Qualcomm Incorporated Transforming spherical harmonic coefficients
CN104424953B (en) * 2013-09-11 2019-11-01 华为技术有限公司 Audio signal processing method and device
FR3018015B1 (en) 2014-02-25 2016-04-29 Arkamys AUTOMATED ACOUSTIC EQUALIZATION METHOD AND SYSTEM

Also Published As

Publication number Publication date
FR3050601A1 (en) 2017-10-27
US20190132695A1 (en) 2019-05-02
EP3449643B1 (en) 2020-06-10
US10659902B2 (en) 2020-05-19
FR3050601B1 (en) 2018-06-22
WO2017187053A1 (en) 2017-11-02
CN109661824A (en) 2019-04-19

Similar Documents

Publication Publication Date Title
EP1563485B1 (en) Method for processing audio data and sound acquisition device therefor
EP2000002B1 (en) Method and device for efficient binaural sound spatialization in the transformed domain
EP1836876B1 (en) Method and device for individualizing hrtfs by modeling
EP1992198B1 (en) Optimization of binaural sound spatialization based on multichannel encoding
EP1600042B1 (en) Method for the treatment of compressed sound data for spatialization
CN104581610B (en) A kind of virtual three-dimensional phonosynthesis method and device
WO2007110520A1 (en) Method for binaural synthesis taking into account a theater effect
EP1586220B1 (en) Method and device for controlling a reproduction unit using a multi-channel signal
US20160212564A1 (en) Apparatus and Method for Compressing a Set of N Binaural Room Impulse Responses
US20130243201A1 (en) Efficient control of sound field rotation in binaural spatial sound
FR3065137A1 (en) SOUND SPATIALIZATION METHOD
EP3559947A1 (en) Processing in sub-bands of an actual ambisonic content for improved decoding
WO2018050292A1 (en) Device and method for capturing and processing a three-dimensional acoustic field
EP3449643B1 (en) Method and system of broadcasting a 360° audio signal
WO2021212287A1 (en) Audio signal processing method, audio processing device, and recording apparatus
EP3384688B1 (en) Successive decompositions of audio filters
US11432092B2 (en) Method and system for processing an audio signal including ambisonic encoding
CN116615919A (en) Post-processing of binaural signals

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20181113

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20191126

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

Ref country code: AT

Ref legal event code: REF

Ref document number: 1280121

Country of ref document: AT

Kind code of ref document: T

Effective date: 20200615

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602017018008

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: FRENCH

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200610

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200910

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200610

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200610

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20200610

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200610

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200610

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200610

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200910

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1280121

Country of ref document: AT

Kind code of ref document: T

Effective date: 20200610

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200610

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200610

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201012

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200610

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200610

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200610

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200610

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200610

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200610

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200610

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200610

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200610

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201010

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602017018008

Country of ref document: DE

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200610

26N No opposition filed

Effective date: 20210311

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200610

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200610

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20210420

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210420

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20210430

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210420

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210430

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210430

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210420

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201010

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210430

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230512

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200610

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20170420

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200610

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200610

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200610

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240429

Year of fee payment: 8

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20240426

Year of fee payment: 8

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200610