EP2815589A1 - Transaural synthesis method for sound spatialization - Google Patents

Transaural synthesis method for sound spatialization

Info

Publication number
EP2815589A1
EP2815589A1 EP13710449.3A EP13710449A EP2815589A1 EP 2815589 A1 EP2815589 A1 EP 2815589A1 EP 13710449 A EP13710449 A EP 13710449A EP 2815589 A1 EP2815589 A1 EP 2815589A1
Authority
EP
European Patent Office
Prior art keywords
signal
sound
channels
signals
stereo
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP13710449.3A
Other languages
German (de)
French (fr)
Other versions
EP2815589B1 (en
Inventor
Franck Rosset
Jean-Luc HAURAIS
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Publication of EP2815589A1 publication Critical patent/EP2815589A1/en
Application granted granted Critical
Publication of EP2815589B1 publication Critical patent/EP2815589B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S3/00—Systems employing more than two channels, e.g. quadraphonic
    • H04S3/002—Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R5/00—Stereophonic arrangements
    • H04R5/033—Headphones for stereophonic communication
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S3/00—Systems employing more than two channels, e.g. quadraphonic
    • H04S3/02—Systems employing more than two channels, e.g. quadraphonic of the matrix type, i.e. in which input signals are combined algebraically, e.g. after having been phase shifted with respect to each other
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
    • H04S2400/01—Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
    • H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]

Definitions

  • the present invention relates to the field of spacialized sound spatialization of sound signals, integrating in particular a room effect, particularly in the field of transaural techniques.
  • the term "binaural” refers to the reproduction on a stereophonic headphones, or a pair of headphones or a pair of speakers, a sound signal with nevertheless spatialization effects.
  • the invention is however not limited to the aforementioned technique and applies, in particular, to techniques derived from "binaural” such as "transaural" (commercial name) restitution techniques, that is to say on remote loudspeakers, installed for example in a concert hall or cinema with a multi-point sound system.
  • a specific application of the invention is, for example, the enrichment of audio content broadcast by a pair of speakers to immerse a listener in a spatialized sound scene, including in particular a room effect or outdoor space.
  • the state of the art defines a transfer function, or filter, of a sound signal between a position of a sound source in the space and the two ears of a listener.
  • the acoustic transfer function of the aforementioned head is designated HRTF for "IHead Related Transfer Function” in English in its form. frequency and HRIR for "JHead Related Impulse Response” in English in its temporal form.
  • HRTF for one direction of space, we finally get two HRTFs: one for the right ear and one for the left ear.
  • the binaural technique consists in applying such acoustic transfer functions of the head to monophonic audio signals, in order to obtain a stereophonic signal which makes it possible, when listening to headphones, to have the feeling that the sources sounds come from a particular direction of space.
  • the signal from the right ear is obtained by filtering the monophonic signal by the HRTF of the right ear and the left ear signal is obtained by filtering the same monophonic signal by the HRTF of the left ear.
  • FIG. 1 represents a general block diagram of the installation intended for the phase of constructing the pulse signal database.
  • FIG. 2 represents a schematic view of the installation for acquiring pulse signals
  • the method according to the invention comprises a first processing (1) of producing a database of pulse signals from the acquisition of acoustic signals in a plurality of physical spaces, by
  • the method consists in applying a succession of
  • the method comprises a preliminary stage (2) of constructing a signal N.i from the stereo signal
  • This stereo signal can then be broadcast by a pair of standard loudspeakers, to restore a spatialized sound environment corresponding to the space that has been used to produce the impulse response signals or a combination of such spaces.
  • This step is replicated a plurality of times.
  • impulses to be arranged in a physical space such as a concert hall, an open or closed place, a given room, a series of known loudspeakers (5 to 11; 17) associated with an amplifier (14), preference of recognized quality, as well as a microphone pair (12, 13) whose position relative to the speaker series (5 to 11; 17) is fixed for the series being acquired.
  • Each of the speakers (5 to 11) is then successively applied to an original multifrequency signal using the amplifier (14).
  • This original signal is for example a sequence of a duration of between 10 and 90 seconds, with a frequency variation in the sound spectrum.
  • This signal is for example a linear variation between 20 Hz and 20 kHz, or any signal covering the entire spectrum of the enclosure.
  • the sound signal produced by the active speaker is picked up by the microphone torque (12, 13) and produces a recorded stereo signal. From this signal, 96 kHz sampling is carried out in a known manner and at a
  • This step is reproduced for each of the speakers (5 to 11) of the series, then for different physical spaces where a series of identical or identical speakers is re-inserted.
  • This first step leads to the construction of a database of stereo impulse responses.
  • This step makes it possible to construct a spatial stereo audio signal from a multichannel signal N.i
  • This step consists in selecting in the database formed during the initial step N + i impulse responses.
  • the selection will consist in associating with each of the N + 1 signals one of the impulse responses of said database, ensuring that the acquisition position in the space of the impulse response corresponds to the position in the space of the channel with which it is associated.
  • a convolution processing is applied to calculate a stereo spatialized signal cut S sG and S sD .
  • channel equalization is performed to improve the dynamics of the signals.
  • the final step is to recombine the signals to build a right and left signal pair
  • the channels are equalized to improve the dynamics of the two channels.
  • Case of a stereo start signal increasing the number of channels and creating intermediate channels
  • the signal to be spatialized is not of the Ni type but simply a stereo signal
  • a step is taken intermediate consisting of building a signal Ni by phase extraction treatments between the left and right track, to build different new signals.
  • This extraction by phase consists in producing a signal corresponding to a reconstructed central channel, by a processing consisting in adding the signal of the left channel with a signal of the straight line which is out of phase, for example in phase opposition.
  • the left and right tracks are phase-shifted, with different phase-shift angles, and the out-of-phase signal pairs are added, with determined weights.
  • frequency filters are applied to right and left signals when creating "reconstructed" channels, in order to increase signal dynamics and maintain high fidelity sound quality.
  • FIG. 3 represents a schematic view of the reproduction installation, from a pair of actual speakers (17, 18).
  • This pair of speakers (17, 18) receives a signal for simulating calculated speakers (20 to 27 and 30 to 37).
  • the effective number of calculated speakers (20 to 27) corresponds to the number of physical speakers (5 to 11; 17) used for the production of the pulse signal database, or to the number of virtual speakers reconstructed by the method referred to above.
  • Virtual speakers (30 to 37) are also created which produce a perception in the sound space of a combination of real neighboring speakers, to fill the sound holes.
  • These virtual speakers are created by a modification of the signal supplying the neighboring real speakers.
  • the signals are distributed according to their right, left or central component to produce a left signal (17) for the left speaker, and a right signal for the right speaker (18):
  • the "right” signal corresponds to the addition of the calculated “right” signals (21, 22, 23) and virtual “right” signals (30, 31, 32), as well as the "central” signals calculated (20, 27) and virtual (33) with an amplitude weighting of 50%
  • the "left" signal corresponds to the addition of the left calculated signals (24, 25, 26) and the left virtual signals (34, 35, 36), as well as the calculated central (20, 27) and virtual (33 ) with an amplitude weighting of 50%.
  • This stereo signal is then applied to a conventional audio equipment, connected to a pair of speakers (18, 19), which will reproduce a spatialized sound environment corresponding to the sound environment of the installation that was used to build the base impulse signals, or a virtual sound environment corresponding to the combination of several original atmospheres, where appropriate enriched with virtual atmospheres.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Algebra (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Mathematical Physics (AREA)
  • Pure & Applied Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Stereophonic System (AREA)

Abstract

The present invention relates to a method for producing a digital spatialized stereo audio file from an original multichannel audio file, characterized in that it comprises: a step of performing a processing on each of the channels for cross-talk cancelation; a step of merging the channels in order to produce a stereo signal; and a dynamic filtering and specific equalization step for increasing the sound dynamics.

Description

PROCÉDÉ DE SYNTHÈSE TRANSAURALE POUR LA SPATIALISATION SONORE  TRANSAURAL SYNTHESIS METHOD FOR SOUND SPACE DELIVERY
Domaine de 1 ' invention La présente invention concerne le domaine de la spatialisation sonore, dite rendu spacialisé, de signaux audio, intégrant en particulier un effet de salle, notamment dans le domaine des techniques transaurales. Le terme " binaural " vise la restitution sur un casque stéréophonique, ou une paire d'écouteurs ou encore une paire d'enceintes, d'un signal sonore avec néanmoins des effets de spatialisation. L'invention ne se limite toutefois pas à la technique précitée et s'applique, notamment, à des techniques dérivées du "binaural" telles que les techniques de restitution "transaurale" (nom commercial), c'est-à-dire sur des hauts parleurs distants, installés par exemple dans une salle de concert ou de cinéma avec un système sonore multipoint . FIELD OF THE INVENTION The present invention relates to the field of spacialized sound spatialization of sound signals, integrating in particular a room effect, particularly in the field of transaural techniques. The term "binaural" refers to the reproduction on a stereophonic headphones, or a pair of headphones or a pair of speakers, a sound signal with nevertheless spatialization effects. The invention is however not limited to the aforementioned technique and applies, in particular, to techniques derived from "binaural" such as "transaural" (commercial name) restitution techniques, that is to say on remote loudspeakers, installed for example in a concert hall or cinema with a multi-point sound system.
Une application spécifique de l'invention est, par exemple, l'enrichissement des contenus audio diffusé par une paire d'enceintes afin de plonger un auditeur dans une scène sonore spatialisée, incluant en particulier un effet de salle ou d'espace extérieur. A specific application of the invention is, for example, the enrichment of audio content broadcast by a pair of speakers to immerse a listener in a spatialized sound scene, including in particular a room effect or outdoor space.
Etat de la technique State of the art
Pour la mise en oeuvre des techniques "binaurales" sur casque ou haut-parleurs, on définit dans l'état de la technique une fonction de transfert, ou filtre, d'un signal sonore entre une position d'une source sonore dans l'espace et les deux oreilles d'un auditeur. La fonction de transfert acoustique de la tête précitée est désignée HRTF pour "IHead Related Transfer Function" en anglais dans sa forme fréquentielle et HRIR pour "JHead Related Impulse Response" en anglais dans sa forme temporelle. Pour une direction de l'espace, on obtient au final deux HRTF : une pour l'oreille droite et une pour l'oreille gauche. For the implementation of "binaural" techniques on headphones or loudspeakers, the state of the art defines a transfer function, or filter, of a sound signal between a position of a sound source in the space and the two ears of a listener. The acoustic transfer function of the aforementioned head is designated HRTF for "IHead Related Transfer Function" in English in its form. frequency and HRIR for "JHead Related Impulse Response" in English in its temporal form. For one direction of space, we finally get two HRTFs: one for the right ear and one for the left ear.
En particulier, la technique binaurale consiste à appliquer de telles fonctions de transfert acoustique de la tête à des signaux audio monophoniques, afin d'obtenir un signal stéréophonique qui permet, lors d'une écoute au casque, d'avoir la sensation que les sources sonores proviennent d'une direction particulière de l'espace. Le signal de l'oreille droite est obtenu en filtrant le signal monophonique par la HRTF de l'oreille droite et le signal de l'oreille gauche est obtenu en filtrant ce même signal monophonique par la HRTF de l'oreille gauche.  In particular, the binaural technique consists in applying such acoustic transfer functions of the head to monophonic audio signals, in order to obtain a stereophonic signal which makes it possible, when listening to headphones, to have the feeling that the sources sounds come from a particular direction of space. The signal from the right ear is obtained by filtering the monophonic signal by the HRTF of the right ear and the left ear signal is obtained by filtering the same monophonic signal by the HRTF of the left ear.
Lorsque, dans le rendu spatial, l'on prend en compte le fait, pour l'auditeur, de percevoir les sources sonores plus ou moins éloignées de la tête, phénomène connu sous le nom d ' externalisation, et ce de manière indépendante de la direction de provenance des sources sonores, il arrive fréquemment, dans un rendu 3D binaural, que les sources soient perçues à l'intérieur de la tête par l'auditeur. La source ainsi perçue est dite non externalisée . When, in the spatial rendering, we take into account the fact, for the listener, to perceive sound sources more or less distant from the head, a phenomenon known as outsourcing, and this independently of the Direction of provenance of the sound sources, it happens frequently, in a binaural 3D rendering, that the sources are perceived inside the head by the listener. The source thus perceived is said to be not outsourced.
Différents travaux ont montré que l'ajout d'un effet de salle dans les méthodes de rendu 3D binaurales permet d'augmenter considérablement 1 ' externalisation des sources sonores .  Various works have shown that the addition of a room effect in binaural 3D rendering methods makes it possible to considerably increase the outsourcing of sound sources.
On connaît dans l'état de la technique la demande de brevet US 2007/011025A décrivant un procédé de spatialisation de son comportant une étape de détermination d'une matrice acoustique pour un ensemble réel de sources sonores à un emplacement réel et une étape de calcul d'une matrice acoustique pour la transmission d'un signal acoustique d'un ensemble de sources sonores apparentes, à des emplacement différents des emplacements réels de l'auditeur. La méthode inclut plus loin une étape de résolution d'une matrice de fonction de transfert pour présenter à l'auditeur signal audio créant une image audio de son émanant la source apparente. Inconvénients de l'art antérieur The prior art is known from US patent application 2007 / 011025A describing a sound spatialization method comprising a step of determining an acoustic matrix for a real set of sound sources at a real location and a calculation step an acoustic matrix for transmitting an acoustic signal from a set of apparent sound sources at locations different from the actual locations of the listener. The method further includes a step of resolving a transfer function matrix for presenting to the listener an audio signal creating an audio image of sound emanating from the apparent source. Disadvantages of prior art
Les solutions de l'art antérieur sont figés et ne permettent pas de choisir une ambiance spatiale parmi plusieurs ambiances possibles. Elles sont généralement basées sur une matrice de transformation calculée à partir d'une tête virtuelle . The solutions of the prior art are fixed and do not allow to choose a spatial atmosphere among several possible environments. They are usually based on a transformation matrix calculated from a virtual head.
Les solutions de l'art antérieur ne permettent généralement pas une impression d'externalisation de l'environnement sonore.  The solutions of the prior art do not generally allow an impression of externalization of the sound environment.
Solution apportée par l'invention Solution provided by the invention
Les salles physiques et enceintes physiques permettent de calculer les filtres qui seront utilisés pour générer les multicanaux. Physical rooms and physical enclosures make it possible to calculate the filters that will be used to generate the multichannels.
Description détaillée d'un exemple de réalisation non limitatif La présente invention sera mieux comprise à la lecture de la description qui suit, faisant référence aux dessins annexés où : DETAILED DESCRIPTION OF A NON-LIMITATIVE EXEMPLARY EMBODIMENT The present invention will be better understood on reading the description which follows, with reference to the appended drawings in which:
- la figure 1 représente un schéma de principe général de l'installation destiné à la phase de construire de la base de données de signaux impulsionnelle  FIG. 1 represents a general block diagram of the installation intended for the phase of constructing the pulse signal database.
- la figure 2 représente une vue schématique de l'installation pour l'acquisition des signaux impulsionnels  FIG. 2 represents a schematic view of the installation for acquiring pulse signals
- la figure 3 représente un schéma de principe de l'installation d'écoute. Le procédé selon l'invention comporte un premier traitement (1) consistant à produire une base de données de signaux impulsionnels à partir de l'acquisition de signaux acoustiques dans une pluralité d'espaces physiques, par - Figure 3 shows a block diagram of the listening installation. The method according to the invention comprises a first processing (1) of producing a database of pulse signals from the acquisition of acoustic signals in a plurality of physical spaces, by
l'enregistrement des signaux produits par des enceintes recording signals produced by speakers
acoustiques en réponse à un signal multifréquence de référence. acoustically in response to a reference multifrequency signal.
Ensuite, pour chaque séquence audio à spatialiser, le procédé consiste à appliquer une succession de  Then, for each audio sequence to be spatialized, the method consists in applying a succession of
traitements : treatments:
- lorsque le signal à spatialiser est un signal stéréo, le procédé comporte une étape préliminaire (2) de construction d'un signal N.i à partir du signal stéréo  when the signal to be spatialized is a stereo signal, the method comprises a preliminary stage (2) of constructing a signal N.i from the stereo signal
- une étape (3) de transformation du signal de chacun des N.i canaux à partir de l'un des fichiers de réponse impulsionnel sélectionné dans la base de données susvisée  a step (3) of transforming the signal of each of the N.i channels from one of the impulse response files selected in the abovementioned database
- une étape (4) de recombinaison des signaux des N.i canaux ainsi transformés pour construire un signal stéréo spatialisé .  a step (4) of recombining the signals of the N.i channels thus transformed in order to construct a spatialized stereo signal.
Ce signal stéréo pourra ensuite être diffusé par un couple d'enceintes acoustiques standard, pour restituer une ambiance sonore spatialisée correspondant à l'espace qui a servi à la production des signaux de réponse impulsionnel ou à une combinaison de tels espaces. Etape initiale de construction de la base de  This stereo signal can then be broadcast by a pair of standard loudspeakers, to restore a spatialized sound environment corresponding to the space that has been used to produce the impulse response signals or a combination of such spaces. Initial stage of construction of the base of
réponses impulsionnelles. impulse responses.
Cette étape est répliquée une pluralité de fois. This step is replicated a plurality of times.
Elle est illustrée par la figure 2. It is illustrated in Figure 2.
Elle consiste, pour chaque série de réponses  It consists, for each series of responses
impulsionnelles, à disposer dans un espace physique tel qu'une salle de concert, un lieu ouvert ou fermé, un local donnée, une série d'enceintes acoustiques (5 à 11 ; 17) connues, associées à un amplificateur (14), de préférence de qualité reconnue, ainsi qu'un couple de microphone (12, 13) dont la position par rapport à la série d'enceintes (5 à 11 ; 17) est figée pour la série en cours d'acquisition. impulses, to be arranged in a physical space such as a concert hall, an open or closed place, a given room, a series of known loudspeakers (5 to 11; 17) associated with an amplifier (14), preference of recognized quality, as well as a microphone pair (12, 13) whose position relative to the speaker series (5 to 11; 17) is fixed for the series being acquired.
On applique ensuite successivement à chacune des enceintes (5 à 11) un signal multifréquence d'origine à l'aide de l'amplificateur (14). Ce signal d'origine est par exemple une séquence d'une durée comprise entre 10 et 90 secondes, avec une variation fréquentielle dans le spectre sonore. Ce signal est par exemple une variation linéaire entre 20Hz et 20 Khz, ou encore un signal quelconque couvrant l'ensemble du spectre de l'enceinte.  Each of the speakers (5 to 11) is then successively applied to an original multifrequency signal using the amplifier (14). This original signal is for example a sequence of a duration of between 10 and 90 seconds, with a frequency variation in the sound spectrum. This signal is for example a linear variation between 20 Hz and 20 kHz, or any signal covering the entire spectrum of the enclosure.
Le signal sonore produit par l'enceinte active est capté par le couple de microphone (12, 13) et produit un signal stéréo enregistré. A partir de ce signal on procède à un échantillonnage à 96 Khz de manière connue et à une  The sound signal produced by the active speaker is picked up by the microphone torque (12, 13) and produces a recorded stereo signal. From this signal, 96 kHz sampling is carried out in a known manner and at a
déconvolution par transformée de Fourier rapide entre le signal d'origine et le signal enregistré, pour construire une réponse impulsionnelle pour l'enceinte considérée dans deconvolution by fast Fourier transform between the original signal and the recorded signal, to construct an impulse response for the enclosure considered in
l'espace physique considéré. the physical space considered.
On reproduit cette étape pour chacune des enceintes (5 à 11) de la série, puis pour différents espaces physiques où on réimplante une série d'enceintes, identiques ou  This step is reproduced for each of the speakers (5 to 11) of the series, then for different physical spaces where a series of identical or identical speakers is re-inserted.
différentes, avec un amplificateur identique ou différent et des microphones identiques. different, with the same or different amplifier and identical microphones.
Cette première étape conduit à la construction d'une base de données de réponses impulsionnelles stéréo.  This first step leads to the construction of a database of stereo impulse responses.
Etape de préparation d'un signal spatialisé Preparation stage of a spatialized signal
Cette étape permet de construire un signal audio stéréo spatialisé à partir d'un signal multicanaux N.i This step makes it possible to construct a spatial stereo audio signal from a multichannel signal N.i
correspondant à un enregistrement numérique traditionnel. corresponding to a traditional digital recording.
Cette étape consiste à sélectionner dans la base de données constituée lors de l'étape initiale N+i réponses impulsionnelles . La sélection va consister à associer à chacun des N+1 signaux l'une des réponses impulsionnelles de ladite base de données, en veillant à ce que la position d'acquisition dans l'espace de la réponse impulsionnelle correspond à la position dans l'espace du canal auquel elle est associée. This step consists in selecting in the database formed during the initial step N + i impulse responses. The selection will consist in associating with each of the N + 1 signals one of the impulse responses of said database, ensuring that the acquisition position in the space of the impulse response corresponds to the position in the space of the channel with which it is associated.
Pour chaque couple «signal mono/réponse impulsionnel stéréo», on applique un traitement de convolution pour calculer une coupe de signaux spatialisé stéréo SsG et SsD. For each pair "mono signal / stereo impulse response", a convolution processing is applied to calculate a stereo spatialized signal cut S sG and S sD .
On produit ainsi N+i couples de j signaux  N + i pairs of signals are thus produced
spatialisés S3 sG et S3 sD, avec J compris entre 1 et N+i. spatialized S 3 sG and S 3 sD , with J between 1 and N + i.
Par exemple, si l'enregistrement de départ était de type 5.1, on va construire 6 couples de signaux spatialisés.  For example, if the starting record was of type 5.1, we will construct 6 pairs of spatialized signals.
Optionnellement , on procède à une égalisation des canaux pour améliorer la dynamique des j signaux.  Optionally, channel equalization is performed to improve the dynamics of the signals.
Construction du signal stéréo spatialisé Construction of the spatial stereo signal
L'étape finale consiste à recombiner les j signaux pour construire un couple de signaux droit et gauche The final step is to recombine the signals to build a right and left signal pair
spatialisé. spatialized.
Pour cela, on additionne les j signaux S3 sG For this, we add the signals S 3 sG
correspondant à l'espace situé à gauche, pour construire la voie gauche du signal stéréo spatialisé. On procède de même pour les j signaux S3 sD correspondant à l'espace situé à droite, pour construire la voie droite du signal stéréo spatialisé. corresponding to the space on the left, to build the left channel of the spatialized stereo signal. The same procedure is followed for the S 3 sD signals corresponding to the space on the right, to construct the right channel of the spatial stereo signal.
Optionnellement, on procède à une égalisation des canaux pour améliorer la dynamique des deux voies. Cas d'un signal de départ stéréo ; augmentation du nombre de canaux et création de canaux intermédiaires Optionally, the channels are equalized to improve the dynamics of the two channels. Case of a stereo start signal; increasing the number of channels and creating intermediate channels
Lorsque le signal à spatialiser n'est pas de type N.i mais simplement un signal stéréo, on procède à une étape intermédiaire consistant à construire un signal N.i par des traitements d'extraction par phase entre la piste gauche et droite, pour construire différents signaux nouveaux. When the signal to be spatialized is not of the Ni type but simply a stereo signal, a step is taken intermediate consisting of building a signal Ni by phase extraction treatments between the left and right track, to build different new signals.
Cette extraction par phase consiste à produire un signal correspondant à une voie centrale reconstruite, par un traitement consistant à additionner le signal de la voie gauche avec un signal de la voie droite déphasée, par exemple en opposition de phase.  This extraction by phase consists in producing a signal corresponding to a reconstructed central channel, by a processing consisting in adding the signal of the left channel with a signal of the straight line which is out of phase, for example in phase opposition.
Pour créer les autres voies « reconstruites » , on procède à des déphasages des pistes gauche et droite, avec des angles de déphasage différents, et on additionne les couples de signaux déphasés, avec des pondérations déterminées  To create the other "reconstructed" channels, the left and right tracks are phase-shifted, with different phase-shift angles, and the out-of-phase signal pairs are added, with determined weights.
empiriquement afin de restituer une ambiance sonore empirically to restore a soundscape
spacialisée . spacialized.
On applique de surcroit des filtres fréquentiels sur les signaux droit et gauche, lors de la créations de canaux « reconstruits », afin d'augmenter la dynamique du signal et conserver une qualité de haute fidélité du son. Restitution du signal  In addition, frequency filters are applied to right and left signals when creating "reconstructed" channels, in order to increase signal dynamics and maintain high fidelity sound quality. Signal restitution
La figure 3 représente une vue schématique de l'installation de restitution, à partir d'une paire d'enceintes réelles (17, 18). FIG. 3 represents a schematic view of the reproduction installation, from a pair of actual speakers (17, 18).
Ce couple d'enceintes (17, 18) reçoit un signal permettant de simulé des enceintes calculées (20 à 27 et 30 à 37) .  This pair of speakers (17, 18) receives a signal for simulating calculated speakers (20 to 27 and 30 to 37).
Le nombre effectif d'enceintes calculées (20 à 27) correspond au nombre d'enceintes physiques (5 à 11 ; 17) utilisés pour la production de la base de données de signaux impulsionnels, ou au nombre d'enceintes virtuelles reconstruites selon le procédé susvisé.  The effective number of calculated speakers (20 to 27) corresponds to the number of physical speakers (5 to 11; 17) used for the production of the pulse signal database, or to the number of virtual speakers reconstructed by the method referred to above.
On crée en outre des enceintes virtuelles (30 à 37) produisant une perception dans l'espace sonore d'une combinaison des enceintes réelles voisines, afin de combler les trous sonores. Virtual speakers (30 to 37) are also created which produce a perception in the sound space of a combination of real neighboring speakers, to fill the sound holes.
Ces enceintes virtuelles sont créées par une modification du signal alimentant les enceintes réelles voisines.  These virtual speakers are created by a modification of the signal supplying the neighboring real speakers.
On produit ainsi quinze fichiers sonores, 8 (7.1) correspondant au traitement à partir des signaux impulsionnels, et 7 calculés par une combinaisons de ces quinze fichiers.  Fifteen sound files are produced, 8 (7.1) corresponding to the processing from the pulse signals, and 7 calculated by a combination of these fifteen files.
On répartie les signaux en fonction de leur composante droite, gauche ou centrale pour produire un signal gauche (17) destiné à l'enceinte gauche, et un signal droit destiné à l'enceinte droite (18) :  The signals are distributed according to their right, left or central component to produce a left signal (17) for the left speaker, and a right signal for the right speaker (18):
- le signal « droite» correspond à l'addition des signaux « droite «calculés (21, 22, 23) et des signaux « droite » virtuels (30, 31, 32), ainsi que les signaux « centraux » calculé (20, 27) et virtuel (33) avec une pondération de l'amplitude de 50%  the "right" signal corresponds to the addition of the calculated "right" signals (21, 22, 23) and virtual "right" signals (30, 31, 32), as well as the "central" signals calculated (20, 27) and virtual (33) with an amplitude weighting of 50%
- le signal « gauche » correspond à l'addition des signaux calculés gauche (24, 25, 26) et des signaux virtuels gauche (34, 35, 36), ainsi que les signaux centraux calculé (20, 27) et virtuel (33) avec une pondération de l'amplitude de 50%.  the "left" signal corresponds to the addition of the left calculated signals (24, 25, 26) and the left virtual signals (34, 35, 36), as well as the calculated central (20, 27) and virtual (33 ) with an amplitude weighting of 50%.
Ce signal stéréo est ensuite appliqué à un équipement audio classique, raccordé à une paire d'enceintes (18, 19), qui reproduiront une ambiance sonore spatialisée correspondant à l'ambiance sonore de l'installation qui a servi à la construction de la base de signaux impulsionnels, ou à une ambiance sonore virtuelle correspondant à la combinaison de plusieurs ambiances originelles, le cas échéant enrichie avec des ambiances virtuelles.  This stereo signal is then applied to a conventional audio equipment, connected to a pair of speakers (18, 19), which will reproduce a spatialized sound environment corresponding to the sound environment of the installation that was used to build the base impulse signals, or a virtual sound environment corresponding to the combination of several original atmospheres, where appropriate enriched with virtual atmospheres.

Claims

Revendications  claims
1 — Procédé pour la production d'un fichier numérique audio stéréo spatialisé à partir d'un fichier audio multicanal originel, caractérisé en ce qu'il comporte : 1 - Process for producing a spatial stereo audio digital file from an original multichannel audio file, characterized in that it comprises:
• une étape de traitement, sur chacun des canaux, pour la suppression des trajets croisés (cross talk cancelation) A processing step, on each of the channels, for the deletion of cross paths cancelation
• une étape de fusion des canaux pour construite un signal stéréo • a channel merge step to build a stereo signal
· une étape de filtrage dynamique et d'équalisation spécifique pour l'augmentation de la dynamique du son.  · A step of dynamic filtering and specific equalization for the increase of the dynamics of the sound.
2 — Procédé pour la production d'un fichier numérique audio stéréo spatialisé selon la revendication principale caractérisé en ce que l'étape de suppression des trajets croisés consiste à ajouter au signal de chacun des canaux un signal correspondant au signal déphasé et pondéré des autres canaux. 3 — Procédé pour la production d'un fichier numérique audio stéréo spatialisé selon la revendication principale caractérisé en ce que le signal originel est un signal multicanal 5.n natif. 4 — Procédé pour la production d'un fichier numérique audio stéréo spatialisé selon la revendication principale caractérisé en ce que le signal originel est un signal multicanal 5.n natif calculé à partir d'un signal stéréo. 2 - Process for the production of a spatial stereo audio digital file according to the main claim, characterized in that the step of eliminating the crossed paths consists in adding to the signal of each of the channels a signal corresponding to the phase-shifted and weighted signal of the other channels. . 3 - Process for the production of a digital spatial stereo audio file according to the main claim characterized in that the original signal is a native 5.n multichannel signal. 4 - Process for the production of a spatial stereo audio digital file according to the main claim characterized in that the original signal is a native multichannel signal 5.n calculated from a stereo signal.
EP13710449.3A 2012-02-13 2013-02-11 Transaural synthesis method for sound spatialization Active EP2815589B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
FR1251328A FR2986932B1 (en) 2012-02-13 2012-02-13 PROCESS FOR TRANSAURAL SYNTHESIS FOR SOUND SPATIALIZATION
PCT/FR2013/050278 WO2013121136A1 (en) 2012-02-13 2013-02-11 Transaural synthesis method for sound spatialization

Publications (2)

Publication Number Publication Date
EP2815589A1 true EP2815589A1 (en) 2014-12-24
EP2815589B1 EP2815589B1 (en) 2017-04-05

Family

ID=47901163

Family Applications (1)

Application Number Title Priority Date Filing Date
EP13710449.3A Active EP2815589B1 (en) 2012-02-13 2013-02-11 Transaural synthesis method for sound spatialization

Country Status (10)

Country Link
EP (1) EP2815589B1 (en)
JP (1) JP6421385B2 (en)
KR (1) KR20140128412A (en)
CN (1) CN104160722B (en)
BR (1) BR112014019926A2 (en)
FR (1) FR2986932B1 (en)
HK (1) HK1204188A1 (en)
IN (1) IN2014DN06776A (en)
RU (1) RU2639955C2 (en)
WO (1) WO2013121136A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018185733A1 (en) 2017-04-07 2018-10-11 A3D Technologies Llc Sound spatialization method

Family Cites Families (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020133327A1 (en) * 1998-03-31 2002-09-19 Mcgrath David Stanley Acoustic response simulation system
US20030035553A1 (en) * 2001-08-10 2003-02-20 Frank Baumgarte Backwards-compatible perceptual coding of spatial cues
JP4062959B2 (en) * 2002-04-26 2008-03-19 ヤマハ株式会社 Reverberation imparting device, reverberation imparting method, impulse response generating device, impulse response generating method, reverberation imparting program, impulse response generating program, and recording medium
US6937737B2 (en) * 2003-10-27 2005-08-30 Britannia Investment Corporation Multi-channel audio surround sound from front located loudspeakers
KR20050060789A (en) * 2003-12-17 2005-06-22 삼성전자주식회사 Apparatus and method for controlling virtual sound
JP2005252332A (en) * 2004-03-01 2005-09-15 Clarion Co Ltd Sound field reproducing apparatus and control method thereof
US8175286B2 (en) * 2005-05-26 2012-05-08 Bang & Olufsen A/S Recording, synthesis and reproduction of sound fields in an enclosure
JP2006339694A (en) * 2005-05-31 2006-12-14 D & M Holdings Inc Audio signal output device
US7970626B2 (en) * 2005-07-08 2011-06-28 Oltine Acquistitions NY LLC Facilitating payments to health care providers
KR100619082B1 (en) * 2005-07-20 2006-09-05 삼성전자주식회사 Method and apparatus for reproducing wide mono sound
TWI396188B (en) * 2005-08-02 2013-05-11 Dolby Lab Licensing Corp Controlling spatial audio coding parameters as a function of auditory events
WO2007110103A1 (en) * 2006-03-24 2007-10-04 Dolby Sweden Ab Generation of spatial downmixes from parametric representations of multi channel signals
MX2008011994A (en) * 2006-03-24 2008-11-27 Dolby Sweden Ab Generation of spatial downmixes from parametric representations of multi channel signals.
JP2008301427A (en) * 2007-06-04 2008-12-11 Onkyo Corp Multichannel voice reproduction equipment
RU2439717C1 (en) * 2008-01-01 2012-01-10 ЭлДжи ЭЛЕКТРОНИКС ИНК. Method and device for sound signal processing
EP2248352B1 (en) * 2008-02-14 2013-01-23 Dolby Laboratories Licensing Corporation Stereophonic widening
UA101542C2 (en) * 2008-12-15 2013-04-10 Долби Лабораторис Лайсензин Корпорейшн Surround sound virtualizer and method with dynamic range compression
KR101764175B1 (en) * 2010-05-04 2017-08-14 삼성전자주식회사 Method and apparatus for reproducing stereophonic sound

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See references of WO2013121136A1 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018185733A1 (en) 2017-04-07 2018-10-11 A3D Technologies Llc Sound spatialization method

Also Published As

Publication number Publication date
JP2015510348A (en) 2015-04-02
FR2986932B1 (en) 2014-03-07
EP2815589B1 (en) 2017-04-05
FR2986932A1 (en) 2013-08-16
RU2639955C2 (en) 2017-12-25
BR112014019926A2 (en) 2017-07-04
CN104160722A (en) 2014-11-19
HK1204188A1 (en) 2015-11-06
WO2013121136A1 (en) 2013-08-22
KR20140128412A (en) 2014-11-05
JP6421385B2 (en) 2018-11-14
CN104160722B (en) 2018-01-12
RU2014133066A (en) 2016-04-10
IN2014DN06776A (en) 2015-05-22

Similar Documents

Publication Publication Date Title
KR100626233B1 (en) Equalisation of the output in a stereo widening network
WO2007110520A1 (en) Method for binaural synthesis taking into account a theater effect
Rafaely et al. Spatial audio signal processing for binaural reproduction of recorded acoustic scenes–review and challenges
KR20080060640A (en) Method and apparatus for reproducing a virtual sound of two channels based on individual auditory characteristic
KR20110053600A (en) Apparatus for generating multi-channel sound signal
US20190394596A1 (en) Transaural synthesis method for sound spatialization
JP6361000B2 (en) Method for processing audio signals for improved restoration
EP2009891B1 (en) Transmission of an audio signal in an immersive audio conference system
US20200059750A1 (en) Sound spatialization method
EP2815589B1 (en) Transaural synthesis method for sound spatialization
US20150036827A1 (en) Transaural Synthesis Method for Sound Spatialization
KR100275779B1 (en) A headphone reproduction apparaturs and method of 5 channel audio data
US9609454B2 (en) Method for playing back the sound of a digital audio signal
Nishimura et al. B-format for binaural listening of higher order Ambisonics
CA3192986A1 (en) Sound reproduction with multiple order hrtf between left and right ears
Gamper et al. Instant BRIR acquisition for auditory events in audio augmented reality using finger snaps
Tsakostas Binaural Simulation applied to standard stereo audio signals aiming to the enhancement of the listening experience
KR20050069859A (en) 3d audio signal processing(acquisition and reproduction) system using rigid sphere and its method

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20140811

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAX Request for extension of the european patent (deleted)
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 1204188

Country of ref document: HK

17Q First examination report despatched

Effective date: 20151125

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAJ Information related to disapproval of communication of intention to grant by the applicant or resumption of examination proceedings by the epo deleted

Free format text: ORIGINAL CODE: EPIDOSDIGR1

INTG Intention to grant announced

Effective date: 20160823

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTC Intention to grant announced (deleted)
INTG Intention to grant announced

Effective date: 20161018

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 882820

Country of ref document: AT

Kind code of ref document: T

Effective date: 20170415

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: FRENCH

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602013019417

Country of ref document: DE

REG Reference to a national code

Ref country code: NL

Ref legal event code: FP

REG Reference to a national code

Ref country code: CH

Ref legal event code: NV

Representative=s name: ABREMA AGENCE BREVET ET MARQUES, GANGUILLET, CH

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 882820

Country of ref document: AT

Kind code of ref document: T

Effective date: 20170405

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170706

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170805

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602013019417

Country of ref document: DE

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 6

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

26N No opposition filed

Effective date: 20180108

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 602013019417

Country of ref document: DE

Representative=s name: KOENIG SZYNKA TILMANN VON RENESSE PATENTANWAEL, DE

Ref country code: DE

Ref legal event code: R081

Ref document number: 602013019417

Country of ref document: DE

Owner name: A3D TECHNOLOGIES LLC, LOS ANGELES, US

Free format text: FORMER OWNERS: HAURAIS, JEAN-LUC, PARIS, FR; ROSSET, FRANCK, BRUXELLES, BE

REG Reference to a national code

Ref country code: HK

Ref legal event code: GR

Ref document number: 1204188

Country of ref document: HK

REG Reference to a national code

Ref country code: FR

Ref legal event code: TP

Owner name: A3D TECHNOLOGIES LLC, US

Effective date: 20180412

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

REG Reference to a national code

Ref country code: GB

Ref legal event code: 732E

Free format text: REGISTERED BETWEEN 20180709 AND 20180711

REG Reference to a national code

Ref country code: NL

Ref legal event code: PD

Owner name: A3D TECHNOLOGIES LLC; US

Free format text: DETAILS ASSIGNMENT: CHANGE OF OWNER(S), ASSIGNMENT; FORMER OWNER NAME: ROSSET, FRANCK

Effective date: 20180703

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20180228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180211

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180211

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20130211

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170405

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

REG Reference to a national code

Ref country code: CH

Ref legal event code: PFUS

Owner name: A3D TECHNOLOGIES LLC, US

Free format text: FORMER OWNER: A3D TECHNOLOGIES LLC, US

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20240123

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240123

Year of fee payment: 12

Ref country code: CH

Payment date: 20240301

Year of fee payment: 12

Ref country code: GB

Payment date: 20240123

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20240123

Year of fee payment: 12

Ref country code: FR

Payment date: 20240123

Year of fee payment: 12