EP2898707B1 - Optimized calibration of a multi-loudspeaker sound restitution system - Google Patents
Optimized calibration of a multi-loudspeaker sound restitution system Download PDFInfo
- Publication number
- EP2898707B1 EP2898707B1 EP13774728.3A EP13774728A EP2898707B1 EP 2898707 B1 EP2898707 B1 EP 2898707B1 EP 13774728 A EP13774728 A EP 13774728A EP 2898707 B1 EP2898707 B1 EP 2898707B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- reflections
- impulse responses
- signal
- amplitude
- direct wave
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000004044 response Effects 0.000 claims description 110
- 230000005236 sound signal Effects 0.000 claims description 40
- 239000011159 matrix material Substances 0.000 claims description 38
- 238000000034 method Methods 0.000 claims description 34
- 238000001914 filtration Methods 0.000 claims description 25
- 239000004303 calcium sorbate Substances 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 7
- 239000004283 Sodium sorbate Substances 0.000 claims description 5
- 230000001629 suppression Effects 0.000 claims description 5
- 239000004302 potassium sorbate Substances 0.000 claims description 4
- 238000012545 processing Methods 0.000 description 24
- 230000006870 function Effects 0.000 description 20
- 238000004458 analytical method Methods 0.000 description 19
- 230000015654 memory Effects 0.000 description 13
- 230000008447 perception Effects 0.000 description 13
- 238000012937 correction Methods 0.000 description 10
- 230000000694 effects Effects 0.000 description 10
- 238000009877 rendering Methods 0.000 description 6
- 238000001228 spectrum Methods 0.000 description 6
- 230000002123 temporal effect Effects 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 5
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 230000009466 transformation Effects 0.000 description 3
- 238000000354 decomposition reaction Methods 0.000 description 2
- 230000009977 dual effect Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000012732 spatial analysis Methods 0.000 description 2
- 238000011282 treatment Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000007429 general method Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 229940050561 matrix product Drugs 0.000 description 1
- 230000013707 sensory perception of sound Effects 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
- 230000003936 working memory Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
- H04S7/301—Automatic calibration of stereophonic sound system, e.g. with test microphone
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
- H04S2420/11—Application of ambisonics in stereophonic audio systems
Definitions
- the present invention relates to a method and a device for calibrating a sound reproduction system comprising a plurality of loudspeakers or sound reproduction elements.
- the calibration makes it possible to optimize the quality of listening of the restitution system that constitutes all the restitution elements, including the loudspeaker device and the listening room.
- the restitution systems particularly concerned are the sound reproduction systems of the multichannel type (5.1, 7.1, 10.2, 22.2, etc.) or of the ambisonic type (Ambisonics in English or Higher Order Ambisonics (HOA)).
- current devices for calibrating the acoustics of the listening environment are based on a general method of the "multichannel equalization" type in which the impulse responses of each loudspeaker of the restitution are measured using one or more microphones at one or more points of the listening location and a frequency equalization filtering is performed on each speaker, independently, by reversing all or part of the measured impulse response for the speaker concerned.
- the inversion aims to correct the response of the loudspeaker so that it approaches as well as possible a "target" curve generally defined in the frequency domain to improve the rendering of the timbre of the sound sources.
- This type of calibration or correction focuses on the correction of the frequency aspect of the response of the system of reproduction of the listening location without exploiting the temporal information such as the phenomena of reflections and in particular the first reflections of the sound signals.
- the analysis of the impulse responses carried out in the existing calibration methods is of monophonic type, that is to say that it does not take into account either the spatial information of the reflections such as the direction of incidence. .
- the techniques of the state of the art are based on the application of correction filters on each of the channels of the multi-channel signal, that is to say that each speaker of the system of restitution is corrected individually without taking into account the whole network of speakers.
- the present invention improves the situation.
- the effect of the first reflections of the sound waves diffused by the reproduction system on the auditory perception of direct waves is evaluated and taken into account to adapt the processing.
- applied to the channels of the multi-channel signal according to the specific perceptual effect associated with each reflection.
- the filtering of the channels of the multi-channel signal thus takes into account exclusively the reflections which have an impact on the auditory perception of direct waves.
- the constraints of the correction are lightened by the fact that they take into account perceptual impulse responses instead of raw impulse responses.
- some of the non-perceptible reflections which are eliminated from the impulse responses obtained correspond to components of the impulse response which are precisely at the origin of processing instabilities (in particular components with non-minimal phase). With perceptual impulse responses, the risks of instabilities and artefacts which can be generated during treatments taking into account all of the reflections are thus reduced.
- the error signal thus determined makes it possible to take into account in the calculation of the filtering matrix, only the reflections which have an impact on the auditory perception of the direct wave. Indeed, only the reflections which are not perceptible are removed for the determination of the error signal.
- the perceptibility threshold can be obtained from characteristics determined by the step of analyzing the multidirectional impulse responses of the loudspeakers.
- the perceptibility threshold is determined as a function of the direction of incidence of the direct wave and / or of its amplitude, and of the directions of incidence of the first reflections and / or of their arrival times relative to the direct wave.
- the effect of a reflection on the perception of the direct wave generally depends on five parameters in total; on the one hand it depends on two characteristics of the direct wave: its amplitude and its direction; on the other hand it depends on three characteristics of the reflection: its amplitude, its instant of arrival and its incidence.
- the perceptual effect of the reflection by fixing the missing characteristic at an arbitrary value, by taking for example the value corresponding to the case more unfavorable in order to increase the perceptibility.
- the direction value can be fixed and the perceptibility threshold determined only according to the value of the time of arrival.
- the target response signal corresponds to the response of the direct wave alone without any reflection.
- the target response signal corresponds to the response of a direct wave associated with reflections representative of a predetermined listening location.
- the reference response can then be voluntarily chosen as a desired listening location in which the sound is at a desired quality.
- the target response signal corresponds to the response of a direct wave associated with reflections representative of a different restitution set.
- the reference response is here chosen as a function of a chosen reference rendering system, in which the number and the position of the loudspeakers can be different from the restitution system being corrected.
- This device has the same advantages as the method described above, which it implements.
- the invention also relates to an audio decoder comprising a calibration device as described.
- It relates to a computer program comprising code instructions for implementing the steps of the calibration method as described, when these instructions are executed by a processor.
- the invention relates to a storage medium, readable by a processor, integrated or not into the calibration device, possibly removable, storing a computer program implementing a calibration method as described above.
- the figure 1 therefore illustrates an example of a sound reproduction system in which the calibration method according to an embodiment of the invention is implemented.
- This system includes a processing device 100 comprising a calibration device E according to an embodiment of the invention controlling a rendering unit 180 which comprises a plurality of rendering elements (speakers, loudspeakers, etc.) represented here by loudspeakers HP 1 , HP 2 , HP 3 , HP i and HP N.
- a rendering unit 180 which comprises a plurality of rendering elements (speakers, loudspeakers, etc.) represented here by loudspeakers HP 1 , HP 2 , HP 3 , HP i and HP N.
- These loudspeakers are arranged in a listening location in which a microphone or set of MA microphones is also provided.
- a processing device 100 which can be a decoder such as a living room decoder of the "set top box" type for reading or broadcasting audio or video content, a processing server capable of processing audio and video content and to retransmit them to the restitution unit, a conference bridge capable of processing the audio signals from different conference locations or any multi-channel audio signal processing device.
- a decoder such as a living room decoder of the "set top box” type for reading or broadcasting audio or video content
- a processing server capable of processing audio and video content and to retransmit them to the restitution unit
- a conference bridge capable of processing the audio signals from different conference locations or any multi-channel audio signal processing device.
- the processing device 100 comprises a calibration device E according to an embodiment of the invention and a filtering matrix 170 composed of a plurality of processing filters which are determined by the calibration device according to a calibration method such that '' illustrated later with reference to figure 2 .
- This filtering matrix receives as input a multi-channel signal Si and transmits as output the signals SC 1 , SC 2 , SC i , SC N capable of being restored by the restitution assembly 180.
- the calibration device E comprises a reception and transmission module 110 capable of transmitting on the one hand reference audio signals (Sref) to the various speakers of the reproduction unit 180 and to receive by the microphone or the 'set of microphones MA, the multidirectional impulse responses (RIs) of these different speakers corresponding to the broadcasting of these reference signals.
- Sref one hand reference audio signals
- RIs multidirectional impulse responses
- a multidirectional impulse response contains temporal and spatial information relating to all of the sound waves induced by the loudspeaker considered in the reproduction room.
- the reference signals are for example signals whose frequency increases logarithmically over time, these signals being called in English “chirps” or “logarithmic sweeps”.
- the microphone capable of measuring the multidirectional impulse responses of the loudspeakers is a HOA type microphone placed at a point of the listening place, for example in the center of the loudspeakers of the reproduction unit.
- This microphone will receive, for each speaker reproducing a reference audio signal, the sound reproduced in several directions.
- the HOA microphone is made up of a plurality of microphones.
- the spatial information of the different sounds picked up can be extracted.
- this type of microphone one can refer to the document entitled " Study and realization of advanced spatial encoding tools for the sound spatialization technique Higher Order Ambisonics: 3D microphone and distance control "by S. Moreau cited in Univ. Of Maine, PhD thesis, 2006 .
- the HOA microphone then retrieves the multidirectional impulse responses from each of the speakers to transmit them to the calibration device or to store them in memory in a local or remote memory space.
- the analysis module 120 of the device E performs a joint analysis of the impulse responses obtained, which makes it possible to obtain these characteristics and in particular the characteristics of the first reflections of the restored signals.
- the multidirectional impulse responses are obtained in a spatio-temporal representation where the spatial information is described on the basis of the spherical harmonics and makes it possible to identify the directions of incidence different sound components.
- the analysis of impulse responses is done on a predetermined time scale, including the moments of the first reflections.
- this time window is between 50 and 100 ms in length, which corresponds to the time scale of the instants of arrival of the first reflections.
- the embodiment thus described is adapted to the field of representation of spherical harmonics but it is entirely possible to carry out these same steps in a field of WFS representation (for "Wave Field Synthesis" in English) or in the plane wave domain.
- the means for capturing the signals reproduced by the loudspeakers will have to be adapted to these areas of representation in order to obtain multidirectional impulse responses, without departing from the scope of the invention.
- the calibration device E also includes a module 130 for comparing and identifying non-perceptible reflections.
- This module implements a step of comparing the amplitudes of the reflections, obtained by the analysis module 120, with a predetermined perceptibility threshold Se.
- This perceptibility threshold is determined by the module 140 from a predefined table of values stored in a memory space.
- a step of identifying these "non-perceptible" reflections is then implemented by the module 130. These identified reflections make it possible to implement by the module 150 a step of determining perceptual impulse responses which are deduced from the impulse responses obtained by module 110 by suppressing reflections judged as not perceptible.
- the figure 2 illustrates in the form of a flowchart, the main steps implemented in an embodiment of the calibration method according to the invention.
- step E201 the multidirectional impulse responses from the various loudspeakers of the reproduction unit as described with reference to the figure 1 , are obtained. They are obtained by the calibration device, either by simple reading in memory if these were saved beforehand, either by receiving the microphone or a set of microphones having carried out the measurement.
- These multidirectional impulse responses are the responses of each loudspeaker following the reproduction of a reference signal as described with reference to the figure 1 .
- a step E202 of analysis of the multidirectional impulse responses thus obtained is then implemented.
- This analysis is carried out in a space-time representation domain.
- Spatial information can for example be described in the field of representation of spherical harmonics.
- each point has as spherical coordinates, a distance r from the origin 0, an angle ⁇ of azimuth or orientation in the horizontal plane and an angle ⁇ of elevation or orientation in the vertical plane.
- an acoustic wave is perfectly described if we define at any point at each time t, the acoustic pressure noted p (r, ⁇ , ⁇ , t) whose temporal Fourier transform is noted P (r, ⁇ , ⁇ , f) where f denotes the time frequency.
- the spatial components are ambisonic components B min ⁇ which correspond to the decomposition of the sound pressure wave p on the basis of spherical harmonics.
- B min ⁇ S t .
- the P min (sin ⁇ ) are the associated Legendre functions.
- FIG. 3b An illustration of the spherical harmonic functions is represented in figure 3b .
- the omnidirectional component Y 00 1 (designated as the “W component” in ambisonic terminology) corresponding to the order 0
- the bidirectional components Y 10 1 , Y 11 1 , Y 11 - 1 (designated respectively as the “Z, X and Y components” in ambisonic terminology) corresponding to order 1, and the components of higher orders.
- Decomposition on the basis of spherical harmonics can be considered as the dual transform between spatial coordinates and spatial frequencies.
- Components B min ⁇ therefore define a spatial spectrum.
- a multidirectional impulse response is obtained which is made up of K impulse responses corresponding to the K components of the chosen spatial representation.
- K impulse responses
- the multidirectional impulse response which is associated with it thus consists of K elementary responses H jl (t) where the index l locates the index of the spatial component and t corresponds to the time sample.
- the reproduction system comprises a total of N loudspeakers
- the set of multidirectional impulse responses measured for the N loudspeakers and the K spatial components defines a matrix H of size KxN, in which the jth column corresponds to the impulse response multidirectional associated with the jth speaker.
- the K spatial components contained in the vector h j (t) represent the spatial spectrum of the sounds picked up by the microphone.
- This inverse transformation is carried out by reconstructing the pressure wave p (r, ⁇ , ⁇ , t) by linear combination of spherical harmonics, each harmonic being weighted by the amplitude of the component associated with it.
- This spatial decoding step is for example described in the document entitled “ Ambisonics encoding of other audio formats for multiple listening conditions "by the authors Jérians Daniel, Jean-Bernard Rault and Jean-Dominique Polack in AES 105th Convention, September 1998 .
- this transformation of the spatial frequencies (ambisonic components) towards the spatial coordinates is carried out by multiplying, for each loudspeaker and each temporal sample t, the vector h j (t) by a decoding matrix D.
- each column consists of the values of the K spherical harmonics for a given loudspeaker.
- the precision of estimation of these characteristics therefore depends on the number P of virtual loudspeakers used for this analysis.
- the characteristics of the direct wave are determined on the one hand, such as its amplitude A D (j), its instant of arrival on the microphone T D (j) or its direction of incidence.
- the characteristics of the reflections such as their amplitudes A Ri (j), their instants of arrival on the microphone T Ri (j) or their directions of incidences C Ri (j).
- the first reflections of a reproduced audio signal depend on the listening location in which the reproduction unit is placed. Generally, these first reflections appear in a time located in a range from 50 to 100 ms after the direct wave.
- the analysis time window of step E202 will, in an adapted embodiment, be between 50 and 100 ms.
- Step E203 compares the amplitudes obtained by the analysis step with a perceptibility threshold Se of the reflections which has been defined beforehand and stored in memory.
- Step E204 makes it possible to find the predefined threshold value as a function of characteristics of each reflection and of the associated direct wave, obtained in the analysis step E202.
- the value of the characteristic of the time of arrival of the reflection is fixed, for example the most critical value (the one which gives maximum perceptibility) and the value of the perceptibility threshold is determined only relative to the value of leadership.
- the direction value can be fixed, for example the most critical value (the one which gives maximum perceptibility), and determine the perceptibility threshold according to the value of the arrival time.
- the threshold value can be determined, with better precision, as a function of these two characteristics.
- an array of perceptibility threshold values is stored in memory.
- An example of such a table is illustrated with reference to the figure 4 .
- the threshold is defined as the relative level of the reflection, that is to say it represents the difference between the amplitude values (expressed in dB) of the reflection and of the direct wave considered.
- This table of values is an example of threshold values defined from psycho-acoustic experiences carried out by considering different types of sound signal (speech, clicks, music, etc.), different angles of incidence and different times of arrival of reflections and of the direct wave.
- a threshold of perceptibility of these reflections is defined according to these parameters.
- the figure 5 shows different perceptibility threshold curves expressed in dB (which always corresponds to the relative threshold corresponding to the difference between the level of the reflection and that of the direct wave). These different curves correspond to different positions of the direct wave (azimuth of 0 ° for D1, 60 ° for D2, 90 ° for D3 and 150 ° for D4) and represent the thresholds of perceptibility as a function of the direction of reflection, this for a fixed arrival time (corresponding in this case to 15 ms).
- step E204 the threshold value corresponding to the characteristics obtained in the analysis step is recovered.
- This threshold value is compared to the amplitude value of each reflection in step E203.
- the value of the amplitude of the reflection is referenced to that of the associated direct wave and expressed in dB: 20log ( AN Ri ( j )).
- Step E203 thus makes it possible to identify all the reflections which have no impact on the perception of the direct wave. Step E203 therefore identifies all the reflections for which the amplitude is below the perceptibility threshold.
- the figure 6 represents an example of impulse response, for a given direction, of one of the loudspeakers of the reproduction unit in comparison with the curve in broken line representing the perceptibility threshold (RMT for "Reflection Masked Threshold") obtained by the table described above with reference to the figure 4 . Reflections whose level is lower than the threshold curve are thus identified. Note that in the illustrated case, the first reflections occurring in the first 15 ms are not perceptible.
- RTT perceptibility threshold
- this operation is carried out for example by a thresholding operation.
- the value of the perceptibility threshold Se is subtracted from the impulse response signal which was obtained in step E201.
- the processing can also be applied in the dual domain of space coordinates. In the following, we will describe the operation performed in the case of the spatial spectrum.
- the thresholding operation consists in comparing for each identified reflection its amplitude with the perceptibility threshold Se associated with its characteristics.
- the perceptual impulse responses only retain the reflections having a significant impact on the perception of the direct wave.
- step E206 This filtering matrix is then used to process the multi-channel audio signal before its sound reproduction by the system reproduction unit.
- a possible embodiment includes a step of determining an error signal defined by the difference between a predetermined target response signal from the set of restitution and a response signal reconstructed from the perceptual impulse responses and a multichannel inversion step by minimization of the error signal thus determined.
- the error signal thus obtained therefore only takes into account the perceptible reflections since it is calculated from a reconstructed signal based on the perceptual impulse responses.
- the inversion can be performed by a gradient descent algorithm or its variants.
- An example of a possible inversion algorithm is that of the ISTA type (for “Iterative Shrinkage-Thresholding algorithm) as described in the document entitled” A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems "by the authors Amir Beck & Marc Teboulle, published in SIAM J. IMAGING SCIENCES, Vol. 2, No. 1, pp. 183-202 in 2009 .
- the problem which arises to calculate the filters of the processing matrix is as follows.
- N loudspeakers which constitute the real reproduction system.
- the space of spatial representation is of dimension K. Spatial information is therefore described by K coefficients.
- the objective is to reproduce with the N speaker system, a set of V signals defining the input multichannel audio signal.
- V signals are dedicated to an ideal reproduction system consisting of V loudspeakers.
- This ideal system defines the V target signals which one wishes to reproduce and which therefore correspond to the responses of a fictitious system of V virtual loudspeakers.
- the resolution of this operation can be carried out in two stages.
- the correction filters are calculated by correcting only the room effect of the restitution place, that is to say that we take into account the actual loudspeaker device, ie N loud- speakers.
- the arrangement of the loudspeakers is compensated for in order to adapt the V signals to a restitution according to a non-ideal configuration of N loudspeakers.
- the V signals are distributed by matrixing on the N channels associated with the real reproduction system in order to emulate a system of V virtual loudspeakers.
- the elements of the matrix H include the perceptual impulse responses as obtained in step E205.
- the target responses may vary depending on the expected sound reproduction result.
- this target response corresponds to the impulse response given by the direct wave alone without any reflection. This is equivalent to suppressing all the room effect in the expected signal.
- the target response signal corresponds to the response of a direct wave associated with reflections representative of a predetermined listening location.
- a characteristic listening location with good listening quality may be desired (for example the listening location in the Pleyel TM room).
- the processing filters will be calculated to obtain a sound reproduction close to this listening quality.
- the target response signal corresponds to the response of a direct wave associated with reflections representative of a restitution set different from that used to restore the resulting signal.
- a desired restitution system for example comprising more speakers, is taken as a reference for obtaining a restitution close to that which would have been obtained with such a system.
- the implementation of the described method makes it possible to obtain a better quality of listening during the restitution of a multi-channel audio signal thanks to the taking into account only of the perceptible reflections of the signals by the restitution unit in the listening place.
- the figure 7 shows an example of a hardware embodiment of a calibration device according to the invention. This can be an integral part of an audio / video decoder, a processing server, a conference bridge or any other audio or video playback or broadcasting equipment.
- This type of device comprises a ⁇ P processor cooperating with a memory block MEM comprising a storage and / or working memory.
- the memory block can advantageously include a computer program comprising code instructions for implementing the steps of the calibration method within the meaning of the invention, when these instructions are executed by the processor, and in particular the steps for obtaining responses.
- multidirectional impulse signals from the loudspeakers of the reproduction unit for reproducing a predetermined audio signal, for analyzing the multidirectional impulse responses obtained, in a space-time representation domain, over at least one time window including the instants d arrival of the first reflections of the predetermined audio signal reproduced to determine a set of characteristics of the first reflections, of comparing the amplitude of each of the reflections at a predetermined perceptibility threshold and of identifying the non-perceptible reflections for which the amplitude is less than the predetermined threshold, modification of the impulse responses obtained to obtain perceptual impulse responses, by deletion of the reflections identified as non-perceptible and determination of a filtering matrix from the perceptual impulse responses for an application of this filtering matrix to the multi-channel audio signal before sound reproduction.
- the description of the figure 2 takes the steps of an algorithm of such a computer program.
- the computer program can also be stored on a memory medium readable by a reader of the device or downloadable in the memory space of the latter.
- the memory MEM stores a table of perceptibility threshold values as a function of characteristics of the sound components consisting of the direct wave and of the reflections used in the method according to an embodiment of the invention and in general, all the necessary data. to the implementation of the process.
- Such a device comprises an input module I capable of receiving impulse responses from a restitution assembly and an output module S capable of transmitting to a processing module, the filters calculated from a filtering matrix.
- the device thus described may also include the processing functions by implementing the processing matrix on reception at I of a multi-channel signal Si to output processed signals SCi capable of be returned by the return package.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Signal Processing (AREA)
- Acoustics & Sound (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Stereophonic System (AREA)
Description
La présente invention se rapporte à un procédé et un dispositif de calibration d'un système de restitution sonore comportant une pluralité de haut-parleurs ou d'éléments de restitution sonore. La calibration permet d'optimiser la qualité d'écoute du système de restitution que constitue l'ensemble des éléments de restitution, comprenant le dispositif des haut-parleurs et la salle d'écoute.The present invention relates to a method and a device for calibrating a sound reproduction system comprising a plurality of loudspeakers or sound reproduction elements. The calibration makes it possible to optimize the quality of listening of the restitution system that constitutes all the restitution elements, including the loudspeaker device and the listening room.
Les systèmes de restitution particulièrement concernés sont les systèmes de restitution sonore de type multicanal (5.1, 7.1, 10.2, 22.2, etc...) ou encore de type ambisonique (Ambisonics en anglais ou Higher Order Ambisonics (HOA)).The restitution systems particularly concerned are the sound reproduction systems of the multichannel type (5.1, 7.1, 10.2, 22.2, etc.) or of the ambisonic type (Ambisonics in English or Higher Order Ambisonics (HOA)).
Pour permettre une restitution de bonne qualité des signaux multicanaux, les dispositifs actuels de calibration de l'acoustique du lieu d'écoute sont basés sur une méthode générale de type "égalisation multicanale" dans laquelle les réponses impulsionnelles de chaque haut-parleur du système de restitution sont mesurées à l'aide d'un ou plusieurs microphones en un ou plusieurs points du lieu d'écoute et un filtrage d'égalisation fréquentielle est effectué sur chaque haut-parleur, indépendamment, en inversant tout ou partie de la réponse impulsionnelle mesurée pour le haut-parleur concerné.To allow good quality reproduction of multichannel signals, current devices for calibrating the acoustics of the listening environment are based on a general method of the "multichannel equalization" type in which the impulse responses of each loudspeaker of the restitution are measured using one or more microphones at one or more points of the listening location and a frequency equalization filtering is performed on each speaker, independently, by reversing all or part of the measured impulse response for the speaker concerned.
L'inversion vise à corriger la réponse du haut-parleur de façon à ce qu'elle se rapproche au mieux d'une courbe "cible" généralement définie dans le domaine fréquentiel pour améliorer le rendu du timbre des sources sonores.The inversion aims to correct the response of the loudspeaker so that it approaches as well as possible a "target" curve generally defined in the frequency domain to improve the rendering of the timbre of the sound sources.
Une telle méthode est par exemple décrite dans le document intitulé "
Ce type de calibration ou correction se focalise sur la correction de l'aspect fréquentiel de la réponse du système de restitution du lieu d'écoute sans exploiter les informations temporelles comme les phénomènes de réflexions et notamment les premières réflexions des signaux sonores.This type of calibration or correction focuses on the correction of the frequency aspect of the response of the system of reproduction of the listening location without exploiting the temporal information such as the phenomena of reflections and in particular the first reflections of the sound signals.
Or les premières réflexions de signaux sonores ont un impact non négligeable sur la perception auditive du signal sonore restitué.However, the first reflections of sound signals have a significant impact on the auditory perception of the restored sound signal.
De plus, l'analyse des réponses impulsionnelles effectuée dans les méthodes de calibration existantes est de type monophonique, c'est-à-dire qu'elle ne prend pas non plus en compte l'information spatiale des réflexions comme la direction d'incidence.In addition, the analysis of the impulse responses carried out in the existing calibration methods is of monophonic type, that is to say that it does not take into account either the spatial information of the reflections such as the direction of incidence. .
L'absence de données temporelles et spatiales des réflexions, ne permet pas de prendre en compte le rôle des ces réflexions sur la perception de l'onde directe du signal sonore par un auditeur, et ainsi d'ajuster la correction en fonction de leur effet spécifique. La qualité du signal sonore restitué et perçue par l'auditeur n'est alors pas optimale.The absence of temporal and spatial data of the reflections, does not allow to take into account the role of these reflections on the perception of the direct wave of the sound signal by a listener, and thus to adjust the correction according to their effect specific. The quality of the sound signal reproduced and perceived by the listener is therefore not optimal.
Les techniques de l'état de l'art sont basées sur l'application de filtres de correction sur chacun des canaux du signal multi canal, c'est-à-dire que chaque haut-parleur du système de restitution est corrigé individuellement sans tenir compte de l'ensemble du réseau de haut-parleurs.The techniques of the state of the art are based on the application of correction filters on each of the channels of the multi-channel signal, that is to say that each speaker of the system of restitution is corrected individually without taking into account the whole network of speakers.
Le document intitulé «Validation de la correction des réflexions sur la base d'une représentation en harmoniques sphériques » de
Il existe donc un besoin d'optimisation du calibrage effectué sur les systèmes de restitution de signaux audio multicanaux pour d'une part prendre en compte les propriétés temporelles et spatiales des réflexions sonores qui impactent la perception auditive des ondes directes, afin d'ajuster l'effort de traitement selon la perceptibilité des dégradations et ainsi limiter les artefacts audibles susceptibles d'être générés par les traitements trop contraints effectués dans les méthodes de calibration existantes ; et d'autre part utiliser de façon conjointe les différents haut-parleurs, afin de répartir l'effort de traitement sur l'ensemble des haut-parleurs.There is therefore a need to optimize the calibration carried out on multichannel audio signal restitution systems on the one hand to take into account the temporal and spatial properties of sound reflections which impact the auditory perception of direct waves, in order to adjust the 'processing effort according to the perceptibility of degradations and thus limit the audible artefacts likely to be generated by excessively constrained processing carried out in existing calibration methods; and on the other hand to use the different speakers jointly, in order to distribute the processing effort over all of the speakers.
La présente invention vient améliorer la situation.The present invention improves the situation.
Elle propose à cet effet, un procédé de calibration d'un ensemble de restitution sonore d'un signal sonore multi canal comportant une pluralité de haut-parleurs. Le procédé est tel qu'il comporte les étapes suivantes:
- obtention de réponses impulsionnelles multidirectionnelles des haut-parleurs de l'ensemble de restitution à la reproduction d'un signal audio de référence;
- analyse des réponses impulsionnelles multidirectionnelles obtenues, dans un domaine de représentation spatio-temporelle, sur au moins une fenêtre temporelle englobant les instants d'arrivée des premières réflexions du signal audio de référence reproduit pour déterminer un ensemble de caractéristiques des ondes directes et des premières réflexions comprenant au moins l'amplitude;
- comparaison de l'amplitude de chacune des réflexions à un seuil de perceptibilité déterminé en fonction de caractéristiques de l'onde directe et des premières réflexions du signal audio de référence et identification des réflexions non perceptibles pour lesquelles l'amplitude est inférieure au seuil déterminé;
- modification des réponses impulsionnelles obtenues pour obtenir des réponses impulsionnelles perceptives, par suppression des réflexions identifiées comme non perceptibles;
- détermination d'une matrice de filtrage par les étapes de :
- détermination d'un signal d'erreur défini par la différence entre un signal de réponse cible d'un ensemble de restitution et un signal de réponse reconstruit à partir des réponses impulsionnelles perceptives;
- inversion multicanale par minimisation du signal d'erreur ainsi déterminé pour obtenir les filtres de la matrice de filtrage,
- obtaining multidirectional impulse responses from the loudspeakers of the reproduction unit to the reproduction of a reference audio signal;
- analysis of the multidirectional impulse responses obtained, in a space-time representation domain, over at least one time window including the arrival times of the first reflections of the reproduced reference audio signal to determine a set of characteristics of the direct waves and the first reflections comprising at least the amplitude;
- comparison of the amplitude of each of the reflections to a perceptibility threshold determined as a function of characteristics of the direct wave and of the first reflections of the reference audio signal and identification of the non-perceptible reflections for which the amplitude is less than the determined threshold;
- modification of the impulse responses obtained to obtain perceptual impulse responses, by suppression of the reflections identified as not perceptible;
- determination of a filtering matrix by the steps of:
- determining an error signal defined by the difference between a target response signal from a rendering set and a response signal reconstructed from the perceptual impulse responses;
- multichannel inversion by minimizing the error signal thus determined to obtain the filters of the filtering matrix,
Ainsi, dans la mise en œuvre de la correction du système de restitution audio multi canal, l'effet des premières réflexions des ondes sonores diffusées par le système de restitution sur la perception auditive des ondes directes est évalué et pris en compte pour adapter le traitement appliqué aux canaux du signal multi canal selon l'effet perceptif spécifique associé à chaque réflexion. Le filtrage des canaux du signal multi canal prend ainsi en compte exclusivement les réflexions qui ont un impact sur la perception auditive des ondes directes.Thus, in the implementation of the correction of the multi-channel audio reproduction system, the effect of the first reflections of the sound waves diffused by the reproduction system on the auditory perception of direct waves is evaluated and taken into account to adapt the processing. applied to the channels of the multi-channel signal according to the specific perceptual effect associated with each reflection. The filtering of the channels of the multi-channel signal thus takes into account exclusively the reflections which have an impact on the auditory perception of direct waves.
Ceci permet donc d'augmenter la qualité du signal audio restitué.This therefore makes it possible to increase the quality of the audio signal reproduced.
De plus, comme il n'est pas nécessaire de prendre en compte les réflexions qui ne sont pas perceptibles, au sens ou leur amplitude est inférieure à un seuil de perceptibilité, les contraintes de la correction sont allégées du fait qu'elles prennent en compte les réponses impulsionnelles perceptives au lieu des réponses impulsionnelles brutes. De plus, certaines des réflexions non perceptibles qui sont éliminées des réponses impulsionnelles obtenues correspondent à des composantes de la réponse impulsionnelle qui sont justement à l'origine d'instabilités du traitement (notamment des composantes à phase non minimale). Avec les réponses impulsionnelles perceptives, on diminue ainsi les risques d'instabilités et d'artefacts qui peuvent être générés lors de traitements prenant en compte la totalité des réflexions.In addition, as it is not necessary to take into account the reflections which are not perceptible, in the sense that their amplitude is less than a perceptible threshold, the constraints of the correction are lightened by the fact that they take into account perceptual impulse responses instead of raw impulse responses. In addition, some of the non-perceptible reflections which are eliminated from the impulse responses obtained correspond to components of the impulse response which are precisely at the origin of processing instabilities (in particular components with non-minimal phase). With perceptual impulse responses, the risks of instabilities and artefacts which can be generated during treatments taking into account all of the reflections are thus reduced.
Le signal d'erreur ainsi déterminé permet de prendre en compte dans le calcul de la matrice de filtrage, uniquement les réflexions qui ont un impact sur la perception auditive de l'onde directe. En effet, seules les réflexions qui ne sont pas perceptibles sont enlevées pour la détermination du signal d'erreur.The error signal thus determined makes it possible to take into account in the calculation of the filtering matrix, only the reflections which have an impact on the auditory perception of the direct wave. Indeed, only the reflections which are not perceptible are removed for the determination of the error signal.
L'influence des réflexions sur la perception de l'onde directe dépend en effet de plusieurs caractéristiques des réflexions. Avantageusement, le seuil de perceptibilité peut être obtenu à partir de caractéristiques déterminées par l'étape d'analyse des réponses impulsionnelles multidirectionnelles des haut-parleurs.The influence of reflections on the perception of the direct wave depends on several characteristics of the reflections. Advantageously, the perceptibility threshold can be obtained from characteristics determined by the step of analyzing the multidirectional impulse responses of the loudspeakers.
Les différents modes particuliers de réalisation mentionnés ci-après peuvent être ajoutés indépendamment ou en combinaison les uns avec les autres, aux étapes du procédé défini ci-dessus.The various particular embodiments mentioned below can be added independently or in combination with one another, to the steps of the process defined above.
Plus particulièrement, le seuil de perceptibilité est déterminé en fonction de la direction d'incidence de l'onde directe et/ou de son amplitude, et des directions d'incidences des premières réflexions et/ou de leurs délais d'arrivée par rapport à l'onde directe.More particularly, the perceptibility threshold is determined as a function of the direction of incidence of the direct wave and / or of its amplitude, and of the directions of incidence of the first reflections and / or of their arrival times relative to the direct wave.
L'effet d'une réflexion sur la perception de l'onde directe dépend généralement de cinq paramètres au total ; d'une part il dépend de deux caractéristiques de l'onde directe : son amplitude et sa direction; d'autre part il dépend de trois caractéristiques de la réflexion : son amplitude, son instant d'arrivée et son incidence.The effect of a reflection on the perception of the direct wave generally depends on five parameters in total; on the one hand it depends on two characteristics of the direct wave: its amplitude and its direction; on the other hand it depends on three characteristics of the reflection: its amplitude, its instant of arrival and its incidence.
Cependant, si l'une des caractéristiques de l'onde directe n'est pas connue, il est possible d'estimer la caractéristique manquante en fixant à une valeur arbitraire l'autre caractéristique.However, if one of the characteristics of the direct wave is not known, it is possible to estimate the missing characteristic by fixing the other characteristic at an arbitrary value.
De même, si l'une des informations concernant les réflexions n'est pas connue, on peut par exemple estimer l'effet perceptif de la réflexion en fixant à une valeur arbitraire la caractéristique manquante, en prenant par exemple la valeur correspondant au cas le plus défavorable afin de majorer la perceptibilité. Ainsi, dans le cas où seule l'information de direction des réflexions est connue, il est possible de fixer une valeur à la caractéristique d'instant d'arrivée de la réflexion pour déterminer une valeur du seuil de perceptibilité uniquement par rapport à la valeur de la direction, de même si seule l'information d'instant d'arrivée de la réflexion est connue, on peut fixer la valeur de direction et déterminer le seuil de perceptibilité uniquement selon la valeur de l'instant d'arrivée. Enfin, dans le cas où les deux caractéristiques sont connues, la valeur du seuil peut être déterminée en fonction de ces deux caractéristiques.Similarly, if one of the pieces of information concerning the reflections is not known, one can for example estimate the perceptual effect of the reflection by fixing the missing characteristic at an arbitrary value, by taking for example the value corresponding to the case more unfavorable in order to increase the perceptibility. Thus, in the case where only the information of direction of the reflections is known, it is possible to fix a value to the characteristic of instant of arrival of the reflection in order to determine a value of the perceptibility threshold only with respect to the value direction, similarly if only the information of the time of arrival of the reflection is known, the direction value can be fixed and the perceptibility threshold determined only according to the value of the time of arrival. Finally, in the case where the two characteristics are known, the value of the threshold can be determined as a function of these two characteristics.
Dans un mode de réalisation possible, le signal de réponse cible correspond à la réponse de l'onde directe seule sans aucune réflexion.In one possible embodiment, the target response signal corresponds to the response of the direct wave alone without any reflection.
Ceci permet de prendre en compte comme signal de référence un signal dépourvu de tout effet de salle.This allows to take into account as reference signal a signal devoid of any room effect.
Dans une première variante de réalisation, le signal de réponse cible correspond à la réponse d'une onde directe associée à des réflexions représentatives d'un lieu d'écoute prédéterminé.In a first alternative embodiment, the target response signal corresponds to the response of a direct wave associated with reflections representative of a predetermined listening location.
La réponse de référence peut alors être volontairement choisie comme un lieu d'écoute voulue dans lequel le son est à une qualité souhaitée.The reference response can then be voluntarily chosen as a desired listening location in which the sound is at a desired quality.
Dans une seconde variante de réalisation, le signal de réponse cible correspond à la réponse d'une onde directe associée à des réflexions représentatives d'un ensemble de restitution différent.In a second variant embodiment, the target response signal corresponds to the response of a direct wave associated with reflections representative of a different restitution set.
La réponse de référence est ici choisie en fonction d'un système de restitution de référence choisi, dans lequel le nombre et la position des haut-parleurs peuvent être différents du système de restitution faisant l'objet de la correction.The reference response is here chosen as a function of a chosen reference rendering system, in which the number and the position of the loudspeakers can be different from the restitution system being corrected.
La présente invention vise également un dispositif de calibration d'un ensemble de restitution sonore d'un signal sonore multi canal comportant une pluralité de haut-parleurs. Ce dispositif est tel qu'il comporte:
- un module d'obtention de réponses impulsionnelles multidirectionnelles des haut-parleurs de l'ensemble de restitution à la reproduction d'un signal audio de référence;
- un module d'analyse des réponses impulsionnelles multidirectionnelles obtenues, dans un domaine de représentation spatio-temporelle, sur au moins une fenêtre temporelle englobant les instants d'arrivée des premières réflexions du signal audio de référence reproduit pour déterminer un ensemble de caractéristiques des ondes directe et des premières réflexions associées comprenant au moins l'amplitude;
- un module de comparaison de l'amplitude de chacune des réflexions à un seuil de perceptibilité déterminé en fonction de caractéristiques de l'onde directe et des premières réflexions du signal audio de référence et d'identification des réflexions non perceptibles pour lesquelles l'amplitude est inférieure au seuil déterminé;
- un module de modification des réponses impulsionnelles obtenues pour obtenir des réponses impulsionnelles perceptives, par suppression des réflexions identifiées comme non perceptibles par le module d'identification;
- un module de calcul d'une matrice de filtrage apte à mettre en œuvre les étapes de:
- détermination d'un signal d'erreur défini par la différence entre un signal de réponse cible d'un ensemble de restitution et un signal de réponse reconstruit à partir des réponses impulsionnelles perceptives;
- inversion multicanale par minimisation du signal d'erreur ainsi déterminé pour obtenir les filtres de la matrice de filtrage ;
- a module for obtaining multidirectional impulse responses from the loudspeakers of the reproduction unit for the reproduction of a reference audio signal;
- a module for analyzing the multidirectional impulse responses obtained, in a field of spatio-temporal representation, over at least one time window including the instants of arrival of the first reflections of the reproduced reference audio signal to determine a set of characteristics of direct waves and first associated reflections comprising at least the amplitude;
- a module for comparing the amplitude of each of the reflections at a perceptibility threshold determined as a function of characteristics of the direct wave and of the first reflections of the reference audio signal and of identifying non-perceptible reflections for which the amplitude is below the determined threshold;
- a module for modifying the impulse responses obtained in order to obtain perceptual impulse responses, by deleting the reflections identified as not perceptible by the identification module;
- a module for calculating a filtering matrix capable of implementing the steps of:
- determining an error signal defined by the difference between a target response signal from a rendering set and a response signal reconstructed from the perceptual impulse responses;
- multichannel inversion by minimizing the error signal thus determined to obtain the filters of the filtering matrix;
Ce dispositif présente les mêmes avantages que le procédé décrit précédemment, qu'il met en œuvre.This device has the same advantages as the method described above, which it implements.
L'invention vise également un décodeur audio comportant un dispositif de calibration tel que décrit.The invention also relates to an audio decoder comprising a calibration device as described.
Elle vise un programme informatique comportant des instructions de code pour la mise en œuvre des étapes du procédé de calibration tel que décrit, lorsque ces instructions sont exécutées par un processeur.It relates to a computer program comprising code instructions for implementing the steps of the calibration method as described, when these instructions are executed by a processor.
Enfin l'invention se rapporte à un support de stockage, lisible par un processeur, intégré ou non au dispositif de calibration, éventuellement amovible, mémorisant un programme informatique mettant en œuvre un procédé de calibration tel que décrit précédemment.Finally, the invention relates to a storage medium, readable by a processor, integrated or not into the calibration device, possibly removable, storing a computer program implementing a calibration method as described above.
D'autres caractéristiques et avantages de l'invention apparaîtront plus clairement à la lecture de la description suivante, donnée uniquement à titre d'exemple non limitatif, et faite en référence aux dessins annexés, sur lesquels :
- la
figure 1 représente un système de restitution sonore et un dispositif de calibration du système de restitution selon un mode de réalisation de l'invention; - la
figure 2 représente sous forme d'organigramme les étapes principales d'un procédé de calibration selon un mode de réalisation de l'invention; - la
figure 3a est une représentation d'un repère sphérique; - la
figure 3b , illustre les composantes harmoniques sphériques dans le cas d'une représentation spatiale ambisonique d'ordre 3; - la
figure 4 représente un exemple de tableau de valeurs en dB que peut prendre le seuil de perceptibilité utilisé dans le procédé de calibration selon un mode de réalisation de l'invention, pour un son direct d'angle d'incidence de 60°, en fonction de l'angle d'incidence (exprimé en degrés) de la réflexion et du temps d'arrivée (exprimé en ms) de cette réflexion par rapport à l'instant t0 d'arrivée de l'onde directe; le seuil de perceptibilité est défini comme le niveau (en dB) de la réflexion auquel est soustrait le niveau (en dB) de l'onde directe ; - la
figure 5 propose une autre illustration des valeurs prises par le seuil de perceptibilité : le seuil est cette fois représenté en fonction de l'incidence de la réflexion, et ceci pour différentes directions de l'onde directe ; dans tous les cas, le retard de la réflexion par rapport à l'onde directe est fixe et vaut 15 ms; - la
figure 6 représente un exemple d'une réponse impulsionnelle d'un haut-parleur d'un système de restitution ; le seuil de perceptibilité associé à chaque réflexion est également reproduit par une courbe pointillée; - la
figure 7 représente un exemple de réalisation matérielle d'un dispositif de calibration selon un mode de réalisation de l'invention.
- the
figure 1 represents a sound reproduction system and a device for calibrating the reproduction system according to an embodiment of the invention; - the
figure 2 represents in the form of a flowchart the main steps of a calibration method according to an embodiment of the invention; - the
figure 3a is a representation of a spherical coordinate system; - the
figure 3b , illustrates the spherical harmonic components in the case of an ambisonic spatial representation of order 3; - the
figure 4 represents an example of a table of values in dB that the perceptibility threshold used in the calibration process can take according to an embodiment of the invention, for a direct sound with an angle of incidence of 60 °, as a function of the angle of incidence (expressed in degrees) of the reflection and the arrival time (expressed in ms) of this reflection relative to the time t0 of arrival the direct wave; the perceptibility threshold is defined as the level (in dB) of the reflection from which the level (in dB) of the direct wave is subtracted; - the
figure 5 offers another illustration of the values taken by the perceptibility threshold: the threshold is this time represented as a function of the incidence of the reflection, and this for different directions of the direct wave; in all cases, the delay of the reflection with respect to the direct wave is fixed and is worth 15 ms; - the
figure 6 represents an example of an impulse response from a speaker of a reproduction system; the perceptibility threshold associated with each reflection is also reproduced by a dotted curve; - the
figure 7 shows an example of a hardware embodiment of a calibration device according to an embodiment of the invention.
La
Ces haut-parleurs sont agencés dans un lieu d'écoute dans lequel un microphone ou ensemble de microphones MA est aussi prévu.These loudspeakers are arranged in a listening location in which a microphone or set of MA microphones is also provided.
Ces haut-parleurs et microphones sont pilotés par un dispositif de traitement 100 qui peut être un décodeur tel qu'un décodeur de salon de type "set top box" pour lire ou diffuser des contenus audio ou vidéo, un serveur de traitement apte à traiter des contenus audio et vidéo et à les retransmettre à l'ensemble de restitution, un pont de conférence apte à traiter les signaux audio de différents lieux de conférence ou tout dispositif de traitement audio de signal multi canal.These speakers and microphones are controlled by a
Le dispositif de traitement 100 comporte un dispositif de calibration E selon un mode de réalisation de l'invention et une matrice de filtrage 170 composée d'une pluralité de filtres de traitement qui sont déterminés par le dispositif de calibration selon un procédé de calibration tel qu'illustré ultérieurement en référence à la
Cette matrice de filtrage reçoit en entrée un signal multi canal Si et transmet en sortie les signaux SC1, SC2, SCi, SCN aptes à être restitués par l'ensemble de restitution 180.This filtering matrix receives as input a multi-channel signal Si and transmits as output the signals SC 1 , SC 2 , SC i , SC N capable of being restored by the
Le dispositif de calibration E comporte un module de réception et d'émission 110 apte à transmettre d'une part des signaux audio de référence (Sref) aux différents haut-parleurs de l'ensemble de restitution 180 et à recevoir par le microphone ou l'ensemble de microphones MA, les réponses impulsionnelles multidirectionnelles (RIs) de ces différents haut-parleurs correspondant à la diffusion de ces signaux de référence.The calibration device E comprises a reception and transmission module 110 capable of transmitting on the one hand reference audio signals (Sref) to the various speakers of the
Une réponse impulsionnelle multidirectionnelle contient l'information temporelle et l'information spatiale relatives à l'ensemble des ondes sonores induites par le haut-parleur considéré dans la salle de reproduction.A multidirectional impulse response contains temporal and spatial information relating to all of the sound waves induced by the loudspeaker considered in the reproduction room.
Les signaux de référence sont par exemple des signaux dont la fréquence augmente de façon logarithmique avec le temps, ces signaux étant appelés en anglais "chirps" ou "sweeps" logarithmiques.The reference signals are for example signals whose frequency increases logarithmically over time, these signals being called in English "chirps" or "logarithmic sweeps".
La convolution du signal mesuré à la sortie du haut-parleur avec un signal de référence inverse permet d'obtenir directement la réponse impulsionnelle du haut-parleur.The convolution of the signal measured at the output of the loudspeaker with an inverse reference signal makes it possible to directly obtain the impulse response of the loudspeaker.
Dans un mode de réalisation particulier adapté au domaine de représentation des harmoniques sphériques lié au format ambisonique ou HOA, le microphone apte à mesurer les réponses impulsionnelles multidirectionnelles des haut-parleurs est un microphone de type HOA placé en un point du lieu d'écoute, par exemple au centre des haut-parleurs de l'ensemble de restitution.In a particular embodiment adapted to the field of representation of spherical harmonics linked to the ambisonic or HOA format, the microphone capable of measuring the multidirectional impulse responses of the loudspeakers is a HOA type microphone placed at a point of the listening place, for example in the center of the loudspeakers of the reproduction unit.
Ce microphone va recevoir, pour chaque haut-parleur restituant un signal audio de référence, le son restitué dans plusieurs directions. En effet, le microphone HOA est constitué d'une pluralité de microphones. Par un traitement approprié, l'information spatiale des différents sons captés peut être extraite. Pour plus de détails sur ce type de microphone, on peut se référer au document intitulé "
Le microphone HOA récupère alors les réponses impulsionnelles multidirectionnelles de chacun des haut-parleurs pour les transmettre au dispositif de calibration ou pour les stocker en mémoire dans un espace mémoire local ou distant.The HOA microphone then retrieves the multidirectional impulse responses from each of the speakers to transmit them to the calibration device or to store them in memory in a local or remote memory space.
Lorsque que ces informations sont stockées en mémoire, l'obtention de ces réponses impulsionnelles multidirectionnelles par le dispositif de calibration selon l'invention, s'effectue alors par une simple lecture en mémoire.When this information is stored in memory, the obtaining of these multidirectional impulse responses by the calibration device according to the invention is then carried out by a simple reading in memory.
Ces réponses impulsionnelles multidirectionnelles permettent d'obtenir des informations sur les directions d'arrivée des ondes directes et des réflexions du signal restitué ainsi que des informations de temps d'arrivée à la fois des ondes directes et des réflexions.These multidirectional impulse responses make it possible to obtain information on the directions of arrival of the direct waves and reflections of the restored signal as well as information on the time of arrival of both the direct waves and the reflections.
Le module d'analyse 120 du dispositif E effectue une analyse conjointe des réponses impulsionnelles obtenues, ce qui permet d'obtenir ces caractéristiques et notamment les caractéristiques des premières réflexions des signaux restitués. Dans le mode de réalisation particulier adapté au domaine de représentation des harmoniques sphériques, les réponses impulsionnelles multidirectionnelles sont obtenues dans une représentation spatio-temporelle où l'information spatiale est décrite sur la base des harmoniques sphériques et permet d'identifier les directions d'incidence des différentes composantes sonores. Ainsi, on obtient au final l'ensemble des informations sur l'amplitude des réflexions, leurs directions d'arrivée et leurs temps d'arrivée en comparaison au temps d'arrivée de l'onde directe. Cette étape sera décrite ultérieurement en référence à la
L'analyse des réponses impulsionnelles est faite sur une échelle temporelle prédéterminée, englobant les instants des premières réflexions.The analysis of impulse responses is done on a predetermined time scale, including the moments of the first reflections.
Dans un exemple de réalisation cette fenêtre temporelle est de longueur comprise entre 50 et 100 ms, ce qui correspond à l'échelle temporelle des instants d'arrivée des premières réflexions.In an exemplary embodiment, this time window is between 50 and 100 ms in length, which corresponds to the time scale of the instants of arrival of the first reflections.
Bien entendu, le mode de réalisation ainsi décrit est adapté au domaine de représentation des harmoniques sphériques mais il est tout à fait envisageable d'effectuer ces même étapes dans un domaine de représentation WFS (pour "Wave Field Synthesis" en anglais) ou dans le domaine des ondes planes. Dans ces cas de figures, les moyens de captation des signaux restitués par les haut-parleurs seront à adapter à ces domaines de représentation pour obtenir des réponses impulsionnelles multidirectionnelles, sans que cela s'éloigne du cadre de l'invention.Of course, the embodiment thus described is adapted to the field of representation of spherical harmonics but it is entirely possible to carry out these same steps in a field of WFS representation (for "Wave Field Synthesis" in English) or in the plane wave domain. In these cases, the means for capturing the signals reproduced by the loudspeakers will have to be adapted to these areas of representation in order to obtain multidirectional impulse responses, without departing from the scope of the invention.
Le dispositif de calibration E comporte également un module 130 de comparaison et d'identification des réflexions non perceptibles. Ce module met en œuvre une étape de comparaison des amplitudes des réflexions, obtenues par le module d'analyse 120, à un seuil de perceptibilité Se prédéterminé. Ce seuil de perceptibilité est déterminé par le module 140 à partir d'une table de valeurs prédéfinie et stockée dans un espace mémoire.The calibration device E also includes a
La détermination de ce seuil de perceptibilité sera explicitée ultérieurement en référence aux
Dans le cas où l'amplitude d'une réflexion est inférieure au seuil de perceptibilité tel que défini, cela veut dire que cette réflexion n'a pas d'impact significatif sur la perception auditive de l'onde directe du signal restitué.In the case where the amplitude of a reflection is less than the perceptibility threshold as defined, this means that this reflection has no significant impact on the auditory perception of the direct wave of the restored signal.
Une étape d'identification des ces réflexions "non perceptibles "est alors mise en œuvre par le module 130. Ces réflexions identifiées permettent de mettre en œuvre par le module 150 une étape de détermination de réponses impulsionnelles perceptives qui sont déduites des réponses impulsionnelles obtenues par le module 110 par suppression des réflexions jugées comme non perceptibles.A step of identifying these "non-perceptible" reflections is then implemented by the
Ainsi, seules les réflexions qui ont un impact sur la perception des ondes directes sont prises en compte pour calculer dans le module 160, la matrice de filtrage Filt. du module de filtrage matriciel 170.Thus, only the reflections which have an impact on the perception of direct waves are taken into account to calculate in
La
A l'étape E201, les réponses impulsionnelles multidirectionnelles des différents haut-parleurs de l'ensemble de restitution tel que décrit en référence à la
Ces réponses impulsionnelles multidirectionnelles sont les réponses de chaque haut-parleur suite à la reproduction d'un signal de référence tel que décrit en référence à la
Une étape E202 d'analyse des réponses impulsionnelles multidirectionnelles ainsi obtenues est alors mise en œuvre. Cette analyse s'effectue dans un domaine de représentation spatio-temporelle. L'information spatiale peut par exemple être décrite dans le domaine de représentation des harmoniques sphériques. Dans cette représentation illustrée à la
Dans le contexte de spatialisation ambisonique d'ordre supérieur (HOA), les composantes spatiales sont des composantes ambisoniques
Une illustration des fonctions harmoniques sphériques est représentée en
Une représentation spatiale tridimensionnelle ou "3D" dite "d'ordre M' comprend K = (M+1)2 composantes dont les triplets d'indices {m,n, σ} sont tels que 0≤m≤M, 0≤n≤m, σ=±1. Une représentation bidimensionnelle ou "2D" d'ordre M comprend un sous-ensemble de ces composantes en ne retenant que les indices m=n, soit K=2M+1 composantes.A three-dimensional or "3D" spatial representation called "of order M ' comprises K = ( M +1) 2 components whose triplets of indices { m, n, σ } are such that 0≤ m ≤ M , 0≤ n ≤ m , σ = ± 1. A two-dimensional or "2D" representation of order M includes a subset of these components, retaining only the indices m = n, ie K = 2 M +1 components.
La décomposition sur la base des harmoniques sphériques peut être considérée comme la transformée duale entre coordonnées spatiales et les fréquences spatiales. Les composantes
Pour chaque haut-parleur, on obtient à l'issue de l'étape E201, une réponse impulsionnelle multidirectionnelle qui est constituée de K réponses impulsionnelles correspondant aux K composantes de la représentation spatiale choisie. Dans le cas de la représentation des harmoniques sphériques, il s'agit des K composantes sur les K=2M+1 harmoniques sphériques considérés. Pour le jième haut-parleur, la réponse impulsionnelle multidirectionnelle qui lui est associée se compose ainsi de K réponses élémentaires Hjl(t) où l'indice l repère l'indice de la composante spatiale et t correspond à l'échantillon temporel. Par la suite, on désigne par hj(t) le vecteur des K composantes spatiales mesurées pour le jième haut-parleur :
Si le système de reproduction comprend au total N haut-parleurs, l'ensemble des réponses impulsionnelles multidirectionnelles mesurées pour les N haut-parleurs et les K composantes spatiales définit une matrice H de taille KxN, dans laquelle la jième colonne correspond à la réponse impulsionnelle multidirectionnelle associée au jième haut-parleur.If the reproduction system comprises a total of N loudspeakers, the set of multidirectional impulse responses measured for the N loudspeakers and the K spatial components defines a matrix H of size KxN, in which the jth column corresponds to the impulse response multidirectional associated with the jth speaker.
Pour chaque haut-parleur, les K composantes spatiales contenues dans le vecteur hj(t) représentent le spectre spatial des sons captés par le microphone. Pour accéder à l'information de direction des sons, il convient donc d'effectuer une transformation inverse pour repasser d'une représentation en fonction des fréquences spatiales à une représentation en fonction des coordonnées spatiales.For each loudspeaker, the K spatial components contained in the vector h j (t) represent the spatial spectrum of the sounds picked up by the microphone. To access the sound direction information, it is therefore necessary to perform an inverse transformation to go from a representation as a function of spatial frequencies to a representation as a function of spatial coordinates.
Cette transformation inverse est réalisée en reconstruisant l'onde de pression p(r, θ, δ, t) par combinaison linéaire des harmoniques sphériques, chaque harmonique étant pondéré par l'amplitude de la composante qui lui est associée. On retrouve ces éléments dans la thèse de S. Moreau citée ci-dessus.This inverse transformation is carried out by reconstructing the pressure wave p (r, θ, δ, t) by linear combination of spherical harmonics, each harmonic being weighted by the amplitude of the component associated with it. These elements are found in the thesis of S. Moreau cited above.
On peut alors évaluer l'onde de pression p(r, θ, δ, t) en tout point d'une sphère centrée sur le point de mesure des réponses impulsionnelles multidirectionnelles en reconstruisant l'onde de pression point par point par combinaison linéaire des harmoniques sphériques. On peut par exemple évaluer cette pression sur un réseau de P points définissant un « échantillonnage régulier » de la sphère au sens défini dans le mémoire de thèse de S. Moreau. Cette opération s'apparente alors au décodage spatial des composantes ambisoniques pour une restitution par un réseau sphérique régulier de P haut-parleurs virtuels. Cette étape de décodage spatial est par exemple décrite dans le document intitulé "
En pratique, cette transformation des fréquences spatiales (composantes ambisoniques) vers les coordonnées spatiales s'effectue en multipliant, pour chaque haut-parleur et chaque échantillon temporel t, le vecteur hj(t) par une matrice de décodage D. Par exemple, la matrice D peut être obtenue comme D=YT, où la matrice Y est calculée en évaluant les K harmoniques sphériques
Le maximum de cette fonction Gj(t) identifie une réflexion. Si Gj(t) présente plusieurs maxima, ces différents maxima identifient chacun une réflexion. Ainsi, pour chaque réflexion identifiée, ses caractéristiques sont déterminées selon la procédure suivante : son instant d'arrivée correspond à l'échantillon tRi = t pour lequel elle est identifiée, son incidence correspond aux coordonnées spatiales
Ainsi, à partir des réponses impulsionnelles multidirectionnelles obtenues, considérées sur une fenêtre d'analyse temporelle englobant les instants des premières réflexions du signal audio reproduit par les haut-parleurs, il est possible de déterminer, et ce pour chaque haut-parleur, les caractéristiques de l'onde directe et les caractéristiques des réflexions qui lui sont associées. Ainsi, pour le jième haut-parleur, sont déterminées d'une part les caractéristiques de l'onde directe comme son amplitude AD(j), son instant d'arrivée sur le microphone TD(j) ou sa direction d'incidence CD(j) ; et d'autre part les caractéristiques des réflexions comme leurs amplitudes ARi(j), leurs instants d'arrivée sur le microphone TRi(j) ou leurs directions d'incidences CRi(j). Dans la suite, on utilisera plutôt l'amplitude normalisée par l'amplitude de l'onde directe :
Les premières réflexions d'un signal audio restitué dépendent du lieu d'écoute dans lequel est placé l'ensemble de restitution. D'une façon générale, ces premières réflexions apparaissent dans un temps situé dans une plage allant de 50 à 100ms après l'onde directe.The first reflections of a reproduced audio signal depend on the listening location in which the reproduction unit is placed. Generally, these first reflections appear in a time located in a range from 50 to 100 ms after the direct wave.
De façon avantageuse, la fenêtre temporelle d'analyse de l'étape E202 sera, dans un mode de réalisation adapté, d'une taille comprise entre 50 et 100 ms.Advantageously, the analysis time window of step E202 will, in an adapted embodiment, be between 50 and 100 ms.
L'étape E203 compare les amplitudes obtenues par l'étape d'analyse à un seuil de perceptibilité Se des réflexions qui a été défini au préalable et stocké en mémoire. L'étape E204 permet de retrouver la valeur de seuil prédéfinie en fonction de caractéristiques de chaque réflexion et de l'onde directe associée, obtenues à l'étape d'analyse E202.Step E203 compares the amplitudes obtained by the analysis step with a perceptibility threshold Se of the reflections which has been defined beforehand and stored in memory. Step E204 makes it possible to find the predefined threshold value as a function of characteristics of each reflection and of the associated direct wave, obtained in the analysis step E202.
En effet, plusieurs cas de figure peuvent se présenter. Dans un premier exemple de réalisation, seule l'information de direction des réflexions est connue et récupérée de l'étape d'analyse. Pour retrouver le seuil de perceptibilité correspondant, on fixe la valeur de la caractéristique d'instant d'arrivée de la réflexion, par exemple la valeur la plus critique (celle qui donne une perceptibilité maximale) et on détermine la valeur du seuil de perceptibilité uniquement par rapport à la valeur de la direction.Indeed, several scenarios can arise. In a first embodiment, only the information on the direction of the reflections is known and recovered from the analysis step. To find the corresponding perceptibility threshold, the value of the characteristic of the time of arrival of the reflection is fixed, for example the most critical value (the one which gives maximum perceptibility) and the value of the perceptibility threshold is determined only relative to the value of leadership.
De même si seule l'information d'instant d'arrivée de la réflexion est connue, on peut fixer la valeur de direction, par exemple la valeur la plus critique (celle qui donne une perceptibilité maximale), et déterminer le seuil de perceptibilité selon la valeur de l'instant d'arrivée.Likewise if only the information of time of arrival of the reflection is known, the direction value can be fixed, for example the most critical value (the one which gives maximum perceptibility), and determine the perceptibility threshold according to the value of the arrival time.
Enfin, dans le cas où les deux caractéristiques sont connues, la valeur du seuil peut être déterminée, avec une meilleure précision, en fonction de ces deux caractéristiques.Finally, in the case where the two characteristics are known, the threshold value can be determined, with better precision, as a function of these two characteristics.
Pour cela, un tableau de valeurs de seuil de perceptibilité est stocké en mémoire. Un exemple d'un tel tableau est illustré en référence à la
Ce tableau de valeurs est un exemple de valeurs seuils définies à partir d'expériences psycho-acoustiques réalisées en considérant différents types de signal sonore (parole, clics, musique, etc...), différents angles d'incidences et différents temps d'arrivée des réflexions et de l'onde directe. Un seuil de perceptibilité de ces réflexions est défini en fonction de ces paramètres.This table of values is an example of threshold values defined from psycho-acoustic experiences carried out by considering different types of sound signal (speech, clicks, music, etc.), different angles of incidence and different times of arrival of reflections and of the direct wave. A threshold of perceptibility of these reflections is defined according to these parameters.
Pour compléter l'illustration des valeurs du seuil de perceptibilité de la
Ainsi, à l'étape E204, la valeur de seuil correspondant aux caractéristiques obtenues à l'étape d'analyse est récupérée. On compare cette valeur de seuil à la valeur d'amplitude de chaque réflexion à l'étape E203. Pour être comparée au seuil de perceptibilité, la valeur de l'amplitude de la réflexion est référencée à celle de l'onde directe associée et exprimée en dB: 20log(ANRi (j)).Thus, in step E204, the threshold value corresponding to the characteristics obtained in the analysis step is recovered. This threshold value is compared to the amplitude value of each reflection in step E203. To be compared to the perceptibility threshold, the value of the amplitude of the reflection is referenced to that of the associated direct wave and expressed in dB: 20log ( AN Ri ( j )).
Dans le cas où la valeur d'amplitude de la réflexion est inférieure à la valeur de seuil de perceptibilité, cela veut dire que cette réflexion n'a pas d'impact sur la perception que peut avoir un auditeur de l'onde directe. Cette réflexion n'est donc pas à prendre en compte pour le traitement d'un signal multi canal avant restitution. L'étape E203 permet ainsi d'identifier toutes les réflexions qui n'ont pas d'impact sur la perception de l'onde directe. L'étape E203 identifie donc toutes les réflexions pour lesquelles l'amplitude est inférieure au seuil de perceptibilité.In the case where the amplitude value of the reflection is lower than the perceptibility threshold value, this means that this reflection has no impact on the perception that a listener of the direct wave can have. This reflection is therefore not to be taken into account for the processing of a multi-channel signal before restitution. Step E203 thus makes it possible to identify all the reflections which have no impact on the perception of the direct wave. Step E203 therefore identifies all the reflections for which the amplitude is below the perceptibility threshold.
Pour illustrer cette étape E203, la
A partir de cette identification des réflexions non perceptibles, l'étape E205 effectue une modification des réponses impulsionnelles hj(t) obtenues à l'étape E201 pour les j=1 à N haut-parleurs, pour obtenir des réponses impulsionnelles perceptives hpj(t). Pour cela, la modification consiste à éliminer les réflexions non perceptibles identifiées à l'étape E203 dans les réponses impulsionnelles.From this identification of non-perceptible reflections, step E205 modifies the impulse responses h j (t) obtained in step E201 for j = 1 to N loudspeakers, to obtain perceptual impulse responses hp j (t). For this, the modification consists in eliminating the non-perceptible reflections identified in step E203 in the impulse responses.
De façon plus détaillée, cette opération s'effectue par exemple par une opération de seuillage. A chaque instant t, la valeur du seuil de perceptibilité Se est retranchée au signal de réponse impulsionnelle qui a été obtenue à l'étape E201.In more detail, this operation is carried out for example by a thresholding operation. At each instant t, the value of the perceptibility threshold Se is subtracted from the impulse response signal which was obtained in step E201.
Préférentiellement ce traitement est appliqué sur le spectre spatial défini par les K composantes hj(t) = [Hj1(t) ... Hjl(t) ... HjK(t)] dans le domaine de représentation spatiale choisi, correspondant par exemple à la représentation sur la base des harmoniques sphériques. Cependant le traitement peut aussi s'appliquer dans le domaine dual des coordonnées d'espace. Dans la suite, nous allons décrire l'opération réalisée dans le cas du spectre spatial.Preferably, this treatment is applied to the spatial spectrum defined by the K components h j (t) = [H j1 (t) ... H jl (t) ... H jK (t)] in the area of spatial representation. chosen, corresponding for example to the representation on the basis of spherical harmonics. However, the processing can also be applied in the dual domain of space coordinates. In the following, we will describe the operation performed in the case of the spatial spectrum.
L'opération de seuillage consiste à comparer pour chaque réflexion identifiée son amplitude au seuil de perceptibilité Se associé à ses caractéristiques. Ainsi, pour la ième réflexion identifiée pour le jième haut-parleur, le seuil Se(i) est déterminé en fonction de ses caractéristiques [τRi (j), CRi(j)]. Cette réflexion est localisée à l'instant ti donné par:
Pour réaliser le seuillage, on considère donc la réponse impulsionnelle à cet instant, soit hj(ti), ou plus exactement sur le spectre spatial associé et constitué des K composantes [Hj1(ti) ... Hjl(ti) ... HjK(ti)]. Plusieurs stratégies sont alors possibles. La plus simple consiste à préserver l'amplitude relative des composantes du spectre spatial, c'est-à-dire qu'on applique un traitement identique à toutes les composantes. Dans ce cas, pour chaque composante Hjl(ti), l'opération de seuillage peut se traduire par les équations suivantes:
Ainsi, les réponses impulsionnelles perceptives ne conservent que les réflexions ayant un impact significatif sur la perception de l'onde directe.Thus, the perceptual impulse responses only retain the reflections having a significant impact on the perception of the direct wave.
Ces réponses impulsionnelles perceptives sont alors utilisées pour déterminer la matrice de filtrage, à l'étape E206. Cette matrice de filtrage est ensuite utilisée pour traiter le signal audio multi canal avant sa restitution sonore par l'ensemble de restitution du système.These perceptual impulse responses are then used to determine the filtering matrix, in step E206. This filtering matrix is then used to process the multi-channel audio signal before its sound reproduction by the system reproduction unit.
Pour obtenir l'ensemble de filtres constituant la matrice de filtrage Filt du dispositif de traitement, un mode de réalisation possible comporte une étape de détermination d'un signal d'erreur défini par la différence entre un signal de réponse cible prédéterminé de l'ensemble de restitution et un signal de réponse reconstruit à partir des réponses impulsionnelles perceptives et une étape d'inversion multicanale par minimisation du signal d'erreur ainsi déterminé.To obtain the set of filters constituting the Filt filtering matrix of the processing device, a possible embodiment includes a step of determining an error signal defined by the difference between a predetermined target response signal from the set of restitution and a response signal reconstructed from the perceptual impulse responses and a multichannel inversion step by minimization of the error signal thus determined.
Le signal d'erreur ainsi obtenu ne prend donc en compte que les réflexions perceptibles puisque qu'il est calculé à partir d'un signal reconstruit basé sur les réponses impulsionnelles perceptives.The error signal thus obtained therefore only takes into account the perceptible reflections since it is calculated from a reconstructed signal based on the perceptual impulse responses.
L'inversion peut être réalisée par un algorithme de descente de gradient ou ses variantes. Un exemple d'algorithme d'inversion possible est celui de type ISTA (pour "Iterative Shrinkage-Thresholding algorithm) tel que décrit dans le document intitulé "
D'une façon générale, le problème qui se pose pour calculer les filtres de la matrice de traitement, est le suivant. Il y a N haut-parleurs qui constituent le système réel de reproduction. Dans le contexte de spatialisation ambisonique d'ordre supérieur (HOA), l'espace de représentation spatiale est de dimension K. L'information spatiale est donc décrite par K coefficients. L'objectif est de reproduire avec le système de N haut-parleurs, un ensemble de V signaux définissant le signal audio multicanal d'entrée. Ces V signaux sont dédiés à un système idéal de reproduction constitués de V haut-parleurs. Ce système idéal définit les V signaux cibles qu'on souhaite reproduire et qui correspondent donc aux réponses d'un système fictif de V haut-parleurs virtuels. Dans le cas le plus simple, le système réel de reproduction comporte aussi N=V haut-parleurs. Mais dans le cas général, on est capable d'émuler un système de V haut-parleurs virtuels à partir d'un dispositif de N haut-parleurs réels.In general, the problem which arises to calculate the filters of the processing matrix is as follows. There are N loudspeakers which constitute the real reproduction system. In the context of higher order ambisonic spatialization (HOA), the space of spatial representation is of dimension K. Spatial information is therefore described by K coefficients. The objective is to reproduce with the N speaker system, a set of V signals defining the input multichannel audio signal. These V signals are dedicated to an ideal reproduction system consisting of V loudspeakers. This ideal system defines the V target signals which one wishes to reproduce and which therefore correspond to the responses of a fictitious system of V virtual loudspeakers. In the simplest case, the actual reproduction system also includes N = V loudspeakers. But in the general case, we are able to emulate a system of V virtual speakers from a device of N real speakers.
L'équation à résoudre est la suivante:
- avec H, la matrice de dimension KxN comportant les réponses impulsionnelles des N éléments du système de restitution dans le domaine d'analyse spatiale,
- W, la matrice comportant les filtres de correction à calculer, de dimension NxV,
- T, la matrice contenant les V réponses cibles définies dans le domaine d'analyse spatiale, de dimension KxV,
- et l'opération dénotée par « * » est un produit matriciel convolutif où un élément Tij de la matrice T est obtenu de la façon suivante :
L'objectif de l'opération d'inversion est de trouver les éléments de la matrice W.The equation to solve is as follows:
- with H, the matrix of dimension KxN comprising the impulse responses of the N elements of the restitution system in the field of spatial analysis,
- W, the matrix comprising the correction filters to be calculated, of dimension NxV,
- T, the matrix containing the V target responses defined in the area of spatial analysis, of dimension KxV,
- and the operation denoted by "*" is a convolutional matrix product where an element T ij of the matrix T is obtained in the following way:
The objective of the inversion operation is to find the elements of the matrix W.
La résolution de cette opération peut s'effectuer en deux temps. Tout d'abord, on calcule les filtres de correction en ne corrigeant que l'effet de salle du lieu de restitution, c'est-à-dire qu'on prend en compte le dispositif réel de haut-parleurs, soit N haut-parleurs. Dans une seconde étape, on compense la disposition des haut-parleurs pour adapter les V signaux à une restitution selon une configuration non idéale de N haut-parleurs. Dans ce but, les V signaux sont répartis par matriçage sur les N canaux associés au système réel de reproduction afin d'émuler un système de V haut-parleurs virtuels.The resolution of this operation can be carried out in two stages. First of all, the correction filters are calculated by correcting only the room effect of the restitution place, that is to say that we take into account the actual loudspeaker device, ie N loud- speakers. In a second step, the arrangement of the loudspeakers is compensated for in order to adapt the V signals to a restitution according to a non-ideal configuration of N loudspeakers. For this purpose, the V signals are distributed by matrixing on the N channels associated with the real reproduction system in order to emulate a system of V virtual loudspeakers.
Dans le cas présent, pour mettre en œuvre l'invention, les éléments de la matrice H comportent les réponses impulsionnelles perceptives telles qu'obtenues à l'étape E205.In the present case, to implement the invention, the elements of the matrix H include the perceptual impulse responses as obtained in step E205.
Les réponses cibles peuvent varier selon le résultat de restitution sonore attendue.The target responses may vary depending on the expected sound reproduction result.
Dans un mode de réalisation, cette réponse cible correspond à la réponse impulsionnelle donnée par l'onde directe seule sans aucune réflexion. Cela revient à supprimer tout l'effet de salle dans le signal attendu.In one embodiment, this target response corresponds to the impulse response given by the direct wave alone without any reflection. This is equivalent to suppressing all the room effect in the expected signal.
Dans une première variante de réalisation, le signal de réponse cible correspond à la réponse d'une onde directe associée à des réflexions représentatives d'un lieu d'écoute prédéterminé.In a first alternative embodiment, the target response signal corresponds to the response of a direct wave associated with reflections representative of a predetermined listening location.
Un lieu d'écoute caractéristique qui présente une bonne qualité d'écoute peut être souhaitée (par exemple le lieu d'écoute de la salle Pleyel™). Dans ce cas, les filtres de traitement seront calculés pour obtenir une restitution sonore proche de cette qualité d'écoute.A characteristic listening location with good listening quality may be desired (for example the listening location in the Pleyel ™ room). In this case, the processing filters will be calculated to obtain a sound reproduction close to this listening quality.
Dans une deuxième variante de réalisation, le signal de réponse cible correspond à la réponse d'une onde directe associée à des réflexions représentatives d'un ensemble de restitution différent de celui utilisé pour restituer le signal résultant.In a second variant embodiment, the target response signal corresponds to the response of a direct wave associated with reflections representative of a restitution set different from that used to restore the resulting signal.
Ainsi, un système de restitution souhaité, par exemple comportant plus de haut-parleurs, est pris comme référence pour obtenir une restitution proche de celle qui aurait été obtenu avec un tel système.Thus, a desired restitution system, for example comprising more speakers, is taken as a reference for obtaining a restitution close to that which would have been obtained with such a system.
D'autres signaux de réponse cibles peuvent bien évidemment être choisi selon l'effet de la restitution souhaitée.Other target response signals can obviously be chosen according to the effect of the desired restitution.
Ainsi, la mise en œuvre du procédé décrit permet d'obtenir une meilleure qualité d'écoute lors de la restitution d'un signal audio multi canal grâce à la prise en compte seule des réflexions perceptibles des signaux par l'ensemble de restitution dans le lieu d'écoute.Thus, the implementation of the described method makes it possible to obtain a better quality of listening during the restitution of a multi-channel audio signal thanks to the taking into account only of the perceptible reflections of the signals by the restitution unit in the listening place.
La
Ce type de dispositif comporte un processeur µP coopérant avec un bloc mémoire MEM comportant une mémoire de stockage et/ou de travail.This type of device comprises a µP processor cooperating with a memory block MEM comprising a storage and / or working memory.
Le bloc mémoire peut avantageusement comporter un programme informatique comportant des instructions de code pour la mise en œuvre des étapes du procédé de calibration au sens de l'invention, lorsque ces instructions sont exécutées par le processeur, et notamment les étapes d'obtention de réponses impulsionnelles multidirectionnelles des haut-parleurs de l'ensemble de restitution à la reproduction d'un signal audio prédéterminé, d'analyse des réponses impulsionnelles multidirectionnelles obtenues, dans un domaine de représentation spatio-temporelle, sur au moins une fenêtre temporelle englobant les instants d'arrivée des premières réflexions du signal audio prédéterminé reproduit pour déterminer un ensemble de caractéristiques des premières réflexions, de comparaison de l'amplitude de chacune des réflexions à un seuil de perceptibilité prédéterminé et d'identification des réflexions non perceptibles pour lesquelles l'amplitude est inférieure au seuil prédéterminé, de modification des réponses impulsionnelles obtenues pour obtenir des réponses impulsionnelles perceptives, par suppression des réflexions identifiées comme non perceptibles et de détermination d'une matrice de filtrage à partir des réponses impulsionnelles perceptives pour une application de cette matrice de filtrage au signal audio multi canal avant restitution sonore.The memory block can advantageously include a computer program comprising code instructions for implementing the steps of the calibration method within the meaning of the invention, when these instructions are executed by the processor, and in particular the steps for obtaining responses. multidirectional impulse signals from the loudspeakers of the reproduction unit for reproducing a predetermined audio signal, for analyzing the multidirectional impulse responses obtained, in a space-time representation domain, over at least one time window including the instants d arrival of the first reflections of the predetermined audio signal reproduced to determine a set of characteristics of the first reflections, of comparing the amplitude of each of the reflections at a predetermined perceptibility threshold and of identifying the non-perceptible reflections for which the amplitude is less than the predetermined threshold, modification of the impulse responses obtained to obtain perceptual impulse responses, by deletion of the reflections identified as non-perceptible and determination of a filtering matrix from the perceptual impulse responses for an application of this filtering matrix to the multi-channel audio signal before sound reproduction.
Typiquement, la description de la
La mémoire MEM enregistre une table de valeurs de seuil de perceptibilité en fonction de caractéristiques des composantes sonores constituées de l'onde directe et des réflexions utilisée dans le procédé selon un mode de réalisation de l'invention et de manière générale, toutes les données nécessaires à la mise en œuvre du procédé.The memory MEM stores a table of perceptibility threshold values as a function of characteristics of the sound components consisting of the direct wave and of the reflections used in the method according to an embodiment of the invention and in general, all the necessary data. to the implementation of the process.
Un tel dispositif comporte un module d'entrée I apte à recevoir des réponses impulsionnelles d'un ensemble de restitution et un module de sortie S apte à transmettre à un module de traitement, les filtres calculés d'une matrice de filtrage.Such a device comprises an input module I capable of receiving impulse responses from a restitution assembly and an output module S capable of transmitting to a processing module, the filters calculated from a filtering matrix.
Dans un mode possible de réalisation, le dispositif ainsi décrit peut également comporter les fonctions de traitement par la mise en œuvre de la matrice de traitement à la réception en I d'un signal multi canal Si pour transmettre en sortie des signaux traités SCi aptes à être restitués par l'ensemble de restitution.In one possible embodiment, the device thus described may also include the processing functions by implementing the processing matrix on reception at I of a multi-channel signal Si to output processed signals SCi capable of be returned by the return package.
Claims (9)
- Method for calibrating an assembly for sound playback of a multi-channel sound signal having a plurality of loudspeakers, the method having the following steps:- obtaining (E201) multi-directional impulse responses from the loudspeakers of the playback assembly upon reproduction of a reference audio signal;- analyzing (E202) the multi-directional impulse responses obtained, in a domain of spatio-temporal representation, over at least one time window encompassing the instants of arrival of the early reflections of the reproduced reference audio signal in order to determine a set of characteristics of the direct waves and of the associated early reflections comprising at least the amplitude;the method being characterized in that is has the following steps:- comparing (E203) the amplitude of each of the reflections with a determined perceptibility threshold (E204) as a function of characteristics of the direct wave and of the early reflections of the reference audio signal and identifying (E203) the non-perceptible reflections for which the amplitude is below the determined threshold;- modifying (E205) the impulse responses obtained in order to obtain perceptual impulse responses, by suppression of the reflections identified as non-perceptible;- determining (E206) a filtering matrix with the steps of:- determination of an error signal defined by the difference between a target response signal for a playback assembly and a response signal reconstructed from the perceptual impulse responses;- multi-channel inversion by minimization of the error signal thus determined in order to obtain the filters of the filtering matrix,for an application of this filtering matrix to the multi-channel audio signal before sound playback.
- Method according to Claim 1, characterized in that the perceptibility threshold is determined as a function of the direction of incidence of the direct wave (CD) and/or its amplitude (AD), and the directions of incidence of the early reflections (CRi) and/or their arrival times (τRi) with respect to the direct wave.
- Method according to Claim 1, characterized in that the target response signal corresponds to the response of the direct wave alone without any reflection.
- Method according to Claim 1, characterized in that the target response signal corresponds to the response of a direct wave associated with reflections representing a predetermined listening site.
- Method according to Claim 1, characterized in that the target response signal corresponds to the response of a direct wave associated with reflections representing a different playback assembly.
- Device for calibrating an assembly for sound playback of a multi-channel sound signal having a plurality of loudspeakers, the device having:- a module (110) for obtaining multi-directional impulse responses from the loudspeakers of the playback assembly upon reproduction of a reference audio signal;- a module (120) for analyzing the multi-directional impulse responses obtained, in a domain of spatio-temporal representation, over at least one time window encompassing the instants of arrival of the early reflections of the reproduced reference audio signal in order to determine a set of characteristics of the direct waves and of the associated early reflections comprising at least the amplitude;the device being characterized in that it comprises:- a module (120) for comparing the amplitude of each of the reflections with a determined perceptibility threshold (140) as a function of characteristics of the direct wave and of the early reflections of the reference audio signal and for identifying (120) the non-perceptible reflections for which the amplitude is below the determined threshold;- a module (150) for modifying the impulse responses obtained in order to obtain perceptual impulse responses, by suppression of the reflections identified as non-perceptible by the identification module;- a module (130) for computing a filtering matrix able to implement the steps of:- determination of an error signal defined by the difference between a target response signal for a playback assembly and a response signal reconstructed from the perceptual impulse responses;- multi-channel inversion by minimization of the error signal thus determined in order to obtain the filters of the filtering matrix;for an application of this filtering matrix to the multi-channel audio signal before sound playback.
- Audio decoder having a calibration device according to Claim 6.
- Computer program having code instructions for the implementation of the steps of the calibration method according to one of Claims 1 to 5 when these instructions are executed by a processor.
- Storage medium, readable by a processor, on which a computer program is stored comprising code instructions for the execution of the steps of the calibration method according to one of Claims 1 to 5.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
FR1258760A FR2995754A1 (en) | 2012-09-18 | 2012-09-18 | OPTIMIZED CALIBRATION OF A MULTI-SPEAKER SOUND RESTITUTION SYSTEM |
PCT/FR2013/052047 WO2014044948A1 (en) | 2012-09-18 | 2013-09-05 | Optimized calibration of a multi-loudspeaker sound restitution system |
Publications (2)
Publication Number | Publication Date |
---|---|
EP2898707A1 EP2898707A1 (en) | 2015-07-29 |
EP2898707B1 true EP2898707B1 (en) | 2020-04-22 |
Family
ID=47215616
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP13774728.3A Active EP2898707B1 (en) | 2012-09-18 | 2013-09-05 | Optimized calibration of a multi-loudspeaker sound restitution system |
Country Status (4)
Country | Link |
---|---|
US (1) | US9584947B2 (en) |
EP (1) | EP2898707B1 (en) |
FR (1) | FR2995754A1 (en) |
WO (1) | WO2014044948A1 (en) |
Families Citing this family (23)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9084058B2 (en) | 2011-12-29 | 2015-07-14 | Sonos, Inc. | Sound field calibration using listener localization |
US9219460B2 (en) | 2014-03-17 | 2015-12-22 | Sonos, Inc. | Audio settings based on environment |
US9106192B2 (en) | 2012-06-28 | 2015-08-11 | Sonos, Inc. | System and method for device playback calibration |
US9565497B2 (en) * | 2013-08-01 | 2017-02-07 | Caavo Inc. | Enhancing audio using a mobile device |
US9264839B2 (en) | 2014-03-17 | 2016-02-16 | Sonos, Inc. | Playback device configuration based on proximity detection |
US9952825B2 (en) | 2014-09-09 | 2018-04-24 | Sonos, Inc. | Audio processing algorithms |
US12087311B2 (en) | 2015-07-30 | 2024-09-10 | Dolby Laboratories Licensing Corporation | Method and apparatus for encoding and decoding an HOA representation |
EP3329486B1 (en) * | 2015-07-30 | 2020-07-29 | Dolby International AB | Method and apparatus for generating from an hoa signal representation a mezzanine hoa signal representation |
US9693165B2 (en) | 2015-09-17 | 2017-06-27 | Sonos, Inc. | Validation of audio calibration using multi-dimensional motion check |
JP6437695B2 (en) | 2015-09-17 | 2018-12-12 | ソノズ インコーポレイテッド | How to facilitate calibration of audio playback devices |
US9779759B2 (en) * | 2015-09-17 | 2017-10-03 | Sonos, Inc. | Device impairment detection |
US9743207B1 (en) | 2016-01-18 | 2017-08-22 | Sonos, Inc. | Calibration using multiple recording devices |
US10003899B2 (en) | 2016-01-25 | 2018-06-19 | Sonos, Inc. | Calibration with particular locations |
US9864574B2 (en) | 2016-04-01 | 2018-01-09 | Sonos, Inc. | Playback device calibration based on representation spectral characteristics |
US9860662B2 (en) | 2016-04-01 | 2018-01-02 | Sonos, Inc. | Updating playback device configuration information based on calibration data |
US9763018B1 (en) | 2016-04-12 | 2017-09-12 | Sonos, Inc. | Calibration of audio playback devices |
US9794710B1 (en) | 2016-07-15 | 2017-10-17 | Sonos, Inc. | Spatial audio correction |
CN112492502B (en) * | 2016-07-15 | 2022-07-19 | 搜诺思公司 | Networked microphone apparatus, method thereof, and media playback system |
US10372406B2 (en) | 2016-07-22 | 2019-08-06 | Sonos, Inc. | Calibration interface |
US10459684B2 (en) | 2016-08-05 | 2019-10-29 | Sonos, Inc. | Calibration of a playback device based on an estimated frequency response |
CN109863764B (en) * | 2016-10-19 | 2020-09-08 | 华为技术有限公司 | Method and device for controlling acoustic signals to be recorded and/or reproduced by an electroacoustic sound system |
US10299061B1 (en) | 2018-08-28 | 2019-05-21 | Sonos, Inc. | Playback device calibration |
US10734965B1 (en) | 2019-08-12 | 2020-08-04 | Sonos, Inc. | Audio calibration of a portable playback device |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE10351793B4 (en) * | 2003-11-06 | 2006-01-12 | Herbert Buchner | Adaptive filter device and method for processing an acoustic input signal |
-
2012
- 2012-09-18 FR FR1258760A patent/FR2995754A1/en active Pending
-
2013
- 2013-09-05 EP EP13774728.3A patent/EP2898707B1/en active Active
- 2013-09-05 WO PCT/FR2013/052047 patent/WO2014044948A1/en active Application Filing
- 2013-09-05 US US14/429,291 patent/US9584947B2/en active Active
Non-Patent Citations (1)
Title |
---|
None * |
Also Published As
Publication number | Publication date |
---|---|
WO2014044948A1 (en) | 2014-03-27 |
US20150223004A1 (en) | 2015-08-06 |
EP2898707A1 (en) | 2015-07-29 |
FR2995754A1 (en) | 2014-03-21 |
US9584947B2 (en) | 2017-02-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2898707B1 (en) | Optimized calibration of a multi-loudspeaker sound restitution system | |
EP2374124B1 (en) | Advanced encoding of multi-channel digital audio signals | |
EP2374123B1 (en) | Improved encoding of multichannel digital audio signals | |
EP1836876B1 (en) | Method and device for individualizing hrtfs by modeling | |
EP1992198B1 (en) | Optimization of binaural sound spatialization based on multichannel encoding | |
EP1946612B1 (en) | Hrtfs individualisation by a finite element modelling coupled with a corrective model | |
EP2002424B1 (en) | Device and method for scalable encoding of a multichannel audio signal based on a principal component analysis | |
EP3807669B1 (en) | Location of sound sources in a given acoustic environment | |
EP3427260B1 (en) | Optimized coding and decoding of spatialization information for the parametric coding and decoding of a multichannel audio signal | |
EP1479266B1 (en) | Method and device for control of a unit for reproduction of an acoustic field | |
EP1586220B1 (en) | Method and device for controlling a reproduction unit using a multi-channel signal | |
EP3706119A1 (en) | Spatialised audio encoding with interpolation and quantifying of rotations | |
EP3895446B1 (en) | Method for interpolating a sound field and corresponding computer program product and device | |
EP1652406B1 (en) | System and method for determining a representation of an acoustic field | |
EP3559947B1 (en) | Processing in sub-bands of an actual ambisonic content for improved decoding | |
EP3025514B1 (en) | Sound spatialization with room effect | |
WO2018050292A1 (en) | Device and method for capturing and processing a three-dimensional acoustic field | |
EP3384688B1 (en) | Successive decompositions of audio filters | |
EP4042418B1 (en) | Determining corrections to be applied to a multichannel audio signal, associated coding and decoding | |
EP3934282A1 (en) | Method for converting a first set of signals representing a sound field into a second set of signals and associated electronic device | |
WO2009081002A1 (en) | Processing of a 3d audio stream as a function of a level of presence of spatial components | |
FR3143932A1 (en) | Obtaining an impulse response from a room | |
FR2943867A1 (en) | Three dimensional audio signal i.e. ambiophonic signal, processing method for computer, involves determining equalization processing parameters according to space components based on relative tolerance threshold and acquisition noise level | |
WO2014102199A1 (en) | Device and method for the spatial interpolation of sounds |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20150417 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: NICOL, ROZENN Inventor name: DEPREZ, ROMAIN |
|
DAX | Request for extension of the european patent (deleted) | ||
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
17Q | First examination report despatched |
Effective date: 20180912 |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
INTG | Intention to grant announced |
Effective date: 20191220 |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: NICOL, ROZENN Inventor name: DEPREZ, ROMAIN |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602013068191 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: FRENCH |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 1261742 Country of ref document: AT Kind code of ref document: T Effective date: 20200515 |
|
RAP2 | Party data changed (patent owner data changed or rights of a patent transferred) |
Owner name: ORANGE |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20200422 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200824 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200422 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200722 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200422 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200422 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200422 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200822 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200723 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 1261742 Country of ref document: AT Kind code of ref document: T Effective date: 20200422 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200422 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200422 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200422 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200722 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200422 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602013068191 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200422 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200422 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200422 Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200422 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200422 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200422 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200422 Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200422 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200422 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200422 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
26N | No opposition filed |
Effective date: 20210125 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200422 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200422 |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20200930 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20200905 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20200930 Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20200905 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20200930 Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20200930 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200422 Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200422 Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200422 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200422 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20240820 Year of fee payment: 12 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20240820 Year of fee payment: 12 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20240820 Year of fee payment: 12 |