EP3844981A1 - Procédé pour une restitution sonore spatialisée d'un champ sonore audible en une position d'un auditeur se déplaçant et système mettant en ouvre un tel procédé - Google Patents
Procédé pour une restitution sonore spatialisée d'un champ sonore audible en une position d'un auditeur se déplaçant et système mettant en ouvre un tel procédéInfo
- Publication number
- EP3844981A1 EP3844981A1 EP19778569.4A EP19778569A EP3844981A1 EP 3844981 A1 EP3844981 A1 EP 3844981A1 EP 19778569 A EP19778569 A EP 19778569A EP 3844981 A1 EP3844981 A1 EP 3844981A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- listener
- sub
- zone
- loudspeakers
- mic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
- H04S7/302—Electronic adaptation of stereophonic sound system to listener position or orientation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
- H04S7/302—Electronic adaptation of stereophonic sound system to listener position or orientation
- H04S7/303—Tracking of listener position or orientation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
- H04R1/00—Details of transducers, loudspeakers or microphones
- H04R1/20—Arrangements for obtaining desired frequency or directional characteristics
- H04R1/32—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only
- H04R1/40—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers
- H04R1/403—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers loud-speakers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers
- H04R3/005—Circuits for transducers for combining the signals of two or more microphones
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
- H04S7/301—Automatic calibration of stereophonic sound system, e.g. with test microphone
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
- H04S2400/13—Aspects of volume control, not necessarily automatic, in stereophonic sound systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
- H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
Definitions
- the invention is in the field of spatialized audio and sound field control.
- the purpose of the method is to restore at least one sound field in an area, for a listener, according to the position of the listener.
- the method aims to restore the sound field by taking into account the movements of the listener.
- the area is covered by a network of speakers, powered by respective control signals to each transmit an audio signal continuously.
- a respective weight is applied to each speaker control signal in order to restore the sound field according to the position of the listener. From the weights, a set of filters is determined, each filter of the set of filters corresponding to each loudspeaker. The signal to be distributed to the listener is then filtered by the filter set and broadcast by the loudspeaker corresponding to the filter.
- the iterative methods used use the weights calculated in the previous iteration to calculate the new weights.
- the filter set therefore has a memory of the previous iterations.
- part of the sound field which was restored in the previous iteration (or in the old position of the listener) is absent from the new position of the listener. It is therefore no longer constrained and the part of the weights allowing this previous restitution is no longer useful but remains in memory.
- the sound field restored at the previous position of the listener, at the previous iteration is no longer useful for calculating the weights at the current position of the listener, or at the current iteration, but remains in memory.
- the present invention improves the situation.
- the method comprises iteratively and continuously for each listener: obtaining the current position of a listener in the area by means of a position sensor;
- a position of said point being defined dynamically as a function of the current position of the listener, said point corresponding to a virtual microphone position
- the present method is therefore based directly on the displacement of the listener to vary the forget factor at each iteration. This reduces the memory effect due to the calculation of the weights in the previous iterations. The precision of the field restitution is greatly improved, while not requiring too expensive computing resources.
- a plurality of points forming the respective positions of a plurality of virtual microphones is defined in the area to estimate a plurality of respective acoustic pressures in the area taking into account the respective weight applied to each speaker , each comprising a forgetting factor respectively, and transfer functions specific to each loudspeaker into each virtual microphone, the plurality of points being centered on the position of the listener.
- the sound pressure is estimated at a plurality of points in the area, surrounding the listener.
- This allows weights to be applied to each speaker taking into account the differences in sound pressures that may occur at different points in the area.
- the estimation of the acoustic pressures is therefore carried out in a homogeneous and precise manner around the listener, which makes it possible to increase the precision of the method.
- the zone comprises a first sub-zone in which the selected sound field is to make audible and a second sub-zone in which the chosen sound field is to make inaudible, the first sub-zone being defined dynamically as corresponding to the position of the listener and of said virtual microphone, the virtual microphone being a first virtual microphone, and the second sub-zone being defined dynamically as being complementary to the first sub-zone, the second sub-zone being covered by at least one second virtual microphone whose position is defined dynamically as a function of said second sub-zone, the method further comprising iteratively:
- the method therefore allows to restore different sound fields in the same area, using the same speaker system, based on a movement of the listener.
- the sound field actually restored in the two sub-zones is evaluated so that each time the listener moves, the sound pressure in each of the sub-zones effectively reaches the target sound pressure.
- the position of the auditor can be used to determine the sub-area in which the field sound is to be made audible.
- the sub-zone in which the sound field is to be made inaudible is then defined dynamically each time the listener moves.
- the forgetting factor is therefore calculated iteratively for each of the two sub-zones, so that the sound pressure in each of the sub-zones reaches its target sound pressure.
- the zone comprises a first sub-zone in which the selected sound field is to make audible and a second sub-zone in which the chosen sound field is to make inaudible, the second sub-zone being defined dynamically as corresponding to the position of the listener and of said virtual microphone, the virtual microphone being a first virtual microphone, and the first sub-zone being defined dynamically as being complementary to the second sub-zone, the first sub-zone being covered by at least one second virtual microphone whose position is defined dynamically as a function of said first sub-zone, the method further comprising iteratively:
- each sub-zone comprises at least one virtual microphone and two speakers, and preferably each sub-zone comprises at least ten virtual microphones and at least ten speakers. The method is therefore able to operate with a plurality of microphones and speakers.
- a value of the forgetting factor increases if the listener moves and decreases if the listener does not move.
- the forgetting factor is defined by , with g (h) the forget factor, n the current iteration, y max the maximum forget factor, c a parameter defined by the designer equal to m an adaptation step, m a variable defined according to d '' a displacement of the listener having as a maximum c and a variable allowing to adjust the speed of increase or decrease of the forgetting factor.
- the forgetting factor is directly estimated based on a displacement of the listener.
- the forgetting factor depends on the distance traveled by each listener by the listener, in other words the speed of movement of the listener. A different forgetting factor can therefore be estimated for each listener.
- the values of the variables can also be adjusted during iterations so as to really take into account the displacement of the listener.
- the forgetting factor is between 0 and 1.
- the present invention also relates to a system of spatialized sound reproduction from a network of loudspeakers covering an area, with a view to broadcasting a selected sound field, selectively audible in a position of a listener in the area, characterized in that it comprises a processing unit suitable for the processing and the implementation of the method according to the invention.
- the present invention also relates to a storage medium for a computer program, which can be loaded into a memory associated with a processor, and comprising portions of code for implementing a method according to the invention during the execution of said program by the processor.
- FIG. 1 represents an example of a system according to an embodiment of the invention
- FIGS. 2a and 2b illustrate, in the form of a flowchart, the main steps of a particular embodiment of the method
- FIG. 3 schematically illustrates an embodiment in which two sub-zones are defined dynamically as a function of the geolocation data of an auditor
- Figures 4a and 4b illustrate, in the form of a flowchart, the main steps of a second embodiment of the method.
- FIG. 1 schematically illustrates a SYST system according to an exemplary embodiment.
- the SYST system comprises a network of loudspeakers HP comprising N loudspeakers (HP I , ..., HP n ), with N at least equal to 2, and preferably at least equal to 10.
- the loudspeaker network loudspeakers cover a zone Z.
- the loudspeakers HP are supplied by respective control signals to each emit an audio signal continuously, for the purpose of a spatialized sound diffusion of a sound field chosen in zone Z. More precisely , the selected sound field is to be reproduced at a position al by a listener U.
- the loudspeakers can be defined by their position in the zone.
- the position al of the listener U can be obtained by means of a position sensor POS.
- the area is also covered by MIC microphones.
- the area is covered by a network of M MIC microphones, with M at least equal to 1 and preferably at least equal to 10.
- the MIC microphones are virtual microphones.
- the term “microphone MIC” is used, the microphones being able to be real or virtual.
- MIC microphones are identified by their position in the Z zone.
- the virtual microphones are defined as a function of the position al of the listener U in the zone Z.
- the virtual microphones MIC can be defined so as to surround the listener U.
- the position of the virtual microphones MIC changes as a function of the position al of the listener U.
- the array of microphones MIC surrounds the position al of the listener U. Then, when the listener U moves towards position a2, the array of MIC microphones is redefined to surround the listener's position a2.
- the displacement of listener U is shown diagrammatically by the arrow F.
- the SYST system further comprises a TRAIT processing unit capable of implementing the steps of the method.
- the TRAIT processing unit notably includes a memory, forming a storage medium for a computer program comprising portions of code for implementing the method described below with reference to FIGS. 2a and 2b.
- the TRAIT processing unit further comprises a processor PROC capable of executing the code portions of the computer program.
- the TRAIT processing unit receives, continuously and in real time, the position of the microphones MIC, the position of the listener U, the positions of each speaker HP, the audio signal to be reproduced S (U) intended for the listener U and the target sound field P t to be reached at the position of the listener.
- the TRAIT processing unit also receives the estimated acoustic pressure P at the position of the listener U. From this data, the TRAIT processing unit calculates the FILT filter to be applied to the signal S in order to restore the sound field target P t .
- the TRAIT processing unit outputs the filtered signals S (HPi ... HPN) to be broadcast respectively on the speakers HPi to HPN .
- Figures 2a and 2b illustrate the main steps of a method for the reproduction of a selected sound field in a position of a listener, when the listener is moving.
- the process steps are implemented by the TRAIT processing unit continuously and in real time.
- step S1 the position of the listener U in the zone is obtained by means of a position sensor. From this geolocation data, a network of virtual microphones MIC is defined in step S2.
- the network of virtual MIC microphones can take any geometric shape such as a square, a circle, a rectangle ...
- the network of virtual microphones MIC can be centered around the position of the listener U.
- the network of virtual microphones MIC defined for example a perimeter of a few tens of centimeters to a few tens of meters around the listener U.
- the network of virtual microphones MIC comprises at least two virtual microphones, and preferably at least ten virtual microphones. The number of virtual microphones and their arrangement define limits in the quality of reproduction of the area.
- step S3 the position of each speaker HP is determined.
- the area includes a network of speakers comprising at least two loudspeakers.
- the speaker network comprises ten or so HP loudspeakers.
- the HP speakers can be distributed throughout the area so that the entire area is covered by the speakers.
- step S4 a distance between each pair of loudspeaker HP and microphone MIC is calculated. This makes it possible to be able to calculate each of the transfer functions Ftransf, for each pair of loudspeaker HP / microphone MIC, in step S5.
- the loudspeakers (HRI,.,., HR N ) are controlled by a set of weights grouped in the vector is the transpose operator.
- the sound field propagation path between each pair of loudspeaker HP and microphone MIC can be defined by a set of transfer functions G (w, h) assembled in the matrix
- G mi Jpck e ⁇ kRml , with R mi the distance between a speaker and microphone pair, k 4nR mi
- the sound pressure P is determined at the position of the listener U. More precisely, the sound pressure P is determined within the perimeter defined by the network of virtual microphones MIC. Even more precisely, the sound pressure P is determined in each virtual microphone.
- the sound pressure P is the sound pressure from the signals broadcast by the loudspeakers in the area.
- the sound pressure P is determined from the transfer functions Ftransf, calculated in step S5, and from a weight applied to the control signals supplying each loudspeaker.
- the initial weight applied to the control signals of each of the loudspeakers is zero. This corresponds to the weight applied to the first iteration. Then, with each new iteration, the weight applied to the control signals tends to vary, as described below.
- the sound pressure P includes all of the sound pressures determined at each of the positions of the virtual microphones.
- the sound pressure estimated at the position of the listener U is more representative. This makes it possible to obtain a homogeneous result at the end of the process.
- Step S7 makes it possible to define the value of the target sound pressure Pt at the position of the listener U. More precisely, the value of the target sound pressure Pt is initialized at this step. The target sound pressure Pt can be chosen by the designer. It is then transmitted to the TRAIT processing unit in the form of the vector defined above.
- step S8 the error between the target pressure Pt and the estimated pressure P at the position of the listener U is calculated.
- the error may be due to the fact that an adaptation step is applied to me so that the target pressure Pt is not reached immediately.
- the target pressure Pt is reached after a certain number of process iterations. This minimizes the computational resources necessary to reach the target pressure at the position of the listener U. It also ensures the stability of the algorithm.
- the adaptation step m is also chosen so that the error calculated in step S8 has a small value, in order to stabilize the filter.
- step S 12 the forgetting factor g (h) is calculated in order to calculate the weights to be applied to each control signal from the loudspeakers.
- the forget factor g (h) has two roles. On the one hand, it helps to regularize the problem. In other words, it prevents the process from diverging when it is in a stationary state.
- the forget factor g (h) makes it possible to attenuate the weights calculated in the previous iterations. So when the listener moves, previous weights do not affect future weights.
- the forgetting factor g (h) is determined based directly on a possible displacement of the listener. This calculation is illustrated in steps S9 to Sl l.
- step S9 the position of the listener in the previous iterations is retrieved. It is for example possible to retrieve the position of the listener in all the previous iterations. As a variant, it is possible to recover the position of the listener only for part of the previous iterations, for example the last ten or the last hundred iterations.
- a speed of movement of the listener is calculated in step S 10.
- the speed of movement can be calculated in meters per iteration.
- the listener’s speed may be zero.
- step Sl 1 the forgetting factor g (h) is calculated according to the formula:
- the forget factor g is limited between 0 and y max . According to this definition, y, llax therefore corresponds to a maximum percentage of weight to be forgotten between each iteration.
- m The choice of the value of m is variable during iterations. It is chosen such that if there is a displacement of the listener, then the forgetting factor increases. When there is no movement, it decreases. In other words, when the listener's speed is positive, the forget factor increases and when the listener's speed is zero it decreases.
- variable a mainly influences the speed of convergence of the process. In other words, it allows you to choose the number of iterations for which the maximum value y max and / or minimum of the forget factor is reached.
- variables l u and correspond respectively to an up step and a down step of the forgetting factor. They are defined according to the speed of movement of the listener and / or according to a modification of the sound field chosen to be reproduced.
- the rise step l u has a greater value if the preceding weights are to be quickly forgotten during movement (for example in the case where the speed of movement of the listener is high).
- the step of descent has a greater value if the previous weights are completely forgotten at the end of a movement of the listener.
- step S12 the forgetting factor is modified there if necessary, depending on the result of the calculation in step S 11.
- the calculation and modification of the forget factor in step S 12 is used to calculate the weights to be applied to the control signals of the loudspeakers HP. More precisely, at the first iteration, the weights are initialized to zero (step S 13). Each speaker broadcasts an unweighted control signal. Then, at each iteration, the value of the weights varies as a function of the error and the forgetting factor (step S14). The loudspeakers then broadcast a weighted control signal, which may be different with each new iteration.
- This modification of the control signals explains in particular that the acoustic pressure P estimated at the position of the listener U can be different at each iteration.
- m the adaptation step which can vary with each iteration
- g (h) the forgetting factor which can vary.
- step S15 the FILT filters to be applied to the loudspeakers are calculated.
- a speaker filter is calculated. So there can be as many filters as there are speakers.
- To obtain filters in the time domain from the weights calculated in the previous step it is possible to perform a symmetry of the weights calculated in the frequency domain by taking their conjugate complex. Then, an inverse Fourier transform is performed to obtain the filters in the time domain.
- the filters calculated may not respect the principle of causation. A time shift of the filter, corresponding for example to half the length of the filter, can be achieved. Thus, a plurality of filters, for example one filter per speaker, is obtained.
- step S16 the audio signal to be broadcast to the listener is obtained. It is then possible to perform a real-time filtering of the audio signal S (U) in order to broadcast the signal over the loudspeakers.
- the signal S (U) is filtered in step S 17 by the filters calculated in step S15 and broadcast by the loudspeaker corresponding to the filter in steps S18 and S19.
- the FILT filters are calculated as a function of the filtered signals S (HP I , ..., HP n ), weighted at the previous iteration and broadcast by the loudspeakers, as perceived by the microphone network.
- the FILT filters are applied to the signal S (U) to obtain new control signals S (HP I , ..., HP n ) to be broadcast respectively on each speaker of the speaker network.
- step S6 The process is then restarted from step S6 in which the sound pressure in the position of the listener is determined.
- the speaker network HP covers an area comprising a first sub-area SZ1 and a second sub-area SZ2.
- the HP loudspeakers are supplied with respective control signals to each emit an audio signal continuously, with a view to spatialized sound diffusion of a selected sound field.
- the selected sound field is to be made audible in one of the sub-zones, and to be made inaudible in the other sub-zone.
- the selected sound field is audible in the first subzone SZ1.
- the selected sound field is to be made inaudible in the second subzone SZ2.
- the speakers can be defined by their position in the area.
- Each subzone SZ can be defined by the position of the listener U. It is then possible to define, as a function of the geolocation data of the listener, the first subzone SZ1, in which the listener U hears the selected sound field.
- the subzone SZ1 has for example predefined dimensions.
- the first sub-area can correspond to an area of a few tens of centimeters to a few tens of meters, of which the listener U is the center.
- the second subzone SZ2, in which the selected sound field is to be made inaudible can be defined as the complementary subzone.
- the position of the auditor U can define, in the same manner as described above, the second subzone SZ2.
- the first subzone SZ1 is defined as complementary to the second subzone SZ2.
- part of the MIC microphone array covers the first subzone SZ1 while the other part covers the second subzone SZ2.
- Each sub-zone includes at least one virtual microphone.
- the zone is covered by M microphones Ml to MIC M ⁇
- the first sub-zone is covered by microphones MICi to MIC N , with N less than M.
- the second sub-zone is covered by microphones MIC N + I to MIC M ⁇
- sub-zones are defined according to the position of the listener, they change as the listener moves.
- the position of virtual microphones is changing in the same way.
- the first subzone SZ1 is defined by the position al of the listener U (shown in solid lines).
- the MIC microphone array is defined to cover the first subzone SZ1.
- the second subzone SZ2 is complementary to the first subzone SZ1.
- the arrow F illustrates a displacement of the listener LF towards a position a2.
- the first subzone SZ1 is then redefined around the LF listener (in dotted lines).
- the MIC microphone array is redefined to cover the new first subzone SZ1.
- the rest of the area represents the new second SZ2 subzone.
- the first subzone SZ1 initially defined by the position al of the listener is in the second subzone SZ2.
- the TRAIT processing unit receives as input the position of the microphones MIC, the geolocation data of the listener U, the positions of each speaker HP, the audio signal to reproduce S (U) intended for the listener U and the target sound fields Pt l Pt 2 to be reached in each sub-zone. From this data, the processing unit TRAIT calculates the FILT filter to be applied to the signal S (U) in order to restore the target sound fields Pt l Pt 2 in the sub-zones. The TRAIT processing unit also receives the acoustic pressures Pi, P 2 estimated in each of the sub-zones. The TRAIT processing unit outputs the filtered signals S (HPi ... HPN) to be broadcast respectively on the speakers HPi to HPN .
- FIGS 4a and 4b illustrate the main steps of the method according to the invention.
- the process steps are implemented by the TRAIT processing unit continuously and in real time.
- the purpose of the method is to make the selected sound field inaudible in one of the subzones, for example in the second subzone SZ2 while following the movement of a listener whose position defines the subzones .
- the method is based on an estimation of acoustic pressures in each of the sub-zones, so as to apply a desired level of sound contrast between the two sub-zones.
- the audio signal S (U) is filtered as a function of the estimated acoustic pressures and the level of sound contrast in order to obtain the control signals S (HP I ... HP n ) to be broadcast on the loudspeakers.
- step S20 the position of the listener U is determined, for example by means of a position sensor POS. From this position, the two subzones SZ1, SZ2 are defined.
- the first sub-zone corresponds to the position of the listener U.
- the first sub-zone SZ1 is for example defined as being an area of a few tens of centimeters to a few tens of meters in circumference, of which the first listener Ul is the center.
- the second subzone SZ2 can be defined as being complementary to the first subzone SZ1.
- the second subzone SZ2 which is defined by the position of the listener, the first subzone SZ1 being complementary to the second subzone SZ2.
- step S21 the network of microphones MIC is defined, at least one microphone covering each of the subzones SZ1, SZ2.
- step S22 the position of each speaker HP is determined, as described above with reference to Figures 2a and 2b.
- step S23 a distance between each pair of loudspeaker HP and microphone MIC is calculated. This makes it possible to be able to calculate each of the transfer functions Ftransf, for each speaker pair HP / microphone MIC, in step S4. More specifically, the target sound field can be defined as a vector
- the exponent T is the transposition operator.
- the sound field propagation path between each pair of loudspeaker HP and microphone MIC can be defined by a set of transfer functions G (w, h) assembled in the matrix
- G mi with R mi the distance between a speaker and microphone pair
- step S25 the acoustic pressures P x and P 2 are determined respectively in the first subzone SZ1 and in the second subzone SZ2.
- the acoustic pressure P x in the first subzone SZ1 can be the acoustic pressure originating from the signals broadcast by the loudspeakers in the first subzone.
- the sound pressure P 2 in the second sub-area, in which the sound signals are to be made inaudible, may correspond to the induced sound pressure resulting from the signals broadcast by the loudspeakers supplied by the control signals associated with the pressure P x induced in the first subzone.
- the acoustic pressures P l 5 P 2 are determined from the transfer functions Ftransf calculated in step S24, and from an initial weight applied to the control signals of each loudspeaker. The initial weight applied to the control signals of each of the loudspeakers is zero. Then, the weight applied to the control signals tends to vary with each iteration, as described below.
- the acoustic pressures P l 5 P 2 each include all of the acoustic pressures determined at each of the positions of the virtual microphones.
- the sound pressure estimated in the sub-zones is more representative. This makes it possible to obtain a homogeneous result at the end of the process.
- a sound pressure determined in a single position P 2 , P 2 is respectively estimated for the first subzone SZ1 and for the second subzone SZ2. This makes it possible to limit the number of calculations, and therefore to decrease the processing time and consequently the reactivity of the system.
- step S26 the sound levels Li and L 2 are determined respectively in the first subzone SZ1 and in the second subzone SZ2.
- the sound levels Li and L 2 are determined at each position of the MIC microphones.
- This step converts the estimated sound pressure values P l 5 P 2 into measurable values in decibels. In this way, the sound contrast between the first and second subzones can be calculated.
- step S27 a desired sound contrast level C c between the first sub-area and the second sub-area is defined.
- the desired sound contrast C c between the first subzone SZ1 and the second subzone SZ2 is previously defined by a designer as a function of the selected sound field and / or the perception of a listener U.
- the average sound level in a sub-area can be defined as:
- This coefficient is determined by the amplitude of the sound pressure to be given to each microphone so that the sound level in the second sub-area is homogeneous.
- the principle is therefore to use the pressure field present in the second sub-zone which is induced by the sound pressure in the first sub-zone, then to attenuate or to amplify the individual values of estimated sound pressures. in each microphone, so that they correspond to the target sound field in the second sub-area on all of the microphones.
- x [x 1 ,. . . , x hi , ... , x M ] t .
- the attenuation coefficients are calculated so as to meet the contrast criterion defined by the designer.
- the attenuation coefficient is defined so that the difference between the sound contrast between the two subzones SZ2 and the desired sound contrast Ce is close to zero.
- Step S30 to S32 make it possible to define the value of the target acoustic pressures Pt- L , Pt 2 in the first and second subzones SZ1, SZ2.
- Step S30 includes the initialization of the target acoustic pressures Pt ⁇ Pt 2 , respectively in the first and second subzones SZ1, SZ2.
- the target acoustic pressures Pt 1 5 Pt 2 characterize the target sound field to be diffused in the sub-zones.
- the target acoustic pressure Pt x in the first subzone SZ1 is defined as being a target pressure Pt 15 chosen by the designer.
- the target pressure Pt- L in the first subzone SZ1 is greater than zero, so that the target sound field is audible in this first subzone.
- the target sound pressure Pt 2 in the second subzone is initialized to zero.
- the target pressures Pt- L , Pt 2 are then transmitted to the processing unit TRAIT in step S31, in the form of a vector Pt.
- the target pressure Pt 2 in the second sub-zone takes the value of the attenuated sound pressure P 2 (step S29). This allows, at each iteration, to redefine the target sound field to be reproduced in the second sub-zone, taking into account the perception of the listener and the control signals from the speakers.
- the target acoustic pressure Pt 2 of the second subzone is equal to zero only during the first iteration. In fact, as soon as the loudspeakers broadcast a signal, a sound field is perceived in the first sub-zone, but also in the second sub-zone.
- the target pressure Pt 2 in the second sub-area is calculated as follows.
- step S33 the error between the target pressure Pt 2 and the estimated pressure P 2 in the second sub-area is calculated.
- the error is due to the fact that an adaptation step m is applied so that the target pressure Pt 2 is not immediately reached.
- the target pressure Pt 2 is reached after a certain number of iterations of the process. This minimizes the computing resources necessary to reach the target pressure Pt 2 in the second subzone SZ2. This also ensures the stability of the algorithm.
- the adaptation step m is also chosen so that the error calculated in step S33 has a small value, in order to stabilize the filter.
- the forget factor g (h) is then calculated to calculate the weights to be applied to each speaker control signal.
- the forgetting factor g (h) helps to regularize the problem and reduce the weights calculated in previous iterations. So when the listener moves, previous weights do not affect future weights.
- the forget factor g (h) is determined based directly on a possible displacement of the listener. This calculation is illustrated in steps S34 to S36.
- step S34 the position of the listener in the previous iterations is recovered. It is for example possible to recover the position of the listener in all the previous iterations. As a variant, it is possible to recover the position of the listener only for part of the previous iterations, for example the last ten or the last hundred iterations.
- a speed of movement of the listener is calculated in step S35. Travel speed can be calculated in meters per iteration. The listener's speed may be zero.
- step S36 the forgetting factor g (h) is calculated according to the formula described above:
- step S33 the forget factor g (h) is modified if necessary, depending on the result of the calculation of step S36.
- step S37 The calculation and modification of the forget factor in step S37 is used to calculate the weights to be applied to the control signals of the loudspeakers. More specifically, at the first iteration the weights are initialized to zero (step S38). Each speaker broadcasts an unweighted control signal. Then, at each iteration, the value of the weights varies according to the error and the forgetting factor (step S39). The loudspeakers then broadcast the weighted control signal.
- q (n + 1) q (n) (l— mg (h) ' ) + / rG H (n) (G (n) q (n) - Pt (n)).
- the FILT filters to apply to the speakers are then determined in step S40. For example, an HP speaker filter is calculated. So there can be as many filters as there are speakers.
- the type of filters applied to each speaker includes, for example, an inverse Fourier transform.
- Step S41 is an initialization step, implemented only at the first iteration of the method.
- the audio signal to be reproduced S (U) is intended respectively for the listener U.
- the filters FILT are applied to the signal S (U), in order to obtain N control signals S (HPi, ..., HP n ) filtered to be broadcast respectively by the loudspeakers (HRi, ..,, HR N ) in step S43.
- the control signals S (HP I , ..., HP n ) are broadcast respectively by each speaker (HP I , ..., HP n ) of the speaker network in step S44.
- the HP loudspeakers broadcast the control signals continuously.
- the FILT filters are calculated as a function of the signals S (HP I , ..., HP n ) filtered at the previous iteration and broadcast by the loudspeakers, as perceived by the network microphones.
- FILT filters are applied to the S (U) signal to obtain new control signals S (HPi, ..., HP N ) to be broadcast respectively on each speaker of the speaker network.
- step S35 The method is then restarted from step S35 in which the acoustic pressures P l 5 P 2 of the two subzones SZ1, SZ2 are estimated.
- the method can be implemented for a plurality of listeners Ui to UN ⁇
- an audio signal S (Ui, U N ) can be provided respectively for each listener.
- the steps of the method can be implemented for each of the listeners, so that the chosen sound field of each listener is restored to their position, and taking into account their movements.
- a plurality of forgetting factors can be calculated for each of the listeners.
- the selected sound field is a first sound field, at least a second chosen sound field being broadcast by the speaker network HP.
- the second selected sound field is audible in the second sub-zone for a second listener and is to be made inaudible in the first sub-zone for a first listener.
- the loudspeakers are powered by the first control signals to each emit a continuous audio signal corresponding to the first sound field chosen, and are also powered by second control signals to each emit a continuous audio signal corresponding to the second sound field selected.
- the process steps as described above can be applied to the first subzone SZ1, so that the second selected sound field is made inaudible in the first subzone SZ1 taking into account the movements of the two listeners.
- the first and second sub-areas are not complementary.
- a first sub-zone can be defined with respect to a first listener Ul and a second sub-zone can be defined with respect to a second listener U2.
- the sound field must be made audible in the first sub-zone and inaudible in the second sub-zone.
- the sound field in the rest of the area may not be controlled.
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Otolaryngology (AREA)
- General Health & Medical Sciences (AREA)
- Stereophonic System (AREA)
- Circuit For Audible Band Transducer (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| FR1857774A FR3085572A1 (fr) | 2018-08-29 | 2018-08-29 | Procede pour une restitution sonore spatialisee d'un champ sonore audible en une position d'un auditeur se deplacant et systeme mettant en oeuvre un tel procede |
| PCT/FR2019/051952 WO2020043979A1 (fr) | 2018-08-29 | 2019-08-22 | Procédé pour une restitution sonore spatialisée d'un champ sonore audible en une position d'un auditeur se déplaçant et système mettant en œuvre un tel procédé |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| EP3844981A1 true EP3844981A1 (fr) | 2021-07-07 |
| EP3844981B1 EP3844981B1 (fr) | 2023-09-27 |
Family
ID=65951625
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| EP19778569.4A Active EP3844981B1 (fr) | 2018-08-29 | 2019-08-22 | Procédé pour une restitution sonore spatialisée d'un champ sonore audible en une position d'un auditeur se déplaçant et système mettant en ouvre un tel procédé |
Country Status (5)
| Country | Link |
|---|---|
| US (1) | US11432100B2 (fr) |
| EP (1) | EP3844981B1 (fr) |
| CN (1) | CN112840679B (fr) |
| FR (1) | FR3085572A1 (fr) |
| WO (1) | WO2020043979A1 (fr) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11417351B2 (en) * | 2018-06-26 | 2022-08-16 | Google Llc | Multi-channel echo cancellation with scenario memory |
| CN114199368B (zh) * | 2021-11-30 | 2024-04-26 | 北京工商大学 | 一种全频带pp声强自动测量装置和测量方法 |
| CN115701142A (zh) * | 2022-10-21 | 2023-02-07 | 苏州触达信息技术有限公司 | 一种超声防丢追踪器指向信息的校正方法 |
| CN116489573A (zh) * | 2022-12-21 | 2023-07-25 | 瑞声科技(南京)有限公司 | 一种声场控制方法、装置、设备及可读存储介质 |
| JP7681700B2 (ja) * | 2022-12-21 | 2025-05-22 | エーエーシー テクノロジーズ (ナンジン) カンパニーリミテッド | 音場制御方法、装置、デバイスおよびコンピュータ読み取り可能な記録媒体 |
| EP4521778A1 (fr) * | 2023-09-07 | 2025-03-12 | Ask Industries Societa' per Azioni | Système et procédé de création de zones d'écoute individuelles dans un environnement, en particulier à l'intérieur d'un véhicule automobile, et véhicule automobile comprenant un tel système |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AUPR647501A0 (en) | 2001-07-19 | 2001-08-09 | Vast Audio Pty Ltd | Recording a three dimensional auditory scene and reproducing it for the individual listener |
| EP2056627A1 (fr) * | 2007-10-30 | 2009-05-06 | SonicEmotion AG | Procédé et dispositif pour améliorer la précision de rendu de champ sonore dans une région d'écoute préférée |
| GB2457508B (en) * | 2008-02-18 | 2010-06-09 | Ltd Sony Computer Entertainmen | System and method of audio adaptaton |
| WO2012068174A2 (fr) * | 2010-11-15 | 2012-05-24 | The Regents Of The University Of California | Procédé de commande d'un réseau de haut-parleurs permettant de produire un son d'ambiance virtuel binaural spatialisé localisé |
| US9031268B2 (en) * | 2011-05-09 | 2015-05-12 | Dts, Inc. | Room characterization and correction for multi-channel audio |
| WO2013149867A1 (fr) | 2012-04-02 | 2013-10-10 | Sonicemotion Ag | Procédé pour reproduction efficace de son 3d haute qualité |
| WO2014036121A1 (fr) * | 2012-08-31 | 2014-03-06 | Dolby Laboratories Licensing Corporation | Système conçu pour le rendu et la lecture d'un son basé sur un objet dans divers environnements d'écoute |
| JP2015206989A (ja) * | 2014-04-23 | 2015-11-19 | ソニー株式会社 | 情報処理装置、情報処理方法及びプログラム |
| EP3354043B1 (fr) * | 2015-10-14 | 2021-05-26 | Huawei Technologies Co., Ltd. | Système de suppression adaptative d'échos |
| US10979843B2 (en) * | 2016-04-08 | 2021-04-13 | Qualcomm Incorporated | Spatialized audio output based on predicted position data |
-
2018
- 2018-08-29 FR FR1857774A patent/FR3085572A1/fr not_active Ceased
-
2019
- 2019-08-22 US US17/270,528 patent/US11432100B2/en active Active
- 2019-08-22 CN CN201980065289.6A patent/CN112840679B/zh active Active
- 2019-08-22 WO PCT/FR2019/051952 patent/WO2020043979A1/fr not_active Ceased
- 2019-08-22 EP EP19778569.4A patent/EP3844981B1/fr active Active
Also Published As
| Publication number | Publication date |
|---|---|
| US11432100B2 (en) | 2022-08-30 |
| CN112840679B (zh) | 2022-07-12 |
| CN112840679A (zh) | 2021-05-25 |
| US20210360363A1 (en) | 2021-11-18 |
| EP3844981B1 (fr) | 2023-09-27 |
| WO2020043979A1 (fr) | 2020-03-05 |
| FR3085572A1 (fr) | 2020-03-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3844981B1 (fr) | Procédé pour une restitution sonore spatialisée d'un champ sonore audible en une position d'un auditeur se déplaçant et système mettant en ouvre un tel procédé | |
| WO2014044948A1 (fr) | Calibration optimisee d'un systeme de restitution sonore multi haut-parleurs | |
| EP2518724A1 (fr) | Combiné audio micro/casque comprenant des moyens de débruitage d'un signal de parole proche, notamment pour un système de téléphonie "mains libres" | |
| FR2473233A1 (fr) | Systeme d'egalisation perfectionne | |
| EP1836876A2 (fr) | Procédé et dispositif d'individualisation de hrtfs par modélisation | |
| EP3815395B1 (fr) | Procédé pour une restitution sonore spatialisée d'un champ sonore audible sélectivement dans une sous-zone d'une zone | |
| EP3842923B1 (fr) | Enceinte connectée comprenant une interface lan et une interface wpan | |
| EP4302114B1 (fr) | Procede de determination d'une direction de propagation d'une source sonore par creation de signaux sinusoidaux a partir des signaux sonores recus par des microphones | |
| CA2165512C (fr) | Dispositif de prise de sons comprenant un systeme video pour le reglage de parametres et procede de reglage | |
| FR2550903A1 (fr) | Procede et dispositif de commande et de regulation de chaine electroacoustique | |
| FR3107982A1 (fr) | Procédé et système de détermination de filtres d’égalisation sonore d’un système audio | |
| FR3052319A1 (fr) | Combine audio micro/casque comprenant des moyens de detection d'activite vocale multiples a classifieur supervise. | |
| FR2967848A1 (fr) | Systeme de correction de spectre destine notamment a une salle de spectacle | |
| CA2974156C (fr) | Amplificateur a reglage de niveau sonore automatique | |
| FR3157767A1 (fr) | Traitement de réduction de canaux d’un signal audio stéréophonique par remise en phase optimisée | |
| FR2957185A1 (fr) | Procede de determination d'un seuil a appliquer a un signal sonore, procede d'attenuation de bruit, dispositif et programme d'ordinateur associes | |
| FR3051959A1 (fr) | Procede et dispositif pour estimer un signal dereverbere | |
| FR2963844A1 (fr) | Procede de determination de parametres definissant des filtres applicables a des haut-parleurs, dispositif et programme associes | |
| EP3900402A1 (fr) | Calibration d'un système de restitution sonore distribué | |
| FR2943867A1 (fr) | Traitement d'egalisation de composantes spatiales d'un signal audio 3d | |
| FR3158605A1 (fr) | Procédé de contrôle MIMO amélioré ; Système de contrôle et programme d’ordinateur associés. | |
| WO2025132526A1 (fr) | Transition entre deux modes de traitement de réduction de canaux avec contrôle de niveau optimisé | |
| FR3138750A1 (fr) | Procédé de calibrage d’un dispositif audio nomade, système de calibrage d’un dispositif audio nomade et produit programme d’ordinateur associés | |
| WO2026032789A1 (fr) | Methode d'optimisation de la reponse en frequence d'au moins deux enceintes placees dans un environnement specifique | |
| WO2024213556A1 (fr) | Traitement optimisé de réduction de canaux d'un signal audio stéréophonique |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
| PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
| 17P | Request for examination filed |
Effective date: 20210216 |
|
| AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
| RAP3 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: ORANGE |
|
| DAV | Request for validation of the european patent (deleted) | ||
| DAX | Request for extension of the european patent (deleted) | ||
| GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
| INTG | Intention to grant announced |
Effective date: 20230404 |
|
| GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
| GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
| AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
| REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH |
|
| REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
| REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602019038275 Country of ref document: DE |
|
| REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: FRENCH |
|
| REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG9D |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231228 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230927 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230927 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231227 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230927 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230927 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230927 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231228 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230927 |
|
| REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20230927 |
|
| REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 1616558 Country of ref document: AT Kind code of ref document: T Effective date: 20230927 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230927 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240127 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230927 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230927 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230927 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230927 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240127 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230927 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230927 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230927 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230927 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240129 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230927 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230927 Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230927 |
|
| REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602019038275 Country of ref document: DE |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230927 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230927 |
|
| PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
| 26N | No opposition filed |
Effective date: 20240628 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230927 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230927 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230927 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230927 |
|
| REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20240822 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230927 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20240831 |
|
| REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20240831 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20240831 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20240822 |
|
| PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20250724 Year of fee payment: 7 |
|
| PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20250725 Year of fee payment: 7 |
|
| PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20250725 Year of fee payment: 7 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20190822 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20190822 |