EP3192073B1 - Unterscheidung und dämpfung von vorechos in einem digitalen audiosignal - Google Patents

Unterscheidung und dämpfung von vorechos in einem digitalen audiosignal Download PDF

Info

Publication number
EP3192073B1
EP3192073B1 EP15771686.1A EP15771686A EP3192073B1 EP 3192073 B1 EP3192073 B1 EP 3192073B1 EP 15771686 A EP15771686 A EP 15771686A EP 3192073 B1 EP3192073 B1 EP 3192073B1
Authority
EP
European Patent Office
Prior art keywords
sub
echo
block
signal
attenuation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP15771686.1A
Other languages
English (en)
French (fr)
Other versions
EP3192073A1 (de
Inventor
Balazs Kovesi
Stéphane RAGOT
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Orange SA
Original Assignee
Orange SA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Orange SA filed Critical Orange SA
Publication of EP3192073A1 publication Critical patent/EP3192073A1/de
Application granted granted Critical
Publication of EP3192073B1 publication Critical patent/EP3192073B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
    • G10L25/21—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters the extracted parameters being power information
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • G10L19/0204—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using subband decomposition
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
    • G10L19/26—Pre-filtering or post-filtering
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination

Definitions

  • the invention relates to a method and a device for discriminating and processing pre-echo attenuation when decoding a digital audio signal.
  • compression processes for the transmission of digital audio signals over telecommunications networks, whether for example fixed or mobile networks, or for the storage of signals, compression processes (or source coding) using coding systems which are generally of the linear coding type by linear prediction or by transform frequency coding.
  • the method and the device which are the subject of the invention, thus have as their field of application the compression of sound signals, in particular frequency-coded digital audio signals.
  • the figure 1 represents for illustrative purposes, a block diagram of the coding and decoding of a digital audio signal by transform including an addition / overlap synthesis analysis according to the prior art.
  • Certain musical sequences such as percussion and certain segments of speech like the plosives (/ k /, / t /, ...), are characterized by extremely sudden attacks which result in very fast transitions and a very strong variation signal dynamics in a few samples.
  • An example of a transition is given to the figure 1 from sample 410.
  • the input signal is cut into blocks of samples of length L whose boundaries are represented on the figure 1 by dotted vertical lines.
  • the input signal is denoted x ( n ), where n is the index of the sample.
  • L 160 samples.
  • the division in blocks, also called frames, operated by the transform coding is totally independent of the sound signal and the transitions can therefore appear at any point in the analysis window. But after transform decoding, the reconstructed signal is tainted by "noise” (or distortion) generated by the quantization (Q) -quantization inverse (Q -1 ) operation.
  • This coding noise is temporally distributed relatively uniform throughout the temporal support of the transformed block, that is to say over the entire length of the window length 2L samples (with overlap of L samples).
  • the energy of the coding noise is generally proportional to the energy of the block and is a function of the coding / decoding rate.
  • the level of the coding noise is typically lower than that of the signal for the high energy segments that immediately follow the transition, but the level is higher than that of the signal for the lower energy segments, especially on the part preceding the transition (samples 160 - 410 of the figure 1 ).
  • the signal-to-noise ratio is negative and the resulting degradation can appear very troublesome to listen.
  • Pre-echo is the coding noise prior to the transition and post-echo the noise after the transition.
  • the human ear also performs a post-masking of a longer duration, from 5 to 60 milliseconds, during the passage of high energy sequences to low energy sequences.
  • the rate or level of inconvenience acceptable for post-echoes is therefore greater than for pre-echoes.
  • MPEG AAC Advanced Audio Coding
  • MPEG AAC Advanced Audio Coding
  • Transform encoders used for conversational applications such as ITU-T G.722.1, G.722.1C or G.719, often use a 20 ms frame length and a 40 ms window at 16, 32 or 48 kHz (respectively). It should be noted that the ITU-T G.719 encoder incorporates a window switch mechanism with transient detection, however the pre-echo is not completely reduced at low bit rate (typically at 32 kbit / s).
  • Window switching has been mentioned previously; it requires transmitting auxiliary information to identify the type of windows used in the current frame.
  • Another solution is to apply adaptive filtering. In the area preceding the attack, the reconstructed signal is seen as the sum of the original signal and the quantization noise.
  • the aforementioned filtering process does not allow to find the original signal, but provides a strong reduction of pre-echoes. However, it requires transmitting the additional parameters to the decoder.
  • Other definitions of the factor g ( k ) are possible, for example as a function of the energy En ( k ) in the current sub-block and the energy En ( k -1) in the preceding sub-block.
  • the factor g ( k ) is set to an attenuation-inhibiting attenuation value, i.e. 1. Otherwise, the attenuation factor is between 0 and 1.
  • the frame that precedes the pre-echo frame has a homogeneous energy that corresponds to the energy of a low energy segment (typically a background noise).
  • a background noise typically a background noise
  • the attenuation factors (or gains) g ( k ) determined by sub-blocks can then be smoothed by a sample-by-sample applied smoothing function to avoid abrupt changes in the attenuation factor at the block boundaries.
  • L ' represents the length of a sub-block.
  • g pre (-1) is the last attenuation factor obtained for the last sample of the previous sub-block
  • the signal is sampled at 32 kHz
  • part a) of the figure 2 a frame of an original signal sampled at 32 kHz is shown.
  • An attack (or transition) in the signal is located in the sub-block beginning at the index 320.
  • This signal has been coded by a low rate (24 kbit / s) MDCT type transform coder.
  • part (b) of the figure 2 the result of the decoding without pre-echo processing is illustrated.
  • the pre-echo can be observed from sample 160, in the preceding sub-blocks the one containing the attack.
  • Part c) shows the evolution of the pre-echo attenuation factor (solid line) obtained by the method described in the aforementioned prior art patent application.
  • the dotted line represents the factor before smoothing. Note here that the position of the attack is estimated around the sample 380 (in the block delimited by the samples 320 and 400).
  • Part d) illustrates the result of the decoding after application of pre-echo processing (multiplication of signal b) with signal c)).
  • pre-echo has been attenuated.
  • the figure 2 also shows that the smoothed factor does not go back to 1 at the moment of the attack, which implies a decrease in the amplitude of the attack. The noticeable impact of this decrease is very small but can nevertheless be avoided.
  • the figure 3 illustrates the same example as the figure 2 , in which, before smoothing, the attenuation factor value is forced to 1 for the few samples of the sub-block preceding the sub-block where the attack is located. Part (c) of the figure 3 give an example of such a correction.
  • the value of factor 1 has been assigned to the last 16 samples of the sub-block preceding the attack, starting from the index 364.
  • the smoothing function progressively increases the factor to have a value close to 1 at the moment. of the attack.
  • the amplitude of the attack is then preserved, as illustrated in part d) of the figure 3 however, some pre-echo samples are not attenuated.
  • Attenuation pre-echo reduction does not reduce the pre-echo to the attack level because of gain smoothing.
  • This pre-echo reduction technique is however perfectible for certain types of signals such as modern music signals, for example. Indeed, in some cases, a false pre-echo detection can take place.
  • the figure 4 illustrates an example of such an original signal, not coded so without pre-echo. This is a beat of an electronic / synthetic percussion instrument. It can be observed that before the net attack to the index 1600 there is a synthetic noise which starts towards the index 1250. This synthetic noise which is therefore part of the signal would be detected as a pre-echo by the algorithm pre-echo detection described above, assuming perfect signal coding / decoding. The pre-echo attenuation processing would therefore suppress this component of the signal. This would distort the decoded signal (when the coding / decoding is perfect), which is undesirable.
  • the present invention improves the state of the art.
  • the energy director coefficient calculated for the sub-blocks preceding the position of the attack makes it possible to check the tendency of increase of the energy of the signal in the pre-echo zone. This makes reliable detection of pre-echoes by avoiding false detection of pre-echoes.
  • the pre-echo has a typical characteristic: its energy has a growing tendency in approaching the original pre-echo attack.
  • the shape of the weighting windows of the addition-overlap explain this. Even if the pre-echo has a nearly constant energy before the overlap-addition, the signals at the input of the add-over module are multiplied by weighting windows whose weight decreases towards the past.
  • the energy of the signal before the attack is approximately constant which makes it possible to differentiate it from a pre-echo.
  • verification of increasing signal energy in the pre-echo area increases the reliability of the pre-echo detection.
  • the method further comprises a step of decomposing the digital audio signal into at least two sub-signals according to a frequency criterion and in that the comparison calculation steps are performed for at least one of the subsignals.
  • the energy of two sub-blocks is used in the pre-echo zone to calculate a directional coefficient and compare it to a threshold. With only two points, only the verification for the high-frequency sub-signal in the case of two sub-signal decomposition is sufficient to detect a false pre-echo detection.
  • the method further comprises a step of decomposing the digital audio signal into at least two sub-signals as a function of a frequency criterion and in that the calculation and comparison steps are performed for each of the sub-signals, the inhibition of the pre-echo attenuation processing in the pre-echo zone of all the sub-signals performing when a calculated master coefficient is below the predefined threshold for at least one sub-signal.
  • the division into sub-signals thus makes it possible to carry out a pre-echo attenuation independently and adapted in the sub-signals.
  • the detection reliability of the pre-echo zone is enhanced for each of the sub-signals by checking the value of the respective coefficient coefficients.
  • a different threshold is defined by sub-signal.
  • the steering coefficient is calculated using a least squares estimation method.
  • This calculation method is of low complexity.
  • the steering coefficient is normalized.
  • a direction coefficient calculated for the previous frame is used for the comparison step.
  • the invention relates to a decoder of a digital audio signal comprising a device as described above.
  • the invention also relates to a computer program comprising code instructions for implementing the steps of the method as described above, when these instructions are executed by a processor.
  • the invention relates to a storage medium, readable by a processor, integrated or not to the processing device, optionally removable, storing a computer program implementing a method of treatment as described above.
  • a discrimination device 600 and pre-echo reduction processing is described.
  • the attenuation processing device 600 as described below is included in a decoder comprising a reverse quantization module 610 (Q -1 ) receiving a signal S, a reverse transformation module 620 (MDCT -1 ), a module 630 of addition / recap signal reconstruction as described with reference to FIG. figure 1 and delivering a reconstructed signal x rec ( n ) to the attenuation discrimination and processing device according to the invention.
  • a decoder comprising a reverse quantization module 610 (Q -1 ) receiving a signal S, a reverse transformation module 620 (MDCT -1 ), a module 630 of addition / recap signal reconstruction as described with reference to FIG. figure 1 and delivering a reconstructed signal x rec ( n ) to the attenuation discrimination and processing device according to the invention.
  • MDCT transform which is the most common in speech and audio coding
  • the device 600 also applies to any other type of transform (F
  • a processed signal Sa is provided in which a pre-echo attenuation has been performed.
  • the device 600 implements a discrimination and pre-echo attenuation processing method in the decoded signal x rec ( n ) .
  • the discrimination and attenuation processing method includes a step of detecting (E601) attacks that may generate a pre-echo, in the decoded signal x rec ( n ) .
  • the device 600 comprises a detection module 601 able to implement a step of detecting (E601) the position of an attack in a decoded audio signal.
  • An onset is a rapid transition and a sudden change in the dynamics (or amplitude) of the signal.
  • This type of signal may be referred to by the more general term "transient”.
  • transient we will use only the terms of attack or transition to designate also transients.
  • L 640 samples (20 ms) at 32 kHz
  • L' 80 samples (2.5 ms)
  • K 8.
  • the size of these sub-blocks is therefore identical but the invention remains valid and easily generalizable when the sub-blocks have a variable size. This can be the case for example when the length of the frame L is not divisible by the number of sub-blocks K or if the frame length is variable.
  • Special low-delay analysis-synthesis windows similar to those described in ITU-T G.718 are used for the analysis part and for the synthesis part of the MDCT transformation.
  • An example of such windows is illustrated with reference to the figure 6 .
  • the delay generated by the transformation is only 280 samples in contrast to the delay of 640 samples in the case of use of conventional sinusoidal windows.
  • the MDCT memory with special low-delay analysis-synthesis windows contains only 140 independent samples (not folded with the current frame) unlike the 320 samples in the case of using conventional sinusoidal windows.
  • the MDCT x MDCT ( n ) memory is used which gives a time-folding version of the future signal.
  • L m (0) 140
  • the figure 1 shows that the pre-echo influences the frame that precedes the frame where the attack is located, and it is desirable to detect an attack in the future frame which is partly contained in the MDCT memory.
  • the current frame and the MDCT memory can be seen as concatenated signals forming a signal cut into (K + K ') consecutive sub-blocks.
  • Other pre-echo detection criteria are possible without changing the nature of the invention.
  • the device 600 also comprises a pre-echo zone discrimination module 602 implementing a step of determining (E602) a pre-echo zone (ZPE) preceding the detected driving position.
  • Pre-echo zone is here referred to as the zone covering the samples before the estimated position of the attack which are disturbed by the pre-echo generated by the attack and where attenuation of this pre-echo is desirable.
  • the pre-echo area can be determined on the decoded signal.
  • the energies En ( k ) are concatenated in chronological order, with the time envelope of the decoded signal first, then the envelope of the signal of the following estimated frame from the memory of the MDCT transform. According to this concatenated temporal envelope and mean energies In and In of the previous frame, the presence of pre-echo is detected for example if the ratio R ( k ) exceeds a threshold, typically this threshold is 16.
  • the device 600 comprises a calculation module 603 capable of implementing a step of calculating a director coefficient (or variation trend indicator) of the energies of the sub-blocks preceding the sub-block in which an attack has been detected.
  • the guideline gives information on the (mean) trend of energy change.
  • a positive directing coefficient signals an increase in energies.
  • a value close to 0 indicates constant energy.
  • b 1 ⁇ t i - t ⁇ e i - e ⁇ ⁇ t i - t ⁇ 2
  • This alternative solution has a higher computational complexity because it requires calculating a square root.
  • the steering coefficient is calculated with at most 3 sub-blocks. This makes it possible to limit the maximum complexity of the calculation of the steering coefficient.
  • the normalized standard coefficient b 1n thus obtained is then compared with the step E604 by a comparator module 604 at a predefined threshold.
  • the threshold may be predefined to a fixed value or may be variable depending for example on the classification of the signal according to a speech or music criterion. Typically this threshold is equal to 0 if we only check that the energy does not decrease or equal to 0.2 if we impose a slight increase in energy in the pre-echo zone. If the standardized guideline b 1n is below this threshold it is concluded that the signal in the pre-echo zone does not correspond to a typical pre-echo and the pre-echo attenuation in this zone at step E602 is inhibited. Thus, it is avoided that a decoded signal whose original input signal contains a low energy component before an attack is erroneously modified / altered by the pre-echo attenuation module by detecting that component as a pre-echo.
  • a pre-echo attenuation is implemented in step E607 by the attenuation module 607 for the discriminated pre-echo area.
  • the attenuation factor is for example calculated as in the request FR 08 56248 .
  • the attenuation factor can be forced to 1 thus inhibiting the attenuation or the discrimination module 602 does not discriminate this zone as a pre-echo zone. echo, the attenuation module is not requested.
  • the device 600 further comprises a signal decomposition module 605, able to perform a step E605 of decomposing the decoded signal into at least two sub-signals according to a predetermined criterion.
  • a signal decomposition module 605 able to perform a step E605 of decomposing the decoded signal into at least two sub-signals according to a predetermined criterion. This method is notably described in the application FR12 62598 which we recall here some elements.
  • a constant value c ( n ) 0.25 is used.
  • the combination of the attenuated sub-signals for obtaining the attenuated signal Sa is made by simply adding the attenuated sub-signals to the step E608 described later.
  • a step E606 for calculating pre-echo attenuation factors is implemented in the calculation module 606. This calculation is done separately for the two sub-signals.
  • Attenuation factors are obtained by sampling the pre-echo zone determined in E602 as a function of the frame in which the attack was detected and the previous frame.
  • the attenuation factors are calculated by sub-block. In the method described here, they are additionally calculated separately for each sub signal. For samples preceding the detected attack, the attenuation factors g pre, ss 1 '( n ) and g pre, ss 2 ' ( n ) are thus calculated. Then these attenuation values are optionally smoothed to obtain the attenuation values per sample.
  • g pre, ss 2 '( n ) The calculation of the attenuation factor of a sub-signal (for example g pre, ss 2 '( n )) can be similar to that described in the patent application.
  • R ( k ) also used for the detection of the attack
  • the factor is then set to a attenuation value that inhibits the attenuation, that is to say 1. Otherwise, the attenuation factor is between 0 and 1. This initialization can be common for all the sub-signals. .
  • the attenuation values are then refined by sub-signal to be able to adjust the optimal sub-signal attenuation level based on the characteristics of the decoded signal.
  • the attenuations can be limited according to the average energy of the sub-signal of the previous frame because it is not desirable that after the pre-echo attenuation processing, the signal energy becomes less than the average energy per sub-block of the signal preceding the processing zone (typically that of the previous frame or that of the second half of the previous frame).
  • the limit value of lim g, ss 2 ( k ) can be calculated in order to obtain exactly the same energy as the average energy per sub-block of the segment preceding the sub-block to be processed. .
  • the ' - 1 ; k 0 ... , K - 1
  • the attenuations associated with the sub-block samples of the attack are all set to 1 even if the attack is towards the end of this sub-block.
  • the start position of the attack pos is refined in the sub-block of the attack, for example by cutting the sub-block into sub-sub-blocks and observing the evolution of the energy of these sub-sub-blocks.
  • the calculation of the attenuation values based on the sub-signal x rec, ss 1 ( n ) may be similar to the calculation of the attenuation values by based on the decoded signal x rec ( n ) .
  • the attenuation values can be determined based on the decoded signal x rec ( n ) . In the case where the detection of attacks is made on the decoded signal, it is no longer necessary to recalculate energies of the sub-blocks because for this signal the energy values by sub-block are already calculated to detect the attacks.
  • the attenuation factors g pre , ss1 ( n ) and g pre , ss 2 ( n ) determined by sub-blocks can then be smoothed by an applied smoothing function sample by sample to avoid abrupt changes in the attenuation factor at block boundaries. This is particularly important for sub-signals containing low frequency components such as sub-signal x rec, ss 1 ( n ) but not necessary for sub-signals containing only high frequency components such as sub-signal x rec, ss 2 ( n ) .
  • the figure 7 illustrates an example of applying an attenuation gain with smoothing functions represented by the L arrows.
  • This figure illustrates in a), an example of an original signal, in b), the decoded signal without pre-echo attenuation, in c), the attenuation gains for the two sub-signals obtained according to the decomposition step E605 and in d), the decoded signal with pre-echo attenuation of steps E607 and E608 (i.e. after combining the two attenuated sub-signals).
  • the attenuation gain represented in dashed line and corresponding to the gain calculated for the first sub-signal comprising low frequency components comprises smoothing functions as described above.
  • the attenuation gain represented in solid line and calculated for the second sub-signal comprising high frequency components does not include smoothing gain.
  • the signal represented in d) clearly shows that the pre-echo has been effectively attenuated by the attenuation processing implemented.
  • the pre-echo zone (the number of attenuated samples) may therefore be different for the 2 sub-signals processed separately, even if the detection of the attack is made in common on the basis of the decoded signal. .
  • the smoothed attenuation factor does not go back to 1 at the time of the attack, which implies a decrease in the amplitude of the attack. The noticeable impact of this decrease is very small but must nevertheless be avoided.
  • the attenuation factor value can be forced to 1 for the u-1 samples preceding the index pos where the onset of the attack is. This is equivalent to advancing the pos marker of u-1 samples for the sub signal where the smoothing is applied.
  • the smoothing function gradually increases the factor to have a value 1 at the time of the attack. The amplitude of the attack is then preserved.
  • the verification of the increase of the energy of the pre-echo zone according to the invention is carried out for at least one sub-signal or for each of these sub-signals.
  • the comparison threshold used may be different depending on the sub-signals and the number of sub-blocks available before the attack.
  • the normalized steering coefficient b 1n is less than the threshold of this sub-signal, the pre-echo attenuation is inhibited for all the sub-signals.
  • pre-echo processing can be done for example by setting the attenuation factors to 1 or not discriminating the area as a pre-echo zone, the module of pre-echo attenuation processing is not then requested as illustrated by way of example in the embodiment of the figure 5 by the link between block 604 and 602.
  • the attenuation will be inhibited separately for each sub-signal as soon as the normalized steering coefficient b 1n is lower than the threshold of this sub-signal.
  • the inhibition may for example be implemented by setting the attenuation factors to 1 or by not soliciting the pre-echo module for the sub-signal considered.
  • the evolution of the two sub-signals is checked in both sub-signals. energy of the sub-blocks preceding the sub-block where the attack was detected, by linear regression.
  • This verification can be done according to the steps E603 and E604, at any time after the division of the decoded signal into sub-signals (E605) and before the application of the pre-echo attenuation factors (E607). Verification is possible if at least two sub-blocks precede the sub-block where the attack was detected. If the attack is detected in the first or second sub-block verification according to the invention is not possible.
  • the attack is detected in the fourth sub-block or a sub-block of index higher than 4, one checks the evolution of the energy of the last 3 sub-blocks in the pre-echo zone preceding the sub-block block where the attack was detected.
  • the direction coefficient of the high-frequency sub-signal x rec, ss 2 ( n ) is compared with a threshold of value 0.2.
  • E not ss 2 id - 1 - E not ss 2 id - 2 2 E not ss 2 id - 1 + E not ss 2 id - 2 + E not ss 2 id - 3 ⁇ 0.2 is equivalent to E not ss 2 id - 1 - E not ss 2 id - 2 ⁇ 1 7.5 E not ss 2 id - 1 + E not ss 2 id - 2 + E not ss 2 id - 3 thus avoiding a division operation to reduce the complexity and to facilitate the implementation on a DSP (for "Digital Signal Processor") to fixed point arithmetic.
  • DSP Digital Signal Processor
  • the module 607 of the device 600 of the figure 5 implements the pre-echo attenuation step E607 in the pre-echo area of each of the sub-signals by applying to the subsignals of the thus calculated attenuation factors.
  • the pre-echo attenuation is therefore done independently in the sub-signals.
  • the attenuation can be chosen according to the spectral distribution of the pre-echo.
  • the filtering used is not associated with sub-signal decimation operations and the complexity and delay ("lookahead" or future frame) are reduced to a minimum.
  • this device 100 in the sense of the invention typically comprises a ⁇ P processor cooperating with a memory block BM including a storage and / or working memory, and a memory buffer MEM mentioned above as a means for storing all data. necessary for the implementation of the discrimination and attenuation processing method as described with reference to the figure 5 .
  • This device receives as input successive frames of the digital signal Se and delivers the reconstructed signal Sa with pre-echo attenuation in the pre-echo areas discriminated with, if necessary, reconstruction of the attenuated signal by combining attenuated sub-signals.
  • the memory block BM may comprise a computer program comprising the code instructions for implementing the steps of the method according to the invention when these instructions are executed by a ⁇ P processor of the device and in particular the steps of calculating a control coefficient of the energies for at least two sub-blocks preceding the sub-block in which an attack is detected, comparing the steering coefficient to a predefined threshold and inhibiting the pre-echo attenuation processing in the pre-echo area in the case where the calculated coefficient of direction is lower than the predefined threshold.
  • the figure 5 can illustrate the algorithm of such a computer program.
  • This discrimination and attenuation processing device can be independent or integrated in a digital signal decoder.
  • a decoder can be integrated with equipment for storing or transmitting digital audio signals such as communication gateways, communication terminals or servers of a communication network.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Cable Transmission Systems, Equalization Of Radio And Reduction Of Echo (AREA)

Claims (11)

  1. Verfahren zur Unterscheidung und Dämpfung von Vorechos in einem digitalen Audiosignal, das durch eine Transformationscodierung erzeugt wird, wobei bei der Decodierung für einen aktuellen Rahmen, der in Subblöcke zerlegt wird, die Subblöcke mit niedriger Energie, die einem Subblock vorausgehen, in dem ein Übergang oder ein Angriff detektiert wird (E601), eine Vorechozone bestimmen (E602), in der eine Behandlung zur Dämpfung von Vorechos durchgeführt wird (E607),
    wobei das Verfahren dadurch gekennzeichnet ist, dass in dem Fall, wo ein Angriff aus dem dritten Subblock des aktuellen Rahmens detektiert wird, dieses die folgenden Schritte umfasst:
    - Berechnen (E603) eines Energierichtungskoeffizienten für mindestens zwei Subblöcke des aktuellen Rahmens, die dem Subblock vorausgehen, in dem der Angriff detektiert wird;
    - Vergleichen (E604) des Richtungskoeffizienten mit einer vordefinierten Schwelle; und
    - Inhibieren (E602) der Behandlung zur Dämpfung von Vorechos in der Vorechozone in dem Fall, wo der berechnete Richtungskoeffizient kleiner ist als die vordefinierte Schwelle.
  2. Verfahren nach Anspruch 1,
    dadurch gekennzeichnet, dass dieses ferner einen Schritt des Zerlegens des digitalen Audiosignals in mindestens zwei Subsignale als Funktion eines Frequenzkriteriums umfasst, und dadurch, dass die Schritte des Berechnens und Vergleichens für mindestens eines der Subsignale durchgeführt werden.
  3. Verfahren nach Anspruch 1,
    dadurch gekennzeichnet, dass dieses ferner einen Schritt des Zerlegens des digitalen Audiosignals in mindestens zwei Subsignale als Funktion eines Frequenzkriteriums umfasst, und dadurch, dass die Schritte des Berechnens und Vergleichens für jedes der Subsignale durchgeführt werden, wobei die Inhibierung der Behandlung zur Dämpfung von Vorechos in der Vorechozone aller Subsignale erfolgt, wenn ein berechneter Richtungskoeffizient kleiner ist als die vordefinierte Schwelle für mindestens ein Subsignal.
  4. Verfahren nach Anspruch 3,
    dadurch gekennzeichnet, dass eine andere Schwelle pro Subsignal definiert wird.
  5. Verfahren nach einem der Ansprüche 1 bis 4,
    dadurch gekennzeichnet, dass der Richtungskoeffizient nach einer Schätzmethode der kleinsten Quadrate berechnet wird.
  6. Verfahren nach einem der Ansprüche 1 bis 5,
    dadurch gekennzeichnet, dass der Richtungskoeffizient normalisiert wird.
  7. Verfahren nach Anspruch 1,
    dadurch gekennzeichnet, dass in dem Fall, wo ein Angriff in dem ersten oder zweiten Subblock des aktuellen Rahmens detektiert wird, ein für den vorausgehenden Rahmen berechneter Richtungskoeffizient für den Vergleichsschritt verwendet wird.
  8. Vorrichtung zur Unterscheidung und Dämpfung von Vorechos in einem digitalen Audiosignal, das durch einen Transformationscodierer erzeugt wird, wobei die Vorrichtung mit einem Decodierer assoziiert ist und ein Modul (601) zur Detektion eines Übergangs oder Angriffs, ein Modul zur Unterscheidung einer Vorechozone (602) und ein Modul (607) zur Behandlung zur Dämpfung von Vorechos umfasst, wobei eine Behandlung zur Dämpfung von Echos für einen aktuellen Rahmen, der in Subblöcke zerlegt ist, in den Subblöcken mit niedriger Energie durchgeführt wird, die einem Subblock vorausgehen, in dem ein Übergang oder ein Angriff detektiert wird, und die eine Vorechozone definieren,
    wobei die Vorrichtung dadurch gekennzeichnet ist, dass diese ferner umfasst:
    - ein Berechnungsmodul (603), das einen Energierichtungskoeffizienten für mindestens zwei Subblöcke des aktuellen Rahmens berechnet, die dem Subblock vorausgehen, in dem ein Angriff detektiert wird, in dem Fall, wo ein Angriff aus dem dritten Subblock des aktuellen Rahmens detektiert wird;
    - einen Komparator (604), der geeignet ist, einen Vergleich des Richtungskoeffizienten mit einer vordefinierten Schwelle vorzunehmen; und
    - ein Unterscheidungsmodul (602), das geeignet ist, die Behandlung zur Dämpfung von Vorechos in der Vorechozone in dem Fall zu inhibieren, wo der berechnete Richtungskoeffizient kleiner ist als die vordefinierte Schwelle.
  9. Decodierer für ein digitales Audiosignal, umfassend eine Vorrichtung zur Unterscheidung und Dämpfung von Vorechos nach Anspruch 8.
  10. Computerprogramm, umfassend Codeinstruktionen zur Durchführung der Schritte des Verfahrens nach einem der Ansprüche 1 bis 7, wenn diese Instruktionen von einem Prozessor ausgeführt werden.
  11. Speichermedium, das von einer Vorrichtung zur Unterscheidung und Behandlung zur Dämpfung von Vorechos lesbar ist, auf dem ein Computerprogramm gespeichert ist, das Codeinstruktionen zur Ausführung der Schritte des Verfahrens zur Unterscheidung und Behandlung zur Dämpfung von Vorechos nach einem der Ansprüche 1 bis 7 umfasst, wenn das Speichermedium auf einem Rechner funktioniert.
EP15771686.1A 2014-09-12 2015-09-11 Unterscheidung und dämpfung von vorechos in einem digitalen audiosignal Active EP3192073B1 (de)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
FR1458608A FR3025923A1 (fr) 2014-09-12 2014-09-12 Discrimination et attenuation de pre-echos dans un signal audionumerique
PCT/FR2015/052433 WO2016038316A1 (fr) 2014-09-12 2015-09-11 Discrimination et atténuation de pré-échos dans un signal audionumérique

Publications (2)

Publication Number Publication Date
EP3192073A1 EP3192073A1 (de) 2017-07-19
EP3192073B1 true EP3192073B1 (de) 2018-08-01

Family

ID=51842602

Family Applications (1)

Application Number Title Priority Date Filing Date
EP15771686.1A Active EP3192073B1 (de) 2014-09-12 2015-09-11 Unterscheidung und dämpfung von vorechos in einem digitalen audiosignal

Country Status (8)

Country Link
US (1) US10083705B2 (de)
EP (1) EP3192073B1 (de)
JP (2) JP6728142B2 (de)
KR (1) KR102000227B1 (de)
CN (2) CN112086107B (de)
ES (1) ES2692831T3 (de)
FR (1) FR3025923A1 (de)
WO (1) WO2016038316A1 (de)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR3025923A1 (fr) * 2014-09-12 2016-03-18 Orange Discrimination et attenuation de pre-echos dans un signal audionumerique
EP3382700A1 (de) * 2017-03-31 2018-10-03 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und verfahren zur nachbearbeitung eines audiosignals mit transienten-positionserkennung
US11100942B2 (en) * 2017-07-14 2021-08-24 Dolby Laboratories Licensing Corporation Mitigation of inaccurate echo prediction
JP7172030B2 (ja) * 2017-12-06 2022-11-16 富士フイルムビジネスイノベーション株式会社 表示装置及びプログラム

Family Cites Families (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
NL249503A (de) 1959-03-19
JP3104400B2 (ja) * 1992-04-27 2000-10-30 ソニー株式会社 オーディオ信号符号化装置及び方法
FR2739736B1 (fr) * 1995-10-05 1997-12-05 Jean Laroche Procede de reduction des pre-echos ou post-echos affectant des enregistrements audio
JP3660599B2 (ja) * 2001-03-09 2005-06-15 日本電信電話株式会社 音響信号の立ち上がり・立ち下がり検出方法及び装置並びにプログラム及び記録媒体
WO2005057801A2 (en) * 2003-12-05 2005-06-23 Plexus Networks, Inc. Low-power mixed-mode echo/crosstalk cancellation in wireline communications
CA2457988A1 (en) * 2004-02-18 2005-08-18 Voiceage Corporation Methods and devices for audio compression based on acelp/tcx coding and multi-rate lattice vector quantization
TWI275074B (en) * 2004-04-12 2007-03-01 Vivotek Inc Method for analyzing energy consistency to process data
FR2897733A1 (fr) * 2006-02-20 2007-08-24 France Telecom Procede de discrimination et d'attenuation fiabilisees des echos d'un signal numerique dans un decodeur et dispositif correspondant
JP5295372B2 (ja) 2008-09-17 2013-09-18 フランス・テレコム デジタルオーディオ信号におけるプリエコーの減衰
KR20110001130A (ko) * 2009-06-29 2011-01-06 삼성전자주식회사 가중 선형 예측 변환을 이용한 오디오 신호 부호화 및 복호화 장치 및 그 방법
KR101405022B1 (ko) * 2009-09-18 2014-06-10 돌비 인터네셔널 에이비 입력 신호를 전위시키기 위한 시스템 및 방법, 상기 방법을 수행하는 소프트웨어 프로그램 및 컴퓨터 프로그램 제품을 포함하는 저장 매체
US8582443B1 (en) * 2009-11-23 2013-11-12 Marvell International Ltd. Method and apparatus for virtual cable test using echo canceller coefficients
CN103325379A (zh) * 2012-03-23 2013-09-25 杜比实验室特许公司 用于声学回声控制的方法与装置
CN103391381B (zh) * 2012-05-10 2015-05-20 中兴通讯股份有限公司 回声消除方法及装置
FR2992766A1 (fr) 2012-06-29 2014-01-03 France Telecom Attenuation efficace de pre-echos dans un signal audionumerique
CN103730125B (zh) * 2012-10-12 2016-12-21 华为技术有限公司 一种回声抵消方法和设备
FR3000328A1 (fr) * 2012-12-21 2014-06-27 France Telecom Attenuation efficace de pre-echos dans un signal audionumerique
FR3011408A1 (fr) * 2013-09-30 2015-04-03 Orange Re-echantillonnage d'un signal audio pour un codage/decodage a bas retard
FR3015754A1 (fr) * 2013-12-20 2015-06-26 Orange Re-echantillonnage d'un signal audio cadence a une frequence d'echantillonnage variable selon la trame
FR3023036A1 (fr) * 2014-06-27 2016-01-01 Orange Re-echantillonnage par interpolation d'un signal audio pour un codage / decodage a bas retard
FR3025923A1 (fr) * 2014-09-12 2016-03-18 Orange Discrimination et attenuation de pre-echos dans un signal audionumerique

Also Published As

Publication number Publication date
JP7008756B2 (ja) 2022-01-25
CN112086107B (zh) 2024-04-02
WO2016038316A1 (fr) 2016-03-17
JP2020170187A (ja) 2020-10-15
ES2692831T3 (es) 2018-12-05
JP6728142B2 (ja) 2020-07-22
US10083705B2 (en) 2018-09-25
KR20170055515A (ko) 2017-05-19
KR102000227B1 (ko) 2019-07-15
FR3025923A1 (fr) 2016-03-18
EP3192073A1 (de) 2017-07-19
CN106716529B (zh) 2020-09-22
CN106716529A (zh) 2017-05-24
US20170263263A1 (en) 2017-09-14
CN112086107A (zh) 2020-12-15
JP2017532595A (ja) 2017-11-02

Similar Documents

Publication Publication Date Title
EP2002428B1 (de) Verfahren zur trainierten diskrimination und dämpfung von echos eines digitalsignals in einem decoder und entsprechende einrichtung
EP2867893B1 (de) Wirksame prä-echodämpfung in einem digitalen audiosignal
CA2436318C (fr) Procede et dispositif de reduction de bruit
EP2936488B1 (de) Wirksame dämpfung von vorechos in einem digitalen audiosignal
EP2586133B1 (de) Steuerung einer rauschformungs-feedbackschleife in einem digitalen audiosignal-encoder
EP1789956A1 (de) Verfahren zum verarbeiten eines rauschbehafteten tonsignals und einrichtung zur implementierung des verfahrens
EP2153438B1 (de) Nachbearbeitung zur reduzierung des quantifizierungsrauschens eines codierers während der decodierung
FR2741217A1 (fr) Procede et dispositif permettant d'eliminer les bruits parasites dans un systeme de communication
EP3192073B1 (de) Unterscheidung und dämpfung von vorechos in einem digitalen audiosignal
FR2907586A1 (fr) Synthese de blocs perdus d'un signal audionumerique,avec correction de periode de pitch.
EP2347411B1 (de) Vor-echo-dämpfung in einem digitalaudiosignal
EP1016071B1 (de) Verfahren und vorrichtung zur sprachdetektion
EP0506535B1 (de) Verfahren und Einrichtung zur Bearbeitung von Vorechos eines mittels einer Frequenztransformation kodierten digitalen Audiosignals
EP1116216A1 (de) Verfahren und vorrichtung zur sprachaktivitätsdetektion
EP1021805B1 (de) Verfahren und vorrichtung zur verbesserung eines digitalen sprachsignals
EP2515300B1 (de) Verfahren und System für die Geräuschunterdrückung

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20170404

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 602015014397

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: G10L0025210000

Ipc: G10L0019260000

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

RIC1 Information provided on ipc code assigned before grant

Ipc: G10L 25/21 20130101ALI20180306BHEP

Ipc: G10L 19/26 20130101AFI20180306BHEP

Ipc: G10L 25/51 20130101ALI20180306BHEP

INTG Intention to grant announced

Effective date: 20180323

RIN1 Information on inventor provided before grant (corrected)

Inventor name: KOVESI, BALAZS

Inventor name: RAGOT, STEPHANE

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

Ref country code: AT

Ref legal event code: REF

Ref document number: 1025252

Country of ref document: AT

Kind code of ref document: T

Effective date: 20180815

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: FRENCH

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602015014397

Country of ref document: DE

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 4

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2692831

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20181205

Ref country code: NL

Ref legal event code: MP

Effective date: 20180801

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1025252

Country of ref document: AT

Kind code of ref document: T

Effective date: 20180801

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180801

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180801

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180801

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180801

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181101

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180801

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181201

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181102

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181101

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180801

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180801

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180801

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180801

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180801

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180801

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180801

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180801

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180801

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602015014397

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180801

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180801

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180801

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20180930

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180911

26N No opposition filed

Effective date: 20190503

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180911

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180801

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180930

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180930

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180930

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180801

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180801

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180801

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180801

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20150911

Ref country code: MK

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180801

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20230822

Year of fee payment: 9

Ref country code: GB

Payment date: 20230823

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20230822

Year of fee payment: 9

Ref country code: DE

Payment date: 20230822

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20231002

Year of fee payment: 9