EP2080195B1 - Synthese verlorener blöcke eines digitalen audiosignals - Google Patents

Synthese verlorener blöcke eines digitalen audiosignals Download PDF

Info

Publication number
EP2080195B1
EP2080195B1 EP07871872A EP07871872A EP2080195B1 EP 2080195 B1 EP2080195 B1 EP 2080195B1 EP 07871872 A EP07871872 A EP 07871872A EP 07871872 A EP07871872 A EP 07871872A EP 2080195 B1 EP2080195 B1 EP 2080195B1
Authority
EP
European Patent Office
Prior art keywords
signal
repetition period
samples
amplitude
block
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP07871872A
Other languages
English (en)
French (fr)
Other versions
EP2080195A1 (de
Inventor
Balazs Kovesi
Stéphane RAGOT
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Orange SA
Original Assignee
France Telecom SA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by France Telecom SA filed Critical France Telecom SA
Priority to PL07871872T priority Critical patent/PL2080195T3/pl
Publication of EP2080195A1 publication Critical patent/EP2080195A1/de
Application granted granted Critical
Publication of EP2080195B1 publication Critical patent/EP2080195B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/005—Correction of errors induced by the transmission channel, if related to the coding algorithm
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/90—Pitch determination of speech signals
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • G10L19/0204—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using subband decomposition
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • G10L19/022—Blocking, i.e. grouping of samples in time; Choice of analysis windows; Overlap factoring
    • G10L19/025—Detection of transients or attacks for time/frequency resolution switching

Definitions

  • the present invention relates to the processing of digital audio signals (especially speech signals).
  • the present invention relates to a reception processing for improving the quality of the decoded signals in the presence of data block losses.
  • the LTP long-term prediction parameters including the pitch period, represent the fundamental vibration of the speech signal (when it is voiced), while the LPC short-term prediction parameters represent the spectral envelope. of this signal.
  • all of these LPC and LTP parameters thus resulting from a speech coding, can be transmitted in blocks to a peer decoder, via one or more telecommunication networks, to then restore the initial speech signal.
  • the G.722 coder has an ADPCM coding scheme in two sub-bands obtained by a QMF (for "Quadrature Mirror Filter") filter bank.
  • QMF for "Quadrature Mirror Filter”
  • the figure 1 of the state of the art shows the coding and decoding structure according to Recommendation G.722.
  • Blocks 101 to 103 represent the transmission filter bank QMF (spectral separation high 102 and low 100 frequencies and sub-sampling 101 and 103), applied to the input signal Se.
  • the following blocks 104 and 105 respectively correspond to low and high band ADPCM coders.
  • the Low-band ADPCM encoder rate is specified by a mode of 0, 1, or 2, indicating a rate of 6, 5, or 4 bits per sample, respectively, while the high-band ADPCM encoder rate is fixed (two bits per second). sample).
  • the ADPCM decoding equivalent blocks (blocks 106 and 107), the outputs of which are combined in the receiving QMF filterbank (oversampling 108 and 110, inverse filters 109, 111 and joining the low frequency bands and high 112) to generate the synthesis signal Ss.
  • the decoder When a loss of one or more consecutive blocks, the decoder must reconstruct the signal without information on lost or errored blocks. It relies on the previously decoded information from the valid blocks received. This problem, called “lost block correction” (or, hereafter, “erasure correction”) is actually more general than simply extrapolating missing information because the loss of frames often causes a loss of synchronization between encoder and decoder, especially when these are predictive, as well as continuity problems between the extrapolated information and the decoded information after a loss.
  • the correction of erased frames thus also encompasses state restoration, re-convergence, and other techniques.
  • Annex I of ITU-T Recommendation G.711 describes erasure correction for PCM encoding.
  • the frame loss correction is therefore simply to extrapolate the missing information and ensure the continuity between a reconstructed frame and correctly received frames, following a loss.
  • the extrapolation is implemented by repetition of the signal passed synchronously with the fundamental frequency (or conversely, "pitch period"), that is to say by simply repeating pitch periods.
  • the continuity is ensured by a smoothing (or "cross-fading") between samples received and extrapolated samples.
  • a speech signal includes so-called “transient” sounds (non-stationary sounds including typically vowel attacks (beginnings) and sounds called “plosives” which correspond to short consonants such as “p", "b”, “d”, "t", “k”).
  • transient sounds non-stationary sounds including typically vowel attacks (beginnings)
  • plosives sounds which correspond to short consonants such as "p", "b”, “d”, "t", “k”
  • a correction of frame loss by simple repetition will generate a very unpleasant sequence listening to "t” (which will be understood in French as “teu- teu-teu-teu-teu ) in burst for a loss of several successive frames (for example five consecutive losses).
  • the Figures 2a and 2b illustrate this acoustic effect in the case of an expanded band signal encoded by an encoder according to Recommendation G.722.
  • the figure 2a shows a decoded speech signal on an ideal channel (without loss of frame).
  • This signal corresponds, in the example represented, to the French word "temps", divided into two phonemes: / t / then / an /.
  • Vertical dashed lines indicate the boundaries between frames. We consider here the case of frames of length of the order of 10 ms.
  • the figure 2b shows the decoded signal according to a technique similar to the Serizawa et al reference above when a frame loss immediately follows the phoneme / t /. This figure 2b shows the problem of the repetition of the past signal.
  • the phoneme / t / is repeated in the extrapolated frame. It is also present in the following frame or frames because the extrapolation is slightly prolonged after a loss, in the example shown, in order to effect a cross-fade with the decoding under normal conditions (ie in the presence of useful information in the received signal).
  • the method according to the invention is advantageously applied to the processing of a speech signal, both in the case of a voiced signal and in the case of an unvoiced signal.
  • the repetition period simply consists of the pitch period and step a) of the method aims in particular to determine a pitch period (typically given by the inverse of a fundamental frequency) a tone of the signal (for example the tone of a voice in a speech signal) in at least one valid block preceding the loss.
  • the valid signal received is not voiced, there is not really a detectable pitch period.
  • it can be expected to set a given arbitrary number of samples which will be considered as the length of the pitch period (which can then be called generically "repetition period”) and carry out the process. sense of the invention on the basis of this repetition period.
  • the longest possible pitch period typically 20 ms (corresponding to 50 Hz of a very deep voice), or 160 samples at 8 kHz sampling frequency.
  • the sample correction step b) is applied to all samples of the last period of repetition, taken one by one as a current sample.
  • step b) is repeatedly copied to form the replacement blocks.
  • This amplitude chosen from the amplitudes of the samples of said neighborhood is preferably the maximum amplitude in absolute value.
  • a damping (gradual attenuation) of the amplitude of the samples in the replacement blocks is usually applied.
  • a transient character of the signal is detected before the loss of blocks, and, where appropriate, a faster damping is applied than for a stationary (non-transitory) signal.
  • the digital audio signal is a speech signal
  • a degree of voicing is advantageously detected in the speech signal and the correction of step b) is not implemented if the speech signal is strongly voiced (which manifests itself by a correlation coefficient close to "1" in the search for a pitch period).
  • this correction is implemented only if the signal is not voiced or if it is weakly voiced.
  • step b) avoids applying the correction of step b) and unnecessarily attenuating the signal in the replacement blocks, if the valid signal received is strongly voiced (thus stationary), which actually corresponds to the pronunciation of the signal. a stable vowel (eg "aaaa”).
  • the present invention is directed to the signal modification before repetition period repetition (or "pitch" for a voiced speech signal), for the synthesis of lost blocks at the decoding of digital audio signals.
  • Transient repeat effects are avoided by comparing samples of a pitch period with those of the previous pitch period.
  • the signal is modified preferentially by taking the minimum between the current sample and at least one sample substantially of the same position of the previous pitch period.
  • the invention offers several advantages, particularly in the context of decoding in the presence of block losses.
  • it makes it possible to avoid artifacts coming from the erroneous repetition of transients (when a simple repetition of pitch period is used).
  • it performs a transient detection which can be used to adapt the energy control of the extrapolated signal (via a variable attenuation).
  • the decoder in the sense of the invention again presents an architecture in two subbands with the reception QMF filter banks (blocks 310 to 314).
  • the decoder of the figure 3 further integrates a device 320 for clearing erased frames.
  • the G.722 decoder generates an output signal Ss sampled at 16 kHz and cut into time frames (or sample blocks) of 10, 20 or 40 ms. Its operation differs according to the presence or not of loss of frames.
  • the erased frame is extrapolated in the block 301 from the signal x1 passed (pitch copy in particular) and the states of the ADPCM decoder are updated. in block 302.
  • the erased frame is extrapolated in block 305 from the passed signal xh and the states of the ADPCM decoder are updated in block 306.
  • the extrapolation yh is a simple repetition of the last period of the past xh signal.
  • This signal uh is advantageously filtered to give the signal vh.
  • the G.722 coding is a recursive predictive coding scheme (of the "backward" type). It uses in each subband an ARMA prediction operation (for "Auto-Regressive Moving Average") and a procedure for adapting the ARMA filter quantization and adaptation pitch, identical to the encoder and to the decoder. The prediction and the pitch adaptation are based on the decoded information (prediction error, reconstructed signal).
  • the transmission errors lead to a desynchronization between the decoder and the encoder variables.
  • the pitch adaptation and prediction procedures are then erroneous and skewed over a long period of time (up to 300-500 ms).
  • this bias can result, among other artifacts, in the appearance of a continuous component of very low amplitude (of the order of +/- 10 for a maximum dynamic signal +/- 32767 ).
  • this DC component is found in the form of a sinusoid at 8kHz audible and very troublesome to the hearing.
  • the transformation of the DC component (or “DC component”) into a sinusoid at 8 kHz is explained below.
  • the figure 8a represents a two-channel quadrature filter bank (QMF).
  • the signal x (n) is decomposed into two subbands by the analysis bank. We thus obtain a low band xl (n) and a high band xh (n).
  • the signal obtained after the synthesis filter bank is identical to the signal x (n) with a shift.
  • the filters L (z) and H (z) can be, for example, the QMF filters of 24 coefficients specified in ITU-T Recommendation G.722.
  • the figure 8b shows the spectrum of the signals x (n), xl (n) and xh (n) in the case where the filters L (z) and H (z) are ideal mid-band filters.
  • the frequency response of L (z) over the interval [-f'e / 2, + fe '/ 2] is then given, in the ideal case, by:
  • the f ⁇ 1 if f ⁇ f vs ' / 4 0 other
  • the L (z) and H (z) filters are not ideal. Their non-ideal character results in the appearance of a spectral folding component which is canceled by the synthesis bench. The high band remains inverted, however.
  • Block 308 then performs a high pass filtering (HPF for " high pass filter ”) which removes the DC component (for " DC remove " in English).
  • HPF high pass filter
  • the use of such a filter is particularly advantageous, including outside the scope of the correction of pitch period in the low band within the meaning of the invention.
  • a high-pass filter 308 is provided on the high frequency channel.
  • This high-pass filter 308 is advantageously provided upstream, for example, of the QMF filter bank of this high-frequency channel of the G.722 decoder.
  • This arrangement makes it possible to avoid the folding of the DC component at 8 kHz (value taken from the sampling rate f ' e ) when it is applied to the QMF filter bank.
  • the high-pass filter (308) is preferably provided upstream of this filter bank.
  • this high-pass filter 308 is temporarily applied (for a few seconds for example) during and after a loss of blocks, even if valid blocks are received again.
  • Filter 308 could be used permanently. Nevertheless, it is activated only in case of frame losses, because the disturbance due to the DC component is generated only in this case, so that the output of the modified G.722 decoder (because integrating the mechanism of loss correction) is identical to that of the ITU-T G.722 decoder in the absence of frame loss.
  • This filter 308 is applied only during the frame loss correction and for a few seconds following a loss.
  • the G.722 decoder is desynchronized from the encoder for a period of 100 to 500 ms following a loss and the continuous component in the high band is typically only present for a duration of 1 to 2 seconds.
  • the filter 308 is maintained a little longer to have a safety margin (for example four seconds).
  • Block 400 performs a linear prediction analysis (LPC) on the passed signal xl.
  • LPC linear prediction analysis
  • This analysis is similar to that carried out in particular in the G.729 standardized coder. It can consist of windowing the signal, calculating the autocorrelation and finding the linear prediction coefficients by the Levinson-Durbin algorithm. Preferably, only the last 10 seconds of the signal are used and the LPC order is set to 8.
  • the past excitation signal is calculated by block 401.
  • Block 402 makes an estimate of the fundamental frequency or its inverse: the pitch period T 0 . This estimation is carried out for example in a manner similar to the pitch analysis (called “open loop” especially as in the standardized encoder G.729).
  • the pitch T 0 thus estimated is used by the block 403 to extrapolate the excitation of the current frame.
  • the passed signal x1 is classified in the block 404.
  • the aim is rather to detect whether the signal Se is strongly voiced (for example when the correlation with respect to the pitch period is very close to 1). If the signal is strongly voiced (which corresponds to the pronunciation of a stable vowel, for example "aaaa ##), then the signal Se is free of transients and the pitch period correction in the sense of the invention may not be implemented. Otherwise, preferentially, the correction of the pitch period in the sense of the invention will be applied in all other cases.
  • SYNTH synthesis follows the model well known in the state of the art and called "source-filter”. It consists in filtering the excitation extrapolated by an LPC filter.
  • the excitation can be extrapolated by simply repeating the last pitch period T 0 , that is to say by copying the succession of the last samples of the past excitation, the number of samples in this succession corresponding to the number of samples that includes the pitch period T 0 .
  • this signal modification is not applied if the signal x 1 (and therefore the input signal Se) is strongly voiced. Indeed, in the case of a strongly voiced signal, the simple repetition of the last pitch period, without modification, can give a better result, whereas a modification of the last pitch period and its repetition could result in a slight quality degradation.
  • step 70 we obtain the information according to which the signal x1 is strongly voiced or not, from the module 404 determining the degree of voicing. If the signal is strongly voiced (arrow O at the output of the test 71), it copies the last pitch period of the valid blocks, as such, in the block 403 of the figure 4 and processing continues directly thereafter by the application of the inverse filter 1 / A (z) by the module 405.
  • each sample e (n) of the last pitch period is made to correspond to a NEIGH neighborhood in the preceding pitch period, ie in the penultimate pitch period.
  • This measure is advantageous but not necessary. The advantage it provides will be described later. It is simply indicated here that this neighborhood comprises an odd number of samples 2k + 1, in the example described. Of course, alternatively, this number may be even.
  • ak 1. Indeed, with reference to the figure 6 , it is found that the third sample of the last pitch period noted e (3) is selected (step 74) and NEIGH neighborhood samples associated with it in the penultimate pitch period (step 75) are represented in FIG. and are e (2- T 0 ), e (3- T 0 ) and e (4- T 0 ). They are therefore distributed around e (3- T 0 ).
  • step 76 the maximum, in absolute value, is determined among the NEIGH neighborhood samples (ie the sample e (2- T 0 ) in the example of the figure 6 ). This feature is advantageous but not necessary. The advantage it provides will be described later. Typically, alternatively, one could choose to determine the average on NEIGH neighborhood, for example.
  • step 77 the minimum, in absolute value, is determined between the value of the current sample e (n) and the value of the maximum M found on NEIGH neighborhood in step 76.
  • this minimum between e (3) and e (2- T 0 ) is indeed the sample of the penultimate pitch period e (2- T 0 ).
  • the amplitude of the current sample e (n) is then replaced by this minimum.
  • the amplitude of the sample e (3) becomes equal to that of the sample e (2- T 0 ).
  • the same method is applied to all the samples of the last period, from e (1) to e (12).
  • the corrected samples are represented by dashed lines. Samples of pitch periods extrapolated T j + 1 , T j + 2 , corrected according to the invention, are represented by closed arrows.
  • this step 77 if a plosive is indeed present on the last pitch period T j (high intensity of the signal, in absolute value, as represented on FIG. figure 6 ), we will determine the minimum between this intensity of the plosive and that of the samples substantially at the same time position in the preceding pitch period (the term "substantially” meaning here "to a neighborhood ⁇ k near", hence the advantage of the realization of step 75), and replace, if necessary, the intensity of the plosive by a lower intensity belonging to the penultimate pitch period T j-1 .
  • step 76 the maximum value M in absolute value of the samples of the neighborhood (and not another parameter such as the average on this neighborhood, for example) is determined so as to compensate for the effect of choosing the minimum to step 77 for replace the value e (n).
  • This measurement therefore makes it possible not to limit the amplitude of the pitch pitch periods T j + 1 , T j + 2 ( figure 6 ).
  • the neighborhood determination step 75 is advantageously implemented because a pitch period is not always regular and, if a sample e (n) has a maximum intensity in a period of pitch T 0 , it is not always the same for a sample e (n + T 0 ) in a next pitch period.
  • a pitch period may extend to a time position falling between two samples (at a given sampling frequency). We speak of "fractional pitch". It is therefore always preferable to take a neighborhood centered around a sample e (n- T 0 ), if this sample e (n- T 0 ) must be associated with a sample e (n) positioned at a pitch period next.
  • step 78 consists simply in reassigning the sign of the initial sample e (n) to the modified sample e mod (n).
  • Steps 75 to 78 are repeated for a sample e (n) following (n before n + 1 in step 79), until the pitch period T 0 is exhausted (ie until reaching the last valid sample e ( n 1 )).
  • the modified signal e mod (n) is thus delivered to the inverse filter 1 / A (z) (reference 405 of FIG. figure 4 ) for further decoding.
  • the signal passed x1 has a transient (for example a plosive), which makes it possible to force a fast attenuation by the block 406 on the synthesis signal yl (eg attenuation over 10 ms).
  • a transient for example a plosive
  • the Figure 2c then illustrates the decoded signal when the invention is implemented, for comparison with the Figures 2a and 2b for which a frame with the plosive / t / was lost. Repetition of the phoneme / t / is avoided here, thanks to the implementation of the invention.
  • the differences following the frame loss are not related to the actual plosive detection.
  • attenuation of the signal after the frame loss at the Figure 2c can be explained by the fact that in this case, the G.722 decoder is reset (complete update of the states in the block 302 of the figure 3 ), whereas in the case of figure 2b , the G.722 decoder is not reset. It will be understood, however, that the invention relates to the detection of plosives for the extrapolation of an erased frame and not to the problem of restarting after a loss of frame.
  • the present invention also relates to a computer program intended to be stored in memory of a device for synthesizing a digital audio signal.
  • This program then comprises instructions for implementing the method within the meaning of the invention, when it is executed by a processor of such a synthesis device.
  • the figure 7 described above can illustrate a flowchart of such a computer program.
  • the synthesis device SYN within the meaning of the invention comprises means such as a working memory MEM (or storage of the aforementioned computer program) and a PROC processor cooperating with this memory MEM, for the implementation of the method within the meaning of the invention, and thus to synthesize the current block from at least one of the preceding blocks of the signal e (n).
  • a working memory MEM or storage of the aforementioned computer program
  • PROC processor cooperating with this memory MEM, for the implementation of the method within the meaning of the invention, and thus to synthesize the current block from at least one of the preceding blocks of the signal e (n).
  • the present invention also relates to a decoder of a digital audio signal consisting of a succession of blocks, this decoder comprising the device 403 in the sense of the invention for synthesizing invalid blocks.
  • the present invention is not limited to the embodiments described above by way of example; it extends to other variants.
  • signal detection and modification can be performed in the signal domain (rather than the field of excitation).
  • the excitation is extrapolated by repetition of pitch and possibly addition of a random contribution and this excitation is filtered by a filter of type 1 / A (z), where A (z) is derived from the last correctly received predictor filter.
  • sample correction in step b), followed by copying of the corrected samples into the replacement block (s).
  • sample correction and copying can be steps that can occur in any order and, in particular, be reversed.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Computational Linguistics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Transmission Systems Not Characterized By The Medium Used For Transmission (AREA)
  • Stereophonic System (AREA)
  • Analysing Materials By The Use Of Radiation (AREA)

Claims (12)

  1. Verfahren zur Synthese eines digitalen Audiosignals, das durch aufeinanderfolgende Tastprobenblöcke dargestellt wird, bei dem bei Empfang eines solchen Signals zum Ersatz mindestens eines ungültigen Blocks ein Ersatzblock ausgehend von Tastproben mindestens eines gültigen Blocks erzeugt wird,
    wobei das Verfahren die folgenden Schritte aufweist:
    a) Bestimmen (402) einer Wiederholungsperiode in mindestens einem gültigen Block, und
    b) Kopieren (403) der Tastproben der Wiederholungsperiode in mindestens einen Ersatzblock,
    wobei die Wiederholungsperiode einer Pitch-Periode, wenn das Signal stimmhaft ist, oder einem willkürlich oder ausgehend von einer Korrelationsfunktion bestimmten Wert entspricht, wenn das Signal nicht stimmhaft ist,
    dadurch gekennzeichnet, dass:
    - im Schritt a) eine letzte Wiederholungsperiode (Tj) in mindestens einem gültigen Block bestimmt wird, der direkt vor einem ungültigen Block liegt,
    - im Schritt b) Tastproben (e(3)) der letzten Wiederholungsperiode (Tj) abhängig von Tastproben (e(2-T0), e(3-T0), e(4-T0)) einer vor der letzten Wiederholungsperiode liegenden Wiederholungsperiode (Tj-1) korrigiert werden, um die Amplitude eines möglichen transienten Signals in der letzten Wiederholungsperiode zu begrenzen, und die so korrigierten Tastproben in den Ersatzblock (Tj+1, Tj+2) kopiert werden, wobei die Korrektur bezüglich einer Umgebung durchgeführt wird, die um eine Tastprobe herum zentriert ist, die zeitlich in einer Wiederholungsperiode vor der laufenden Tastprobe positioniert ist.
  2. Verfahren nach Anspruch 1, bei dem das Signal ein stimmhaftes Sprachsignal ist, dadurch gekennzeichnet, dass die Wiederholungsperiode eine Pitch-Periode ist, die dem Kehrwert einer Grundfrequenz des Signals entspricht.
  3. Verfahren nach einem der Ansprüche 1 und 2, dadurch gekennzeichnet, dass im Schritt b) eine laufende Tastprobe (e(3)) der letzten Wiederholungsperiode korrigiert wird, indem verglichen wird:
    - die Amplitude dieser laufenden Tastprobe im Absolutwert,
    - mit der Amplitude im Absolutwert mindestens einer Tastprobe (e(2-T0)), die zeitlich im Wesentlichen in einer Wiederholungsperiode vor der laufenden Tastprobe positioniert ist,
    und indem der laufenden Tastprobe die minimale Amplitude, im Absolutwert, unter diesen zwei Amplituden zugeteilt wird.
  4. Verfahren nach Anspruch 3, dadurch gekennzeichnet, dass für eine laufende Tastprobe (e(3)) der letzten Wiederholungsperiode:
    - eine Gruppe von Tastproben (75) in einer Umgebung gebildet wird, die um eine Tastprobe (e(3-T0)) herum zentriert ist, die zeitlich in einer Wiederholungsperiode vor der laufenden Tastprobe positioniert ist,
    - eine gewählte Amplitude (76) unter den Amplituden der Tastproben der Umgebung, im Absolutwert gesehen, bestimmt wird,
    - und diese gewählte Amplitude mit der Amplitude der laufenden Tastprobe, im Absolutwert, verglichen wird, um der laufenden Tastprobe (e(3)) die minimale Amplitude, im Absolutwert, unter der gewählten Amplitude und der Amplitude der laufenden Tastprobe zuzuweisen (77).
  5. Verfahren nach Anspruch 4, dadurch gekennzeichnet, dass die aus den Amplituden der Tastproben der Umgebung gewählte Amplitude die maximale Amplitude im Absolutwert (M) ist.
  6. Verfahren nach einem der vorhergehenden Ansprüche, bei der eine Dämpfung der Amplitude der Tastproben im Ersatzblock angewendet wird, dadurch gekennzeichnet, dass eine mögliche transiente Eigenschaft des Signals in der letzten Wiederholungsperiode erfasst und ggf. die Korrektur des Schritts b) durchgeführt wird, indem eine schnellere Dämpfung als für ein stationäres Signal angewendet wird.
  7. Verfahren nach Anspruch 6 in Kombination mit einem der Ansprüche 3 und 4, dadurch gekennzeichnet, dass:
    - für mehrere laufende Tastproben der letzten Wiederholungsperiode ein Verhältnis, im Absolutwert, der Amplitude einer laufenden Tastprobe zur gewählten Amplitude gemessen wird, und
    - die Anzahl von Vorkommen, für die laufenden Tastproben, bei denen das Verhältnis über einer ersten vorbestimmten Schwelle liegt, gezählt wird, und
    - das Vorhandensein einer transienten Eigenschaft erfasst wird, wenn die Anzahl von Vorkommen höher als eine zweite vorbestimmte Schwelle ist.
  8. Verfahren nach einem der vorhergehenden Ansprüche, dadurch gekennzeichnet, dass im Fall eines Empfangs von mehreren aufeinanderfolgenden ungültigen Blöcken, die sich über mindestens eine Wiederholungsperiode erstrecken, der Schritt der Korrektur von Tastproben b) bei allen Tastproben der letzten Wiederholungsperiode, einzeln als laufende Tastprobe gesehen, durchgeführt wird.
  9. Verfahren nach Anspruch 8, dadurch gekennzeichnet, dass im Fall eines Empfangs von mehreren aufeinanderfolgenden ungültigen Blöcken, die sich über mehrere Wiederholungsperioden erstrecken, zum Ersetzen der mehreren ungültigen Blöcke die im Schritt b) korrigierte Wiederholungsperiode mehrmals übertragen wird, um die Ersatzblöcke zu formen.
  10. Computerprogramm, das dazu bestimmt ist, im Speicher einer Vorrichtung zur Synthese eines digitalen Audiosignals gespeichert zu werden, dadurch gekennzeichnet, dass es Anweisungen enthält, die für die Durchführung des Verfahrens nach einem der Ansprüche 1 bis 9 geeignet sind, wenn es von einem Prozessor einer solchen Synthesevorrichtung ausgeführt wird.
  11. Vorrichtung zur Synthese eines aus einer Folge von Blöcken bestehenden digitalen Audiosignals, die aufweist:
    - einen Eingang (E) für den Empfang von Blöcken des Signals (e(n)), die mindestens vor einem zu synthetisierenden laufenden Block liegen, und
    - einen Ausgang (S), um das synthetisierte Signal (emod(n)) zu liefern, das mindestens den laufenden Block aufweist,
    dadurch gekennzeichnet, dass sie Einrichtungen (MEM, PROC) aufweist, die für die Durchführung des Verfahrens nach einem der Ansprüche 1 bis 9 geeignet sind, um den laufenden Block ausgehend von mindestens einem der vorhergehenden Blöcke zu synthetisieren.
  12. Decoder eines digitalen Audiosignals, das aus einer Folge von Blöcken besteht, dadurch gekennzeichnet, dass er außerdem eine Vorrichtung (403) nach Anspruch 11 aufweist, um ungültige Blöcke zu synthetisieren.
EP07871872A 2006-10-20 2007-10-17 Synthese verlorener blöcke eines digitalen audiosignals Active EP2080195B1 (de)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PL07871872T PL2080195T3 (pl) 2006-10-20 2007-10-17 Synteza utraconych bloków cyfrowego sygnału akustycznego

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
FR0609227A FR2907586A1 (fr) 2006-10-20 2006-10-20 Synthese de blocs perdus d'un signal audionumerique,avec correction de periode de pitch.
PCT/FR2007/052189 WO2008096084A1 (fr) 2006-10-20 2007-10-17 Synthèse de blocs perdus d'un signal audionumérique, avec correction de période de pitch

Publications (2)

Publication Number Publication Date
EP2080195A1 EP2080195A1 (de) 2009-07-22
EP2080195B1 true EP2080195B1 (de) 2011-03-16

Family

ID=37735201

Family Applications (1)

Application Number Title Priority Date Filing Date
EP07871872A Active EP2080195B1 (de) 2006-10-20 2007-10-17 Synthese verlorener blöcke eines digitalen audiosignals

Country Status (14)

Country Link
US (1) US8417519B2 (de)
EP (1) EP2080195B1 (de)
JP (1) JP5289320B2 (de)
KR (1) KR101406742B1 (de)
CN (1) CN101627423B (de)
AT (1) ATE502376T1 (de)
BR (1) BRPI0718422B1 (de)
DE (1) DE602007013265D1 (de)
ES (1) ES2363181T3 (de)
FR (1) FR2907586A1 (de)
MX (1) MX2009004211A (de)
PL (1) PL2080195T3 (de)
RU (1) RU2432625C2 (de)
WO (1) WO2008096084A1 (de)

Families Citing this family (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8706479B2 (en) * 2008-11-14 2014-04-22 Broadcom Corporation Packet loss concealment for sub-band codecs
KR101622950B1 (ko) * 2009-01-28 2016-05-23 삼성전자주식회사 오디오 신호의 부호화 및 복호화 방법 및 그 장치
JP5456370B2 (ja) * 2009-05-25 2014-03-26 任天堂株式会社 発音評価プログラム、発音評価装置、発音評価システムおよび発音評価方法
US8976675B2 (en) * 2011-02-28 2015-03-10 Avaya Inc. Automatic modification of VOIP packet retransmission level based on the psycho-acoustic value of the packet
JP5932399B2 (ja) * 2012-03-02 2016-06-08 キヤノン株式会社 撮像装置及び音声処理装置
CN105976830B (zh) * 2013-01-11 2019-09-20 华为技术有限公司 音频信号编码和解码方法、音频信号编码和解码装置
FR3001593A1 (fr) * 2013-01-31 2014-08-01 France Telecom Correction perfectionnee de perte de trame au decodage d'un signal.
MX2021000353A (es) * 2013-02-05 2023-02-24 Ericsson Telefon Ab L M Método y aparato para controlar ocultación de pérdida de trama de audio.
ES2664968T3 (es) 2013-02-05 2018-04-24 Telefonaktiebolaget Lm Ericsson (Publ) Encubrimiento de pérdida de trama de audio
WO2014123469A1 (en) 2013-02-05 2014-08-14 Telefonaktiebolaget L M Ericsson (Publ) Enhanced audio frame loss concealment
JP6153661B2 (ja) * 2013-06-21 2017-06-28 フラウンホーファーゲゼルシャフト ツール フォルデルング デル アンゲヴァンテン フォルシユング エー．フアー． 改善されたパルス再同期化を採用するａｃｅｌｐ型封じ込めにおける適応型コードブックの改善された封じ込めのための装置および方法
BR112015031181A2 (pt) 2013-06-21 2017-07-25 Fraunhofer Ges Forschung aparelho e método que realizam conceitos aperfeiçoados para tcx ltp
US9418671B2 (en) * 2013-08-15 2016-08-16 Huawei Technologies Co., Ltd. Adaptive high-pass post-filter
JP6306175B2 (ja) 2013-10-31 2018-04-04 フラウンホッファー−ゲゼルシャフト ツァ フェルダールング デァ アンゲヴァンテン フォアシュンク エー．ファオ 時間ドメイン励振信号に基づくエラーコンシールメントを用いて、復号化されたオーディオ情報を提供するオーディオデコーダおよび復号化されたオーディオ情報を提供する方法
BR122022008603B1 (pt) 2013-10-31 2023-01-10 Fraunhofer-Gesellschaft Zur Forderung Der Angewandten Forschung E.V. Decodificador de áudio e método para fornecer uma informação de áudio decodificada utilizando uma dissimulação de erro que modifica um sinal de excitação no domínio de tempo
NO2780522T3 (de) 2014-05-15 2018-06-09
US9706317B2 (en) * 2014-10-24 2017-07-11 Starkey Laboratories, Inc. Packet loss concealment techniques for phone-to-hearing-aid streaming
JP6611042B2 (ja) * 2015-12-02 2019-11-27 パナソニックＩｐマネジメント株式会社 音声信号復号装置及び音声信号復号方法
GB2547877B (en) * 2015-12-21 2019-08-14 Graham Craven Peter Lossless bandsplitting and bandjoining using allpass filters
CN106970950B (zh) * 2017-03-07 2021-08-24 腾讯音乐娱乐（深圳）有限公司 相似音频数据的查找方法及装置
WO2022045395A1 (ko) * 2020-08-27 2022-03-03 임재윤 파열음 제거를 위한 오디오데이터를 보정하는 방법 및 장치

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3369077A (en) * 1964-06-09 1968-02-13 Ibm Pitch modification of audio waveforms
CA2137459A1 (en) * 1993-05-04 1994-11-10 Stephen V. Cahill Apparatus and method for substantially eliminating noise in an audible output signal
US6597961B1 (en) * 1999-04-27 2003-07-22 Realnetworks, Inc. System and method for concealing errors in an audio transmission
JP2001228896A (ja) * 2000-02-14 2001-08-24 Iwatsu Electric Co Ltd 欠落音声パケットの代替置換方式
US6584438B1 (en) * 2000-04-24 2003-06-24 Qualcomm Incorporated Frame erasure compensation method in a variable rate speech coder
US20030163304A1 (en) * 2002-02-28 2003-08-28 Fisseha Mekuria Error concealment for voice transmission system
US20030220787A1 (en) * 2002-04-19 2003-11-27 Henrik Svensson Method of and apparatus for pitch period estimation
US7411985B2 (en) * 2003-03-21 2008-08-12 Lucent Technologies Inc. Low-complexity packet loss concealment method for voice-over-IP speech transmission
GB2416467B (en) * 2003-05-14 2006-08-30 Oki Electric Ind Co Ltd Apparatus and method for concealing erased periodic signal data
JP4135621B2 (ja) * 2003-11-05 2008-08-20 沖電気工業株式会社 受信装置および方法
CN1989548B (zh) * 2004-07-20 2010-12-08 松下电器产业株式会社 语音解码装置及补偿帧生成方法
EP2054876B1 (de) * 2006-08-15 2011-10-26 Broadcom Corporation Überbrückung von paketverlusten zur prädiktiven, auf der extrapolation der audiowellenform basierenden subband-kodierung

Also Published As

Publication number Publication date
ATE502376T1 (de) 2011-04-15
BRPI0718422A2 (pt) 2013-11-12
KR20090082415A (ko) 2009-07-30
ES2363181T3 (es) 2011-07-26
PL2080195T3 (pl) 2011-09-30
EP2080195A1 (de) 2009-07-22
JP5289320B2 (ja) 2013-09-11
BRPI0718422B1 (pt) 2020-02-11
CN101627423B (zh) 2012-05-02
RU2432625C2 (ru) 2011-10-27
WO2008096084A1 (fr) 2008-08-14
JP2010507121A (ja) 2010-03-04
KR101406742B1 (ko) 2014-06-12
FR2907586A1 (fr) 2008-04-25
RU2009118929A (ru) 2010-11-27
DE602007013265D1 (de) 2011-04-28
US20100318349A1 (en) 2010-12-16
CN101627423A (zh) 2010-01-13
US8417519B2 (en) 2013-04-09
MX2009004211A (es) 2009-07-02

Similar Documents

Publication Publication Date Title
EP2080195B1 (de) Synthese verlorener blöcke eines digitalen audiosignals
EP2277172B1 (de) Verbergung von übertragungsfehlern in einem digitalsignal in einer hierarchischen decodierungsstruktur
EP1316087B1 (de) Übertragungsfehler-verdeckung in einem audiosignal
EP2002428B1 (de) Verfahren zur trainierten diskrimination und dämpfung von echos eines digitalsignals in einem decoder und entsprechende einrichtung
EP3103116B1 (de) Verbesserte frequenzbanderweiterung in einem audiosignaldecodierer
EP2080194B1 (de) Dämpfung von stimmüberlagerung, im besonderen zur erregungserzeugung bei einem decoder in abwesenheit von informationen
WO2015004373A1 (fr) Facteur d'échelle optimisé pour l'extension de bande de fréquence dans un décodeur de signaux audiofréquences
EP3175444B1 (de) Rahmenverlustverwaltung in einem fd-/ldp-übergangskontext
EP2347411B1 (de) Vor-echo-dämpfung in einem digitalaudiosignal
EP3138095B1 (de) Verbesserte frameverlustkorrektur mit sprachinformationen
WO2007107670A2 (fr) Procede de post-traitement d'un signal dans un decodeur audio
EP2203915B1 (de) Übertragungsfehlerverdeckung bei einem digitalem signal mit komplexitätsverteilung
WO2007006958A2 (fr) Procédé et dispositif d'atténuation des échos d'un signal audionumérioue issu d'un codeur multicouches

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20090416

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC MT NL PL PT RO SE SI SK TR

17Q First examination report despatched

Effective date: 20091209

RTI1 Title (correction)

Free format text: SYNTHESIS OF LOST BLOCKS OF A DIGITAL AUDIO SIGNAL

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC MT NL PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REF Corresponds to:

Ref document number: 602007013265

Country of ref document: DE

Date of ref document: 20110428

Kind code of ref document: P

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602007013265

Country of ref document: DE

Effective date: 20110428

REG Reference to a national code

Ref country code: NL

Ref legal event code: VDEP

Effective date: 20110316

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2363181

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20110726

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110617

LTIE Lt: invalidation of european patent or patent extension

Effective date: 20110316

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110616

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

REG Reference to a national code

Ref country code: PL

Ref legal event code: T3

REG Reference to a national code

Ref country code: IE

Ref legal event code: FD4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

Ref country code: IE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110718

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110716

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20111219

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602007013265

Country of ref document: DE

Effective date: 20111219

BERE Be: lapsed

Owner name: FRANCE TELECOM

Effective date: 20111031

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20111031

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20111031

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20111031

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20111031

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20111017

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 9

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 10

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 11

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: TR

Payment date: 20230927

Year of fee payment: 17

Ref country code: IT

Payment date: 20230920

Year of fee payment: 17

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: PL

Payment date: 20230925

Year of fee payment: 17

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20231102

Year of fee payment: 17

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20230920

Year of fee payment: 17

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20240919

Year of fee payment: 18

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20240919

Year of fee payment: 18