EP2203915B1 - Übertragungsfehlerverdeckung bei einem digitalem signal mit komplexitätsverteilung - Google Patents

Übertragungsfehlerverdeckung bei einem digitalem signal mit komplexitätsverteilung Download PDF

Info

Publication number
EP2203915B1
EP2203915B1 EP08838291A EP08838291A EP2203915B1 EP 2203915 B1 EP2203915 B1 EP 2203915B1 EP 08838291 A EP08838291 A EP 08838291A EP 08838291 A EP08838291 A EP 08838291A EP 2203915 B1 EP2203915 B1 EP 2203915B1
Authority
EP
European Patent Office
Prior art keywords
frame
erased
signal
concealment
frames
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP08838291A
Other languages
English (en)
French (fr)
Other versions
EP2203915A1 (de
Inventor
Balazs Kovesi
Stéphane RAGOT
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Orange SA
Original Assignee
France Telecom SA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by France Telecom SA filed Critical France Telecom SA
Publication of EP2203915A1 publication Critical patent/EP2203915A1/de
Application granted granted Critical
Publication of EP2203915B1 publication Critical patent/EP2203915B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
    • G10L19/16—Vocoder architecture
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/005—Correction of errors induced by the transmission channel, if related to the coding algorithm
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • G10L19/0204—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using subband decomposition

Definitions

  • the present invention relates to the processing of digital signals in the telecommunications field. These signals may be, for example, speech, music, video signals or more generally multimedia signals.
  • the present invention intervenes in a coding / decoding system adapted for the transmission / reception of such signals. More particularly, the present invention relates to a reception processing for improving the quality of the decoded signals in the presence of data block losses.
  • disturbances may affect the transmitted signal and produce errors on the bitstream received by the decoder. These errors may occur in isolation in the bit stream but occur very frequently in bursts. It is then a packet of bits corresponding to a complete portion of the signal that is erroneous or not received. This type of problem occurs for example for transmissions on mobile networks. It is also found in transmission on packet networks and in particular on Internet-type networks.
  • the transmission system or the modules in charge of reception detect that the received data are highly erroneous (for example on mobile networks), or that a block of data has not been received or is corrupted by errors
  • error concealment procedures are implemented.
  • LPC Linear Predictive Coding
  • LTP Long Term Prediction
  • the LPC parameters of a frame to be reconstructed are obtained from the LPC parameters of the last valid frame, by simple copy of the parameters or with introduction of a certain damping (technique used for example in the G723.1 standard encoder see also the document US 2006/0171373 ). Then, a voicing or non-voicing in the speech signal is detected to determine a degree of harmonicity of the signal at the erased frame.
  • an excitation signal can be randomly generated (by drawing a codeword of the past excitation, by a slight damping of the gain of the past excitation, by random selection in the past excitement, or still using transmitted codes that may be totally wrong).
  • the pitch period (also called “LTP delay”) is generally the one calculated for the previous frame, possibly with a slight “jitter” (increase of the value of the LTP delay for consecutive error frames, the gain LTP being taken very close to 1 or equal to 1).
  • the excitation signal is therefore limited to the long-term prediction made from a past excitation.
  • the algorithm for concealing erased frames must first estimate the extrapolation parameters from the signal itself. decoded past. This typically requires short-term (LPC) and long-term (LTP) correlation analyzes and possibly signal classification (voiced, voiceless, plosive, etc.) which greatly increases the computational load. These analyzes are described, for example, in the document entitled “ "From B. KOVESI and D. Massaloux,” ISIVC-2004, International Symposium on Image / Video . According to this technique described, the method of concealing an erased frame therefore consists of a first analysis part and a second extrapolation part producing missing samples of the signal corresponding to the erased frame.
  • this calculation load increase due to the past signal analyzes is the same as the erased frame of either 5 ms or 40 ms.
  • the analyzes of the past signal require a given number of operations per frame, regardless of the frame size.
  • the complexity of these analyzes is measured in the number of operations per second. This complexity therefore increases as long as the frame length is short because the number of operations per second is given by the number of operations per frame divided by the frame length - the number of operations per second is therefore inversely proportional to the frame length.
  • the average complexity is also an important parameter because it influences the energy consumption of the processor and thus the duration of autonomy of the battery of the equipment in which it is located, such as a mobile terminal.
  • This computing load in some cases remains reasonable and comparable to the computation load of the normal decoding.
  • an algorithm for hiding low complexity erased frames has been standardized according to the ITU-T Recommendation G.722 appendix IV.
  • the complexity of calculating the extrapolation of an erased frame of 10 ms is in this case 3 WMOPS (for "Weighted Million Operations Per Second"), which is substantially identical to the complexity of the decoding of a valid frame.
  • the complexity of such an algorithm for concealing erased frames can be penalizing in the case of very low complexity coders such as the standard encoder according to the ITU-T Recommendation G.711 (MIC) and these extensions as the G encoder .711 WB in the process of normalization, in particular for the decoding of the low band, sampled at 8 kHz and coded by a G.711 coder followed by an enhancement layer.
  • very low complexity coders such as the standard encoder according to the ITU-T Recommendation G.711 (MIC) and these extensions as the G encoder .711 WB in the process of normalization, in particular for the decoding of the low band, sampled at 8 kHz and coded by a G.711 coder followed by an enhancement layer.
  • the complexity of the MIC coding / decoding is of the order of 0.3 WMOPS, whereas that of a high performance erased frame dissimilar algorithm is typically of the order of 3 WMOPS based on frames of 10 ms.
  • a transmission error concealment method in a digital signal divided into a plurality of successive frames associated with different time intervals in which, on reception, the signal may comprise erased frames and valid frames. and to replace at least the first erased frame after a valid frame, at least two steps are performed, a first preparation step producing no missing sample and having at least one analysis of a valid decoded signal and a second concealment step producing the missing samples of the signal corresponding to the said erased frame.
  • the method is such that the first step and the second step are performed in different time intervals.
  • parameters decoded in the previous valid frames are used for loss concealment.
  • such parameters are not transmitted to the decoder and must be estimated by analysis to synthesize the missing signal when concealing losses.
  • the preparation step is carried out in the time interval associated with a valid frame and the concealment step is performed in the time interval associated with an erased frame.
  • the preparation step is carried out before the time interval corresponding to an erased frame, the second step no longer requires such significant complexity during the time interval corresponding to the erased frame, which reduces the complexity in this interval. It is usually during this interval that the worst case of complexity is measured. This is thus decreased in this embodiment.
  • the preparation step is performed in the time interval associated with an erased frame and the concealment step is performed in a subsequent time interval.
  • the first step is no longer performed systematically when receiving a valid frame but on receiving an erased frame. So we reduce thus both the worst case of complexity by the distribution of the computing load and the average complexity with respect to the first embodiment.
  • the second embodiment of the method according to the invention is such that it is implemented during the decoding of a first frequency band in a decoding system comprising a decoding in a first frequency band and a decoding in a second frequency band, the decoding in the second frequency band having a time delay with respect to the decoding in the first frequency band.
  • the delay introduced by the execution of the second step over the next time interval is transparent for this type of decoding which already has a time delay between the decoding of the first frequency band and the second frequency band.
  • the invention is particularly suitable in the case where the first frequency band corresponds to the low band of a G.711WB type decoding and the second frequency band corresponds to the high band of a G.711 WB type decoding. , the delay of the signal from the concealment step corresponding to the decoding delay of the high band with respect to the low band.
  • the preparation step comprises an LPC analysis step, an LTP analysis step and the concealment step comprises a step of calculating a residual LPC signal, a classification step and a step of extrapolation of missing samples.
  • the preparation step comprises an LPC analysis step, an LTP analysis step, a LPC residual signal calculation step and the concealment step includes a classification step and a step of extrapolation of missing samples.
  • the present invention also relates to a transmission error concealment device in a digital signal cut into a plurality of successive frames associated with different time intervals comprising preparation means producing no missing sample and comprising at least analysis means. a valid decoded signal and means of concealment producing the missing samples of the signal corresponding to an erased frame.
  • the device is such that said means are implemented in different time intervals to replace at least the first erased frame after a valid frame.
  • It also relates to a digital signal decoder comprising a transmission error concealment device according to the invention.
  • the invention relates to a computer program intended to be stored in a memory of a transmission error concealment device.
  • This computer program is such that it includes code instructions for carrying out the steps of the error concealment method according to the invention, when executed by a processor of said transmission error concealment device.
  • the erased frame dissimilarization module Upon detection of a first erased frame (lost or erroneous), the erased frame dissimilarization module analyzes the stored stored signal and then synthesizes (or extrapolates) the missing frame using the estimated parameters.
  • the erasure mask module continues to synthesize the missing signal using the same parameters as in the previous extrapolated frame, possibly slightly attenuated.
  • continuity between the extrapolated signal during erasure and the valid decoded signal is provided by a simple and efficient smoothing or "cross-fading" means.
  • This crossfade is performed in the following manner: for a predetermined length of typically 5-10 ms continues to synthesize the extrapolated signal parallel to the decoding of the signal in the valid frame. The output signal is then the weighted sum of these two signals by progressively decreasing the weight of the extrapolated signal and at the same time increasing the weight of the valid signal.
  • Table 1 below illustrates the evolution of the complexity of such an encoder in the case where only one frame (No. 3) is erased.
  • Example of evolution of complexity in the case of an erased frame ⁇ / b> Number of the frame 1 2 3 4 5 6
  • Nature of the frame valid valid deleted valid valid valid valid encoding (0.15 WMOPS) 1 1 1 1 1 1 1 decoding (0.15 WMOPS) 1 1 0 1 1 1 1 analysis (2.5 WMOPS) 0 0 1 0 0 0 extrapolation (0.5 WMOPS) 0 0 1 1 0 0 dissolve fade (0.05 WMOPS) 0 0 0 1 0 0 total complexity WMOPS) 0.3 0.3 3.15 0.85 0.3 0.3
  • the peak of complexity (3.15 WMOPS) can always be observed during the duration of the first erased frame because again the entire process of concealing the erased frame (part of analysis and part of extrapolation) is executed during the duration of a frame, that of the first erased frame.
  • the complexity for the following erased frames is much lower and the average complexity for these six frames is 0.925 WMOPS, slightly higher than in the case of a single erased frame. Increasing the duration of erasure does not significantly increase complexity.
  • the present invention aims to reduce this complexity by distributing the steps of hiding erased frames over the duration of several frames.
  • the figure 1 illustrates a first embodiment of the invention.
  • the concealment method according to the invention comprises at least two steps, a first preparation step (E1) producing no missing sample, a second concealment step (E2) which includes producing missing samples of the signal corresponding to the erased frame. It is specified that in preparation step is meant operations specific to concealment, which would not be necessary for the decoding of only valid frames.
  • FIG. 1 shows an exemplary embodiment in the case where the frame N, received at the decoder is erased.
  • a first N-2 frame received in a bit stream from the communication channel is processed by a demultiplexing module (DEMUX) 14 and is decoded by a normal decoding module (DE-NO ) 15.
  • DEMUX demultiplexing module
  • DE-NO normal decoding module
  • This decoded signal constitutes the N-2 frame referenced at the output of the decoder sent for example to the sound card 24. It is also provided at the input of a preparation module 16 implementing the first preparation step E1. The result of this step is then stored at 17 (MEM).
  • the preparation step is performed for all valid frames in anticipation of a potential erased frame.
  • the second step of concealment E2 is performed taking into account at least one result stored in the previous frames. This second stage of concealment generates missing samples to constitute the N frame referenced 22 at the output of the decoder.
  • step of demultiplexing of normal decoding like all the valid frames but also a "fade-in" step FOND referenced 19 which will allow to smooth the signal decoded between the reconstructed signal for the N frame and the decoded signal for the N + 1 frame.
  • This step of cross-fading consists of continuing in parallel with the normal decoding, extrapolation EXTR referenced 26 of the missing samples of the step E2.
  • the output signal is then the weighted sum of these two signals by progressively decreasing the weight of the extrapolated signal and at the same time increasing the weight of the valid signal.
  • the signal obtained at the output of the decoder is then for example supplied to a sound card 24 to be restored for example by means of loudspeakers 25.
  • the preparation step E1 may for example contain a first part of the analysis, for example LPC analysis and LTP analysis. These analysis steps are particularly detailed in the document "Method of packet errors cancellation for any speech and sound compression scheme" cited above.
  • the concealment step E2 then contains a step of calculating the residual signal LPC (used in the extrapolation phase), of classification of the signal and extrapolation of the missing samples (generation of the excitation signal from the residual signal and synthesis filtering).
  • the step E1 can contain both the LPC, LTP and the calculation of the residual LPC signal, the E2 step then containing the classification and extrapolation step.
  • step E1 can contain both the LPC analysis, the calculation of the residual LPC signal and the first part of the LTP analysis, step E2 then containing the second part of the LTP analysis. LTP analysis, classification and extrapolation.
  • Table 3 illustrates an encrypted example where the first analysis part (analysis_p1) has a complexity of 1.15 WMOPS, the second analysis part (analysis_p2) has a complexity of 1.35 WMOPS, the preparation step E1 containing the first analysis part (analysis_p1) and the concealment step E2 containing the second analysis part (analysis_p2) and the extrapolation (extrapolation).
  • a second embodiment of the invention offers a solution that decreases both the worst case of complexity without increasing the average complexity. So, with reference to the figure 2 a second embodiment is illustrated in the case where the frame N referenced 31 received at the decoder is erased.
  • the preparation step E1 is executed only in the case where a frame is erased and no longer systematically to each valid frame.
  • the preparation step is thus performed in the time interval corresponding to the erased N frame.
  • the output signal of the decoder therefore has a time delay corresponding to a time interval of one frame.
  • the duration of two frames is available to extrapolate the signal replacing this frame N.
  • the preparation step E1 is performed on the frame decoded and stored signal corresponding to the received N-1 frame.
  • the concealment step E2 comprising the extrapolation of the missing samples corresponding to the frame N is carried out in the time interval corresponding to the N + 1 frame, received at the decoder.
  • the N + 1 frame is also processed by the demultiplexing module, decoded and stored for later use in the time interval corresponding to the N + 2 frame during the FADE 19 step of cross fading.
  • the resulting N + 1 frame is sent to the sound card at 43.
  • a corresponding time offset in this exemplary one-frame embodiment is therefore introduced at the output of the decoder. This is generally acceptable in the case for example of a G.711 encoder / decoder which has a very low delay.
  • FIG. 3a A tabular illustration of this second embodiment is also provided in Figure 3a and Figure 3b .
  • the figure 3a shows an example where frame # 4 is erased.
  • the first line 310 shows the numbers of the frames received at the decoder.
  • the second line 311 shows the number of the decoded frame in the buffer.
  • the preparation step is carried out by starting the analysis (analysis_p1) on the decoded past frames (n ° 1 - n ° 3) as shown on line 312. end of the frame No. 4 is sent to the sound card the frame No. 3 previously stored as illustrated in line 316.
  • the buffer or "buffer” in English
  • P2 analysis the second part of the Online analysis
  • the extrapolated frame # 4 can be sent to the sound card.
  • the decoding of the frame No. 5 is done and the result is stored as illustrated in line 311.
  • we extrapolate the frame No. 5 (line 314) for the crossfade with the frame No. 5 memorized (line 315). The result of this fade is sent to the sound card (line 316). Then we decode and memorize the frame n ° 6.
  • the table represented in figure 3b illustrates the case where both frame # 4 and frame # 5 are erased.
  • the frames received at the decoder are illustrated on line 410.
  • line 411 represents decoded frames and stored in the buffer.
  • the first preparation step (analysis_p1) is performed in the time interval of the first deleted frame (line 412).
  • the second part of the analysis (analysis_p2) is carried out in the following time interval, that is to say here in the interval corresponding to the second erased frame (line 413).
  • Line 416 shows the numbers of the decoder output frames with a time shift of one frame relative to the signal received at the decoder.
  • Table 4 illustrates the evolution of the complexity corresponding to the case of the figure 3a . This time the optimal result (the lowest maximum complexity) is obtained by dividing the analysis as follows: • part 1 ⁇ 1.6 WMOPS, • part 2 ⁇ 0.9 WMOPS.
  • the second embodiment thus described is particularly interesting when it is implemented in some decoders, for example in the G.711WB decoder (for G711- WideBand in English, broadband) being formalized.
  • the figure 4 shows an example of an encoder that falls within the scope of the G.711 WB standardization.
  • the encoder input is an audio signal S 16 sampled at 16 kHz.
  • the encoder comprises a quadrature filter bank 101 separating the low band (50-7000 Hz) and the high band (4000-7000 Hz).
  • An intermediate signal (block 102) calculated by a noise feedback loop (blocks 104 and 105) is removed from the low band.
  • the signal is then encoded by a 64 and 80 kbit / s scalable PCM encoder (Co-MIC) (block 103).
  • the high band is coded (block 107-Co-MDCT) after Modified Discrete Cosine Transform (MDCT) (Block 106).
  • the MDCT transformation is an overlap transformation of 50%, which requires knowing the signal in the future N + 1 frame to encode the current frame N.
  • the coding of the high band introduces a delay of 5 ms (called lookahead in English) because of the MDCT transformation.
  • the bit stream T of each frame is then generated by the multiplexer (block 108). This bit stream may be transmitted to a decoder being truncated or erased.
  • the figure 5 shows a corresponding decoder implementing the method of concealing transmission errors according to the invention.
  • the low band decoded by the scalable PCM decoder (DID-MIC) (block 202) is shifted by one frame (block 203) - ie 5 ms.
  • the high band is additionally decoded (blocks 205 and 206) and the two bands are combined after selecting the appropriate branches (blocks 208 and 209) by the quadrature filter bank (block 210).
  • the invention applies here in the case of hiding erased frames in the low band.
  • the normal decoding in the low band is of low complexity since it is a PCM type decoding.
  • the distribution of the complexity of the process of concealing erased frames is then interesting to implement.
  • the process of hiding erased frames is done in at least two steps that are performed in different time intervals.
  • the first step E1 is carried out by means of preparation implemented in the block 204 on the time interval corresponding to the erased frame and the second step is carried out in the time interval corresponding to the following frame by the means of concealment implemented in block 211.
  • a delay of one frame is necessary to temporally align the low band with the high band (block 203).
  • This delay of a frame between low band and high band is here exploited to implement the invention in its second embodiment detailed above with reference to figures 2 , 3a and 3b . It is not necessary to introduce additional delay.
  • the bit stream T associated with the N frame actually contains the low band (LB) codes of the N + 1 frame.
  • the bitstream associated with the N-1 frame actually contains the low band codes of the N-frame.
  • the low band signal of the N frame is decoded and buffered to be given together with the N-1 frame of the high band at the filter bank. 210.
  • the bitstream associated with the N frame is erased, which means that the low band codes of the N + 1 frame are not available.
  • the first preparation step E1 is executed in the low band, taking into account the decoded and memorized signal of the lowband frame N.
  • the sound card receives the N frame of the low band stored in memory.
  • the bitstream associated with the N + 1 frame is received, which means that the low band codes of the N + 2 frame are received. These are decoded and the result is buffered.
  • the concealment step E2 (second part of the analysis and extrapolation of the N + 1 frame) of the concealment algorithm is executed. So we have the low band signal extrapolated in the N + 1 frame to send it to the sound card.
  • the bitstream associated with the N + 2 frame is received.
  • the low band codes of the N + 3 frame are thus decoded and the decoded signal is memorized.
  • the erased frame concealment algorithm continues the extrapolation for the N + 2 frame of the low band so as to perform a cross-fading with N + 2 frame of the buffered low band to ensure continuity between signal extrapolated and decoded signal normally.
  • the present invention is not limited to an application in this type of encoder / decoder. It can also be implemented according to the second mode of implemented in a G.722 encoder / decoder for decoding the low band, particularly when this decoder deals with a frame length of 5 ms.
  • the present invention also relates to a device 70 for concealing a transmission error in a digital signal
  • a device 70 for concealing a transmission error in a digital signal comprising, as represented at 212, the figure 5 , 204 preparation means capable of implementing the first step E1, 211 means of concealment able to implement the second step E2. These means are implemented in different time intervals corresponding to successive signal frames received at the input of the device.
  • this device in the sense of the invention typically comprises, with reference to the figure 7 , a ⁇ P processor cooperating with a memory block BM including a storage and / or working memory, as well as a memory buffer MEM mentioned above as a means for storing the decoded frames and sent with a time offset.
  • This device receives as input successive frames of the digital signal Se and delivers the synthesized signal Ss comprising the samples of an erased frame.
  • the memory block BM may comprise a computer program comprising the code instructions for implementing the steps of the method according to the invention when these instructions are executed by a ⁇ P processor of the device and in particular a first preparation step producing no missing sample and a second concealment step producing the missing samples of the signal corresponding to the erased frame, the two steps being executed in different time intervals.
  • the figures 1 and 2 can illustrate the algorithm of such a computer program.
  • This concealment device according to the invention can be independent or integrated in a digital signal decoder.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Detection And Prevention Of Errors In Transmission (AREA)

Claims (10)

  1. Verfahren zur Verschleierung von Übertragungsfehlern in einem digitalen Signal, das in eine Vielzahl von aufeinanderfolgenden Zeitrahmen aufgeteilt ist, bei dem das Signal beim Empfang gelöschte Rahmen und gültige Rahmen aufweisen kann, und um mindestens den ersten gelöschten Rahmen (N) nach einem gültigen Rahmen zu ersetzen, mindestens zwei Schritte durchgeführt werden, ein erster Schritt (E1) der Vorbereitung, der keine fehlende Tastprobe erzeugt und mindestens eine Analyse eines gültigen decodierten Signals aufweist, um Verschleierungsparameter zu schätzen, und ein zweiter Schritt (E2) der Verschleierung, der die fehlenden Tastproben des Signals entsprechend dem gelöschten Rahmen mit Hilfe der geschätzten Parameter erzeugt, dadurch gekennzeichnet, dass die Rechenbelastung des Prozesses der Verschleierung gelöschter Rahmen verteilt wird, indem die Schritte durchgeführt werden, die den Prozess der Verschleierung von gelöschten Rahmen über unterschiedliche Zeitintervalle bilden, wobei der erste Schritt in dem für die Verarbeitung eines ersten Rahmens reservierten Zeitintervall ausgeführt wird, und der zweite Schritt in dem für die Verarbeitung eines auf den ersten Rahmen folgenden zweiten Rahmens reservierten Zeitintervall ausgeführt wird.
  2. Verfahren nach Anspruch 1, dadurch gekennzeichnet, dass der Vorbereitungsschritt für jeden gültigen decodierten Rahmen in Voraussicht eines potentiellen gelöschten Rahmens durchgeführt wird, und der Verschleierungsschritt in dem für die Verarbeitung des gelöschten Rahmen reservierten Zeitintervall durchgeführt wird.
  3. Verfahren nach Anspruch 1, dadurch gekennzeichnet, dass der Vorbereitungsschritt in dem für die Verarbeitung des gelöschten Rahmens reservierten Zeitintervall durchgeführt wird, und der Verschleierungsschritt in einem für die Verarbeitung des auf den gelöschten Rahmen folgenden Rahmens reservierten Zeitintervall durchgeführt wird.
  4. Verfahren nach Anspruch 3, dadurch gekennzeichnet, dass es bei der Decodierung eines ersten Frequenzbands in einem Decodiersystem angewendet wird, das eine Decodierung in einem ersten Frequenzband und eine Decodierung in einem zweiten Frequenzband aufweist, wobei die Decodierung eines Signal-Zeitrahmens eine zeitliche Verzögerung im zweiten Frequenzband bezüglich der Decodierung dieses gleichen Zeitrahmens im ersten Frequenzband aufweist.
  5. Verfahren nach Anspruch 4, dadurch gekennzeichnet, dass das erste Frequenzband dem Niedrigband einer Decodierung des Typs G.711WB und das zweite Frequenzband dem Hochband einer Decodierung des Typs G.711WB entspricht.
  6. Verfahren nach einem der Ansprüche 1 bis 5, dadurch gekennzeichnet, dass der Vorbereitungsschritt einen Analyseschritt LPC, einen Analyseschritt LTP aufweist, und der Verschleierungsschritt einen Schritt der Berechnung eines Restsignals LPC, einen Schritt der Zuordnung und einen Schritt der Extrapolierung fehlender Tastproben aufweist.
  7. Verfahren nach einem der Ansprüche 1 bis 5, dadurch gekennzeichnet, dass der Vorbereitungsschritt einen Analyseschritt LPC, einen Analyseschritt LTP, einen Schritt der Berechnung eines Restsignals LPC aufweist, und der Verschleierungsschritt einen Schritt der Zuordnung und einen Schritt der Extrapolierung fehlender Tastproben aufweist.
  8. Vorrichtung zur Verschleierung eines Übertragungsfehlers in einem digitalen Signal, das in eine Vielzahl von aufeinanderfolgenden Zeitrahmen aufgeteilt ist, die Vorbereitungseinrichtungen aufweist, die keine fehlende Tastprobe erzeugen, und mindestens Einrichtungen zur Analyse eines gültigen decodierten Signals, um Verschleierungsparameter zu schätzen, und Verschleierungseinrichtungen aufweist, die mit Hilfe der geschätzten Parameter die fehlenden Tastproben des Signals erzeugen, die einem gelöschten Rahmen entsprechen, dadurch gekennzeichnet, dass die Vorbereitungseinrichtungen in dem für die Verarbeitung eines ersten Rahmens reservierten Zeitintervall angewendet werden, und die Verschleierungseinrichtungen in dem für die Verarbeitung eines auf den ersten Rahmen folgenden zweiten Rahmens reservierten Zeitintervall angewendet werden, um die Rechenlast des Prozesses zur Verschleierung gelöschter Rahmen zu verteilen.
  9. Decoder eines digitalen Signals, dadurch gekennzeichnet, dass er eine Vorrichtung zur Verschleierung eines Übertragungsfehlers nach Anspruch 8 aufweist.
  10. EDV-Programm, das dazu bestimmt ist, in einem Speicher einer Vorrichtung zur Verschleierung von Übertragungsfehlern gespeichert zu werden, dadurch gekennzeichnet, dass es Codeanweisungen zur Durchführung der Schritte des Verfahrens nach einem der Ansprüche 1 bis 7 aufweist, wenn es von einem Prozessor der Vorrichtung zur Verschleierung von Übertragungsfehlern ausgeführt wird.
EP08838291A 2007-09-21 2008-09-19 Übertragungsfehlerverdeckung bei einem digitalem signal mit komplexitätsverteilung Active EP2203915B1 (de)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
FR0757750 2007-09-21
PCT/FR2008/051684 WO2009047461A1 (fr) 2007-09-21 2008-09-19 Dissimulation d'erreur de transmission dans un signal numerique avec repartition de la complexite

Publications (2)

Publication Number Publication Date
EP2203915A1 EP2203915A1 (de) 2010-07-07
EP2203915B1 true EP2203915B1 (de) 2012-07-11

Family

ID=39283823

Family Applications (1)

Application Number Title Priority Date Filing Date
EP08838291A Active EP2203915B1 (de) 2007-09-21 2008-09-19 Übertragungsfehlerverdeckung bei einem digitalem signal mit komplexitätsverteilung

Country Status (7)

Country Link
US (1) US8607127B2 (de)
EP (1) EP2203915B1 (de)
JP (2) JP2010539550A (de)
KR (1) KR101450297B1 (de)
CN (1) CN101802906B (de)
ES (1) ES2391360T3 (de)
WO (1) WO2009047461A1 (de)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
RU2660610C2 (ru) * 2014-03-19 2018-07-06 Фраунхофер-Гезелльшафт Цур Фердерунг Дер Ангевандтен Форшунг Е.Ф. Устройство и способ для генерации сигнала маскирования ошибок с использованием индивидуальных замещающих представлений lpc для информации индивидуальных кодовых книг
US10163444B2 (en) 2014-03-19 2018-12-25 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Apparatus and method for generating an error concealment signal using an adaptive noise estimation
US10224041B2 (en) 2014-03-19 2019-03-05 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Apparatus, method and corresponding computer program for generating an error concealment signal using power compensation

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR3024582A1 (fr) * 2014-07-29 2016-02-05 Orange Gestion de la perte de trame dans un contexte de transition fd/lpd

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA2142391C (en) * 1994-03-14 2001-05-29 Juin-Hwey Chen Computational complexity reduction during frame erasure or packet loss
US6952668B1 (en) 1999-04-19 2005-10-04 At&T Corp. Method and apparatus for performing packet loss or frame erasure concealment
WO2000063885A1 (en) * 1999-04-19 2000-10-26 At & T Corp. Method and apparatus for performing packet loss or frame erasure concealment
FR2813722B1 (fr) * 2000-09-05 2003-01-24 France Telecom Procede et dispositif de dissimulation d'erreurs et systeme de transmission comportant un tel dispositif
ATE422744T1 (de) * 2001-04-24 2009-02-15 Nokia Corp Verfahren zum ändern der grösse eines zitterpuffers und zur zeitausrichtung, kommunikationssystem, empfängerseite und transcoder
JP2004239930A (ja) * 2003-02-03 2004-08-26 Iwatsu Electric Co Ltd パケット損失補償におけるピッチ検出方法と装置
JP2004361731A (ja) * 2003-06-05 2004-12-24 Nec Corp オーディオ復号装置及びオーディオ復号方法
DE602005020130D1 (de) * 2004-05-10 2010-05-06 Nippon Telegraph & Telephone E, sendeverfahren, empfangsverfahren und einrichtung und programm dafür
US7359409B2 (en) 2005-02-02 2008-04-15 Texas Instruments Incorporated Packet loss concealment for voice over packet networks
JP2007114417A (ja) * 2005-10-19 2007-05-10 Fujitsu Ltd 音声データ処理方法及び装置
KR100658907B1 (ko) * 2005-12-29 2006-12-15 포스데이타 주식회사 휴대 인터넷 시스템의 기지국 장치 및 호 제어 방법
US7877253B2 (en) * 2006-10-06 2011-01-25 Qualcomm Incorporated Systems, methods, and apparatus for frame erasure recovery
KR100998396B1 (ko) * 2008-03-20 2010-12-03 광주과학기술원 프레임 손실 은닉 방법, 프레임 손실 은닉 장치 및 음성송수신 장치

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
RU2660610C2 (ru) * 2014-03-19 2018-07-06 Фраунхофер-Гезелльшафт Цур Фердерунг Дер Ангевандтен Форшунг Е.Ф. Устройство и способ для генерации сигнала маскирования ошибок с использованием индивидуальных замещающих представлений lpc для информации индивидуальных кодовых книг
US10140993B2 (en) 2014-03-19 2018-11-27 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Apparatus and method for generating an error concealment signal using individual replacement LPC representations for individual codebook information
US10163444B2 (en) 2014-03-19 2018-12-25 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Apparatus and method for generating an error concealment signal using an adaptive noise estimation
US10224041B2 (en) 2014-03-19 2019-03-05 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Apparatus, method and corresponding computer program for generating an error concealment signal using power compensation
US10614818B2 (en) 2014-03-19 2020-04-07 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Apparatus and method for generating an error concealment signal using individual replacement LPC representations for individual codebook information
US10621993B2 (en) 2014-03-19 2020-04-14 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Apparatus and method for generating an error concealment signal using an adaptive noise estimation
US10733997B2 (en) 2014-03-19 2020-08-04 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Apparatus and method for generating an error concealment signal using power compensation
US11367453B2 (en) 2014-03-19 2022-06-21 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Apparatus and method for generating an error concealment signal using power compensation
US11393479B2 (en) 2014-03-19 2022-07-19 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Apparatus and method for generating an error concealment signal using individual replacement LPC representations for individual codebook information
US11423913B2 (en) 2014-03-19 2022-08-23 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Apparatus and method for generating an error concealment signal using an adaptive noise estimation

Also Published As

Publication number Publication date
JP5604572B2 (ja) 2014-10-08
JP2013250582A (ja) 2013-12-12
US8607127B2 (en) 2013-12-10
CN101802906A (zh) 2010-08-11
KR20100084632A (ko) 2010-07-27
CN101802906B (zh) 2013-01-02
KR101450297B1 (ko) 2014-10-13
JP2010539550A (ja) 2010-12-16
ES2391360T3 (es) 2012-11-23
EP2203915A1 (de) 2010-07-07
WO2009047461A1 (fr) 2009-04-16
US20100306625A1 (en) 2010-12-02

Similar Documents

Publication Publication Date Title
EP2277172B1 (de) Verbergung von übertragungsfehlern in einem digitalsignal in einer hierarchischen decodierungsstruktur
EP2080195B1 (de) Synthese verlorener blöcke eines digitalen audiosignals
EP1316087B1 (de) Übertragungsfehler-verdeckung in einem audiosignal
EP1372289B1 (de) Erzeugung einer Rahmenbeschreibung der Stille zur Erzeugung eines angenehmen Hintergrundrauschens
JP5849106B2 (ja) 低遅延の統合されたスピーチ及びオーディオ符号化におけるエラー隠しのための装置及び方法
EP2727107B1 (de) Verzögerungsoptimierte kodierungs-/dekodierungs-gewichtungsfenster durch überlappungstransformation
EP2452336B1 (de) Verbesserte codierung/decodierung digitaler audiosignale
EP2452337A1 (de) Zuweisung von bits bei einer verstärkten codierung/decodierung zur verbesserung einer hierarchischen codierung/decodierung digitaler tonsignale
WO2004070706A1 (fr) Procede de codage et de decodage audio a debit variable
EP0906613B1 (de) Verfahren und vorrichtung zur kodierung eines audiosignals mittels "vorwärts"- und "rückwärts"-lpc-analyse
EP2080194B1 (de) Dämpfung von stimmüberlagerung, im besonderen zur erregungserzeugung bei einem decoder in abwesenheit von informationen
WO1999040573A1 (fr) Procede de decodage d'un signal audio avec correction des erreurs de transmission
EP1356455B1 (de) Verfahren und vorrichtung zur verarbeitung von mehreren audiobitströmen
EP2347411B1 (de) Vor-echo-dämpfung in einem digitalaudiosignal
EP2203915B1 (de) Übertragungsfehlerverdeckung bei einem digitalem signal mit komplexitätsverteilung
EP2795618B1 (de) Verfahren zur erkennung eines vorgegebenen frequenzbandes in einem audiodatensignal, erkennungsvorrichtung und computerprogramm dafür
WO2005024786A1 (fr) Procede de transmission d'un flux d'information par insertion a l'interieur d'un flux de donnees de parole, et codec parametrique pour sa mise en oeuvre
EP2232833A2 (de) Verarbeitung von binärfehlern in einem digitalaudio-binärrahmen
FR2830970A1 (fr) Procede et dispositif de synthese de trames de substitution, dans une succession de trames representant un signal de parole

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20100413

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA MK RS

DAX Request for extension of the european patent (deleted)
17Q First examination report despatched

Effective date: 20110516

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 566475

Country of ref document: AT

Kind code of ref document: T

Effective date: 20120715

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: FRENCH

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602008017181

Country of ref document: DE

Effective date: 20120906

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2391360

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20121123

REG Reference to a national code

Ref country code: NL

Ref legal event code: VDEP

Effective date: 20120711

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 566475

Country of ref document: AT

Kind code of ref document: T

Effective date: 20120711

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

Effective date: 20120725

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20121111

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120711

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120711

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120711

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120711

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20121011

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120711

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120711

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120711

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20121112

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20121012

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120711

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120711

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120711

BERE Be: lapsed

Owner name: FRANCE TELECOM

Effective date: 20120930

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120711

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120711

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20120930

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120711

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120711

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120711

26N No opposition filed

Effective date: 20130412

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20120919

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20121011

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20120930

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20120930

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20120930

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602008017181

Country of ref document: DE

Effective date: 20130412

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120711

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120711

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20120919

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080919

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 9

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 10

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 11

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20230822

Year of fee payment: 16

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20231002

Year of fee payment: 16

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240820

Year of fee payment: 17

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20240820

Year of fee payment: 17

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20240820

Year of fee payment: 17