EP1103953A2 - Method for concealing erased speech frames - Google Patents
Method for concealing erased speech frames Download PDFInfo
- Publication number
- EP1103953A2 EP1103953A2 EP00310360A EP00310360A EP1103953A2 EP 1103953 A2 EP1103953 A2 EP 1103953A2 EP 00310360 A EP00310360 A EP 00310360A EP 00310360 A EP00310360 A EP 00310360A EP 1103953 A2 EP1103953 A2 EP 1103953A2
- Authority
- EP
- European Patent Office
- Prior art keywords
- excitation
- synthesis filter
- frame
- erased
- decoder
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/005—Correction of errors induced by the transmission channel, if related to the coding algorithm
Definitions
- the invention relates to electronic devices, and more particularly, but not exclusively, to speech coding, transmission, storage, and decoding/synthesis methods and circuitry.
- the performance of digital speech systems using low bit rates has become increasingly important with current and foreseeable digital communications.
- Both dedicated channel and packetized-over-network (e.g., Voice over IP or Voice over Packet) transmissions benefit from compression of speech signals.
- the widely-used linear prediction (LP) digital speech coding compression method models the vocal tract as a time-varying filter and a time-varying excitation of the filter to mimic human speech.
- M the order of the linear prediction filter, is taken to be about 10-12; the sampling rate to form the samples s(n) is typically taken to be 8 kHz (the same as the public switched telephone network sampling for digital transmission); and the number of samples ⁇ s(n) ⁇ in a frame is typically 80 or 160 (10 or 20 ms frames).
- a frame of samples may be generated by various windowing operations applied to the input speech samples.
- ⁇ r(n) 2 yields the ⁇ a i ⁇ which furnish the best linear prediction for the frame.
- the coefficients ⁇ a i ⁇ may be converted to line spectral frequencies (LSFs) for quantization and transmission or storage and converted to line spectral pairs (LSPs) for interpolation between subframes.
- LSFs line spectral frequencies
- LSPs line spectral pairs
- the ⁇ r(n) ⁇ is the LP residual for the frame, and ideally the LP residual would be the excitation for the synthesis filter 1/A(z) where A(z) is the transfer function of equation (1).
- the LP residual is not available at the decoder; thus the task of the encoder is to represent the LP residual so that the decoder can generate an excitation which emulates the LP residual from the encoded parameters.
- the LP compression approach basically only transmits/stores updates for the (quantized) filter coefficients, the (quantized) residual (waveform or parameters such as pitch), and (quantized) gain(s).
- a receiver decodes the transmitted/stored items and regenerates the input speech with the same perceptual characteristics.
- Figures 5-6 illustrate high level blocks of an LP system. Periodic updating of the quantized items requires fewer bits than direct representation of the speech signal, so a reasonable LP coder can operate at bits rates as low as 2-3 kb/s (kilobits per second).
- the decoder typically has methods to conceal such frame erasures, and such methods may be categorized as either interpolation-based or repetition-based.
- An interpolation-based concealment method exploits both future and past frame parameters to interpolate missing parameters.
- interpolation-based methods provide better approximation of speech signals in missing frames than repetition-based methods which exploit only past frame parameters.
- the interpolation-based method has a cost of an additional delay to acquire the future frame.
- future frames are available from a playout buffer which compensates for arrival jitter of packets, and interpolation-based methods mainly increase the size of the playout buffer.
- Repetition-based concealment which simply repeats or modifies the past frame parameters, finds use in several CELP-based speech coders including G.729, G.723.1 and GSM-EFR.
- the repetition-based concealment method in these coders does not introduce any additional delay or playout buffer size, but the performance of reconstructed speech with erased frames is poorer than that of the interpolation-based approach, especially in a high erased-frame ratio or bursty frame erasure environment.
- the ITU standard G.729 uses frames of 10 ms length (80 samples) divided into two 5-ms 40-sample subframes for better tracking of pitch and gain parameters plus reduced codebook search complexity.
- Each subframe has an excitation represented by an adaptive-codebook contribution and a fixed (algebraic) codebook contribution.
- the adaptive-codebook contribution provides periodicity in the excitation and is the product of v(n), the prior frame's excitation translated by the current frame's pitch lag in time and interpolated, multiplied by a gain, gp.
- the algebraic codebook contribution approximates the difference between the actual residual and the adaptive codebook contribution with a four-pulse vector, c(n), multiplied by a gain, g c .
- G.729 handles frame erasures by reconstruction based on previously received information; that is, repetition-based concealment. Namely, replace the missing excitation signal with one of similar characteristics, while gradually decaying its energy by using a voicing classifier based on the long-term prediction gain (which is computed as part of the long-term postfilter analysis).
- the long-term postfilter finds the long-term predictor for which the prediction gain is more than 3 dB by using a normalized correlation greater than 0.5 in the optimal delay determination.
- a 10 ms frame is declared periodic if at least one 5 ms subframe has a long-term prediction gain of more than 3 dB. Otherwise the frame is declared nonperiodic.
- An erased frame inherits its class from the preceding (reconstructed) speech frame. Note that the voicing classification is continuously updated based on this reconstructed speech signal. The specific steps taken for an erased frame are as follows:
- An aspect of the present invention provides concealment of erased frames by frame repetition together with one or more of: excitation signal muting, LP coefficient bandwidth expansion with cutoff frequency, and pitch delay jittering.
- Figure 1 shows a preferred embodiment decoder in block format.
- Figure 2 shows known decoder concealment.
- Figure 3 is a block diagram of a known encoder.
- Figure 4 is a block diagram of a known decoder.
- FIGS 5-6 illustrate systems.
- Preferred embodiment decoders and methods for concealment of frame erasures in CELP-encoded speech or other signal transmissions have one or more of three features: (1) muting the excitation outside of the feedback loop, this replaces the attenuation of the adaptive and fixed codebook gains; (2) expanding the bandwidth of the LP synthesis filter with a threshold frequency for differing expansion factors; and (3) jittering the pitch delay to avoid overly periodic repetition frames.
- Features (2) and (3) especially apply to bursty noise leading to frame erasures.
- Figure 1 illustrates a preferred embodiment decoder using all three concealment features; this contrasts with the G.729 standard decoder concealment illustrated in Figure 2.
- Preferred embodiment systems e.g., Voice over IP or Voice over Packet
- Preferred embodiment concealment methods in decoders.
- Figure 3 illustrates a speech encoder using LP encoding with excitation contributions from both adaptive and algebraic codebook, and preferred embodiment concealment features affect the pitch delay, the codebook gains, and the LP synthesis filter. Encoding proceeds as follows:
- the final codeword encoding the (sub)frame would include bits for: the quantized LSF coefficients, adaptive codebook pitch delay, algebraic codebook vector, and the quantized adaptive codebook and algebraic codebook gains.
- Figure 1 illustrates preferred embodiment decoders and decoding methods which essentially reverse the encoding steps of the foregoing encoding method plus provide repetition-based concealment features for erased frame reconstructions as described in the next section.
- Figure 4 shows a decoder without concealment features, and for the m th (sub)frame proceed as follows:
- Figure 1 shows preferred embodiment concealment features in a preferred embodiment decoder and contrasts with Figure 2.
- the preferred embodiment concealment features construct an (m+j) st frame with one or more of the following modified decoder steps:
- Another alternative preferred embodiment omits the pitch delay jittering but may use the incrementing as in G.729 for erased frames together with excitation muting and LP coefficient bandwidth expansion.
- an alternative preferred embodiment omits the excitation muting and uses the G.729 construction together with the pitch delay jittering and synthesis filter coefficient bandwidth expansion.
- preferred embodiments may use just one of the three features (excitation muting, pitch delay jittering, and synthesis filiter coefficient bandwidth expansion) and follow G.729 in other aspects.
- FIGS 5-6 show in functional block form preferred embodiment systems which use the preferred embodiment encoding and decoding. This applies to speech and also other signals which can be effectively CELP coded.
- the encoding and decoding can be performed with digital signal processors (DSPs) or general purpose programmable processors or application specific circuitry or systems on a chip such as both a DSP and RISC processor on the same chip with the RISC processor controlling.
- DSPs digital signal processors
- Codebooks would be stored in memory at both the encoder and decoder, and a stored program in an onboard or external ROM, flash EEPROM, or ferroelectric memory for a DSP or programmable processor could perform the signal processing.
- Analog-to-digital converters and digital-to-analog converters provide coupling to the real world, and modulators and demodulators (plus antennas for air interfaces) provide coupling for transmission waveforms.
- the encoded speech can be packetized and transmitted over networks such as the Internet.
- the preferred embodiments may be modified in various ways while retaining one or more of the features of erased frame concealment by synthesis filter coefficient bandwidth expansion, pitch delay jittering, and excitation muting.
- interval (frame and subframe) size and sampling rate could differ;
- the bandwidth expansion factor could apply for C B > 0 or C B > 2, the multipliers 0.95 and 1.05 and limits 0.8 and 1.0 could vary, and the 100 Hz threshold could vary;
- the pitch delay jitter could be with a larger or smaller percentage of the pitch delay and could also apply to the first erased frame, and the jitter size could vary with the number of consecutive erased frames or erasure density;
- the excitation muting could vary nonlinearly with number of consecutive erased frames or erasure density, and the multipliers 0.95499 and 1.09648 could vary.
- a software-controlled programmable processing device such as a Digital Signal Processor, microprocessor, other processing devices, data processing apparatus or computer system
- a computer program for configuring a programmable device, apparatus or system to implement the foregoing described methods is envisaged as an aspect of the present invention.
- the computer program may be embodied as source code and undergo compilation for implementation on a processing device, apparatus or system, or may be embodied as object code, for example.
- object code for example.
- the term computer in its most general sense encompasses programmable devices such as referred to above, and data processing apparatus and computer systems.
- the computer program is stored on a carrier medium in machine or device readable form, for example in solid-state memory or magnetic memory such as disc or tape and the processing device utilises the program or a part thereof to configure it for operation.
- the computer program may be supplied from a remote source embodied in a communications medium such as an electronic signal, radio frequency carrier wave or optical carrier wave.
- a communications medium such as an electronic signal, radio frequency carrier wave or optical carrier wave.
- carrier media are also envisaged as aspects of the present invention.
Landscapes
- Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
- Transmission Systems Not Characterized By The Medium Used For Transmission (AREA)
- Materials For Medical Uses (AREA)
- Catalysts (AREA)
- Laminated Bodies (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
Description
by frame repetition together with one or more of: excitation signal muting, LP coefficient bandwidth expansion with cutoff frequency, and pitch delay jittering.
γ(n+1) = max(0.95 γ(n), 0.8) if CB > 1 and LSFBWmin < 100 Hz
γ(n+1) = min(1.05 γ(n), 1.0) otherwise
where CB is a bursty frame erasure counter which counts the number of consecutive erased frames, and LSFBWmin is the minimum LSF bandwidth in the last good frame. The ith LSF bandwidth (LSFBWi) is defined as |fi+1 - fi|. The smaller an LSF bandwidth, the sharper the corresponding LPC spectrum peak (formant). That is, LSFBWmin is the minimum LSFBWi, and so the bandwidth expansion factor may decrease only if at least one pair of LSF frequencies are close together (a sharp formant). Note that for γ(n) decreasing the poles of the
if CB > 1, then CM = 4
else if gP (m+1) < 1.0 and CM > 0, then decrement CM by 1
else, no change in CM
where CB again is the bursty counter which counts consecutive number of erased frames and gP (m+1) is the algebraic codebook gain from step (4) Then the gE (n) updating is:
gE (n+1) = 0.95499 gE (n) if CM (n+1) > 0
gE (n+1) = min(1.09648 gE (n) , 1.0) otherwise
Thus the excitation to the synthesis filter becomes gE (m+1) u(m+1)(n). Similarly for the (m+j)th consecutive erased frame using the corresponding gP (m+j) v(m+j)(n) + gC (m+J) c(m+j)(n) and muting with gE (m+j).
Claims (12)
- A method for decoding digital speech, comprising:(a) forming an excitation for an erased interval of encoded digital speech by a sum of an adaptive codebook contribution and a fixed codebook contribution where said adaptive codebook contribution derives from an excitation and pitch and first gain of intervals prior in time of said encoded digital speech and said fixed codebook contribution derives from a second gain of said intervals prior in time;(b) muting said excitation; and(b) filtering said muted excitation.
- The method of claim 1, wherein:
(a) said filtering includes a synthesis with synthesis filter coefficients derived from filter coefficients of said intervals prior in time. - A method for decoding digital speech, comprising:(a) forming a synthesis filter for an erased interval of encoded digital speech by determining filter coefficients from bandwidth expanded versions of filter coefficients of intervals prior in time of said encoded digital speech; and(b) filtering an excitation for said erased interval with said synthesis filter for said erased interval.
- The method of claim 3, wherein:
(a) said filter coefficients a1, a2, ... aM for said synthesis filter for said erased interval are related to said filter coefficients b1, b2 ... bM for said synthesis filter for a interval prior in time by a1 = f b1, a2 = f2 b2, aM = fM bM, where f is a bandwidth expansion factor. - A method for decoding digital speech, comprising:(a) forming an excitation for an erased interval of encoded digital speech by a sum of an adaptive codebook contribution and a fixed codebook contribution where said adaptive codebook contribution derives from an excitation and pitch and first gain of intervals prior in time of said encoded digital speech with said pitch jittered randomly, and said fixed codebook contribution derives from a second gain of said intervals prior in time; and(b) filtering said excitation.
- The method of claim 5, wherein:
(a) said filtering includes a muting followed by a synthesis with synthesis filter coefficients derived from synthesis filter coefficients of said intervals prior in time. - The method of claim 6, further comprising:
(a) determining synthesis filter coefficients for said interval from bandwidth expanded versions of synthesis filter coefficients of intervals prior in time of said encoded digital speech. - A decoder for CELP encoded signals, comprising:(a) a fixed codebook vector decoder;(b) a fixed codebook gain decoder;(c) an adaptive codebook gain decoder;(d) an adaptive codebook pitch delay decoder;(e) an excitation generator coupled to said decoders;(f) a synthesis filter;(g) a muting gain coupled between an output of said excitation generator and an input to said synthesis filter;(h) wherein when a received frame is erased, said decoders generate substitute outputs, said excitation generator generates a substitute excitation, said synthesis filter generates substitute filter coefficients, and said muting gain mutes said substitute excitation.
- The decoder of claim 8, wherein:
(a) said fixed codebook decoder and said adaptive codebook decoder both generate said substitute outputs by repeating the outputs for the prior frame. - A computer program comprising computer- or machine-readable computer program elements for configuring a computer to implement the method of any one of claims 1 to 7.
- A computer program comprising computer- or machine-readable computer program elements translatable for configuring a computer to implement the method of any one of claims 1 to 7.
- A carrier medium carrying a computer program according to claim 10 or 11.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US16719799P | 1999-11-23 | 1999-11-23 | |
| US167197P | 1999-11-23 |
Publications (3)
| Publication Number | Publication Date |
|---|---|
| EP1103953A2 true EP1103953A2 (en) | 2001-05-30 |
| EP1103953A3 EP1103953A3 (en) | 2002-09-11 |
| EP1103953B1 EP1103953B1 (en) | 2006-08-16 |
Family
ID=22606354
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| EP00310360A Expired - Lifetime EP1103953B1 (en) | 1999-11-23 | 2000-11-22 | Method for concealing erased speech frames |
Country Status (4)
| Country | Link |
|---|---|
| EP (1) | EP1103953B1 (en) |
| JP (1) | JP2001154699A (en) |
| AT (1) | ATE336780T1 (en) |
| DE (1) | DE60030069T2 (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7392179B2 (en) | 2000-11-30 | 2008-06-24 | Matsushita Electric Industrial Co., Ltd. | LPC vector quantization apparatus |
| WO2010000303A1 (en) * | 2008-06-30 | 2010-01-07 | Nokia Corporation | Speech decoder with error concealment |
| US7835916B2 (en) | 2003-12-19 | 2010-11-16 | Telefonaktiebolaget Lm Ericsson (Publ) | Channel signal concealment in multi-channel audio systems |
| RU2431892C2 (en) * | 2006-11-10 | 2011-10-20 | Панасоник Корпорэйшн | Parameter decoding device, parameter encoding device and parameter decoding method |
| EP2770504A4 (en) * | 2011-10-20 | 2015-08-05 | Lg Electronics Inc | Method of managing a jitter buffer, and jitter buffer using same |
| CN110474815A (en) * | 2019-09-23 | 2019-11-19 | 北京达佳互联信息技术有限公司 | Bandwidth prediction method, apparatus, electronic equipment and storage medium |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6826527B1 (en) * | 1999-11-23 | 2004-11-30 | Texas Instruments Incorporated | Concealment of frame erasures and method |
| SE527866C2 (en) * | 2003-12-19 | 2006-06-27 | Ericsson Telefon Ab L M | Channel signal masking in multi-channel audio system |
| WO2006098274A1 (en) * | 2005-03-14 | 2006-09-21 | Matsushita Electric Industrial Co., Ltd. | Scalable decoder and scalable decoding method |
| JP5111430B2 (en) * | 2009-04-24 | 2013-01-09 | パナソニック株式会社 | Speech coding apparatus, speech decoding apparatus, and methods thereof |
| JP5328883B2 (en) * | 2011-12-02 | 2013-10-30 | パナソニック株式会社 | CELP speech decoding apparatus and CELP speech decoding method |
| JP6516099B2 (en) * | 2015-08-05 | 2019-05-22 | パナソニックIpマネジメント株式会社 | Audio signal decoding apparatus and audio signal decoding method |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2142391C (en) * | 1994-03-14 | 2001-05-29 | Juin-Hwey Chen | Computational complexity reduction during frame erasure or packet loss |
| WO1996037964A1 (en) * | 1995-05-22 | 1996-11-28 | Ntt Mobile Communications Network Inc. | Sound decoding device |
-
2000
- 2000-11-22 AT AT00310360T patent/ATE336780T1/en not_active IP Right Cessation
- 2000-11-22 DE DE60030069T patent/DE60030069T2/en not_active Expired - Lifetime
- 2000-11-22 JP JP2000356459A patent/JP2001154699A/en not_active Abandoned
- 2000-11-22 EP EP00310360A patent/EP1103953B1/en not_active Expired - Lifetime
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7392179B2 (en) | 2000-11-30 | 2008-06-24 | Matsushita Electric Industrial Co., Ltd. | LPC vector quantization apparatus |
| US7835916B2 (en) | 2003-12-19 | 2010-11-16 | Telefonaktiebolaget Lm Ericsson (Publ) | Channel signal concealment in multi-channel audio systems |
| RU2431892C2 (en) * | 2006-11-10 | 2011-10-20 | Панасоник Корпорэйшн | Parameter decoding device, parameter encoding device and parameter decoding method |
| WO2010000303A1 (en) * | 2008-06-30 | 2010-01-07 | Nokia Corporation | Speech decoder with error concealment |
| EP2770504A4 (en) * | 2011-10-20 | 2015-08-05 | Lg Electronics Inc | Method of managing a jitter buffer, and jitter buffer using same |
| US9324336B2 (en) | 2011-10-20 | 2016-04-26 | Lg Electronics Inc. | Method of managing a jitter buffer, and jitter buffer using same |
| CN110474815A (en) * | 2019-09-23 | 2019-11-19 | 北京达佳互联信息技术有限公司 | Bandwidth prediction method, apparatus, electronic equipment and storage medium |
| US11374825B2 (en) | 2019-09-23 | 2022-06-28 | Beijing Daijia Internet Information Technology Co., Ltd. | Method and apparatus for predicting bandwidth |
Also Published As
| Publication number | Publication date |
|---|---|
| EP1103953B1 (en) | 2006-08-16 |
| DE60030069D1 (en) | 2006-09-28 |
| ATE336780T1 (en) | 2006-09-15 |
| EP1103953A3 (en) | 2002-09-11 |
| JP2001154699A (en) | 2001-06-08 |
| DE60030069T2 (en) | 2007-02-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP1235203B1 (en) | Method for concealing erased speech frames and decoder therefor | |
| US6826527B1 (en) | Concealment of frame erasures and method | |
| RU2419891C2 (en) | Method and device for efficient masking of deletion of frames in speech codecs | |
| CA2177421C (en) | Pitch delay modification during frame erasures | |
| JP5412463B2 (en) | Speech parameter smoothing based on the presence of noise-like signal in speech signal | |
| EP3039676B1 (en) | Adaptive bandwidth extension and apparatus for the same | |
| EP0747883B1 (en) | Voiced/unvoiced classification of speech for use in speech decoding during frame erasures | |
| RU2432625C2 (en) | Synthesis of lost blocks of digital audio signal with pitch period correction | |
| US7606703B2 (en) | Layered celp system and method with varying perceptual filter or short-term postfilter strengths | |
| JP2004508597A (en) | Simulation of suppression of transmission error in audio signal | |
| KR20020052191A (en) | Variable bit-rate celp coding of speech with phonetic classification | |
| HK1040807A1 (en) | Variable rate speech coding | |
| JPWO2001020595A1 (en) | Audio encoding and decoding device | |
| KR20030046451A (en) | Codebook structure and search for speech coding | |
| US6847929B2 (en) | Algebraic codebook system and method | |
| EP1103953A2 (en) | Method for concealing erased speech frames | |
| EP1397655A1 (en) | Method and device for coding speech in analysis-by-synthesis speech coders | |
| JP2853170B2 (en) | Audio encoding / decoding system | |
| JP3274451B2 (en) | Adaptive postfilter and adaptive postfiltering method | |
| JP3071800B2 (en) | Adaptive post filter | |
| WO2001009880A1 (en) | Multimode vselp speech coder | |
| HK1124157A (en) | Method and device for efficient frame erasure concealment in speech codecs | |
| HK1117937A (en) | Variable rate speech coding |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
| AK | Designated contracting states |
Kind code of ref document: A2 Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE TR |
|
| AX | Request for extension of the european patent |
Free format text: AL;LT;LV;MK;RO;SI |
|
| PUAL | Search report despatched |
Free format text: ORIGINAL CODE: 0009013 |
|
| RIC1 | Information provided on ipc code assigned before grant |
Free format text: 7G 10L 19/00 A, 7G 10L 19/12 B |
|
| AK | Designated contracting states |
Kind code of ref document: A3 Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE TR |
|
| AX | Request for extension of the european patent |
Free format text: AL;LT;LV;MK;RO;SI |
|
| 17P | Request for examination filed |
Effective date: 20030226 |
|
| AKX | Designation fees paid |
Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE TR |
|
| 17Q | First examination report despatched |
Effective date: 20050211 |
|
| GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
| GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
| GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
| AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE TR |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT;WARNING: LAPSES OF ITALIAN PATENTS WITH EFFECTIVE DATE BEFORE 2007 MAY HAVE OCCURRED AT ANY TIME BEFORE 2007. THE CORRECT EFFECTIVE DATE MAY BE DIFFERENT FROM THE ONE RECORDED. Effective date: 20060816 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20060816 Ref country code: LI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20060816 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20060816 Ref country code: BE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20060816 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20060816 Ref country code: CH Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20060816 |
|
| REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
| REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
| REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
| REF | Corresponds to: |
Ref document number: 60030069 Country of ref document: DE Date of ref document: 20060928 Kind code of ref document: P |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20061116 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20061116 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20061122 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20061127 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20061130 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20070116 |
|
| NLV1 | Nl: lapsed or annulled due to failure to fulfill the requirements of art. 29p and 29m of the patents act | ||
| REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
| ET | Fr: translation filed | ||
| PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
| 26N | No opposition filed |
Effective date: 20070518 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20061117 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20060816 Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20061122 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20060816 |
|
| REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 16 |
|
| PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20151027 Year of fee payment: 16 Ref country code: DE Payment date: 20151130 Year of fee payment: 16 |
|
| PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20151027 Year of fee payment: 16 |
|
| REG | Reference to a national code |
Ref country code: DE Ref legal event code: R119 Ref document number: 60030069 Country of ref document: DE |
|
| GBPC | Gb: european patent ceased through non-payment of renewal fee |
Effective date: 20161122 |
|
| REG | Reference to a national code |
Ref country code: FR Ref legal event code: ST Effective date: 20170731 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: FR Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20161130 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GB Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20161122 Ref country code: DE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20170601 |