EP0527529B1 - Procédé et appareil pour manipuler la durée d'un signal audio physique et support de données contenant une représentation d'un tel signal audio physique - Google Patents

Procédé et appareil pour manipuler la durée d'un signal audio physique et support de données contenant une représentation d'un tel signal audio physique Download PDF

Info

Publication number
EP0527529B1
EP0527529B1 EP19920202374 EP92202374A EP0527529B1 EP 0527529 B1 EP0527529 B1 EP 0527529B1 EP 19920202374 EP19920202374 EP 19920202374 EP 92202374 A EP92202374 A EP 92202374A EP 0527529 B1 EP0527529 B1 EP 0527529B1
Authority
EP
European Patent Office
Prior art keywords
audio
signal
equivalent signal
audio equivalent
duration
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
EP19920202374
Other languages
German (de)
English (en)
Other versions
EP0527529A3 (en
EP0527529A2 (fr
Inventor
Leonardus Lambertus Maria Vogten
Chang Xue Ma
Werner Desiré Elisabeth Verhelst
Josephus Hubertus Eggen
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Koninklijke Philips NV
Original Assignee
Koninklijke Philips Electronics NV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips Electronics NV filed Critical Koninklijke Philips Electronics NV
Priority to EP19920202374 priority Critical patent/EP0527529B1/fr
Publication of EP0527529A2 publication Critical patent/EP0527529A2/fr
Publication of EP0527529A3 publication Critical patent/EP0527529A3/en
Application granted granted Critical
Publication of EP0527529B1 publication Critical patent/EP0527529B1/fr
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/04—Time compression or expansion

Definitions

  • the invention relates to a method for manipulating an audio equivalent signal, comprising positioning of a chain of mutually overlapping time windows with respect to the audio equivalent signal, as based on periodicity measurements on said audio equivalent signal, and wherein a positional displacement between adjacent windows substantially corresponds to a principal period of said periodicity, and synthesizing an audio output signal by chained superposition of segment signals, each deriving from the audio equivalent signal through weighting with the associated window function.
  • the known method is used during speech synthesis for changing the prosody or pitch of synthesised speech, or to change the duration of stretches of speech.
  • the known method uses voice marks determined manually for placing the windows. It is preferred that such a manipulation method can be performed automatically, is robust against noise, and retains a high audio quality for the output signal.
  • the inventors of the present invention have realized that the manipulation of the duration can be used in various situations where there are external constraints to the total length of a self-contained unit of speech, which constraints may specify both the maximum and the minimum duration of such unit.
  • the object is realized in that the invention is characterized by manipulating a duration of said output signal through systematically repeating, maintaining, and/or suppressing said segment signals, to a resulting predetermined overall length that differs from a corresponding duration of said audio equivalent signal.
  • An advantage of the method of positioning windows according to the junior reference is that it can be machine-executed without any window-to-window human control being necessary. Furthermore, it has been found that the duration can be changed by a factor between 2 and 1 ⁇ 2 without seriously impairing understandability of speech. For lesser degrees of manipulating the duration, such as by + or - 30%, not only remains the understandability very good, but also the natural quality of speech is maintained, and a listener would hardly feel the change of duration as unnatural. A prerequisite to applying the method is that the pitch can indeed be measured, which for human speech is a problem knowing various solutions.
  • the invention relates also to an apparatus for executing the method and to a storage medium containing a representation of audio signal equivalent.
  • the invention allows to fill the available space for a unit of speech (sentence, partial sentence, exclamation, or other) well nigh completely.
  • CD-I Compact Disc Interactive
  • a particular application is Compact Disc Interactive, especially so in a multi-language environment. Editing CD-I is by itself a complicated task. Sizing the duration of speech utterances may now be performed by the machine for relieving the program editor from this tedium.
  • CD-I is a well-published storage medium with associated development platform, the storage itself being an extension from Compact Disc Audio.
  • the audio or speech equivalent signal may be direct analog speech, or it may be speech that is stored as a sequence of codes for on the basis thereon generating synthetic speech.
  • the length of the various windows may be non-uniform, and in a particular embodiment, the length of each window may be substantially equal to a local actual pitch period length.
  • the window function is uniform, which means that the window function scales linearly with the width of the window, which means that generally, there may be an appreciable variation between the widths of successive windows.
  • the systematical character of the repeating, maintaining, or suppression implies that there is a certain prescription for the sequence of window positions that, first, restricts to either repeating or suppressing, either possibly in combination with maintaining, and furthermore, that the repeating or maintaining is done under control of an actual or emulated recurrent cycle. Examples are:
  • the different representations in parallel may be different languages; it has been found that the same sentence, translated to another language, would have different length, counted for example, as a number of syllables: in particular, the German language caused a longer duration as compared with English and French.
  • the pictorial material 200 is shown with accompanying speech representations in French (202), German (204) and English (206) before editing. It is intended to lend each language representation (among which a user may choose) exactly the same duration as the pictorial material (movie, animation, etcetera).
  • a single window is suppressed
  • five windows are suppressed.
  • six windows are repeated one (crosses). The result after editing is not shown. It has been found that analysis of the results can prove infringement. Especially the occurrence of the repeated windows is well traceable.
  • the substantially equal lengths of the various representations is, together with the high subjective quality of the rendering is a clear indication for the use of the present technology.
  • the slowing down or speeding may lend the speech a character, such as nervous (fast) or lively (slow). Also such use is sometimes advantageous.
  • Changing the duration of the audio equivalent signal may be combined with changing the pitch.
  • the two types of manipulation may be both in the same direction, for example in that both effectively shorten the duration. In other circumstances, they could to some degree compensate the effects, so that the change in duration would be less or even be zero.
  • the change of duration may be according to a time-varying pattern, whereby the overall change of duration is the integral or sum of the elementary changes-of-duration.
  • Figures 2a, 2b and 2c show speech signals with marks 52 placed apart by distances determined with a pitch meter (that may be conventional), that is, without a fixed phase reference.
  • a pitch meter that may be conventional
  • two successive periods where marked as voiceless by placing their pitch period length indication outside the scale.
  • the pitch marks (lower scale) where obtained by interpolating the period length.
  • the incremental placement of windows also solves another problem.
  • unvoiced stretches that contain fricatives like the sound “ssss", in which the vocal cords are not excited
  • the windows are placed incrementally just like for voiced stretches.
  • the pitch period length is interpolated between the lengths measured for unvoiced stretches adjacent to the voiced stretch. This provides regularly spaced windows without audible artefacts.
  • the placement of windows is easy if the input audio equivalent signal is monotonous.
  • the windows may be placed simply at fixed distances from each other. This may be effected by preprocessing the signal, so as to change its pitch to a single monotonous value. The final manipulation to obtain a desired pitch and/or duration starting can then be performed with windows at uniform spacing.
  • Figure 3 shows an exemplary embodiment of an apparatus for changing the pitch and/or duration of an audible signal.
  • the input audio equivalent signal arrives at an input 60, and the output signal leaves at an output 63.
  • the input signal is multiplied by the window function in multiplication means 61, and stored segment signal by segment signal in segment slots in storage means 62.
  • speech samples from various segment signals are summed in summing means 64.
  • the manipulation of speech signals in terms of pitch change and/or duration manipulation, is effected by addressing the storage means 62 and selecting window function values. Accordingly, selection of storage addresses for storing the segments is controlled by window position selection means 65, which also control window function value selection means 69; selection of readout addresses is controlled by combination means 66.
  • Figure 4 shows the multiplication means 61 and the window function value selection means 69.
  • the respective t values t a , t b described above are multiplied by the inverse of the period length L i (determined from the period length in an invertor 74) in scaling multipliers 70a, 70b to determine the corresponding arguments of the window function W.
  • These arguments are supplied to window function evaluators 71a, 71b (implemented for example in case of discrete arguments as a lookup table) which outputs the corresponding values of the window function, which are multiplied with the input signal in two multipliers 72a, 72b. This produces the segment signal values Si, Si+1 at two inputs 73a, 73b to the storage means 62.
  • segment signal values are stored in the storage means 62 in segment slots at addresses in the slots corresponding to their respective time point values t a , t b and to respective slot numbers. These addresses are controlled by window position selection means 65. Window position selection means suitable for implementing the invention are shown in Figure 5.
  • the time point values t a , t b are addressed by counters 81, 82, the segment slots numbers are addressed by indexing means 84, (which output the segment indices i, i+1).
  • the counters 81, 82 and the indexing means 84 output addresses with a width as appropriate to distinguish the various positions within the slots and the various slot respectively, but are shown symbolically only as single lines in Figure 5.
  • the two counters 81, 82 are clocked at a fixed clock rate and count from an initial value loaded from a load input (L), upon a trigger signal at trigger input (T).
  • the indexing means 84 increment the index values upon reception of this trigger signal.
  • pitch measuring means 86 determine a pitch value from input 60, and control the scale factor for the scaling multipliers 70a, 70b, and provide the initial value of the first counter 81 (the initial count being minus the pitch value), whereas the trigger signal is generated internally in the window position selection means, once the counter reaches zero, as detected by a comparator 88. This means that successive windows are placed by incrementing the location of a previous window by the time needed by the first counter 81 to reach zero.
  • a monotonized signal is applied to the input 60 (this monotonized signal being obtained by prior processing in which the pitch is adjusted to a time independent value).
  • a constant value, corresponding to the monotonized pitch is fed as initial value to the first counter 81.
  • the scaling multipliers 70a, 70b can be omitted since the windows have a fixed size.
  • the combination means 66 of Figure 3 are shown in Figure 10.
  • the sum being limited to index values i for which -L i ⁇ t-T i ⁇ L i+1 ; in principle, any number of index values may contribute to the sum at one time point t. But when the pitch is not changed by more than a factor of 3/2, at most 3 index values will contribute at a time.
  • Figures 3 and 7 show an apparatus which provides for only three active indices at a time; extension to more than three segments is straightforward.
  • the combination means 66 are quite similar to the input side: they comprise three counters 101, 102, 103 (clocked at a fixed rate), outputting the time point values t-T i for the three segment signals.
  • the three counters receive the same trigger signal, which triggers loading of minus the desired output pitch interval in the first of the three counters 101.
  • the trigger signal is generated by a comparator 104, which detects zero crossing of the first counter 101.
  • the trigger signal also updates indexing means 106.
  • the indexing means address the segment slot numbers which must be read out and the counters address the position within the slots.
  • the counters and indexing means address three segments, which are output from the storage means 62 to the summing means 64 in order to produce the output signal.
  • the duration of the speech signal is controlled by a duration control input 68b to the indexing means. Without duration manipulation, the indexing means simply produce three successive segment slot numbers.
  • the values of the first and second output are copied to the second an third output respectively, and the first output is increased by one.
  • the duration is increased, the first output is kept constant once every so many cycles, as determined by the duration control input 68b.
  • the first output is increased by two every so many cycles. The change in duration is determined by the net number of skipped or repeated indices.
  • Figure 3 only provides one embodiment by way of example.
  • the principal point is the incremental placement of windows at the input side with a phase determined from the phase of a previous window.
  • the addresses may be generated using a computer program, and the starting addresses need not have the values given in the example.
  • Figure 3 can be implemented in various ways, for example using digital samples at input 60, where the sampling rate has at any convenient value, for example 10000 samples per second; conversely, it may use continuous signal techniques, where the clocks 81, 82, 101, 102, 103 provide continuous ramp signals, and the storage means provide for continuously controlled access like a magnetic disk. Furthermore, in Figure 3 in practice segment slots may be reused after some time, as they are not needed permanently. Not all components of Figure 4 need to be implemented by discrete function blocks: often it may be implemented in whole or part by a computer.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Quality & Reliability (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Signal Processing For Digital Recording And Reproducing (AREA)

Claims (7)

  1. Procédé de manipulation d'un signal audio équivalent, comprenant : le positionnement d'une chaíne de fenêtres temporelles se chevauchant mutuellement par rapport au signal audio équivalent, dans lequel un déplacement de position entre des fenêtres adjacentes correspond sensiblement à une période principale sur la base de mesures de périodicité sur ledit signal audio équivalent;
    la formation de signaux segmentés Si, chacun dérivant du signal audio équivalent par pondération avec une fonction de fenêtre de la fenêtre associée Wi; et la synthèse d'un signal audio de sortie par superposition chaínée des signaux segmentés, caractérisé en ce que :
    l'étape de positionnement de la chaíne de fenêtres temporelles se chevauchant mutuellement comprend le décalage de chaque fenêtre Wi par rapport à une fenêtre précédente Wi-1 dans la chaíne sur une longueur de période tonale réelle Li dudit signal audio équivalent, la fenêtre Wi ayant une fonction de fenêtre formée par extension linéaire d'une première moitié d'une fonction de fenêtre normalisée par Li et une seconde moitié de la fonction de fenêtre normalisée par Li+1; et
    la manipulation d'une durée dudit signal de sortie se fait par répétition, maintien et/ou suppression systématiques desdits signaux segmentés, à une longueur prédéterminée du matériau pictural correspondant audit signal audio équivalent, ladite longueur différant d'une durée dudit signal audio équivalent.
  2. Procédé selon la revendication 1, dans lequel ladite longueur prédéterminée s'applique à une pluralité de signaux équivalents vocaux en parallèle dont le contenu correspond, mais ont des différences de représentation.
  3. Procédé selon la revendication 2, dans lequel lesdites différences proviennent de ladite pluralité de signaux audio équivalents qui sont dans autant de langues différentes.
  4. Procédé selon la revendication 1, 2 ou 3, dans lequel ladite longueur prédéterminée appartient à une pause entre les signaux audio équivalents non manipulés.
  5. Procédé selon l'une quelconque des revendications 1 à 4 pour la postsynchronisation de la parole humaine, telle qu'elle est caractérisée par un article vidéo représentable.
  6. Procédé de production d'un titre de logiciel à partir d'un matériel pictural prédéterminé et d'au moins un signal audio équivalent correspondant, le procédé comprenant :
    la manipulation du signal audio équivalent en positionnant une chaíne de fenêtres temporelles se chevauchant mutuellement par rapport au signal audio équivalent, sur la base de mesures de périodicité sur ledit signal audio équivalent, un déplacement de position entre des fenêtres adjacentes correspondant sensiblement à une période principale de ladite périodicité, l'obtention de signaux segmentés à partir du signal audio équivalent par pondération avec la fonction de la fenêtre associée; et la synthèse d'un signal audio de sortie par superposition chaínée desdits signaux segmentés, une durée dudit signal de sortie audio étant manipulée sur une longueur prédéterminée du matériau pictural par répétition, maintien et/ou suppression systématiques desdits signaux segmentés, ladite longueur différant d'une durée dudit signal audio équivalent, et
    le stockage du matériel pictural et du signal de sortie audio obtenu dans un support de stockage unitaire pour une reproduction synchronisée.
  7. Appareil pour manipuler un signal audio équivalent, l'appareil comprenant :
    des moyens pour positionner une chaíne de fenêtres temporelles se chevauchant mutuellement par rapport au signal audio équivalent, sur la base de mesures de périodicité sur ledit signal audio équivalent, en décalant chaque fenêtre Wi par rapport à une fenêtre précédente Wi-1 dans la chaíne sur une longueur de période tonale réelle Li dudit signal audio équivalent, la fenêtre Wi ayant une fonction de fenêtre formée par extension linéaire d'une première moitié d'une fonction de fenêtre normalisée par Li et une seconde moitié de la fonction de fenêtre normalisée de Li+1, et
    des moyens pour obtenir des signaux segmentés à partir du signal audio équivalent par pondération avec la fonction de fenêtre associée, et
    des moyens pour synthétiser un signal audio de sortie par superposition chaínée desdits signaux segmentés par manipulation d'une durée dudit signal de sortie à une longueur prédéterminée du matériel pictural correspondant audit signal audio équivalent par répétition, maintien et/ou suppression systématiques desdits signaux segmentés, ladite longueur différant d'une durée dudit signal audio équivalent.
EP19920202374 1991-08-09 1992-07-31 Procédé et appareil pour manipuler la durée d'un signal audio physique et support de données contenant une représentation d'un tel signal audio physique Expired - Lifetime EP0527529B1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP19920202374 EP0527529B1 (fr) 1991-08-09 1992-07-31 Procédé et appareil pour manipuler la durée d'un signal audio physique et support de données contenant une représentation d'un tel signal audio physique

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
EP91202044 1991-08-09
EP91202044 1991-08-09
EP92200521 1992-02-24
EP92200521 1992-02-24
EP19920202374 EP0527529B1 (fr) 1991-08-09 1992-07-31 Procédé et appareil pour manipuler la durée d'un signal audio physique et support de données contenant une représentation d'un tel signal audio physique

Publications (3)

Publication Number Publication Date
EP0527529A2 EP0527529A2 (fr) 1993-02-17
EP0527529A3 EP0527529A3 (en) 1993-05-05
EP0527529B1 true EP0527529B1 (fr) 2000-07-19

Family

ID=27234119

Family Applications (1)

Application Number Title Priority Date Filing Date
EP19920202374 Expired - Lifetime EP0527529B1 (fr) 1991-08-09 1992-07-31 Procédé et appareil pour manipuler la durée d'un signal audio physique et support de données contenant une représentation d'un tel signal audio physique

Country Status (1)

Country Link
EP (1) EP0527529B1 (fr)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001513225A (ja) 1997-12-19 2001-08-28 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ 伸長オーディオ信号からの周期性の除去
WO1999059139A2 (fr) 1998-05-11 1999-11-18 Koninklijke Philips Electronics N.V. Codage de la parole base sur la determination d'un apport de bruit du a un changement de phase
DE69932786T2 (de) 1998-05-11 2007-08-16 Koninklijke Philips Electronics N.V. Tonhöhenerkennung
EP1628288A1 (fr) * 2004-08-19 2006-02-22 Vrije Universiteit Brussel Procédé et système pour la synthèse de son

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS58102298A (ja) * 1981-12-14 1983-06-17 キヤノン株式会社 電子機器
CA1204855A (fr) * 1982-03-23 1986-05-20 Phillip J. Bloom Methode et appareil utilises dans le traitement des signaux
US5055939A (en) * 1987-12-15 1991-10-08 Karamon John J Method system & apparatus for synchronizing an auxiliary sound source containing multiple language channels with motion picture film video tape or other picture source containing a sound track
FR2636163B1 (fr) * 1988-09-02 1991-07-05 Hamon Christian Procede et dispositif de synthese de la parole par addition-recouvrement de formes d'onde

Also Published As

Publication number Publication date
EP0527529A3 (en) 1993-05-05
EP0527529A2 (fr) 1993-02-17

Similar Documents

Publication Publication Date Title
US5611002A (en) Method and apparatus for manipulating an input signal to form an output signal having a different length
US5479564A (en) Method and apparatus for manipulating pitch and/or duration of a signal
US5828994A (en) Non-uniform time scale modification of recorded audio
US7277856B2 (en) System and method for speech synthesis using a smoothing filter
US6950798B1 (en) Employing speech models in concatenative speech synthesis
EP0561752B1 (fr) Méthode et arrangement pour la synthèse du langage
EP0527529B1 (fr) Procédé et appareil pour manipuler la durée d'un signal audio physique et support de données contenant une représentation d'un tel signal audio physique
US20060074678A1 (en) Prosody generation for text-to-speech synthesis based on micro-prosodic data
JP2002108382A (ja) リップシンクを行うアニメーション方法および装置
JP3728173B2 (ja) 音声合成方法、装置および記憶媒体
Dutilleux et al. Time‐segment Processing
US4092495A (en) Speech synthesizing apparatus
JP3394281B2 (ja) 音声合成方式および規則合成装置
van Santen Quantitative modeling of pitch accent alignment
JP2583883B2 (ja) 音声分析装置および音声合成装置
JP2785628B2 (ja) ピッチパタン生成装置
Rodet Sound analysis, processing and synthesis tools for music research and production
Tychtl Phase-mismatch-free and data efficient approach to natural sounding harmonic concatenative speech synthesis
Nebbia et al. Eight-channel digital speech synthesizer based on LPC techniques
JP2573587B2 (ja) ピッチパタン生成装置
Reddy SPEECH ANALYSIS-SYNTHESIS FOR SPEAKER CHARACTERISTIC MODIFICATION
EP1256933A2 (fr) Procédé et dispositif de contrôle d'un appareil de synthèse d'émotion
JPH01283600A (ja) 残差駆動型音声合成装置
Ding Articulatory speech synthesis
JPH04280B2 (fr)

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): DE FR GB IT

PUAL Search report despatched

Free format text: ORIGINAL CODE: 0009013

AK Designated contracting states

Kind code of ref document: A3

Designated state(s): DE FR GB IT

17P Request for examination filed

Effective date: 19931026

17Q First examination report despatched

Effective date: 19970221

RAP3 Party data changed (applicant data changed or rights of an application transferred)

Owner name: KONINKLIJKE PHILIPS ELECTRONICS N.V.

GRAG Despatch of communication of intention to grant

Free format text: ORIGINAL CODE: EPIDOS AGRA

GRAG Despatch of communication of intention to grant

Free format text: ORIGINAL CODE: EPIDOS AGRA

GRAH Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOS IGRA

RIC1 Information provided on ipc code assigned before grant

Free format text: 7G 10L 21/04 A

GRAH Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOS IGRA

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): DE FR GB IT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT;WARNING: LAPSES OF ITALIAN PATENTS WITH EFFECTIVE DATE BEFORE 2007 MAY HAVE OCCURRED AT ANY TIME BEFORE 2007. THE CORRECT EFFECTIVE DATE MAY BE DIFFERENT FROM THE ONE RECORDED.

Effective date: 20000719

REF Corresponds to:

Ref document number: 69231266

Country of ref document: DE

Date of ref document: 20000824

ET Fr: translation filed
PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed
REG Reference to a national code

Ref country code: GB

Ref legal event code: IF02

REG Reference to a national code

Ref country code: GB

Ref legal event code: 732E

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20031224

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20031231

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20040115

Year of fee payment: 12

REG Reference to a national code

Ref country code: FR

Ref legal event code: TP

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20040731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20050201

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20040731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20050331

REG Reference to a national code

Ref country code: FR

Ref legal event code: ST