EP1743324B1 - Dispositif et procede pour analyser un signal d'information - Google Patents

Dispositif et procede pour analyser un signal d'information Download PDF

Info

Publication number
EP1743324B1
EP1743324B1 EP05744658A EP05744658A EP1743324B1 EP 1743324 B1 EP1743324 B1 EP 1743324B1 EP 05744658 A EP05744658 A EP 05744658A EP 05744658 A EP05744658 A EP 05744658A EP 1743324 B1 EP1743324 B1 EP 1743324B1
Authority
EP
European Patent Office
Prior art keywords
time
short
spectra
spectrum
information signal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Not-in-force
Application number
EP05744658A
Other languages
German (de)
English (en)
Other versions
EP1743324A1 (fr
Inventor
Christian Dittmar
Christian Uhle
Jürgen HERRE
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Original Assignee
Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV filed Critical Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Publication of EP1743324A1 publication Critical patent/EP1743324A1/fr
Application granted granted Critical
Publication of EP1743324B1 publication Critical patent/EP1743324B1/fr
Not-in-force legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use

Definitions

  • the present invention relates to the analysis of information signals, such as audio signals, and more particularly to the analysis of information signals consisting of a superposition of sub-signals, wherein a sub-signal may originate from a single source or a group of individual sources.
  • the aim is also to "enrich" audio data with metadata to z. B. to recover a piece of music based on a fingerprint metadata.
  • the "fingerprint” should on the one hand be meaningful, and on the other hand be as short and concise as possible. "Fingerprint” thus refers to a compressed generated from a music signal Information signal, which does not contain the metadata, but for referencing to the metadata eg by searching in a database is used, for example in a system for the identification of audio material ("AudioID").
  • music data consists of the superposition of sub-signals from single sources. While pop music typically has relatively few individual sources, namely the singer, the guitar, the bass guitar, the drums, and a keyboard, the number of sources for an orchestral piece can become very large.
  • An orchestral piece and a pop music piece for example, consist of a superposition of the tones emitted by the individual instruments.
  • An orchestral piece or piece of music thus represents a superimposition of partial signals from individual sources, the partial signals being the sounds produced by the individual instruments of the orchestra or pop music ensemble, and the individual instruments being individual sources.
  • groups of original sources can also be considered as individual sources, so that at least two individual sources can be assigned to one signal.
  • An analysis of a general information signal is shown below by way of example only with reference to an orchestra signal.
  • the analysis of an orchestra signal can be done in many ways.
  • Further possibilities of the analysis are to extract a dominant rhythm, whereby a rhythm extraction on the basis of the percussion instruments better rather than on the basis of the more sound-giving instruments, which are also referred to as harmonic-sustained or "harmonic sustained" instruments.
  • harmonic-sustained or "harmonic sustained” instruments While percussion instruments typically include timpani, drums, rattles or other percussion instruments, the harmonic sustained instruments include all other instruments such as violins, wind instruments, etc.
  • the percussion instruments include all those acoustic or synthetic tone generators that contribute to the rhythm section due to their sound characteristics (e.g., rhythm guitar).
  • rhythm extraction of a piece of music it would be desirable for the rhythm extraction of a piece of music to extract only percussive parts from the entire piece of music and then perform rhythm recognition on the basis of these percussive parts, without the rhythm recognition being "disturbed” by signals from the harmonically sustained instruments.
  • any analysis aimed at extraction of metadata that requires only information from the harmonic sustained instruments e.g., a harmonic or melodic analysis
  • BSS Blind Source Separation
  • ICA Independent Component Analysis
  • the term BSS includes techniques for separating signals from a mix of signals with a minimum of prior knowledge of the nature of the signals and the mixing process.
  • the ICA is a method that makes use of the assumption that the sources underlying a mix are, at least to a degree, statistically independent of each other.
  • the mixing process is assumed to be fixed in time and the number of mixed signals observed is not less than the number of source signals underlying the mixture.
  • a method for the separation of single sources from mono audio signals is presented.
  • an application for a separation into single tracks and then the rhythm analysis is given.
  • a component analysis is performed to obtain a separation into percussive and non-percussive sounds of a polyphonic piece.
  • ICA Independent Component Analysis
  • amplitude bases which are obtained by means of generally calculated frequency bases from a spectrogram representation of a drum track. This is done for the purpose of transcription.
  • this method is extended to polyphonic music pieces.
  • an exhaustive pairwise similarity computation is performed over all the component signals, resulting in a similarity matrix in which all component signals are plotted along a y-axis, and in which all component signals are also plotted along the x-axis.
  • This two-dimensional array provides a similarity measure for each component signal, each time with a different component signal.
  • the Ixegram ie the two-dimensional matrix, is now used to perform a clustering, for which a grouping is performed using a cluster algorithm based on diadic data.
  • a cost function is defined that measures the compactness within a cluster and determines the homogeneity between clusters.
  • the cost function is minimized, so that ultimately results in an assignment of individual components to individual subspaces.
  • the subspan results in the speaker, the reconstructed information signal of the speaker subspace showing significant attenuation of the waterfall noise.
  • a disadvantage of the concepts described is the fact that the case occurs very likely that the signal components of a source come to lie on different component signals. This is why, as stated above, a complex and computation-intensive similarity calculation among all the component signals is performed to obtain the two-dimensional similarity matrix, on the basis of which, by means of a cost function to be minimized, then a division of component signals into subspaces is carried out.
  • the Independent Subspace Analysis can thus be used to decompose a time-frequency representation, such as a spectrogram, of an audio signal into independent component spectra.
  • a time-frequency representation such as a spectrogram
  • the previous methods described above rely either on a calculation-intensive determination of frequency and amplitude bases from the entire spectrogram or on a priori defined frequency bases.
  • a priori defined frequency bases or profile spectra consist for example in that one says that a trumpet is most likely to be found in one piece, and then an example spectrum of a trumpet is used for signal analysis.
  • a spectrogram typically consists of a sequence of individual spectra, wherein a hopping period is defined between the individual spectra, and wherein a spectrum represents a specific number of samples, so that a spectrum has a specific time length, ie a block of samples of the signal.
  • the duration represented by the block of samples from which a spectrum is computed is represented will be much larger than the hopping time to obtain a satisfactory spectrogram in view of the required frequency resolution and time resolution required.
  • this spectrogram representation is extremely redundant.
  • each sample occurs in 10 consecutive spectra.
  • the redundancy generated thereby can drive the computational time requirements to astronomical heights, especially when a larger number of instruments are being searched for.
  • the approach of working on the basis of the entire spectrogram is disadvantageous for cases in which not all the sources contained in a signal are to be extracted, but only, for example, sources of a certain type, that is to say sources having a specific characteristic .
  • a characteristic may involve percussive sources, ie percussion instruments, or so-called pitched instruments, also referred to as harmonic sustained instruments, which are typical melody instruments such as trumpet, violin, etc.
  • a method that works on the basis of all these sources is then too time-consuming and ultimately also not robust enough, for example, if only a few sources, namely the sources that are to fulfill a specific characteristic, are to be extracted.
  • the object of the present invention is to provide a robust and computationally efficient concept for analyzing an information signal.
  • the present invention is based on the finding that a robust and efficient information signal analysis is achieved by first extracting significant short-term spectra or short-term spectra derived from significant short-term spectra, such as difference spectra etc. from the entire information signal or from the spectrogram of the information signal, respectively Short-term spectra are extracted, which come closer to a specific characteristic than other short-term spectra of the information signal.
  • short-term spectra having percussive portions are extracted, and thus short-term spectra having harmonic components are not extracted.
  • the specific characteristic is a percussive or percussion characteristic.
  • the extracted short-term spectra or short-term spectra derived from the extracted short-term spectra are then fed to means for decomposing the short-term spectra into component signal spectra, a component signal spectrum representing a profile spectrum of a sound source, which generates a tone which corresponds to the desired characteristic, and wherein another component signal spectrum represents a different profile spectrum of a sound source which generates a sound which also corresponds to the desired characteristic.
  • an amplitude envelope is calculated over time, whereby the profile spectra and the original short-term spectra are used for the calculation of the amplitude envelope over time, so that for each time point at which a short-term spectrum was taken off Amplitude value is obtained.
  • the information thus obtained namely different profile spectra and amplitude envelopes for the profile spectra, provides a complete description of the music or information signal with respect to the specified characteristic which has been extracted so that this information may already be sufficient
  • To make a transcription so to first identify with the concepts of feature extraction and segmentation, which instrument "belongs" to the profile spectrum, and what rhythmic is present, so what climbs and drop events are present that indicate notes of this instrument played at certain times.
  • the present invention is advantageous in that for calculating the component analysis, that is to say for disassembling, not the entire spectrogram is used, but only extracted short-term spectra, ie that the calculation of the independent subspace analysis (ISA) takes place only on the basis of a subset of all spectra, so that the computational requirements be lowered. It also increases the robustness of finding certain sources, especially other short-term spectra that do not meet the specified characteristics are not present in the component analysis and thus do not represent any disturbance or "blurring" of the actual spectra.
  • ISA independent subspace analysis
  • the inventive concept is advantageous in that the profile spectra are determined directly from the signal, without resulting in the problem of prefabricated profile spectra, which in turn would lead to either inaccurate results or to increased computational effort.
  • the concept according to the invention is used to detect and classify percussive, non-harmonic instruments in polyphonic audio signals in order to obtain profile spectra as well as amplitude envelopes for the individual profile spectra.
  • Fig. 1 shows a preferred embodiment of an inventive apparatus for analyzing an information signal supplied via an input line 10 to a device 12 for providing a sequence of short-term spectra representing the information signal.
  • the information signal may also be supplied, for example, in time, to means 16 for extracting significant short-term spectra or short-term spectra derived from the short-term spectra from the information signal is designed for extracting to extract such short-term spectra, which come closer to a specific characteristic than other short-term spectra of the information signal.
  • the extracted spectra ie the original short-term spectra or the short-term spectra derived from the original short-term spectra, for example by differentiating, differentiating and rectifying or by other operations, are fed to a device 18 for decomposing the extracted short-term spectra into component signal spectra, wherein a component signal spectrum represents a profile spectrum of a sound source, which generates a sound corresponding to the characteristic sought, and wherein another profile spectrum represents another sound source which generates a sound which also corresponds to the sought characteristic.
  • the profile spectra are finally fed to an amplitude envelope calculating means 20 for the one sound source, the amplitude envelope indicating how the profile spectra of a sound source change over time, and in particular how the intensity or weighting of a profile spectrum changes over time.
  • the device 20 is designed to work on the basis of the sequence of short-term spectra on the one hand and on the basis of the profile spectra on the other hand, as can be seen from FIG.
  • the means 20 for calculating provides amplitude envelopes for the sources, while the means 18 provides profile spectra for the sound sources.
  • the profile spectra and the associated amplitude envelopes provide a complete description of the portion of the information signal that corresponds to the specific characteristic.
  • this portion is the percussive portion of a piece of music.
  • this share could also be the harmonic component.
  • the means for extracting significant short-term spectra would be designed differently, as in the case where the specific characteristic is a percussive characteristic.
  • FIG. 2 a preferred embodiment of the present invention is shown.
  • a detection and classification of percussive, non-harmonic instruments is performed, as also shown by a block 22 in FIG. This will be discussed later.
  • the means 12 for providing a train of short-term spectra is adapted to provide, by means of an appropriate time-frequency transformation to generate an amplitude spectrogram X.
  • the time / frequency device 12 is preferably a device for performing a short-time Fourier transform with a certain hopping period, or comprises filter banks.
  • a phase spectrogram is also obtained as an additional information source, as shown in FIG. 2 by a phase arrow 13.
  • a difference spectrogram ⁇ is obtained, as represented by the differentiator 16a.
  • the negative components resulting from differentiation are set to zero or - alternatively - made positive. This results in a non-negative difference spectrogram ⁇ .
  • the difference spectrogram is fed to a maximum searcher 16c, which is designed to search for the occurrence of local maxima in a detection function e, which is calculated before the maximum searcher 16c, after the times t, that is, for the indices of the corresponding spectrogram columns.
  • the detection function can be obtained, for example, by summing over all rows of X and then smoothing.
  • phase information provided via phase line 13 from block 12 to block 16c is used as an indicator of the reliability of the found maxima.
  • the spectra for which the maximum seeker detects a maximum in the detection function are used as X t and represent the extracted ones
  • a Principle Component Analysis is performed.
  • a sought number of components d is first set.
  • the PCA is performed by a suitable method such as Singular Value Decomposition or Eigenvalue Decomposition over the columns of the matrix X t .
  • X ⁇ X ⁇ t ⁇ T
  • the transformation matrix T effects a dimensional reduction on X, which results in a reduction in the number of columns of this matrix. Furthermore, a decorrelation and variance normalization is achieved.
  • a non-negative independent component analysis is then performed.
  • the method of non-negative Independent Component Analysis shown in [6] is performed on X to calculate a separation matrix A. According to the following equation, X is decomposed into independent components.
  • F A ⁇ X ⁇
  • the amplitude base is interpreted as a set of time-varying amplitude envelopes of the corresponding spectral profiles.
  • the spectral profile is obtained from the music signal itself.
  • the computational complexity compared to the previous method is reduced, and it is achieved a higher robustness compared to stationary signal components, ie signal components due to Harmonic Sustained instruments.
  • a feature extraction and a classification operation are then performed.
  • the components are divided into two subsets, namely first in a subset with the properties not percussive, so quasi-harmonic, and in another percussive subset ..
  • the components with the property percussive / dissonant further classified in various instrument classes.
  • a decision can then be made for drum inserts or an acceptance or acceptance of percussive maxima.
  • maxima with a transient rise in the amplitude envelope above a variable threshold are assumed to be a percussive event, while maxima having a transient rise below the variable threshold are discarded or recognized as an artifact and ignored.
  • the variable threshold preferably varies with the total amplitude over a larger range around the maximum.
  • the output is in an appropriate form that associates with the time of percussive events an instrument class, an intensity, and possibly other information, such as note or rhythm information in MIDI format.
  • the means 16 for extracting significant short-term spectra may be designed to perform this extraction on the basis of actual short-time spectra, as obtained, for example, in a short-time Fourier transformation.
  • the specific characteristic is the percussion characteristic
  • the differentiation results in the sequence of short-term spectra into a sequence of derived or differentiated spectra, each (differentiated) short-term spectrum now containing the changes between an original spectrum and the next spectrum.
  • the PCA 18a and the non-negative ICA 18b that is, more generally, the decomposition operation for decomposing the extracted short-term spectra in the block 18 of FIG. 1 not with the original short-term spectra but with the derived short-time spectra.
  • the effect is exploited that for strongly transient signals the differentiated signal is very similar to the original signal before differentiation, which is the case in particular when there are very rapid changes in a signal. This applies to percussive instruments.
  • typical digital audio signals are initially preprocessed by preprocessing the device 8. Further, as a PCM audio signal inputted to the preprocessing means 8, it is preferable to supply 16-bit-per-second-wide mono-files at a sampling frequency of 44.1 Hz. These audio signals, that is, this stream of audio samples, which may also be a stream of video samples and generally a stream of information samples, are fed to pre-processor 8 for time-domain preprocessing using software-based emulation an acoustic effect device, often referred to as an "exciter". In this concept, the pre-processing stage 8 amplifies the high-frequency portion of the audio signal.
  • a spectral representation of the preprocessed time signal is then obtained using the time / frequency means 12, which preferably performs a short-time Fourier transform (STFT).
  • STFT short-time Fourier transform
  • a relatively large block size of preferably 4096 values and high overlap are preferred.
  • the temporal resolution is increased to a desired accuracy by obtaining a small hop size, that is, a small hop interval between adjacent blocks.
  • a small hop size that is, a small hop interval between adjacent blocks.
  • 4096 samples per block subjected to a short-time Fourier transform which corresponds to a time block length of 92 ms.
  • the hop size is 10 ms. This means that each sample occurs over 9 consecutive times in a short-term spectrum.
  • the device 12 is designed to obtain an amplitude spectrum X.
  • the phase information can also be calculated and, as will be explained later, used in the extreme value or maximum searcher 16c.
  • the magnitude spectrum X now has n frequency bins or frequency coefficients and m columns or frames, ie individual short-time spectra.
  • the time-variant changes of each spectral coefficient are differentiated over all frames, by differentiator 16a, to minimize the impact of harmonic-sustaining sound sources, and to facilitate the subsequent detection of transients.
  • the differentiation which preferably has a difference between two short-term spectra of the sequence, may also have certain normalizations.
  • the maximum searcher 16c performs event detection, which will be discussed below.
  • the detection of multiple local extremes, and preferably local maxima associated with transient deployment events in the music signal, is performed by first defining a time tolerance that separates two consecutive percussion inserts.
  • a time of 68 ms is used as a constant value derived from the time resolution and knowledge about the music signal.
  • this value determines the number of individual spectra or differentiated individual spectra which must occur at least between two consecutive inserts.
  • This minimum distance is also supported by the observation that a sixteenth note takes 60 ms at an upper tempo limit of a very high tempo of 250 bpm.
  • a detection function is derived from the differentiated and rectified spectrum, ie from the sequence of rectified (different) short-term spectra, on the basis of which the maximum search can be carried out.
  • a sum over all frequency coefficients or all spectral bins is simply determined.
  • a convolution of the function obtained is performed with a suitable Hann window, so that a relatively smooth function e is obtained.
  • a sliding window of the tolerance length is "pushed" over the entire path e in order to obtain the ability to obtain a maximum per step.
  • the reliability of the Maximas search is improved by preferentially retaining only the maxima that appear in a window for more than one time, since they are most likely to be the peaks of interest.
  • the maxima which represent a maximum over a predetermined threshold of times, for example three times, the threshold ultimately depending on the ratio of the block length to the hop size. From this it can be seen that a maximum, if it really is a significant maximum, must actually be a maximum a certain number of times, ie ultimately a certain number of overlapping spectra, if it is thought that with the numerical values previously shown, each sample in at least 9 consecutive short-term spectra "mixed in".
  • the unwrapped phase information of the original spectrogram is used as the reliability function. It has been found that in the phase information, a significant positive-going phase shift must occur in addition to an estimated insertion time t, thereby avoiding that small ripples are erroneously considered as "onsets".
  • a small section of the difference spectrogram namely a short-term spectrum produced by differentiation, is then extracted and fed to the subsequent decomposition device.
  • the functionality of the device 18a for performing a principal component analysis will be discussed below. From the steps described in the previous section, the information about the time of occurrence t and the spectral compositions of the inserts, ie the extracted short-term spectra X t , are derived. For real music signals, one typically finds a large number of transient events within the duration of the piece of music. Even with a simple example of a piece at a speed of 120 beats per minute (bpm), it turns out that in a four-minute segment, 480 events can be set, assuming that only quarter notes occur.
  • bpm beats per minute
  • PCA principal component analysis
  • T describes a transformation matrix that is actually a subset of the manifold of eigenvectors.
  • the reciprocal values of the eigenvalues become used as scaling factors, which not only leads to a decorrelation, but also provides a variance normalization, which in turn leads to a whitening effect or a whitening effect.
  • a singular value decomposition (SVD) of X t can also be used. It has been found that the SVD is equivalent to the PCA with EVD.
  • the whitened components X are subsequently fed to the ICA stage 18b, which will be discussed below.
  • ICA Independent Component Analysis
  • PDF common probability density function
  • the first concept is always met, since the vectors subjected to the ICA result from the differentiated and half-wave equally weighted version X of the original spectrogram X , which thus never contains values less than zero, but certainly values equal to zero.
  • the second constraint is taken into account when the spectra collected at operating times are considered to be the linear combinations of a small set of original source spectra containing the instruments considered characterize. This, of course, is a pretty rough approximation, but it turns out to be sufficiently good in a variety of cases.
  • the spectra having inserts, and in particular the spectra of actual percussion instruments have no invariant structures, but are not subject to any changes in their spectral composition here. Nevertheless, it can be assumed, however, that there are characteristic features characteristic of spectral profiles of percussion sounds, thus allowing the whitened components X to be separated into their potential source or profile spectra F according to the following equation.
  • F A ⁇ X ⁇
  • A denotes a dxd demixing matrix determined by the ICA process which actually separates the individual components X.
  • the sources F are also called profile spectra in this document.
  • Each profile spectrum has n frequency bins, just like a spectrum of the original spectrogram, but is identical for all times except the amplitude normalization, ie the amplitude envelope. This means that such a profile spectrum contains only the spectral information related to an onset spectrum of an instrument.
  • the spectral profiles obtained after demixing still have certain dependencies. However, this should not be considered a faulty behavior.
  • Tests with spectral profiles of individual drum sounds have shown that the spectral profiles also have a strong dependence between the input spectra of different percussive instruments.
  • One way to measure the degree of mutual overlap and similarity along the frequency axis is to perform crosstalk measurements.
  • the spectral profiles obtained from the ICA process may be considered as a transfer function of highly frequency selective parts in a filterbank, with overlapping passbands leading to crosstalk in the output of the filterbank channels.
  • the crosstalk measure between two spectral profiles is calculated according to the following equation.
  • C i . j F i ⁇ F j T F i ⁇ F i T
  • i ranges from 1 to d
  • j ranges from 1 to d
  • j is other than i.
  • this value is related to the known cross-correlation coefficient, but uses a different normalization.
  • amplitude envelope determination is now performed in block 20 of FIG.
  • the original spectrogram that is to say the sequence of short-time spectra obtained, for example, by means 12 of FIG. 1 or in time / frequency / converter 12 of FIG. 2, is used.
  • the inventive concept gives highly specialized spectral profiles that are very close to the spectra of the instruments that actually appear in the signal. Nevertheless, the extracted amplitude envelopes are only in certain cases beautiful capture functions with sharp peaks, for example for dance-oriented music with very dominant percussive rhythm parts. Often, the amplitude envelopes contain smaller peaks and plateaus, which may be due to the above mentioned crosstalk effects.
  • a maximum number d of components in the PCA or ICA process is specified.
  • the extracted components are classified using a set of spectral-based and time-based features.
  • the classification should provide two pieces of information. First, the components are to be eliminated from the further process, which are recognized with high certainty as non-percussive. Furthermore, the remaining components are to be assigned to predefined instrument classes.
  • FIG. 3a shows a very fast and very high amplitude envelope for a percussive source
  • Fig. 3a is an amplitude envelope for a kick drum
  • Fig. 4a is an amplitude envelope for a trumpet.
  • the amplitude envelope for the trumpet shows a relatively rapid rise, and then a relatively slow decay, as is typical of harmonic sustained instruments.
  • the amplitude envelope for a percussive element increases very rapidly and very sharply, but also drops again just as fast and steeply, as a drum sound typically does not linger very long due to the nature of the generation of this tone or decays.
  • amplitude envelopes can thus be used as well for classification or feature extraction as the profiled spectra explained below, which in the case of a percussive source (Fig. 3b, Hi-Hat) and Fig. 4b in the case of a harmonically sustained instrument (guitar) clearly differ.
  • the harmonically sustained instrument shows a clear manifestation of the harmonics, whereas the percussive source has a rather noisy spectrum, which does not have pronounced harmonics, but overall has an area in which energy is concentrated Energy is concentrated, is very broadband.
  • a spectrally-based measure that is, a measure derived from the profile spectra (eg, FIGS. 3b and 4b), is used to obtain spectra of harmonically sustained tones of spectra related to percussive sounds separate.
  • a modified version of the calculation of this measure is used, showing tolerance to spectral lag phenomena, all harmonics dissonance and proper normalization.
  • a higher degree of computational efficiency is achieved by replacing an original dissonance function with a frequency pair weighting matrix.
  • the assignment of spectral profiles to a-priori-defined classes of percussive instruments is provided by a simple classifier for classifying the k nearest neighbors with spectral profiles of individual instruments as a training database.
  • the distance function is calculated from at least one correlation coefficient between a query profile and a database profile.
  • additional features that provide detailed information about the shape of the spectral profile are extracted. These include the previously mentioned individual features.
  • Drum-type inserts are detected in the amplitude envelopes, such as in the amplitude envelope in Figure 3a, using conventional peak selection methods, also referred to as peak picking. Only peaks in a tolerance range in addition to the original times t, ie the times at which the maximum seeker 16c gave a result, are considered as candidates for missions. Remaining peaks extracted from the amplitude envelopes are first stored for further consideration. The value of the amount of the amplitude envelope is assigned to each insert candidate at its position. If this value does not exceed a predetermined dynamic threshold, then the mission is not accepted. The threshold varies over the amount of energy in a larger time range surrounding the inserts.
  • an automatic detection and preferably also an automatic classification of non-pitched percussive instruments in real polyphonic music signals is thus achieved, the starting basis for this being the profile spectra on the one hand and the amplitude envelope on the other hand.
  • the rhythmic information of a piece of music can be well extracted, which in turn should lead to a favorable note-to-note transcription.
  • the inventive method for analyzing an information signal can be implemented in hardware or in software.
  • the implementation may be on a digital storage medium, in particular a floppy disk or CD with electronically readable control signals, which can cooperate with a programmable computer system, that the method is performed.
  • the invention thus also consists in a computer program product with a program code stored on a machine-readable carrier for carrying out the method when the computer program product runs on a computer.
  • the invention can thus be realized as a computer program with a program code for carrying out the method when the computer program runs on a computer.

Claims (25)

  1. Dispositif pour analyser un signal d'information, aux caractéristiques suivantes:
    un moyen (16) pour extraire des spectres de courte durée significatifs ou des spectres de courte durée significatifs dérivés de spectres de courte durée du signal d'information d'une succession dans le temps de spectres de courte durée représentant le signal d'information, le moyen (16) pour extraire étant réalisé de manière à extraire de la succession dans le temps les spectres de courte durée qui se rapprochent plus d'une caractéristique spécifique que d'autres spectres de courte durée du signal d'information;
    un moyen (18) pour décomposer les spectres de courte durée extraits en spectres de signal à composantes, un spectre de signal à composantes représentant un spectre de profil d'une source sonore qui génère un son correspondant à la caractéristique recherchée, et un autre spectre de signal à composantes représentant un spectre de profil d'une autre source sonore qui génère un son correspondant à la caractéristique recherchée; et
    un moyen (20) pour calculer une courbe enveloppante d'amplitude pour les sources sonores, une courbe enveloppante d'amplitude pour une source sonore indiquant la manière dont varie dans le temps un spectre de profil de la source sonore, à l'aide des spectres de profil et d'une succession de spectres de courte durée représentant le signal d'information.
  2. Dispositif selon la revendication 1, dans lequel le moyen (16) pour extraire est réalisé de manière à prétraiter le signal d'information (8) de sorte que les parties de signal dans le signal d'information à hautes fréquences sont mises en évidence par rapport aux parties de signal dans le signal d'information à basses fréquences dans le signal d'information.
  3. Dispositif selon la revendication 2, dans lequel le moyen (16) pour extraire est réalisé de manière à, lors du prétraitement (8),
    soumettre le signal d'information, à une filtration passe-haut,
    distorsionner de manière non linéaire la version filtrée passe-haut du signal d'information, et
    ajouter le signal distorsionné non linéairement au signal d'information original.
  4. Dispositif selon l'une des revendications précédentes, dans lequel le moyen (16) pour extraire est réalisé de manière à soumettre le signal d'information à une conversion domaine du temps-domaine de la fréquence (12), pour obtenir une succession de spectres de courte durée, deux spectres de courte durée adjacents dans le temps se rapportant à des segments du signal d'information qui se recouvrent à un intervalle de saut près.
  5. Dispositif selon la revendication 4, dans lequel chaque spectre de courte durée présente une succession de coefficients spectraux, et
    dans lequel le moyen (16) pour extraire est réalisé de manière à différentier la succession de spectres de courte durée en ce qui concerne le temps (16a), pour obtenir une succession de spectres de courte durée différentiés, un spectre de courte durée différentié donnant des informations sur des variations dans un spectre de courte durée par rapport à un spectre de courte durée précédent dans le temps ou suivant dans le temps.
  6. Dispositif selon la revendication 5, dans lequel le moyen (16) pour extraire est réalisé de manière à obtenir un spectre de courte durée différentié en formant, pour chaque coefficient spectral, une différence du coefficient spectral dans un spectre de courte durée actuel et un spectre de courte durée précédent ou suivant.
  7. Dispositif selon la revendication 5 ou 6, dans lequel le moyen (16) pour extraire est réalisé de manière à redresser les spectres de courte durée différentiés (16b) de sorte qu'un spectre de courte durée différentié redressé ne présente pas de valeurs négatives.
  8. Dispositif selon l'une des revendications 5 à 7, dans lequel le moyen (16) pour extraire est réalisé de manière à déterminer les spectres de courte durée significatifs sur base des spectres de courte durée différentiés.
  9. Dispositif selon la revendication 8, dans lequel le moyen (16) pour extraire est réalisé de manière à additionner, pour chaque spectre de courte durée différentié, des coefficients spectraux ou des valeurs dérivées des coefficients spectraux du spectre de courte durée différentié (16c), pour obtenir, pour un spectre de courte durée, une valeur de somme, de sorte qu'il s'ensuive une fonction de détection dans le temps.
  10. Dispositif selon la revendication 9, dans lequel le moyen (16) pour extraire est réalisé de manière à lisser la fonction de détection dans le temps.
  11. Dispositif selon la revendication 9 ou 10, dans lequel le moyen (16) pour extraire est réalisé de manière à trouver à un moment des maximums dans la fonction de détection (16c), et à utiliser un spectre de courte durée différentié ou un spectre de courte durée comme spectre significatif auquel est associé un moment auquel la fonction de détection présente un maximum.
  12. Dispositif selon l'une des revendications 9 à 11, dans lequel le moyen (16) pour extraire est réalisé de manière à ne considérer comme significatifs que des maximums de la fonction de détection qui sont distants l'un de l'autre dans le temps de plus d'un laps de temps prédéfini.
  13. Dispositif selon l'une des revendications 4 à 12, dans lequel le moyen (16) pour extraire est réalisé de manière à déterminer, comme succession de spectres de courte durée, des spectres de quantité et à utiliser des informations de phase des spectres de courte durée lors de l'extraction des spectres de courte durée significatifs.
  14. Dispositif selon l'une des revendications précédentes, dans lequel le moyen (18) pour décomposer est réalisé de manière à additionner, pondérés, les spectres de courte durée extraits (18a), pour obtenir un nombre réduit de spectres de courte durée extraits.
  15. Dispositif selon l'une des revendications 1 à 14, dans lequel le moyen (18) pour décomposer est réalisé de manière à effectuer, en vue d'une réduction de dimension, une analyse de composantes principales (18a), pour obtenir des spectres de courte durée traités.
  16. Dispositif selon l'une des revendications précédentes, dans lequel le moyen (18) pour décomposer est réalisé de manière à effectuer une 'independent component analysis' (18b), pour générer une pluralité de signaux à composantes, à un signal à composantes étant associée une source d'information qui contribue au signal d'information.
  17. Dispositif selon l'une des revendications précédentes, dans lequel le moyen (20) pour calculer la courbe enveloppante d'amplitude est réalisé de manière à multiplier une matrice comprenant les spectres de profil et une matrice comprenant une succession de spectres de courte durée du signal d'information, pour obtenir pour les sources sonores les courbes enveloppantes d'amplitude.
  18. Dispositif selon l'une des revendications précédentes, dans lequel le moyen pour calculer la courbe enveloppante d'amplitude est réalisé, par ailleurs, de manière à déterminer une courbe enveloppante d'amplitude différenciée à l'aide des spectres de profil pour les sources sonores et à l'aide du spectrogramme différentiel.
  19. Dispositif selon l'une des revendications précédentes, comprenant, par ailleurs, un moyen (22) pour classifier les signaux à composantes en signaux à composantes percussives et en signaux à composantes non percussives.
  20. Dispositif selon la revendication 19, dans lequel le moyen (22) pour classifier est réalisé de manière à classifier sur base des spectres de profil et/ou des courbes enveloppantes d'amplitude.
  21. Dispositif selon la revendication 19 ou 20, dans lequel le moyen (20) pour classifier est réalisé de manière à extraire une caractéristique des spectres de profil ou des courbes enveloppantes d'amplitude et à la comparer avec les caractéristiques de sources connues dans une banque de données.
  22. Dispositif selon l'une des revendications précédentes, présentant, par ailleurs, un moyen (24) pour examiner les courbes enveloppantes d'amplitude pour une source sonore, pour accepter un maximum dans la courbe enveloppante d'amplitude comme insertion d'un signal de la source sonore lorsque le moyen (16) pour extraire avait extrait, à un moment similaire dans un seuil, un spectre de courte durée significatif.
  23. Dispositif selon l'une des revendications précédentes, dans lequel le moyen (20) pour calculer la courbe enveloppante d'amplitude est réalisé de manière à calculer la courbe enveloppante d'amplitude pour une source sonore de sorte que la courbe enveloppante d'amplitude indique la manière dont varie dans le temps une intensité ou une pondération d'un spectre de profil de la source sonore.
  24. Procédé pour analyser un signal d'information, aux étapes suivantes consistant à:
    extraire (16) des spectres de courte durée significatifs ou des spectres de courte durée significatifs dérivés de spectres de courte durée du signal d'information d'une succession dans le temps de spectres de courte durée représentant le signal d'information, de la succession dans le temps étant extraits les spectres de courte durée qui se rapprochent plus d'une caractéristique spécifique que d'autres spectres de courte durée du signal d'information;
    décomposer (18) les spectres de courte durée extraits en spectres de signal à composantes, un spectre de signal à composantes représentant un spectre de profil d'une source sonore qui génère un son correspondant à la caractéristique recherchée, et un autre spectre de signal à composantes représentant un spectre de profil d'une autre source sonore qui génère un son correspondant à la caractéristique recherchée; et
    calculer (20) une courbe enveloppante d'amplitude pour les sources sonores, une courbe enveloppante d'amplitude pour une source sonore indiquant la manière dont varie dans le temps un spectre de profil de la source sonore, à l'aide des spectres de profil et d'une succession de spectres de courte durée représentant le signal d'information.
  25. Programme d'ordinateur avec un code de programme adapté pour réaliser un procédé pour analyser un signal d'information selon la revendication 24 lorsque le programme d'ordinateur est exécuté sur un ordinateur.
EP05744658A 2004-05-07 2005-04-29 Dispositif et procede pour analyser un signal d'information Not-in-force EP1743324B1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102004022660A DE102004022660B4 (de) 2004-05-07 2004-05-07 Vorrichtung und Verfahren zum Analysieren eines Informationssignals
PCT/EP2005/004685 WO2005114651A1 (fr) 2004-05-07 2005-04-29 Dispositif et procede pour analyser un signal d'information

Publications (2)

Publication Number Publication Date
EP1743324A1 EP1743324A1 (fr) 2007-01-17
EP1743324B1 true EP1743324B1 (fr) 2007-10-31

Family

ID=34968451

Family Applications (1)

Application Number Title Priority Date Filing Date
EP05744658A Not-in-force EP1743324B1 (fr) 2004-05-07 2005-04-29 Dispositif et procede pour analyser un signal d'information

Country Status (5)

Country Link
EP (1) EP1743324B1 (fr)
JP (1) JP2007536587A (fr)
AT (1) ATE377240T1 (fr)
DE (2) DE102004022660B4 (fr)
WO (1) WO2005114651A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113723200B (zh) * 2021-08-03 2024-01-12 同济大学 一种非平稳信号的时频谱结构特征提取方法

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0675562A (ja) * 1992-08-28 1994-03-18 Brother Ind Ltd 自動採譜装置
US6140568A (en) * 1997-11-06 2000-10-31 Innovative Music Systems, Inc. System and method for automatically detecting a set of fundamental frequencies simultaneously present in an audio signal
US6201176B1 (en) * 1998-05-07 2001-03-13 Canon Kabushiki Kaisha System and method for querying a music database
GB2363227B (en) * 1999-05-21 2002-02-20 Yamaha Corp Method and system for supplying contents via communication network
US7117149B1 (en) * 1999-08-30 2006-10-03 Harman Becker Automotive Systems-Wavemakers, Inc. Sound source classification
US6453252B1 (en) * 2000-05-15 2002-09-17 Creative Technology Ltd. Process for identifying audio content
JP2002207482A (ja) * 2000-11-07 2002-07-26 Matsushita Electric Ind Co Ltd 自動演奏装置、及び自動演奏方法
JP2004029274A (ja) * 2002-06-25 2004-01-29 Fuji Xerox Co Ltd 信号パターン評価装置、信号パターン評価方法及び信号パターン評価プログラム

Also Published As

Publication number Publication date
DE502005001838D1 (de) 2007-12-13
WO2005114651A1 (fr) 2005-12-01
DE102004022660A1 (de) 2005-12-15
DE102004022660B4 (de) 2006-03-23
EP1743324A1 (fr) 2007-01-17
ATE377240T1 (de) 2007-11-15
JP2007536587A (ja) 2007-12-13

Similar Documents

Publication Publication Date Title
EP1606798B1 (fr) Dispositif et procede pour analyser un signal d'information audio
EP1405222B9 (fr) Procede et dispositif pour produire une empreinte digitale et procede et dispositif pour identifier un signal audio
US7565213B2 (en) Device and method for analyzing an information signal
EP1368805B1 (fr) Procede et dispositif de caracterisation d'un signal et procede et dispositif de production d'un signal indexe
EP1371055B1 (fr) Dispositif pour l'analyse d'un signal audio concernant des informations de rythme de ce signal a l'aide d'une fonction d'auto-correlation
DE10134471C2 (de) Verfahren und Vorrichtung zum Charakterisieren eines Signals und Verfahren und Vorrichtung zum Erzeugen eines indexierten Signals
WO2006039995A1 (fr) Procede et dispositif pour le traitement harmonique d'une ligne melodique
DE10123366C1 (de) Vorrichtung zum Analysieren eines Audiosignals hinsichtlich von Rhythmusinformationen
WO2005122136A1 (fr) Dispositif et procede permettant de determiner un type d'accord a la base d'un signal d'essai
WO2005122135A1 (fr) Dispositif et procede de transformation d'un signal d'information en une representation spectrale a resolution variable
EP1743324B1 (fr) Dispositif et procede pour analyser un signal d'information
EP1671315B1 (fr) Procede et dispositif pour caracteriser un signal audio
EP1377924B1 (fr) Procede et dispositif permettant d'extraire une identification de signaux, procede et dispositif permettant de creer une banque de donnees a partir d'identifications de signaux, et procede et dispositif permettant de se referencer a un signal temps de recherche
Krusche Visualization and auralization of features learned by neural networks for musical instrument recognition

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20061102

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU MC NL PL PT RO SE SI SK TR

RIN1 Information on inventor provided before grant (corrected)

Inventor name: DITTMAR, CHRISTIAN

Inventor name: UHLE, CHRISTIAN

Inventor name: HERRE, JUERGEN

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

DAX Request for extension of the european patent (deleted)
GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU MC NL PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REF Corresponds to:

Ref document number: 502005001838

Country of ref document: DE

Date of ref document: 20071213

Kind code of ref document: P

GBT Gb: translation of ep patent filed (gb section 77(6)(a)/1977)

Effective date: 20071129

ET Fr: translation filed
NLV1 Nl: lapsed or annulled due to failure to fulfill the requirements of art. 29p and 29m of the patents act
PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071031

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080131

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080211

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080131

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080229

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071031

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071031

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071031

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080331

REG Reference to a national code

Ref country code: IE

Ref legal event code: FD4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071031

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071031

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071031

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071031

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20080801

BERE Be: lapsed

Owner name: FRAUNHOFER-GESELLSCHAFT ZUR FORDERUNG DER ANGEWAN

Effective date: 20080430

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071031

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20080430

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080201

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071031

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071031

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20080430

REG Reference to a national code

Ref country code: GB

Ref legal event code: 732E

Free format text: REGISTERED BETWEEN 20090409 AND 20090415

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071031

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20080429

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20090430

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20090430

REG Reference to a national code

Ref country code: FR

Ref legal event code: TP

REG Reference to a national code

Ref country code: FR

Ref legal event code: TP

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20080429

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080501

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20100506

Year of fee payment: 6

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071031

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20100428

Year of fee payment: 6

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20100426

Year of fee payment: 6

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20080430

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20110429

REG Reference to a national code

Ref country code: FR

Ref legal event code: ST

Effective date: 20111230

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20111101

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20110502

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 502005001838

Country of ref document: DE

Effective date: 20111101

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20110429