EP1368805B1 - Verfahren und vorrichtung zum charakterisieren eines signals und verfahren und vorrichtung zum erzeugen eines indexierten signals - Google Patents

Verfahren und vorrichtung zum charakterisieren eines signals und verfahren und vorrichtung zum erzeugen eines indexierten signals Download PDF

Info

Publication number
EP1368805B1
EP1368805B1 EP02718164A EP02718164A EP1368805B1 EP 1368805 B1 EP1368805 B1 EP 1368805B1 EP 02718164 A EP02718164 A EP 02718164A EP 02718164 A EP02718164 A EP 02718164A EP 1368805 B1 EP1368805 B1 EP 1368805B1
Authority
EP
European Patent Office
Prior art keywords
signal
tonality
measure
spectral components
quotient
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
EP02718164A
Other languages
German (de)
English (en)
French (fr)
Other versions
EP1368805A2 (de
Inventor
Eric Allamanche
Jürgen HERRE
Oliver Hellmuth
Bernhard FRÖBA
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
M2any GmbH
Original Assignee
Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV filed Critical Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Publication of EP1368805A2 publication Critical patent/EP1368805A2/de
Application granted granted Critical
Publication of EP1368805B1 publication Critical patent/EP1368805B1/de
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H1/00—Details of electrophonic musical instruments
    • G10H1/0033—Recording/reproducing or transmission of music for electrophonic musical instruments
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H1/00—Details of electrophonic musical instruments
    • G10H1/0008—Associated control or indicating means
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • G10L19/0204—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using subband decomposition
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
    • G10H2210/031—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
    • G10H2210/081—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal for automatic key or tonality recognition, e.g. using musical rules or a knowledge base
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2240/00—Data organisation or data communication aspects, specifically adapted for electrophonic musical tools or instruments
    • G10H2240/011—Files or data streams containing coded musical information, e.g. for transmission
    • G10H2240/046—File format, i.e. specific or non-standard musical file format used in or adapted for electrophonic musical instruments, e.g. in wavetables
    • G10H2240/061—MP3, i.e. MPEG-1 or MPEG-2 Audio Layer III, lossy audio compression
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2240/00—Data organisation or data communication aspects, specifically adapted for electrophonic musical tools or instruments
    • G10H2240/121—Musical libraries, i.e. musical databases indexed by musical parameters, wavetables, indexing schemes using musical parameters, musical rule bases or knowledge bases, e.g. for automatic composing methods
    • G10H2240/131—Library retrieval, i.e. searching a database or selecting a specific musical piece, segment, pattern, rule or parameter set
    • G10H2240/135—Library retrieval index, i.e. using an indexing scheme to efficiently retrieve a music piece
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2250/00—Aspects of algorithms or signal processing methods without intrinsic musical character, yet specifically adapted for or used in electrophonic musical processing
    • G10H2250/131—Mathematical functions for musical analysis, processing, synthesis or composition
    • G10H2250/215—Transforms, i.e. mathematical transforms into domains appropriate for musical signal processing, coding or compression
    • G10H2250/235—Fourier transform; Discrete Fourier Transform [DFT]; Fast Fourier Transform [FFT]
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2250/00—Aspects of algorithms or signal processing methods without intrinsic musical character, yet specifically adapted for or used in electrophonic musical processing
    • G10H2250/541—Details of musical waveform synthesis, i.e. audio waveshape processing from individual wavetable samples, independently of their origin or of the sound they represent
    • G10H2250/571—Waveform compression, adapted for music synthesisers, sound banks or wavetables
    • G10H2250/601—Compressed representations of spectral envelopes, e.g. LPC [linear predictive coding], LAR [log area ratios], LSP [line spectral pairs], reflection coefficients

Definitions

  • the present invention relates to characterization of audio signals with regard to their content and in particular on a concept for classifying or indexing Audio pieces in terms of their content, for researchability to enable such multimedia data.
  • U.S. Patent No. 5,918,223 discloses a method for the Content-based analysis, storage, recovery and Segmentation of audio information.
  • An analysis of audio data generates a set of numerical values, also called Feature vector is referred to, and used for this can determine the similarity between individual audio pieces that typically in a multimedia database or on the World Wide Web are stored, classified and ranked.
  • the analysis also enables the description of user-defined Classes of audio pieces based on an analysis of a set of audio pieces that all members of a Are user-defined class.
  • the system is able individual sections of sound within a longer piece of sound find what enables audio recording to automatically segmented into a series of shorter audio segments becomes.
  • MFCCs Mel Frequency Cepstral Coefficients
  • the database system is able to measure the distance in an n-dimensional Space between two n-dimensional vectors quantify. It is also possible to have classes of audio pieces to generate by specifying a set of audio pieces who belongs in a class. Example classes are twittering birds, Rock music, etc.
  • the user is enabled to the audio track database using specific ones Search procedures. The result of a search is one List of sound files ordered by their distance from that specified n-dimensional vector are listed.
  • the User can search the database for similarity characteristics, with regard to acoustic or psychoacoustic Characteristics, in terms of subjective characteristics or in terms of special noises, e.g. Bee buzz, search.
  • Audio pieces suggested such as Animal sounds, bell sounds, Crowd sounds, laughter, machine noises, Music instruments, male language, female language, Telephone noises or water noises.
  • U.S. Patent No. 5,510,572 discloses an apparatus for Analyze and harmonize a tune using results of a melody analysis.
  • a melody in the form of a Sequence of notes played by a keyboard is read in and broken down into melody segments, a melody segment, i.e. a phrase, e.g. B. four bars of the melody includes.
  • a tonality analysis is done with each phrase, to determine the key of the melody in that phrase. To do this, the pitch of a note in the phrase is determined and then a pitch difference between the current one considered note and the previous note. Further becomes a pitch difference between the current note and the following note. Because of the pitch differences becomes a previous coupling coefficient and a subsequent coupling coefficient determined.
  • the coupling coefficient for the current grade then results from the previous coupling coefficient and the following Coupling coefficient and the note length. This process will repeated for each note of the melody in the phrase to the Key of the melody or a candidate for the key of the Determine melody.
  • the key of the phrase is used to a grade type classifier for interpretation to control the meaning of each note in a phrase.
  • the key information, which were obtained by the tonality analysis is also used to create a transpose module to control the one in a reference key in a database stored chord progression in the by tonality analysis transposed certain key for a considered melody phrase.
  • Document US-B1-6185527 discloses a classification and an indexing of audio data based on a tonality determination.
  • the object of the present invention is an improved Concept for characterizing or indexing a To create signal that has audio content.
  • This task is accomplished through a characterization process of a signal according to claim 1, by a method for Generating an indexed signal according to claim 11, by a device for characterizing a signal Claim 14 or by a device for generating a indexed signal according to claim 15 solved.
  • the present invention is based on the finding that when selecting the characteristic for characterization or indexing of a signal especially for robustness Distortions of the signal must be taken into account.
  • the usefulness of characteristics or combinations of characteristics depends on how strongly by irrelevant changes such as B. by a MP3 coding, can be changed.
  • the tonality of a signal i. H. the property of a signal, a rather flat spectrum with pronounced lines or rather a spectrum with the same height Having lines that are more robust to distortion is more common Is like Distortion caused by a lossy coding method, such as. MP3.
  • the essence of the signal is taken its spectral appearance, and related to the individual spectral lines or groups of Spectral lines.
  • the tonality also provides great flexibility with regard to the computing effort to be carried out in order to to determine the tonality measure.
  • the tonality measure can be taken from the Tonality of all spectral components of a piece derived or from the tonality of groups of spectral components, etc.
  • tonalities of successive short-term spectra of the signal under investigation either individually or weighted or statistically evaluated be used.
  • the tonality depends on the present Registration based on the audio content. Is the audio content or the signal under consideration with the audio content has noisy, so it has a different tonality than a less noisy signal.
  • a noise-like signal typically has a lower one Tonality value as a less noisy, i.e. H. more tonal, Signal. The latter signal has a higher tonality value.
  • the tonality i.e. H. the noise or tonality of a signal
  • H. the noise or tonality of a signal
  • a concept based on a tonality measure Characterizing or indexing signals therefore provides a robust recognition, which shows that the tonality essence of a signal is not beyond recognition is changed if the signal is distorted.
  • Distortion is, for example, a transmission of the signal from a loudspeaker via an air transmission channel to a microphone.
  • the robustness property of the tonality feature is significant with regard to lossy compression methods.
  • the tonality measure of a signal through lossy data compression such as according to one of the MPEG standards not or hardly being affected. It also provides a distinguishing feature based on the tonality of the signal a sufficiently good one Essence for the signal so that two different from each other Audio signals also have sufficiently different tonality measures deliver. The content of the audio signal is therefore strong correlates with the tonality measure.
  • the main advantage of the present invention is thus in that the tonality measure of the signal compared to disturbed, d. H. distorted, signals is robust. This robustness exists in particular against filtering, i. H. equalization, Dynamic compression, lossy data reduction, such as. MPEG-1/2 Layer 3, an analog transmission, etc. It also provides the tonality property of a signal has a high correlation to the content of the signal.
  • Fig. 1 shows a basic block diagram of an inventive Device for characterizing a signal that a Represents audio content.
  • the device includes an entrance 10, in which the signal to be characterized are entered can, the signal to be characterized compared to a original signal, for example a lossy one Has undergone audio coding.
  • the one to be characterized Signal is in a device 12 for determining a measure for the tonality of the signal.
  • the measure of that Tonality for the signal is via a connecting line 14 a device 16 for making a statement about the content of the signal supplied.
  • the device 16 is designed to this statement based on the transmitted by the device 12 Measure of the tonality of the signal and delivers this statement about the content of the signal at an output 18 of the system.
  • FIG. 2 shows an inventive device for generating an indexed signal that has audio content.
  • the Signal for example an audio piece as it is generated in the recording studio has been stored on a compact disc, is via an input 20 in the device shown in Fig. 2 fed.
  • a device 22 that is basically the same how the device 12 of FIG. 12 can be constructed, determines a measure of the tonality of the signal to be indexed and delivers this measure via a connecting line 24 to a device 26 for recording the measurement as an index for the signal.
  • the output 28 of the device shown in FIG. 2 to generate an indexed signal, the Signal fed in at input 20 together with a tonality index be issued.
  • the one in FIG Device shown be designed so that at the output 28 a table entry is generated, the tonality index linked to an identification mark, the identification mark clearly assigned to the signal to be indexed is.
  • the device shown in Fig. 2 provides one Index for the signal, where the index is assigned to the signal and indicates the audio content of the signal.
  • the database When a plurality of signals by the one shown in Fig. 2 Device is processed, gradually creates a database from indices for audio pieces, for example for the pattern recognition system outlined in FIG. 5 can be used can.
  • the database optionally contains the Audio pieces themselves Tonality properties can be easily searched to identify a piece by the device shown in FIG. 1 and classify them, in terms of tonality or in terms of similarities to others Pieces or distances between two pieces.
  • the device shown in Fig. 2 provides one possibility to create pieces with an associated meta description, d. H. the tonality index. Therefore it is possible Records e.g. to index according to given tonality indices and search so that according to the present invention an efficient search and find of Multimedia pieces is possible.
  • Various can be used to calculate the tonality measure of a piece Procedures are applied.
  • Fig. 3 is a time signal to be characterized by means of a device 30 are implemented in the spectral range, to a block from a block of temporal samples of generating spectral coefficients.
  • a separate tonality value can be determined in order for example using a yes / no determination, whether a spectral component is tonal or not.
  • the Tonality values can be determined by the device 32 then by means of a device 34 the tonality measure for the Signal calculated in a variety of different ways become.
  • Pieces can be classified as similar if their tonality measures only about a difference less than one differentiate predetermined threshold while pieces other than can be classified differently if their tonality indices differ by a difference that is greater than is a dissimilarity threshold.
  • Two tonality measures can be used to determine the tonality distance other sizes are used between two pieces, such as B. the difference between two absolute values, the square a difference, the quotient between two tonality measures less one, the correlation between two tonality measures, the distance metric between two tonality measures, the n-dimensional Are vectors, etc.
  • the signal to be characterized does not necessarily have to be a time signal, but that it is the same can also be an MP3-encoded signal, for example, which consists of a sequence of Huffman code words consisting of quantized spectral values have been generated.
  • the quantized spectral values were from the original Spectral values generated by quantization, the quantization was chosen such that the quantization introduced quantization noise below the psychoacoustic Masking threshold is.
  • directly the encoded MP3 data stream can be used, for example the spectral values using an MP3 decoder calculate (device 40 in Fig. 4). It is not necessary before the determination of the tonality an implementation in the time domain and then again implement a conversion into the spectral range, but it can be inside the MP3 decoder calculated spectral values can be taken directly to the Tonality per spectral component or as shown in FIG.
  • the measure for spectral flatness (SFM) is calculated using the following equation.
  • X (n) stands for the square of one Spectral component with the index n, while N for the total number is the spectral coefficient of a spectrum.
  • the SFM is equal to the quotient from the geometric mean of the spectral components to arithmetic mean of the spectral components.
  • the geometric mean is always smaller or at most equal to the arithmetic mean so that the SFM has a range of values between 0 and 1.
  • a value indicates close to 0 to a tonal signal and a value close to 1 to a closer noise-like signal with a flat spectral curve.
  • the SFM is in "Digital Coding of Waveforms", Englewood Cliffs, NJ, Prentice-Hall, N. Jayant, P. Noll, 1984 and was originally used as a measure of the maximum to be achieved Coding gain defined from a redundancy reduction.
  • the SFM can then be determined by a device 44 of the tonality measure the tonality measure can be determined.
  • Another way to determine the tonality of the spectral values, performed by a device 32 of FIG. 3 can be determined by determining peaks in the Power density spectrum of the audio signal as found in MPEG-1 audio ISO / IEC 11172-3, Annex D1 "Psychoacoustic Model 1" is.
  • the level of a spectral component determined.
  • the levels of two become the one spectral component surrounding spectral components determined.
  • a Classification of the spectral component as tonal then takes place instead when the level of the spectral component is a predetermined Factor is greater than a level of a surrounding Spectral component.
  • the predetermined threshold is in the state of technology adopted as 7dB, being for the present invention however, any other predetermined thresholds are used can be. This allows for each spectral component whether it is tonal or not.
  • the measure of tonality can then by means 34 of FIG. 3 under Use of the tonality values for the individual components and the energy of the spectral components can be specified.
  • Another way to determine the tonality of a Spectral component consists in evaluating the temporal Predictability, d. H. Predictability, the spectral component.
  • MPEG-1 Audio ISO / IEC 11172-3, Annex D2 "Psychoacoustic Model 2".
  • General will a current block of samples of the to be characterized Signal converted into a spectral representation to a to get current block of spectral components.
  • hereupon become the spectral components of the current block of spectral components using information from samples of the signal to be characterized that corresponds to the current Go ahead block, so using historical information, predicted. This will result in a prediction error from which a tonality measure is then derived can.
  • U.S. Patent No. 5,918,203 Another possibility for determining the tonality is in U.S. Patent No. 5,918,203.
  • the amounts or squares of amounts of the spectral components initially logarithmic compressed and then using a filter with differentiating Characteristic filtered to differentiate a block of to get filtered spectral components.
  • the amounts of Spectral components first with a filter with differentiating Characteristic filtered to get a counter and then with a filter with an integrating characteristic filtered to get a denominator.
  • the quotient of one differentially filtered amount of a spectral component and the integrally filtered amount of the same spectral component then gives the tonality value for this spectral component.
  • Tonality value is calculated per spectral component
  • it will preferred in view of a lower computing effort for example always the amount squares of two neighboring ones Add spectral components and then for each result the addition of a tonality value by one of the above Calculate procedure.
  • Any kind of additive grouping of amount squares or amounts of spectral components can be used to set tonality values for more than one Calculate spectral component.
  • Another way to determine the tonality of a Spectral component is the level of a spectral component with an average of levels of spectral components to compare in a frequency band.
  • the band is chosen narrow.
  • the band could also be chosen broadly, or also according to psychoacoustic Aspects. As a result, the influence can be brief Performance drops in the spectrum can be reduced.
  • the tonality of an audio signal was determined based on its spectral components, this can also in the time domain, i.e. using the samples of the Audio signal happen. This could be an LPC analysis of the signal be performed to gain a prediction for the Estimate signal.
  • the prediction gain is inversely proportional to the SFM and is also a measure of tonality of the audio signal.
  • the tonality measure is a multidimensional vector of tonality values.
  • the short-term spectrum in four adjacent and preferably not overlapping areas or frequency bands are divided, with a tonality value for example for each frequency band by the device 34 of FIG. 3 or by the device 44 of Fig. 4 is determined.
  • This is for a short-term spectrum of the signal to be characterized is a 4-dimensional one Preserve tonality vector.
  • a tonality measure which is a 16-dimensional vector or generally an n x m-dimensional Is vector, where n is the number of tonality components per frame or block of samples, while m for the number of blocks or short-term spectra under consideration stands.
  • the tonality measure would then, as stated, a 16-dimensional vector.
  • the tonality can thus consist of parts of the whole Spectrum can be calculated. So it is possible to Tonality / noiseiness of a sub-spectrum or several Determine sub-spectra and thus a finer characterization to achieve the spectrum and thus the audio signal.
  • short-term statistics from tonality values such as e.g. Mean, variance and central moments of higher order, can be calculated as a measure of tonality.
  • mean mean
  • central moments of higher order can be calculated as a measure of tonality.
  • Tonality vectors or linearly filtered tonality values are used, for example as a linear filter IIR filters or FIR filters can be used.
  • Pattern recognition system between two operating modes, namely training mode 50 and classification mode 52.
  • data is "trained", i.e. H. the System added and then recorded in a database 54.
  • Fig. 1 Device In classification mode an attempt is made to characterize one Signal with the entries in the database 54 to compare and order.
  • the invention shown in Fig. 1 Device can be in classification mode 52 be used when there are tonality indices of other pieces, with which the tonality index of the current piece can be compared to a statement about the piece too to meet.
  • the device shown in Fig. 2, however, is advantageous used in training mode 50 of Fig. 5 to the Database to be filled gradually.
  • the pattern recognition system comprises a device 56 for signal preprocessing, a downstream device 58 for Feature extraction, a device 60 for feature processing, a device 62 for cluster generation, and means 64 for performing a classification to for example, as a result of classification mode 52 such a statement about the content of the signal to be characterized to meet that signal with the signal xy that is in a Previous training mode has been trained identically is.
  • Block 56 together with block 58, forms a feature extractor, while block 60 represents a feature processor.
  • Block 56 sets an input signal to a uniform one Target format, such as B. the number of channels, the sampling rate, the resolution (in bits per sample) etc. This is insofar as it makes sense and is necessary because there are no requirements about the source from which the input signal originates should.
  • the feature 58 for feature extraction is used to do the usual large amount of information at the exit of the facility 56 to a small amount of information.
  • the too investigating signals usually have a high data rate, so a high number of samples per time period.
  • the restriction on a small amount of information must take place that the essence of the original signal, that is, the peculiarity the same, is not lost.
  • characteristic properties as general for example loudness, fundamental frequency, etc. and / or, according to the present invention, tonality features or the SFM, extracted from the signal.
  • the tonality characteristics thus obtained are said to be the essence of the signal under investigation include.
  • the previously calculated feature vectors can are processed.
  • the processing is simple Standardization of the vectors.
  • Possible processing of characteristics are linear transformations, such as the Karhunen-Loeve transformation (KLT) or linear discriminant analysis (LDA), which are known in the art. More in particular nonlinear transformations are also available Feature processing applicable.
  • KLT Karhunen-Loeve transformation
  • LDA linear discriminant analysis
  • the class generator is used to process the feature vectors to combine into classes. These classes correspond a compact representation of the associated signal.
  • the Classifier 64 is finally used to generate a feature vector a predefined class or a predefined Assign signal.
  • the table presents detection rates using a database (54) of FIG. 5 with a total of 305 pieces of music, of which the first 180 seconds each as reference data were trained.
  • the detection rate gives the percentage Number of correctly recognized pieces depending on the signal influence on.
  • the second column represents the detection rate if loudness is used as a characteristic.
  • the loudness was calculated in four spectral bands, then logarithmizing the loudness values, and then a difference of logarithmic loudness values for corresponding spectral bands in succession carried out. The result obtained was used as a feature vector used for loudness.
  • the SFM was used as the feature vector for four bands used.
  • tonality according to the invention as a classification feature for a 100% recognition rate of MP-3 encoded pieces when a snippet of 30 seconds is considered while the detection rates are both in the inventive feature as well as in the Decrease loudness as a characteristic if shorter sections (e.g. 15 s) of the signal to be examined is used for detection become.
  • FIG Device used to do the shown in FIG Train detection system.
  • the in Fig. 2 device shown can be used for any Multimedia records meta descriptions, d. H. Generating indexes so that it is possible to view records regarding their Search for tonality values or records from a database to output that have a certain tonality vector or are similar to a certain tonality vector.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Computational Linguistics (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Communication Control (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Signal Processing For Digital Recording And Reproducing (AREA)
  • Auxiliary Devices For Music (AREA)
  • Electrical Discharge Machining, Electrochemical Machining, And Combined Machining (AREA)
EP02718164A 2001-02-28 2002-02-26 Verfahren und vorrichtung zum charakterisieren eines signals und verfahren und vorrichtung zum erzeugen eines indexierten signals Expired - Lifetime EP1368805B1 (de)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
DE10109648 2001-02-28
DE10109648A DE10109648C2 (de) 2001-02-28 2001-02-28 Verfahren und Vorrichtung zum Charakterisieren eines Signals und Verfahren und Vorrichtung zum Erzeugen eines indexierten Signals
PCT/EP2002/002005 WO2002073592A2 (de) 2001-02-28 2002-02-26 Verfahren und vorrichtung zum charakterisieren eines signals und verfahren und vorrichtung zum erzeugen eines indexierten signals

Publications (2)

Publication Number Publication Date
EP1368805A2 EP1368805A2 (de) 2003-12-10
EP1368805B1 true EP1368805B1 (de) 2004-08-18

Family

ID=7675809

Family Applications (1)

Application Number Title Priority Date Filing Date
EP02718164A Expired - Lifetime EP1368805B1 (de) 2001-02-28 2002-02-26 Verfahren und vorrichtung zum charakterisieren eines signals und verfahren und vorrichtung zum erzeugen eines indexierten signals

Country Status (9)

Country Link
US (1) US7081581B2 (ja)
EP (1) EP1368805B1 (ja)
JP (1) JP4067969B2 (ja)
AT (1) ATE274225T1 (ja)
AU (1) AU2002249245A1 (ja)
DE (2) DE10109648C2 (ja)
DK (1) DK1368805T3 (ja)
ES (1) ES2227453T3 (ja)
WO (1) WO2002073592A2 (ja)

Families Citing this family (38)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7277766B1 (en) 2000-10-24 2007-10-02 Moodlogic, Inc. Method and system for analyzing digital audio files
US7890374B1 (en) 2000-10-24 2011-02-15 Rovi Technologies Corporation System and method for presenting music to consumers
DE10134471C2 (de) * 2001-02-28 2003-05-22 Fraunhofer Ges Forschung Verfahren und Vorrichtung zum Charakterisieren eines Signals und Verfahren und Vorrichtung zum Erzeugen eines indexierten Signals
DE10157454B4 (de) * 2001-11-23 2005-07-07 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Verfahren und Vorrichtung zum Erzeugen einer Kennung für ein Audiosignal, Verfahren und Vorrichtung zum Aufbauen einer Instrumentendatenbank und Verfahren und Vorrichtung zum Bestimmen der Art eines Instruments
US7027983B2 (en) * 2001-12-31 2006-04-11 Nellymoser, Inc. System and method for generating an identification signal for electronic devices
DE10232916B4 (de) * 2002-07-19 2008-08-07 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zum Charakterisieren eines Informationssignals
WO2004010352A1 (en) * 2002-07-22 2004-01-29 Koninklijke Philips Electronics N.V. Determining type of signal encoder
US20040194612A1 (en) * 2003-04-04 2004-10-07 International Business Machines Corporation Method, system and program product for automatically categorizing computer audio files
KR101008022B1 (ko) * 2004-02-10 2011-01-14 삼성전자주식회사 유성음 및 무성음 검출방법 및 장치
JP2006018023A (ja) * 2004-07-01 2006-01-19 Fujitsu Ltd オーディオ信号符号化装置、および符号化プログラム
DE102004036154B3 (de) * 2004-07-26 2005-12-22 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zur robusten Klassifizierung von Audiosignalen sowie Verfahren zu Einrichtung und Betrieb einer Audiosignal-Datenbank sowie Computer-Programm
DE102004047032A1 (de) * 2004-09-28 2006-04-06 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zum Bezeichnen von verschiedenen Segmentklassen
DE102004047069A1 (de) * 2004-09-28 2006-04-06 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zum Ändern einer Segmentierung eines Audiostücks
EP1816639B1 (en) * 2004-12-10 2013-09-25 Panasonic Corporation Musical composition processing device
US7567899B2 (en) * 2004-12-30 2009-07-28 All Media Guide, Llc Methods and apparatus for audio recognition
JP4940588B2 (ja) * 2005-07-27 2012-05-30 ソニー株式会社 ビート抽出装置および方法、音楽同期画像表示装置および方法、テンポ値検出装置および方法、リズムトラッキング装置および方法、音楽同期表示装置および方法
US8068719B2 (en) 2006-04-21 2011-11-29 Cyberlink Corp. Systems and methods for detecting exciting scenes in sports video
JP4597919B2 (ja) * 2006-07-03 2010-12-15 日本電信電話株式会社 音響信号特徴抽出方法、抽出装置、抽出プログラム、該プログラムを記録した記録媒体、および該特徴を利用した音響信号検索方法、検索装置、検索プログラム、並びに該プログラムを記録した記録媒体
US8450592B2 (en) * 2006-09-18 2013-05-28 Circle Consult Aps Method and a system for providing sound generation instructions
US7873634B2 (en) * 2007-03-12 2011-01-18 Hitlab Ulc. Method and a system for automatic evaluation of digital files
CA2690433C (en) 2007-06-22 2016-01-19 Voiceage Corporation Method and device for sound activity detection and sound signal classification
US8412340B2 (en) 2007-07-13 2013-04-02 Advanced Bionics, Llc Tonality-based optimization of sound sensation for a cochlear implant patient
CA2715432C (en) * 2008-03-05 2016-08-16 Voiceage Corporation System and method for enhancing a decoded tonal sound signal
US7923624B2 (en) * 2008-06-19 2011-04-12 Solar Age Technologies Solar concentrator system
CN101847412B (zh) * 2009-03-27 2012-02-15 华为技术有限公司 音频信号的分类方法及装置
US8620967B2 (en) * 2009-06-11 2013-12-31 Rovi Technologies Corporation Managing metadata for occurrences of a recording
US20110041154A1 (en) * 2009-08-14 2011-02-17 All Media Guide, Llc Content Recognition and Synchronization on a Television or Consumer Electronics Device
US8677400B2 (en) * 2009-09-30 2014-03-18 United Video Properties, Inc. Systems and methods for identifying audio content using an interactive media guidance application
US20110078020A1 (en) * 2009-09-30 2011-03-31 Lajoie Dan Systems and methods for identifying popular audio assets
US8161071B2 (en) 2009-09-30 2012-04-17 United Video Properties, Inc. Systems and methods for audio asset storage and management
US8886531B2 (en) 2010-01-13 2014-11-11 Rovi Technologies Corporation Apparatus and method for generating an audio fingerprint and using a two-stage query
US20110173185A1 (en) * 2010-01-13 2011-07-14 Rovi Technologies Corporation Multi-stage lookup for rolling audio recognition
US8812310B2 (en) * 2010-08-22 2014-08-19 King Saud University Environment recognition of audio input
JP5851455B2 (ja) * 2013-08-06 2016-02-03 日本電信電話株式会社 共通信号含有区間有無判定装置、方法、及びプログラム
CA2990891A1 (en) 2015-06-30 2017-01-05 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forchung E.V. Method and device for associating noises and for analyzing
US9743138B2 (en) 2015-07-31 2017-08-22 Mutr Llc Method for sound recognition task trigger
CN105741835B (zh) * 2016-03-18 2019-04-16 腾讯科技（深圳）有限公司 一种音频信息处理方法及终端
CN109584904B (zh) * 2018-12-24 2022-10-28 厦门大学 应用于基础音乐视唱教育的视唱音频唱名识别建模方法

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5210820A (en) * 1990-05-02 1993-05-11 Broadcast Data Systems Limited Partnership Signal recognition system and method
US5510572A (en) * 1992-01-12 1996-04-23 Casio Computer Co., Ltd. Apparatus for analyzing and harmonizing melody using results of melody analysis
JPH06110945A (ja) * 1992-09-29 1994-04-22 Fujitsu Ltd 音楽データベース作成装置及びその検索装置
DE19505435C1 (de) * 1995-02-17 1995-12-07 Fraunhofer Ges Forschung Verfahren und Vorrichtung zum Bestimmen der Tonalität eines Audiosignals
US5918223A (en) * 1996-07-22 1999-06-29 Muscle Fish Method and article of manufacture for content-based analysis, storage, retrieval, and segmentation of audio information
US6185527B1 (en) * 1999-01-19 2001-02-06 International Business Machines Corporation System and method for automatic audio content analysis for word spotting, indexing, classification and retrieval

Also Published As

Publication number Publication date
WO2002073592A3 (de) 2003-10-02
EP1368805A2 (de) 2003-12-10
JP4067969B2 (ja) 2008-03-26
DE10109648A1 (de) 2002-09-12
WO2002073592A2 (de) 2002-09-19
US7081581B2 (en) 2006-07-25
US20040074378A1 (en) 2004-04-22
DK1368805T3 (da) 2004-11-22
ATE274225T1 (de) 2004-09-15
AU2002249245A1 (en) 2002-09-24
JP2004530153A (ja) 2004-09-30
DE50200869D1 (de) 2004-09-23
ES2227453T3 (es) 2005-04-01
DE10109648C2 (de) 2003-01-30

Similar Documents

Publication Publication Date Title
EP1368805B1 (de) Verfahren und vorrichtung zum charakterisieren eines signals und verfahren und vorrichtung zum erzeugen eines indexierten signals
EP1407446B1 (de) Verfahren und vorrichtung zum charakterisieren eines signals und zum erzeugen eines indexierten signals
EP1405222B9 (de) Verfahren und vorrichtung zum erzeugen eines fingerabdrucks und verfahren und vorrichtung zum identifizieren eines audiosignals
EP1787284B1 (de) Vorrichtung und verfahren zur robusten klassifizierung von audiosignalen sowie verfahren zu einrichtung und betrieb einer audiosignal-datenbank sowie computer-programm
DE10232916B4 (de) Vorrichtung und Verfahren zum Charakterisieren eines Informationssignals
DE60203436T2 (de) Verfahren und System zum Erkennen, Indizieren und Suchen von akustischen Signalen
DE60215495T2 (de) Verfahren und system zur automatischen erkennung ähnlicher oder identischer segmente in audioaufzeichnungen
EP1371055B1 (de) Vorrichtung zum analysieren eines audiosignals hinsichtlich von rhythmusinformationen des audiosignals unter verwendung einer autokorrelationsfunktion
DE10117870A1 (de) Verfahren und Vorrichtung zum Überführen eines Musiksignals in eine Noten-basierte Beschreibung und Verfahren und Vorrichtung zum Referenzieren eines Musiksignals in einer Datenbank
EP1388145B1 (de) Vorrichtung und verfahren zum analysieren eines audiosignals hinsichtlich von rhythmusinformationen
DE10157454B4 (de) Verfahren und Vorrichtung zum Erzeugen einer Kennung für ein Audiosignal, Verfahren und Vorrichtung zum Aufbauen einer Instrumentendatenbank und Verfahren und Vorrichtung zum Bestimmen der Art eines Instruments
DE102004028693B4 (de) Vorrichtung und Verfahren zum Bestimmen eines Akkordtyps, der einem Testsignal zugrunde liegt
DE69026474T2 (de) System zur Spracherkennung
EP1377924B1 (de) VERFAHREN UND VORRICHTUNG ZUM EXTRAHIEREN EINER SIGNALKENNUNG, VERFAHREN UND VORRICHTUNG ZUM ERZEUGEN EINER DAZUGEHÖRIGEN DATABANK und Verfahren und Vorrichtung zum Referenzieren eines Such-Zeitsignals
Thiruvengatanadhan Music genre classification using mfcc and aann
EP1743324B1 (de) Vorrichtung und verfahren zum analysieren eines informationssignals
DE102008040163A1 (de) Vorrichtung und Verfahren zur Bereithaltung von Zuordnungsdaten, welche Audiodatensätze in Teilmengen aufteilen

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE TR

AX Request for extension of the european patent

Extension state: AL LT LV MK RO SI

17P Request for examination filed

Effective date: 20030822

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

RIN1 Information on inventor provided before grant (corrected)

Inventor name: HELLMUTH, OLIVER

Inventor name: ALLAMANCHE, ERIC

Inventor name: FROEBA, BERNHARD

Inventor name: HERRE, JUERGEN

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE TR

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT;WARNING: LAPSES OF ITALIAN PATENTS WITH EFFECTIVE DATE BEFORE 2007 MAY HAVE OCCURRED AT ANY TIME BEFORE 2007. THE CORRECT EFFECTIVE DATE MAY BE DIFFERENT FROM THE ONE RECORDED.

Effective date: 20040818

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20040818

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20040818

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: GERMAN

REF Corresponds to:

Ref document number: 50200869

Country of ref document: DE

Date of ref document: 20040923

Kind code of ref document: P

GBT Gb: translation of ep patent filed (gb section 77(6)(a)/1977)

Effective date: 20040913

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20041118

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20041118

REG Reference to a national code

Ref country code: DK

Ref legal event code: T3

LTIE Lt: invalidation of european patent or patent extension

Effective date: 20040818

ET Fr: translation filed
PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20050226

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20050226

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20050228

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2227453

Country of ref document: ES

Kind code of ref document: T3

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20050519

REG Reference to a national code

Ref country code: CH

Ref legal event code: PUE

Owner name: M2ANY GMBH

Free format text: FRAUNHOFER-GESELLSCHAFT ZUR FOERDERUNG DER ANGEWANDTEN FORSCHUNG E.V.#HANSASTRASSE 27 C#80686 MUENCHEN (DE) -TRANSFER TO- M2ANY GMBH#LICHTENBERGSTRASSE 8#85748 GARCHING (DE)

NLS Nl: assignments of ep-patents

Owner name: M2ANY GMBH

Effective date: 20060131

REG Reference to a national code

Ref country code: GB

Ref legal event code: 732E

REG Reference to a national code

Ref country code: FR

Ref legal event code: TP

REG Reference to a national code

Ref country code: ES

Ref legal event code: PC2A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20050118

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 15

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 16

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 17

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20190224

Year of fee payment: 18

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IE

Payment date: 20190222

Year of fee payment: 18

Ref country code: GB

Payment date: 20190225

Year of fee payment: 18

Ref country code: CH

Payment date: 20190222

Year of fee payment: 18

Ref country code: DE

Payment date: 20181204

Year of fee payment: 18

Ref country code: ES

Payment date: 20190315

Year of fee payment: 18

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: BE

Payment date: 20190222

Year of fee payment: 18

Ref country code: AT

Payment date: 20190222

Year of fee payment: 18

Ref country code: FR

Payment date: 20190224

Year of fee payment: 18

Ref country code: DK

Payment date: 20190222

Year of fee payment: 18

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 50200869

Country of ref document: DE

REG Reference to a national code

Ref country code: DK

Ref legal event code: EBP

Effective date: 20200229

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: NL

Ref legal event code: MM

Effective date: 20200301

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 274225

Country of ref document: AT

Kind code of ref document: T

Effective date: 20200226

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20200226

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20200229

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200229

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200229

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200226

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200301

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200226

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200901

Ref country code: DK

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200229

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200226

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200229

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200229

REG Reference to a national code

Ref country code: ES

Ref legal event code: FD2A

Effective date: 20210707

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200227