DE102004022659B3 - Apparatus for characterizing a sound signal - Google Patents

Abstract

To characterize a sound signal, a sequence of quantized use times for each of at least two sound sources over time is provided based on a quantization grid.  Then a common period length, which is the basis for the at least two sound sources, determined using the consequences of use times.  Then the sequence of deployment times is divided into respective subsequences, where a length of a subsequence is equal to the common period length.  Finally, the subsequences for the first sound source are combined into a first combined subsequence and for the second sound source into a second combined subsequence, namely z.  B.  using a pattern histogram to characterize the audio signal through the first combined subsequence and the second combined subsequence, e.g.  B.  in terms of rhythm, speed or genre.

Description

The The present invention relates to the analysis of audio signals and in particular to the analysis of sound signals for the purposes of Classification and identification of sound signals to the sound signals to characterize.

The progressive development of digital distribution media for multimedia Contents leads to a big one Diversity of offered data. For the human user is the limit of the manageable long exceeded. Thus, the content description of the data by metadata increasingly gains in importance. in principle the goal is not only text files but also z. Music files, Video files or other information signal files searchable with the same comfort as usual text databases becomes. An approach for this is the well-known MPEG 7 standard.

Especially in the analysis of audio signals, ie signals, the music and / or Language is the extraction of fingerprints from greater Importance.

The aim is further "enriching" audio data with metadata in order, for example, to a piece of music to recover metadata based on a fingerprint. The "fingerprint" should on the one hand meaningful be, and on the other hand as possible short and concise be. "Fingerprint" means thus a compressed information signal generated from a music signal, which does not contain the metadata but for referencing to the metadata e.g. by searching in a database, e.g. in a system for the identification of Audio material ("AudioID").

Usually consist of music data from the overlay of sub-signals from single sources. While listening to a pop music typically there are relatively few individual sources, namely the Singers who Guitar, bass guitar, drums and a keyboard, so can the number of sources for an orchestral piece get very tall. An orchestral piece and a pop piece of music For example, consist of a superposition of the individual Instruments emitted tones. An orchestral piece or any piece of music thus provides a superposition of Partial signals from individual sources, the sub-signals the from the individual instruments of the orchestra or pop music ensemble generated sounds and the individual instruments are single sources.

alternative can also groups of original ones Sources are understood as single sources, giving a signal at least two individual sources can be assigned.

A Analysis of a general information signal will be hereinafter merely exemplified by an orchestral signal. The analysis of a Orchestra signal can be performed in many ways. So there may be a desire to recognize the individual instruments and to extract the individual signals of the instruments from the total signal and if necessary to convert it into a musical notation, the musical notation would act as "metadata" Possibilities of Analysis is to extract a dominant rhythm being a rhythm extraction based on the percussion instruments better than on the basis of the more sound-giving instruments, which are also referred to as harmonic-sustained or "harmonic sustained" instruments become. While Percussion instruments typically timpani, drums, rattles or otherwise Including percussion instruments are among the most harmoniously endured All other instruments, such as violins, wind instruments, Etc.

Farther Become the percussion instruments all those acoustic or synthetic sound generators counted which contribute to the rhythm section due to their sound characteristics (e.g., rhythm guitar).

So would it be for example, for rhythm extraction of a piece of music desirable from the entire piece of music only to extract percussive parts and a rhythm detection then to carry out on the basis of these percussive shares, without that the rhythm recognition by signals of the harmoniously endured Instruments "disturbed" is.

In There are various ways of using the technique automatically different patterns of music pieces to extract or to detect the presence of patterns. In Coyle, E.J., Shmulevich, I., "A System for Machine Recognition of Music Patterns ", IEEE Int. Conf. on Acoustic, Speech, and Signal Processing, 1998, http://www2.mdanderson.org/app/ilya/Publications/icassp98mp r.pdf, is looking for melodic themes. For this purpose, a topic is given. Then it is searched where it occurs.

In Schroeter, T., Doraisamy, S., Rüger, S., "From Raw Polyphonic Audio to Locating Recurring Themes", ISMIR, 2000, http://ismir2000.ismir.net/poster/shroeter_ruger.pdf, is becoming melodic Searching for topics in a transcribed representation of the musical signal. and it is searched where it occurs.

Corresponding the usual Structure occidental Music is accompanied by melodic fragments as opposed to rhythmic ones Structure mostly not periodic. That's why many limit themselves Method for searching melodic fragments for individual finding their occurrence. In contrast, in the field of rhythmic analysis the interest is preferred in finding periodic structures.

In Meudic, B., "Musical Pattern Extraction: from Repetition to Musical Structure ", in Proc. CMMR, 2003, http://www.ircam.fr/equipes/repmus/RMPapers/ CMMR-meudic2003.pdf melodic patterns are identified using a self-similarity matrix.

In Meek, Colin, Birmingham, W.P., "Thematic Extractor ", ISMIR, 2001, http://ismir2001.ismir.net/pdf/meek.pdf, becomes melodic Topics searched. In particular, sequences are searched, the Length of one Sequence of two notes can be up to a predetermined number.

In Smith, L., Medina, R. "Discovering Themes by Exact Pattern Matching ", 2001, http://citeseer.ist.psu. edu / 498226.html turns to melodic themes with a self-similarity matrix searched.

In Lartillot, O., "Perception-Based Musical Pattern Discovery ", in proc. IFMC, 2003, http://www.ircam.fr/equipes/repmus/lartillot/cmmr/cmmr.pdf is also looking for melodic themes.

In Brown, J.C., "Determination of the Meter of Musical Scores by Autocorrelation ", J. of the Acoust, Soc. Of America, vol. 94, no. 4, 1993 becomes symbolic Presentation of the music signal, namely based on a MIDI representation using a periodicity function (Autocorrelation function) determines the time signature of the underlying piece of music.

Similarly in Meudic, B., "Automatic Meter Extraction from MIDI files ", Proc. JIM, 2002, http://www.ircam. fr / equipes / repmus / RMPapers / JIM-benoit2002.pdf proceeded where on the estimate of periodicities a speed and timing estimate is made of audio signals.

method to identify melodic themes are only very limited for the identification of periodicities present in a sound signal, as stated though musical themes are repetitive, however not so much describing a basic periodicity in a piece of music, but rather, if any, higher Periodizitätinformationen to have in oneself. In any case, methods of identification melodic themes very elaborate, as in the search for melodic Topics considering the different variations of the topics Need to become. So it is known from the music world that topics usually varies be, namely for example, by transposition, mirroring, etc.

The WO 02/11123 A2 discloses systems and methods for detecting Sound and music signals despite high noise and high distortion. For this are reference times and associated fingerprints of the Calculated signal. The reference times occur at reproducible Positions within a file while fingerprint features of the signal at or near the reference times. Around to perform a signal recognition, the reference times and fingerprints are used to match fingerprints to find from a database. The database also provides a File identification, after which the unknown piece is identified becomes.

The DE 101 574 54 A1 discloses a method and apparatus for generating an identifier for an audio signal, wherein first a discrete amplitude-time representation of the audio signal is generated. From this, an identifier for the audio signal is extracted from the amplitude-time representation. From several identifiers for several audio signals, which include sounds of several instruments, an instrument database is built. By means of a test identifier for audio signals generated by an unknown instrument, the type of test instrument is determined using the instrument database.

The The object of the present invention is to provide an efficient and reliable Concept for characterizing a sound signal.

These The object is achieved by a device for characterizing a sound signal according to claim 1 solved.

Of the The present invention is based on the finding that an efficient predictable and meaningful in terms of much information a sound signal based on a sequence of deployment times by period length determination, Division into subsequences and summary into one summarized Subsequence can be determined as a characteristic.

Further, preferably not only one Considered succession of use times of a single instrument, so a single sound source along the time, but at least two episodes of use times of two different sound sources that occur in parallel in the piece of music considered. Since it can typically be assumed that all sound sources or at least one subset of sound sources, such as the percussive sound sources in a piece of music, are based on the same period length, a common period length is determined using the sequences of application times of the two sound sources based on at least two sound sources. According to the invention, each sequence of deployment times is then subdivided into respective subsequences, where at a length of a subsequence is equal to the common period length.

The Characteristic extraction then takes place on the basis of a summary the subsequences for the first sound source into a first combined suborder and based on a summary of the subsequences for the second Sound source in a second combined suborder instead, where the summarized subsequences as a characteristic for the sound signal serve and can be used for further processing, such as for example, to extract semantically meaningful information about the entire piece of music, such as genre, tempo, time signature, similarity to other pieces of music etc.

The combined subsequence for the first sound source and the combined subsequence for the second Sound source thus form a drum pattern of the sound signal when the both sound sources, taking into account the sequence of application times are percussive sound sources, such as drums, other percussion instruments or any other percussive Instruments that are characterized by not their pitch, so their pitch decides, but that their characteristic spectrum or the rise and fall of an output sound and not the pitch from higher musical meaning.

The inventive approach is used thus for the automatic extraction of preferably drum patterns from a preferably transcribed, so z. B. note representation a music signal. This representation may be in MIDI format or automatically from an audio signal using digital methods Signal processing can be determined, such as with the Independent Component Analysis (ICA) or specific variations same, such as the non-negative Independent Component Analysis, or more generally with concepts, which are under the keyword "Blind Source Separation "(BSS) are known.

at a preferred embodiment of The present invention is for the extraction of a drum pattern first a Recognition of the stakes, So start times, each instrument and each pitch made with tonal instruments. Alternatively, a read held a score, this reading in one Reading in a MIDI file can consist of or in a scanning and Image processing of a musical notation or in the acceptance of manually typed notes.

hereupon in a preferred embodiment the present invention determines a grid, according to which the Note assignment times are then quantized, then the note times are quantized become.

hereupon becomes the length of the drum pattern as a length a musical measure, as an integer multiple of the length of a musical measure or as an integer multiple of the length of a musical beat determined.

hereupon is a determination of a frequency of the Occurrence of a particular instrument per metric position performed with a pattern histogram.

Then a selection of the relevant entries is made, finally a Shape of the drum pattern as a preferred characteristic for the audio signal to obtain. Alternatively, the pattern histogram as such are processed. The pattern histogram is also a compressed representation the musical events, i. of the score, and contains information about the degree the variation and preferred beats, where flatness of the histogram indicates a strong variation, while a very "mountainous" histogram on one rather stationary Signal in the sense of self-similarity points.

to Improvement of the informative value of the histogram, it is preferred to first perform a preprocessing to a signal in characteristic mutually similar regions of the signal to subdivide and a drum pattern only for mutually similar Extract regions in the signal and for others characteristic Regions in the signal to determine a different drum pattern.

The present invention is advantageous in that a robust and efficient way of calculating a characteristic of a sound signal is obtained, in particular due to the subdivision carried out, which is very robust and equally feasible for all signals according to the period length which can also be determined by statistical methods. Furthermore, this is invented The concept according to the invention is scalable to the extent that the informative value and accuracy of the concept can be increased at the price of a higher computing time without further ado that more and more sequences of occurrence times of more and more different sound sources, ie instruments, in the determination of the common period length and in the Determination of the drum pattern can be included, so that the calculation of the summarized subsequences becomes more and more complex.

A However, alternative scalability is also a certain number of combined subsequences for a given number from sound sources, depending on the processing interest to rework the obtained summarized subsequences and so as to reduce their significance as needed. histogram entries below a certain threshold z. B. be ignored. histogram entries can but also quantized per se or only generally depending on the threshold decision to be binarized that a histogram only contains the statement that in the summarized suborder at a given time a histogram entry is or not.

The inventive concept is due to the fact that many subsequences to one summarized Subsequent "fused", a robust Method, which, however, is still efficiently executable since no numeric intensive processing steps are needed.

Especially play percussive instruments without pitch, which in the following also Drums, an essential role especially in popular music. Lots of information about Rhythm and musical genre are in the "notes" played by drums, which, for example, in an intelligent and intuitive search can be used in music archives could to carry out classifications or at least pre-classifications can.

The notes played by drums often form recurring patterns, which are also called drum pattern. A drum pattern can serve as a compressed representation of the notes played, by from a longer one Score picture a score of length a drum pattern is extracted. This can turn drum patterns into semantic meaningful information about the entire piece of music such as genre, tempo, time signature, similarity to other pieces of music, Etc.

preferred embodiments The present invention will be described below with reference to FIG the accompanying drawings explained in detail. Show it:

1 a block diagram of a device according to the invention for characterizing a sound signal;

2 a schematic representation for explaining the determination of the Noteneinsatzpunkte;

3 a schematic diagram showing a quantization grid and a quantization of the notes on the grid;

4 an exemplary representation of common period lengths that can be obtained by statistical period length determinations using all instruments;

5 an exemplary pattern histogram as an example of summarized subsequences for the individual sound sources (instruments); and

6 a postprocessed pattern histogram as an example of an alternative characteristic of the audio signal.

1 shows an inventive device for characterizing a sound signal. At first includes 1 An institution 10 for providing a sequence of usage times for each sound source from at least two sound sources over time. The deployment times are preferably already quantized deployment times, which are present in a quantization grid. While 2 a sequence of application times of notes from different sound sources, ie instruments 1, 2, ..., n shows that in 2 with "x" indicates 3 one in a grid that is in 3 is shown, quantized sequence of quantized use times for each sound source, that is for each instrument 1, 2, ..., n.

3 represents at the same time a matrix or list of deployment times, with one column in 3 corresponds to a distance between two grid points or grid lines and thus represents a time interval in which, depending on the sequence of use times a note insert is present or not. At the in 3 shown embodiment is z. B. in the column, with the reference numeral 30 from instrument 1 a note insert is present, and this also applies to the instrument 2, as indicated by the "x" in the two lines 1 and 2 associated with the instruments 3 is indicated. On the other hand, the instrument n has no use of notes time in the by the reference numeral 30 shown time interval.

The multiple sequences of preferably quantized deployment times are provided by the facility 10 to a facility 12 supplied for determining a common period length. The device 12 for determining a common period length is designed so as not to determine its own period length for each succession of application times, but to find a common period length which is most likely to underlie the at least two sound sources. This is based on the fact that even if z. B. play several percussion instruments in one piece, all play more or less the same rhythm, so that a common period length must exist to which virtually all instruments that contribute to the audio signal, so all sound sources will hold.

The common tone period length is then a device 14 for dividing each sequence of insertion times to obtain on the output side a set of subsequences for each sound source.

If, for example 4 is considered, it can be seen that a common period length 40 has been found, for all instruments 1, 2, ..., n, the device 14 is designed to divide into subsequences to all sequences of deployment times in subsequences of the length of the common period length 40 divide. The sequence of deployment times for the instrument would then, as in 4 is drawn, in a first suborder 41 , a subsequent second subsequence 42 and a subsequent subsequence 43 be split, thus for the in 4 Example shown for the sequence for the instrument 1 to obtain three subsequences. Similarly, the other sequences for the instruments 2,..., Ne are also split into corresponding contiguous subsequences, as illustrated by the sequence of deployment times for the instrument 1.

The sets of subsequences for the sound sources then become one device 16 for summing each sound source to obtain a combined subsequence for the first sound source and a combined subsequence for the second sound source as a characteristic for the sound signal. The summary preferably takes place in the form of a pattern histogram. The subsequences for the first instrument are superimposed on each other in such a way that the first interval of each subsequence is, so to speak, "above" the first interval of every other subsequence 5 2, the entries in each slot of a combined suborder or in each histogram bin of the pattern histogram are counted. The combined suborder for the first sound source would be in the 5 example shown so a first line 50 the pattern histogram. For the second sound source, so z. As the instrument 2, the combined subsequence would be the second line 52 of the pattern histogram, etc. Overall, the pattern histogram sets in 5 thus the characteristic for the sound signal, which can then be used for various other purposes.

Hereinafter, various embodiments for determining the common period length in step 12 received. The finding of the pattern length can be realized in various ways, namely, for example, from an a-priori criterion, which immediately provides an estimate of the periodicity / pattern length due to the existing note information, or alternatively z. By a preferably iterative search algorithm which accepts a number of hypotheses for the pattern length and checks their plausibility based on the resulting results. This can, for example, again by evaluation of a pattern histogram, as well as by the device 16 is preferably implemented to summarize, or made using other self-similarity measures.

As it has been stated, the pattern histogram, as shown in 5 shown by the institution 16 be created to summarize. Alternatively, the pattern histogram may also consider the intensities of the individual notes in order to weight the notes according to their relevance. Alternatively, as it is in 5 has been shown, the histogram only contain information as to whether there is a sound in a subsequence or in a bin or time slot of a subsequence or not. In this case, a weighting of the individual notes would not be included in the histogram in terms of their relevance.

In a preferred embodiment of the present invention, the in 5 shown characteristic, which is here preferably a pattern histogram, processed even further. In this case, note selection can be made on the basis of a criterion, for example by comparing the frequency or the combined intensity values with a threshold value. This threshold may also depend on the type of instrument or the flatness of the histogram, among other things. The Drum Pattern entries may be Boolean sizes, where a "1" would represent the fact that a note is occurring, while a "0" would indicate the fact that no note occurs. Alternatively, an entry in the histogram can also be a measure of how high the intensity (loudness) or relevance of the note occurring in this time slot over the music signal is considered. If 6 is considered, it can be seen that the threshold was chosen so that all time slots or bins in the pattern histogram for each instrument are marked with an "x", in which the number of entries is greater than or equal to 3. On the other hand all bins are deleted in which the number of entries is less than 3, for example, 2 or 1 amounts.

Thus, according to the invention a musical "result" or score from percussive Instruments that are not or not significantly characterized by a pitch be generated. A musical event is called the occurrence of a Sounds of a musical instrument defined. Preferably only percussive Instruments considered without a significant pitch. Events are detected in the audio signal and classified into instrument classes, where the temporal positions of the events on a quantization grid, which is also referred to as a Tatum grid, are quantized. Further will the musical measure or the length of a clock in milliseconds or a number of quantization intervals furthermore preferably also identifies upbeats become. The identification of rhythmic structures on the base the frequency of the occurrence of musical events at certain Positions in the drum pattern allows a robust identification of the tempo and gives valuable hints for the Positioning of the timing lines, if also used musical background knowledge becomes.

It it should be noted that the musical score or the characteristic preferably the rhythmic information, such as Start time and duration includes. Although the estimation of this metric information, same a time signature, not necessarily for automatic synthesis the transcribed music needed is she still for the generation of a valid musical scores and for the reproduction needed by human reproducers. Therefore An automatic transcription process can be divided into two tasks be, namely the recording and classification of musical events, thus notes, and the production of a musical score from the recorded notes, so the drum pattern, as already above explained has been. For this purpose, preferably the metric structure of the music estimated, where also a quantization of the temporal positions of the detected Notes as well as a recognition of the beginning and a determination of the position the timing lines can be made. In particular, the extraction the musical score for percussive instruments without significant pitch information described from polyphonic musical audio signals. The capture and Classification of events is preferably done with the procedure the independent one Subspace analysis performed.

A Extension of the ICA provides the Independent Subspace Analysis (ISA) Here the components are divided into independent subspaces or Subspaces whose components are not statistically independent have to. Through a transformation of the music signal becomes a multidimensional Representation of the mixed signal determined and the last assumption for the ICA met. Various methods for calculating the independent components have been developed in recent years. Relevant references that partly dealing with the analysis of audio signals the following:

• 1. J. Karhunen, "Neural Approaches to independent component analysis and source separation ", Proceedings of the European Symposium on Artificial Neural Networks, pp. 249-266, Bruges, 1996.
• 2. M.A. Casey and A. Westner, "Separation of Mixed Audio Sources by Independent Subspace Analysis ", Proceedings of the International Computer Music Conference, Berlin, 2000.
• 3rd J.-F. Cardoso, "Multidimensional independent component analysis ", Proceedings of ICASSP'98, Seattle, 1998th
• 4. A. Hyvärinen, P.O. Hoyer and M. Inki, "Topographic Independent analysis ", Neural Computation, 13 (7), pp. 1525-1558, 2001.
• 5. S. Dubnov, "Extracting Sound Objects by Independent Subspace Analysis" Proceedings of AES 22 ^nd International Conference on Virtual, Synthetic and Entertainment Audio, Helsinki., 2002
• 6. J.-F. Cardoso and A. Souloumiac, "Blind beamforming for non Gaussian signals "IEE Proceedings, Vol. 140, No. 6, pp. 362-370, 1993.

An event is defined as the occurrence of a note of a musical instrument. The appearance time of a note is thus the time at which the note occurs in the musical piece. The audio signal is segmented into parts, with a segment of the audio signal having similar rhythmic properties. This is done using a distance measure between short frames of the audio signal represented by a vector of low-level audio features. The tatum grid and higher metric levels are determined separately from the segmented parts. It is assumed that the metric structure does not change within a segmented portion of the audio signal. The detected events are preferably aligned with the estimated tatum grid. This process is similar to the well-known quantization function in common MIDI sequencer software programs for music production. The bar length is taken from the quanti estimated event list, and recurring rhythmic structures are identified. The knowledge about the rhythmic structures is used for the correction of the estimated tempo and for the identification of the position of the timing lines using musical background knowledge.

Hereinafter, preferred embodiments of various inventive elements will be discussed. Preferably, the device performs 10 for providing sequences of use times for a plurality of sound sources, quantization by. The detected events are preferably quantized in the Tatum grid. The tatum grid is estimated using the note usage times of the recorded events along with note times that operate using conventional note-taking techniques. The generation of the Tatum grid on the basis of the recorded percussive events works reliably and robustly. It should be noted that the distance between two halftone dots in a piece of music usually represents the fastest played note. Thus, if there are at most sixteenth notes in a piece of music and no ones faster than the sixteenth notes, the distance between two dots of the tatum grid is equal to the length of a sixteenth note of the tone signal.

in the general case, the distance between two grid points the largest note value, the needed is to order by forming integer multiples of this note value to represent all occurring note values or temporal periods. The grid spacing is thus the largest common divisor of all occurring note durations / period lengths etc.

following are two alternative approaches to Determination of the Tatum grid shown. First, as a first solution, The tatum grid is constructed using a 2-way mismatch procedure (TWM). A series of experimental values for the Tatum period, So for the Spacing of two grid points, will be a histogram for an inter-onset interval (IOI) derived. The calculation of the IOI is not on consecutive Onsets limited, but on virtually all pairs of onsets in a time frame. Tatum candidates are considered integer fractions the most common IOI calculated. The candidate is selected, the best the harmonic structure of the IOI according to the 2-way mismatch error function predicts. The estimated Tatum period is subsequently calculated by calculating the error function between the comb grid, which is derived from the tum period and the onset times of the signal. So it becomes the histogram of the IOI generated and smoothed by means of an FIR low-pass filter. Become Tatum candidates that is, by splitting the IOI according to the peaks in the IOI histogram by a set of values between e.g. B. 1 and 4 received. One raw estimate for the Tatum period is taken from the IOI histogram after applying the Derived TWM. Subsequently become the phase of the Tatum grid and an exact estimate of the Tatum period by means of the TWM between the notes times and several date grids calculated with periods close to the previously estimated tatum period.

The second method refines and presents the tatum grid by computing the best match between the note insert vector and the tatum grid, using a correlation coefficient R _xy between the note insert vector x and the tatum y.

Around To follow small tempo variations, the tatum grid becomes adjacent Frame with z. B. a length estimated by 2, 5 sec. The transitions between the tatum grids of adjacent frames are filtered by low pass filtering of the IOI vector smoothed the tatum grid points, and the tatum grid is restored from the smoothed IOI vector. Subsequently Each event is assigned to its closest grid position. This will, so to speak a quantization performed.

The score can then be written as a matrix T _ik , i = 1,... N and j = 1,..., M, where n denotes the number of instruments acquired, and m is equal to the number of tatum grid Is equal to the number of columns in the matrix. The intensity of the detected events can either be removed or used, resulting in a Boolean matrix or resulting in a matrix of intensity values.

The following will focus on specific embodiments of the device 12 for determining a common period length. The quantized representation of the percussive events provides valuable information for the estimation of the musical measure or a periodicity that underlies the playing of the sound sources. The periodicity at the clock level, for example, is determined in two stages. First, a periodicity is calculated to then estimate the cycle length.

The periodic functions used are preferably the autocorrelation function (ACF) or the mean absolute difference function (RMDF), as shown in the following equations are placed.

The AMDF will also be available for the estimate the fundamental frequency for Music and voice signals and for the estimate of musical measure.

In the general case, a periodicity function measures the similarity or dissimilarity between the signal and its temporally different version. Different similarity measures are known. For example, there is the Hamming distance (HD), which calculates a dissimilarity between two Boolean vectors B ₁ and B ₂ according to the following equation. HD = sum (b 1 ⊻b 2 )

A suitable extension for the comparison of the rhythmic structures results from the different weighting of similar hits and rest periods. The similarity B between two sections of a score T ₁ and T ₂ is then calculated by weighted summation of the Boolean operations, as shown below. B = a · T 1 ⋀T 2 + b · ¬T 1 ⋀¬T 2 - c · T 1 ⊻T 2

In From the above equation, the weights a, b and c are originally on a = 1, b = 0, 5 and c = 0 set. a weights the occurrence of common Grades, b weighted the occurrence of common breaks and c weighted the occurrence of a difference, d. H. one score occurs Note on and no note occurs in the other score. The similarity measure becomes M obtained by summing the elements of B, as follows is set forth.

This similarity measure is similar to the Hamming distance in that differences between matrix elements are similarly accounted for. Subsequently, a modified Hamming distance (MHD) is used as the distance measure. In addition, the influence of distinct instruments can be controlled by means of a weighting vector ν _i , i = 1,..., N, determined either by using musical foresight, e.g. By placing more importance on small drums (snare drums) or on deep instruments, or depending on the frequency and regularity of the appearance of the instruments:

In addition, the similarity measures for Boolean matrices can be extended by weighting B with the average of T ₁ and T ₂ to account for intensity values. Distances or dissimilarities are regarded as negative similarities. The periodicity function P = f (M, l) is calculated by calculating the similarity measure M between the score T and a shifted version thereof, based on a displacement l. The time signature is determined by comparing P with a number of metric models. The implemented metric models Q consist of a train of spikes with typical accent positions for different time signatures and micro times. A micro-time is the integer ratio between the duration of a musical beat, that is, the note value that determines the musical tempo (eg, quarter-note), and the duration of a tatum period.

The best match between P and Q is obtained when the correlation coefficient assumes its maximum. In the present State of the system will be 13 metric models for seven different time signatures implemented.

Recurring structures are recorded in order to get started. B. to capture, and to obtain a robust pace estimation. For the acquisition of drum patterns, a score T is obtained from the length of a stroke b by summation of the matrix elements T with a similar metric position according to the following equation:

In the above equation, b denotes an estimated clock length and p denotes the number of clocks in T. Hereinafter, T 'is referred to as a score histogram or a pattern histogram. Drum patterns are obtained from the score histogram T 'by searching for score elements T' _{i, j} with large histogram values. Patterns longer than one clock are retrieved by repeating the procedure described above for integer values of the measured length. The pattern length with the most hits, relative to the pattern length itself, is selected to obtain a maximum representative pattern as a further or alternative characteristic for the sound signal.

Preferably, the identified rhythmic patterns are interpreted using a set of rules derived from musical Knowledge derived. Preferably, equidistant events of occurrence of individual instruments are identified and evaluated with reference to the instrument class. This leads to an identification of playing styles that often occur in popular music. An example is the very frequent use of the snare-drum or tambourines, or hand claps in the second and fourth beat in a four-fourth beat. This concept, called backbeat, serves as an indicator of the position of the bar lines If a backbeat pattern is present, a bar starts between two stops on the small drum.

One further note for the positioning of the timing lines is the occurrence of kick-drum events, that is, events of a typically foot-operated big drum.

It It is believed that the start of a musical measure by the metric position is marked where most kick drum notes occur.

A preferred application of the characteristic, as determined by the device 16 to summarize for each sound source, as in 1 has been shown and described, is obtained, as z. In 5 or 6 is in the genre classification of popular music. From the drum patterns obtained, various high-level features can be derived to identify typical playing styles. A classification procedure evaluates these features in conjunction with information about the musical measure, ie the speed, in z. B. beats per minute or beats per minute and using the percussive instruments used. The concept is based on the fact that all percussive instruments carry rhythm information and are often played repetitively. Drum patterns have genre-specific characteristics. Therefore, these drum patterns can be used to classify the music genre.

For this becomes a classification of different playing styles (Playing Style), each associated with individual instruments. This is how it is Playing style, for example, in that events only on every quarter note occur. An associated instrument for this style of play is the kick-drum, So the big one drum operated by foot the drums. This style of playing is abbreviated FS.

One For example, alternate play style is that events every second and fourth quarter note of a four-fourth measure occur. This is mainly from the small drum (snare drum) and tambourines, so the hand claps played. This style of play is abbreviated as BS. Exemplary others Playing styles are that notes often on the first and the third note of a triplet occur. This is abbreviated as SP and often seen in a hi-hat or a cymbal.

It So are game styles for different musical instruments specific. For example, that is first feature FS a boolean value and true if kick-drum events only occur on every quarter note. Only for certain values no boolean Variables are calculated but certain numbers are determined such as for the relation between the number of off-beat events and the Number of on-beat events, such as those of one Hi-hat, a shaker or a tambourine.

typical Combinations of drum instruments become one of the different drum set types classified, such as rock, jazz, Latin, disco and techno, to another feature for to get the genre classification. The classification of the drum set is not derived using the instrument tones, but by general investigation of the occurrence of drum instruments in different pieces, which belong to the individual genres. For example, the drum set type Rock is characterized by the fact that a kick drum, a snare drum, a hi-hat and a pelvis. In contrast, comes in the type "Latin" a bongo, a conga, Claves and shakers in front.

One another set of features becomes from the rhythmic features derived from the drum score or drum pattern. These features include musical Tempo, time signature, micro time, etc. In addition, a measure of the variation the occurrence of kick drum notes by counting the number of different ones IOI that occur in the drum pattern received.

The classification of the musical genre using the drum pattern is performed using a rule-based decision network. Potential genre candidates will be rewarded if they fulfill a hypothesis currently under investigation, and will be "punished" if they do not fulfill aspects of a currently investigated hypothesis, which will result in the selection of favorable feature combinations for each genre derived from observations of representative pieces and from musical knowledge itself, values for reward and punishment are empirically adjusted taking into account the robustness of the extraction concept: the resulting decision for a particular musical genre is hit for the genre candidate who has the maximum number of rewards. For example, the disco genre is recognized when a drum set type is disco, when the tempo is in the range of 115 to 132 bpm, when a time signature is 4/4 bit and the micro time is equal to 2. Further, another feature of the genre Disco is that a play style FS z. B. is present, and that z. B. yet another style of play is present, namely the events occur on each off-beat position. Similar criteria can be applied to other genres such as hip-hop, soul / funk, drum and bass, jazz / swing, rock / pop, heavy metal, Latin, waltz, polka / punk or techno.

Depending on the circumstances, the inventive method for characterizing a Sound signals can be implemented in hardware or in software. The Implementation can be on a digital storage medium, in particular a floppy disk or CD with electronically readable control signals done so interact with a programmable computer system can that the procedure is carried out becomes. Generally, the invention thus also consists in a computer program product with a program code stored on a machine-readable carrier for execution of the procedure when the computer program product on a machine expires. In other words Thus, the invention can be thought of as a computer program with a program code to carry out the process can be realized when the computer program is up a computer expires.

Claims (18)

Apparatus for characterizing a sound signal, comprising: a device ( 10 ) for providing a sequence of application timings of sounds for at least one sound source; a facility ( 12 ) for determining a common period length underlying the at least one sound source using the at least one sequence of times of use; a facility ( 14 ) for dividing the at least one sequence of deployment times into respective subsequences, wherein a length of a subsequence is equal to the common period length or derived from the common period length; and a facility ( 16 ) for combining the subsequences for the at least one sound source into a combined subsequence, wherein the combined subsequence represents a characteristic for the sound signal, wherein the device ( 16 ) is designed to combine to produce a histogram as a combined subsequence. Device according to claim 1, in which the device ( 10 ) is provided for providing at least two episodes of use time points for at least two sound sources, in the device ( 12 ) is designed to determine the common period length for the at least two sound sources, in which the one device ( 14 ) is arranged for splitting to divide the at least two sequences of use times according to the common period length, and in which the device ( 16 ) is configured to combine the subsequences for the second sound source into a second combined subsequence, the first combined subsequence and the second combined subsequence representing the characteristic for the audio signal. Apparatus according to claim 1, wherein the means for providing ( 10 ) is configured to provide, for each of the at least two sound sources, a sequence of quantized application times, wherein the application times are quantized relative to a quantization grid, wherein a halftone dot spacing between two halftone dots equals a shortest distance between two tones in the tone signal or equal to the largest common divisor the duration of sounds in the music signal is. Device according to Claim 1, 2 or 3, in which the device ( 10 ) is designed to provide the deployment times of percussive instruments, but not deployment times of harmonic instruments. Device according to one of the preceding claims, in which the means for determining ( 12 ) is adapted to determine a probability measure for each of a plurality of hypothetical common period lengths, and to select the hypothetical common period length of the plurality of hypothetical common period lengths as the common period length whose probability measure indicates that the hypothetical common period length is the common period length for which is at least two sound sources. Device according to claim 5, in which the device ( 12 ) for determining to determine the probability measure based on a first probability measure for the first sound source and based on a second probability measure for the second sound source. Device according to claim 5 or 6, in which the device ( 12 ) is designed to determine calculate the probability measures by comparing the sequence of times of use with a shifted sequence of times of use. Device according to one of the preceding claims, in which the device ( 14 ) is adapted to generate a list for each subsequence, the list having associated information for each screen point and for each audio source relating to whether or not the screen dot is an application time of a sound. Device according to one of the preceding claims, in which the device ( 10 ) for providing a list for each sound source, the list having, for each raster point of a raster, associated information on whether at the raster point a time of use of a sound is. Device according to Claim 9, in which the device ( 16 ) for combining to generate the histogram such that each halftone dot of a tonal raster of the composite substring represents a histogram bin. Device according to Claim 1 or 10, in which the device ( 16 ) for summarizing, for each subsequence for a sound source when an entry is found, to increment a count for an associated bin in the histogram, or to increase it by adding a measure determined by the entry, the entry being a measure of an intensity of a Is clay, which has a use at the time of use. Device according to one of the preceding claims, in which the device ( 16 ) is configured to combine to output in the first combined subsequence and the second combined subsequence only values of the subsequences that are above a threshold. Device according to one of the preceding claims, in which the device ( 16 ) is configured to normalize the subsequences for the common length or to normalize the first combined subsequence or the second combined subsequence with respect to the common length. Device according to one of the preceding claims, in which the device ( 10 ) is provided for providing segments with a uniform rhythmic structure from an audio signal, and wherein the device ( 16 ) for combining to produce the characteristic for a segment having a uniform rhythmic structure. Device according to one of the preceding claims, which further comprising the following feature: a device for extracting a feature of the characteristic for the audio signal; and a Device for determining a music genre to which the sound signal belongs, using the feature. Apparatus according to claim 15, wherein the device designed to determine a rule-based decision network, to use a pattern recognition device or a classifier. Device according to one of the preceding claims, which and means for extracting a tempo from the characteristic having. Apparatus according to claim 17, wherein the device is designed to extract the pace based on the common period length to determine.