EP2180463A1 - Method to detect note patterns in pieces of music - Google Patents

Abstract

The method involves repeatedly segmenting channels (ch1-chp) i.e. musical instrument digital interface channels. Identical segments are determined and stored in lists (L11-Lpn) of music candidate patterns with instances for each segmentation type. Coincidence values (K21-12, K21-31) for each list of each channel are calculated relative to the lists of the channels. Resemblance values (E12, E21) and the coincidence values of each list are concatenated to a total value (Gpn) per list. The patterns with the highest total value in each channel are used as detected note pattern of the channels.

Description

The present invention relates to a method of recognizing similar recurring patterns of notes in a piece of music containing note sequences distributed in parallel channels.

Recognition of recurrent note patterns in pieces of music, e.g. from loops, riffs, phrases, themes, themes, stanzas, choruses, transitions, movements, etc., has in recent years become a vast field of research with concrete and promising technical applications. As some application examples, the automated analysis of musical structures of pieces of music in computer-aided recording studio, audio workstation and music production environments are mentioned, which must rely on a reliable pattern recognition for archiving and sorting purposes as well as for the resynthesis of existing note patterns to new compositions. Another concrete technical application is the analysis and indexing of large music databases, e.g. of music archives or online music shops, for identifiable music score for the new field of "music information retrieval" (MIR), for example, to process fuzzy queries automated, keyword "query by humming".

For pattern recognition in single-channel pieces of music, various methods have been proposed in the past which also adopt concepts from other areas of pattern recognition, such as "string-matching" techniques from the field of DNA sequence analysis, see eg Kilian Jürgen, Hoos Holger H .: "MusicBLAST - Gapped Sequence Alignment for MIR", International Conference on Music Information Retrieval (ISMIR), 2004 , "String matching" methods are based often on the use of "Dynamic Programming" algorithms for alignment and similarity comparison of note subsequences, cf. eg Hu Ning, Dannenberg Roger B., Lewis Ann L .: "A Probabilistic Model of Melodic Similarity", Proceedings of the ICMC, 2002 ,

Specifically for recognizing identical recurrent note patterns for music analysis and MIR purposes, in Hsu Jia-Lien, Liu Chih-Chin, Chen Arbee LP: "Discovering Nontrivial Repeating Patterns in Musical Data", IEEE Transactions On Multimedia, Vol. 3, 2001 proposed the use of a correlation matrix which allows non-trivial, ie non-mutually exclusive, identically recurring patterns to be found in a channel.

All previously known methods have the property of separately analyzing each channel of a multi-channel piece of music. The inventors of the present method have recognized that therein is a significant drawback of the known methods, because with it structure information that is precisely in the musical parallelism of the channels, i. their rhythmic, melodic and polyphonic context, are completely disregarded, which is reflected in the unsatisfactory recognition rate and quality of the known methods.

There is therefore an unmet need for an improved pattern recognition method for multi-channel pieces of music. The invention sets itself the goal of creating such a method.

1. a) wiederholtes Segmentieren jedes Kanals unter Variierung von Segmentlänge und -beginn und, für jede Segmentierungsart, Bestimmen zueinander ähnlicher Segmente und Speichern derselben in Listen von Kandidatenmustern mit ihren jeweiligen Instanzen, und zwar jeweils einer Liste pro Segmentierungsart und Kanal;
2. b) Berechnen eines Eigenähnlichkeitswerts für jede Liste, welcher auf den Ähnlichkeiten der Instanzen jedes Kandidatenmusters einer Liste untereinander basiert;
3. c) Berechnen von Koinzidenzwerten für jede Liste jedes Kanals gegenüber den Listen aller anderen Kanäle, welcher jeweils auf den Überlappungen von Instanzen eines Kandidatenmusters der einen Liste mit Instanzen eines Kandidatenmuster der anderen Liste basiert, wenn sich diese zumindest zweimal überlappen; und
4. d) Verknüpfen der Eigenähnlichkeits- und Koinzidenzwerte jeder Liste zu einem Gesamtwert pro Liste und Verwenden der Musterkandidaten der Listen mit dem höchsten Gesamtwert in jedem Kanal als erkannte Notenmuster des Kanals.

This object is achieved by a method of the type mentioned in the introduction, which is characterized by the following steps:

1. a) repeatedly segmenting each channel by varying segment length and start and, for each type of segmentation, determining segments and memories similar to each other the same in lists of candidate patterns with their respective instances, one list per segmentation type and channel;
2. b) calculating a self-similarity value for each list based on the similarities of the instances of each candidate pattern of a list with each other;
3. c) calculating coincidence values for each list of each channel against the lists of all other channels, each based on the overlaps of instances of a candidate pattern of the one list with instances of a candidate pattern of the other list, if they overlap at least twice; and
4. d) associating the self-similarity and coincidence values of each list to a total value per list, and using the pattern candidates of the highest-total-value lists in each channel as recognized pattern notes of the channel.

The method of the invention thus takes into account, for the first time and significantly, the parallel structural information of a multi-channel piece of music which may be concealed in the temporal coincidences of potential patterns ("candidate patterns") of different channels, and associates these with an assessment of the robustness of found candidate patterns due to the similarities between them Instances, their so-called "fitness". As a result, a much more reliable, meaningful and more accurate pattern recognition result is achieved than with all previously known methods.

It should be noted at this point that the term "channel" used here is to be understood in its most general form for a multi-channel piece of music, ie both in the sense of a single (monophonic) voice of a polyphonic (polyphonic) sentence in the sense of a (possibly also polyphonic) Instrumental voice, such as a bass, trumpet, string, Percussion, piano parts, etc., as well as in the sense of a technical channel such as a midi channel, which can contain both monophonic and polyphonic voices, parts or combinations thereof, eg a drum pattern, a chord progression, a string replacement etc.

• a1) Detektieren der in einem Kanal ident wiederkehrenden Muster, daraus Auswählen der den Kanal bestabdeckenden Muster und Speichern derselben in einer weiteren Liste von Kandidatenmustern mit ihren jeweiligen Instanzen pro Kanal.

A particularly advantageous embodiment of the invention is characterized in that the following step is additionally performed in step a):

• a1) detecting the pattern that identifies in a channel identically, selecting therefrom the pattern best covering the channel and storing it in another list of candidate patterns with their respective instances per channel.

As a result, the degree of recognition can be further increased. The channel-related pattern recognition is thus placed on two equal bases, once an identifier and once a similarity detection, for which variants different methods can be used. By including the recognition results of both variants in one and the same set of candidate patterns, an implicit combination of the two methods results in the subsequent list evaluation by means of the self-similarity and coincidence values, because the results of the two methods compete there. The method of the invention thus becomes "self-adaptive" for different types of input signals which respond differently to different types of recognition methods.

Preferably, in step a1) the detection of identical patterns is carried out by means of the correlation matrix method, as known per se from Hsu Jia-Lien et al. (supra) is known. Particularly preferably, in step a1) the best covering patterns are selected by iteratively selecting the respectively most frequent and / or longest pattern from the detected patterns.

According to a further preferred feature of the invention, in step a) the segment length is varied in multiples of the clock unit of the piece of music, which limits the possible variations to an appropriate level and saves computing time. It is particularly advantageous if the segment length is varied from twice the average note duration of the piece of music to half the length of the piece of music.

According to a further advantageous embodiment of the invention, in step a) the determination of segments which are similar to each other is carried out by aligning the notes of two segments, determining a degree of agreement of the two segments and recognizing similarity, if the degree of agreement exceeds a predetermined threshold value. These measures can be implemented quickly with reasonable computing power.

In particular, the alignment of the notes preferably takes place by means of the "Dynamic Programming" method, as described in Kilian Jürgen et al. (supra) or Hu Ning et al. (supra, with further evidence) known per se.

According to a preferred embodiment of the method, the eigen similarity value is calculated in step b) by establishing for each candidate pattern of the list a similarity matrix of its instances whose values are linked to the self-similarity value of the list, preferably weighted by the channel coverage of the candidate patterns of the list. It has been found that this embodiment leads to a rapid and stable implementation.

To further improve the recognition result, optionally at the end of step b), those lists of a channel whose self-similarity value does not reach a predetermined threshold can be deleted. Preferably, this predetermined threshold value is adaptive, in particular a percentage of the highest self-similarity value of all lists of the channel, particularly preferred at least 70%. In a practically particularly suitable embodiment, the threshold is about 85%.

A further advantageous variant of the method of the invention is that in step c) only the overlaps to those instances of the other list are taken into account for a specific candidate pattern of a list, with which the time-longest overlaps exist. Practical experiments have shown that this leads to a satisfactory recognition rate and simplifies the process in this step.

According to a further preferred variant of the invention, it is provided that, in linking step e), for each list of each channel, only those coincidence values to the lists of the other channels are taken into account, which represent the highest value there, which further improves the recognition rate.

For the same reason, it is preferably provided that, in the combination of step e), the coincidence values taken into account for a list are respectively summed up and the accumulated coincidence values are particularly preferably multiplied by the intrinsic similarity value of the list to said total value.

• die Fig. 1 und 2 ein beispielhaftes mehrkanaliges Musikstück als Eingangssignal des vorliegenden Verfahrens in Musiknotation (Fig. 1) und Notensequenzschreibweise (Fig. 2);
• Fig. 3 ein globales Flußdiagramm des erfindungsgemäßen Verfahrens;
• Fig. 4 ein Beispiel einer Korrelationsmatrix für den Schritt a1) des Verfahrens;
• Fig. 5 das Ergebnis der Detektionsphase von Schritt a1);
• Fig. 6 ein Flußdiagramm für die Auswahlphase für die bestabdeckenden Muster in Schritt a1);
• Fig. 7 das Ergebnis von Schritt a1) in Form einer ersten Liste von Kandidatenmustern und ihren Instanzen für einen Kanal;
• Fig. 8 die Bedeutung der Liste von Fig. 7 in Bezug auf die Kanalabdeckung;
• Fig. 9 verschiedene Segmentierungsarten eines Kanals für die Ähnlichkeitsbestimmung in Schritt a) des Verfahrens;
• Fig. 10 ein Beispiel eines "Dynamic Programming"-Algorithmus zur Ausrichtung zweier Segmente;
• Fig. 11 das Ergebnis der Ausrichtung von Fig. 11 für den Ähnlichkeitsvergleich zweier Segmente;
• Fig. 12 ähnliche und transitiv-ähnliche Segmente eines Kanals, welche die Instanzen eines erkannten Kandidatenmusters darstellen;
• Fig. 13 das Ergebnis von Schritt a) in Form einer weiteren Liste von Kandidatenmustern und ihren Instanzen für einen Kanal und eine bestimmte Segmentierungsart dieses Kanals;
• Fig. 14 das gesamte Ergebnis des Schrittes a), dargestellt als Satz von mehreren Listen für einen Kanal;
• Fig. 15 die Bedeutung der Listen von Fig. 14 in Form verschiedener möglicher Abdeckungen eines Kanals mit jeweils den Kandidatenmustern seiner Listen;
• Fig. 16 eine Ähnlichkeitsmatrix für die Instanzen eines Kandidatenmusters einer Liste als Grundlage für die Berechnung des Eigenähnlichkeitswerts einer Liste gemäß Schritt b);
• Fig. 17 einen Überlappungsvergleich zwischen den Musterinstanzen zweier Listen als Grundlage für die Berechnung der Koinzidenzwerte einer Liste gemäß Schritt c);
• Fig. 18 die Verknüpfung der Eigenähnlichkeits- und Koinzidenzwerte und die Berechnung des Gesamtwerts einer Liste gemäß Schritt d); und
• die Fig. 19 und 20 das Ergebnis der Anwendung des Verfahrens auf das Eingangssignal der Fig. 1 und 2 in Form der möglichen (Fig. 19) und der besten (Fig. 20) Kanalabdeckungen, welch letztere die in den Kanälen erkannten Notenmuster darstellen.

The invention will be explained in more detail below by means of preferred embodiments with reference to the accompanying drawings, in which:

• the Fig. 1 and 2 an exemplary multi-channel piece of music as an input signal of the present method in musical notation ( Fig. 1 ) and note sequence notation ( Fig. 2 );
• Fig. 3 a global flow chart of the method according to the invention;
• Fig. 4 an example of a correlation matrix for step a1) of the method;
• Fig. 5 the result of the detection phase of step a1);
• Fig. 6 a flow chart for the selection phase for the Bestabdeckenden pattern in step a1);
• Fig. 7 the result of step a1) in the form of a first list of candidate patterns and their instances for a channel;
• Fig. 8 the meaning of the list of Fig. 7 in relation to the channel cover;
• Fig. 9 various types of segmentation of a similarity determination channel in step a) of the method;
• Fig. 10 an example of a "Dynamic Programming" algorithm for aligning two segments;
• Fig. 11 the result of the alignment of Fig. 11 for the similarity comparison of two segments;
• Fig. 12 similar and transitive-like segments of a channel representing the instances of a recognized candidate pattern;
• Fig. 13 the result of step a) in the form of another list of candidate patterns and their instances for a channel and a particular segmentation type of that channel;
• Fig. 14 the entire result of step a), represented as a set of multiple lists for a channel;
• Fig. 15 the meaning of the lists of Fig. 14 in the form of various possible covers of a channel, each with the candidate patterns of its lists;
• Fig. 16 a similarity matrix for the instances of a candidate pattern of a list as a basis for the calculation of the self-similarity value of a list according to step b);
• Fig. 17 an overlap comparison between the pattern instances of two lists as a basis for the calculation of the coincidence values of a list according to step c);
• Fig. 18 the concatenation of the self-similarity and coincidence values and the calculation of the total value of a list according to step d); and
• the Fig. 19 and 20 the result of applying the method to the input signal of the Fig. 1 and 2 in the form of the possible ( Fig. 19 ) and the best ( Fig. 20 ) Channel covers, the latter representing the note patterns recognized in the channels.

Fig. 1 shows a section of a piece of music which contains notch sequences q ₁ , q ₂ and q ₃ (generally q _p ) distributed in parallel channels ch ₁ , ch ₂ and ch ₃ (generally chp), which in Fig. 2 are shown schematically. The channels ch _p are, for example, separate MIDI channels for the various instruments or voices of the piece of music, although this is not mandatory, as explained above.

For the sake of simplicity, in the present examples in the note sequences q _p only the pitches and times of occurrence of the individual notes are taken into account, but not other note parameters such as note duration, volume, velocity, envelope, sound, key context, etc. It is understood, however, that all in the The comparisons of individual notes or note patterns as described below can equally well extend to such parameters, if desired, ie multilevel or multidimensional identity or similarity comparisons between several parameters can correspondingly also be carried out in these comparisons.

Moreover, in the present examples, for the sake of simplicity, only monophonic note sequences are considered in each channel. It is understood, however, that the method presented here is just as suitable for polyphonic note sequences in the channels, for which accordingly extended identity comparisons, eg chord comparisons and key-context comparisons etc., can be made.

Thus, as will be apparent to those skilled in the art, the method presented herein is readily scalable to multiple note parameter comparisons and polyphonic note sequences.

• a1) Detektieren der in einem Kanal ident wiederkehrenden Muster, daraus Auswählen der den Kanal bestabdeckenden Muster und Speichern derselben in einer Liste von Kandidatenmustern mit ihren jeweiligen Instanzen pro Kanal.

1. a) wiederholtes Segmentieren jedes Kanals unter Variierung von Segmentlänge und -beginn und, für jede Segmentierungsart, Bestimmen zueinander ähnlicher Segmente und Speichern derselben in weiteren Listen von Kandidatenmustern mit ihren jeweiligen Instanzen, und zwar jeweils einer Liste pro Segmentierungsart und Kanal;
2. b) Berechnen eines Eigenähnlichkeitswerts für jede Liste, welcher auf den Ähnlichkeiten der Instanzen jedes Kandidatenmusters einer Liste untereinander basiert;
3. c) Berechnen von Koinzidenzwerten für jede Liste jedes Kanals gegenüber den Listen aller anderen Kanäle, welcher jeweils auf den Überlappungen von Instanzen eines Kandidatenmusters der einen Liste mit Instanzen eines Kandidatenmuster der anderen Liste basiert, wenn sich diese zumindest zweimal überlappen; und
4. d) Verknüpfen der Eigenähnlichkeits- und Koinzidenzwerte jeder Liste zu einem Gesamtwert pro Liste und Verwenden der Musterkandidaten der Listen mit dem höchsten Gesamtwert in jedem Kanal als erkannte Notenmuster des Kanals.

Fig. 3 shows the global flow of the method by its basic five steps a1), a), b), c) and d), which are discussed in detail below. These five global steps are:

• a1) detecting the pattern recurring in a channel, selecting therefrom the pattern best covering the channel and storing it in a list of candidate patterns with their respective instances per channel.

1. a) repeatedly segmenting each channel by varying segment length and start and, for each type of segmentation, determining mutually similar segments and storing them in further lists of candidate patterns with their respective instances, one list per segmentation type and channel;
2. b) calculating a self-similarity value for each list based on the similarities of the instances of each candidate pattern of a list with each other;
3. c) calculating coincidence values for each list of each channel against the lists of all other channels, each based on the overlaps of instances of a candidate pattern of the one list with instances of a candidate pattern of the other list, if they overlap at least twice; and
4. d) associating the self-similarity and coincidence values of each list to a total value per list, and using the pattern candidates of the highest-total-value lists in each channel as recognized pattern notes of the channel.

The illustrated sequence of steps a1) -a) -b) -c) -d) is only compulsory insofar as some steps presuppose the result of others; otherwise the sequence is arbitrary. For example the sequence of steps a1) and a) could be reversed, or the sequence of steps b) and c), etc.

In a simplified embodiment of the method, step a1) can optionally be dispensed with, with a correspondingly limited range of applications of the method, as discussed above.

Steps a1) to d) will now be described in detail.

a1) pattern detection by correlation matrix

In step a1), a correlation matrix according to Hsu Jia-Lien et al. Is first detected for each channel chp in order to detect the pattern of patterns recurring in a channel chp (identical "loops") . (supra). Fig. 4 shows an example of such a correlation matrix: the first row and the first column each contain the entire note sequence of a channel in which patterns are to be detected; and only one triangle of the matrix is relevant. The first entry "1" in a row means that a note in the sequence already occurs the second time ; an entry "2" means that the pattern of length 2 ("2-loop") consisting of this and the preceding note occurs a second time; the entry "3" means that the existing from this, the preceding and two times before the pattern marks of length 3 ( "3-Loop") for the second time occurs in this line, etc. For details of the correlation matrix method is to Hsu Jia- Lien et al. (supra).

By statistical evaluation of the entries in the correlation matrix Fig. 4 For each channel a provisional list can be set according to Fig. 5 in which note patterns m ₁ , m _II , m _III , m _IV , etc., found as identically recurring, with the positions of their appearance or occurrence in the note sequence q _p , ie their so-called "instances", as well as their length and frequency are listed.

From the preliminary list of Fig. 5 will now be using the in Fig. 6 outlined sketch method those note patterns as "candidate pattern" for further processing, which cover the channel ch _p as largely as possible and also without overlap. This is done according to Fig. 6 in a loop the preliminary list Fig. 5 and (i) searched for the "best" pattern m _I , m _II , etc., (ii) this as a candidate pattern m _1a , m _2a , etc. in a first list L ₁ ( _FIG . Fig. 7 ) and its instances, and (iii) all patterns overlapping with this candidate pattern are removed from the preliminary list Fig. 5 deleted.

• Es wird das häufigste Muster ausgewählt, außer es gibt ein längeres Kandidatenmuster, das mehr als 75% des Kanals abdeckt und mindestens 2/3 so oft auftritt.

The "best" pattern in step (i) is the one in the preliminary list Fig. 5 most frequent and / or longest patterns m _I , m _II , etc. The following criterion is most preferably used for the "best" pattern:

• The most common pattern is selected unless there is a longer candidate pattern that covers more than 75% of the channel and occurs at least 2/3 as often .

As a result of step a1), this results in each channel chp a first list L ₁ of candidate patterns m _{1 a} m _1b (generally m _1x), that the channel chp or its sequence of notes q _p without overlap, and as far as possible, that is, cover lükkenfrei possible, please refer Fig. 8 ,

a) pattern detection by means of segment similarity comparison

In step a) a second approach is followed. Each channel chp (or its note sequence q _p ) is repeated and respectively segmented in various ways, varying with segment length and start. Fig. 9 shows five exemplary types of segmentation I - V, wherein the segment length in multiples of the clock unit of the piece of music, ie the duration of a beat of the piece of music is varied; For example, at a 4/4 time, the clock unit is a quarter note.

The segmentation types I and II shown are based on a segmentation into segments with a length of two beats, wherein in segmentation II the beginning of the segment has been offset by one beat.

The segmentation types III - V are based on a segment length of three beats and a successive offset of the beginning of the segment by one beat at a time.

It goes without saying that this concept can be extended to any segmentation lengths, starts and also arbitrarily fine quantization units (beats) of the note sequences.

Preferably, the segment length is varied from twice the average note duration of the piece of music to a maximum of half the length of the entire piece of music, since the maximum length of a note pattern can be at most half the length of the piece of music. If desired, for the sake of brevity, the method could also be terminated earlier, i. For example, the segment length may only be varied up to a predetermined number of clocks.

For each possible segmentation type I, II, III, etc., the similarity of the segments S ₁ , S ₂ , etc. is now determined among each other, preferably with the aid of the "Dynamic Programming" method known in the art.

For an explanation of this method, let me just briefly Fig. 10 referenced in which note sequences of two exemplary segments S _s and S _t are compared to each other in a matrix. According to the rules of the "Dynamic Programming" algorithm We now assign weightings for the progression from cell to cell, eg in the present example the "Dynamic Programming" weights {0, 0, 0, 1} as {"Penalty for Insert", "Penalty for Delete", "Penalty for Replace "," Points for Match "}. For details of the "Dynamic Programming" method, please refer to Kilian Jürgen et al. (supra) and Hu Ning et al. (loc. cit.), which also contains further references.

Using the "Dynamic Programming" alignment procedure of Fig. 10 Even non-identical, ie only similar and even unequal length note sequences in the segments can be aligned with each other. Fig. 11 shows the obtained alignment result.

The similarity of the segments S _s and S _t is then evaluated using a correspondingly selected scoring scheme between 0% (unlike) and 100% (ident), for example based on the number of identical scores, the number of gaps, the pitch of deviating scores, etc. Two Segments S _s , S _t are then recognized as "similar" if their similarity value determined in this way is above a predetermined threshold value, preferably above 50%.

In this way, all segments S _{s are} compared with all other segments S _{t of} a segmentation type I, II, etc. of a channel chp. For example, for the segmentation type II of the channel chp, this leads to the recognition of a similarity between the segments S ₁ , S ₃ and S ₆ , as in FIG Fig. 12 The segments S ₁ and S ₃ here are 50% similar, the segments S ₃ and S ₆ 60% similar, and the segments S ₁ and S ₆ "transitivähnlich" to 40%.

All similar or even transitive-like segments are now again considered to be instances i _{i of} a candidate pattern resulting from the note sequence of one (eg the first) of these segments. The on this Candidate patterns found for a segmentation type of a channel are stored in the form of another list L ₂ of candidate patterns m _2a , m _2b , etc. with their respective instances i ₁ , i ₂ , etc., see Fig. 13 ,

All lists L ₂ , L ₃ , etc. for all possible sequencing types I, II, etc. of a channel ch _p , together with the first list L ₁ of step a1) discussed above, yield a set of lists L _n for each channel ch _p , please refer Fig. 14 which represents various possible covers of the channel chp with candidate patterns, see Fig. 15 ,

The lists L _n are now evaluated in the following steps b), c) and d).

b) Calculation of Eigen Similarity Values

First, in step b), for each list L _n, a self-similarity value E _{n is} calculated on the basis of similarity matrices for all candidate patterns m _na , m _nb , etc. (generally m _nx ) of the list L _n . Fig. 16 Figure 10 shows an example similarity matrix for the instances i ₁ , i ₂ , i ₃ and i _{4 of} a candidate pattern m _{n of} the list L _n : The cells of the matrix give the degree of similarity, for example as determined according to the Dynamic Programming step of step a) , again; For example, here the similarity between instance i ₁ and instance i _{3 is} 80%.

From all values of the similarity matrix Fig. 16 a self-similarity value E _nx for the candidate pattern m _nx is now determined, for example by summing up in the form: $$e_{\mathit{nx}}=\underset{k.l}{\mathrm{\Sigma }}\ddot{A}\mathit{similarity\; between}{i}_k\mathit{and}{i}_l$$

• wenn mindestens eine Zelle pro Zeile den Eintrag "1" hat, dann wird E_nx um 2 inkrementiert d.h. $${\mathrm{E}}_{\mathrm{nx}}:={\mathrm{E}}_{\mathrm{nx}}+2;$$
  
• wenn nicht, dann wird E_nx nur um den Durchschnittswert aller Zellen dieser Zeile inkrementiert, d.h. $${\mathrm{E}}_{\mathrm{nx}}:={\mathrm{E}}_{\mathrm{nx}}+\mathrm{Zeilendurchschnitt}\mathrm{.}$$
  

Alternatively, an evaluation scheme can also be used which statistically evaluates or evaluates the values in the cells of the similarity matrix, preferably in the form:

• if at least one cell per line has the entry "1", then E _nx is incremented by 2 ie $${\mathrm{e}}_{\mathrm{nx}}:={\mathrm{e}}_{\mathrm{nx}}+2;$$
  
• if not, then E _{nx is} only incremented by the average of all cells of that row, ie $${\mathrm{e}}_{\mathrm{nx}}:={\mathrm{e}}_{\mathrm{nx}}+\mathrm{Line\; average}\mathrm{,}$$
  

The self-similarity value E _{nx of} the candidate pattern m _nx is also referred to as "loopfitness" of the candidate pattern m _nx .

The self-similarity value E _{n of} the list L _n is then obtained as the sum of the eigen similarity values E _{nx of} all candidate patterns m _{nx of} the list L _n multiplied by the channel coverage P reaching all instances of all candidate _patterns m _{nx of} the list L _n , ie $$e_n={\displaystyle \underset{x}{\Sigma }}{e}_{\mathit{nx}}*P_n\mathrm{,}$$

• die zeitliche Abdeckung des Kanals verstanden, als Summe der Zeitdauern t_nxi aller Instanzen i aller Kandidatenmuster m_nx des Kanals, bezogen auf die Gesamtdauer Tp des Kanals chp; oder
• die notenmäßige Abdeckung des Kanals, als Summe der Notenanzahlen n_nxi in allen Instanzen i aller Kandidatenmuster m_nx des Kanals, bezogen auf die Gesamtanzahl N_p von Noten des Kanals chp; oder bevorzugt
• sowohl die zeitliche als auch die notenmäßige Abdekkung in gewichteter Form, beispielsweise gleich gewichtet, d.h.: $$P_n=\frac{1}{2}\left(\frac{{\displaystyle \sum _{x,i}}{t}_{\mathit{nxi}}}{T_p}+\frac{{\displaystyle \sum _{x,i}}{n}_{\mathit{nxi}}}{N_p}\right)$$
  

Under the channel coverage P _{n of} a list L _{n of} a channel ch _p becomes either

• the temporal coverage of the channel, as the sum of the durations t _{nxi of} all instances i of all candidate _patterns m _{nx of} the channel, relative to the total duration Tp of the channel chp; or
• the notional coverage of the channel, as the sum of the number of notes n _nxi in all instances i of all candidate _patterns m _{nx of} the channel, relative to the total number N _p of notes of the chp channel; or preferred
• both the temporal and the nominal coverage in weighted form, for example, equally weighted, ie: $$P_n=\frac{1}{2}\left(\frac{{\displaystyle \underset{x.i}{\Sigma }}{t}_{\mathit{nxi}}}{T_p}+\frac{{\displaystyle \underset{x.i}{\Sigma }}{n}_{\mathit{nxi}}}{N_p}\right)$$
  

In an optional step that lists L can, after determining the intrinsic similarity values E _n of the lists L _n, for example, immediately after step b), for a particular channel chp all _n of the channel ch are deleted _p whose intrinsic similarity values E _n a predetermined threshold value do not reach. The threshold value can preferably be specified adaptively or dynamically, for example as a percentage of the highest eigennormality value E _{n of} all channels L _{n of} the channel chp, eg as at least 70% or more preferably as approximately 85% of the highest self-similarity value E _{n of} all lists L _{n of} the channel ch _p .

c) Calculation of the coincidence values

In step c), for each list L _n, coincidence values are calculated, between each list L _{n of} each channel chp and each list L _{n of} each other channel ch _{p '} , as in FIGS Fig. 17 and 18 outlined.

Fig. 18 shows - representative of all these coincidence value calculations - the first list L _{21 of} the channel ch ₂ , which is in each case compared with all other lists (but not with the lists of its own channel ch ₂ ) in order to obtain coincidence values K _21-12 , K _{21- 31} , etc., generally K _{pn-p'n '} (with p' ≠ p), from which then a total coincidence value K _pn for each list L _{pn is} determined, as described below.

According to Fig. 17 is a coincidence value from the temporal overlaps u of the instances i _{i of} two lists to be compared - Fig. 17 for the sake of simplicity only referred to as L ₁ and L ₂ - calculated: The coincidence value K _{pn-p'n '} is the sum of all time durations t _{i of} those instance overlaps u, which are considered below, based on the time duration T of the entire considered channel chp.

Only those overlaps u of instances i _{i of} a candidate pattern m _{1x of} the list L ₁ with instances i _{i of} the candidate patterns m _{2x of} the other list L _{2 are} taken into account, which occur at least twice, and then only those overlaps u which match the candidate candidates - generate the longest overlap times t _i . In the example of Fig. 17 this means that the candidate pattern m _1b (ie its three instances i ₁ , i ₂ , i ₃ ) overlaps three times with instances of one and the same candidate pattern of the second list L ₂ , with the three instances i ₁ , i ₂ and i ₅ the candidate pattern m _2a at the overlap times t ₁ , t ₂ and t ₅ ; and only these overlap periods to be considered for the candidate pattern m _1b.

All further overlaps of the candidate pattern m _1b with instances of other candidate patterns, for example the instances i ₁ and i ₄ of m _2b , are _disregarded because these overlaps are shorter than the aforementioned ones. Also, the repeated overlapping of the instance i ₂ of m _1b with the instance i ₃ of m _2a is not counted, but only a single double overlap per instance of the first list L ₁ , namely the time-longest.

Likewise, repeated overlaps v of the instances i ₁ and i _{2 of} the candidate pattern m _1a with the instances i ₁ and i _{2 of} the candidate _pattern m _{2b are} disregarded because already the overlaps u of the instances i ₃ and i ₄ of m _1a with the instances i ₁ and i _u m were taken into account by _2a.

The coincidence _value K _{pn-p'n '} can optionally be for instances _coinciding exactly in their beginning or end - in the example shown in _FIG Fig. 17 the coincident beginnings of the first instances i _{1 of} the candidate patterns m _1b and m _2a and the coincidence of the ends of the third instances i ₃ of m _1a and m _2a and the starts of the fourth instances i ₄ of m _1a and m _2a - for each coincidence especially increased, for example by a predetermined "bonus value" be incremented.

Coming back to the general notation of Fig. 18 Thus, for example, for the list L _{pn of} the channel chp, the following coincidence _values K _{pn-p'n 'result} from the lists of all other channels: $$K_{\mathit{pn}-\mathit{p\text{'}n\text{'}}}=\frac{\underset{i}{\Sigma }{t}_{\mathit{pn}.i}}{T_p}\mathrm{With}{\mathrm{T}}_{\mathrm{p}}\mathrm{=}{\mathrm{T}}_{\mathrm{p\text{'}}}\mathrm{,}$$

d) Linking the self-similarity and coincidence values

The inherent similarity values determined for each list L E _{pn pn pn} and coincidence values K _{p 'n'} are linked to form a total value of G _{pn pn} the list L, for example, by summing, multiplication or other mathematical operations.

Preferably, the following relationship is applied: As in Fig. 18 For example, for a list, for example the first list L _{21 of} the second channel ch ₂ , only those coincidence _values K _{21-p'n 'are} taken into account with respect to the lists L _{p'n' of} the other channels ch _{p '} which are present in each channel each have the highest value. In the example shown, these are the coincidence value K _21-12 to the second list L _{12 of} the first channel ch ₁ and the coincidence value K _21-31 to the first list L _{31 of} the third channel ch ₃ .

These channel-maximum coincidence values are added up to a total coincidence value K _pn for the list L _pn , ie: $$K_{\mathit{pn}}={\displaystyle \underset{\mathit{p\text{'}}}{\Sigma }}\underset{\mathit{in}}{\mathrm{m}}\underset{\mathit{p\text{'}}}{\mathrm{ax}}\left(K_{\mathit{pn}-\mathit{p\text{'}n\text{'}}}\right)\mathrm{,}$$

The total coincidence value K _{pn of} the list L _pn is then multiplied by the intrinsic similarity value E _{pn of} the list L _pn to give a total value G _pn for the list L _pn : $${\mathit{G}}_{\mathit{pn}}\mathit{=}{\mathit{e}}_{\mathit{pn}}\mathit{*}{\mathit{K}}_{\mathit{pn}}$$

hat. In dem in Fig. 19 gezeigten Beispiel, welches auf den Eingangssequenzen der Fig. 1 und 2 beruht, sind das die Liste L₁₂ als Ergebnisliste L₁ des ersten Kanals ch₁, die Liste L₂₁ als Ergebnisliste L₂ des zweiten Kanals ch₂ und die Liste L₃₃ als Ergebnisliste L₃ des dritten Kanals ch₃.Subsequently, in each channel ch _p, respectively that list L _{p is} sought which has the highest total value Gp $$G_p=\underset{\mathit{in}}{\mathrm{m}}\underset{p}{\mathrm{ax}}\left(G_{\mathit{pn}}\right)$$

Has. In the in Fig. 19 shown example, which on the input sequences of Fig. 1 and 2 that is, the list L ₁₂ as the result list L _{1 of} the first channel ch ₁ , the list L ₂₁ as the result list L _{2 of} the second channel ch _2, and the list L ₃₃ as the result list L _{3 of} the third channel ch ₃ .

The candidate patterns m _{px of} the lists Lp represent the respectively best-known, similarly recurring note patterns of the channel for each channel chp, taking into account its structural relationships to all other channels, as in Fig. 20 shown.

The invention is not limited to the illustrated embodiments, but includes all variants and modifications that fall within the scope of the appended claims.

Claims (15)

A method of recognizing similar recurring patterns of notes in a piece of music comprising note sequences (q) distributed on parallel channels (ch), comprising the steps of: a) repeatedly segmenting each channel (ch) by varying segment length and start and, for each type of segmentation, determining mutually similar segments (S) and storing them in lists (L) of candidate patterns (m) with their respective instances (i), one list per segmentation type and channel; b) calculating a self-similarity value (E) for each list (L) based on the similarities of the instances (i) of each candidate pattern (m) of a list with one another; c) calculating coincidence values (K) for each list (L) of each channel (ch) against the lists of all other channels, each on the overlaps (u) of instances (i) of a candidate pattern (m) of the one list (L) is based on instances (i) of a candidate pattern (m) of the other list (L) if they overlap at least twice; and d) associating the self-similarity and coincidence values (E, K) of each list (L) with a total value (G) per list and using the pattern candidates (m) of the highest totality lists (L) in each channel (ch ) as recognized note patterns of the channel. Method according to Claim 1, characterized in that the following additional step is carried out in step a): a1) detecting the pattern (m) recurring in a channel (ch), selecting therefrom the pattern bestabdekkenden and storing it in another list (L) of candidate patterns (m) with their respective instances (i) per channel. Method according to Claim 2, characterized in that, in step a1), the detection of identical recurring patterns (m) is carried out by means of the correlation matrix method known per se. Method according to Claim 2 or 3, characterized in that, in step a1), the selection of the best-covering patterns (m) is carried out by iteratively selecting the respectively most frequent and / or longest pattern from the detected patterns. Method according to one of claims 1 to 4, characterized in that in step a) the segment length is varied in multiples of the clock unit of the piece of music. A method according to claim 5, characterized in that the segment length is varied from twice the average note duration of the piece of music to half the length of the piece of music. Method according to one of claims 1 to 6, characterized in that in step a) determining mutually similar segments (S) by aligning the scores of two segments, determining a degree of agreement of the two segments and recognizing similarity, if the degree of matching a predetermined threshold exceeds, takes place. A method according to claim 7, characterized in that the alignment of the notes by means of the known "Dynamic Programming" method is carried out. Method according to one of Claims 1 to 8, characterized in that in step b) a similarity matrix of its instances (i) is set up for each candidate pattern (m) of the list (L) whose values correspond to the self-similarity value (E) of the list (L). , preferably weighted by the channel coverage (P) of the candidate pattern (m) of the list (L). Method according to one of Claims 1 to 9, characterized in that at the end of step b) those lists (L) of a channel (ch) whose self-similarity value (E) does not reach a predetermined threshold value are deleted. A method according to claim 10, characterized in that the predetermined threshold value is a percentage of the highest self-similarity value (E) of all the lists (L) of the channel (ch), preferably at least 70%, more preferably about 85%. Method according to one of claims 1 to 11, characterized in that in step c) only the overlaps (u) to those instances (i) of the other list (L) are taken into account for a particular candidate pattern of a list (L), with which the longest overlaps. Method according to one of Claims 1 to 12, characterized in that, in linking step e), for each list (L) of each channel (ch) only those coincidence values (K) to the lists (L) of the other channels (ch) are considered which represent the highest value there. Method according to one of Claims 1 to 13, characterized in that in the combination of step e) the coincidence values (K) taken into account for a list (L) are respectively summed up. A method according to claim 14, characterized in that, in the linking of step e), the accumulated coincidence values (K) are multiplied by the self-similarity value (E) of the list (L) to said total value (G).

Images