EP2351017B1 - Method for recognizing note patterns in pieces of music - Google Patents

Description

The present invention relates to a method of recognizing similar recurring patterns of notes in a piece of music containing note sequences distributed in parallel channels.

Recognition of recurrent note patterns in pieces of music, e.g. from loops, riffs, phrases, motifs, themes, stanzas, choruses, transitions, movements, etc., has in recent years become a vast field of research with concrete and promising technical applications. As some application examples, the automated analysis of musical structures of pieces of music in computer-aided recording studio, audio workstation and music production environments are mentioned, which must rely on a reliable pattern recognition for archiving and sorting purposes as well as for the resynthesis of existing note patterns to new compositions. Another concrete technical application is the analysis and indexing of large music databases, e.g. of music archives or online music shops, for identifiable music score for the new field of "music information retrieval" (MIR), for example, to process fuzzy queries automated, keyword "query by humming".

For pattern recognition in single-channel pieces of music, various methods have been proposed in the past which also adopt concepts from other areas of pattern recognition, such as "string-matching" techniques from the field of DNA sequence analysis, see eg Kilian Jürgen, Hoos Holger H .: "MusicBLAST - Gapped Sequence Alignment for MIR", International Conference on Music Information Retrieval (ISMIR), 2004 , String matching techniques are often based on the use of dynamic programming algorithms for alignment and similarity comparison Grade subsequences, cf. eg Hu Ning, Dannenberg Roger B., Lewis Ann L .: "A Probabilistic Model of Melodic Similarity", Proceedings of the ICMC, 2002 ,

Specifically for recognizing identical recurrent note patterns for music analysis and MIR purposes, in Hsu Jia-Lien, Liu Chih-Chin, Chen Arbee LP: "Discovering Nontrivial Repeating Patterns in Musical Data", IEEE Transactions On Multimedia, Vol. 3, 2001 proposed the use of a correlation matrix which allows non-trivial, ie non-mutually exclusive, identically recurring patterns to be found in a channel.

All previously known methods have the property of separately analyzing each channel of a multi-channel piece of music. In the document EFFICIENT DETECTION OF EXACT REDUNDANCIES IN AUDIO SIGNALS by Zapata et al. (125th AES Convention 02-10-2008, San Francisco, CA, USA ) discloses the term "Finding Inter-Channel Redundancy" (see Section 4), but gives no hint to the realization of such a task. The inventors of the present method have recognized that this is a significant disadvantage of the known methods, because structural information which is contained precisely in the musical parallelism of the channels, ie their rhythmic, melodic and polyphonic contexts, remains completely disregarded, as reflected in the low satisfactory detection rate and quality of the known methods reflected.

There is therefore an unmet need for an improved pattern recognition method for multi-channel music pieces. The invention sets itself the goal of creating such a method.

1. a) wiederholtes Segmentieren jedes Kanals unter Variierung von Segmentlänge und -beginn und, für jede Segmentierungsart, Bestimmen zueinander ähnlicher Segmente und Speichern derselben in Listen von Kandidatenmustern mit ihren jeweiligen Instanzen, und zwar jeweils einer Liste pro Segmentierungsart und Kanal;
2. b) Berechnen eines Eigenähnlichkeitswerts für jede Liste, welcher auf den Ähnlichkeiten der Instanzen jedes Kandidatenmusters einer Liste untereinander basiert;
3. c) Berechnen von Koinzidenzwerten für jede Liste jedes Kanals gegenüber den Listen aller anderen Kanäle, welcher jeweils auf den Überlappungen von Instanzen eines Kandidatenmusters der einen Liste mit Instanzen eines Kandidatenmuster der anderen Liste basiert, wenn sich diese zumindest zweimal überlappen; und
4. d) Verknüpfen der Eigenähnlichkeits- und Koinzidenzwerte jeder Liste zu einem Gesamtwert pro Liste und Verwenden der Musterkandidaten der Listen mit dem höchsten Gesamtwert in jedem Kanal als erkannte Notenmuster des Kanals.

This object is achieved by a method of the type mentioned in the introduction, which is characterized by the following steps:

1. a) repeatedly segmenting each channel by varying segment length and start and, for each type of segmentation, determining mutually similar segments and storing them in lists of candidate patterns with their respective instances, one list per segmentation type and channel;
2. b) calculating a self-similarity value for each list based on the similarities of the instances of each candidate pattern of a list with each other;
3. c) calculating coincidence values for each list of each channel against the lists of all other channels, each of which is based on the overlaps of instances of a candidate pattern of the one list of instances of a candidate pattern other list if they overlap at least twice; and
4. d) associating the self-similarity and coincidence values of each list to a total value per list, and using the pattern candidates of the highest-total-value lists in each channel as recognized pattern notes of the channel.

The method of the invention thus takes into account, for the first time and significantly, the parallel structural information of a multi-channel piece of music which may be concealed in the temporal coincidences of potential patterns ("candidate patterns") of different channels and associates these with an assessment of the robustness of found candidate patterns due to the similarities between them Instances, their so-called "fitness". As a result, a much more reliable, meaningful and more accurate pattern recognition result is achieved than with all previously known methods.

It should be noted that the term "channel" used herein for a multi-channel piece of music is to be understood in its most general form, i. both in the sense of a single (monophonic) voice of a polyphonic sentence, in the sense of a (possibly polyphonic) instrumental voice, such as a bass, trumpet, string, percussion, piano parts, etc., as well as in the sense of a technical Channels like a midi channel, which can contain both monophonic and polyphonic voices, parts or their combinations, eg a drum pattern, a chord progression, a string replacement, etc.

• a1) Detektieren der in einem Kanal ident wiederkehrenden Muster, daraus Auswählen der den Kanal bestabdeckenden Muster und Speichern derselben in einer weiteren Liste von Kandidatenmustern mit ihren jeweiligen Instanzen pro Kanal.

A particularly advantageous embodiment of the invention is characterized in that the following step is additionally performed in step a):

• a1) detecting the pattern that identifies in a channel identically, selecting therefrom the pattern best covering the channel and storing it in another list of candidate patterns with their respective instances per channel.

As a result, the degree of recognition can be further increased. The channel-related pattern recognition is thus placed on two equal bases, once an identifier and once a similarity detection, for which variants different Method can be used. By including the recognition results of both variants in one and the same set of candidate patterns, an implicit combination of the two methods results in the subsequent list evaluation by means of the self-similarity and coincidence values, because the results of the two methods compete there. The method of the invention thus becomes "self-adaptive" for different types of input signals which respond differently to different types of recognition methods.

Preferably, in step a1) the detection of identical patterns is carried out by means of the correlation matrix method, as known per se from Hsu Jia-Lien et al. (supra) is known. Particularly preferably, in step a1) the best covering patterns are selected by iteratively selecting the respectively most frequent and / or longest pattern from the detected patterns.

According to a further preferred feature of the invention, in step a) the segment length is varied in multiples of the clock unit of the piece of music, which limits the possible variations to an appropriate level and saves computing time. It is particularly advantageous if the segment length is varied from twice the average note duration of the piece of music to half the length of the piece of music.

According to a further advantageous embodiment of the invention, in step a) the determination of segments which are similar to each other is carried out by aligning the notes of two segments, determining a degree of agreement of the two segments and recognizing similarity, if the degree of agreement exceeds a predetermined threshold value. These measures can be implemented quickly with reasonable computing power.

In particular, the alignment of the notes preferably takes place by means of the "Dynamic Programming" method, as described in Kilian Jürgen et al. (supra) or Hu Ning et al. (supra, with further evidence) known per se.

According to a preferred embodiment of the method, the eigen similarity value is calculated in step b). in that, for each candidate pattern of the list, a similarity matrix of its instances is set up whose values are linked to the self-similarity value of the list, preferably weighted by the channel coverage of the candidate patterns of the list. It has been found that this embodiment leads to a rapid and stable implementation.

To further improve the recognition result, optionally at the end of step b), those lists of a channel whose self-similarity value does not reach a predetermined threshold can be deleted. Preferably, this predetermined threshold value is adaptive, in particular a percentage of the highest self-similarity value of all lists of the channel, particularly preferably at least 70%. In a practically particularly suitable embodiment, the threshold is about 85%.

A further advantageous variant of the method of the invention is that in step c) only the overlaps to those instances of the other list are taken into account for a specific candidate pattern of a list, with which the time-longest overlaps exist. Practical experiments have shown that this leads to a satisfactory recognition rate and simplifies the process in this step.

According to a further preferred variant of the invention it is provided that in linking step d) for each list of each channel only those coincidence values to the lists of the other channels are taken into account, which there represent the highest value, which further improves the recognition rate.

For the same reason, it is preferably provided that, in the combination of step d), the coincidence values taken into account for a list are respectively summed up and the accumulated coincidence values are particularly preferably multiplied by the intrinsic similarity value of the list to said total value.

• die Fig. 1 und 2 ein beispielhaftes mehrkanaliges Musikstück als Eingangssignal des vorliegenden Verfahrens in Musiknotation (Fig. 1) und Notensequenzschreibweise (Fig. 2);
• Fig. 3 ein globales Flußdiagramm des erfindungsgemäßen Verfahrens;
• Fig. 4 ein Beispiel einer Korrelationsmatrix für den Schritt a1) des Verfahrens;
• Fig. 5 das Ergebnis der Detektionsphase von Schritt a1);
• Fig. 6 ein Flußdiagramm für die Auswahlphase für die bestabdeckenden Muster in Schritt a1);
• Fig. 7 das Ergebnis von Schritt a1) in Form einer ersten Liste von Kandidatenmustern und ihren Instanzen für einen Kanal;
• Fig. 8 die Bedeutung der Liste von Fig. 7 in Bezug auf die Kanalabdeckung;
• Fig. 9 verschiedene Segmentierungsarten eines Kanals für die Ähnlichkeitsbestimmung in Schritt a) des Verfahrens;
• Fig. 10 ein Beispiel eines "Dynamic Programming"-Algorithmus zur Ausrichtung zweier Segmente;
• Fig. 11 das Ergebnis der Ausrichtung von Fig. 11 für den Ähnlichkeitsvergleich zweier Segmente;
• Fig. 12 ähnliche und transitiv-ähnliche Segmente eines Kanals, welche die Instanzen eines erkannten Kandidatenmusters darstellen;
• Fig. 13 das Ergebnis von Schritt a) in Form einer weiteren Liste von Kandidatenmustern und ihren Instanzen für einen Kanal und eine bestimmte Segmentierungsart dieses Kanals;
• Fig. 14 das gesamte Ergebnis des Schrittes a), dargestellt als Satz von mehreren Listen für einen Kanal;
• Fig. 15 die Bedeutung der Listen von Fig. 14 in Form verschiedener möglicher Abdeckungen eines Kanals mit jeweils den Kandidatenmustern seiner Listen;
• Fig. 16 eine Ähnlichkeitsmatrix für die Instanzen eines Kandidatenmusters einer Liste als Grundlage für die Berechnung des Eigenähnlichkeitswerts einer Liste gemäß Schritt b);
• Fig. 17 einen Überlappungsvergleich zwischen den Musterinstanzen zweier Listen als Grundlage für die Berechnung der Koinzidenzwerte einer Liste gemäß Schritt c);
• Fig. 18 die Verknüpfung der Eigenähnlichkeits- und Koinzidenzwerte und die Berechnung des Gesamtwerts einer Liste gemäß Schritt d); und
• die Fig. 19 und 20 das Ergebnis der Anwendung des Verfahrens auf das Eingangssignal der Fig. 1 und 2 in Form der möglichen (Fig. 19) und der besten (Fig. 20) Kanalabdeckungen, welch letztere die in den Kanälen erkannten Notenmuster darstellen.

The invention will be explained in more detail below by means of preferred embodiments with reference to the accompanying drawings, in which:

• the Fig. 1 and 2 an exemplary multi-channel piece of music as an input signal of the present method in musical notation ( Fig. 1 ) and note sequence notation ( Fig. 2 );
• Fig. 3 a global flow chart of the method according to the invention;
• Fig. 4 an example of a correlation matrix for step a1) of the method;
• Fig. 5 the result of the detection phase of step a1);
• Fig. 6 a flow chart for the selection phase for the Bestabdeckenden pattern in step a1);
• Fig. 7 the result of step a1) in the form of a first list of candidate patterns and their instances for a channel;
• Fig. 8 the meaning of the list of Fig. 7 in relation to the channel cover;
• Fig. 9 various types of segmentation of a similarity determination channel in step a) of the method;
• Fig. 10 an example of a "Dynamic Programming" algorithm for aligning two segments;
• Fig. 11 the result of the alignment of Fig. 11 for the similarity comparison of two segments;
• Fig. 12 similar and transitive-like segments of a channel representing the instances of a recognized candidate pattern;
• Fig. 13 the result of step a) in the form of another list of candidate patterns and their instances for a channel and a particular segmentation type of that channel;
• Fig. 14 the entire result of step a), represented as a set of multiple lists for a channel;
• Fig. 15 the meaning of the lists of Fig. 14 in the form of various possible covers of a channel, each with the candidate patterns of its lists;
• Fig. 16 a similarity matrix for the instances of a candidate pattern of a list as a basis for the calculation of the self-similarity value of a list according to step b);
• Fig. 17 an overlap comparison between the pattern instances of two lists as a basis for the calculation of the coincidence values of a list according to step c);
• Fig. 18 the concatenation of the self-similarity and coincidence values and the calculation of the total value of a list according to step d); and
• the Fig. 19 and 20 the result of applying the method to the input signal of the Fig. 1 and 2 in the form of the possible ( Fig. 19 ) and the best ( Fig. 20 ) Channel covers, the latter representing the note patterns recognized in the channels.

Fig. 1 shows a section of a piece of music which contains notch sequences q ₁ , q ₂ and q ₃ (generally q _p ) distributed in parallel channels ch ₁ , ch ₂ and ch ₃ (generally ch _p ), which in Fig. 2 are shown schematically. The channels ch _p are, for example, separate MIDI channels for the various instruments or voices of the piece of music, although this is not mandatory, as explained above.

For the sake of simplicity, in the present examples in the note sequences q _p only the pitches and times of occurrence of the individual notes are taken into account, but not other note parameters such as note duration, volume, velocity, envelope, sound, key context, etc. It is understood, however, that all in the The comparisons of individual notes or note patterns as described below can equally well extend to such parameters, if desired, ie multilevel or multidimensional identity or similarity comparisons between several parameters can correspondingly also be carried out in these comparisons.

Moreover, in the present examples, for the sake of simplicity, only monophonic note sequences are considered in each channel. It should be understood, however, that the method presented herein is equally well suited for polyphonic note sequences in the channels, and accordingly, extended identity comparisons, e.g. Chord comparisons and key-context comparisons, etc., can be made.

Thus, as will be apparent to those skilled in the art, the method presented herein is readily scalable to multiple note parameter comparisons and polyphonic note sequences.

• a1) Detektieren der in einem Kanal ident wiederkehrenden Muster, daraus Auswählen der den Kanal bestabdeckenden Muster und Speichern derselben in einer Liste von Kandidatenmustern mit ihren jeweiligen Instanzen pro Kanal.
  1. a) wiederholtes Segmentieren jedes Kanals unter Variierung von Segmentlänge und -beginn und, für jede Segmentierungsart, Bestimmen zueinander ähnlicher Segmente und Speichern derselben in weiteren Listen von Kandidatenmustern mit ihren jeweiligen Instanzen, und zwar jeweils einer Liste pro Segmentierungsart und Kanal;
  2. b) Berechnen eines Eigenähnlichkeitswerts für jede Liste, welcher auf den Ähnlichkeiten der Instanzen jedes Kandidatenmusters einer Liste untereinander basiert;
  3. c) Berechnen von Koinzidenzwerten für jede Liste jedes Kanals gegenüber den Listen aller anderen Kanäle, welcher jeweils auf den Überlappungen von Instanzen eines Kandidatenmusters der einen Liste mit Instanzen eines Kandidatenmuster der anderen Liste basiert, wenn sich diese zumindest zweimal überlappen; und
  4. d) Verknüpfen der Eigenähnlichkeits- und Koinzidenzwerte jeder Liste zu einem Gesamtwert pro Liste und Verwenden der Musterkandidaten der Listen mit dem höchsten Gesamtwert in jedem Kanal als erkannte Notenmuster des Kanals.

Fig. 3 shows the global process of the method based on its basic five steps a1), a), b), c) and d), which will be discussed in detail below. These five global steps are:

• a1) detecting the pattern recurring in a channel, selecting therefrom the pattern best covering the channel and storing it in a list of candidate patterns with their respective instances per channel.
  1. a) repeatedly segmenting each channel by varying segment length and start and, for each type of segmentation, determining mutually similar segments and storing them in further lists of candidate patterns with their respective instances, one list per segmentation type and channel;
  2. b) calculating a self-similarity value for each list based on the similarities of the instances of each candidate pattern of a list with each other;
  3. c) calculating coincidence values for each list of each channel against the lists of all other channels, each based on the overlaps of instances of a candidate pattern of the one list with instances of a candidate pattern of the other list, if they overlap at least twice; and
  4. d) associating the self-similarity and coincidence values of each list to a total value per list, and using the pattern candidates of the highest-total-value lists in each channel as recognized pattern notes of the channel.

The illustrated sequence of steps a1) -a) -b) -c) -d) is only compulsory insofar as some steps presuppose the result of others; otherwise the sequence is arbitrary. For example, the sequence of steps a1) and a) could be reversed, or the sequence of steps b) and c), etc.

In a simplified embodiment of the method, step a1) can optionally be dispensed with, with a correspondingly limited range of applications of the method, as discussed above.

Steps a1) to d) will now be described in detail.

a1) pattern detection by correlation matrix

In step a1) for detecting the identical recurring in a channel ch _p touch pattern (identical "loops") at first, a correlation matrix according to Hsu Jia-Lien et al for each channel ch _p. (supra). Fig. 4 shows an example of such a correlation matrix: the first row and the first column each contain the entire note sequence of a channel in which patterns are to be detected; and only one triangle of the matrix is relevant. The first entry "1" in a row means that a note in the sequence already occurs the second time ; an entry "2" means that the pattern of length 2 ("2-loop") consisting of this and the preceding note occurs a second time; the entry "3" indicates that the pattern of length 3 ("3-loop") consisting of this, the previous and the previous notes appears a second time in this line, etc. For details of the correlation matrix method, reference is made to Hsu Jia. Lien et al. (supra).

By statistical evaluation of the entries in the correlation matrix Fig. 4 For each channel a provisional list can be set according to Fig. 5 in which identity patterns m _I , m _II , m _III , m _IV , etc. found as identically identifiable, with the positions of their occurrence or occurrence in the note sequence q _p , ie their so-called "instances", as well as their length and frequency are listed.

From the preliminary list of Fig. 5 will now be using the in Fig. 6 outlined sketch method those note patterns as "candidate pattern" for further processing, which cover the channel ch _p as largely as possible and also without overlap. This is done according to Fig. 6 in a loop the preliminary list Fig. 5 and (i) searched for the "best" pattern m _I , m _II , etc., (ii) this as a candidate pattern m _1a , m _2a , etc. in a first list L ₁ ( _FIG . Fig. 7 ) and its instances, and (iii) all patterns overlapping with this candidate pattern are removed from the preliminary list Fig. 5 deleted.

• Es wird das häufigste Muster ausgewählt, außer es gibt ein längeres Kandidatenmuster, das mehr als 75% des Kanals abdeckt und mindestens 2/3 so oft auftritt.

The "best" pattern in step (i) is the one in the preliminary list Fig. 5 most common and / or longest patterns m _I , m _II , etc. It is particularly preferable to use the following criterion for the "best" pattern:

• The most common pattern is selected unless there is a longer candidate pattern that covers more than 75% of the channel and occurs at least 2/3 as often.

As a result of step a1), this results in each channel ch _p a first list L ₁ of candidate patterns m _{1 a} m _1b (generally m _1x), that the channel ch _p or its sequence of notes q _p without overlap, and as far as possible, that is without gaps as possible cover, see Fig. 8 ,

a) pattern detection by means of segment similarity comparison

In step a) a second approach is followed. Each channel ch _p (or its note sequence q _p ) is repeated and each segmented in different ways, with varying segment length and start. Fig. 9 shows five exemplary types of segmentation I - V, wherein the segment length in multiples of the clock unit of the piece of music, ie the duration of a beat of the piece of music is varied; For example, at a 4/4 time, the clock unit is a quarter note.

The segmentation types I and II shown are based on a segmentation into segments with a length of two beats, wherein in segmentation II the beginning of the segment has been offset by one beat.

The segmentation types III - V are based on a segment length of three beats and a successive offset of the beginning of the segment by one beat at a time.

It goes without saying that this concept can be extended to any segmentation lengths, starts and also arbitrarily fine quantization units (beats) of the note sequences.

Preferably, the segment length is varied from twice the average note duration of the piece of music to a maximum of half the length of the entire piece of music, since the maximum Length of a note pattern can be at most half the length of the piece of music. If desired, for shortening the method could also be terminated earlier, ie the segment length can be varied, for example, only up to a predetermined number of clocks.

For each possible segmentation type I, II, III, etc., the similarity of the segments S ₁ , S ₂ , etc. is now determined among each other, preferably with the aid of the "Dynamic Programming" method known in the art.

For an explanation of this method, let me just briefly Fig. 10 referenced in which note sequences of two exemplary segments S _s and S _t are compared to each other in a matrix. According to the rules of the "Dynamic Programming" algorithm, weights are now allocated for the progression from cell to cell, eg in the present example the "Dynamic Programming" weights {0, 0, 0, 1} as ("punishment for insert"). , "Penalty for Delete", "Punishment for Replace", "Points for Match"} For details of the "Dynamic Programming" procedure refer to Kilian Jürgen et al. (Ibid.) And Hu Ning et al. (Ibid.) , which also contain other literature references.

Using the "Dynamic Programming" alignment procedure of Fig. 10 Even non-identical, ie only similar and even unequal length note sequences in the segments can be aligned with each other. Fig. 11 shows the obtained alignment result.

The similarity of the segments S _s and S _t is then evaluated using a correspondingly selected scoring scheme between 0% (unlike) and 100% (ident), for example based on the number of identical scores, the number of gaps, the pitch of deviating scores, etc. Two Segments S _s , S _t are then recognized as "similar" if their similarity value determined in this way is above a predetermined threshold value, preferably above 50%.

In this way, all segments S _{s are} compared with all other segments S _{t of} a segmentation type I, II, etc. of a channel ch _p . For example, this leads to the recognition of a similarity for the segmentation type II of the channel ch _p between the segments S ₁ , S ₃ and S ₆ , as in Fig. 12 The segments S ₁ and S ₃ here are 50% similar, the segments S ₃ and S ₆ 60% similar, and the segments S ₁ and S ₆ "transitivähnlich" to 40%.

All similar or even transitive-like segments are now again understood as instances i _i, a candidate pattern resulting from the note sequence of one (eg the first) of these segments. The candidate patterns thus found for a segmentation type of a channel are stored in the form of another list L ₂ of candidate patterns m _2a , m _2b , etc. with their respective instances i ₁ , i ₂ , etc., see Fig. 13 ,

All lists L ₂ , L ₃ , etc. for all possible sequencing types I, II, etc. of a channel ch _p , together with the first list L ₁ of step a1) discussed above, yield a set of lists L _n for each channel ch _p , please refer Fig. 14 which represents various possible covers of the channel ch _p with candidate patterns, see Fig. 15 ,

The lists L _n are now evaluated in the following steps b), c) and d).

b) Calculation of Eigen Similarity Values

First, in step b), for each list L _n, a self-similarity value E _{n is} calculated on the basis of similarity matrices for all candidate patterns m _na , m _nb , etc. (generally m _nx ) of the list L _n . Fig. 16 Figure 10 shows an exemplary similarity matrix for the instances i ₁ , i ₂ , i ₃ and i _{4 of} a candidate pattern m _{n of} the list L _n : The cells of the matrix give the degree of similarity, for example as determined according to the "Dynamic Programming" step of step a) , again; For example, here the similarity between instance i ₁ and instance i _{3 is} 80%.

From all values of the similarity matrix Fig. 16 a self-similarity value E _nx for the candidate pattern m _nx is now determined, for example by summing up in the form: $$e_{\mathit{nx}}={\displaystyle \underset{k.l}{\Sigma }}\ddot{A}\mathit{similarity\; between}{i}_k\mathrm{and}{i}_l\mathrm{,}$$

• wenn mindestens eine Zelle pro Zeile den Eintrag "1" hat, dann wird E_nx um 2 inkrementiert d.h. $${\mathrm{E}}_{\mathrm{nx}}\mathrm{:}\mathrm{=}{\mathrm{E}}_{\mathrm{nx}}\mathrm{+}\mathrm{2}\mathrm{;}$$
  
• wenn nicht, dann wird E_nx nur um den Durchschnittswert aller Zellen dieser Zeile inkrementiert, d.h. $${\mathrm{E}}_{\mathrm{nx}}\mathrm{:}\mathrm{=}{\mathrm{E}}_{\mathrm{nx}}\mathrm{+}\mathrm{Zeilendurchschnitt}\mathrm{.}$$
  

Alternatively, an evaluation scheme can also be used which statistically evaluates or evaluates the values in the cells of the similarity matrix, preferably in the form:

• if at least one cell per line has the entry "1", then E _nx is incremented by 2 ie $${\mathrm{e}}_{\mathrm{nx}}\mathrm{:}\mathrm{=}{\mathrm{e}}_{\mathrm{nx}}\mathrm{+}\mathrm{2}\mathrm{;}$$
  
• if not, then E _{nx is} only incremented by the average of all cells of that row, ie $${\mathrm{e}}_{\mathrm{nx}}\mathrm{:}\mathrm{=}{\mathrm{e}}_{\mathrm{nx}}\mathrm{+}\mathrm{Line\; average}\mathrm{,}$$
  

The self-similarity value E _{nx of} the candidate pattern m _nx is also referred to as "loopfitness" of the candidate pattern m _nx .

The self-similarity value E _{n of} the list L _n is then obtained as the sum of the eigen similarity values E _{nx of} all candidate patterns m _{nx of} the list L _n multiplied by the channel coverage P which reach all instances of all candidate _patterns m _{nx of} the list L _n , ie $$e_n={\displaystyle \underset{x}{\Sigma }}{e}_{\mathit{nx}}*P_n\mathrm{,}$$

• die zeitliche Abdeckung des Kanals verstanden, als Summe der Zeitdauern t_nxi aller Instanzen i aller Kandidatenmuster m_nx des Kanals, bezogen auf die Gesamtdauer T_p des Kanals ch_p; oder
• die notenmäßige Abdeckung des Kanals, als Summe der Notenanzahlen n_nxi in allen Instanzen i aller Kandidatenmuster m_nx des Kanals, bezogen auf die Gesamtanzahl N_p von Noten des Kanals ch_p; oder bevorzugt
• sowohl die zeitliche als auch die notenmäßige Abdekkung in gewichteter Form, beispielsweise gleich gewichtet, d.h.: $$P_n=\frac{1}{2}\left(\frac{{\displaystyle \sum _{\mathit{x}\mathit{,}i}}{t}_{\mathit{nxi}}}{T_p}+\frac{{\displaystyle \sum _{\mathit{x}\mathit{,}\mathit{i}}}{n}_{\mathit{nxi}}}{N_p}\right)$$
  

Under the channel coverage P _{n of} a list L _{n of} a channel ch _p becomes either

• the temporal coverage of the channel, as the sum of the durations t _{nxi of} all instances i of all candidate _patterns m _{nx of} the channel, relative to the total duration T _{p of} the channel ch _p ; or
• the nominal coverage of the channel, as the sum of the number of notes n _nxi in all instances i of all candidate _patterns m _{nx of} the channel, relative to the total number N _p of notes of the channel ch _p ; or preferred
• both the temporal and the nominal coverage in weighted form, for example, equally weighted, ie: $$P_n=\frac{1}{2}\left(\frac{{\displaystyle \underset{\mathit{x}\mathit{.}i}{\Sigma }}{t}_{\mathit{nxi}}}{T_p}+\frac{{\displaystyle \underset{\mathit{x}\mathit{.}\mathit{i}}{\Sigma }}{n}_{\mathit{nxi}}}{N_p}\right)$$
  

In an optional step that lists L can, after determining the intrinsic similarity values E _n of the lists L _n, for example, immediately after step b), for a specific channel ch _p all _n of the channel ch are deleted _p whose intrinsic similarity values E _n a predetermined Do not reach threshold. The threshold value can preferably be specified adaptively or dynamically, for example as a percentage of the highest eigennormality value E _{n of} all lists L _{n of} the channel ch _p , eg as at least 70% or particularly preferably as approximately 85% of the highest eigennormality value E _{n of} all lists L _n des Channel ch _p .

c) Calculation of the coincidence values

In step c), for each list L _n, coincidence values are calculated, between each list L _{n of} each channel ch _p and each list L _{n of} each other channel ch _{p '} , as in US _Pat Fig. 17 and 18 outlined.

Fig. 18 shows - representative of all these coincidence value calculations - the first list L _{21 of} the channel ch ₂ , which is in each case compared with all other lists (but not with the lists of its own channel ch ₂ ) in order to obtain coincidence values K _21-12 , K _{21- 31} , etc., generally K _{pn-p'n '} (with p' ≠ p), from which then a total coincidence value K _pn for each list L _{pn is} determined, as described below.

According to Fig. 17 is a coincidence value from the temporal overlaps u of the instances i _{i of} two lists to be compared - Fig. 17 for the sake of simplicity only referred to as L ₁ and L ₂ - calculated: The coincidence value K _{pn-p'n '} is the sum of all time durations t _{i of} those instance overlaps u, which are considered below, based on the time duration T of the entire considered channel ch _p .

Only those overlaps u of instances i _{i of} a candidate pattern m _{1x of} the list L ₁ with instances i _{i of} the candidate patterns m _{2x of} the other list L _{2 are} taken into account, which occur at least twice, and then only those overlaps u which match the candidate candidates - longest overlap times generate t _i . In the example of Fig. 17 this means that the candidate pattern m _1b (ie its three instances i _i , i ₂ , i ₃ ) overlaps three times with instances of one and the same candidate pattern of the second list L ₂ , namely with the three instances i _i , i ₂ and i ₅ the candidate pattern m _2a at the overlap times t ₁ , t ₂ and t ₅ ; and only these overlap times are considered for the candidate pattern m _1b .

All further overlaps of the candidate pattern m _1b with instances of other candidate patterns, for example the instances i ₁ and i ₄ of m _2b , are _disregarded because these overlaps are shorter than the aforementioned ones. Also, the repeated overlapping of the instance i ₂ of m _1b with the instance i ₃ of m _2a is not counted, but only a single double overlap per instance of the first list L ₁ , namely the time-longest.

Likewise, repeated overlaps v of the instances i ₁ and i _{2 of} the candidate pattern m _1a with the instances i ₁ and i _{2 of} the candidate _pattern m _{2b are} disregarded because already the overlaps u of the instances i ₃ and i ₄ of m _1a with the instances i ₁ and i _u of m _{2a were} taken into account.

The coincidence _value K _{pn-p'n '} can optionally be for instances _coinciding exactly in their beginning or end - in the example shown in _FIG Fig. 17 the coincident beginnings of the first instances i _{1 of} the candidate patterns m _1b and m _2a and the coincidence of the ends of the third instances i ₃ of m _1a and m _2a and the starts of the fourth instances i ₄ of m _1a and m _2a - for each coincidence especially increased, for example by a predetermined "bonus value" be incremented.

Coming back to the general notation of Fig. 18 Thus, for example, for the list L _{pn of} the channel ch _p, the following coincidence _values K _{pn-p'n 'result} from the lists of all other channels: $$K_{\mathit{pn}-\mathit{p\text{'}n\text{'}}}=\frac{{\displaystyle \underset{i}{\Sigma }}{t}_{\mathit{pn}.i}}{T_p}\mathrm{With}{\mathrm{T}}_{\mathrm{p}}={\mathrm{T}}_{\mathrm{p\text{'}}}\mathrm{,}$$

d) Linking the self-similarity and coincidence values

The eigen similarity values E _pn and coincidence _values K _{pn-p'n '} determined for each list L _pn are linked to a total value G _{pn of} the list L _pn , for example by summing, multiplying or other mathematical operations.

Preferably, the following relationship is applied: As in Fig. 18 For example, for a list, for example the first list L _{21 of} the second channel ch ₂ , only those coincidence _values K _{21-p'n 'are} taken into account with respect to the lists L _{p'n' of} the other channels ch _{p '} which are present in each channel each have the highest value. In the example shown, these are the coincidence value K _21-12 to the second list L _{12 of} the first channel ch ₁ and the coincidence value K _21-31 to the first list L _{31 of} the third channel ch ₃ .

These channel-maximum coincidence values are added up to a total coincidence value K _pn for the list L _pn , ie: $$K_{\mathit{pn}}={\displaystyle \underset{\mathit{p\text{'}}}{\Sigma }}\underset{\mathit{in\; p\text{'}}}{\mathrm{Max}}\left(K_{\mathit{pn}-\mathit{p\text{'}n\text{'}}}\right)\mathrm{,}$$

The total coincidence value K _{pn of} the list L _pn is then multiplied by the intrinsic similarity value E _{pn of} the list L _pn to give a total value G _pn for the list L _pn : $$G_{\mathit{pn}}=e_{\mathit{pn}}*K_{\mathit{pn}}$$

hat. In dem in Fig. 19 gezeigten Beispiel, welches auf den Eingangssequenzen der Fig. 1 und 2 beruht, sind das die Liste L₁₂ als Ergebnisliste L₁ des ersten Kanals ch₁, die Liste L₂₁ als Ergebnisliste L₂ des zweiten Kanals ch₂ und die Liste L₃₃ als Ergebnisliste L₃ des dritten Kanals ch₃.Subsequently, in each channel ch _p that list L _{p is} sought which has the highest total value G _p $$G_{\mathit{p}}=\underset{\mathit{in\; p}}{\mathrm{Max}}\left(G_{\mathit{pn}}\right)$$

Has. In the in Fig. 19 shown example, which on the input sequences of Fig. 1 and 2 that is, the list L ₁₂ as the result list L _{1 of} the first channel ch ₁ , the list L ₂₁ as the result list L _{2 of} the second channel ch _2, and the list L ₃₃ as the result list L _{3 of} the third channel ch ₃ .

The candidate _patterns m _{px of} the lists L _p thus represent the best known for each channel ch _p , taking into account its structural relationships with all other channels, similar recurring note pattern of the channel, as in Fig. 20 shown.

The invention is not limited to the illustrated embodiments, but includes all variants and modifications that fall within the scope of the appended claims.

Claims (15)

1. Method for recognising similarly recurring patterns of notes in a piece of music, which contains note sequences (q) distributed on parallel channels (ch), with the following steps:

   a) repeatedly segmenting each channel (ch) by varying segment length and segment beginning and, for each type of segmentation, determining segments (S) that are similar to one another and storing these in lists (L) of candidate patterns (m) with their respective instances (i), i.e. one list respectively for each type of segmentation and channel;

   b) calculating an intrinsic similarity value (E) for each list (L), which is based on the similarities of the instances (i) of each candidate pattern (m) of a list with one another;

   c) calculating coincidence values (K) for each list (L) for each channel (ch) with respect to the lists for all other channels, which is respectively based on the overlaps (u) of instances (i) of a candidate pattern (m) of one list (L) with instances (i) of a candidate pattern (m) of the other list (L) when these overlap at least twice; and

   d) combining the intrinsic similarity and coincidence values (E, K) for each list (L) to form a total value (G) for each list and using the pattern candidates (m) in the lists (L) with the highest total value (G) in each channel (ch) as recognised note patterns in the channel.
2. Method according to claim 1, characterised in that in step a) the following step is additionally conducted:

   a1) detecting the patterns (m) identically recurring in a channel (ch), selecting therefrom the patterns best covering the channel and storing these in a further list (L) of candidate patterns (m) with their respective instances (i) for each channel.
3. Method according to claim 2, characterised in that in step a1) the detection of identically recurring patterns (m) is conducted by means of the correlation matrix method known per se.
4. Method according to claim 2 or 3, characterised in that in step a1) the selection of the best covering patterns (m) is achieved by iterative selection of the respective most frequent and/or longest pattern from the detected patterns.
5. Method according to one of claims 1 to 4, characterised in that in step a) the segment length is varied in multiples of the rhythmic unit of the piece of music.
6. Method according to claim 5, characterised in that the segment length is varied from double the average note duration of the piece of music to half the length of the piece of music.
7. Method according to one of claims 1 to 6, characterised in that in step a) the determination of segments (S) that are similar to one another is achieved by aligning the notes of two segments with one another, determining a degree of consistency of both segments and recognising similarity when the degree of consistency exceeds a preset threshold value.
8. Method according to claim 7, characterised in that the alignment of the notes is achieved by means of the "dynamic programming" method known per se.
9. Method according to one of claims 1 to 8, characterised in that in step b) for each candidate pattern (m) for the list (L) a similarity matrix of its instances (i) is drawn up, the values of which are combined to form the intrinsic similarity value (E) for the list (L), preferably with weighting by the channel coverage (P) of the candidate patterns (m) for the list (L).
10. Method according to one of claims 1 to 9, characterised in that at the end of step b) those lists (L) for a channel (ch) whose intrinsic similarity value (E) do not reach a preset threshold value are deleted.
11. Method according to claim 10, characterised in that the preset threshold value is a percentage of the highest intrinsic similarity value (E) of all lists (L) for the channel (ch), preferably at least 70%, particularly preferred about 85%.
12. Method according to one of claims 1 to 11, characterised in that in step c) for a specific candidate pattern of a list (L) only the overlaps (u) with those instances (i) of the other list (L), with which the longest overlaps in time are present, are taken into consideration.
13. Method according to one of claims 1 to 12, characterised in that in combining step d) for each list (L) for each channel (ch) only those coincidence values (K) to the lists (L) of the other channels (ch) that represent the respectively highest value there are taken into consideration.
14. Method according to one of claims 1 to 13, characterised in that in combining step d) the coincidence values (K) taken into consideration for a list (L) are respectively added up.
15. Method according to claim 14, characterised in that in combining step d) the added coincidence values (K) are multiplied by the intrinsic similarity value (E) for the list (L) to form the said total value (G).

Images