WO2006034743A1 - Device and method for arranging in groups temporal segments of a piece of music - Google Patents

Abstract

The invention relates to a method for arranging in various segment classes temporal segments of an audio piece which is structured into main parts which recur in the audio piece. For this purpose, a similarity representation of the segments is established (10). Said similarity representation comprises for each segment an associated plurality of similarity values which indicate the degree of similarity of the segment with every other segment of the piece of music. The similarity values associated with the segment are used to calculate a similarity threshold value for a segment (12) in order to allocate a segment to a segment class (14) if the similarity value of the segment meets a defined relationship with respect to the similarity threshold value. The invention provides a clustering method which works efficiently and correctly even in cases where the segments have very different or almost identical combined similarity values.

Description

 Apparatus and method for grouping temporal segments of a piece of music

description

The present invention relates to the audio segmentation and in particular to the analysis of pieces of music on the individual Haupttei¬ contained in the pieces of music, which may occur repeatedly in the piece of music.

Music from the rock and Popbereicri consists mostly of more or less unique segments, such as intro, verse, chorus, bridge, Oαtro, etc. The beginning and end times of such segments to detect and the segments according to their affiliation to the most important Klas¬ Grouping the stanza (verse and chorus) is the goal of audio segmentation. Correct segmentation and also identification of the calculated segments can be usefully used in various areas. For example, pieces of music from online providers such as Amazon, Mu- sicline, etc. can be intelligently "played".

Most providers on the Internet limit their listening examples to a short excerpt from the available pieces of music. In this case, it would of course also make sense to offer the interested party not just the first 30 seconds or any 30 seconds, but a section of the song that is as representative as possible. This could be z. For example, the chorus, or: but also a summary of the song, consisting of segments that belong to the various main classes (stanza _e chorus, ...).

Another application example for the technique of audio segmentation is the integration of the segmentation / Grouping / marking algorithm into a music player. The information about segment beginnings and segment ends makes it possible to navigate through a piece of music. Due to the class affiliation of the segments, ie whether a segment is a verse, a chorus, etc., z. B. also jump directly to the next chorus or the next stanza. Such an application is of interest to large music markets, offering their customers the opportunity to listen to complete albums. As a result, the customer spares himself the annoying, searching fast-forward to characteristic points in the song, which could perhaps lead him to actually buy a piece of music in the end.

There are different approaches in the field of audio segmentation. In the following, the approach of Jonathan Foote and Matthew Cooper is exemplified. This process is described in FOOTE, J.T. / Cooper, M.L. : Summarizing Populary Music via Structural Similarity Analysis. Proceedings of the IEEE Workshop on Signal Processing to Audio and Acoustics 2003. FOOTE, J.T. / COOPER, M.L .: Media Segmentation using Self-Similar Decomposition. Proceedings of SPIE Storage and Retrieval for Multimedia Databases, Vol. 5021, pp. 167-75, January 2003.

The known method of Foote is explained by way of example with reference to the block diagram of FIG. First, a WAV file 500 is provided. In a subsequent extraction block 502, a feature extraction then takes place, wherein as a feature the spectral coefficients per se or alternatively the mel frequency cepstral coefficients (MFCCs) are extracted. Prior to this extraction, a short-time Fourier transform (STFT) is performed with 0.05 second wide non-overlapping windows with the WAV file. The MFCC features are then extracted in the spectral range. It should be noted that the parameterization is not optimized for compression, transmission or reconstruction, but for a Audio analysis. The requirement is that similar audio pieces produce similar features.

The extracted features are then stored in a memory 504.

The feature extraction algorithm now has a segmentation algorithm that ends in a similarity matrix, as shown in a block 506. First, however, the feature matrix is read in (508), to then group feature vectors (510), to then build a similarity matrix based on the grouped feature vectors, which consists of a distance measurement between each of all features. In particular, all pairs of audio window pairs are compared using a quantitative similarity measure, distance.

The structure of the similarity matrix is shown in FIG. 8. Thus, in Fig. 8, the music piece is represented as a stream or stream 800 of audio samples. The piece of audio is windowed, as has been stated, with a first window having i and a second window being j. Overall, the audio piece has z. B. K window. This means that the similarity matrix has K rows and K columns. Then, for each window i and for each window j, a similarity measure to each other is calculated, and the calculated similarity measure or distance measure D (i, j) is input to the row or column designated by i and j in the similarity matrix. One column therefore shows the similarity of the window designated by j to all other audio windows in the piece of music. The similarity of window j to the very first window of the piece of music would then be in column j and in line 1. The similarity of window j to the second window of the piece of music would then be in column j, but now in line 2. By contrast, the similarity of the second window - A -

to the first window in the second column of the matrix and in the first row of the matrix.

It can be seen that the matrix is redundant in that it is symmetric to the diagonal, and that on the diagonal the similarity of a window is to itself, which is the trivial case of 100% similarity.

An example of a similarity matrix of a piece can be seen in FIG. Here again the completely symmetrical structure of the matrix with respect to the main diagonal is recognizable, the main diagonal being visible as a light stripe. It should also be noted that due to the small window length compared to the relatively coarse time resolution in Fig. 6, the main diagonal is not seen as a lighter solid line, but from Fig. 6 is only approximately recognizable.

This is done using the similarity matrix as described, for. For example, as shown in FIG. 6, a kernel correlation 512 is performed on a kernel matrix 514 to obtain a novelty measure, also known as a novelty score, that could be averaged and smoothed into The smoothing of these Novelty Scores is shown schematically in FIG. 5 by a block 516.

Then, in a block 518, the segment boundaries are read out using the smoothed novelty value profile, for which purpose the local maxima in the smoothed novelty curve are determined and, if appropriate, must be shifted by a constant number of samples caused by the smoothing in order actually to produce the correct segment - To obtain limits of the audio piece as absolute or relative Zeitan¬ gift. - D

Then, as is already apparent in a block of FIG. 5 designated with clustering, a so-called segment similarity representation or segment similarity matrix is created. An example of a segment similarity matrix is shown in FIG. The similarity matrix in FIG. 7 is basically similar to the feature similarity matrix of FIG. 6, but now, as in FIG. 6, features from windows are no longer used, but features from a whole segment. The segment-like

It is similar to the feature similarity matrix, but with a much coarser resolution, which is of course desired when considering that window lengths are in the order of 0.05 seconds, while reasonably long segments in the range of perhaps

1.5 10 seconds of a piece lie.

Then, in a block 522, a clustering is carried out, that is, an arrangement of the segments into segment classes (an arrangement of similar segments into the same

20 segment class), then in a block 524 those found

Labeling also determines which segment class contains segments which are stanzas, which are reflections, which are intros, outros, bridges, etc. 25

Finally, in a block labeled 526 in FIG. 5, a music score is created, which is e.g. B. can be provided to a user, without redundancy of a piece only z. B. a stanza, a chorus and the intro

30 to hear.

The individual blocks will be discussed in more detail below.

As has already been explained, the actual segmentation of the piece of music does not take place until the feature matrices have been generated and stored (Block - Q -

Depending on which feature the music piece is to be examined for its structure, the corresponding feature matrix is read out and loaded into a main memory for further processing. The feature matrix has the dimension number of analysis windows times the number of Merkiinalskoeffizienten.

By the similarity matrix, the characteristic curve of a piece ± n is given a two-dimensional representation. For each pairwise combination of feature vectors, the distance measure is calculated, which is kept fixed in the similarity matrix. There are various possibilities for calculating the distance measure between two vectors, for example the Euclidean distance measurement and the cosine distance measurement. A result D (i, j) between the two feature vectors is stored in the i, jth element of the window similarity matrix (block 506). The main diagonal of the similarity matrix represents the course over the entire piece. Accordingly, the elements of the Hamptdiagonalen result from the respective comparison of a window with itself and always have the value of the greatest similarity. For the cosine distance measurement this is de: r value 1, for the simple scalar difference and the Euclid distance this value is equal to 0.

To visualize a similarity matrix, as shown in FIG. 6, each element i, j is assigned a gray value. The gray values are graduated in proportion to the similarity values, so that the maximum similarity (the main diagonal) corresponds to the maximum similarity. By means of this representation, the structure of a song can already be visually recognized on the basis of the matrix. Areas of similar feature expression correspond to quadrants of similar brightness along the main diagonal. Finding the boundaries between the areas is the task of the actual segmentation. The structure of the similarity matrix is important to the novelty measure calculated in kernel correlation 512. The measure of novelty arises from the correlation of a special kernel along the main diagonal of the similarity matrix. An exemplary kernel K is shown in FIG. If one compares this kernel matrix along the main diagonal of the similarity matrix S, and sums up all the products of the superimposed matrix elements for each time point i of the piece, then one obtains the measure of novelty, which is shown by way of example in FIG. 9 in a smoothed form , Preferably, not the kernel K in FIG. 5 is used, but an enlarged kernel, which is additionally superimposed with a Gaussian distribution, so that the edges of the matrix tend towards zero.

The selection of: r? striking maxima in the novelty course is important for the segmentation. The selection of all maxima of the unsmoothed Neiαheitsverlaufs would lead to a strong Über¬ segmentation of the audio signal.

Therefore, the novelty measure should be smoothed with different filters, such as IIR filters or FIR filters.

If the segment boundaries of a piece of music are extracted, then similar segments must be identified as such and grouped into classes.

Foote and Cooper describe the computation of a segment-based similarity matrix using a Cullback-Leibler distance. For this purpose, individual segment feature matrices are extracted from the entire feature matrix on the basis of the segment boundaries obtained from the course of novelty, ie each of these matrices is a submatrix of the entire feature matrix. The resulting segment similarity matrix 520 is now subjected to a singular value decomposition (SVD, SVD = Singular Value Decomposition). Then one obtains singular values in descending order.

In block 526, an automatic summary of a piece is then carried out on the basis of the segments and clusters of a piece of music. To do this, first select the two clusters with the largest x-ray values. Then, the segment with the maximum value of the corresponding cluster indicator is added to this summary. This means that the summary includes a stanza and a chorus. Alternatively, all repeated segments can also be removed to ensure that all information of the piece is provided, but always exactly once.

Concerning further techniques for segmentation / music analysis, CHU, s. / LOGAN B .: Music Summary using Key Phases. Technical Report, Cambridge Research Laboratory 2000, BARTSCH, M.A. / WAKEFIELD, g. H.: To Catch a Chorus: Using Chroma-Based Representation for Audio Thumbnailing. Certifications of the IEEE Workshop on Signal Processing to Audio and Acoustics 2001. http: //musen.engin.umich.edu/papers/bartsch wakefield waspaaOl final.pdf, referenced

A disadvantage of the known method is the fact that the singular value decomposition (SVD) for segment class formation, that is to say the assignment of segments to clusters, is very computationally intensive and is problematic in the evaluation of the results. Thus, if the singular values are nearly equal, then a possibly wrong decision is made to the effect that the two similar singular values actually represent the same segment class and not two different segment classes.

Furthermore, it has been found that the results obtained by the singular value decomposition will become more and more problematic if it has strong similarity values. there are differences, such as stanza and chorus, but also relatively unimportant parts, such as intro, outro or bridge.

Further problematic in the known method is that it is always assumed that the cluster among the two clusters with the highest singular values having the first segment in the song is the cluster "stanza", and that the other cluster is the cluster "chorus "is. This procedure is based on the fact that it is assumed in the known method that a song always starts with a stanza. Experience has shown that this will result in considerable faulty faults. This is problematic insofar as labeling is effectively the "harvesting" of the entire process, ie what the user experiences directly.Were the preceding steps were still so precise and complex, everything gets relativised, if in the end wrong is labeled, because then the user could lose confidence in the entire concept as a whole.

It should also be noted at this point that there is a particular need for automatic music analysis methods without the result always being able to be checked and, if appropriate, corrected. Instead, a method can only be used on the market if it can run automatically without human correction.

The object of the present invention is to provide an improved and at the same time efficient concept for grouping temporal segments of a piece of music.

This object is achieved by a device for grouping according to claim 1, a method for grouping according to claim 16 or a computer program according to patent claim 17. The present invention is based on the finding that the assignment of a segment to a segment class is to be carried out on the basis of an adaptive similarity mean value for a segment such that the overall similarity score is taken into account by the average of the average has a segment throughout the piece. After such a similarity mean has been calculated for a segment, for the calculation of which the number of segments and the similarity values of the plurality of similarity values assigned to the segment are required, then the actual allocation of a segment becomes a segment class, ie a cluster , performed on the basis of this similarity mean. If a similarity value of a segment to the segment just considered is above the similarity mean, for example, the segment is assigned as belonging to the segment class currently being considered. On the other hand, if the similarity value of a segment to the segment under consideration is below this similarity mean, it is not assigned to the segment class.

In other words, this means that the assignment is no longer made dependent on the absolute size of the similarity values, but relative to the mean of similarity. This means that for a segment which has a relatively low similarity score, ie, for example, For example, for a segment having an intro or outro, the similarity mean will be lower than for a segment that is a stanza or chorus. Thus, the strong deviations of the similarities of segments in pieces or the frequency of the occurrence of certain segments in pieces are taken into account, whereby z. B. numerical problems and thus ambiguities and da¬ associated false allocations can be avoided.

The concept according to the invention is particularly suitable for music pieces which do not consist only of stanzas and choruses, ie have the segments which belong to the segment class. have the same similarity values, but also for pieces that have other parts in addition to stanza and chorus, namely an introduction (Intro), an interlude (Bridge) or a finale (Outro).

In a preferred embodiment of the present invention, the calculation of the adaptive similarity mean and the assignment of a segment are performed iteratively, ignoring assigned segments at the next iteration run. Thus, the similarity absolute value, that is to say the sum of the similarity values in a column of the similarity matrix, changes again for the next iteration run since already assigned segments have been set to 0.

In a preferred embodiment of the present invention, a segmentation post-correction is carried out, in such a way that after the segmentation z. B. due to the novelty value (the local maxima of novelty value) and after a subsequent assignment to segment classes relatively short segments are examined to see if they can be assigned to the predecessor segment or the successor ger segment, as segments below a minimal segment length is likely to indicate over-segmentation.

In an alternative preferred embodiment of the present invention, a labeling is carried out after the final segmentation and assignment into the segment classes, specifically using a special selection algorithm in order to obtain the most correct possible labeling of the segment classes as a stanza or chorus.

Preferred embodiments of the present invention will be explained in detail below with reference to the accompanying drawings. Show it: 1 shows a block diagram of the device according to the invention for grouping according to a preferred embodiment of the present invention;

FIG. 2 shows a flow chart for illustrating a preferred embodiment of the invention for iteratively assigning; FIG.

3 is a block diagram of the operation of the segmentation correction device;

Figures 4a and 4b show a preferred embodiment of the segment class designator;

5 shows an overall block diagram of an audio analysis tool;

6 shows an illustration of an exemplary feature similarity matrix;

7 shows an exemplary representation of a segment similarity matrix;

FIG. 8 shows a schematic representation for illustrating the elements in a similarity matrix S; FIG. and

9 is a schematic representation of a smoothed novelty value.

1 shows a device for grouping temporal segments of a piece of music, which is subdivided into main parts which repeatedly appear in the piece of music, into different segment classes, one segment class being assigned to a main part. The present invention thus relates particularly to pieces of music which are subject to a certain structure in which similar sections appear several times and alternate with other sections. The Most rock and pop songs have a clear structure in terms of their main parts.

The literature deals with the topic of music analysis mainly on the basis of classical music, but it also applies to rock and pop music. The main parts of a piece of music are also called "large shaped parts." A large shaped part of a piece is a section which has a relatively uniform character with regard to various features, eg, melody, rhythm, texture, etc. Definition applies generally in music theory.

Large moldings in rock and pop music are z. Eg verse, chorus, bridge and solo. In classical music, an interplay of refrain and other parts (couplets) of a composition is also called rondo. In general, the couplets contrast with the refrain, for example with regard to melody, rhythm, harmony, key or instrumentation. This can also be transferred to modern light music. Just as there are various forms in the rondo (chain rondo, bowed rondo, sonata rondo), there are also proven patterns in rock and pop music for setting up a lie. These are, of course, only a few possibilities. Ultimately, of course, the composer decides how his piece is constructed. An example of a typical structure of a rock song is the pattern.

A-B-A-B-C-D-A-B,

where A equals strophe, B equals refrain, C equals bridge, and D equals solo. Often a piece of music is introduced with a prelude (Intro). Intros often consist of the same chord progression as the stanza, but with different instrumentation, eg. B. without drums, without bass or distortion of the guitar in rock songs etc.

The device according to the invention initially comprises a device 10 for providing a similarity representation for the segments, the similarity representation for each segment having an associated plurality of similarity values, the similarity values indicating how similar the segment is to each other segment. The similarity representation is preferably the segment similarity matrix shown in FIG. It has for each segment (in Fig. 7 segments 1-10) its own column which has the index " ^jλλ . Furthermore, the similarity representation has a separate line for each segment, one line being designated by a line index i. This will be referred to below with reference to the exemplary segment 5. The element (5, 5) in the main diagonal of the matrix of FIG. 7 is the similarity value of the segment 5 with itself, ie the maximum similarity value. Furthermore, the segment 5 is still medium-le to the segment no. 6, as it is denoted by the element (6,5) or by the element (5,6) of the matrix in Fig. 7. Moreover, the segment 5 is still similar to the segments 2 and 3 as shown by elements (2, 5) or (3, 5) or (5, 5) or (5, 3) in FIG. 7. To the other segments 1, 4, 7, 8, 9, 10, the segment No. 5 has a similarity, which is no longer visible in Fig. 7.

A plurality of similarity values assigned to the segment is, for example, a column or a row of the segment similarity matrix in FIG. 7, this column or row indicating by its column / row index which segment it refers to, namely, for example, to the fifth segment, and where this row / column includes the similarities of the fifth segment to each other segment in the piece. Thus, the plurality of similarity values is, for example, a row of the similarity matrix or, alternatively, a column of the similarity matrix of FIG. 7.

The device for grouping temporal segments of the piece of music further comprises means 12 for calculating a similarity mean value for a segment, using the segments and the similarity values of the segment Segment associated with a plurality of similarity values. The device 12 is designed to z. For example, to calculate a similarity mean for column 5 in FIG. If the arithmetic mean value is used in a preferred exemplary embodiment, the device 12 will add the similarity values in the column and divide them by the total number of segments. To eliminate the self - similarity, could of the. As a result of addition, the similarity of the segment to itself can also be deducted, whereby, of course, a division is then no longer to be carried out by all elements, but by all elements less than one.

The means 12 for calculating could alternatively also calculate the geometric mean value, ie each similarity value of a column for: square, in order to sum the quadrated results, in order then to calculate a root from the summation result, which is given by the number The elements in the column (or the number of elements in the column less "1) is to be divided in. Any other average values, such as the median value, etc., can be used as long as the mean value for each column is Similarity matrix is calculated adaptively, that is, a value that is calculated using the similarity values of the plurality of similarity values associated with the segment.

The adaptively calculated similarity threshold is then provided to a segment 14 for assigning a segment to a segment class. The means 14 for assigning is designed to assign a segment to a segment class if the similarity value of the segment fulfills a predetermined condition with respect to the mean of similarity. For example, if the similarity mean value is such that a larger value indicates a greater similarity and a smaller value indicates a lower similarity, the predetermined relationship will be that the similarity value of a segment equal to or above the Similarity mean, in order for the segment to be assigned to a segment class.

In a preferred embodiment of the present invention, further devices exist to realize special embodiments, which will be discussed later. These devices are a segment selection device 16, a segment assignment conflicting device 18, a segmentation correction device 20 and a segment class designation device 22.

The segment selector 16 in FIG. 1 is designed to first calculate an overall similarity value V (j) for each column in the matrix of FIG. 7, which is determined as follows:

V (j) = ΣS _s (i, j) -Sj j = \, ..., P

(= 1

P is the number of segments. SÄ is the value of the self-similarity of a segment with itself. Depending on the verwen¬ deter- mined technique, the value z. B. zero or one. The segment selector 16 will first: calculate the value V (j) for each segment to then find the vector element i of the maximum value vector V. In other words, this means that the column in FIG. 7 is selected, which achieves the greatest value or score when the individual similarity values in the column are added up. This segment could, for example, be the segment No. 5 or the column 5 of the matrix in FIG. 7, since this segment has at least a certain similarity with three other segments. Another candidate in which to Example of Fig. 7 also Segmen t ^~ could be No. 7., Since this segment has a certain similarity also to three other segments which also is still greater than the similarity of the segment 5 to Segments 2 and 3 (higher shade of gray in Fig. 7). For the following example, it is now assumed that the segment selection device 16 selects segment No. 7, since it has the highest similarity score on the basis of the matrix elements (1, 7), (4, 7) and (10, 7) , In other words, this means that V (7) is the component of the vector V which has the maximum value among all the components of V.

Now, the similarity score of column 7, that is, for segment no. 7, is still divided by the number "9" in order to obtain from the means 12 the similarity threshold value for the segment.

Then, in the segment similarity matrix for the seventh row or column, it is checked which segment similarities are above the calculated threshold value, ie. H. with which segments the i-th segment has an above-average similarity. All these segments are now also assigned to a first segment class like the seventh segment.

For the present example, it is assumed that the similarity of the segment 10 to the segment 7 is below average, but that the similarities of the segment 4 and the segment 1 to the segment 7 are above average. Therefore, in addition to segment no. 7, segment no. 4 and segment no. 1 are also classified in the first segment class. By contrast, segment No. 10 is not classified in the first segment class due to the below-average similarity to segment No. 7.

After the assignment, the corresponding vector elements V (j) of all segments which were assigned to a cluster in this threshold value analysis are set to 0. In the example these are beside V (7) also the components V (4) and V (I). This immediately means that the 7th, 4th, and 4th columns of the matrix are no longer available for a later maximum search that they are zero, so that they can never be a maximum.

This is roughly equivalent to setting the entries (1,7), (4,7), (7,7) and (10,7) of the segment similarity matrix to 0. The same procedure will be used for the

Column 1 (elements (1,1), (4,1) and (7,1)) and column 4

(Elements (1,4), (4,4), (7,4) and (10, 4)).

Due to the ease of handling, however, the matrix is not changed, but the components of

V, which belong to an assigned segment, are ignored in the next maximum search in a later iteration step.

In a next iteration step, a new maximum is now selected from the remaining elements of V, that is to say V (2), V (3), V (5), V (6), V (8), V (9) and V (IO) searched. The segment no. 5, ie V (5), is expected to yield the largest similarity score. The second segment class then obtains segments 5 and 6. Due to the fact that the similarities to segments 2 and 3 are below average, segments 2 and 3 are not placed in the second order clusters. Thus, the elements V (6) and V (5) are set to 0 by the vector V due to the assignment that has been made, while still the components V (2), V (3), V (8), V { 9) and V (IO) of the vector for the selection of the third-order cluster remain.

Then again a new maximum among the mentioned remaining elements of V is searched for. The new maximum could be V (IO), ie the component of V for the segment 10. Segment 10 thus comes in the third-order segment class. Thus, it could also be found that the segment 7 also has an above-average similarity to the segment 10, although the segment 7 is already identified as belonging to the first segment class. This results in an assignment conflict, which is resolved by the segment assignment conflicting device 18 of FIG. A simple kind of resolution could be to simply not assign the segment 7 into the third segment class and e.g. For example, instead of assigning the segment 4, if for the segment 4 would not also conflict exist.

Preferably, however, in order not to disregard the similarity between the segment 7 and the segment 10, the similarity between 7 and 10 is taken into account in the following algorithm.

Generally, the invention is designed not to discount the similarity between i and k. Therefore, the similarity values S _s (i, k) of segment i and k are compared with the similarity value S _s (i ^* , k), where i ^{* is} the first segment assigned to the cluster C ^* . The cluster or the segment class C ^* is the cluster to which segment k is already assigned on the basis of a previous examination. The similarity value S _s (i ^* , k) is decisive for the fact that the segment k belongs to the cluster C ^* . If S _s (i ^* , k) is greater than S _s (i _/ k), segment k remains in cluster C ^* . If S _s (i ^* , k) is smaller than S _s (i, k), the segment k is taken out of the cluster C ^* and assigned to the cluster C. For the first case, ie if the segment k does not change the cluster membership, a tendency towards the cluster i is noted for the cluster C ^* . Preferably, however, this tendency is also noted when segment k changes cluster membership. In this case, a tendency of this segment to the cluster in which it was originally recorded is noted. These tendencies may advantageously be used in a segmentation correction carried out by the segmentation correction device 20.

The similarity value check is based on the fact that the segment 7 is the "original segment" in the first segment class, in favor of the first segment class. It will remain in the first segment class. However, this fact is taken into account by the fact that segment no. 10 in the third segment class is attested a trend towards the first segment class.

According to the invention, it is thus taken into account that, in particular, for the segments whose segment similarities to two different segment classes exist, these similarities are not ignored but, if appropriate, are taken into account later by the trend or the tendency ,

The procedure continues until all segments in the segment similarity matrix are assigned, which is the case when all elements of vector V are set to zero.

For the example shown in FIG. 7, this would mean that next, in the fourth segment class, the maximum of V (2), V (3), V (8), V (9), ie the segment 2 and 3 einge¬ be classified, then, in a fifth segment class, the segments 8 and 9 to classify until all segments have been assigned. Thus, the iterative algorithm shown in FIG. 2 is completed.

In the following, the preferred implementation of the segmentation correcting device 20 will be described in detail with reference to FIG. 3.

Thus, it follows that in the calculation of the segment boundaries by means of the kernel correlation, but also in the calculation of segment boundaries by means of other measures, an over-segmentation of a piece often occurs, ie too many segment boundaries or generally too short segments te calculated. An over-segmentation, z. B. caused by an incorrect subdivision of the stanza is erfin¬ according to corrected by the fact that due to the segment length and the information in which segment class a predecessor or successor segment has been sorted is corrected. In other words, the correction serves to completely eliminate segments that are too short, ie to merge with adjacent segments, and to segments that are short but not too short, ie that are short in length but longer than that Minimal lengths are still to undergo a special investigation, whether they may not yet be merged with a predecessor segment or a successor segment. In principle, successive segments which belong to the same segment class are always fused together. If the scenario shown in FIG. B. that the segments 2 and 3 come in the same segment class, they are automatically ver¬ melted together, while the segments in the first Segmentklas¬ se, ie the segments 7, 4, 1 are spaced apart and therefore (at least initially) can not be merged. This is indicated in FIG. 3 by a block 30. Now, in a block 31, it is examined whether segments have a segment length smaller than a minimum length. Thus, there are preferably different minimum lengths.

Relatively short segments shorter than 11 seconds (a first threshold) are examined at all, while later on even shorter segments (a second threshold smaller than the first) shorter than 9 seconds are examined and, later, any remaining segments which are shorter than 6 seconds (a third threshold which is shorter than the second threshold) will be treated again alternatively.

In the preferred embodiment of the present invention in which this staggered length check occurs, the segment length check in block 31 is initially directed to finding the segments shorter than 11 seconds. For the segments that are longer than 11 seconds, no post processing is done, as can be recognized by a "No" at block 31. For segments which are shorter than 11 seconds, a trend check (block 32) is first of all carried out, so that at first a check is made as to whether a segment 1 has an associated trend or associated tendency In the example of Fig. 7, this would be the segment 10 which has a trend towards the segment 7 or a trend towards the first segment class the tenth segment is shorter than 11 seconds, in the example shown in FIG. 7, nothing would happen, however, also due to the trend check, since a fusion of the considered segment takes place only if there is no tendency for any cluster, but has a tendency to a cluster of an adjacent (before or after) segment, but this is for the segment 10 in the example shown in FIG not the case.

In order to avoid too short segments, which have no tendency to the cluster of an adjacent segment, the procedure is as shown in the blocks 33a, 33b, 33c and 33d in FIG. Thus, nothing is done on segments longer than 9 seconds, but shorter than 11 seconds. They are left. In a block 33a, however, for segments from the cluster X which are shorter than 9 seconds, and in which both the precursor segment and the successor segment belong to the cluster Y, an assignment to the cluster Y is made, which automatically means that such a segment is merged with both the predecessor and the successor segment, so that an overall longer segment is formed, which is composed of the considered segment and the predecessor as well as the successor segment. Thus, a subsequent merger may result in a combination of initially separate segments over an intervening segment to be merged. In a block 33b it is further explained what happens to a segment that is shorter than 9 seconds and that is the only segment in a segment group. For example, segment no. 10 is the only segment in the third segment class. If it were shorter than 9 seconds, it is automatically assigned to the segment class to which segment No. 9 belongs. This automatically leads to a fusion of the segment 10 with the segment 9. If the segment 10 län¬ ger than 9 seconds, this merger is not made.

In a block 33c an examination is then made for segments shorter than 9 seconds which are not the only segment in a corresponding cluster X than in a corresponding segment group. They are subjected to a more detailed check in which a regularity in the cluster sequence is to be ascertained. At first all segments from the segment group X are searched, which are shorter than the minimum length. Subsequently, it is checked for each of these segments whether the predecessor and successor segments each belong to a uniform cluster. If all predecessor segments are from a uniform cluster, all segments that are too short from cluster X are assigned to the predecessor cluster. If, on the other hand, all successor segments are from a uniform cluster, the segments too short from cluster X are each assigned to the successor cluster.

In a block 33d is executed what happens, even if this condition is not fulfilled for segments that are shorter than 9 seconds. In this case, a novelty value check is performed by resorting to the novelty value curve shown in FIG. 9. In particular, the novelty curve, which has arisen from the kernel correlation, is read out at the locations of the affected segment boundaries, and the maximum of these values is determined. If the maximum occurs at the beginning of a segment, the segments that are too short become the cluster of the successor assigned to ge segments. If the maximum occurs at a segment end, the segments that are too short are assigned to the cluster of the precursor segment. If the segment labeled 90 in Fig. 9 were a segment shorter than 9 seconds, the novelty check at the beginning of the segment 90 would give a higher novelty value 91 than at the end of the segment, with the novelty value at the end of the segment 92 is designated. This would mean that the segment 90 would be assigned to the successor segment since the: novelty value to the successor segment is less than the: novelty value to the predecessor segment.

If segments still remain that are shorter than 9 seconds and have not yet been allowed to merge, a staggered selection is carried out once again below them. In particular, now all segments among the remaining segments shorter than 6 seconds are selected. The segments whose length is between 6 and 9 seconds from this group are allowed "untouched".

However, the segments that are shorter than 6 seconds are now all subjected to the novelty check explained with reference to the elements 90, 91, 92 and assigned to either the predecessor or the successor segment, so that at the end of the in FIG. 3 Nachkorrekturalgorithmus all too short segments, namely all segments below a length of 6 seconds, have been merged intelli¬ gent with predecessor and successor segments.

This procedure according to the invention has the advantage that no elimination of parts of the piece has been carried out, ie that no simple elimination of the segments which are too short has been carried out by setting them to zero, but that the entire complete piece of music is still the one Entity of segments is represented. Due to the segmentation therefore no loss of information auf¬ occurs, which would be, however, if you z. B. as a reaction on over-segmentation, simply eliminating all too short segments "regardless of losses".

Referring now to FIGS. 4a and 4b, a preferred implementation of the segment class designator 22 of FIG. 1 is illustrated. According to the invention, two clusters are assigned the labels "stanza" and "refrain" during labeling.

According to the invention, not a largest singular value of a singular value decomposition and the associated cluster is used as the refrain, and the cluster for the second largest singular word is used as the stanza. Furthermore, it is not fundamentally assumed that each song begins with a stanza, ie that the cluster with the first segment is the stanza cluster and the other cluster is the refrain cluster. Instead, according to the invention, the cluster in the candidate selection having the last segment is called a refrain, and the other cluster is called a stanza.

Thus, for the two clusters finally available for stanza / refrain selection, the cluster which has the segment which occurs as the last segment of the segments of the two segment groups in the song progression is checked (40) to designate the same as chorus.

The last segment may actually be the last segment in the song or else a segment which occurs later in the song than all segments of the other segment class. If this segment is not the actual last segment in the song, this means that there is still an outro.

This decision is based on the realization that in most cases the refrain comes in a song behind the last stanza, ie directly as the last segment of the song

Song, if a piece z. B. hidden with the refrain or as a segment in front of an outro, which follows a recess and with which the piece is terminated.

If the last segment is from the first segment group, then all segments of this first (highest-order) segment class are referred to as a refrain, as represented by a block 41 in FIG. 4b. In addition, in this case, all segments of the other segment class which is to be selected are marked as "stanza", since typically one of the two candidate segment classes will have one class of the refrain and thus immediately the other class will have the strokes.

If, on the other hand, the examination in block 40 reveals that which segment class in the selection has the last segment in the music segment, that this is the second, ie rather low-order segment class, a search is made in block 42 as to whether the second segment class has the first segment in the piece of music. This study is based on the finding that the likelihood is very high that a song will begin with a stanza, not a chorus.

If the question is answered with "no" in block 42, that is to say if the second segment class does not have the first segment in the piece of music, then the second segment class is designated as a refrain, and the first segment category is called a stanza, as it is is indicated in a block 43. If, on the other hand, the query in block 42 is answered with "yes", then, contrary to the rule, the second segment group is designated as a stanza and the first segment group as a refrain, as indicated in a block 44 The denomination in block 44 occurs because the probability that the second segment class will utter the chorus is quite small.If the improbability that a piece of music is introduced with a chorus, there is some evidence of a clustering error out, eg that the last considered segment was erroneously assigned to the second segment class.

FIG. 4b shows how the stanza / refrain determination has been carried out on the basis of two available segment classes. After this stanza / refrain determination, the remaining segment classes can then be designated in a block 45, an outro possibly being the segment class which has the last segment of the piece, while an intro will be the segment class which has the first segment of a piece in itself.

In the following it will be shown, with reference to FIG. 4a, how the two segment classes are determined which output the candidates for the algorithm shown in FIG. 4b.

In general, an assignment of the labels "stroke" and "refrain" is carried out in labeling, whereby one segment group is marked as a stanza segment group, while the other segment group is marked as a refrain segment group. Basically, this concept is based on the assumption (Al) that the two clusters (segment groups) with the highest similarity values, that is, cluster 1 and cluster 2, correspond to the regular and stanza clusters. The last of these two clusters is the refrain cluster, assuming that a verse follows a chorus.

The experience from numerous tests has shown that cluster 1 in most cases corresponds to the refrain. For cluster 2, however, the assumption (Al) is often not fulfilled. This situation usually occurs when there is either a third, frequently repeating part in the play, eg. B. a bridge, with a high Ähn¬ probability of intro and outro, or for the not sel¬ th occurring case that a segment in the piece has a high similarity to the chorus, thus also a high However, the resemblance to the chorus is not enough to be part of Cluster 1.

Research has shown that this situation often occurs for variations of the refrain at the end of the piece. In order to mark chorus and stanza correctly (as labein) with the greatest possible certainty, the segment selection described in FIG. 4b is improved such that, as illustrated in FIG. 4a, the two candidates are dependent on the chorus chorus selection is determined by the segments present in the same.

First, in a step 46, the cluster or the segment group with the highest similarity value (value of the component of V which was once a maximum for the first-determined segment class, ie segment 7 in the example of FIG. 7, was ), that is, the segment group determined in the first pass of FIG. 1, is included in the stanza refrain selection as the first candidate.

It is now questionable which other segment group will be the second participant in the verse-chorus selection. The most probable candidate is the second highest segment class, ie the segment class which is found on the second pass through the concept described in FIG. However, this does not always have to be this way. Therefore, firstly for the second highest segment class (segment 5 in FIG. 1) ₁ , cluster 2 is checked whether this class has only a single segment or exactly two segments, one of the two segments being the first segment and the other segment being both are the last segment in the song (block 47).

If, on the other hand, the answer to the question is "no", then the second highest segment class, for example, has at least three segments, or two segments, one of which is within the piece and not at the "edge" of the piece second segment class initially in the selection and is henceforth referred to as "Second Cluster".

If the question in block 47 is answered with "yes", then the second highest class is eliminated (block 48a), then it is replaced by the segment class which occurs most frequently in the entire song (in other words, which contains the most segments) ) and not the highest segment class (cluster 1) .This segment class will henceforth be referred to as "second cluster".

Second clusters, as explained below, still have to measure themselves with a third segment class (48b), which is referred to as a "third cluster" in order to ultimately survive the selection process as a candidate.

The segment class "Third Cluster" corresponds to the cluster which occurs most frequently in the entire song, however, that the highest segment class (cluster 1) still corresponds to the segment class "second cluster", so to speak the next most frequently (often equally frequently) occurring clusters after cluster 1 and second clusters.

With regard to the so-called bridge problem, it is now checked for "third cluster" whether it belongs more in the stanza-refrain selection than "second cluster" or not. This happens because "second clusters" and "third clusters" often occur the same number of times, so one of them may represent a bridge or another recurring intermediate part. In order to ensure that the segment class is selected by the two which most closely corresponds to the stanza or chorus, ie not to a bridge or another intermediate piece, the investigations shown in blocks 49a, 49b, 49c are carried out.

The first examination in block 49a is to examine whether each segment of third cluster ne certain minimum length has, as a threshold z. B. 4% of the total song length is preferred. Other values between 2% and 10% can also lead to meaningful results.

In a block 49b, it is then examined whether ThirdCluster has a greater total portion of the song than SecondCluster. For this purpose, the total time of all segments in ThirdCluster is added up and compared with the correspondingly added total number of all segments in SecondCluster, in which case ThirdCluster has a larger overall proportion of the song than Se¬ condCluster, if the added segments in ThirdC¬ luster give a larger value than the added up Segments in SecondCluster.

In block 49c, it is finally checked whether the distance of the segments from third cluster to the segments from cluster 1, ie the most frequent cluster, is constant, ie. H. whether a regularity is evident in the sequence.

If all these three conditions are answered with "yes", then ThirdCluster enters the stanza-refrain selection, but if at least one of these conditions is not met, ThirdCluster does not enter the stanza-refrain selection the stanza-refrain selection, as represented by a block 50 in Fig. 4a, completes the "candidate search" for the stanza-refrain selection, and the algorithm shown in Fig. 4b is started in the end it is determined which segment class comprises the stanzas, and which segment class comprises the chorus.

It should be noted at this point that the three conditions in blocks 49a, 49b, 49c could alternatively also be weighted, so that z. For example, a no answer in block 49a is "overruled" if both the query in block 49b and the query in block 49c are answered "yes". Alternatively, it could also be a condition the three conditions are highlighted so that z. For example, it only examines whether there is regularity of the sequence between the third segment class and the first segment class, while the queries in blocks 49a and 49b are not performed or are only performed if the query in block 49c reads " No answer is given, but for example a relatively large total proportion in block 49b and relatively large minimum quantities are determined in block 49a.

Alternative combinations are also possible, and for a low-level examination only the query of one of the blocks 49a, 49b, 49c will be sufficient for certain implementations.

Hereinafter, exemplary implementations of the block 526 for performing a music summary are set forth. So there are different possibilities, which can be stored as music summary. Two of them are described below, namely the option entitled "Refrain" and the option entitled "Medley".

The refrain possibility is to select a version of the female as a summary. This will attempt to choose a run of the Refxain that lasts between 20 and 30 seconds if possible. If a segment with such a length is not contained in the refrain cluster, then a version is chosen which has the smallest possible deviation to a length of 25 seconds. If the selected chorus is longer than 30 seconds, it is blanked out for 30 seconds in this embodiment and is shorter than 20 seconds, so that it is extended to 30 seconds with the following segment.

Storing a medley for the second option is more like an actual summary of a piece of music. Here, a section of the strobe, a section of the chorus and a section of a third segment in its actual chronological order as a medley constructed. The third segment is selected from a cluster that has the largest total portion of the song and is not a verse or chorus.

The following priority is used to search for the most appropriate sequence of segments:

- "third segment" stanza refrain;

- stanza refrain - "third segment"; or

- stanza - "third segment" ~ chorus.

The selected segments are not installed in their full length in the medley. The length is preferably set to a fixed 10 seconds per segment, so that a total of 30 seconds is created again. However, alternative values are also readily feasible.

Preferably, after the feature extraction in block 502 or after block 508, a grouping of a plurality of feature vectors in block 510 is performed to save computation time by forming an average over the grouped feature vectors. The grouping may be the next. Processing step, the calculation of the similarity matrix, saving computing time. To calculate the similarity matrix, a distance is determined between all possible combinations of j & two feature vectors. This yields n x n calculations for n vectors over the entire piece. A grouping factor g indicates how many; consecutive feature vectors are grouped into a vector via the averaging. This can reduce the number of calculations.

The grouping is also a type of noise suppression _r in which small changes in the feature expression of successive vectors are compensated on the average. the. This property has a positive effect on finding large song structures.

The concept according to the invention makes it possible to navigate through the calculated segments by means of a special music player and selectively select individual segments, so that a consumer in a music shop can easily immediately return to the Re by pressing a certain key or activating a certain software command - Frain of a piece can jump to determine whether the chorus pleases him, and then perhaps listen to a stanza, so that the consumer can finally make a Kaufent¬ divorce. It is thus comfortably possible for a buyer to hear exactly what he is particularly interested in from a single piece, while he is, for example, interested in doing so. B. the solo or the bridge then actually save for the listening pleasure at home.

Alternatively, the concept according to the invention is also of great advantage for a music shop, since the customer can listen in and thus quickly and ultimately buy, so that the customers do not have to wait long to listen in, but also quickly get their turn , This is due to the fact that a user does not have to constantly go back and forth, but receives in a targeted and rapid manner all the information of the piece that he would like to have.

Furthermore, reference is made to a significant advantage of the inventive concept, namely that no information of the piece is lost, in particular due to the post-correction of the segmentation. Thus, although all Segmen¬ te, which are preferably shorter than 6 seconds, merged with the predecessor or successor segment. However, no segments as short as they are will be eliminated. This has the advantage that the user can in principle hear everything in the play, so that a short but a user but very good liking striking piece that would have been omitted in a segmentation post-correction, the actually had completely eliminated a section of the piece, yet is available to the user, so perhaps he might make a deliberate purchase decision just because of the short distinctive piece.

However, the present invention is also applicable in other applica tion scenarios, for example in advertising monitoring, ie where an advertiser wants to check whether the audio piece for which he has bought advertising time, has actually been played over the entire length. An audio piece may include, for example, music segments, speaker segments, and noise segments. The segmentation algorithm, that is to say the segmentation and subsequent classification into segment groups, then makes it possible to carry out a quick and considerably less complicated check than a complete sample-wise comparison. The efficient checking would simply consist in a segment class statistic, ie a comparison of how many segment classes were found and how many segments are in the individual segment classes, with a specification based on the ideal advertising piece. It is thus easily possible for an advertiser to recognize whether a broadcaster or television station has actually broadcast all the main parts (sections) of the commercial signal or not.

The present invention is further advantageous in that it can be used for searching in large music databases, for example, only to listen through the choruses of many pieces of music in order to then perform a music program selection. In this case, only individual segments from the segment class marked "chorus" would be selected from many different pieces and provided by a program provider, Alternatively, there could also be an interest, for example from an artist, for all the guitar solos According to the invention, these can likewise be provided without difficulty by always having one or more segments (if present) in the range marked "Solo". segment class from a large number of pieces of music, for. B. assembled and provided as a file.

Yet other possible applications include mixing stanzas and choruses from various audio tracks, which will be of particular interest to DJs and opens up completely new possibilities for creative music synthesis, which can be carried out precisely and, above all, automatically automatically. Thus, the inventive concept can be easily automated, since it requires at no point a user intervention. This means that users of the inventive concept by no means require special training, except for. For example, a common skill in dealing with normal software user interfaces.

Depending on the practical circumstances, the inventive concept can be implemented in hardware or in software. The implementation can be carried out on a digital storage medium, in particular a floppy disk or CD with electronically readable control signals, which can cooperate with a programmable computer system in such a way that the corresponding method is executed. In general, the invention thus also consists in a computer program product with a program code stored on a machine-readable carrier for carrying out the method according to the invention when the computer program product runs on a computer. In other words, the invention thus represents a computer program with a program code for carrying out the method when the computer program runs on a computer.

Claims

claims

1. An apparatus for grouping time segments of an audio piece, which is divided into main parts which repeatedly occur in the audio piece, into different segment classes, one segment class being assigned to a main part, having the following features:

means (10) for providing a similarity representation for the segments, the similarity representation having for each segment an associated plurality of similarity values, the similarity values indicating how similar the segment is to each other segment of the audio piece;

means (12) for calculating a similarity threshold for a segment using the plurality of similarity values associated with the segment; and

means (14) for assigning a segment to a segment class when the similarity value of the segment satisfies a predetermined condition regarding the similarity threshold.

2. Apparatus according to claim 1, further comprising:

a segment selection device (16) for determining an extreme segment whose associated plurality of similarity values taken together has an extremum,

wherein the means (12) for calculating is adapted to calculate the similarity threshold value for the extreme segment, and wherein the means (14) for assigning is adapted to mark the segment class with an indication of the extreme segment.

3. Apparatus according to claim 1 or 2, wherein the Ein¬ direction (14) is adapted to assign a segment that does not meet the predetermined condition with respect to the similarity threshold, the Segment¬ class not assign but for an assignment to another segment class, and

wherein the means (14) is adapted to assign, for an associated segment no longer taking into account the similarity value of the associated segment in an assignment to another segment class.

4. Device according to one of the preceding claims, wherein the means (12) for calculating the similarity threshold value is formed in a later passage in order, after an earlier assignment of a segment class, similarity values for previously assigned segments in the majority to ignore similarity values, and

wherein the means (14) is adapted to allocate to, in a later pass, perform an assignment to a segment class other than the segment class in an earlier pass.

5. Device according to one of the preceding claims, further comprising the following feature:

a segment allocation conflicting device (18) which is designed to have a first similarity value of the conflict segment in the case in which a conflict segment should be assigned to two different segment classes by the device (14) for assigning determine a segment of a first segment class, and to determine a second similarity value of the conflict segment with a segment of a second segment class, and

wherein the means (14) is adapted to assign, in the case in which the second similarity value points to a stronger similarity of the conflict segment with the segment of the second segment class, the conflict segment from the first segment class remove and assign the second segment class sen.

6. Device according to claim 5, wherein the segment allocation conflicting device (18) is designed to be in

If the segment is removed from the first segment class, the segment has a tendency to assign the segment to the first segment class or, in the case of a non-removal of the segment, to assign the segment a bias to the second segment class.

A device according to any one of the preceding claims, further comprising

a Segmentierungskorrektureinrichtung (20), which is aus¬ formed to correct a segmentation of the audio piece, the Segmentierungskorrektureinrich¬ device (20) is adapted to merge segments depending on segment class information for the segments with a preceding segment or a subsequent segment.

8. The apparatus of claim 7, wherein the Segmentie¬ rungskorrektureinrichtung (20) is designed to determine for a segment that is shorter than a predetermined Mindest¬ length, determine whether a trend of the segment coincides with a segment class, the unmit¬ temporally preceding segment, and in this case, to merge the segment with the immediately preceding segment, or is designed to determine, for a segment that is shorter than a predetermined minimum length, whether a trend of the segment indicates a segment class that is temporally belongs immediately following segment, and in this case to merge the segment with the immediate subsequent segment.

9. Device according to one of the preceding claims, which has a Segmentierungskorrektureinrichtung (20) auf¬, which is designed to merge temporally successive segments that belong to the same segment class.

10. Device according to one of claims 6 to 9, wherein the Segmentierungskorrektureinrichtung (20) is ausgebil¬ det to select for correcting the segments only segments having a temporal segment length that is shorter than a predetermined minimum length -Lst.

11. The device according to claim 10, wherein the segmentation correction means (20) is designed to include a selected segment from a second segment class whose temporal precursor segment and its temporal successor segment of a first segment class belong to the predecessor segment and the successor segment to merge.

12. Apparatus according to claim 10 or 11, wherein the segmentation correction means (20) is designed to ver a segment which is in a segment class, which comprises only a single segment, with the vorhergehen¬ the segment or the subsequent segment ¬ melt.

13. The apparatus of claim 10, 11 or 12, wherein the Segmentierungskorrektureinrichtung (20) is adapted to merge a plurality of selected segments that are in the same segment class, each with a time-leading segment or each a temporally aufeinander¬ ing segment if all selected segments of the segment class include precursor segments from one and the same segment class or successor segments from one and the same segment class.

14. Device according to one of claims 7 to 13, wherein the Segmentierungskorrektureinrichtung is adapted to determine for a segment having a smaller length of time than a predetermined minimum length, a first novelty value at a beginning of the segment, and a second to determine novelty value at an end of the segment, and to merge the segment with a temporally following segment, when the first novelty value is greater than the second novelty ^'worth, or to the segment with a temporally vor¬ reciprocating segment to merge when the first novelty value is less than the second novelty value.

15. Device according to one of claims 7 to 14, wherein the Segmentierungskorrektureinrrichtung (20) is ausgebil¬ det to durchzu¬ perform different correction measures depending on different predetermined segment lengths.

16. Device according to one of the preceding claims, further comprising a segment class designation means, which is designed to perform depending on a time position of segments in different segment classes, a designation of segment classes to different main parts.

17. The device according to claim 16, wherein the segment class designation means (22) is designed to before a segment class designation into a main part "stanza" and in a main part "refrain" two segment class candidates for the consideration of the segments in the segment classes to select.

18. Device according to claim 16 or 17, in which the segment class designation device (22) is designed to designate a candidate segment class as a refrain class if the candidate segment class comprises the segment that is in the audio piece occurs temporally after all other segments of the other candidate segment class.

The apparatus of any one of claims 16 to 18, wherein the segment class designator (22) is configured to designate a candidate segment class as a stanza class if the candidate segment class does not include the segment included in the audio piece occurs temporally after all other segments of the other candidate segment class.

20. A method for grouping temporal segments of an audio piece, which is subdivided into main parts which repeatedly occur in the audio piece, into different segment classes, one segment class being assigned to a main part, with the following steps:

Providing (10) a similarity representation for the segments, the similarity representation having for each segment an associated plurality of similarity values, the similarity values indicating how similar the segment is to each other segment of the audio piece;

Calculating (12) a similarity threshold for a segment using the plurality of similarity values associated with the segment; and Assigning (14) a segment to a segment class if the similarity value of the segment satisfies a predetermined condition with respect to the similarity threshold.

21. Computer program with a program code for carrying out the method according to claim 20, when the computer program runs on a computer.