Classifications

• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
  • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
  • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
  • G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
  • G10H2210/031—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
  • G10H2210/061—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal for extraction of musical phrases, isolation of musically relevant segments, e.g. musical thumbnail generation, or for temporal structure analysis of a musical piece, e.g. determination of the movement sequence of a musical work

Definitions

• the present invention relates to the audio segmentation and in particular to the analysis of pieces of music on the individual Kleintei ⁇ contained in the pieces of music, which may occur repeatedly in the piece of music.
• Music from the rock and pop area usually consists of more or less distinct segments, such as intro, verse, chorus, bridge, outro, etc.
• the beginning and end times of such segments to detect and the segments according to their affiliation to the most important Klas ⁇ Grouping the stanza (verse and chorus) is the goal of audio segmentation.
• Correct segmentation and also identification of the calculated segments can be usefully used in various areas. For example, pieces of music from online providers such as Amazon, Mu- sicline, etc. can be intelligently "played”.
• Another application example for the technique of audio segmentation is the integration of the segmentation / Grouping / marking algorithm into a music player.
• the information about segment beginnings and segment ends makes it possible to navigate through a piece of music. Due to the class affiliation of the segments, ie whether a segment is a verse, a chorus, etc., z. B. also jump directly to the next chorus or the next stanza.
• Such an application is of interest to large music markets, offering their customers the opportunity to listen to complete albums. As a result, the customer spares himself the annoying, searching prelude to characteristic passages in the song, which might perhaps lead him to actually buy a piece of music in the end.
• a WAV file 500 is provided.
• a feature extraction then takes place, wherein as a feature the spectral coefficients per se or alternatively the mel frequency cepstral coefficients (MFCCs) are extracted.
• MFCCs mel frequency cepstral coefficients
• STFT short-time Fourier transform
• the MFCC features are then extracted in the spectral range. It should be noted that the parameterization is not optimized for compression, transmission or reconstruction, but for a O
• Audio analysis The requirement is that similar audio pieces produce similar features.
• the extracted features are then stored in a memory 504.
• the feature extraction algorithm now has a segmentation algorithm that ends in a similarity matrix, as shown in a block 506.
• the feature matrix is read in (508), to then group feature vectors (510), to then build a similarity matrix based on the grouped feature vectors, which consists of a distance measurement between each of all features.
• all pairs of audio window pairs are compared using a quantitative similarity measure, distance.
• the music piece is represented as a stream or stream 800 of audio samples.
• the piece of audio is windowed, as has been stated, with a first window having i and a second window being j.
• the audio piece has z. B. K window.
• the similarity matrix has K rows and K columns.
• a similarity measure to each other is calculated, and the calculated similarity measure or distance measure D (i, j) is input to the row or column designated by i and j in the similarity matrix.
• One column therefore shows the similarity of the window designated by j to all other audio windows in the piece of music.
• the matrix is redundant in that it is symmetric to the diagonal, and that on the diagonal the similarity of a window is to itself, which is the trivial case of 100% similarity.
• FIG. 6 An example of a similarity matrix of a piece can be seen in FIG.
• the completely symmetrical structure of the matrix with respect to the main diagonal is recognizable, the main diagonal being visible as a light stripe.
• the main diagonal is not seen as a lighter solid line, but from Fig. 6 is only approximately recognizable.
• a kernel correlation 512 is performed on a kernel matrix 514 to obtain a novelty measure, also known as a novelty score, that could be averaged and smoothed into
• a novelty measure also known as a novelty score
• segment boundaries are read out using the smoothed novelty value profile, for which purpose the local maxima in the smoothed novelty curve are determined and, if appropriate, must be shifted by a constant number of samples caused by the smoothing in order actually to produce the correct segment - To obtain limits of the audio piece as absolute or relative Zeitan ⁇ gift.
• segment similarity representation or segment similarity matrix is created.
• An example of a segment similarity matrix is shown in FIG.
• the similarity matrix in FIG. 7 is basically similar to the feature similarity matrix of FIG. 6, but now, as in FIG. 6, features from windows are no longer used, but features from a whole segment.
• the segment similarity matrix has a similar proposition to the feature similarity matrix, but with a much coarser resolution, which of course is desired when considering that window lengths are in the range of 0.05 seconds, while reasonably long segments are in the range of maybe 10 seconds of a piece lying.
• a clustering is performed, ie an arrangement of the segments into segment classes (an arrangement of similar segments in the same segment class), in order then to mark the found segment classes in a block 524, which is also referred to as "labeling
• labeling it is determined which segment class contains segments which are stanzas, which are reflections, which are intros, outros, bridges, etc.
• a music score is created, which is e.g. B. can be provided to a user, without redundancy of a piece only z. B. a verse, a chorus and the intro to hear.
• the corresponding feature matrix is read out and loaded into a main memory for further processing.
• the feature matrix has the dimension number of analysis windows times the number of feature coefficients.
• the similarity matrix brings the feature course of a piece into a two-dimensional representation. For each pairwise combination of feature vectors, the distance measure is calculated, which is kept fixed in the similarity matrix. There are various possibilities for calculating the distance measure between two vectors, namely, for example, the Euclidean distance measurement and the cosine distance measurement.
• a result D (i, j) between the two feature vectors is stored in the i, jth element of the window similarity matrix (block 506).
• the main diagonal of the similarity matrix represents the course over the entire piece. Accordingly, the elements of the main diagonal result from the respective comparison of a window with itself and always have the value of the greatest similarity. For the cosine distance measurement this is the value 1, for the simple scalar difference and the Euclidean distance this value is 0.
• each element i, j is assigned a gray value.
• the gray values are graduated in proportion to the similarity values, so that the maximum similarity (the main diagonal) corresponds to the maximum similarity.
• the structure of a song can already be visually recognized on the basis of the matrix. Areas of similar feature expression correspond to quadrants of similar brightness along the main diagonal. Finding the boundaries between the areas is the task of the actual segmentation.
• the structure of the similarity matrix is important to the novelty measure calculated in kernel correlation 512. The measure of novelty arises from the correlation of a special kernel along the main diagonal of the similarity matrix.
• An exemplary kernel K is shown in FIG.
• the novelty measure is obtained, which is shown by way of example in FIG. 9 in a smoothed form .
• the kernel K in FIG. 5 is not used, but rather an enlarged kernel, which is additionally superimposed with a Gaussian distribution, so that the edges of the matrix tend towards 0.
• the selection of the striking maxima in the novelty course is important for the segmentation.
• the selection of all maxima of the unsmoothed novelty course would lead to a strong over-segmentation of the audio signal.
• the novelty measure should be smoothed with different filters, such as IIR filters or FIR filters.
• segment boundaries of a piece of music are extracted, then similar segments must be identified as such and grouped into classes.
• Foote and Cooper describe the computation of a segment-based similarity matrix using a Cullback-Leibler distance.
• individual segment feature matrices are extracted from the entire feature matrix on the basis of the segment boundaries obtained from the course of novelty, ie each of these matrices is a submatrix of the entire feature matrix.
• an automatic summary of a piece is then carried out on the basis of the segments and clusters of a piece of music. For this purpose, first the two clusters with the largest singular values are selected. Then, the segment with the maximum value of the corresponding cluster indicator is added to this summary. This means that the summary includes a stanza and a chorus. Alternatively, all repeated segments can be removed to ensure that all information of the piece is provided, but always exactly once.
• a disadvantage of the known method is the fact that the singular value decomposition (SVD) for segment class formation, that is to say the assignment of segments to clusters, is very computationally intensive and is problematic in the evaluation of the results. Thus, if the singular values are nearly equal, then a possibly wrong decision is made that the two similar singular values actually represent the same segment class and not two different segment classes.
• SSVD singular value decomposition
• a further disadvantage of the known concept is the fact that in the segmentation the segmentation calculated by the singular value decomposition is established. In other words, this means that both the clustering and the final labeling are based on the segmentation determined by singular value decomposition.
• clustering and labeling, and thus also the music summary which is the actual product of the entire process for the listener, can never be better than the underlying segmentation. If an over-segmentation takes place, as is frequently the case for kernel-correlation-based concepts, one will end up with far too many segment classes that have to be reworked in order to avoid disturbing segment classes that are actually not appropriate for a major part ⁇ if necessary completely remove.
• Dd_ese "post-repair" is unfavorable in that it eliminates audio information, since a listener, when navigating through the audio track due to the already-mentioned segment classes, can not hear the entire audio information, since insignificant segments do not actually belong to any one Main part correspond, have been completely eliminated in this Verfallren.
• the object of the present invention is to achieve a more precise segmentation concept, which should also be compatible with an already existing first segmentation of the audio piece.
• This object is achieved by a device for changing a segmentation of an audio piece according to claim 1, a method for changing a segmentation of an audio piece - IT -
• the present invention is based on the finding that the over-segmentation is effectively counteracted if, after an original segmentation and subsequent segmentation class assignment, the originally already completed original segmentation is corrected.
• the device according to the invention comprises a segmentation correction device for correcting the segmentation, which is designed to merge a segment having a length that is shorter than a predetermined minimum length with a temporal preceding segment or a successor segment, by a modified segmentation of the audio track.
• this post-correction takes place after the first segmentation and the allocation to the segment classes which adjoins the first segmentation, thus also after the clustering.
• the segment merge is preferably carried out only ciann, so to speak, as a last resort, if a corresponding speaking short segment could not be merged by prior verifications, in which the segment class affiliation of the concerned precursor / successor segments were taken into account.
• adaptive segment allocation is performed on the basis of the primary segmentation, however, when segmentation allocation conflicts occur, segments that are actually associated with a first segment class tend to have a different class of segment that caused the conflict become. If it then turns out that a segment with such a tendency is at the same time a short segment, and it further states that the tendency is due to a segment class h_ which also includes the temporally preceding or the temporally subsequent segment, then receive a segment merger that meets the requirements of the original representation of similarity on the basis of this tendency or trend.
• the inventive concept is particularly advantageous in that no portion of the audio piece is completely eliminated.
• the user who then navigates through the audio piece when all processing has finished, will find segments which constitute the changed segmentation whose total length is still equal to the original length of the audio piece.
• the minimum length of a segment can be adjusted variably, solely on the basis of a temporal threshold value specification, which opens possibilities, particularly permissible minimum segment lengths, to the object, particularly in connection with a music identity identification.
• a temporal threshold value specification which opens possibilities, particularly permissible minimum segment lengths, to the object, particularly in connection with a music identity identification.
• adapted music genre especially since different genre Musik ⁇ different lengths of segments bring with it Rön ⁇ NEN.
• the concept according to the invention also makes it possible to reduce the number of segment classes by assigning short segments only on the basis of a minimum length threshold value, until an expected number is fulfilled, without the segment display of the audio piece having holes includes.
• the assignment of a segment to a segment class is based on an adaptive similarity mean for a segment, such that the mean of similarity takes into account which overall similarity score a segment as a whole takes into account Piece has.
• an adaptive similarity mean for a segment, such that the mean of similarity takes into account which overall similarity score a segment as a whole takes into account Piece has.
• the similarity mean will be lower than for a segment that is a stanza or chorus.
• the concept according to the invention is particularly suitable for music pieces which consist not only of stanzas and choruses, that is to say have the segments which belong to segment class having equal similarity values, but also for pieces which, in addition to stanza and chorus, also have others Parts have, namely an introduction (Intro), an interlude (Bridge) or a conclusion (Outro).
• the calculation of the adaptive similarity mean and the assignment of a segment are performed iteratively, similarity values of assigned segments being ignored in the next iteration run. This results in a new maximum similarity absolute value for the next iteration run, ie the sum of the similarity values in a column of the similarity matrix, since the similarity absolute values corresponding to the previously assigned segments have been set to zero.
• a segmentation post-correction is carried out in such a way that after the segmentation z. For example, based on the novelty value (the local maxima of the novelty value) and after a subsequent assignment to segment classes, relatively short segments are examined in order to see whether they can be assigned to the predecessor segment or the successor segment Segments below a minimum segment length indicate a high degree of probability of an over-segmentation.
• a labeling is carried out after the final segmentation and assignment into the segment classes, specifically using a special selection algorithm in order to obtain the most correct possible labeling of the segment classes as a stanza or chorus.
• FIG. 1 shows a block diagram of the device according to the invention for grouping according to a preferred embodiment of the present invention
• FIG. 2 shows a flow chart for illustrating a preferred embodiment of the invention for iteratively assigning
• FIG. 3 shows a block diagram of the mode of operation of the segmentation correcting device
• Figures 4a and 4b show a preferred embodiment of the segment class designator
• FIG. 5 shows an overall block diagram of an audio analysis tool
• FIG. 6 is an illustration of an exemplary feature similarity matrix
• FIG. 8 shows a schematic illustration for illustrating the elements in a similarity matrix S
• FIG. and 9 is a schematic representation of a smoothed novelty value.
• FIG. 1 shows a device for grouping temporal segments of a piece of music, which is subdivided into main parts which repeatedly appear in the piece of music, into different segment classes, one segment class being assigned to one main part.
• the present invention thus relates particularly to pieces of music which are subject to a certain structure in which similar sections appear several times and alternate with other sections. Most rock and pop songs have a clear structure in terms of their main parts.
• a large shaped part of a piece is understood to mean a section which has a relatively uniform quality with regard to various features, eg, melody, rhythm, texture, etc. Definition applies generally in music theory.
• ⁇ ntros often consist of the same chord progression as dd_e stanza, but with different instrumentation, eg. B. without drums, without bass or distortion of the guitar in rock songs etc.
• the device according to the invention initially comprises a device 10 for providing a similarity representation for the segments, the similarity representation having an associated plurality of similarity values for each segment, the similarity values indicating how the segment is similar to any other segment is.
• the similarity representation is preferably the segment similarity matrix shown in FIG. It has a separate column for each segment (segments 1-10 in Fig. 7) having the index "j.” Furthermore, the similarity representation has a separate row for each segment, one row being designated by a row index i This is referred to below by means of the exemplary segment 5.
• the element (5, 5) in the main diagonal of the matrix of Fig. 7 is the similarity value of the segment 5 with itself, ie the maximum similarity value still mid-like to the segment No.
• segment # 5 has a similarity that is not visible in Figure 7;
• a plurality of similarity values assigned to the segment is, for example, a column or a row of the segment similarity matrix in FIG. 7, which indicates this column or row on the basis of its column / row index which segment it refers to, namely, for example, to the fifth segment, and where this row / column comprises the similarities of the fifth segment to each other segment in the piece.
• the plurality of similarity values is, for example, a row of the similarity matrix or, alternatively, a column of the similarity matrix of FIG. 7.
• the device for grouping temporal segments of the piece of music further comprises a means 12 for calculating a similarity mean value for a segment, using the segments and the similarity values of the plurality of similarity values assigned to the segment.
• the device 12 is designed to z. For example, to calculate a similarity mean for column 5 in FIG. If the arithmetic mean value is used in a preferred exemplary embodiment, the device 12 will add the similarity values in the column and divide them by the total number of segments. In order to eliminate the self-similarity, the similarity of the segment to itself could also be deducted from the result of addition, whereby of course then a division should no longer be carried out by all elements but by all elements less 1.
• the means 12 for calculating could alternatively calculate the geometric mean value, that is to say, squareness each etch value of a column individually in order to sum the quadrated results, in order then to calculate a root from the summation result, which is given by the number Elements in the column ' (or the number of Ele ⁇ elements in the column less 1) to share. Any other average values, such as the median value, etc., may be used as long as the mean value for each column of the similarity matrix is adaptively calculated, that is, a value calculated using the similarity values of the plurality of similarity values associated with the segment.
• the adaptively calculated similarity threshold is then provided to a segment 14 for assigning a segment to a segment class.
• the means 14 for assigning is designed to assign a segment to a segment class if the similarity value of the segment fulfills a predetermined condition with respect to the mean of similarity. For example, if the similarity mean is such that a larger value indicates greater similarity and a smaller value indicates lesser similarity, then the predetermined relationship will be that the similarity value of a segment must be equal to or above the similarity mean, thus the segment is assigned to a segment class.
• a segment selection device 16 In a preferred exemplary embodiment of the present invention, further devices exist to realize special embodiments, which will be discussed later. These devices are a segment selection device 16, a segment assignment conflicting device 18, a segmentation correction device 20 and a segment class designation device 22.
• the segment selection device 16 in FIG. 1 is designed to first calculate an overall similarity value V (j) for each column in the matrix of FIG. 7, which is determined as follows:
• P is the number of segments.
• SA is the value of the self-similarity of a segment to itself.
• the value z. B. zero or one.
• the segment selector 16 will first calculate the value V (j) for each segment to then find the vector element i of the maximum value vector V. In other words, this means that the column in FIG. is selected, which achieves the greatest value or score in the addition of the individual similarity values in the column.
• This segment could, for example, be the segment No. 5 or the column 5 of the matrix in FIG. 7, since this segment has at least a certain similarity with three other segments.
• Another candidate in the example of FIG. 7 could also be the segment with the number 7, since this segment also has a certain similarity to three other segments, which is even greater than the similarity of the segment 5 to the segments 2 and 3 (higher gray shade in Fig. 7).
• segment selector 16 selects the segment # 7 because it has the highest similarity score due to the matrix elements (1,7), (4,7) and (10,7) ,
• segment similarity matrix for the seventh row or column it is checked which segment similarities are above the calculated threshold value, ie. H. with which segments the ith segment has an above-average similarity. All these segments are now also assigned to a first segment class like the seventh segment.
• segment no. 10 becomes the ISIr segment due to the below-average similarity. 7 not classified in the first segment class.
• a new maximum is now selected from the remaining elements of V, that is to say V (2), V (3), V (5), V (6), V (8), V (9) and V (IO) searched.
• the segment no. 5, ie V (5), is expected to yield the largest similarity score.
• the second segment class then obtains segments 5 and 6. Due to the fact that the similarities to segments 2 and 3 are below average, the segments 2 and 3 are not placed in the second order clusters.
• the elements V (6) and V (5) are set to 0 by the vector " V on the basis of the following assignment, while the components V (2), V (3), V (8), V (9) and V (IO) of the third-order cluster selection vector remain.
• a simple kind of resolution could be to simply not assign the segment 7 into the third segment class and e.g. For example, instead of assigning the segment 4, if for the segment 4 would not also conflict exist.
• the similarity between 7 and 10 is taken into account in the following algorithm.
• the invention is designed not to discount the similarity between i and k. Therefore, the similarity values S s (i, k) of segment i and k are compared with the similarity value S s (i * , k), where i * is the first segment assigned to the cluster C * .
• the cluster or the segment class C * is the cluster to which segment k is already assigned on the basis of a previous examination.
• the similarity value S s (i * , k) is decisive for the fact that the segment k belongs to the cluster C * . If S s (i * , k) is greater than S s (i, k), the segment k is in cluster C * .
• S s (i * , k) is smaller than S s (i, k)
• the segment k is taken out of the cluster C * and assigned to the cluster C.
• a tendency towards the cluster i is noted for the cluster C * .
• this tendency is also noted " when segment k changes cluster membership.
• a tendency of this segment to the cluster in which it was originally recorded is noted.
• the similarity value check is based on the fact that the segment 7 is the "original segment” in the first segment class, in favor of the first segment class e. but it will remain in the first segment class, but this fact is taken into account by the fact that segment no. 10 Ln of the third segment class is attested a trend towards the first segment class.
• the correction serves to completely eliminate too short segments, ie to fuse with adjacent segments, and to segments which are short but are not too short, that is to say they have a short length, but longer than that Minimum length is still to undergo a special investigation, whether they may not still be merged with a predecessor segment or a successor segment.
• successive segments which belong to the same segment class are always merged. If the scenario shown in FIG. B. that the segments 2 and 3 come in the same segment class, they are automatically ver ⁇ melted together, while the segments in the first segment class se, ie the segments 7, 4, 1 are spaced apart and therefore (at least initially) can not be merged. This is indicated in FIG. 3 by a block 30. Now, in a block 31, it is examined whether segments have a segment length that is smaller than a minimum length. Thus, there are preferably different minimum lengths.
• Relatively short segments which are shorter than 11 seconds are only examined at all, while later even shorter segments (a second threshold): smaller than the first one) shorter than 9 seconds, and later remaining segments shorter than 6 seconds (a third threshold shorter than the second threshold) are again alternately treated ,
• the segment length check in block 31 is initially directed to finding the segments shorter than 11 seconds. For the segments that are longer than 11 seconds, no post-processing is performed, as can be seen by a "No" at block 31. For segments which are shorter than 11 seconds, a trend check (block 32) is first carried out Thus, it is first examined whether a segment has an associated trend or an associated tendency due to the functionality of the segment assignment conflicting device 18 of Figure 1. In the example of Figure 7, this would be the segment 10 that is a trend If the tenth segment is shorter than 11 seconds, then in the example shown in Fig.
• segment no. 10 is the only segment in the third segment class. If it were shorter than 9 seconds, it is automatically assigned to the segment class to which segment No. 9 belongs. This automatically leads to a fusion of the segment 10 with the segment 9. If the segment 10 is longer than 9 seconds, then this merger is not taken vor ⁇ .
• a block 33c an examination is then made for segments shorter than 9 seconds which are not the only segment in a corresponding cluster X than in a corresponding segment group. They are subjected to a more detailed check in which a regularity in the cluster sequence is to be ascertained. Initially, all segments from the segment group X that are shorter than the minimum length are searched for. Subsequently, it is checked for each of these segments whether the predecessor and successor segments each belong to a uniform cluster. If all predecessor segments are from a uniform cluster, all segments that are too short from cluster X are assigned to the predecessor cluster. If, on the other hand, all successor segments are from a uniform cluster, the segments too short from cluster X are each assigned to the successor cluster.
• a novelty value check is performed by resorting to the novelty value curve shown in FIG. 9.
• the novelty curve which has arisen from the kernel correlation, is read out at the locations of the affected segment boundaries, and the maximum of these values is determined. If the maximum occurs at the beginning of a segment, the segments that are too short are assigned to the cluster of the successor segment. If the maximum occurs at a segment end, the segments that are too short are assigned to the cluster of the predecessor segment. If the segment labeled 90 in Fig.
• This procedure according to the invention has the advantage that no elimination of parts of the piece has been carried out, that is, that no simple elimination of the segments which are too short has been carried out by setting them to zero, but that the entire complete piece of music is still due to the
• the totality of the segments is represented by the segmentation therefore no loss of information auf ⁇ occurred, which would be, however, if one z.
• simply eliminating all too short segments "without regard to losses" would be easy.
• segment class designator 22 of FIG. 1 a preferred implementation of the segment class designator 22 of FIG. 1 is illustrated.
• two clusters are assigned the labels "stanza” and "refrain” during labeling.
• a largest singular value of a singular value decomposition and the associated cluster are used as a refrain and the cluster for the second largest singular value as a stanza.
• each song begins with a stanza, ie that the cluster with the first segment is the stanza cluster and the other cluster is the refrain cluster.
• the cluster: in the candidate selection having the last segment is called a refrain, and the other cluster is called a str-ophe.
• This decision is based on the insight that the Re ⁇ frain in the vast majority of cases in a songissuerm behind the last stanza, so directly as the last segment of the song, if a piece z. B. is eroded with the refrain, or as a segment before an outro, which follows a Re ⁇ frain and with which the piece is terminated.
• a refrain As represented by a block 41 in FIG. 4b.
• all the segments of the other segment class that is to be selected are identified as "stxophe", since typically one class of the refrain and thus the other class will have the strokes of the two candidate segment classes.
• the examination in block 40 reveals that which segment class in the selection has the last segment in the music piece history, that this is the second, ie rather non-valued segment class, then in a block 42 it is examined whether the second segment class has the first segment in the piece of music. This investigation is based on the commentary that the probability is very high that a song starts with a stream, not a chorus.
• the second segment class is designated as a refrain, and the first segment class is characterized as a stanza, as it is is indicated in a block 43.
• the second segment group is called a stanza and the first segment group as a refrain, as indicated in a block 44.
• the designation in block 44 is because the probability The fact that the second segment class corresponds to the chorus is already quite small: If the untruthful coincidence that a piece of music is introduced with a refrain adds up to a mistake in the clustering, for example, that the segment last considered is more incorrect Has been assigned to the second segment class.
• FIG. 4b shows how the stanza / refrain determination has been carried out on the basis of two classes of segments available. After this stanza / refrain determination, the remaining segment classes can then be designated in a block 45, where an outro will possibly be the segment class having the last segment of the piece, while an intro will be the segment class which has the first segment of a piece in itself.
• an assignment of the labels "stroke” and "refrain” is carried out in the labeling, whereby one segment group is marked as a stanza segment group, while the other segment group is marked as a refrain segment group.
• this concept is based on the assumption (Al) that the two clusters (Segmentolir ⁇ ) with the highest similarity values, ie cluster 1 and cluster 2, the Re ⁇ frain and stanza clusters correspond. The last of these two clusters is the refrain cluster, assuming that a verse follows a chorus. The experience from numerous tests has shown that cluster 1 in most cases corresponds to the refrain. For cluster 2, however, the assumption (Al) is often not fulfilled.
• This situation usually occurs when there is either a third, frequently repeating part in the play, eg. B. a bridge, with a high ⁇ hn ⁇ probability of intro and outro ⁇ or for the not sel ⁇ th occurring case that a segment in the piece has a high similarity to the chorus, thus also has a high overall similarity, the similarity to the refrain but gera ⁇ de is not big enough to belong to the cluster 1 yet.
• a third, frequently repeating part in the play eg. B. a bridge, with a high ⁇ hn ⁇ probability of intro and outro ⁇ or for the not sel ⁇ th occurring case that a segment in the piece has a high similarity to the chorus, thus also has a high overall similarity, the similarity to the refrain but gera ⁇ de is not big enough to belong to the cluster 1 yet.
• the cluster or the segment group with the highest similarity value (value of the component of V, which was once a maximum for the first-determined segment class, ie segment 7 in the example of FIG ), that is, the segment group determined in the first run of Fig. 1, is included in the stanza-refrain selection as the first candidate.
• cluster 2 checks whether this class has only: a single segment or exactly two segments, one of them the two segments is the first segment and the other segment of the two is the last segment in the song (block 47).
• the second highest segment class for example, has at least three segments, or two segments, one of which is within the piece and not at the "edge" of the piece, then the second segment class remains initially in the selection and is henceforth referred to as "Second Cluster".
• Second clusters still have to measure themselves with a third segment class (48b), which is referred to as a "third cluster" in order to ultimately survive the selection process as a candidate.
• the segment class "Third Cluster” corresponds to the cluster which occurs most frequently in the entire song, but the highest segment class (cluster 1) still corresponds to the segment class "second cluster", so to speak the next most frequently (often equally frequently) occurring clusters after cluster 1 and "second clusters".
• the first examination in block 49a is that it is examined whether each segment of third cluster has a certain minimum length, wherein as threshold value z. B. 4% of the 'entire song length is preferred. Other values between 2% and 10% can also lead to meaningful results.
• a block 49b it is then examined whether ThirdCluster has a greater total portion of the song than SecondCluster. For this purpose, the total time of all segments in ThirdCluster is added up and compared with the correspondingly added total number of all segments in SecondCluster, in which case ThirdCluster has a greater overall proportion of the song than SecondCluster, if the added segments in ThirdChuster give a greater value than the added ones Segments in SecondCluster.
• ThirdCluster enters the stanza-refrain selection, but if at least one of these conditions is not met, ThirdCluster does not enter the stanza-refrain selection the stanza-refrain selection, as represented by a block 50 in Fig. 4a, completes the "candidate search" for the stanza-refrain selection, and the algorithm shown in Fig. 4b is started which is the final one Segment class includes the stanzas, and which Segmentklas ⁇ se includes the chorus.
• the three conditions in the blocks 49a, 49b, 49c could alternatively also be weighted, so that z.
• a no answer in block 49a is "overruled” if both the query in block 49b and the query in block 49c are answered "yes".
• a condition of the three conditions could be emphasized, so that z.
• it only examines whether there is regularity of the sequence between the third segment class and the first segment class, while the queries in blocks 49a and 49b are not performed or are only performed if the query in block 49c reads " No answer is given, but for example a relatively large total proportion in block 49b and relatively large minimum quantities are determined in block 49a.
• the refrain possibility consists of selecting a version of the reiner as a summary. This will attempt to choose a chorus version that lasts between 20 and 30 seconds if possible. If a segment with such a length is not contained in the refrain cluster, then a version is chosen which has the smallest possible deviation to a length of 25 seconds. Is the chosen chorus If it is longer than 30 seconds, it will be hidden for more than 30 seconds in this example and if it is shorter than 20 seconds, it will be extended to 30 seconds with the following segment.
• Storing a medley for the second option is more like an actual summary of a piece of music.
• a section of the stroke, a section of the refrain and a section of a third segment are constructed in their actual chronological order as a medley.
• the third segment is selected from a cluster that has the largest total portion of the song and is not a verse or chorus.
• the selected segments are not installed in their full length in the medley.
• the length is preferably set to a fixed 10 seconds per segment, so that a total of 30 seconds is created again.
• alternative values are also readily feasible,
• a grouping of a plurality of feature vectors in block 510 is performed to save computation time by forming an average over the grouped feature vectors.
• the grouping can save computing time.
• a distance is determined between all possible combinations of two feature vectors.
• Dara.us For n vectors calculations are made over the entire piece of nxn calculations.
• a grouping factor g indicates how many consecutive feature vectors are grouped into a vector via the mean value formation. This can reduce the number of calculations.
• the grouping is also a kind of noise suppression, in which small changes in the feature expression of successive vectors are compensated on average. This property has a positive effect on finding large song structures.
• the concept according to the invention makes it possible to navigate through the calculated segments by means of a special music player and to selectively select individual segments, so that a consumer in a music shop can easily start immediately by, for example, pressing a certain key or by activating a specific software command Re ⁇ frain of a piece can jump to determine whether the chorus pleases him, and then perhaps listen to a strife, so that the consumer can finally make a purchase decision.
• a buyer it is easily possible for a buyer to hear from a piece exactly what he is particularly interested in, while he himself z. , B. the solo or the bridge then actually save for the listening pleasure at home.
• the concept according to the invention is also of great advantage for a music shop, since the customer can listen in and therefore quickly and ultimately buy, so that the customers do not have to wait long to listen in, but also quickly get their turn , This is due to the fact that a user does not have to constantly toggle back and forth, but receives targeted and quick all the information of the piece that he also wants to have.
• the present invention is also applicable in other A.nwen-dungsszenarien, for example, inferfer monitoring, ie where an advertiser wants to check whether the Aud lot, for which he has bought advertising time, actually played over the entire length has been.
• An audio piece may include, for example, music segments, speaker segments, and noise segments.
• the segmentation algo- rithm, ie the segmentation and subsequent classification into segment groups, then makes it possible to carry out a scully-free and considerably less complicated check than a complete sample-wise comparison A comparison of how many segment classes are found, what are and how many segments are in the individual segment classes, with a specification based on the ideal advertising piece. whether a broadcaster or television station has actually aired all or part of the advertising section of the commercial.
• the present invention is further advantageous in that it can be used for research in large music data banks in order, for example, only to listen through the RefTrains of many pieces of music in order then to listen to a music score. -? R -
• the inventive concept d_n hardware or can be implemented in software.
• the implementation can be carried out on a digital storage medium, in particular a floppy disk or CD with control signals which can be electronically read-out, which can cooperate with a programmable computer system in such a way that the corresponding method is executed.
• the invention thus also consists in a computer program product with a program code stored on a machine-readable carrier for carrying out the method according to the invention when the computer program product runs on a computer.
• the invention thus represents a computer program with a program code for carrying out the method when the computer program runs on a computer.

Applications Claiming Priority (2)

Publications (2)

Family

ID=34978834

Family Applications (1)

Country Status (6)

Families Citing this family (28)

Family Cites Families (34)

• 2004
  • 2004-09-28 DE DE102004047069A patent/DE102004047069A1/de not_active Withdrawn
• 2005
  • 2005-02-01 US US11/048,681 patent/US7345233B2/en active Active
  • 2005-02-01 US US11/048,679 patent/US7282632B2/en not_active Expired - Fee Related
  • 2005-07-15 JP JP2007533881A patent/JP2008515011A/ja active Pending
  • 2005-07-15 WO PCT/EP2005/007750 patent/WO2006034742A1/fr active IP Right Grant
  • 2005-07-15 DE DE502005003500T patent/DE502005003500D1/de active Active
  • 2005-07-15 AT AT05762452T patent/ATE390681T1/de not_active IP Right Cessation
  • 2005-07-15 EP EP05762452A patent/EP1794745B1/fr not_active Not-in-force
• 2011
  • 2011-05-02 JP JP2011102839A patent/JP5565374B2/ja not_active Expired - Fee Related

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events