JP2007536587A - Apparatus and method for analyzing information signals - Google Patents

Abstract

In order to analyse an information signal, a significant short-time spectrum is extracted from the information signal. The extraction device (16) is embodied in such a way as to extract the short-time spectra which come closer to a specific characteristic than other short-time spectra of the information signal. The extracted short-time spectra are then decomposed (18) into component signals, by ICA analysis, a component signal spectrum representing a profile spectrum of a sound source which generates a sound corresponding to the required characteristic. An amplitude envelope is calculated (20) for each profile spectrum from a series of short-time spectra of the information signal and from the determined profile spectra, said envelope indicating how the profile spectrum of a sound source generally varies over time. The profile spectra and associated amplitude envelopes describe the information signal that can be further evaluated, e.g. for the purposes of a transcription in the case of a music signal.

Description

  The present invention relates to analyzing information signals, such as audio signals, and in particular to analyzing information signals consisting of partial signal superpositions that can originate from individual sources or from a group of individual sources. About.

  The ongoing development of digital distribution media for multimedia content has led to the wide variety of data provided. The vast variety of data provided has greatly exceeded the ease of handling for human users. Thus, the description of data contents by metadata becomes more and more important. In principle, the purpose is to be able to search not only text files, but for example music files, video files or other information signal files, while on the other hand the same convenience as a general text database Is assumed. One approach in this situation is the well-known MPEG7 standard.

  In particular, extracting fingerprints is very important in analyzing audio signals, i.e. signals containing music and / or speech.

  Also, for example, for music, it is envisaged to “enrich” audio data with metadata in order to retrieve the metadata based on the fingerprint. A “fingerprint” is on the one hand to provide a sufficient amount of relevant information and on the other hand to be as short and concise as possible. Thus, a “fingerprint” is generated from a music signal and does not include metadata, but for example by searching a database in a system for identifying audio material (“audio ID”), for example, Fig. 2 shows a compressed information signal useful for referencing metadata.

  Music data usually consists of a superposition of partial signals from individual sources. In pops, there are typically relatively few individual sources: singer, guitar, bass guitar, drums and keyboard, but the number of sources can be very large due to the orchestra part. Orchestra parts and pops consist of, for example, a superposition of tones emitted by individual instruments. Thus, an orchestra portion or any music represents a superposition of partial signals from individual sources, where the partial signals are tones generated by individual instruments in an orchestra and / or pop formation, Each instrument is an individual source.

  Also, even multiple groups of original sources are considered individual sources so that one signal is assigned to at least two individual sources.

  The analysis of a general information signal is shown below for an orchestra signal, for example. The orchestra signal analysis is performed in various ways. For example, if you want to recognize individual instruments, extract individual instrument signals from the entire signal, and convert them to notes as much as possible when the notes act as "metadata" There is. Another possibility of the analysis is to extract the main rhythm, making it easier to extract rhythms based on percussion instruments than on instruments that produce significant tones called harmonically sustaining instruments. It is. Percussion instruments typically include kettle drums, drums, rattles or other percussion instruments, while harmonically sustaining instruments include all other instruments such as violins, wind instruments, and the like.

  In addition, percussion instruments include all their acoustic or synthetic sound producers that contribute to the rhythm section because of their sound characteristics (eg, rhythm guitar).

  Thus, for example, to extract rhythms in music, only the percussion instrument parts are extracted from the entire music, and those percussion instrument parts that do not "interfer" with the rhythm detection by a signal from a musical instrument with sustain sustainably. It is desirable to perform rhythm detection based on

  On the other hand, any analysis pursuing the purpose of extracting metadata that requires information only about instruments that are sustainably in harmony (eg harmonic or melodic analyzes) Benefit from further processing of the part where there is sustain.

  Most recently in this situation, there have been reports on the use of blind source separation (BSS) and independent component analysis (ICA) techniques for signal processing and signal analysis. Application fields are in particular biomedical technology, communication technology, artificial intelligence and image processing.

  BSS typically includes techniques for separating signals from a mixture of signals with knowledge of the mixing process and the nature of the signal or with minimal prior experience. ICA is a method based on the assumption that the underlying sources of a mixture are statistically independent of each other at least in part. Furthermore, the mixing process is assumed to be invariant in time and the number of observed mixed signals is assumed to be greater than or equal to the number of source signals underlying the mixture.

Independent subspace analysis (ISA) represents an extension of ICA. For ISA, the components are subdivided into independent subspaces, and the components need not be statistically independent. By transforming the music signal, a multidimensional representation of the mixed signal is determined and later assumptions for ICA are met. In the last few years, various methods for calculating independent components have been developed. What follows are relevant literature that deals in part with analyzing audio signals:
[1] M. A. Casey and A. Westner "Separation of Mixed Audio Sources by Subbase Analysis International Computer," In Proc. Of the International Computer Music Conference, Berlin, 2000 [2] “Ridim: Based on Independent Subspace Analysis” Rhythm analysis and decomposition tool (Riddim: A rhythm analysis and decomposition tool based on independent subspace analysis ", Master thesis, Dartmouth College, Hannover, New Hampshire, 2001 [3] C. Uhle, C. Ditmar (C. Dittmar) Sperer's "Extraction of Drum Track from Polyphonic Musical Independent Independent Analysis", the 4th International Proceedings of the International Symposium on Independent Component Analysis Proc. Of the Fourth Inter National Symposium on Independent Component Analysis, Nara, Japan, 2003 [4] “Drum transcription for D. Fitzgerald, B. Lawror and E. Coyle Prior Subspace Analysis for Drum Transcription ”, in Proc. Of the 114th AES Convention, Amsterdam, 2003 [5] D Fitzgerald (D. Fitzgerald, B. Lawror, and E. Coyle, “Drum Transcription in the presence of pitched instrument in the presence of a standard pitch instrument using conventional subspace analysis. "Instruments use Prior Subspace Analysis"", ISSC Proceeding (in Proc. of the ISSC), Limerick, Ireland, 2003 [6]" Algorithm for non-negative independent component analysis " (Algorithms for Non-Negative Independent Component Analysis) ", Neura In IEEE Transactions on network (in IEEE Transactions on Neural Networks, 14 (3), pp. 534 - pp. 543, May 2003

  In [1], a method for separating individual sources of a monaural audio signal is described. [2] shows the application for subdivision into signal traces and then rhythm analysis. In [3], component analysis is performed to achieve a subdivision of the polyphony portion into percussion and non-percussion instrument sounds. In [4], independent component analysis (ICA) is applied to the amplitude base obtained from the spectral representation of the drum trace, generally by the calculated frequency base. This is done for transcription purposes. In [5], this method is extended to include a polyphony portion of music.

  The first aforementioned publication by Casey is represented below as an example of the prior art. The publication describes how to separate mixed audio sounds by independent subspace analysis techniques. This includes dividing the audio signal into individual component signals using BSS technology. In order to determine which of the individual component signals belong to the multi-component subspace, grouping is performed on the effect that the mutual similarity of the components is represented by so-called ixograms. An exegram is called an independent component cross-entropy matrix. It is calculated by examining all individual component signals in pairs in the correlation calculation to find the degree of similarity between the two components. In this way, a thorough pair-wise similarity calculation is performed across all component signals, so the result is that all component signals are plotted along the y-axis, and all component signals are A similarity matrix plotted along the x-axis. For each component signal, this two-dimensional array provides a degree of similarity with each other component signal. Exegrams, or two-dimensional matrices, are currently used to perform clustering, for which purpose grouping is performed using a cluster algorithm based on a set of data. A cost function is defined that measures compactness among clusters and determines homogeneity between clusters in order to perform an optimal segmentation of exegrams into k categories. The cost function is minimized so that eventually the result is an assignment of the individual components to the individual subspaces. If this is applied to a signal representing a speaker in the situation of a continuous waterfall noise, the resulting subspace is a speaker, and the reconstructed information signal of the speaker subspace is the waterfall noise. Exhibits significant attenuation.

  A disadvantage with the described concept is that there is a great possibility that the signal part of the source will be in different component signals. Therefore, as mentioned above, a complex and time-intensive similarity calculation is performed between all component signals to obtain a two-dimensional similarity matrix, and based on that, the classification of component signals into subspaces is After all, it is implemented with a cost function that is minimized.

  Also, the disadvantage is that if there are several individual sources, ie the output signal is not speculatively known, even if the similarity distribution is after a little long computation, the similarity distribution itself is actually Is not to give a real idea about the audio scene. Thus, the viewer simply knows that the particular component signals are similar to each other with respect to the minimized cost function. However, it is not known which information will eventually be included in these subspaces obtained and / or which original individual source or which group of individual sources is represented by the subspace. . Thus, independent subspace analysis (ISA) may be used to decompose a time-frequency representation or spectrogram of an audio signal into independent component spectra. To this end, the conventional methods described above rely on frequency and amplitude-based computationally intensive determinations from the entire spectrogram or on a frequency base determined speculatively. Such a speculatively determined frequency-based and / or profile spectrum is said to be very likely, for example, that the portion is characterized by a trumpet, and further, an exemplary spectrum of the trumpet is used for signal analysis. It is to be used for.

  This procedure has the disadvantage of having to know everything that characterizes the instrument in advance, which is in principle contrary to the automated process. A further disadvantage is that if you want to work in a very careful way, for example, not only the trumpet, but all of them differ in sound quality or timbre, so there are also many different types of trumpet that differ in their spectrum That's what it means. If this approach is to use all types of exemplary spectra for component analysis, the method will again be very time consuming, expensive and exhibit very high redundancy. The reason is that typically, not all possible different types of trumpet, but a single type, i.e. a single profile spectrum, or possibly a few different tones, i.e. a few profile spectra. This is because only the trumpet is a major feature in one part. This problem is exacerbated depending on the pitch, especially when the different notes of the trumpet result in each tone having a spread / shrink profile spectrum. Taking this into account involves enormous computational costs.

  On the other hand, if the entire spectrogram is used, decomposition based on the ISA concept is extremely computationally intensive and susceptible to interference. It is pointed out that a spectrogram typically consists of a series of individual spectra, a hopping time period defined between the individual spectra, and a spectrum representing a specific number of samples, so that the spectrum has a specific time associated with it. It has a period or sample of one block of the signal. Typically, this interval, represented by the block of samples from which the spectrum is calculated, is much longer than the hopping time in order to obtain a satisfactory spectrogram with respect to the required frequency resolution and with respect to the required time resolution. However, on the other hand, it can be seen that this spectrogram representation is very verbose. For example, considering the case where the hopping time period reaches 10 milliseconds and the spectrum is based on a block of samples having a time period of, for example, 100 milliseconds, every sample occurs in 10 consecutive spectra. The redundancy generated in this way makes the calculation time requirement extremely large, especially when a relatively large number of instruments are searched.

  Furthermore, an approach that works based on the entire spectrogram is that not all the included sources are extracted from the signal, for example, only certain types of sources, i.e. sources with certain characteristics, are extracted. It is disadvantageous in some cases. Such features relate to the source of percussion instruments, ie percussion instruments, or so-called standard pitched instruments, also called harmonically sustaining instruments, which are typical instruments of tones, such as trumpet, violin etc. To do. Methods that operate based on all these sources are too time consuming and expensive, and eventually are not robust enough if, for example, only some sources, ie those sources that meet certain characteristics, are extracted. In this case, the individual spectra of the spectrogram that do not produce such a source or occur only in a very small range will collapse or “blur” the overall result, because these spectra of the spectrogram are important This is because it is clearly included in the final component analysis calculation that is exactly the same as the spectrum.

M. A. Casey and A. Westner, "Separation of Mixed Audio Sources by Independent Subspace Analysis", International Computer Music Conference In Proc. Of the International Computer Music Conference, Berlin, 2000 IF Orife's “Riddim: Rhythm analysis and decomposition tool based on independence analysis, Master thesis”, Riddim: Rhythm analysis and decomposition tool based on independence analysis (Riddim: Independent subspace analysis) Master thesis, Dartmouth College, Hannover, New Hampshire, 2001 “Extraction of Drum Track inspiring Muscular Pulsing” by C. Uhle, C. Dittmar, and T. Sporer Subspace Analysis) ", Proc. Of the Fourth International Symposium on Independent Component Analysis, Nara, Japan, 2003 (in Proc. Of the Fourth International Symposium on Independent Component Analysis) D. Fitzgerald, B. Lawlor, and E. Coyle, “Prior Subspace Analysis for Drum Transcription 114, Drum Transcription” In Proc. Of the 114th AES Convention, Amsterdam, 2003 D. Fitzgerald, B. Lawlor and E. Coyle, “Drum transcription in the presence of a standard pitch instrument using conventional subspace analysis (Drum "Transcribation in the presence of pitched instruments using Prior Subspace Analysis", ISSC proceeding (in Proc. Of the ISSC), Limerick, Ireland, 2003 M. Plumpley's “Algorithms for Non-Negative Independent Component Analysis”, IEEE Transactions on Neural Networks (in IEEE Transactions on N Pages 534-543, May 2003

  It is an object of the present invention to provide a robust and computation time efficient concept for analyzing information signals.

  This object is achieved by an apparatus for analyzing an information signal according to claim 1, a method for analyzing an information signal according to claim 24, or a computer program according to claim 25.

  The present invention provides a short-time spectrum in which a robust and efficient information signal analysis is derived from the entire information signal and / or from the spectrogram of the information signal from an important short-time spectrum or an important short-period spectrum, for example a difference spectrum. Based on the findings achieved by first extracting, the extracted short-period spectrum is such a short-term spectrum that is closer to a particular feature than other short-term spectra of the information signal.

  What is preferably extracted is a short-time spectrum having a percussion instrument part, so that a short-time spectrum having a harmonic part is not extracted. In this case, the particular feature is a percussion instrument or drum feature.

  Next, the extracted short period or the short period spectrum derived from the extracted short period spectrum is a profile spectrum of the tone source that generates the tones corresponding to the short period spectrum, the component signal spectrum, the feature to be sought. And a means for decomposing the signal into other component signal spectra representing other profile spectra of the tone source that generate tones corresponding to the feature being sought.

  Eventually, the amplitude envelope is calculated over time based on the tone source profile spectrum, and the profile spectrum determined in the same way as the original short-time spectrum is used to calculate the amplitude envelope over time, so that each The amplitude value is similarly obtained when a short-time spectrum is taken for the time.

  The information obtained in this way, ie the various profile spectra as well as the amplitude envelope for the profile spectrum, provide a comprehensive description of the music and / or information signal with respect to the specific features regarding what the extraction has been performed on. As a result, this information is used to perform transcription, i.e., by the feature extraction and segmentation concept, which instruments "belong" to the profile spectrum and which rhythmic movements are nearby, That is, it is sufficient to initially establish whether it is a high or low event that indicates the notes of this instrument played at a particular point in time.

  The present invention has the advantage that only the extracted short-time spectrum is used to calculate the component analysis, i.e. for decomposition, from the overall spectrogram, so that independent subspace analysis (ISA) calculation is achieved. Is performed only to use a subset of all spectra, resulting in lower computational requirements. Furthermore, the robustness with respect to finding a particular source is that there are no other short-term spectra in the component analysis that do not meet a particular feature in particular, so any interference and / or “blurring” of the actual spectrum. It increases so as not to represent.

  Furthermore, the inventive concept has the advantage that the profile spectrum is determined directly from the signal without incurring the problems of existing profile spectra that again lead to inaccurate results or additional computational costs.

  Preferably, the inventive concept is used to detect and classify percussive anharmonic instruments in polyphony audio signals to obtain both profile spectra and amplitude envelopes for individual profile spectra. The

Preferred embodiments of the invention are described in detail below with reference to the accompanying drawings, which include:
FIG. 1 shows a block diagram of an apparatus of the present invention for analyzing information signals,
FIG. 2 shows a block diagram of a preferred embodiment of the apparatus of the present invention for analyzing information signals,
FIG. 3a shows an example of an amplitude envelope for a percussion source,
FIG. 3b shows an example profile spectrum for a percussion instrument source,
FIG. 4a shows an example of an amplitude envelope for a musical instrument with harmonic sustain.
FIG. 4b shows an example profile spectrum for a musical instrument with harmonic sustain.

  FIG. 1 shows a preferred embodiment of the apparatus of the present invention for analyzing an information signal that is sent via an input line 10 to a means 12 for providing a sequence of short-time spectra representing the information signal. As represented by the detour routing 14 depicted by the dashed lines in FIG. 1, the information signal extracts an important short-time spectrum or a short-time spectrum derived from the short-time spectrum, for example in a temporal form. The means for sending to and extracting means 16 are configured to extract such short time spectra that are closer to a particular feature than other short time spectra of the information signal.

  The extracted spectrum, i.e., the original short-time spectrum or the short-time spectrum derived from the original short-time spectrum, is composed of the extracted short-time spectrum by, for example, differentiation, differentiation and rectification, or other operations. The signal spectrum, one component signal spectrum that represents the profile spectrum of the tone source that generates the tone that corresponds to the seeked feature, and the other that represents the other tone source that produces the tone that corresponds to the seeked feature Is sent to means 18 for resolving into the following profile spectrum.

  The profile spectrum is eventually sent to the means 20 for calculating the amplitude envelope for one tone source, which determines how the tone source profile spectrum changes over time and, in particular, the profile spectrum. It shows how the intensity or weighting of changes over time. As can be seen from FIG. 1, the means 20 is arranged to function on the one hand on the basis of the short-time spectrum sequence and on the other hand on the profile spectrum. On the output side, means 20 for calculating provides an amplitude envelope for the source, while means 18 provides a profile spectrum for the tone source. The profile spectrum provides a comprehensive description of the portion of the information signal that corresponds to a particular feature, as well as the associated amplitude envelope. Preferably, this part is a musical percussion part. However, this part may alternatively be a harmonious part. In this case, the means for extracting the important short-time spectrum is configured differently from the case where the specific feature is a percussion instrument feature.

  With reference to FIG. 2, a preferred embodiment of the present invention is represented below. Preferably, perharmonic instrument detection and classification is performed by profile spectrum F and amplitude envelope E, as represented by block 22 of FIG. However, this will be described in more detail later.

  The amplitude reference is interpreted as a set of time variable amplitude envelopes of the corresponding spectral profile.

  According to the invention, the spectral profile is obtained from the music signal itself. Thereby, the computational complexity is reduced compared to the previous method, and increased robustness is achieved for the stationary signal part, ie the signal part with a harmonically sustaining instrument.

  At block 22, feature extraction and classification operations are performed. In particular, the components are differentiated into two subsets, namely the first “non-percussive”, ie the subset with harmonious properties and the other percussion. In addition, components having “percussive / disharmonic” characteristics are further classified into various classes of instruments.

  Percussion or spectral discordant features are used for classification into two subsets.

The following features are used to classify instruments:
A smoothed version of the spectral profile as a search pattern in a training database with individual instrument profiles, spectral centers, spectrum distribution, spectral distortion, center frequency, intensity, spread, most obvious partial line distortion, etc.

The classification is performed, for example, on the following class of instruments:
Kick drum, snare drum, hi-hat, cymbal, tom, bongo, conga, wood block, cowbell, tambar, shaker, tabla, tambourine, triangle, double mosquito, castanets, clapping.

  In order to further increase the robustness of the inventive concept, the decision to start the percussion instrument and / or the acceptance of the percussion instrument maximum is performed in block 24. Thus, a maximum value with a temporary rise in the amplitude envelope above the variable threshold is considered a percussion event, while a maximum value with a temporary rise below the variable threshold is discarded, Or it is recognized as an artifact and ignored. The variable threshold preferably varies with the overall amplitude over a relatively wide range around the maximum value. The output is performed in a suitable form that associates the time of the event of the percussion instrument with the instrument class, intensity and, where possible, further information such as, for example, note and / or rhythm information in the MIDI form.

  It is pointed out here that the means 16 for extracting the important short-time spectrum may be configured to perform this extraction using an actual short-time spectrum, for example obtained by a short-time Fourier transform. Is done. Particularly for the application example of the present invention, the particular feature is that of a percussion instrument, which preferably extracts a short-time spectrum from a differentiated spectrogram or difference spectrum rather than an actual short-time spectrum. The differentiation shown in block 16a of FIG. 2 leads to a sequence of short-time spectra into a sequence of derived and / or differentiated spectra, where each (differentiated) short-time spectrum is now the original spectrum and the next. Including changes occurring between the spectra. In this way, the stationary part of the signal, i.e. the signal part of a harmonically sustaining instrument, for example, is removed in a robust and reliable manner. This is because differentiation emphasizes signal changes and suppresses the same part. However, percussion instruments are characterized by the fact that the tones produced by these instruments are very transient with respect to their passage of time.

  Further, to decompose the short-time spectrum extracted in block 18 of FIG. 1 against PCA 18a and non-negative ICA 18b, ie, more generally speaking, the derived short-time spectrum rather than the original short-time spectrum. It is preferable to perform the disassembling operation. This is an effect that the differentiated signal is very similar to the original signal before differentiation, which is especially true for very very transient signals, especially when there is a very rapid change in the signal. Utilize. This is true for percussion instruments.

  Further, the means 18 for decomposing, performing the PCA 18a with the next non-negative ICA (18b), anyway performs the weighted linear compensation of the extracted spectrum provided by the means to determine the profile spectrum It is pointed out. This means that specific weighting factors calculated by the individual methods are applied to the extracted spectra, or the extracted spectra are combined linearly, ie by subtraction or addition. Thus, at least in part, the means 18 can observe the effect of having the functionality of acting against the derivative to deposit the extracted short-time spectrum, so that it is determined for the tone source. The profile spectrum is not the differentiated profile spectrum but the actual profile spectrum. In any case, the use of the difference spectrum from the difference spectrogram in combination with the differentiated spectrum, i.e. the decomposition algorithm-a decomposition algorithm based on a weighted linear combination of the individual spectra to be extracted, in each high quality means 18. It has been found to lead a profile spectrum for a highly selective tone source. On the one hand, if the stationary part is only processed further, i.e. if a particular feature is not a percussive but harmonious feature, then by integration to reinforce the stationary part compared to the temporary part, That is, it is preferable to achieve spectrogram preprocessing by summing. In this case, it is also preferable to calculate the profile spectrum for the individual—in this case the harmonic—tone source, using the total spectrum, ie the integrated spectrogram.

  The individual functionality of the inventive concept is shown in more detail below. However, in a preferred embodiment of the invention, the representative digital audio signal is first preprocessed by means of preprocessing by means 8. Furthermore, it is preferable to supply a mono file having a 16-bit width for each sample at a sampling frequency of 44.1 Hz as the PCM audio signal input to the preprocessing means 8. These streams of audio signals, which may be streams of video samples, and usually information samples, are based on sound effects device software often referred to as “exciters”. In order to execute the preprocessing within the time range using the emulation, it is sent to the preprocessing means 8. For this concept, the preprocessing stage 8 amplifies the high frequency part of the audio signal. This is accomplished by performing non-linear distortion with a high-pass filtered version of the signal and further adding the distortion result to the original signal. This pre-treatment has been found to be particularly advantageous when there are evaluated hi-hats or idiophones with high pitch and low intensity as well. Their energetic weight with respect to the overall music signal is increased by this step, but most harmonically sustaining and percussion instruments with lower tones are not negatively affected.

  Another positive side effect is that MP3 encoded and decoded files that are inherently low pass filtered by this process again get high frequency information.

  A spectral representation of the pre-processing time signal is preferably obtained using time / frequency means 12 that performs a short time Fourier transform (STFT).

  In order to implement time / frequency means, a relatively large block size of preferably 4096 values and a high degree of overlap are preferred. What is needed first is good spectral resolution for the low frequency range, ie for lower spectral coefficients. Furthermore, the temporal resolution is increased to the desired accuracy by obtaining a small hop size, i.e. a small hop interval between adjacent blocks. In this preferred embodiment, as already explained, 4096 samples per block are by a short time Fourier transform corresponding to a time block period of 92 milliseconds. A value of 10 milliseconds is used as the hop size. This means that each sample occurs more than 9 times in a row in the short-time spectrum.

  It is pointed out that the derivative leads to a negative value, so that half-wave rectification is performed in block 16b to remove this effect. Instead, however, a negative sign is not performed, but can be easily inverted for subsequent decomposition of the components.

  For the rectifier 16b, a non-negative difference spectrogram is obtained and sent to the maximum value searcher 16c.

  The maximum value searcher 16c performs event detection handled below. Detection of several local extremes, preferably local maxima associated with a temporary start event in the music signal, is performed by first defining a time tolerance that separates two consecutive drum starts. The In this preferred embodiment, a time period of 68 milliseconds is used as a constant value derived from the time resolution and from knowledge about the music signal. In particular, this value determines the number of differentiated individual spectra that must occur at least between a frame and / or individual spectra and / or two consecutive starts. The use of this minimum distance is supported by the consideration that a sixteenth note lasts 60 milliseconds at a very high upper speed limit of 250 bpm.

  Maximum search reliability is preferably improved by maintaining only those maximum values that appear in the window for those that are longer than the moment, because it is highly possible that they are interesting peaks. Because it is. Thus, it is preferred to use those maximum values that represent the maximum values that exceed the predetermined threshold of the moment, ie, for example, three moments, which threshold depends on the ratio of the block period and the pop size. This must be the maximum value for a certain number of moments if it is actually the maximum value that is important, ie, for each of the above numbers, each sample is at least 9 consecutive When considering the fact that it is “related to” a short time spectrum, it indicates that it must be the maximum value for a certain number of overlapping spectra.

  According to the invention, a small part of the difference spectrogram, in particular a short-time spectrum formed by differentiation, is extracted and sent to the next decomposition means.

Using this well-known technique, it is possible to reduce the entire set of short-time spectra collected in a limited number of uncorrelated principal components, resulting in a good representation of the original data with small reconstruction errors. For this purpose, the eigenvalue decomposition (EVD) of the covariance matrix of the data set is calculated. From a set of eigenvectors, those eigenvectors with d largest eigenvalues are selected to provide coefficients for a linear combination of the original vectors according to the following equation:

を用いて抽出されるスペクトルプロファイルは、オリジナルスペクトログラムの範囲の値を有する。更なる正規化は、そのＬ２ノルムによる各々のスペクトルプロファイルを分割することによって達成される。 Normalization of R with its absolute maximum yields a weighting factor ranging from -1 to +1, resulting in the following equation

The spectral profile extracted using has a value in the range of the original spectrogram. Further normalization is achieved by dividing each spectral profile by its L2 norm.

As mentioned above, independent and invariant assumptions are not necessarily 100 percent satisfactory for a given short-time spectrum. Therefore, it is not unexpected that the spectral profile obtained after unmixing still exhibits a certain dependence. However, this should not be regarded as incomplete behavior. Tests performed using the spectral profiles of individual percussion instrument tones have been found that the spectral profile also exhibits a great deal of dependence between the starting spectra of different percussion instruments. One possibility to measure the degree of mutual overlap and similarity along the frequency axis is to make crosstalk measurements. For illustration purposes, the spectral profile obtained from the ICA process can be viewed as a transfer function of a very frequency selective part in the filter bank and can lead to crosstalk at the output of the filter bank channel for the passband. The crosstalk that exists between the two spectral profiles is calculated according to the following equation:

  In the above equation, i ranges from 1 to d, j ranges from 1 to d, and j is different from i. In fact, this value is related to the well-known correlation coefficient, but the latter uses a different normalization.

Based on the determined spectral profile, amplitude envelope determination is performed in block 20 of FIG. For this purpose, the original spectrum, i.e. a sequence of short-time spectra obtained, for example, by means 12 of FIG. 1 or in the time / frequency converter 12 of FIG. 2, is used. The following equation applies:

As a second source of information, a differentiated version of the amplitude envelope can be determined from the difference spectrogram according to the following equation:

  Essential to this concept is that no further ICA calculations are performed by the amplitude envelope. The inventive concept also provides a very specialized spectral profile that is very close to the spectrum of those instruments that actually occur in the signal. Nevertheless, it is only in certain cases that the extracted amplitude envelope is an excellent detection function with a sharp peak, for example for music that is oriented to dance with the rhythmic part of a very major percussion instrument. is there. The amplitude envelope often includes relatively small peaks and plateaus that may be due to the above-described crosstalk effect.

  A more detailed implementation of the means 22 for feature extraction and classification is indicated below. It is well known that the actual number of components is not initially known for a real music signal. In this context, “component” refers to both the spectral profile and the corresponding amplitude envelope. If the number d of extracted components is too low, unaccounted component artifacts can very much occur in other components. On the other hand, if too many components are extracted, the most prominent components are divided into several components. Unfortunately, this division occurs even with the correct number of components, sometimes making it difficult to detect real components.

  To overcome this problem, the maximum number of components d is specified in the PCA or ICA process. The extracted components are then classified using a set of spectral and time-based features. Classification is to provide two types of information. Initially, those components that are detected will be removed from further procedures, like non-percussion instruments, with a high degree of certainty. Furthermore, the remaining components will be assigned to a predetermined class of musical instruments.

  Thus, the amplitude envelope, as explained below, has a distinct spectrum spectrum in the case of percussion instrument sources (FIG. 3b; hi-hat) and harmonically sustaining instruments (FIG. 4b; guitar). Equally well used for classification and / or feature extraction. Thus, harmonically sustaining instruments produce harmonics strongly, whereas percussive sources do not have clearly pronounced harmonics, but the overall energy concentration range. And this range of concentrated energy has a very broad spectrum, which is very broadband.

  Thus, spectrum-based measurements, ie, measurements derived from profile spectra (eg, FIGS. 3b and 4b), are preferably used to separate harmonically sustained tone spectra from those associated with percussion instrument tones. . Again, in a preferred embodiment, a modified version that calculates this measurement is used, exhibiting tolerances for spectral delay phenomena, discord with all harmonics, and proper normalization. A higher degree of computational efficiency is achieved by exchanging the original anharmonic function with a weight matrix for frequency pairs.

  Assigning spectral profiles to a given class of percussion instruments is provided by a simple classifier to classify the k next neighbors with the individual instrument's spectral profile as a training database. The distance function is calculated from at least one correlation coefficient between the query profile and the database profile. Additional information that provides detailed information about the form of the spectral profile, in order to check the classification in the case of low confidence, i.e. for low correlation coefficients or to check for multiple occurrences of the same instrument Features are extracted. These features include the individual features already mentioned above.

  In the following, the functionality of the determiner 24 of FIG. 2 will be dealt with. A drum-like start is detected in an amplitude envelope, such as the amplitude envelope of FIG. 3a, using a common peak selection method, also called peak picking. Only peaks that occur within an acceptable range in addition to the original time t, i.e., the time that the maximum value searcher 16c provides results, are mainly considered candidates for start. Any remaining peaks extracted from the amplitude envelope are initially stored for further consideration. A value for the magnitude of the amplitude envelope is associated with each starting candidate at that position. If this value does not exceed a predetermined dynamic threshold, the start is not accepted. In a relatively large time range surrounding the start, the threshold varies across the amount of energy. Most crosstalk effects of harmonically sustaining instruments and percussion instruments played at the same time are reduced in this step. Furthermore, it is preferable to differentiate as to whether the simultaneous onset of various percussion instruments actually exists or only based on the crosstalk effect. The solution to this problem preferably accepts these further occurrences having relatively high values when compared to the values of most intense instruments at the start.

  According to the invention, automatic detection and preferably automatic classification of non-standard pitch percussion instruments in real polyphony music signals is achieved, the starting criterion for this being on the one hand the profile spectrum and on the other hand the amplitude envelope. It is. Furthermore, music rhythm information can be easily extracted from percussion instruments, which are likely to lead to desirable note-to-note transcription one after another.

  In some situations, the inventive scheme for analyzing information signals may also be implemented in hardware or software. This implementation is capable of interacting with a programmable computer system such that the method is carried out, and has a control signal that can be read electronically, in particular a digital storage medium, in particular Propy® It can be done on a disc or CD. The present invention also generally resides in a computer program product having program code for performing this method stored on a machine readable carrier when the computer program product is executed on a computer. In other words, the present invention can also be realized as a computer program having a program code for executing this method when the computer program is executed on a computer.

FIG. 1 shows a block diagram of an apparatus of the present invention for analyzing information signals. FIG. 2 shows a block diagram of a preferred embodiment of the apparatus of the present invention for analyzing information signals. FIG. 3a shows an example of an amplitude envelope for a percussion instrument source. FIG. 3b shows an example of a profile spectrum for a percussion instrument source. FIG. 4a shows an example of an amplitude envelope for a musical instrument with harmonic sustain. FIG. 4b shows an example profile spectrum for a musical instrument with harmonic sustain.

Claims (25)

An apparatus for analyzing an information signal,
Means (16) for extracting an important short-time spectrum or an important short-time spectrum derived from the information signal from the information signal, the means (16) for extracting comprising: Means (16) configured to extract such a short time spectrum that is closer to a particular feature than other short time spectra of said information signal;
The extracted short-time spectrum is divided into a component signal spectrum, a component signal spectrum representing a profile spectrum of a tone source that generates a tone corresponding to the feature to be sought, and a tone corresponding to the feature to be sought. Means (18) for decomposing into other component signal spectra representing profile spectra of other tone sources that are generated;
Means (20) for calculating an amplitude envelope for the tone source, wherein the amplitude envelope for the tone source determines how the profile spectrum of the tone source changes over time, and the profile spectrum and Means (20) for indicating with a sequence of short-time spectra representing said information signal.   The means for extracting (16) is enhanced in the information signal, wherein the signal part present in the information signal at a higher frequency is compared to the signal part present in the information signal at a lower frequency. The apparatus of claim 1, wherein the apparatus is configured to pre-process (8) the information signal. Said means (16) for extracting comprises:
In the pre-processing (8),
High-pass filtering the information signal;
Distorting the high-pass filtered version of the information signal in a non-linear manner,
The apparatus of claim 2, wherein the apparatus is configured to add the nonlinearly-distorted signal to the original information signal.   The means for extracting (16) is configured to time domain / frequency domain transform (12) the information signal to obtain a sequence of short time spectra, wherein two short time spectra adjacent in time are: An apparatus according to any preceding claim, associated with the portion of the information signal that overlaps except for a hopping interval. Each short-time spectrum comprises a sequence of spectral coefficients,
The means (16) for extracting is configured to differentiate (16a) the sequence of short time spectra with respect to time to obtain a differentiated short time sequence, the differentiated time spectrum preceding 5. The apparatus of claim 4, wherein the apparatus provides information about changes in the short-time spectrum as compared to the next short-time spectrum.   Said means for extracting (16) is, for each spectral coefficient, a short time differentiated by forming a difference of said spectral coefficient in the current short time spectrum and the previous or next short time spectrum. The apparatus of claim 5, configured to obtain a spectrum.   The means (16) for extracting is configured to rectify (16b) the differentiated short-time spectrum such that the differentiated and rectified short-time spectrum does not exhibit any negative value; Apparatus according to claim 5 or claim 6.   8. An apparatus according to any of claims 5 to 7, wherein the means (16) for extracting is configured to determine an important signal based on the differentiated short-time spectrum.   Said means (16) for extracting, from each differentiated short-time spectrum, for each differentiated short-time spectrum, a spectral coefficient or a spectral coefficient is obtained from said differentiated short-time spectrum. 9. The apparatus of claim 8, wherein the apparatus is configured to sum (16c) values derived from the result, resulting in a detection function over time.   The apparatus according to claim 9, wherein the means (16) for extracting is configured to smooth the detection function over time.   The means for extracting (16) finds a maximum value in the detection function at a certain time (16c) and is differentiated as an important spectrum having a time point associated with it at which the detection function exhibits a maximum value. 11. An apparatus according to claim 9 or claim 10 configured to use a short time spectrum or a short time spectrum.   The means for extracting (16) is configured to consider only such maximum values of the detection function as significant spaced apart in time beyond a predetermined time period, 12. A device according to any one of claims 9 to 11.   The means for extracting (16) is configured to determine a total amount spectrum as a sequence of short-time spectra and to use the phase information of the short-time spectra when extracting the important short-time spectrum, The apparatus according to any one of claims 4 to 12.   The means for decomposing (18) is configured to add (18a) the extracted short-time spectrum in a weighting method to obtain a reduced number of extracted short-time spectra An apparatus according to any of the claims.   The means (18) for decomposing is configured to perform a principal component analysis (18a) for dimensionality reduction to obtain a processed short-time spectrum. The device according to any one of the above.   The means for decomposing (18) is configured to perform independent component analysis (18b) to produce a plurality of component signals, the component signals being associated with information sources contributing to the information signal. A device according to any of the preceding claims.   The means (20) for calculating the amplitude envelope includes a matrix containing the profile spectrum and a sequence of short-time spectra of the information signal to obtain the amplitude envelope for the tone source. An apparatus according to any preceding claim, wherein the apparatus is configured to multiply with a matrix.   The means for calculating the amplitude envelope is configured to further determine a differentiated amplitude envelope using the profile spectrum for the tone source and further using the difference spectrogram. An apparatus according to any of the claims.   An apparatus according to any preceding claim, further comprising means (22) for classifying the component signals into percussion instrument component signals and non-percussion instrument component signals.   20. Apparatus according to claim 19, wherein the means (22) for classifying is configured to perform classification based on the profile spectrum and / or the amplitude envelope.   21. The means (20) for classifying is configured to extract features from the profile spectrum or the amplitude envelope and compare them to features of known sources in a database. The device described in 1.   When the means for extracting (16) extracts an important short-term spectrum at a time similar to a threshold, the amplitude envelope for the tone source is set to the maximum of the amplitude envelope as the start of the signal from the tone source. Apparatus according to any of the preceding claims, further comprising means (24) for considering to accept the value.   The means (20) for calculating the amplitude envelope indicates the amplitude envelope for a tone source and how the amplitude envelope changes the intensity or weighting of the tone source profile spectrum over time. An apparatus according to any preceding claim, wherein the apparatus is configured to calculate. A method for analyzing an information signal, comprising:
Extracting an important short-time spectrum from the information signal or an important short-time spectrum derived from the short-time spectrum of the information signal, wherein the extracted short-time spectrum includes the information signal Step (16), which is such a short-term spectrum that is closer to a particular feature than the short-term spectrum of
The extracted short-time spectrum is divided into a component signal spectrum, a component signal spectrum representing a profile spectrum of a tone source that generates a tone corresponding to the feature to be sought, and a tone corresponding to the feature to be sought. Resolving (18) into other component signal spectra representing profile spectra of other tone sources to be generated;
Calculating (20) an amplitude envelope for the tone source, wherein the amplitude envelope for the tone source determines how the profile spectrum of the tone source changes over time, the profile spectrum and the information; And (20) indicated with a sequence of short time spectra representing the signal.   A computer program comprising program code for performing the method for analyzing an information signal as claimed in claim 24 when the computer program runs on a computer.

Images