JP4926044B2 - Apparatus and method for describing characteristics of sound signals - Google Patents

Description

  The present invention relates to sound signal analysis, and more particularly to sound signal analysis performed for the purpose of classifying and identifying sound signals in order to describe the characteristics of the sound signals.

  Media for digital distribution of multimedia content is constantly being developed, and a large amount of data is being provided. For human users, this total volume is already much higher. Therefore, it is becoming increasingly important to describe data in text with metadata. Basically, the goal is not only to create text files, but also to create searchable music files, video files, and other information signal files, for example. The goal is to be. One approach for this is known as the MPEG7 standard.

  In particular, when analyzing speech signals, i.e. when analyzing signals containing music and / or language, distinct feature extraction is very important.

  For example, it is further desirable to “enrich” audio data with metadata in order to search for metadata based on the fingerprint of a piece of music. The “fingerprint” needs to be expressive on the one hand and as short and concise as possible on the other hand. The “fingerprint” thus represents a compressed information signal generated from a music signal, and does not include metadata, but is for referring to metadata. For example, reference is made by searching a database or searching for a system that identifies an audio material (“audio ID”), for example.

  Usually, music data is composed of superposed partial signals of individual sound sources. In one piece of popular music, there are relatively few individual sound sources, i.e. singers, guitars, bass guitars, drums and keyboards, but in orchestral songs there may be a very large number of sound sources. The orchestral music and popular music are composed of, for example, superimposed sounds emitted from individual musical instruments. Accordingly, each orchestra song or arbitrary song represents a partial signal superimposed from an individual sound source. The partial signal is a sound generated from an individual instrument of each orchestra or popular music ensemble, and each individual instrument is an individual sound source.

  Alternatively, since the original sound source group can also be interpreted as individual sound sources, at least two individual sound sources can be associated with one signal.

  In the following, a general information signal analysis will be described with reference to an orchestra signal as a mere example. The analysis of the orchestra signal can be done in many ways. Therefore, it is desirable to recognize individual instruments and extract individual instrument signals from the overall signal. If applicable, convert them to notes. The notes function as “metadata”. Furthermore, the possibility of analysis is to extract the main rhythm. Rhythm extraction is preferably performed based on a percussion instrument, not a musical instrument that generates sound, also called a harmony-maintaining instrument. In general, percussion instruments include kettle drums, drums, rattles or other percussion instruments, but harmony sustaining instruments are any other instruments such as violins, wind instruments, and the like.

  Furthermore, all acoustic or synthetic sound generators are counted among percussion instruments (eg, rhythm guitars) that become rhythm sections, depending on the characteristics of the sound.

  Therefore, without recognizing the rhythm where the signal of the harmony sustaining instrument is “disturbed”, for example, extracting only the percussion part from the music of one whole song and performing the rhythm recognition based on these percussion parts, This is desirable for extracting the rhythm of a piece of music.

  With this technique, there is another possibility of automatically extracting different patterns from several pieces of music or detecting the presence of each pattern. “A System for Machine Recognition of Music Patterns” by Coyle, EJ, Shmalevich, I., 1998 IEEE International Conference, Sound, Voice, Signal Processing The subcommittee (http://www2.mdanderson.org/app/ilya/-Publications/icassp98mpr.pdf) searches for the main melody of the melody. For this, the main melody is given. And the search of the location where this main melody occurs is performed.

  "From Raw Raw Polyphonic Audio to Locating Recurring Recurring Theme, Searching for the main melody generated from the polyphonic sound of the original sound (From Raw Polyphonic Audio to Locating Recurring Therme, Schlosser, T.) ) ", 2000 ISMIR (http: /ismir2000.ismir.net/posters/shorter ruger.pdf), the main melody of the melody in the expression obtained by recording the music signal is searched. Again, the main melody is given, and then a search is made for the location where this main melody occurs.

  According to the conventional structure of Western music, in contrast to the rhythm structure, the melody part generally does not occur periodically. For this reason, many of the methods for searching for a melody part are limited to detecting what has occurred individually. In contrast, the object of interest in the field of rhythm analysis is mainly directed to the detection of periodic structures.

  Medic, B., “Music Pattern Extraction: From Musical Structure Extraction”, 2003 CMMR Bulletin (http: //www.ircamu.ircamu.ircamu. repmus / RMPerpers / CMMR-music-2003.pdf) specifies a melody pattern using a self-similarity matrix.

  Meek, Collin, Birmingham, WP, “Thematic Extractor”, 2001 ISMIR (http://ismir2001.ismir.net/pdf/mek.pdf/mek.pdf/mek.pdf ) Retrieves the main melody of the melody. In particular, sequences are searched and the length of one sequence can be from two to a predetermined number of notes.

  Smith, L., Medina, R., "Discovering Theme by Exact Pattern Matching" 2001 (http://sitesee.ist.psu.edu/ -498226.html), the main melody of a melody having a self-similarity matrix is searched.

  Lartillot, O., “Perception-Based Musical Pattern Discovery”, 2003 IFMC Bulletin (http://www.ircam.fr/-equilpes/mm .Pdf), the main melody of the melody is also retrieved.

  Brown (JC), “Determining of the Meter of Musical Score by Autocorrelation”, Journal of the Acoustical Society of Japan (J. of Acoustic, Soc. Of America) 94th. In Vol. 4 No. 1993, a reference value rhythm type that is the basis of one piece of music is calculated from a symbol representation of a music signal, that is, using a periodic function based on a MIDI representation.

  The same can be said of Medic, B., “Automatic Meter Extraction from MIDI files”, 2002 JIM Bulletin (http://www.ircam.fr/equipes). /Repmus/RMPapers/JIM-benoit.2002.pdf). As soon as the periodicity is estimated, the tempo of the audio signal and the reference value rhythm are estimated.

  There is a limit to the method for specifying the main melody of the melody, and the main melody of music is repeated, which is suitable for specifying the periodicity present in the sound signal. However, as already explained, these methods do not describe the basic periodicity of a piece of music and do not include any high order periodicity information. In any case, since it is necessary to consider that there are different variations in the main melody in the search for the main melody of the melody, the method for specifying the main melody of the melody is very expensive. Therefore, it is well known in the music world that the main melody changes normally, for example, by switching, mirroring and the like.

“A System for Machine Recognition of Music Patterns” by Coyle, EJ, Shmalevich, I., 1998 IEEE International Conference, Sound, Voice, Signal Processing Section (http://www2.mdanderson.org/app/ilya/-Publications/icassp98mpr.pdf) "From Raw Raw Polyphonic Audio to Locating Recurring Recurring Theme, Searching for the main melody generated from the polyphonic sound of the original sound (From Raw Polyphonic Audio to Locating Recurring Therme, Schlosser, T.) ) ", 2000 ISMIR (http://ismir2000.ismir.net/posters/soreter ruger.pdf) Medic, B., “Music Pattern Extraction: From Musical Structure Extraction”, 2003 CMMR Bulletin (http: //www.ircamu.ircamu.ircamu. repmus / RMPapers / CMMR-music-2003.pdf) Meek, Collin, Birmingham, WP, “Thematic Extractor”, 2001 ISMIR (http://ismir2001.ismir.net/pdf/mek.pdf/mek.pdf/mek.pdf ) Smith, L., Medina, R., "Discovering Theme by Exact Pattern Matching" 2001 (http://sitesee.ist.psu.edu/ -498226. Html) Lartillot, O., “Perception-Based Musical Pattern Discovery”, 2003 IFMC Bulletin (http://www.ircam.fr/-equilpes/mm .Pdf) Brown (JC), “Determining of the Meter of Musical Score by Autocorrelation”, Journal of the Acoustical Society of Japan (J. of Acoustic, Soc. Of America) 94th. Volume 4 No. 1993 Medic, B., “Automatic Meter Extraction from MIDI files”, 2002 JIM Bulletin (http://www.ircamm.fr/equipes/reprums/reprums/reprums/reprums/reprums/reprums/reprums/repmus/reverse JIM-benoit.2002.pdf)

  The object of the present invention provides an efficient and reliable concept describing the characteristics of a sound signal.

This object is achieved device describes the characteristics of the sound signal according to claim 1, it is achieved by claim 1 way to describe the characteristics of the sound signal according to 3 or a computer program according to claim 1 4,.

  The knowledge based on the present invention is that, for a large number of information that can be calculated efficiently, based on the sequence of entry times by period length determination, division into subsequences, and aggregation into aggregated subsequences, This is to determine the expression characteristics of the signal.

  Furthermore, preferably not only consider one entry time sequence of one instrument over time, ie one entry time sequence of individual sound sources, but also occur in parallel in one piece of music. Consider also a sequence of at least two entry times of two different sound sources. Usually, all sound sources, or at least a subset of sound sources, such as percussion sound sources in a piece of music, for example, are considered to have the same underlying period length. Using a sequence of entry times of two sound sources, a common period length that is the basis of at least two sound sources is obtained. According to the present invention, each entry time sequence is then divided into subsequences. The subsequence length is equal to the common period length.

  Feature extraction is performed based on synthesizing the first sound source sub-sequence into the first synthesized sub-sequence and synthesizing the second sound source sub-sequence into the second synthesized sub-sequence. The synthesized subsequence functions as a feature of the sound signal, and can be further processed using this. For example, semantically important information about music of one whole song such as genre, tempo, reference value rhythm type, similarity to other songs, and the like is extracted.

  Therefore, when the two sound sources assumed for the entry time sequence are percussion sound sources, for example, the characteristic spectrum of the output sound, not the pitch of the sound, or the rising or falling edge of the output sound, not the pitch, In the case of drums, other drum instruments, or any other percussion instrument that distinguishes them due to the fact that they have higher musical meanings, the synthesized sub-sequence of the first sound source and the second sound source are combined. The subsequence forms a drum pattern of the sound signal.

  Therefore, preferably, according to the present invention, a drum pattern is automatically extracted from a music signal that has been recorded, that is, for example, from a musical note representation of the music signal. This expression may be described in the MIDI format, or may be automatically obtained from the audio signal by a digital signal processing method. For example, a well-known concept is used with an independent component analysis (ICA) or a modified method such as a non-negative independent component analysis method, or the keyword “blind source separation” (BSS).

  In the preferred embodiment of the present invention, the drum pattern is first extracted by recognizing the note entry, that is, by recognizing the start time for each different instrument and each pitch of the timbre instrument. Alternatively, note expression may be read out. This reading may include reading a MIDI file, or sampling and image processing of a score, and receiving manually entered notes.

  Here, in a preferred embodiment of the present invention, a raster is determined, whereby the note entry time is quantized and the note entry time is quantized again.

  Here, the length of the drum pattern is obtained as the length of the music measure, the integral multiple of the length of the music measure, or the integral multiple of the length of the music count time.

  Here, the frequency at which a specific musical instrument appears for each reference position is obtained using the pattern histogram.

  Next, in order to obtain a preferable drum pattern form as a characteristic of the sound signal, a corresponding entry is selected. Alternatively, the pattern histogram can be processed as such. The pattern histogram is also a compressed representation of music events, i.e. note generation, and contains information about the degree of change and the preferred count time. The flatness of the histogram represents a strong change, and a “mountain-like” histogram represents a stationary signal in the sense of self-similarity.

  In order to improve the expression of the histogram, the signal is divided into regions with similar signal features, drum patterns are extracted only for regions of similar signals, and other feature regions in the signal are separated. In order to obtain the drum pattern, it is preferable to perform pre-processing first.

  The present invention is advantageous in that it provides an efficient and functioning method for calculating the characteristics of a sound signal. In particular, it is based on performing a division according to a periodic length determined by a statistical method so that it functions correctly and is equal for all signals. In addition, the concept of the present invention can be expanded so that the calculation of the synthesized subsequence is more costly, although the calculation time of the upper level takes time, the determination of the common period length and the drum pattern Judgment also includes more time-of-occurrence sequences, and the fact that more and more different sound sources, ie instruments, are also included, can easily improve the representation and accuracy of this concept. Is a point.

  However, another extensibility depends on whether further processing is required, and if necessary, in order to post-process the resulting synthesized sub-sequence to reduce sub-sequences related to the representation. To calculate a specific number of synthesized subsequences for a specific number of sound sources. For example, histogram entries below a certain threshold can be ignored. However, depending on whether the threshold is determined as to whether the histogram contains only such statements for histogram entries or whether the histogram contains only statements that there is a histogram entry in the synthesized subsequence at a particular time. It may be priced.

  The concept of the present invention is a correctly functioning method based on the fact that it “integrates” with subsequences to synthesize many subsequences. Although this method is performed efficiently anyway, the number of processing steps of the present invention does not require much.

  In particular, the non-pitch percussion instrument referred to below as a drum plays a fundamental role, especially in popular music. Many pieces of information about rhythm and music genre are contained in "notes" played by the drum. For example, it can also be used for intelligent and intuitive searching of music archives so that classification or at least pre-classification respectively can be performed.

  The notes played by the drum frequently form repeating patterns, also called drum patterns. The drum pattern can also function as a compressed expression of a played note by extracting a note image of the length of the drum pattern from a long note image. Thereby, semantically meaningful information about the music of one whole song can be extracted from the drum pattern. For example, information such as genre, tempo, reference value rhythm type, similarity to other music pieces, and the like.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.
FIG. 1 shows a block diagram of an apparatus for describing the characteristics of a sound signal of the present invention.
FIG. 2 shows a schematic diagram for explaining the determination of note entry points.
FIG. 3 shows a schematic diagram representing the quantization raster and the quantization of the notes using the raster.
FIG. 4 is a diagram illustrating an example of a common period length obtained by statistically obtaining the length of time using an arbitrary musical instrument.
FIG. 5 shows an exemplary pattern histogram as an example of a synthesized subsequence of individual sound sources (instruments).
FIG. 6 shows a pattern histogram subjected to post-processing as another example of the sound signal.

  FIG. 1 shows an apparatus for describing the characteristics of a sound signal of the present invention. First, FIG. 1 includes means 10 for providing an entry time sequence for each of at least two sound sources over time. Preferably, the entry time is an already quantized entry time that exists in the quantized raster. FIG. 2 shows a sequence of note entry times of different sound sources, ie instruments 1, 2,... Represented by “x” in FIG. . . FIG. 3 shows a sequence of quantized entry times of each quantized sound source in the raster shown in FIG. 3, that is, each instrument 1, 2,. . . The sequence of the quantization entry time of the musical instrument n is shown.

  FIG. 3 simultaneously shows a matrix or list of entry times. The vertical column in FIG. 3 represents the distance between two raster points or raster lines and thus represents the time interval. Depending on the sequence of entry times, note entries may or may not exist. In the embodiment shown in FIG. 3, for example, in the vertical column indicated by reference numeral 30, note entries of the musical instrument 1 exist. The same applies to the musical instrument 2, as indicated by "x" in the two lines associated with the two musical instruments 1 and 2 in FIG. In contrast, instrument n has no note entry time in the time interval indicated by reference numeral 30.

  Preferably, a sequence of several quantized entry times is supplied from means 10 to means 12 to calculate a common period length. Rather than determining the individual period length of each entry time sequence, the means 12 for calculating the common period length is executed in order to detect the common period length that is the best basis for at least two sound sources. This is because, for example, if you are playing several percussion instruments in a song, all percussion instruments have more or less the same rhythm, so virtually all instruments that make up the sound signal, ie all This is based on the fact that there is always a common period length for the sound sources.

  Here, in order to obtain a set of subsequences of each sound source on the output side, a common sound period length is supplied to the means 14 for dividing each entry time sequence.

  For example, now consider FIG. 4, ie, any instrument 1, 2,. . . It can be seen that a common period length 40 is detected for the musical instrument n. In order to divide a sequence of arbitrary entry times into subsequences having a common period length of 40, means 14 for dividing into subsequences is executed. Next, for example, in order to obtain three sub-sequences for the sequence of the musical instrument 1 shown in the example of FIG. 4, as shown in FIG. The second sub-sequence 42 and the next sub-sequence 43 are divided. Similarly, as described in the entry time sequence for instrument 1, instruments 2,. . . Other sequences of the musical instrument n are also divided into corresponding adjacent sub-sequences.

  Next, in order to obtain the synthesized subsequence of the first sound source and the synthesized subsequence of the second sound source as features of the sound signal, a set of sound source subsequences is supplied to the means 16 for synthesizing each sound source. The Preferably, the synthesis is performed in the form of a pattern histogram. The sub-sequences of the first musical instrument are arranged above each other so that the first interval between the sub-sequences is arranged “above” the so-called first interval between the sub-sequences. Next, as shown in FIG. 5, each entry in each slot of the synthesized subsequence or each entry in each histogram component of the pattern histogram is counted. Therefore, the synthesized sub-sequence of the first sound source becomes the first stage 50 of the pattern histogram in the example shown in FIG. For the second sound source, that is, for example, the musical instrument 2, the synthesized subsequence is the second stage 52 of the pattern histogram. Overall, the pattern histogram of FIG. 5 thus represents the characteristics of the sound signal, which can then be used for further various purposes.

  In the following, different embodiments for obtaining a common period length in step 12 will be described. Pattern length detection can also be implemented in different ways. That is, iterative, eg, based on current note information, eg, from speculative criteria, or preferably estimating many assumptions of pattern length, and verifying their validity using the results obtained The periodic search algorithm can directly generate periodicity / pattern length estimates. Again, this may be done, for example, by interpreting the pattern histogram. For example, it is executed by the synthesizing means 16 or by using other self-similarity measuring means.

  As already shown in FIG. 5, the pattern histogram may be generated by the synthesizing unit 16. Alternatively, in order to weight the notes according to their validity, the pattern histogram can also take into account the intensity of individual notes. Alternatively, as shown in FIG. 5, the histogram may simply contain information regarding whether sound is present in the subsequence or in the bin or time slot of the subsequence. Here, the weighting of individual notes for validity is not included in the histogram.

  In the preferred embodiment of the invention, the features shown in FIG. 5, which are preferably pattern histograms, are further processed here. In doing this, note selection may be performed using criteria such as, for example, comparing the frequency or the combined intensity value with a threshold. Among other things, this threshold may depend on the instrument type or the flatness of the histogram. The entry in the drum pattern may be a Boolean size. “1” indicates the fact that a note has occurred, and “0” indicates that a note has not occurred. Alternatively, the entry in the histogram may be a unit of how large the intensity (volume) or validity of the note generated in this time slot is in the music signal. Considering FIG. 6, it can be seen that each arbitrary time slot or bin has selected a threshold, as indicated by “x” in the pattern histogram of each instrument. The number of entries is 3 or more. In contrast, remove any bins. The number of entries is less than 3, that is, 2 or 1, for example.

  In accordance with the present invention, a musical “result” or musical score is generated from a percussion instrument that is not or not well characterized by pitch. A music event is defined as the occurrence of a musical instrument sound. Preferably, only percussion instruments with no substantial pitch are considered. Events are detected in the audio signal and the instrument type is classified. The temporal position of the event is quantized for the quantized raster. This is also called a tatum grid. In addition, a measure of music, or a measure length in milliseconds, or a number of quantization intervals are respectively calculated, and preferably, an upbeat is also specified. By specifying a rhythm structure based on the frequency at which a music event occurs at a specific position in a drum pattern, the tempo can be specified reliably, and when using musical background knowledge, it is useful to position a bar line Display is obtained.

  Note that each score or feature preferably includes rhythm information such as start time and duration. The estimated value of the reference value information, that is, the estimated value of the time signature is not always necessary for automatic analysis of the recorded music. However, for an artificial reproduction device, generation and reproduction of an effective score is possible. Is necessary to do. Therefore, the automatic music recording process can be divided into two tasks. That is, as already described above, the detection and classification of music events, that is, notes, and the generation of a musical score surface from the detected notes, that is, drum patterns. For this purpose, preferably, the reference value structure of the music is estimated, and the temporal position of the detected note is quantized, the upbeat is detected, and the position of the bar line is determined. In particular, it describes the extraction of the percussion musical score surface without significant pitch information of polyphonic music audio signals. Preferably, event detection and classification is performed using independent subspace analysis.

Extended ICA is represented by independent subspace analysis (ISA). Here, the components are divided into independent subspaces having components that do not need to be statistically independent. By converting the music signal, a multi-dimensional representation of the mixed signal is sought, which is matched with the last estimated ICA. In the past, different methods of calculating the independent component have been developed. The relevant literature, which deals in part with speech signal analysis, is:
1. J. et al. Karhunen, “Neural approaches to independent analysis and source separation” Bulletin pp. 249-266, 1996 Artificial Neural Network European Symposium.
2. M.M. A. Casey and A.C. Westner, “Separation of Mixed Audio Sources by Independent Subspace Analysis” 2000 Bulletin of the International Computer Music Conference, Berlin.
3. J. et al. -F. Cardoso, "Multidimensional independent component analysis" 1998, ICASSP 1998 Bulletin, Seattle.
4). A. Hyvarinen, P.A. O. Hoyer and M.C. Inki, “Topographic Independent Analysis” 2001 Neurocomputing 13 (7), pages 1525-1558.
5. S. Dunovov, “Extracting Sound Objects by Independent Subspace Analysis” 2002 Virtual, Synthetic and Entertainment Speech, Bulletin of the 22nd International Conference on AES, Helsinki.
6). J. et al. -F. Cardoso and A.I. Suloumiac, “Blind beamforming for non Gaussian signals”, 1993 IEEE Bulletin, Vol. 140, No. 6, pages 362-370.

  An event is defined as the occurrence of a musical instrument note. The note generation time is the time when a note is generated in one piece of music. Segment audio signal into parts. Audio signal segments have similar rhythmic characteristics. This is done using a distance criterion between short frames of the audio signal, indicated by a vector of subordinate speech characteristics. The tatum grid and the upper reference level are determined separately from the segmented part. The reference value structure shall not change within the segmented portion of the audio signal. Preferably, the detected event is aligned with the estimated tatum grid. This process roughly corresponds to a quantization function well known in the conventional MIDI sequencer software program for music creation. The length of the bar is estimated from the quantization event list and the repetitive rhythm structure is identified. The estimated tempo is corrected using knowledge about the rhythm structure, and the position of the bar line is specified using musical background knowledge.

  In the following, preferred embodiments of different components of the invention will be described. Preferably, means 10 performs quantization to provide a sequence of entry times for some sound sources. Preferably, the detected event is quantized in the theta grid. A tatum grid is estimated using the note entry time of the detected event together with the note entry time that operates using the conventional note entry detection method. Generation of a theta grid based on detected percussion events will work correctly. It should be noted here that the distance between two raster points in a piece of music usually represents the earliest played note. Therefore, if a piece of music contains at most 1 / 16th note and notes later than 1 / 16th note, the distance between the two raster points of the tatum grid will be 16 / 16th of the sound signal. Equal to the time length of one note.

  In the general case, the distance between two raster points is the largest necessary to represent each generated note value or each temporary period length by forming an integer multiple of this note value. Represents a note value. Thus, the raster distance is the largest common divisor of all generated note durations / period lengths, etc.

  In the following, two alternative approaches for finding a theta grid will be described. First, as a first approach, a theta grid is represented using a bidirectional mismatch procedure (TWM). A series of experimental values of the tatum period, i.e., the distance between two raster points, is derived from a histogram of inter-onset interval (IOI). The IOI calculation is not limited to consecutive pronunciations, but is practically limited to all pairs of pronunciations within a time frame. Tatum candidates are calculated as integer fragments of the most frequently occurring IOI. A candidate that predicts the harmony structure of the IOI is selected according to the optimal bi-directional mismatch error function. Next, the estimated tatam period is calculated by calculating an error function between the comb grid derived from the tatam period and the sounding time of the signal. Therefore, a histogram of IOI is generated and smoothed by the FIR low-pass filter. In addition, the IOI is divided by the peak of the IOI histogram, and for example, a tatum candidate is obtained by setting a value between 1 and 4, for example. After application of the TWM, an approximate estimate of the tatum period is derived from the IOI histogram. Then, using TWM between the note entry time and a number of theta grids that have periods close to the previously estimated tatam period, an accurate estimate of the phase of the tatam grid and the tatam period is calculated. .

The tatum grid is further improved by the second method by calculating using the best match between the note entry vector and the tatum grid, ie, the correlation coefficient Rxy between the note entry vector x and the tatum y. Explain that.

  In order to follow a slight tempo change, the thetagram grid of adjacent frames is estimated to be 2.5 seconds long, for example. The change between adjacent frame's tatam grids is smoothed by low-pass filtering the IOI vectors of the tatam grid points, and the tatam grids are extracted from the smoothed IOI vectors. Each event is then associated with its nearest grid position. Thereby, so-called quantization is performed.

  Next, the musical score is represented by a matrix Tik, i = 1,. . . n and j = 1,. . . , M can also be written. n is the number of detected instruments and m is equal to the number of theta grid elements, ie the number of vertical columns of the matrix. The intensity of the detected event may be deleted or used. This results in a Boolean matrix or a matrix having intensity values.

  In the following, a special embodiment of the means 12 for calculating the common period length will be described. The quantized representation of the percussion event provides useful information about each measure or periodicity estimate of the music on which the sound source is played. For example, the periodicity of the reference value rhythm level is obtained in two stages. First, periodicity is calculated to estimate the bar length.

Preferably, an autocorrelation function (ACF) or an average difference function (AMDF) is used as the periodicity function. These are represented by the following equations.

  Also, AMDF is used for the estimated value of the fundamental frequency of the music signal and the audio signal and the estimated value of the musical measure.

In the general case, the periodicity function measures the similarity or dissimilarity between the signal and its temporally different versions, respectively. Different similarity criteria are well known. Therefore, for example, there is a Hamming distance (HD) for calculating dissimilarity between the two Boolean vectors Bi and B2 by the following equation.

Appropriate expansion and comparison of rhythm structures results from different weighting of similar notes and pauses. Next, as shown below, the similarity B between the two sections of the musical scores T1 and T2 is calculated by weighted addition of Boolean operations.

In the above equation, the weights a, b and c are initially set to a = 1, b = 0.5 and c = 0. a weights the occurrence of a common note, b weights the occurrence of a common pause note, c is the occurrence of a weighting difference, i.e., a note occurs on one musical score, while the other musical score Then, weighting is performed so that notes are not generated. As shown below, the similarity criterion M is obtained by adding the element B.

また、ブール行列の類似性基準が、強度値を考えるために、Ｔ１およびＴ２からの平均値を有する重み付けＢにより拡張してもよい。距離または非類似性それぞれが、負の類似性と見なされてもよい。譜面Ｔと、これをシフトしたバージョンとの間で類似性基準Ｍを算出することにより、周期性関数Ｐ＝ｆ（Ｍ、１）が計算される。シフトは１に基づく。Ｐを基準値モデルの数と比較することにより、拍子記号が求められる。実行した基準値モデルＱは、異なる拍子記号および微少時間の一般的なアクセント位置における一連のスパイクからなる。微少時間は、音楽カウントタイムの継続時間の整数比である。すなわち音楽テンポを確定する音符値（例えば、４分の１音符）と、テータム期間の継続時間との整数比である。 As long as the differences between the matrix elements are considered in a similar way, this similarity criterion is similar to the Hamming distance. Hereinafter, the modified Hamming distance (MHD) is used as the distance reference. Also, weighting vectors ν i, i = 1,. . . n may be used to control the influence of a characteristic instrument. For example, it may be controlled by using a musical background knowledge such as focusing on a small drum (snare drum) or a bass musical instrument, or by a frequency and regularity at which the musical instrument appears.

Also, the Boolean matrix similarity criterion may be extended with a weighting B having an average value from T1 and T2 to consider intensity values. Each distance or dissimilarity may be considered a negative similarity. The periodicity function P = f (M, 1) is calculated by calculating the similarity criterion M between the musical score T and the shifted version. The shift is based on 1. The time signature is determined by comparing P with the number of reference value models. The implemented reference value model Q consists of a series of spikes at different time signatures and a general accent position of a minute time. The minute time is an integer ratio of the duration of the music count time. That is, an integer ratio between a note value (for example, a quarter note) for determining the music tempo and the duration of the theta period.

  When the correlation coefficient is maximum, the best match between P and Q is obtained. In the current system 13, a reference value model is executed for seven different time signatures.

上記の式におけるｂは、推定した小節長さと、Ｔの中の小節数ｐとを示す。以下では、Ｔ’は、それぞれ譜面ヒストグラムまたはパターンヒストグラムとして参照される。大きいヒストグラム値を有する譜面要素Ｔ’ｉ，ｊを検索することにより、譜面ヒストグラムＴ’からドラムパターンが得られる。測定した長さの整数値に対して上述の手順を繰り返し使用することにより、小節を超える長さのパターンが取り出される。音信号のさらなる別の特徴として代表的な最大パターンを得るために、すなわちパターン長さ自体に対して、演奏する音符が最も多いパターン長さが選択される。 For example, a repetitive structure is detected to detect an upbeat and obtain a tempo estimate that functions correctly. In order to detect a drum pattern, a musical score T is obtained from the length of the measure b by adding matrix elements T having similar reference value positions according to the following equation.

In the above formula, b represents the estimated bar length and the number of bars p in T. In the following, T ′ is referred to as a musical score histogram or a pattern histogram, respectively. A drum pattern is obtained from the score histogram T ′ by searching for the score elements T′i, j having a large histogram value. By repeatedly using the above procedure for the integer value of the measured length, a pattern with a length exceeding the measure is extracted. In order to obtain a representative maximum pattern as yet another feature of the sound signal, ie, the pattern length itself, the pattern length with the most notes to be played is selected.

  Preferably, the specified rhythm pattern is interpreted by using a set of rules derived from music knowledge. Preferably, equidistant events in which individual instruments appear are identified and evaluated with reference to instrument classification. This makes it possible to identify performance styles that often appear in popular music. An example is a small drum (snare drum) or tambourine or applause that is used quite often in the second and fourth time signatures of a quarter quarter. This concept called backbeat serves as an indicator of the position of the timeline. If a backbeat pattern is present, time starts between the two beats of the small drum.

  Further, the musical note that positions the time line is the occurrence of a kick drum event, that is, the occurrence of an event of a large drum that is played with a normal foot.

  Assume that the beginning of a musical measure is marked by the reference position where most kick drum notes occur.

  For example, a preferable example of the feature as described in FIG. 5 or 6 as obtained by the sound source synthesizing means 16 shown in FIG. 1 is included in the genre classification of popular music. From the obtained drum pattern, different characteristics at a higher level can be derived in order to specify a normal performance style. The classification procedure evaluates these characteristics using the percussion instrument used in relation to information about the speed of the music bar, ie the time signature, for example, every minute. This concept is based on the fact that any percussion instrument has rhythm information and is frequently played repeatedly. The drum pattern has characteristics specific to the genre. Therefore, these drum patterns can also be used for classification of music genres.

  For this purpose, different performance styles associated with individual musical instruments are classified. Thus, for example, a performance style consists of the fact that an event occurs only on each quarter note. The musical instrument associated with this performance style is a kick drum, that is, a large drum that is played with feet. This performance style is abbreviated as FS.

  Another performance style is, for example, that an event occurs at each second and fourth quarter note of a quarter quarter. It is mainly played with small drums (snare drums) and tambourine, ie applause. This performance style is abbreviated as BS. Further exemplary performance styles consist of the fact that notes often occur on the first and third notes of a triple note. This is abbreviated as SP. Often observed in hi-hats or cymbals.

  Therefore, the performance style is unique to different musical instruments. For example, the first characteristic FS takes a Boolean value and is true only when a kick drum event occurs on each quarter note. For certain values only, Boolean variables are not calculated at all, but as a hi-hat, shaker or tambourine perform, for example, the relationship between the number of offbeat events and the number of onbeat events, A specific number is calculated.

  In order to further obtain the characteristics of genre classification, typical drum instrument combinations are classified into one of the different drum set classifications, for example rock, jazz, latin, disco and techno. Rather than using instrument sounds, the drum set classification is derived by generally detecting the appearance of drum instruments of different musical compositions belonging to individual genres. Thus, for example, drum sets whose classification is rock are distinguished by the fact that they use kick drums, snare drums, hi-hats and cymbals. In contrast, the “Latin” classification uses Bongo, Conga, Claves and Shaker.

  Furthermore, a characteristic set is derived from the drum music score or drum pattern and the rhythm characteristics of each. These characteristics include music tempo, time signature, minute time, and the like. Also, by counting the number of different IOIs that occur in the drum pattern, a reference for the change in which the kick drum note appears can be obtained.

A music genre is classified using a drum pattern using a rule-based decision network. If the currently verified hypothesis is satisfied, possible genre candidates are given, and if the currently verified hypothesis aspect is not satisfied, the candidate is “corrected”. This process results in the selection of a preferred combination of characteristics for each genre. Rules for making wise decisions are derived from observing representative music and the music knowledge itself. Each value for selection or correction is empirically set in consideration of the robustness of the extraction concept. A decision obtained as a particular music genre is considered a genre candidate, including the maximum number of selections.
Therefore, for example, when the drum set type is disco, the tempo range is 115 to 132 bpm, the time signature is 4/4 bits, and the minute time is equal to 2, the genre is recognized as disco. Further, the characteristics of the genre disco are, for example, a performance style FS, for example, that there is another performance style, that is, an event occurs at each offbeat position. Similar criteria may be set for other genres such as hip hop, soul / funk, drums and bass, jazz / swing, rock / pop, heavy metal, Latin, waltz, polka / punk or techno.

  Depending on the situation, the method of the invention describing the characteristics of the sound signal can also be implemented in hardware or software. In practice, the method may also be implemented on a digital recording medium having electronically readable control signals that cooperate with a programmable computer system, in particular a floppy disk or CD. it can. Thus, in general, when a computer program product is executed on a computer, the present invention comprises a computer program product having program code stored on a machine-readable carrier for performing the method. Therefore, in other words, when the computer program is executed on a computer, the present invention can be implemented as a computer program having a program code for executing the method.

FIG. 2 shows a block diagram of an apparatus for describing the characteristics of a sound signal of the present invention. The schematic diagram explaining the determination of a note entry point is shown. FIG. 2 is a schematic diagram illustrating a quantization raster and note quantization using a raster. The figure explaining the example of the common period length obtained by calculating | requiring the length of time statistically using arbitrary musical instruments is shown. An exemplary pattern histogram is shown as an example of a synthesized subsequence of individual sound sources (instruments). As another example of the sound signal, a pattern histogram after post-processing is shown.

Claims (14)

A device for describing the characteristics of a sound signal,
Means (10) for generating a sequence of sound entry times for at least one sound source;
Means (12) for calculating a common period length that is the basis of the at least one sound source using the sequence of the at least one entry time;
And means for dividing pre Symbol least one entry time sequence to each subsequence (14), the length of the subsequence, derives equal to the common period length, or from the common period length Said means for dividing (14);
Said at least one source of said means for combining the sub-sequences into one synthesized subsequence (16), said synthesized subsequence Ri characterized der of said sound signal, said means for combining (16) includes, and means (16) which are Ru, said synthetic configured to generate a histogram as a subsequence which is the synthesis device. Generating means (10) is executed to generate a sequence of at least two entry times for at least two sound sources;
In order to calculate the common period length of the at least two sound sources, calculation means (12) is executed,
In order to divide the sequence of at least two entry times according to the common period length, a dividing means (14) is executed,
The first synthesized subsequence and the second synthesized subsequence represent the characteristics of the sound signal, and a second synthesized subsequence of the second sound source is synthesized into one synthesized subsequence . synthesized in the sub-sequence and to order the synthesis means (16) is executed, according to claim 1.   In order to generate a sequence of one quantized entry time for each of the at least two sound sources, generating means (10) is executed, the entry time is quantized against a quantized raster, and between two raster points The distance between the raster points of the sound signal is equal to the shortest time distance between two sounds in the sound signal or equal to the greatest common divisor of different time durations of different sounds in the sound signal. Equipment.   4. An apparatus according to any one of the preceding claims, wherein generating means (10) is executed to generate the entry time of a percussion instrument rather than the entry time of a harmony instrument.   In order to generate a list for each sound source, generation means (10) is executed, wherein the list for each raster point of the raster contains one associated information as to whether the raster point has a sound entry time. An apparatus according to any one of claims 1 to 4, comprising: Wherein as each raster point of a sound raster synthesized subsequence represents histograms components, in order to generate the histogram, combining means (16) is executed, according to claim 1. When the input is detected, or by adding the reference obtained from the input, the synthesis means (16) is executed, and the count value is increased so that the correspondence in the histogram in each sub-sequence of the sound source is increased. 7. An apparatus according to claim 1 or claim 6 , wherein the count value of the component is increased and the input is a measure of the intensity of the sound having the entry time entry. Combining means (16) is executed for outputting only the values of the subsequences in the first combined subsequence and the second combined subsequence as features exceeding a threshold. Item 8. The device according to any one of Items 7 . Means for extracting characteristics from the features for the sound signal;
Using said characteristic, and means for determining a musical genre belonging to the sound signal, apparatus according to any one of claims 1 to 8. The apparatus according to claim 9 , wherein a determination means is implemented to use a rule-based decision network, pattern recognition means or classifier. Further comprising means for extracting a tempo from the characteristic Apparatus according to any of claims 1 to 1 0. Based on the common period length, to determine the tempo extraction means is executed, according to claim 1 1. A method for describing the characteristics of a sound signal,
Generating (10) a sequence of sound entry times for at least one sound source;
Using the sequence of at least one entry time to calculate a common period length that is the basis of the at least one sound source;
Dividing the sequence of at least one entry time into respective subsequences, wherein the length of the subsequence is equal to or derived from the common period length; The dividing step (14);
Synthesizing the subsequence of the at least one sound source into a synthesized subsequence (16), wherein the synthesized subsequence generates a histogram as the synthesized subsequence, thereby generating a histogram of the sound signal; Combining the step (16) of representing features. When operating the computer program on a computer, the computer program having a program code for performing the method according to the claim 1 3.

Images