JP2008257020A - Method and device for calculating degree of similarity of melody - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method and a device for calculating the degree of similarity of melody with high estimate accuracy while reducing influences of an overtone structure and a noise. <P>SOLUTION: In a calculation of the degree of similarity of melody for calculating the degree of similarity of melody between two music pieces, a sound pitch transition characteristic for representing temporal changes of base sound included in each of the two music pieces to be compared and an energy characteristic for representing the temporal changes of energy of musical instrument sound except the base sound included in each music signal are obtained. After that, the degree of similarity of the sound pitch transition characteristic in each music piece and the degree of similarity of the energy characteristic of the musical instrument sound except the base sound in each music piece are calculated, and the degree of similarity of melody is calculated by using each degree of similarity. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a melody similarity calculation method and similarity calculation device, and more particularly to a melody similarity calculation method and similarity calculation device for calculating the melody similarity between two music pieces.

With the digitization of music, not only searching by keywords such as music titles and composer names, but also searching using music signals directly has become possible. Thus, various search techniques based on the contents of music signals, such as searching for music of the same genre or searching for similar music, are expected. In order to realize such a search technique, a method for calculating the similarity between music pieces has been proposed (see Non-Patent Documents 1 and 2).
Takakawa Kitagawa, Takafumi Nakanishi, Yasushi Shimizu, “Realization of automatic metadata extraction method for music media data and its application to semantic music search,” IEICE Transactions, vol.J85-DI, no. 6, pp. 512-526, 2002 Tetsuya Hatakeyama, Hironori Takashima, “Melody search based on humming singing,” “The IEICE Transactions, vol.J77-DII, no. 8, pp.1543-1551, 1994

In the method of Non-Patent Document 1, a word is given as metadata to music based on a plurality of feature amounts calculated from a music signal. Furthermore, the similarity between music is given using the relationship between words based on human sensitivity. Therefore, the similarity can be calculated based on the contents of the music by appropriately setting the feature amount and the word. However, since metadata is composed of words based on human sensitivity, setting is difficult, and quantification of similarity is difficult. In addition, it is difficult to reflect music components such as music structure and melody in similarity calculation.
On the other hand, in the method of Non-Patent Document 2, the similarity between the sound obtained from singing and the music can be calculated. However, since the purpose is to search for the same music, it is not sufficient for calculating the similarity between different music.
In view of the above, an object of the present invention is to provide a new melody similarity calculation method and calculation apparatus that solve the above-described conventional problems.
Another object of the present invention is to provide a melodic similarity calculation method and calculation apparatus that can reduce the effects of overtone structure and noise and that has good estimation accuracy.
Another object of the present invention is to provide a melody similarity calculation method and a calculation device capable of calculating the similarity even when the expansion and contraction or omission of the melody occurs.

A first aspect of the present invention is a melody similarity calculation method for calculating a melody similarity between two music pieces, and a pitch indicating a temporal change of a bass sound included in each of the two music signals to be compared. A first step of acquiring transition characteristics; a second step of acquiring energy characteristics indicating temporal changes in energy of instrument sounds other than the bass sound included in each music signal; and similarity of pitch transition characteristics of each music. A third step of calculating, a fourth step of calculating the similarity of the energy characteristics of instrument sounds other than the bass sound of each musical piece;
A fifth step of calculating the similarity of the melody using each of the similarities is provided.
In the second step, for each pitch name of each music signal, an energy time obtained by summing the frequency component energy of the pitch name and the frequency component energy of 2 ^k times (k = 1, 2,...) The change characteristic is obtained, and in the fourth step, the degree of difference in energy characteristics for each pitch name of the two music signals is calculated, the difference degree for each pitch name is summed, and the energy characteristic is calculated by the reciprocal of the total value. Calculate similarity. In the fifth step, the similarity of the melody is calculated by multiplying the similarity of the pitch transition characteristic and the similarity of the energy characteristic.

A second aspect of the present invention is a melody similarity calculation device that calculates the melody similarity between two music pieces, and indicates a pitch that indicates a temporal change in the bass sound included in each of the two music signals to be compared. A pitch transition acquisition unit for acquiring transition characteristics, an energy characteristic acquisition unit for acquiring energy characteristics indicating temporal changes in energy of instrument sounds other than the bass sound included in each music signal, and a pitch transition characteristic of each music A first similarity calculation unit for calculating similarity, a second similarity calculation unit for calculating the similarity of energy characteristics of instrument sounds other than the bass sound of each music piece, and a melody similarity using each similarity Has a melody similarity calculation unit.
For each pitch name of each music signal, the energy characteristic acquisition unit calculates the sum of the frequency component energy of the pitch name and the frequency component energy of 2 ^k times (k = 1, 2,...). A plurality of pitch name energy acquisition units for acquiring time-varying characteristics, wherein the second similarity calculation unit calculates a difference degree of energy characteristics for each pitch name of two music signals; It has a calculation part which totals the difference degree for every and calculates the similarity degree of the said energy characteristic by the reciprocal number of this total value. In addition, the melody similarity calculation unit includes a multiplication unit that calculates the similarity of the melody by multiplying the similarity of the pitch transition characteristic and the similarity of the energy characteristic.

According to the present invention, since the energy of the frequency indicated by each pitch name is used as the energy of the instrument sound other than the bass sound, the influence of the harmonic structure and noise can be reduced, and the similarity calculation of the melody with good estimation accuracy can be performed. A method and a calculation device can be provided.
In addition, according to the present invention, by using a bass sound having a fundamental frequency in a low frequency range, it is possible to calculate a similarity degree that further reduces the influence of the harmonic structure.
Furthermore, according to the present invention, since the DTW is used to calculate the similarity, the similarity can be calculated even when the melody is expanded or contracted or missing.

(A) Outline of the present invention The present invention focuses on a melody that is a component of music. A melody in music is a time transition of a fundamental frequency composed of a plurality of sound sources (see Masao Ishigari, “Rakuten-Theory and Practice,” published by Music Notomo, 1980). In accordance with this melody definition, the present invention assumes that the melody is composed of a bass sound and other instrument sounds. Further, based on this assumption, the similarity is obtained by performing matching processing on the transition of energy indicated by the bass sound and the transition of energy indicated by the instrument sound other than the bass sound. As the energy indicated by the bass sound, the power spectrum in the frequency range where the bass sound exists is used, and as the energy indicated by the other instrument sounds, the energy of the frequency indicated by the pitch name such as C, D, E,. . FIG. 1 is an explanatory diagram of pitch names, 1 is a pitch name in Japanese (Haniho Hetoiroha), 2 is a pitch name in English (CDEFGAB), 3 is a note, 4 is a floor name in C major.
By the way, the music signal has the following two characteristics. The first feature is that the musical instrument sound includes many harmonics of the fundamental frequency (hereinafter referred to as harmonic structure), and as a result, the fundamental frequency becomes difficult to identify as the frequency range increases. The second feature is that the music includes noise such as a stringed sound that occurs during pronunciation, and therefore, a frequency that does not exist on the scale may be estimated as the fundamental frequency of the instrument sound. is there.
The present invention effectively copes with a music signal having the above two characteristics by using a transition of energy indicated by a bass sound and a transition of energy indicated by an instrument sound other than the bass sound, for example, a transition of energy of a pitch name. Can do.
Before describing an embodiment of the present invention, a DTW (Dynamic Time Warping) technique used in the present invention, a method for acquiring pitch transition characteristics, and a method for calculating energy indicated by instrument sounds other than the bass sound will be described.

である。また、wは単調写像であり、次式

が成立する。 (B) DTW Technology First, a general DTW will be described, and then the configuration of the DTW used for similarity calculation according to the present invention will be described.
DTW is a technique for calculating the degree of difference between signals by expanding and contracting two one-dimensional signals. Therefore, it is effective for comparing signals that cause expansion and contraction in the time axis direction. Particularly in music signals, since the performance speed frequently changes, it is effective to use DTW for calculating the similarity obtained from the difference. Hereinafter, in calculating the degree of difference, a signal to be referred to is referred to as a reference pattern, and a signal for obtaining the degree of difference from the reference pattern is referred to as a referenced pattern.
First, calculation of the degree of difference between patterns by DTW will be described. Each element included in the one-dimensional reference pattern of length I is sequentially a ₁ , a ₂ ,..., A _I, and each element included in the reference pattern of length J is sequentially b ₁ , b ₂ ,. b Expressed as _J. Furthermore, if the position set of each pattern is expressed by {a ₁ , a ₂ ,..., A _I }, [b ₁ , b ₂ ,..., B _J ], the expansion / contraction map w that determines the correspondence between the elements of the pattern. : [1,2, ..., I] → {1,2, ..., J} satisfies the following properties. That is,
“W matches the start and end points of the pattern.

It is. W is a monotonic map.

Is established.

When such a mapping w is used, the calculation of the dissimilarity between patterns is replaced with a search problem of the shortest path from the lattice point (b ₁ , a ₁ ) to the lattice point (b _J , a _I ) in FIG. be able to. So in DTW,
"Whatever the initial decision of the initial state, subsequent decisions must be appropriate with respect to the state resulting from the first transition."
The above route search problem is solved based on the principle of optimality. That is, the total path length is obtained as the sum of the partial path lengths. The path length of the part is the cost d (j, i) at the grid point (aj, bi) on the path, and between the two grid points (a _j , b _i ), (b _j-1 , a _i-1 ) In addition to c _{j, i} (j-1, i-1), c _{j, i} (j, i-1), c _{j, i} (j-1, i) . A method for calculating the path length of the portion is shown in FIG. Here, the cost d (j, i) on the lattice point is a penalty when the corresponding elements are different between the reference pattern and the referenced pattern, for example, 0 if the elements are the same, and α if they are different. . In addition, the movement cost c _{j, i} (j-1, i-1) is changed from the lattice point (b _j-1 , a _i-1 ) to the lattice point ( b _j , a _i ), for example, 0 if i = j, and β if i ≠ j due to pattern misalignment.
The partial path length is calculated based on the above cost, and the partial path that minimizes the cost of the entire path is selected. Finally, the total path length can be obtained by calculating the sum of the costs for each selected partial path. From the above, it is possible to obtain the degree of difference of the entire pattern from the degree of difference of each part of the pattern.

本発明上記の条件に従ってDTW による相違度の算出を行う。これより相違度は、(4)式を用いて経路長を漸化的に求めることで算出可能となる。

例えば、(４)式より、図４に示す格子点(1,1)の相違度D(１,１)は
D(１,１)=d(1,1)
であり、格子点（1,2)の相違度D(1,2）は
D(1,2）=d(1,2)+c_1,2(1,1,)+D(1,1)
であり、格子点（2,1)の相違度D(2,1）は
D(2,1）=d(2,1)+c_2,1(1,1,)+D(1,1)
である。同様にして第1行目の格子点の相違度及び第1列目の格子点の相違度が求まる。そして、以後、順次(4)式により他の格子点の相違度が求まってゆき、最終的に格子点（5,6）の相違度D（5,6）が求まる。この相違度D（5,6）が図3の2つのパターンa:｛a₁，a₂，…，a₆｝，b:[ b₁，b₂，…，b₅ ]の相違度となる。 In the present invention, since DTW is applied to a music signal, a more detailed method for calculating the degree of difference is proposed in consideration of the characteristics in calculating the similarity of music signals. The present invention focuses on the fact that musical notes do not lose musical notes even when performance speeds of the same musical composition are different. When this feature is applied to calculation of the degree of difference due to movement between lattice points, it means that all elements included in the reference pattern are included in the referenced pattern and the correspondence between the elements is determined. Therefore, the expansion / contraction map w can be subjected to a tilt restriction expressed by the following equation.

According to the present invention, the degree of difference is calculated by DTW. Thus, the degree of difference can be calculated by recursively obtaining the path length using equation (4).

For example, from equation (4), the dissimilarity D (1,1) of the grid point (1,1) shown in FIG.
D (1,1) = d (1,1)
And the dissimilarity D (1,2) of the grid point (1,2) is
D (1,2) = d (1,2) + c _1,2 (1,1,) + D (1,1)
And the dissimilarity D (2,1) of the grid point (2,1) is
D (2,1) = d (2,1) + c _2,1 (1,1,) + D (1,1)
It is. Similarly, the dissimilarity between the grid points in the first row and the dissimilarity between the grid points in the first column are obtained. Thereafter, the dissimilarity of other lattice points is sequentially obtained by the equation (4), and finally the dissimilarity D (5,6) of the lattice point (5,6) is obtained. Two patterns a of the dissimilarity D (5, 6) Figure _{3: {a 1, a 2} , ..., a 6}, b: [b 1, b 2, ..., b 5] the degree of difference .

(C) Acquisition of pitch transition characteristics In the present invention, the transition of the pitch indicated by the bass sound is used as the transition of the bass sound in the music. The pitch is a fundamental frequency indicated by each note described on the score. Therefore, the transition of the pitch means the transition of energy at the main frequencies included in the bass sound. The pitch of the bass sound is estimated according to the pitch estimation method described in Document 1 below.
Reference 1: Koji Konno, et al. “A study on temporal frequency analysis focusing on the structure of music in the low frequency range of music signals”, ITE Technical Report, vol.29, no.46, pp.13-16, 2005
FIG. 5 is a block diagram of the pitch estimation apparatus. The band pass filter 1 passes a 40 to 250 Hz signal component included in the music signal to be processed based on the frequency band in which the bass sound exists. The power spectrum calculation unit 2 calculates a power spectrum from the bandpass filter output signal. The evaluation function calculation unit 3 adds weights based on a Gaussian function in the time axis direction and frequency axis direction of the power spectrum, and finally, the pitch estimation unit 4 calculates the maximum energy in the weighted power spectrum at each time. The given frequency is estimated as the pitch.

なお、v_t(s)は時間軸の重みであり、

ただし、σは音の持続時間の指標となる定数である。また、（5）式においてw(f)は周波数軸の重みであり、

ただし、mを自然数として、

で示されるFm は、DI（Musical Instrument Digital Interface）のm 番目のノートにおける周波数を表す。
(５)式の評価関数R（t，f）は、(6) 式の時間軸方向の重みにより、一定時間持続する基本周波数を音高として推定可能とする。また、(7) 式に示す周波数軸方向の重みにより、音階上に存在する周波数のみを音高として推定可能とする。以降、評価関数R（t，f）の各時刻t において最大値を与える周波数f をベースの音高とし、B(t) と表す。 That is, if the energy calculated by the power spectrum calculation unit 2 at time t (0 ≦ t ≦ T) and frequency f is P (t, f), the evaluation function calculation unit 3 uses the power weighted by equation (5). The spectrum R (t, f) is output as an evaluation function.

Note that v _t (s) is the time axis weight,

However, (sigma) is a constant used as the parameter | index of the duration of a sound. Also, in equation (5), w (f) is the frequency axis weight,

Where m is a natural number

Fm indicated by represents the frequency in the mth note of DI (Musical Instrument Digital Interface).
The evaluation function R (t, f) in equation (5) makes it possible to estimate the fundamental frequency that lasts for a certain period of time as the pitch by the weight in the time axis direction of equation (6). Further, only the frequency existing on the scale can be estimated as the pitch by the weight in the frequency axis direction shown in the equation (7). Hereinafter, the frequency f giving the maximum value at each time t of the evaluation function R (t, f) is defined as B (t) with the base pitch as the base pitch.

(D) Energy Transition by Other Instrument Sounds Next, a method for calculating energy indicated by instrument sounds other than the bass sound will be described. In a general music configuration, since the bass sound is mainly the lowest sound of the music, the other instrument sounds show frequencies higher than the frequency range of the bass sound. In addition, each pitch name has the frequency shown in FIG. 6 in a frequency range higher than the bass tone, and a frequency 2 ^k (k = 1, 2,...) Times each frequency is treated as the same pitch name. Therefore, in the present invention, the energy indicated by the instrument sound other than the bass sound is assumed to be energy having a frequency higher than the frequency of the bass sound and possessed by the pitch name. Further, as the energy of the frequency indicated by each pitch name, the sum of the energy indicated by the frequency 2 ^k times that in FIG. 6 is used. As a result, the present invention reduces the overtone structure of a plurality of musical instruments, and can also be used for similarity calculation for musical instrument sounds that exist in a frequency range where it is difficult to estimate the pitch. The outline of the above procedure is shown in FIG. 7, and the details will be described below. The signal length of the music signal is T seconds, the sampling rate is fs, and the energy at time t (0 ≦ t ≦ T) and frequency f 1 is calculated from the power spectrum and expressed as P (t, f).

ただし、K はを越えない任意の整数とする。(9)式により各音名が示す周波数のエネルギーを定義することで、倍音の影響が軽減可能となる。
2 ）エネルギーの割合の算出：
1）で得られた各音名が示す周波数のエネルギーを全周波数域に対するエネルギーの割合で表現する。これにより、音名毎に時間軸方向での比較が可能となり、推移を得ることが可能となる。音名X が示す周波数のエネルギーの割合p_x(t) は次式で示される。

以上を全てのt、X について施し、得られたp_x(t) をベース音以外の楽器音におけるエネルギーの推移として用いる。 1) Calculation of energy of frequency indicated by pitch name:
The energy of the frequency indicated by each pitch name is calculated from the power spectrum. In FIG. 6, the frequency P _X (t) of the frequency indicated by the pitch name X is defined by the following equation, where f _X is the frequency corresponding to the pitch name X.

However, K is an arbitrary integer not exceeding. By defining the energy of the frequency indicated by each pitch name using equation (9), the influence of overtones can be reduced.
2) Calculation of energy ratio:
Express the energy of the frequency indicated by each pitch name obtained in 1) as the ratio of the energy to the entire frequency range. Thereby, it is possible to compare in the time axis direction for each pitch name, and to obtain a transition. The frequency energy ratio p _x (t) indicated by the pitch name X is expressed by the following equation.

The above is applied to all t and X, and the obtained p _x (t) is used as a transition of energy in instrument sounds other than the bass sound.

FIG. 8 is a block diagram of a pitch name energy acquisition unit for acquiring the energy characteristics of each pitch name. The frequency component of the pitch name and the frequency component of 2 ^k (k = 1, 2 ^,. The bandpass filters 11 _{1 to} 11 _n that pass therethrough, the synthesizing section 12 that outputs the pitch energy P _X (t) by synthesizing the output of each bandpass filter, the energy ratio p _x (t) according to the equation (10) Is provided.

(E) Melody Similarity Calculation Method FIG. 9 is a processing flow of the melody similarity calculation method of the present invention for calculating the melody similarity between two music pieces.
The present invention first assumes that the melody is composed of a bass sound and instrument sounds other than the bass sound. This is because the sound that is simultaneously generated by the bass sound and the other instrument sounds is an index of chords and keys that determine the characteristics of the melody. Based on the above assumption, the present invention calculates the similarity by applying DTW to the energy of each instrument sound.
That is, the pitch B (t) indicating the temporal change of the bass sound included in each of the two music signals to be compared first is acquired (step 101), and the instrument other than the bass sound included in each music signal is acquired. An energy characteristic p _x (t) indicating a temporal change in sound energy is acquired (step 102). Next, using the DTW based on the equation (4), the degree of pitch difference of each music is obtained, and the reciprocal number is calculated to calculate the similarity Sa (step 103). Thereafter, similarly, using DTW, the degree of difference in energy characteristics of instrument sounds other than the bass sound of each musical piece is obtained, and the similarity Sb is calculated by calculating the reciprocal thereof (step 104). Finally, the similarity of the melody is calculated by calculating the product (Sa × Sb) of the similarities obtained for the pitch B (t) and the energy p _x (t) of the instrument sound other than the bass sound ( Step 105).

ただし、α＞βとする。これにより、メロディーの不一致によるコストと比較して、演奏速度の変化等に伴うメロディーのずれに対するコストが小さくなる。以上により得られた相違度をDb と表す。 Details of similarity calculation will be described below. In the difference calculation by DTW, each music signal is divided in a sufficiently small time, and one is used as a reference pattern and the other is used as a referenced pattern.
1) Calculation of difference in pitch B (t):
As described with reference to FIG. 5, the pitch of the base is estimated for each of the melody on the reference side and the melody on the reference side, and the degree of difference is calculated by DTW. Here, each cost used in DTW equation (4) is set as follows.

However, α> β. As a result, the cost for the melody shift due to a change in performance speed or the like is smaller than the cost due to the melody mismatch. The degree of difference obtained as described above is represented as Db.

以上、（13）式により、全ての音名が示す周波数のエネルギーの推移を用いた相違度算出が可能となる。また、（14）式に示すコストを設定することで、エネルギーの大きな周波数に対応する音名が、相違度全体に与える影響を増加する。これにより、メロディーを構成する主要な周波数成分を反映した相違度算出が可能となる。 2) Calculation of the difference between p _x (t):
For each of the melody on the reference side and the melody on the referenced side, the energy p _x r (t), p _x i (t) for each note name X of the instrument sound other than the bass sound is calculated and used for each note name. The difference degree Dax is calculated by DTW. Therefore, 12 dissimilarities are obtained as the number of pitch names. The degree of difference between instrument sounds other than the bass is defined by the sum of the degrees of difference obtained for each pitch name. That is, if the dissimilarity obtained for the pitch name X is Dax, the sound dissimilarity Da by the instruments other than the bass is expressed by the following equation.
Da = Da _C + Da _Cis + Da _D + Da _Dis + Da _E + Da _F + Da _Fis + Da _G + Da _Gis + Da _A + Da _B + Da _H (13)
In addition, the cost used for the difference calculation by DTW is set as follows.

As described above, it is possible to calculate the degree of difference using the transition of the energy of the frequency indicated by all the pitch names by the expression (13). In addition, by setting the cost shown in the equation (14), the influence of the pitch name corresponding to the high energy frequency on the overall dissimilarity is increased. Thereby, it is possible to calculate the degree of difference reflecting the main frequency components constituting the melody.

3) Calculation of each similarity:
By calculating the reciprocal of the dissimilarities Db and Da obtained in 1) and 2) above, the similarity is obtained and expressed as Sb and Sa.
4) Melody similarity calculation:
From the similarity Sb, Sa obtained in 3) above,
S = Sb x Sa (16)
To calculate the similarity S of the melody.
The similarity between melody is calculated by the above process. The similarity between melody is calculated from the similarity between the energy of the sound of the bass sound and other instruments. For this reason, it is possible to reduce the influence of the overtone structure and noise, and it is possible to calculate the degree of similarity reflecting a plurality of instrument sounds constituting the melody. Also, since DTW is used, similarity can be calculated even when expansion or contraction occurs between melodies.

(F) Melody Similarity Calculation Device FIG. 10 is a configuration diagram of a melody similarity calculation device according to the present invention that calculates the melody similarity between two music pieces. The reference-side and referenced-side pitch transition acquisition units 51 and 71 acquire pitch transition characteristics indicating temporal changes of the bass sounds included in the two music signals to be compared, and the reference-side and referenced-side energy. The characteristic acquisition units 52 and 72 acquire energy characteristics indicating temporal changes in the energy of instrument sounds other than the bass sound included in the music signal. Energy characteristic acquisition unit 52 is provided with twelve pitch names twelve energy obtaining unit 52 ₁ to 52 ₁₂ to the energy characteristics to get each. Although not shown, the energy characteristic acquisition unit 72 has the same configuration. The difference calculation unit 53 uses the DTW to determine the pitch difference Db of each music piece, and the calculation unit 54 calculates the similarity Sb by calculating the reciprocal of the difference.
The difference calculation sections 55 _{1 to} 55 ₁₂ for each pitch name use the DTW to calculate the difference in energy characteristics for each pitch name, and the calculation section 56 sums the difference in energy characteristics for each pitch name to calculate the bass sound. The difference Da of the energy characteristics of the other instrument sounds is obtained (see equation (13)), and the calculation unit 57 calculates the similarity Sa by calculating the reciprocal of the difference Da. The melody similarity calculation unit 58 calculates the product S of the similarities Sb and Sa obtained for the pitch B (t) and the energy p _x (t) of the instrument sound other than the bass sound, and the product is obtained as a melody. Is output as the similarity S.

(G) Experiment An experiment was conducted to confirm the effectiveness of the present invention. Three 44.1 kHz, 10-second music signals obtained directly from the CD were used for the experiment. The music signal is a signal cut out at two places from the same piece of music and one place from different pieces of music. Hereafter, these are called signal 1, signal 2, and signal 3, respectively. The parameters used in the present invention were σ = 0: 1, α = 3, β = 1, γ = 0.1, and k = 10, respectively.
The experimental results are shown in FIGS. FIG. 11 shows the similarity finally obtained between the signals. However, the similarity in the figure represents the result when the signal shown at the left end is used as a reference pattern and the signal shown at the upper right end is used as a referenced pattern. On the other hand, FIG. 12 shows the similarity calculated up to each time. However, FIGS. 12A, 12B, and 12C show the similarities obtained when the reference pattern is the signals 1, 2, and 3, and the other two signals are the referenced patterns. Here, the signals 1, 2, and 3 used for the referenced pattern are indicated by a solid line, a broken line, and a dotted line in the drawing, respectively.

First, consider signal 1. From FIG. 11, it can be confirmed that the similarity with signal 2 is higher than that with signal 3. Further, from FIG. 12A, it can be confirmed that the similarity at each time is also high with the signal 2. This is presumably because the signals 1 and 2 are signals obtained by cutting out different times of the same music piece, and thus the sounds that are simultaneously generated over the entire melody are similar. In addition, it can be confirmed that the similarity after 3.5 seconds in FIG. This is considered to be due to the performance of signal 1 in which the melody was emphasized. For this reason, it is considered that the energy of the frequency indicated by each pitch name has increased, and the similarity with the music 2 has increased.
Next, consider signal 2. Similar to signal 1, it can be confirmed from FIG. 11 that the degree of similarity with signal 1 is higher than that of signal 3. However, from FIG. 12B, it can be confirmed that the similarity between the signals 1 and 3 is small when compared at each time. This is probably because the signal 2 is a chorus of music, so there are many instruments that can be sounded simultaneously, and the signal power is extremely strong. For this reason, it is considered that the signals 1 and 3 having lower power than the signal 2 are estimated to be similar.
Finally, consider signal 3. From FIG. 11, it can be confirmed that the similarity between the signal 3 and the signals 1 and 2 which are different music is low. Further, it can be confirmed from FIG. 12 that the similarity to the signals 1 and 2 is low at each time and the change with time is small. From the above, it can be confirmed that the proposed method can calculate the similarity of music. In FIG. 12, it can be confirmed that the similarity increases with respect to the time axis even in the calculation of the similarity of any of the signals 1, 2, and 3. Further, it can be confirmed that the change in the similarity decreases with the passage of time. These means that the similarity is obtained by route search using DTW, so that the accuracy of similarity calculation deteriorates for a short route. For this reason, it is considered necessary to use a signal having a sufficient signal length in calculating the similarity.

It is a pitch name explanatory drawing. It is the 1st explanatory view of DTW (Dynamic Time Warping) technology. It is the 2nd explanatory view of DTW technology. It is explanatory drawing of DTW of this invention. It is a block diagram of a pitch estimation apparatus. It is frequency explanatory drawing of each pitch name. It is procedure outline explanatory drawing of this invention. It is a block diagram of the pitch name energy acquisition part which acquires the energy characteristic of a pitch name. It is a processing flow of the melody similarity calculation method of the present invention for calculating the melody similarity between two music pieces. It is a block diagram of the melody similarity calculation apparatus of this invention which calculates the similarity of a melody in two music. It is a 1st experimental result. It is a 2nd experimental result.

Explanation of symbols

51, 71 Reference-side and referenced-side pitch transition acquisition units 52, 72 Reference-side and referenced-side energy characteristic acquisition units 52 _{1 to} 52 ₁₂ Pitch name energy acquisition unit 53 Pitch difference calculation unit 54 Calculation unit 55 _{1 to} 55 ₁₂ Difference calculation unit 56, 57 for each pitch name Calculation unit 58 Melody similarity calculation unit

Claims (6)

In the melody similarity calculation method for calculating the melody similarity between two songs,
A first step of acquiring a pitch transition characteristic indicating a temporal change of a bass sound included in each of two music signals to be compared;
A second step of acquiring energy characteristics indicating temporal changes in energy of instrument sounds other than the bass sound included in each music signal;
A third step of calculating the similarity of pitch transition characteristics of each song;
A fourth step of calculating the similarity of the energy characteristics of instrument sounds other than the bass sound of each song;
A fifth step of calculating the similarity of the melody using each of the similarities;
A melody similarity calculation method characterized by comprising: In the second step, for each pitch name of each music signal, an energy time obtained by summing the frequency component energy of the pitch name and the frequency component energy of 2 ^k times (k = 1, 2,...) Seeking change characteristics,
In the fourth step, the energy characteristic dissimilarity for each pitch name of the two music signals is calculated, the dissimilarities for each pitch name are summed, and the energy characteristic similarity is calculated by the reciprocal of the total value. ,
The melody similarity calculation method according to claim 1, wherein: In the fifth step, the similarity of the pitch transition characteristic is multiplied by the similarity of the energy characteristic to calculate the similarity of the melody.
The melody similarity calculation method according to claim 1, wherein: In a melody similarity calculation device that calculates melody similarity between two songs,
A pitch transition acquisition unit that acquires a pitch transition characteristic indicating a temporal change of a bass sound included in each of two music signals to be compared;
An energy characteristic acquisition unit that acquires an energy characteristic indicating a temporal change in energy of an instrument sound other than a bass sound included in each music signal;
A first similarity calculator for calculating the similarity of the pitch transition characteristics of each song;
A second similarity calculator for calculating the similarity of the energy characteristics of instrument sounds other than the bass sound of each song;
A melody similarity calculator that calculates the similarity of the melody using each similarity,
A melody similarity calculation device characterized by comprising: For each pitch name of each music signal, the energy characteristic acquisition unit calculates the sum of the frequency component energy of the pitch name and the frequency component energy of 2 ^k times (k = 1, 2,...). It has a plurality of pitch name energy acquisition units that acquire time-varying characteristics,
The second similarity calculation unit is a difference calculation unit for calculating a difference in energy characteristics for each pitch name of two music signals, sums the difference for each pitch name, and calculates the energy by the reciprocal of the total value. A calculation unit for calculating the similarity of characteristics,
The melody similarity calculation apparatus according to claim 4, wherein: The melody similarity calculation unit calculates a melody similarity by multiplying the similarity of the pitch transition characteristic and the similarity of the energy characteristic,
The melody similarity calculation device according to claim 4, further comprising:

Images