CN101430876A - Singing marking system and method - Google Patents

Abstract

The invention discloses a singing grading system and a method thereof. The method comprises the following steps: performing note segmentation and note quantization on singing voice signals, searching the corresponding singing song fragments in songs, computing a plurality of distances and comprehensively obtaining a singing score. The system and the method can help a singer obtain relatively exact score as long as singing data have better consistency with original voice in terms of rhythm, meanwhile, the grading system performs matching by a positioning technology without complete synchronization between the singing and accompaniment, which causes the system to evaluate the singing fragments of the singer without depending on inputting synchronizing information or requiring the singer to sing a song from the beginning to the end.

Description

A kind of points-scoring system of performance and method

Technical field

The present invention relates to the voice process technology field, more particularly, the present invention relates to a kind of points-scoring system and method for performance.

Background technology

Melody is the song most important characteristic, can represent the most quantity of information of song.Melody is made up of each note, embodies by the front and back order of connection of note and the rhythm speed of variation.More particularly, melody can be considered as the sequence of notes that connected to form by a plurality of notes with different attribute (being generally the pitch and the duration of a sound).

Pitch is meant the height of the fundamental frequency of note.Different notes is meant the music base unit with different pitches.Pitch can be that unit is weighed with hertz (Hz), but used unit of measurement is semitone (semitone) usually, particularly represents in the file layout of song with the melody of record at some, and semitone is more common, for example the MIDI file.

The duration of a sound is meant the time span that note continues, and is unit with second (s) or millisecond (ms) generally.

For the evaluation method of singing, the present points-scoring system of Karaoke commonly, this points-scoring system calculates in singer's performance and the database primary sound (perhaps can be considered as the data of the primary sound) difference of some simple physical features between the two, for example relatively sing distance between signal and echo signal sampled point, compare the distance between the sound horizontal curve, differences such as distance between the fundamental frequency sequence relatively, with these differences as evaluation criterion.This points-scoring system is only compared on simple physics feature aspect, when user's performance and the primary sound in the database has than big-difference on these physical features, but but on the melody angle during comparison match, the points-scoring system of Karaoke will show very unstable.So, based on the methods of marking of simple physical feature difference and the vocal music evaluation that is not suitable for singing.

In addition, existing methods of marking is at the situation of singing content complete synchronously with the accompaniment content (or hypothesis fully synchronously), if the performance of actual conditions is asynchronous with accompaniment, the very inaccurate problem of scoring will occur.Based on so synchronous hypothesis, existing methods of marking all can only provide one for whole first song and sing score, but can not provide actual, scoring accurately for singing fragment.

Summary of the invention

Inaccurate and need to sing content and accompaniment content synchronous defective fully for overcoming in the prior art singing marking, the invention provides a kind of points-scoring system and method for performance.

One aspect of the present invention provides a kind of points-scoring system of performance, comprising:

Load module is chosen the performance voice signal that needs marking arbitrarily;

Grading module, relatively sing voice signal on melody with the difference of target song, obtain singing the score of voice signal and target song similarity evaluation.

Described points-scoring system can also comprise:

Note cutting quantization modules, the described performance voice signal that described load module is chosen carries out note cutting and quantification;

Locating module based on the described note after quantizing in the note cutting quantization modules, is set up the pitch difference sequence of singing voice signal, and the corresponding section of speech data is sung in search in the target song storehouse, location;

Distance calculation module, described corresponding section that obtains in the locating module and the pitch difference sequence of singing speech data are carried out multiple distance calculation, described multiple distance is meant sings the distance on melody between voice signal and the target song, comprise the pitch sequence distance, duration of a sound sequence distance, the whole matching distance.

Wherein, described locating module can also comprise: according to the pitch difference sequence of described performance voice signal, and the pitch difference sequence of from the target song database, obtaining target song, use dynamic time warping (DTW) algorithm, realize singing the accurate location of voice signal, obtain singing matching relationship concrete between voice signal and the target song simultaneously.

Wherein, described application dynamic time warping optimal algorithm selection ground comprises: utilize sliding window intercepting target song pitch difference sequence, itself and the pitch difference sequence of singing voice signal are carried out the DTW comparison, obtain representing the cost value of dissimilar degree, the target song segment that replaces the sliding window position correspondence that is worth minimum is as the matching section of singing.

Wherein, the size of described sliding window is not less than the length of singing sequence of notes, preferably, generally gets 2 times to singing sequence of notes length as the sliding window size.

Another aspect of the present invention provides a kind of methods of marking of performance, comprising:

Step 10), choose the performance voice signal that needs marking arbitrarily;

Step 20), relatively sing voice signal on melody with the difference of target song, obtain singing the scoring of voice signal and target song similarity evaluation.

Wherein, step 20) can also comprise:

Step 210), the described performance voice signal of choosing in the step 10) is carried out note cutting and quantification;

Step 220), based on step 210) in described note after quantizing, set up the pitch difference sequence of singing voice signal, the corresponding section of speech data is sung in search in the target song storehouse, location;

Step 230), with step 220) in the pitch difference sequence of the described corresponding section that obtains and performance speech data carry out multiple distance calculation, described multiple distance is meant sings the distance on melody between voice signal and the target song, comprise the pitch sequence distance, duration of a sound sequence distance, the whole matching distance;

Step 240), with step 230) in the described multiple distance that calculates carry out comprehensively, obtain scoring.

Wherein, step 220) further comprises: according to the pitch difference sequence of described performance voice signal, and the pitch difference sequence of from the target song database, obtaining target song, use dynamic time warping (DTW) algorithm, realize singing the accurate location of voice signal, obtain singing matching relationship concrete between voice signal and the target song simultaneously.

Wherein, described application dynamic time warping optimal algorithm selection ground comprises: utilize sliding window intercepting target song pitch difference sequence, itself and the pitch difference sequence of singing voice signal are carried out the DTW comparison, obtain representing the cost value of dissimilar degree, the target song segment that replaces the sliding window position correspondence that is worth minimum is as the matching section of singing.

By using the present invention, make that singing voice signal has consistance preferably with respect to target song, and can be marked accurately, simultaneously, evaluation system of the present invention need not to sing with accompaniment synchronous fully, but utilize location technology to mate, and make the present invention to estimate to singer's performance segment, do not rely on the input of synchronizing information or require the singer to begin to sing ending from song.

Description of drawings

Fig. 1 is the singing marking system composition diagram of one embodiment of the invention;

Fig. 2 is the singing marking process flow diagram of one embodiment of the invention;

Fig. 3 is the path constraint figure of the dynamic time warping (DTW) used of the embodiment of the invention.

Embodiment

Below in conjunction with the drawings and the specific embodiments the present invention is done description further.

As shown in Figure 1, in a preferred embodiment of the invention, comprise load module and grading module, load module is chosen the performance voice signal that needs marking arbitrarily; Grading module mainly by relatively sing voice signal on melody with the difference of target song, obtain singing the scoring of voice signal and target song similarity evaluation.Grading module further comprises, note cutting quantization modules, locating module and distance calculation module.

While is in conjunction with the singing marking process flow diagram of present embodiment shown in Figure 2.

At first in load module, the speech data that input is sung can be the speech data of full songs, also can be the humming sound bite of song.

Then, in note cutting quantization modules, the speech data of this input is carried out the note cutting.In the present embodiment, the speech data form of employing is the 16K sampling rate, and each sampled point is represented with 16 bit numbers (Bit), in the practical application, the invention is not restricted to only handle the 16K*16Bit data, can adopt the sampling rate of any form.Use based on the note cutting method of standout (referring to the Chinese invention patent application, a kind of note cutting method, publication number is CN1963919) based on energy carry out the note cutting.Note slit mode based on standout is a kind of simple and reliable note cutting method based on energy, by harmonic wave and energy standout curve, obtains its effective extreme point, thereby finds the position of note cutting.For the cutting method of note, also can adopt other note cutting methods of this area.

Under the situation that the note cut-off is determined, carry out the quantification of the note pitch and the duration of a sound, in quantizing process, remove the uncertain voice segments that causes because of tidal air, increased the validity of sequence of notes.Pitch quantization unit is semitone (semitone), and duration of a sound quantization unit is a millisecond (ms).Concrete quantification manner is:

The quantification of pitch: at first determine the reference position and the final position of note, be assumed to be T by the note cut-off _s, T _e(embodiment of fundamental frequency extracting method can be referring to document: 1, DikHermes, harmonic wave and fundamental detection method, U.S.'s acoustic journal, in January, 1988, the 257th page the-the 264th page to extract fundamental frequency; 2, Li Ming, Wen Yuan, Yu Tiecheng, based on the high-performance fundamental frequency extracting method of tone feature extraction, international Chinese calculates meeting, calendar year 2001, the 162nd page the-the 166th page), then at T _s～T _eDetermine the less steady component of fundamental frequency rate of change in the scope, with the mean value of the fundamental frequency of this steady component fundamental frequency value f as this note ₀, and then quantize in the semitone unit, concrete formula is as follows:

$N\left(f_0\right)=69+12{\log }_2\frac{f_0}{440},$

N (f ₀) be exactly the pitch value after quantizing.

The quantification of the duration of a sound: be exactly by T _sAnd T _eDecision, concrete formula is:

Dur＝T _e-T _s，

Wherein Dur is exactly the length after quantizing, and unit is a millisecond (ms).

Next step, in locating module, section is sung in the search location from song.Based on the wide in range application requirements of the present invention, the singer's of extraction voice signal need not ending place from the beginning of song to song.The performance of present embodiment location be a kind of be the Orientation and Matching Algorithm of melody based on singing content, according to the result after cutting of previous step note and the quantification, the pitch difference sequence that can obtain singing simultaneously, can be obtained the pitch difference sequence of target song from song melody data storehouse.On the basis of known two groups of pitch difference sequences, use dynamic time warping (Dynamic Time Warping, DTW) algorithm realizes singing the accurate location of segment, and can obtain singing concrete matching relationship between segment and the target song coupling segment simultaneously.For the accurate location of realizing two groups of pitch difference sequences, can adopt those skilled in that art's other algorithms as can be known.

Above-mentioned Orientation and Matching Algorithm, that summarizes says, utilize the dynamic time warping algorithm to search out one section exactly in target song pitch difference sequence and sing the fragment pitch difference sequence that the pitch difference sequence mates most with the singer, the result by dynamic time warping provides concrete matching relationship simultaneously; Specifically, utilize a part of segment (sliding window commonly used be rectangle sliding window) of sliding window intercepting target song pitch difference sequence exactly, with its with sing sequence of notes and carry out the DTW comparison, obtain representing the cost value (cost value) of dissimilar degree.Sliding window slides backward with fixed step size, up to the end position of target song.The each slip, the DTW process can both calculate a cost value.At last, replace the target song segment that is worth minimum sliding window position correspondence, just the most similar target song pitch difference fragment is as the matching section of singing.Wherein, the size of sliding window is relevant with the length of singing sequence of notes, and the length of sliding window is not less than the length of singing sequence of notes, preferably, generally get 2 times to singing sequence of notes length, and the calculating of cost value is mainly according to the dynamic time warping algorithm as the sliding window size.

Dynamic time warping (DTW) algorithm is the application method of pattern match very widely, specifically, is the method for seeking the optimal path of cost minimum under certain path restriction.In this algorithm, the path mode of permission and appropriate cost function have crucial effects.In the present embodiment, choose wherein a kind of DTW algorithm, as shown in Figure 3, the DTW algorithm allows five kinds of path modes, and the short and thick line of black is represented a kind of path respectively.And the calculating of cost value may be summarized to be: the pitch difference sequence of establishing the inquiry melody is X={x1, x2 ..., xNX}, the pitch difference sequence of the melody in the corresponding melody is Y={y1, y2 ..., yNY}.Both dynamic time warping algorithms are as follows:

Initialization: D (0,0)=0

Recurrence:

$D(i,j)=\min \left\{\begin{array}{c}D(i-1,j)+d_{i-1,j}\\ D(i-2,j-1)+d_{i-2,j-1}\\ D(i-1,j-1)+d_{i-1,j-1}\\ D(i-1,j-2)+d_{i-1,j-2}\\ D(i,j-1)+d_{i,j-1}\end{array}\right\}$

Stop: $D_{\min }=\underset{1\≤j\≤N_Y}{\min }D(N_X,j)$

Wherein d is the cost value of each paths, and concrete account form is as follows:

d _i-1，j＝|x _i|+c _p

d _i-2，j-1＝|x _i-1+x _i-y _j|+||y _j|-|x _i-1|-|x _i||+c _p

d _i-1，j-1＝|x _i-y _j|

d _i-1，j-2＝|y _j-1+y _j-x _i|+||x _i|-|y _j-1|-|y _j||+c _p

d _i，j-1＝|y _j|+c _p

Wherein, c _pBe penalty factor, purpose is the cost that increases non-linear consolidation.

Next step, in distance calculation module, segment and the similarity of mating segment are sung in metering, that is to say the distance calculation of singing between segment and the object matching segment.The tolerance that is based on content that the present invention emphasizes is sung the similarity between segment and the object matching segment, and not only be confined to concrete computing method, based on the matching result of previous step DTW, the measure of many similarities can be arranged, two kinds of The common calculation methods just are provided below.

1. first method for optimizing

According to the matching result of DTW process, can calculate the multiple distance of singing between section and the matching section, comprise the pitch sequence distance, duration of a sound sequence distance, whole matching distances etc. are for the calculating of these distances, several different methods can be arranged, and the detailed process that present embodiment preferably calculates is as follows:

At first, calculate the distance of singing pitch difference sequence between section and the matching section, the specific implementation method:

Calculate its Euclidean distance, get average simultaneously:

$D_{\mathrm{note}}={\left(\frac{1}{N}\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{(x_i-y_i)}^2\right)}^{\frac{1}{2}};$

Secondly, calculate the duration of a sound sequence distance of singing between section and the matching section, implementation method:

If the duration of a sound sequence of inquiry melody is A={a1, a2 ..., aNX}, the duration of a sound sequence of the melody in the corresponding melody is B={b1, b2 ..., bNY}, according to the situation of front and back note, do duration of a sound normalization:

$a_1^{\′}=0.0,$

$a_i^{\′}=\log \left(\frac{a_i}{a_{i-1}}\right),i=2...N$

$b_1^{\′}=0.0,$

$b_i^{\′}=\log \left(\frac{b_i}{b_{i-1}}\right),i=2...N$

Calculate its Euclidean distance again, get average:

$D_{\mathrm{dura}}={\left(\frac{1}{N}\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{(a_i^{\′}-b_i^{\′})}^2\right)}^{\frac{1}{2}};$

Then, calculate the whole matching distance, here, no longer define a new distance measure, but with the cost value that obtains in the DTW process as to singer's the performance and the distance of an integral body between target song coupling segment, it at mainly be to sing the pitch difference sequence with respect to problems such as the insertion of object matching segment, deletion errors, be two pitch difference sequences of tolerance a kind of distances of matching relationship on the whole;

At last, calculate the integrate score of singing, computing formula:

${\mathrm{Cost}}_F=100.0(\frac{D_{\mathrm{note}}}{{\mathrm{floor}}_{\mathrm{note}}}\×W_{\mathrm{note}}+\frac{D_{\mathrm{dura}}}{{\mathrm{floor}}_{\mathrm{dura}}}\×W_{\mathrm{dura}})+\frac{1}{N}{\mathrm{Cost}}_{\mathrm{DTW}}\×W_{\mathrm{DTW}}$

Score _F＝max(100.0-Cost _F，0.0)

Wherein, floor _NoteAnd floor _DuraBe respectively maximum pitch sequence distance and the maximum duration of a sound sequence distance that allows, generally speaking, floor _NoteGet 4.0, floor _DuraIt is comparatively suitable to get log4; Cost _DTWIt is the value of the final cost function of DTW process; W _Note, W _DuraAnd W _DTWBe respectively the linear weight that pitch, the duration of a sound and DTW cost function are occupied in final scoring, general W _Note, W _DuraSpan 0.15-0.25 preferably, more excellent gets 0.2, W _DTWSpan 0.55-0.65 preferably, more excellent gets 0.6.Score _FIt is final evaluation score.

2. second method for optimizing

At first, calculate the cosine distance of singing section pitch difference sequence and object matching section pitch difference sequence respectively,

$\mathrm{noteCos}=\frac{\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{x}_i{y}_i}{\sqrt{\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{x}_i^2\×\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{y}_i^2}};$

Secondly, calculate the duration of a sound sequence distance of singing between section and the matching section,

$\mathrm{durCos}=\frac{\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{a}_i{b}_i}{\sqrt{\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{a}_i^2\×\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{b}_i^2}};$

Then, be the calculating of whole matching distance, here, no longer define a new distance measure, but with the cost value that obtains in the DTW process as to singer's the performance and the distance of an integral body between target song coupling segment, it at mainly be to sing the pitch difference sequence with respect to problems such as the insertion of object matching segment, deletion errors, be two pitch difference sequences of tolerance a kind of distances of matching relationship on the whole;

At last, calculate the integrate score of singing,

Cost _F＝Cost _DTW+100.0×(1-noteCos)+10.0×(1-durCos)

Score _F＝100.0-20.0×Cost _F/N _note

Wherein, Cost _FBe cost function, Cost _DTWBe the value of the final cost function of DTW process, and Score _FIt is final evaluation score.

It should be noted that at last, above embodiment is only in order to illustrate that technical scheme of the present invention is not intended to limit, and on using, can extend to other modification, variation, application and embodiment, think that simultaneously all such modifications, variation, application, embodiment are within the spirit and scope of the present invention.

Claims (10)

1, a kind of points-scoring system of performance comprises:

Load module is chosen the performance voice signal that needs marking arbitrarily;

Grading module, relatively sing voice signal on melody with the difference of target song, obtain singing the score of voice signal and target song similarity evaluation.

2, the points-scoring system of claim 1 can also comprise:

Note cutting quantization modules, the described performance voice signal that described load module is chosen carries out note cutting and quantification;

Locating module based on the described note after quantizing in the note cutting quantization modules, is set up the pitch difference sequence of singing voice signal, and the corresponding section of speech data is sung in search in the target song storehouse, location;

Distance calculation module, described corresponding section that obtains in the locating module and the pitch difference sequence of singing speech data are carried out multiple distance calculation, described multiple distance is meant sings the distance on melody between voice signal and the target song, comprise the pitch sequence distance, duration of a sound sequence distance, the whole matching distance.

3, the points-scoring system of claim 2, wherein, described locating module can also comprise: according to the pitch difference sequence of described performance voice signal, and the pitch difference sequence of from the target song database, obtaining target song, use dynamic time warping (DTW) algorithm, realize singing the accurate location of voice signal, obtain singing matching relationship concrete between voice signal and the target song simultaneously.

4, the points-scoring system of claim 3, wherein, described application dynamic time warping optimal algorithm selection ground comprises: utilize sliding window intercepting target song pitch difference sequence, itself and the pitch difference sequence of singing voice signal are carried out the DTW comparison, obtain representing the cost value of dissimilar degree, the target song segment that replaces the sliding window position correspondence that is worth minimum is as the matching section of singing.

5, the points-scoring system of claim 4, wherein, the size of described sliding window is not less than the length of singing sequence of notes, preferably, generally gets 2 times to singing sequence of notes length as the sliding window size.

6, a kind of methods of marking of performance comprises:

Step 10), choose the performance voice signal that needs marking arbitrarily;

Step 20), relatively sing voice signal on melody with the difference of target song, obtain singing the scoring of voice signal and target song similarity evaluation.

7, the method for claim 6, step 20) can also comprise:

Step 210), the described performance voice signal of choosing in the step 10) is carried out note cutting and quantification;

Step 220), based on step 210) in described note after quantizing, set up the pitch difference sequence of singing voice signal, the corresponding section of speech data is sung in search in the target song storehouse, location;

Step 230), with step 220) in the pitch difference sequence of the described corresponding section that obtains and performance speech data carry out multiple distance calculation, described multiple distance is meant sings the distance on melody between voice signal and the target song, comprise the pitch sequence distance, duration of a sound sequence distance, the whole matching distance;

Step 240), with step 230) in the described multiple distance that calculates carry out comprehensively, obtain scoring.

8, the method for claim 7, wherein, step 220) further comprises: according to the pitch difference sequence of described performance voice signal, and the pitch difference sequence of from the target song database, obtaining target song, use dynamic time warping (DTW) algorithm, realize singing the accurate location of voice signal, obtain singing matching relationship concrete between voice signal and the target song simultaneously.

9, the method for claim 8, wherein, described application dynamic time warping optimal algorithm selection ground comprises: utilize sliding window intercepting target song pitch difference sequence, itself and the pitch difference sequence of singing voice signal are carried out the DTW comparison, obtain representing the cost value of dissimilar degree, the target song segment that replaces the sliding window position correspondence that is worth minimum is as the matching section of singing.

10, the method for claim 9, wherein, the size of described sliding window is not less than the length of singing sequence of notes, preferably, generally gets 2 times to singing sequence of notes length as the sliding window size.

Images