CN1737796A - Across type rapid matching method for digital music rhythm - Google Patents
Across type rapid matching method for digital music rhythm Download PDFInfo
- Publication number
- CN1737796A CN1737796A CN 200510029494 CN200510029494A CN1737796A CN 1737796 A CN1737796 A CN 1737796A CN 200510029494 CN200510029494 CN 200510029494 CN 200510029494 A CN200510029494 A CN 200510029494A CN 1737796 A CN1737796 A CN 1737796A
- Authority
- CN
- China
- Prior art keywords
- melody
- note
- segmentation
- coupling
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Auxiliary Devices For Music (AREA)
Abstract
This invention relates to digital music rhythm over high speed matching method, which comprises the following steps: standard rhythm characteristics reading and match index controlling step; rhythm section position testing and moving match controlling step; rhythm section matching step: rhythm matching index result displaying step. The symbol is described by high voice difference, time vector series; the high voice difference is the difference between front symbols as semi part as unit; time is to the starting point of symbol to express rhythm characteristics.
Description
Technical field
What the present invention relates to is the method in a kind of Computer Applied Technology field, specifically is a kind of great-leap-forward high speed matching process of digital music melody.
Background technology
Digital music retrieval mode based on the humming input is called for short singing search (QBH:Query by Humming), and it allows the user to retrieve needed song by the form of humming.As long as the user can remember fragment melody wherein, and hum out by microphone, the QBH searching system just can find its needed song for the user.Aspect singing search, because the user relies on the memory humming of oneself, be the unspecified person humming in addition, how effectively the melody mode of inquiry input carry out regularization, robustness when how to improve the retrieval of melody coupling, and how realizing towards the retrieval high speed of extensive melody etc., these problems also are not well solved, the further further investigation that all awaits of many gordian techniquies.
At singing search, the technology towards the melody coupling that has proposed both at home and abroad includes: approximate symbol string comparison algorithm, typically as DP (Dynamic Programming) method and quick Approximate Match Method, pitch profile geometric match method and linear alignment are similar to melody matching algorithm LAN (Linear Alignment Mahcing).In the method for melody coupling, the melody characteristics pattern is described by the note characteristic sequence at these, and the note feature is then generally characterized by note pitch (or pitch is poor relatively) and note length (note length ratio relatively); When carrying out the pattern match of non-isometric note characteristic sequence between humming melody fragment and the whole first melody melody, all be to carry out mobile continuously one by one the coupling along the standard note sequence direction of whole first melody melody according to the sequence of notes that the note Moving Unit will import the melody fragment.Find through literature search prior art, publication is at volume o. 11th in " computer research and development " November the 40th in 2003, article on the pp.1554-1560 " linear alignment is similar to the melody matching algorithm ", if mate and when moving to K note when input melody fragment moves along the sequence of notes of standard melody, then the core procedure of this algorithm is: the head note that (1) will import the melody fragment earlier aligns with K note of standard melody, and taking-up length serves as to import the sequence of notes section of melody fragment length 1.3 double-lengths as the standard melody matching section under this head note alignment from the standard melody that is starting point with K note; (2) on time shaft, carry out the approximate melody coupling of linear alignment of two melody matching sections, be about to import melody fragment linearity and be extended to the length identical with the sequence of notes section of standard melody, and constantly approaching note takes place in alignment in certain error range, calculate the similarity of melody on rhythm, continue the relatively pitch difference of two sections isometric melody on each time point simultaneously, calculate the similarity of melody on pitch, at last, take all factors into consideration the similarity of rhythm and pitch two aspects, provide the matching score of input melody fragment and this standard melody sequence of notes section.Then, the head note of input melody fragment will continue to move a note backward along the sequence of notes of standard melody, and repeat the coupling of above-mentioned melody sequence of notes section based on core procedure, till moving at last.The approximate melody matching algorithm of linear alignment has rhythm zmodem, advantage that matching precision is high, but because this mobile matching way response time based on the note Moving Unit is longer, is unsuitable for the match retrieval as the large scale digital music libraries.Therefore,, will become increasingly conspicuous, become based on march toward a big bottleneck of practicability of the digital music searching system of humming input towards the response speed and the contradiction between the retrieval precision of melody coupling retrieval along with the scale in digital music storehouse increases day by day.
Summary of the invention
The objective of the invention is at the deficiencies in the prior art, a kind of great-leap-forward high speed matching process of high performance digital music melody is provided, make when maintenance is hummed the fault-tolerance of input to the user, can also improve coupling retrieval rate significantly the humming input.
The present invention is achieved by the following technical solutions, the present invention includes following steps:
(1) the standard melody characteristics reads and mates the retrieval controlled step: control is kept at the reading and mate retrieval process of note characteristic sequence that melody characteristics extracts the note characteristic sequence of the input melody of storage part as a result and is kept at the accurate melody of whole head in the standard melody characteristics storehouse;
(2) melody segmentation position probing and mobile coupling controlled step: detect the feature note that characterizes each melody segmentation position in the melody standard melody, the note characteristic sequence in the standard melody between per two feature notes promptly is defined as a melody characteristics segmentation; Simultaneously, the great-leap-forward move mode when each the melody segmentation in control input melody fragment and the melody standard melody is mated, and the whole first melody matching result of output;
(3) melody segmentation coupling step: import that a certain melody divides intersegmental pattern match in melody fragment and the melody standard melody, the matching result of melody segmentation will return described melody segmentation position probing and move the control process of mating controlled step;
(4) melody mates the result for retrieval step display: show based on the result of the ultimate criterion melody coupling retrieval of input melody fragment, comprise the comparative view of the melody melody characteristics curve that mates with preceding N position and the text attribute of relevant melody thereof;
Described melody characteristics is described by the sequence of (poor, time of the pitch) vector that characterizes the note feature.Wherein, " pitch is poor " refers to the difference with previous note pitch (basic frequency), rise to be in harmonious proportion unspecified person humming with what adapt to different melody, and is unit with semitone (Semitone); " time " refers to the zero hour of this note, and it has expressed the rhythm characteristic of melody.Why selecting the initial moment of note for use but not note duration length is used as characterizing the rhythm parameter of melody, is the rhythm characteristic of considering when the user hums melody.The user is when humming, and is general than being easier to hold going out now of note, and to the persistence length of each note sensitivity or differ greatly less.And ignore this cadence information of time if only adopt pitch information, and along with the expansion of music libraries scale, be matched to power will certainly glide thereupon, it is more and more lower to become.In addition, considering single is the limitation of parameter matching with the pitch information, and the present invention selects the vector sequence of (pitch is poor, the time) to describe the feature of melody.When the pattern match of the note characteristic sequence of importing the melody fragment and the standard note characteristic sequence of whole first melody melody, not to move matching way as traditional melody, promptly the sequence of notes that will import the melody fragment according to the note Moving Unit carries out moving continuously one by one the control of coupling along the sequence of notes direction of whole first melody melody, but serve as that the coupling Moving Unit realizes that great-leap-forward moves the control of coupling, thereby but for realizing that increasing substantially content-based music retrieval matching speed provides a kind of solution of property conscientiously with the feature note position of each the melody segmentation in the detected whole first melody melody.
Is that the input humming melody fragment of N is when mating retrieval in the inventive method to the note number, with the existing mobile one by one matching method of note unit of pressing, approximate melody matching method LAN compares as linear alignment, coupling at the melody melody segmentation that is M of a certain note number, adopt the LAN method to need | M-N|K+K/2 time, promptly needing at least | M-N|K time, the K here is that the segmentation of standard melody is to the fault-tolerant scope of input melody fragment sequence of notes length when adopting the LAN method, and its unit is the note number.And the method for the invention needs 2K time at most, wherein, is input melody slice header and K sublinear under melody melody segmentation head the aligns coupling of aliging the 1st time; And the 2nd time be in order to consider that the user hums fragment and has the note obscission in its initial part, and the head that will import the melody fragment prolongs melody melody segmentation sequence of notes direction and moves a note backward, promptly is equivalent to input melody slice header and K sublinear under the next note of melody melody segmentation head the aligns coupling of aliging.This shows that aspect the coupling retrieval of melody melody segmentation, the speed of matching method of the present invention is faster than LAN method at least | M-N|/2 times; And be the coupling of R and the whole first melody melody that contains L melody segmentation at a certain note number, adopt the LAN method to need (R-N) K+K/2 time, promptly to carry out (R-N) K coupling at least, adopt the method for the invention then to need 2LK time at most, so the speed of coupling descriptor index method of the present invention is faster than LAN method at least | R-N|/2L is doubly.
Therefore, owing to introduced that the melody segmentation detects and be segmented into the notion of mobile matching unit, make and to realize by detecting the note position that characterizes segmentation feature that significantly great-leap-forward moves coupling, thereby solve the problem of melody retrieval high speed with melody.The long more superiority that then can embody the method for the invention more of standard note sequence of whole first melody melody.
Melody segmentation position probing of the present invention and mobile coupling controlled step, for avoiding too much insignificant segmentation, earlier by eliminating the note characteristic sequence that to ignore quiet section (being equivalent to rest) step search criterion melody, if the note length of searching is then deleted this note less than a certain predefined quiet segment length threshold value, and this section is incorporated into the voiced segments of previous note.Because quiet segment length threshold value is generally established lowlyer (as quaver length), so this deletion does not almost influence the result when the coupling retrieval.After having deleted insignificant quiet section, then come each note in the standard melody is detected according to note category feature and note length feature thereof by the detection step of feature note.Feature note classification is divided into the location class note and the class note that stops, and all whether surpasses the feature note threshold value that sets in advance by its note length separately for this two classes note and determines whether this note is the segmentation feature note.Note characteristic sequence in the whole accurate melody of head between per two feature notes promptly is defined as a melody characteristics segmentation.
For location being set at of category feature note: its note length is if then be defined as the segmentation feature note with this note more than or equal to minim when long; For being set at of the category feature note that stops: its note length is if then be defined as the segmentation feature note with such note more than or equal to quaver when long.
Described melody segmentation is carried out based on the feature note, its according to and the feasibility that is used for singing search be according to music theory rudimentary knowledge and a large amount of melody melody carried out draw on the basis of statistical study and checking.At first, for the position divided of whole first melody, can consider that the rest position is a kind of feature note that characterizes the melody segmentation feature.But generally in theme, the number of rest is not a lot, only considers to adopt rest certainly will will cause the melody segmentation long as the cutting position of melody segmentation, thereby does not play real segmenting function.By note signature analysis, find that at melody especially in the song, the melody segmentation that links up mostly finishes at minim/whole note place to a large amount of melody melody.Why like this, this wherein has the notion of a sense-group.Sense-group is meant phrase, phrase or a short sentence with relatively independent meaning, and any interchange all is a notion and the exchanging of the combination of notion.The symbol of no notion or sound are insignificant, also just can not become language.In wirtiting any sentence, article all by key concept in some way structure form.Just as our daily interchange, all be to occur with complete one by one statement or phrase, and can be not suddenly do not exchange since the centre of a phrase, humming also is the same, the formula of being used to for people's thinking, taking certain lyrics " wind in 5,000 years and rain " is example, the humming people generally can not hum with " year wind and " such segmentation, because such segmentation does not constitute an independently sense-group, in melody, corresponding to the phrase one by one that mostly is of sense-group one by one, therebetween with long note or rest as mark at interval, to exchange thematic replacing for, at the intermittence in the performance, reality also is the transformation of sense-group.The basis of coupling so the present invention can be with these residing positions of feature note jumps during as match retrieval.
Melody segmentation coupling step of the present invention, for preventing when carrying out melody segmentation coupling, to be provided with based on the fault-tolerant mobile coupling controlling mechanism of the melody head of note Moving Unit owing to the user hums the mismatch that fragment exists the note obscission to cause itself and the segmentation of melody melody to be mated in its initial part.Promptly when a certain feature melody segmentation that will import melody fragment and melody melody is mated, carry out the linear extendible coupling of melody head alignment separately earlier, the melody head that to import the melody fragment then mates along the linear extendible that the mobile backward note of the sequence of notes direction of this melody melody segmentation carries out a melody head alignment again, till the moving range that the fault-tolerant mobile arrival of this melody head sets, and the highest matching score of getting in the fault-tolerant mobile coupling of its melody head is returned as the coupling output score of this melody segmentation.It is long that this fault-tolerant moving range is made as 2 notes in the present invention.
Melody segmentation coupling step of the present invention also can be made of following characteristics.Promptly contain fault-tolerantly and at the fault-tolerant controlled step of pitch similarity at the rhythm similarity, wherein, rhythm similarity fault-tolerant calculation step is the error on the rhythm of melody of farthest tolerating the user and being produced when the humming.For this reason, before a certain melody segmentation of input melody fragment and melody melody is mated, count the variation range K that N sets the matching length P of sequence of notes corresponding in the segmentation of melody melody, i.e. β with reference to the note of input melody fragment earlier
1N≤P≤β
2N (β
1＜β
2), K=(β
2-β
1) N=α N, wherein, the variation of the matching length P of melody melody segmentation sequence of notes only depends on that its tail position changes and its head position remains unchanged.Then, to import melody fragment sequence of notes with the melody melody sequence of notes matching length P that sets and make the temporal melody head linear stretching that aligns, the note that the sounding that aligns in certain error range is constantly approaching also calculates the rhythm similarity, and N the matching operation of K=α in this melody segmentation in the P matching range finishes.Tolerate that thus the user hums long or the too short and error that cause of note.When calculating rhythmic similarity, be to account for total ratio according to the note that aligns to calculate.Pitch similarity calculation procedure is at two melody sequence of notes after each linear alignment, calculates its similarity based on the pitch difference, and this is to calculate according to the ratio that the approaching melody fragment of pitch accounts for the melody total length.In described method step, rhythm matching after each linear alignment and pitch difference matching result will comprehensively be the parameter index of a melody coupling with the relative weighting form of scope between 0-1, and return as the result of 1 total coupling of this melody segmentation with that time matching result of the top score in the K sublinear alignment coupling.
Melody coupling result for retrieval step display of the present invention also can be made of following characteristics.Promptly contain the melody characteristics curve comparative view generation step that to import the note characteristic sequence matching effect of any melody melody segmentation in melody fragment note characteristic sequence and the preceding N position for the demonstration that the user selectes.Described melody characteristics curve, its transverse axis are the zero-time of each note, and the longitudinal axis is pitch (semitone (Semitone) unit), and the input melody is represented with different colors respectively with the characteristic curve of melody melody.Feature note in the melody melody characteristics curve is represented with the mode of tinting by the note graphics area of its position.The user can browse the pronunciation characteristic of oneself and make the essential analysis assessment easily according to described melody characteristics curve comparative view.
After the present invention is applied to large-scale digital music data storehouse, can be improved significantly for the performance of music retrieval.The ART of implementing by the present invention based on the music retrieval system of humming input reduces about 2/3 than tradition according to the searching system that note unit moves the coupling retrieval mode, system performance improve the effect highly significant.
Description of drawings
Fig. 1 is the function constitution map that is applicable to the music rhythm great-leap-forward high speed coalignment of embodiments of the invention.
Fig. 2 is the process flow diagram that is applicable to the music rhythm great-leap-forward high speed matching treatment of embodiments of the invention.
Fig. 3 A-Fig. 3 B is that the music rhythm great-leap-forward that is applicable to embodiments of the invention mates the synoptic diagram of crossing over moving method at a high speed.
Fig. 4 A-Fig. 4 B is the figure that is used for illustrating the music rhythm great-leap-forward matching process melody melody segmentation feature trace routine of being undertaken by embodiments of the invention.
Fig. 5 A-Fig. 5 E is used for illustrating that the music rhythm great-leap-forward matching process great-leap-forward that is undertaken by embodiments of the invention moves the figure of matcher.
Fig. 6 is the figure that is used to illustrate the music rhythm great-leap-forward matching result written-out program that is undertaken by embodiments of the invention.
Embodiment
Provide following example below in conjunction with accompanying drawing with technical solution of the present invention.
Method based on the present invention's proposition, the music rhythm great-leap-forward that the embodiment of the invention adopts mates the indexing unit structure at a high speed, as shown in Figure 1, specifically extracting as a result storage part 1, standard melody characteristics storehouse and SoundBreeze Clubtunes library storage portion 2, standard melody characteristics by melody characteristics reads and mates retrieval control part 3, melody segmentation position probing and move coupling control part 4, melody segmentation matching part 5, melody matching result storage part 6, melody coupling result for retrieval display part 7 and form.
Melody characteristics extracts storage part 1 as a result and is used to make the melody characteristics result who extracts from the input melody signal suitably to be stored in workspace on the storer.Note characteristic sequence information among the melody characteristics result serves as reasons that the great-leap-forward of this digital music melody mates at a high speed that indexing unit is associated and the note feature vector sequence of the pretreatment system (not having diagram) of the prime that is provided with extracts the information (not having diagram) that flow process obtains.
Standard melody characteristics storehouse and SoundBreeze Clubtunes library storage portion 2 are used to make the SoundBreeze Clubtunes collection making in advance and weave and standard tone character data collection corresponding with it suitably to be stored in database community on the storer.
The standard melody characteristics reads and mates retrieval control part 3 is to be used for controlling from the control of reading and mating retrieval of melody characteristics template file to the note characteristic sequence.The standard melody characteristics read and mate retrieval control part 3 reading and saving melody characteristics extract storage part 1 as a result the input melody characteristics and with its successively with standard melody characteristics storehouse in melody characteristics mate the output area that the result of coupling then is stored on the storer by melody matching result storage part 6.When reading the melody characteristics template file, because system has used melody segmentation feature note, when moving coupling is not to be that unit carries out with the note, but carry out with the melody unit of being segmented into, this device can read the melody node label of note, because template file quantity is bigger, need coupling retrieval control part to carry out the control of each template file matching process.
Melody characteristics in the standard melody characteristics storehouse here is meant the melody basic feature information of taking out and the combination of note characteristic sequence from corresponding melody material data file.Wherein, the melody essential information comprises key bytes such as the note sum, average pitch, loudness of a sound, beat of melody ID, whole first melody and other); The note characteristic sequence then comprises the pitch feature (semitone is a unit) of each note, otonaga features (tick is a unit) and melody segmentation marker.This melody characteristics leaves in the data streaming file with the form that is called as the melody characteristics template file, and between relevant melody template file and SoundBreeze Clubtunes file, set up association, make also to obtain corresponding real music file when retrieving template file sequence number (ID).
Melody segmentation position probing and mobile coupling control part 4 are used for detecting at the note characteristic sequence of single melody characteristics template file the feature note of melody segmentation, when coupling moves, because system has used melody segmentation feature note, when moving coupling is not to be that unit carries out with the note, but carry out with the melody unit of being segmented into, this device can searching and detecting melody characteristics note in the note characteristic sequence, after finishing twice linear alignment algorithm of a melody segmentation, just navigate to the note place that next melody segmentation begins, move coupling control and can finish this work.
Melody segmentation matching part 5 is used to finish the matching process for a melody segmentation, be input melody slice header and K sublinear under melody melody segmentation head the aligns coupling of aliging the 1st time, consider that promptly the user imports note omission or interpolation are arranged, and the linear alignment coupling is carried out in the section of input and the segmentation of standard melody segmentation in fault-tolerant matching length variation range; And the 2nd time be in order to consider that the user hums fragment and has the note obscission in its initial part, and the head that will import the melody fragment prolongs melody melody segmentation sequence of notes direction and moves a note backward, promptly is equivalent to input melody slice header and K sublinear under the next note of melody melody segmentation head the aligns coupling of aliging.The variation range of the fault-tolerant matching length P of the standard note sequence of melody melody segmentation in the present invention is made as: 0.75N～1.33N, wherein N is the note number of input music segments.
Melody matching result storage part 6 is after mating with standard melody template file each time, the result can be stored in this storage part, the template file sequence number that comprises coupling, mate total similarity value, the position of melody segmentation in this document of optimum matching etc., and the matching result of N position is used for output and returns the user before keeping.N gets 10 among the present invention.)
After the result of calculation of melody coupling result for retrieval display part 7 N position before obtaining humming input and standard melody file coupling, the template file sequence number can be corresponded to real music file, for showing the result of preceding N position, the user comprises the filename of humming input, the rank of coupling, detailed music file name, mate total similarity value, the position of melody segmentation in music file of coupling etc.
Fig. 2 has provided the method step of each funtion part shown in Figure 1.Be that its corresponding relation is: the standard melody characteristics reads and mates the retrieval controlled step and reads by the standard melody characteristics and mate retrieval control part 3 and realize, and is read and mated retrieval control part 3 by the standard melody characteristics and judge whether to exist the melody melody; The melody segmentation position probing and the judgement of moving the coupling controlled step and whether having a melody segmentation are by melody segmentation position probing and move coupling control part 4 and melody segmentation matching part 5 is realized; Melody coupling result for retrieval step display is realized by melody coupling result for retrieval display part 7.In addition, the input node among Fig. 2 1., 2. with output node 6. respectively the melody characteristics in the corresponding diagram 1 extract storage part 1, standard melody characteristics storehouse and SoundBreeze Clubtunes library storage portion 2 and melody matching result storage part 6 as a result.
Below embodiment is described.
At the note characteristic sequence (music rhythm: " A Night At Moscow Suburb ") shown in Fig. 4 A that reads and mate the melody melody that the retrieval controlled step reads successively among Fig. 2 by the standard melody characteristics.
Transverse axis is represented the note zero-time among the figure, and its unit is the peculiar TICK of unit of expression note time in MIDI (the Musical Instrument DigitalInterface) file, and the longitudinal axis is then represented pitch, and its unit is a semitone.The tenth note, the 18 note and last note are the defined location of the present invention class melody segmentation feature note among the figure, and the 28 note then is the class melody segmentation feature note that stops.Therefore, as long as can detect these feature notes, then these feature notes can be decided to be the mark position of melody segmentation, i.e. the end note sign of a music rhythm segmentation by detection means.In Fig. 2, this part work is finished automatically by melody segmentation position probing and mobile coupling control part.Be that melody segmentation position probing and mobile coupling controlled step are undertaken by its note category feature and note length feature thereof the detection of the feature note of sign melody segmentation feature position.Characterize the feature note of segmentation feature, its classification is divided into the location category feature note and the category feature note that stops, and all whether surpasses the threshold value that sets in advance separately by its note length separately for this two classes note and determines whether this note is the segmentation feature note.In the present embodiment for location being set at of category feature note: its note length is if then be defined as the segmentation feature note with this note more than or equal to minim when long; For being set at of the category feature note that stops: its note length is if then be defined as the segmentation feature note with such note more than or equal to quaver when long.Note characteristic sequence in the whole accurate melody of head between per two feature notes promptly is defined as a melody segmentation.Shown in Fig. 4 B, just be detected 4 feature notes and be divided into four melody segmentations by this section melody melody shown in Fig. 4 A.
By melody segmentation position probing with move coupling control part 4 the time to the detecting of the category feature note that stops, to quiet section less than quaver, to implement the deletion union operation to it, promptly, find by statistics, length quiet section below quaver can be ignored, and can extend to the concluding time of previous note till the initial moment of pronunciation of next note this moment.
After detecting mark, can generate the music rhythm feature templates data file that has the melody segment information accordingly by melody segmentation position probing and mobile coupling control part 4 in the workspace on storer, as shown in the table.That is, the melody characteristics data layout of touching plate is made up of melodic information head and melody note characteristic sequence.Wherein first field of the data structure of each note feature is represented the pitch difference with last note, but first note then is the perfect pitch value this field record, though do not use the perfect pitch value when coupling, this first note recording mode is essential (as the pitch curve in the present embodiment) for calculating the pitch curve that reappears the description melody characteristics; Its second field record be the initial moment of pronunciation of this note of representing with tick, if common note, be exactly on the occasion of, if and this note is the segmentation feature note, so just before this otonaga features value, add negative sign, the 3rd field of feature note is optional mark position, is used for the actual tone period of recording feature note, equally also is to represent with tick.As shown in the table, to have only when note is the feature note of sign melody segmentation, the feature field of this note just has 3 fields, rather than all notes all have 3 fields.
Therefore, by the standard melody characteristics read and mate the retrieval controlled step to melody segmentation coupling step when the standard melody to input melody fragment and whole first melody mates, just can the sequence of notes of input melody fragment be carried out moving continuously coupling along the sequence of notes direction of whole first melody melody if adopt this standard melody characteristics template that has the melody segment information not according to the note Moving Unit, but can according in every first melody melody in advance the melody of mark be segmented into the coupling Moving Unit and carry out great-leap-forward and move coupling, with reference to Fig. 3, wherein, Fig. 3 A is the explanation that the great-leap-forward under M＞N situation moves the coupling control mode, and the great-leap-forward under Fig. 3 BM=N situation moves the explanation of coupling control mode.Thereby for realizing that but increasing substantially content-based music retrieval matching speed provides a kind of solution of property conscientiously.
| Music ID | The note number | Average pitch | Loudness of a | Beat | Note | 1 | | | |||||||||||
| 3309 | 18 | 69.39 | 127 | 2/4 | 69 | 0 | 3 | 48 | 4 | 96 | |||||||||
| | | | | Note 8 | Note 9 | Note 10 | Note 11 | ||||||||||||
| -4 | 144 | 2 | 192 | -2 | 288 | -1 | 336 | 5 | 432 | -2 | 528 | -5 | -624 | 192 | 3 | 816 | |||
| Note 12 | Note 13 | Note 14 | | Note 16 | Note 17 | Note 18 | |||||||||||||
| 4 | 864 | 3 | 912 | 0 | 960 | 2 | 1008 | -2 | 1104 | -2 | 1152 | -1 | -1200 | 192 | |||||
Be provided with a humming input melody segmentation to be retrieved, the note characteristic sequence of the input melody segmentation of its process feature extraction pre-service (not shown) is shown in Fig. 5 A.The description of note feature also can be read and be mated retrieval control part 3 with Fig. 4 A (for convenience of explanation, all using perfect pitch rather than poor with the pitch of last note with the longitudinal axis in figure below) and be read in by the standard melody characteristics in the note characteristic sequence of this input melody segmentation.Fig. 5 B has provided the note characteristic sequence of first section melody segmentation of the standard melody shown in Fig. 4 B, and the judgement of this melody segmentation and intercepting processing are read by the standard melody characteristics and mated retrieval controlled step to melody segmentation position probing and mobile coupling controlled step and carry out.
The great-leap-forward of narrating below of the present invention mates retrieval at a high speed, will describe linear alignment algorithm and great-leap-forward high speed matching algorithm of the present invention move mode different in matching process in detail.By the melody segmentation position probing among Fig. 2 with move the coupling controlled step and will retrieve reference position earlier and be located at first note place earlier, the first note that is about to the to import melody note characteristic sequence linear alignment of carrying out the melody segmentation of aliging with first note of first melody segmentation of the musical features sequence of standard melody is mated.Omit phenomenon in order to prevent unmusical professional person contingent first note when humming input, algorithm of the present invention is after linear alignment is mated once, move a note backward, carry out the coupling second time again, this moment, the linear alignment algorithm with no melody segmentation mark there is no different.And after twice coupling of end, it is that the 3rd note carries out moving coupling as reference position next time that the linear alignment algorithm can continue to move backward a note, the inventive method then can move backward to the first note of next melody segmentation automatically, the feature note that is detected melody segmentation is an end position, and a note is the reference position that moves coupling next time backward again.Fig. 5 C has provided the reference position of coupling for the third time of the linear alignment algorithm under the no melody segmentation mark, and Fig. 5 D has then provided the reference position of coupling for the third time of great-leap-forward high speed match retrieval algorithm among the present invention.By Fig. 5 C and Fig. 5 D as seen, the inventive method is better than traditional linear alignment algorithm greatly on the coupling translational speed.
Move in the process of coupling in above-mentioned input melody fragment, the coupling of itself and the segmentation of standard melody is to be finished by the coupling of the melody segmentation among Fig. 2 step.And by melody segmentation position probing with move the coupling controlled step and judge that whether the melody segmentation mark of whole first melody exists, until input melody fragment and a complete melody mate finish till.
When in Fig. 2, carrying out the linear alignment coupling of melody segmentation by melody segmentation coupling step, be that note characteristic sequence that note characteristic sequence matching length P with predefined melody melody will import the melody fragment is made the temporal melody head linear stretching that aligns, and in certain error range the constantly approaching note of alignment sounding and calculate the rhythm similarity, N the matching operation of K=α in this melody segmentation in the P matching range finish (Fig. 3 reference).The variation range (K) of the fault-tolerant matching length P of rhythm of the standard note sequence of melody melody segmentation is made as in embodiments of the present invention: 0.75N≤P≤1.33N (K=0.58N), wherein, N is the note number of input melody fragment.Note number as the input section among the embodiment 1 is 8, if the length of standard melody segmentation is less than 6 so, can think that so these two sections melody segmentations can't carry out stretching, promptly can skip this melody segmentation and moves to next segmentation and continue coupling.If any the note characteristic sequence (" EVA ") of standard melody, shown in Fig. 5 E.Can observe first melody segmentation and have only 3 notes, not meet the matching range of importing segment length herein,, skip and do not do coupling so can ignore.
When the great-leap-forward match retrieval of finishing the note characteristic sequence of a standard melody by melody segmentation position probing and mobile coupling controlled step in Fig. 2 was calculated, the calculating of its rhythmic similarity (rhythm_score) was to account for total ratio calculating according to the note that aligns.Because note has only write down zero-time, cause to determine the finish time of last note.In this example, the length of supposing last note is the mean value of front note.
And the calculating of its pitch similarity (pitch_score) is to assess according to the degree of closeness of two sounds and by the sim function.Considering that the user hums always has certain pitch error, so this function is configured to tolerable error within the specific limits.
With rhythm similarity and the addition of pitch similarity, just obtain an overall merit of two sections melody degrees of approximation at last.Rhythm_score and pitch_score are the highest to be 1.0, so the score of Pi Pei two ends melody is 2.0 fully.The input section is comparatively clear in this example, the pre-service result is better, the similarity that obtains when mating with " A Night At Moscow Suburb " is just than higher, the rhythm_score of this matching result and pitch_score are respectively 0.98 and 0.95, therefore total similarity has 1.93, and optimum matching appears at second melody segmentation of this head melody melody.Relative, just seem lower with the similarity of the note characteristic sequence of other standard melody, be respectively 0.32 and 0.45 as similarity with " EVA ", total similarity has only 0.77.
In Fig. 2, read and mate the retrieval controlled step and judge whether that all standard melody characteristics files all mate calculating by the standard melody characteristics, thereby obtain the matching result of current input melody segmentation for each standard melody characteristics file, and export to the user by melody coupling result for retrieval step display, best match position (initial note sequence number finishes the note sequence number) comprising the sequence number of this standard melody characteristics file, total similarity value, current file.
For the melody fragment of importing among the embodiment, can obtain the result that the tag file with standard melody file " A Night At Moscow Suburb " and " EVA " mates respectively:
| ID (A Night At Moscow Suburb) | Total similarity | Best match position |
| 3309 | 1.93 | 11，18 |
| ID (EVA) | Total similarity | Best match position |
| 101 | 0.77 | 9，18 |
In Fig. 2, sort to being kept at the input melody fragment in the melody matching result storage part 6 and the similarity of all standard melody characteristics files by melody coupling result for retrieval step display, the standard melody (ID) that N position score is high before obtaining is as the output result, N gets 10 as default value in this example, specifically can be set by the user.The output interface example as shown in Figure 6.The melody characteristics curve ratio in interface left side than synoptic diagram in, shown the melodic curve on the best match position of current input melody fragment and current standard feature file, click the arrow of top and can select different standard feature files.The tabulation on right side has shown the matching result of the preceding N position of current input melody fragment, comprises coupling rank, ID, matched position, total similarity, melody name, singer and country.
Claims (9)
1. the great-leap-forward high speed matching process of a digital music melody is characterized in that, may further comprise the steps:
(1) the standard melody characteristics reads and mates the retrieval controlled step: control is kept at the reading and mate retrieval process of note characteristic sequence that melody characteristics extracts the note characteristic sequence of the input melody of storage part as a result and is kept at the accurate melody of whole head in the standard melody characteristics storehouse;
(2) melody segmentation position probing and mobile coupling controlled step: detect the feature note that characterizes each melody segmentation position in the melody standard melody, the note characteristic sequence in the standard melody between per two feature notes promptly is defined as a melody characteristics segmentation; Simultaneously, the great-leap-forward move mode when being used for controlling input melody fragment and mating with each melody segmentation of melody standard melody, and first melody matching result is put in order in output;
(3) melody segmentation coupling step: a certain melody divides intersegmental pattern match in input melody fragment and the melody standard melody, and the matching result of melody segmentation will return described melody segmentation position probing and move the control process of coupling controlled step;
(4) melody mates the result for retrieval step display: show based on the result of the ultimate criterion melody coupling retrieval of input melody fragment, comprise the comparative view of the melody melody characteristics curve that mates with preceding N position and the text attribute of relevant melody thereof;
Melody characteristics is described by the pitch sequence poor, time vector that characterizes the note feature, and wherein, " pitch is poor " refers to the difference with previous note frequency, is in harmonious proportion unspecified person humming to adapt to rising of melody, and is unit with the semitone; " time " refers to the zero hour of this note, it has expressed the rhythm characteristic of melody, when the pattern match of the note characteristic sequence of importing the melody fragment and the note characteristic sequence of the standard melody of whole first melody, be segmented into the coupling Moving Unit with the melody in the detected whole first melody melody and carry out the control that great-leap-forward moves coupling.
2. the great-leap-forward high speed matching process of digital music melody as claimed in claim 1, it is characterized in that, described melody segmentation position probing and mobile coupling controlled step, be divided into elimination and can ignore quiet section and detect two steps of melody segmentation feature note and carry out, the note characteristic sequence that it is the search criterion melody that elimination can be ignored quiet section step, search note length less than the note of a certain predefined quiet segment length threshold value and with its deletion, the voiced segments of then this segment length being incorporated into previous note is about to previous note length and prolongs quiet a section of will have been deleted by decision; Detecting melody segmentation feature note is to carry out based on note category feature and note length feature thereof, feature note classification is divided into the location category feature and the category feature note that stops, all whether surpass the threshold value that sets in advance by its note length separately for this two classes note and determine whether this note is the segmentation feature note, the note characteristic sequence in the whole accurate melody of head between per two feature notes promptly is defined as a melody characteristics segmentation.
3. the great-leap-forward high speed matching process of digital music melody as claimed in claim 2 is characterized in that, for being set at of note class note: its note length is if then be defined as the segmentation feature note with this note more than or equal to 2 dieresis when long; For being set at of rest class note: its note length is if then be defined as the segmentation feature note with such note more than or equal to 8 dieresis when long.
4. the great-leap-forward high speed matching process of digital music melody as claimed in claim 1, it is characterized in that, in the described melody segmentation coupling step, when carrying out melody segmentation coupling, be provided with the fault-tolerant controlling mechanism that moves coupling based on note, promptly when a certain feature melody segmentation that will import melody fragment and melody melody is mated, carry out the coupling of melody head alignment separately earlier, the melody head that to import the melody fragment then once mates along the mobile backward note of the sequence of notes of melody feature melody segmentation again, till the set moving range of this fault-tolerant mobile arrival, and the highest matching score of getting in its fault-tolerant moving is returned as the coupling output score of this melody segmentation.
5. the great-leap-forward high speed matching process of digital music melody as claimed in claim 4 is characterized in that it is long that fault-tolerant moving range is made as 2 notes.
6. the great-leap-forward high speed matching process of digital music melody as claimed in claim 1, it is characterized in that, described melody segmentation coupling step, containing the rhythm similarity calculates and pitch similarity calculation procedure, when calculating rhythmic similarity, be to account for total ratio according to the note that aligns to calculate; Pitch similarity calculation procedure is at two melody sequence of notes after each linear alignment, calculates its similarity based on the pitch difference, and this is to calculate according to the ratio that the approaching melody fragment of pitch accounts for the melody total length; With rhythm similarity and the addition of pitch similarity, just obtain an overall merit of two sections melody degrees of approximation at last.
7. the great-leap-forward high speed matching process of digital music melody as claimed in claim 6, it is characterized in that, wherein, rhythm similarity calculation procedure is for farthest tolerating the error on the rhythm of user when humming, before a certain melody segmentation of input melody fragment and melody melody is mated, count the matching length scope of length setting melody melody segmentation sequence of notes earlier with reference to the note of input melody fragment, the matching length variation of melody melody segmentation sequence of notes only depends on that its tail position changes, to import melody fragment sequence of notes with the melody melody sequence of notes matching length of setting then and do temporal linear extendible conversion, the note that the sounding that aligns in error range is constantly approaching also calculates the rhythm similarity, and the matching operation in matching range finishes.
8. as the great-leap-forward high speed matching process of claim 6 or 7 described digital music melody, it is characterized in that, rhythm matching after each linear alignment and pitch difference matching result will comprehensively be the parameter index of a melody coupling with the relative weighting form of scope between 0-1, and return as the result of 1 total coupling of this melody segmentation with that time matching result of the top score in the K sublinear alignment coupling.
9. the great-leap-forward high speed matching process of digital music melody as claimed in claim 1, it is characterized in that, described melody coupling result for retrieval step display, contain the melody characteristics curve comparative view generation step that to import the note characteristic sequence matching effect of any melody melody segmentation in melody fragment note characteristic sequence and the preceding N position for the demonstration that the user selectes, characterize the characteristic curve of melody, its transverse axis is the zero-time of each note, the longitudinal axis is a pitch, the input melody is represented with different colors respectively with the characteristic curve of melody melody, feature note in the melody melody characteristics curve is represented with full coat look mode by the note graphics area of its position.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200510029494 CN1737796A (en) | 2005-09-08 | 2005-09-08 | Across type rapid matching method for digital music rhythm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200510029494 CN1737796A (en) | 2005-09-08 | 2005-09-08 | Across type rapid matching method for digital music rhythm |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN1737796A true CN1737796A (en) | 2006-02-22 |
Family
ID=36080593
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 200510029494 Pending CN1737796A (en) | 2005-09-08 | 2005-09-08 | Across type rapid matching method for digital music rhythm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN1737796A (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1953046B (en) * | 2006-09-26 | 2010-09-01 | 中山大学 | Automatic selection device and method for music based on humming sing |
| CN101916250A (en) * | 2010-04-12 | 2010-12-15 | 电子科技大学 | Humming-based music retrieving method |
| CN102820027A (en) * | 2012-06-21 | 2012-12-12 | 福建星网视易信息系统有限公司 | Accompaniment subtitle display system and method |
| CN101398827B (en) * | 2007-09-28 | 2013-01-23 | 三星电子株式会社 | Method and device for singing search |
| CN101488128B (en) * | 2008-01-14 | 2013-06-12 | 三星电子株式会社 | Music search method and system based on rhythm mark |
| CN103514182A (en) * | 2012-06-19 | 2014-01-15 | 国际商业机器公司 | Music searching method and device |
| CN105244021A (en) * | 2015-11-04 | 2016-01-13 | 厦门大学 | Method for converting singing melody to MIDI (Musical Instrument Digital Interface) melody |
| CN106547797A (en) * | 2015-09-23 | 2017-03-29 | 腾讯科技（深圳）有限公司 | Audio frequency generation method and device |
| CN107039024A (en) * | 2017-02-10 | 2017-08-11 | 美国元源股份有限公司 | Music data processing method and processing device |
| CN110265051A (en) * | 2019-06-04 | 2019-09-20 | 福建小知大数信息科技有限公司 | The sightsinging audio intelligent scoring modeling method of education is sung applied to root LeEco |
| CN111899762A (en) * | 2020-06-30 | 2020-11-06 | 平安科技（深圳）有限公司 | Melody similarity evaluation method and device, terminal equipment and storage medium |
| CN112331170A (en) * | 2020-10-28 | 2021-02-05 | 平安科技（深圳）有限公司 | Method, device and equipment for analyzing similarity of Buddha music melody and storage medium |
| CN113648651A (en) * | 2021-07-02 | 2021-11-16 | 北京金三惠科技有限公司 | Positioning method and system for music teaching foundation improvement game |
| CN111899762B (en) * | 2020-06-30 | 2024-05-31 | 平安科技（深圳）有限公司 | Melody similarity evaluation method and device, terminal equipment and storage medium |
-
2005
- 2005-09-08 CN CN 200510029494 patent/CN1737796A/en active Pending
Cited By (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1953046B (en) * | 2006-09-26 | 2010-09-01 | 中山大学 | Automatic selection device and method for music based on humming sing |
| CN101398827B (en) * | 2007-09-28 | 2013-01-23 | 三星电子株式会社 | Method and device for singing search |
| CN101488128B (en) * | 2008-01-14 | 2013-06-12 | 三星电子株式会社 | Music search method and system based on rhythm mark |
| CN101916250A (en) * | 2010-04-12 | 2010-12-15 | 电子科技大学 | Humming-based music retrieving method |
| CN101916250B (en) * | 2010-04-12 | 2011-10-19 | 电子科技大学 | Humming-based music retrieving method |
| CN103514182A (en) * | 2012-06-19 | 2014-01-15 | 国际商业机器公司 | Music searching method and device |
| CN103514182B (en) * | 2012-06-19 | 2017-05-17 | 国际商业机器公司 | Music searching method and device |
| CN102820027A (en) * | 2012-06-21 | 2012-12-12 | 福建星网视易信息系统有限公司 | Accompaniment subtitle display system and method |
| CN102820027B (en) * | 2012-06-21 | 2014-04-16 | 福建星网视易信息系统有限公司 | Accompaniment subtitle display system and method |
| WO2017050059A1 (en) * | 2015-09-23 | 2017-03-30 | 腾讯科技（深圳）有限公司 | Audio generation method, server, and storage medium |
| CN106547797A (en) * | 2015-09-23 | 2017-03-29 | 腾讯科技（深圳）有限公司 | Audio frequency generation method and device |
| US10261965B2 (en) | 2015-09-23 | 2019-04-16 | Tencent Technology (Shenzhen) Company Limited | Audio generation method, server, and storage medium |
| CN106547797B (en) * | 2015-09-23 | 2019-07-05 | 腾讯科技（深圳）有限公司 | Audio generation method and device |
| CN105244021A (en) * | 2015-11-04 | 2016-01-13 | 厦门大学 | Method for converting singing melody to MIDI (Musical Instrument Digital Interface) melody |
| CN105244021B (en) * | 2015-11-04 | 2019-02-12 | 厦门大学 | Conversion method of the humming melody to MIDI melody |
| CN107039024A (en) * | 2017-02-10 | 2017-08-11 | 美国元源股份有限公司 | Music data processing method and processing device |
| CN110265051A (en) * | 2019-06-04 | 2019-09-20 | 福建小知大数信息科技有限公司 | The sightsinging audio intelligent scoring modeling method of education is sung applied to root LeEco |
| CN111899762A (en) * | 2020-06-30 | 2020-11-06 | 平安科技（深圳）有限公司 | Melody similarity evaluation method and device, terminal equipment and storage medium |
| CN111899762B (en) * | 2020-06-30 | 2024-05-31 | 平安科技（深圳）有限公司 | Melody similarity evaluation method and device, terminal equipment and storage medium |
| CN112331170A (en) * | 2020-10-28 | 2021-02-05 | 平安科技（深圳）有限公司 | Method, device and equipment for analyzing similarity of Buddha music melody and storage medium |
| CN112331170B (en) * | 2020-10-28 | 2023-09-15 | 平安科技（深圳）有限公司 | Method, device, equipment and storage medium for analyzing Buddha music melody similarity |
| CN113648651A (en) * | 2021-07-02 | 2021-11-16 | 北京金三惠科技有限公司 | Positioning method and system for music teaching foundation improvement game |
| CN113648651B (en) * | 2021-07-02 | 2023-11-17 | 北京金三惠科技有限公司 | Positioning method and system for music teaching foundation promotion game |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN1737796A (en) | Across type rapid matching method for digital music rhythm | |
| CN100373383C (en) | Music rhythm sectionalized automatic marking method based on eigen-note | |
| Burred et al. | Hierarchical automatic audio signal classification | |
| Lidy et al. | On the suitability of state-of-the-art music information retrieval methods for analyzing, categorizing and accessing non-western and ethnic music collections | |
| CN100373382C (en) | Rhythm character indexed digital music data-base based on contents and generation system thereof | |
| JP2009508156A (en) | Music analysis | |
| US20060085188A1 (en) | Method for Segmenting Audio Signals | |
| Chai et al. | Music thumbnailing via structural analysis | |
| Aucouturier et al. | Finding repeating patterns in acoustic musical signals: Applications for audio thumbnailing | |
| CN102053998A (en) | Method and system device for retrieving songs based on voice modes | |
| CN100367279C (en) | Leap over type high speed matching device of numerical music melody | |
| Chai et al. | Structural analysis of musical signals for indexing and thumbnailing | |
| Blaß et al. | Content-based music retrieval and visualization system for ethnomusicological music archives | |
| West et al. | Incorporating machine-learning into music similarity estimation | |
| Li et al. | Music data mining: an introduction | |
| Su et al. | High-performance content-based music retrieval via automated navigation and semantic features | |
| Chen | Music sheet score recognition of Chinese Gong-che notation based on Deep Learning | |
| Chai | Structural analysis of musical signals via pattern matching | |
| Shenhuang et al. | Query by humming via multiscale transportation distance in random query occurrence context | |
| Vaglio et al. | The words remain the same: Cover detection with lyrics transcription | |
| Müller | New developments in music information retrieval | |
| Moelants et al. | Problems and opportunities of applying data-& audio-mining techniques to ethnic music | |
| Pinto et al. | A novel xml music information retrieval method using graph invariants | |
| Martin et al. | Indexing musical pieces using their major repetition | |
| Arentz et al. | Retrieving musical information based on rhythm and pitch correlations |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |