CN103412886A - Music melody matching method based on pitch sequence - Google Patents
Music melody matching method based on pitch sequence Download PDFInfo
- Publication number
- CN103412886A CN103412886A CN2013103037249A CN201310303724A CN103412886A CN 103412886 A CN103412886 A CN 103412886A CN 2013103037249 A CN2013103037249 A CN 2013103037249A CN 201310303724 A CN201310303724 A CN 201310303724A CN 103412886 A CN103412886 A CN 103412886A
- Authority
- CN
- China
- Prior art keywords
- sequence
- matching
- algorithm
- pitch sequence
- melody
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Auxiliary Devices For Music (AREA)
Abstract
The invention provides a method utilizing a pitch sequence of multipart music to conduct melody matching. The method includes the steps of conducting normalization operation on the pitch sequence, conducting melody matching on the inquiry pitch sequence and all target sequences in a music library through a SPRING algorithm, ranking the results, and returning to the K target sequences similar to the inquiry sequence. In the matching process, a GPU is used for accelerating the SPRING algorithm to shorten search time. The pitch sequence is used as a characteristic for matching, so that matching accuracy is improved. The GPU is utilized to conduct melody extraction, so that extraction time is decreased from a second level to a millisecond level. The melody matching is conducted through the SPRING algorithm, therefore, algorithm complexity is greatly reduced, the GPU is used for accelerating the SPRING algorithm, and the scale of the music library which can be processed in real time is greatly increased. The method has wide practical value and application prospect in business application and scientific research fields.
Description
Technical field
The present invention proposes a kind of novel music rhythm matching process, be specifically related to a kind of method that pitch sequence that utilizes counterpoint carries out melody matching.The multimedia information retrieval field that belongs to computer science.
Background technology
The purpose of music rhythm coupling is given a piece of music melody, finds the music rhythm similar to it, and then returns to similar music.The most frequently used melody characteristics is the pitch sequence of music now, is also fundamental frequency (F0) sequence of music.Melody matching based on pitch sequence often adopts the Time Series Matching algorithm, and the most frequently used is the subsequence matching algorithm.Conventional subsequence matching algorithm is based on the algorithm of moving window: the pitch sequence that inquire about and all target sequences mate, in the process of coupling, search sequence can be mated as reference position from the optional position of any one target sequence, also namely in the moving window mode, sequentially mates all positions from the reference position of target sequence backward.In all positions, minimum matching distance is as the matching distance of search sequence and this target sequence.After the target sequence with all has mated, according to the most similar K the target sequence of k nearest neighbor rules selection as a result of.
The best applications scene of music rhythm matching algorithm is humming search (Query by Humming), the user hums one section music rhythm, then from this music of search music libraries, this way of search makes the user in the situation that forget that title of the song also can search for song, so good melody matching algorithm will bring very large using value.But, because the complexity of sliding window algorithm and matching algorithm is too high, make it be difficult to be applied in fairly large music libraries.
The method of current solution is mainly to reduce algorithm complex by revising sliding window algorithm.At first amended method utilizes a low lower bound algorithm of complexity to carry out the beta pruning judgement to current location.But this algorithm is unstable, the beta pruning limited ability is in lower bound algorithm, and point that can not beta pruning still adopts complete matching algorithm to carry out sequences match, makes it still be difficult to be applied in extensive music libraries.
We abandon using sliding window algorithm, change and use new matching algorithm-SPRING algorithm to carry out subsequence matching, and algorithm complex is greatly reduced.In addition due to the outstanding floating-point operation ability of GPU, the memory bandwidth of super large and relatively low price, and the improvement of corresponding programming language (CUDA), make GPU bring into play increasing effect in the general-purpose computations field.Utilize GPU the SPRING algorithm to be accelerated and can significantly increase the music libraries scale of real-time processing.
Summary of the invention
The objective of the invention is to utilize the pitch sequence of counterpoint that a kind of melody matching method of music is provided.At first the method carries out the normalization operation to the pitch sequence of music, utilizes afterwards the SPRING algorithm that all target sequences in this sequence and music libraries are carried out to melody matching.In the process of coupling, we utilize GPU to accelerate the SPRING algorithm, to reduce retrieval time.Finally return to the K similar to a search sequence target sequence.
According to an aspect of the present invention, provide a kind of pitch sequence that utilizes counterpoint to carry out the method for melody matching, it is characterized in that: the method concrete steps are as follows:
Step 1: on CPU, pitch sequence is carried out to the normalization operation;
Step 2: utilize the SPRING algorithm that all target sequences in search sequence and music libraries are carried out to melody matching on GPU;
Step 3: on CPU, matching result is sorted, and return to the K similar to a search sequence target sequence.
The accompanying drawing explanation
Fig. 1 is the overall flow figure of music rhythm coupling.
Fig. 2 is the normalized schematic diagram of pitch.
Fig. 3 is the parallel schematic diagram of SPRING algorithm data level.
Embodiment
Below in conjunction with accompanying drawing, technical scheme of the present invention is described.
As shown in Figure 1, the present invention comprises three steps altogether, and wherein first step is moved on CPU, and second step moves on GPU, and an algorithm of the 3rd step is transplanted to again the upper operation of CPU.Concrete step is as follows:
(1) step S1: original pitch sequence is carried out to normalization, make follow-up matching algorithm more accurate;
(2) step S2: adopt the SPRING algorithm to mate normalized pitch sequence and storehouse sequence;
(3) step S3: matching distance is sorted, return to the K the most similar to a search sequence sequence.
Wherein, step S1, the method for normalizing that we adopt is based on the method for normalizing of average: nonzero value in pitch sequence is asked for to average, then all pitches are obtained to normalized pitch sequence divided by average.The normalization formula is:
Wherein Mean_ignore0 means not comprise the average of all pitches of neutral element.
Wherein, step S2, the SPRING algorithm of employing are a kind of subsequence matching algorithms, and algorithm, based on the DTW algorithm of dynamic programming, still utilizes a distance matrix to replace a plurality of distance matrixs to calculate minimal matching spans, and algorithmic formula is as follows:
D(Q，S
s，e)＝min(D(M，t))t＝1，...，N
D(i，0)＝∞；D(0，j)＝0；
SP(i，0)＝0；SP(0，j)＝j；where1≤i≤M，1≤j≤N.
Wherein, D means the matching distance of calculating, and SP means to obtain the melody reference position of corresponding matching distance, dist mean between two pitches apart from computing formula, we adopt Euclidean distance.
In addition, the GPU for the SPRING algorithm accelerates the mode that adopts task level and data level to carry out simultaneously.Task level is parallel is to process different target sequence couplings between different block.Data level is parallel is all threads coupling of target sequence of co-treatment and search sequence in same block, and the strategy of the parallel employing of data level is the clinodiagonal strategy.
Wherein, step S3, the sort method that we adopt is quicksort, K value in actual conditions is 10.
Advantage of the present invention and beneficial effect comprise:
Music rhythm based on pitch sequence matching algorithm proposed by the invention, it compared with prior art, its main advantage is: (1) adopts the pitch sequence can the represent music feature as coupling, and the accuracy rate of melody matching is promoted; (2) pitch sequence that will inquire about and all target sequences in music libraries are carried out to normalization, make coupling more accurate; (3) adopt SPRING algorithm rather than sliding window algorithm as the melody matching algorithm of core, algorithm complex is greatly reduced; (4) adopt speed-up ratio that GPU accelerates the SPRING algorithm near 40 times, make manageable music libraries scale significantly increase; (5) programming is simple, only need to utilize the expansion language CUDA of c language just can programme to GPU, and introduction and development progress are all fast than FPGA.
For making the purpose, technical solutions and advantages of the present invention express clearlyer, the present invention is further described in more detail below in conjunction with drawings and the specific embodiments.
Main thought of the present invention is to utilize the pitch sequence of counterpoint to carry out melody matching, accelerates it can be promoted in the large-scale application scene by adopting new matching algorithm and GPU.
Below with an example, describe, for same snatch of music, obtain respectively the humming fragment of male voice and female voice, length 10s, form are wav.
At first from user's humming fragment, obtain pitch sequence.Different people has different tone colors, hums identical fragment pitch sequence and also can differ greatly (Fig. 2), and while in order to eliminate, mating, the impact of tone color, then carry out normalization to original pitch sequence.Sequence after normalization just can with storehouse in target sequence mate.In order to improve matching speed, we adopt the SPRING algorithm, utilize simultaneously GPU to accelerate the SPRING algorithm.Because the SPRING algorithm is a dynamic programming algorithm, have very strong data dependency, we eliminate data dependence by the mode (Fig. 3) of clinodiagonal and then on GPU, accelerate.Finally result is sorted and returns to a most similar K pitch sequence.
Below describe respectively each step in detail:
(1) step S1: pitch sequence is carried out to normalization:
Pitch sequence is the floating number sequence of a length hundreds of, and each point means a pitch, and whole sequence forms a time series, and null value wherein means that this does not have sound constantly.At first normalization ask for average to the pitch value of non-zero, then each pitch obtained to normalized pitch sequence divided by average.
(2) step S2: melody matching:
A) from the CPU space, copy the pitch sequence of storehouse sequence and humming to the GPU space;
B) mode walked abreast by task level between block is processed different target sequence couplings, so just can have a lot of target sequences with search sequence, to mate simultaneously;
C) all threads in block are by the coupling of target sequence of data level parallel processing and search sequence.If carry out the SPRING algorithm according to the mode on Fig. 3 left side, have serious data dependency, cause the thread in block can not executed in parallel.If computation sequence is revised as to clinodiagonal mode (Fig. 3 right side), the element on every clinodiagonal does not have dependence when calculating.We allow the interior all threads of block process a clinodiagonal at every turn, and each thread is that in block, number of threads is processed the element on clinodiagonal according to stride.When each thread completes calculating, by other threads in this block of synchronous wait, complete calculating, then the common all elements that calculates next clinodiagonal.
(3) step S3: sort result:
The distance that search sequence and all target sequence coupling are drawn is delivered to the CPU space, then by quicksort, obtains a most similar K sequence, is also the K song list that user's most probable is hummed.
It should be noted last that: above embodiment is the unrestricted technical scheme of the present invention in order to explanation only, although with reference to above-described embodiment, the present invention is had been described in detail, those of ordinary skill in the art is to be understood that: still can modify or be equal to replacement the present invention, and not breaking away from any modification or partial replacement of the spirit and scope of the present invention, it all should be encompassed in the middle of claim scope of the present invention.
Claims (4)
1. a pitch sequence that utilizes counterpoint carries out the method for melody matching, and it is characterized in that: the method concrete steps are as follows:
Step 1: on CPU, pitch sequence is carried out to the normalization operation;
Step 2: utilize the SPRING algorithm that all target sequences in search sequence and music libraries are carried out to melody matching on GPU;
Step 3: on CPU, matching result is sorted, and return to the K similar to a search sequence target sequence.
2. a kind of pitch sequence that utilizes counterpoint according to claim 1 carries out the method for melody matching, it is characterized in that: operating on CPU described in step 1 completes, and the method for normalizing of employing is divided by the average method for normalizing that does not comprise neutral element.
3. a kind of pitch sequence that utilizes counterpoint according to claim 1 carries out the method for melody matching, it is characterized in that: described in step 2, operating in that GPU is upper to have walked abreast, is the core of melody matching.Paralleling tactic is that task level is parallel and data level is parallel shared, between block, adopts task level parallel, and the inner data level of clinodiagonal mode that adopts of block walks abreast.
4. a kind of pitch sequence that utilizes counterpoint according to claim 1 carries out the method for melody matching, and it is characterized in that: operating on CPU described in step 3 completes, and utilizes quicksort to sort to result, then returns to K music sequence.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2013103037249A CN103412886A (en) | 2013-07-18 | 2013-07-18 | Music melody matching method based on pitch sequence |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2013103037249A CN103412886A (en) | 2013-07-18 | 2013-07-18 | Music melody matching method based on pitch sequence |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103412886A true CN103412886A (en) | 2013-11-27 |
Family
ID=49605898
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2013103037249A Pending CN103412886A (en) | 2013-07-18 | 2013-07-18 | Music melody matching method based on pitch sequence |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103412886A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103823867B (en) * | 2014-02-26 | 2017-02-15 | 深圳大学 | Humming type music retrieval method and system based on note modeling |
| CN106663110A (en) * | 2014-06-29 | 2017-05-10 | 谷歌公司 | Derivation of probabilistic score for audio sequence alignment |
| CN106778477A (en) * | 2016-11-21 | 2017-05-31 | 深圳市酷浪云计算有限公司 | Tennis racket action identification method and device |
| CN108735231A (en) * | 2018-04-27 | 2018-11-02 | 大连民族大学 | Theme pitch sequence method of estimation |
| CN109273025A (en) * | 2018-11-02 | 2019-01-25 | 中国地质大学（武汉） | A kind of China National Pentatonic emotion identification method and system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102214356A (en) * | 2011-06-07 | 2011-10-12 | 内蒙古大学 | NVIDIA graphics processing unit (GPU) platform-based best neighborhood matching (BNM) parallel image recovering method |
| CN102521281A (en) * | 2011-11-25 | 2012-06-27 | 北京师范大学 | Humming computer music searching method based on longest matching subsequence algorithm |
-
2013
- 2013-07-18 CN CN2013103037249A patent/CN103412886A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102214356A (en) * | 2011-06-07 | 2011-10-12 | 内蒙古大学 | NVIDIA graphics processing unit (GPU) platform-based best neighborhood matching (BNM) parallel image recovering method |
| CN102521281A (en) * | 2011-11-25 | 2012-06-27 | 北京师范大学 | Humming computer music searching method based on longest matching subsequence algorithm |
Non-Patent Citations (3)
| Title |
|---|
| 周勇 等: "非规则流中高维数据流典型相关性分析并行计算方法", 《软件学报》, vol. 23, no. 5, 31 May 2012 (2012-05-31) * |
| 赵明 等: "大系统块对角优分解", 《控制与决策》, vol. 15, no. 5, 30 September 2000 (2000-09-30), pages 585 - 588 * |
| 高成锴: "基于约束动态时间弯曲距离的时间序列相似性匹配", 《中国优秀硕士学位论文全文数据库 信息科技辑》, no. 5, 15 May 2011 (2011-05-15) * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103823867B (en) * | 2014-02-26 | 2017-02-15 | 深圳大学 | Humming type music retrieval method and system based on note modeling |
| CN106663110A (en) * | 2014-06-29 | 2017-05-10 | 谷歌公司 | Derivation of probabilistic score for audio sequence alignment |
| CN106663110B (en) * | 2014-06-29 | 2020-09-15 | 谷歌有限责任公司 | Derivation of probability scores for audio sequence alignment |
| CN106778477A (en) * | 2016-11-21 | 2017-05-31 | 深圳市酷浪云计算有限公司 | Tennis racket action identification method and device |
| CN106778477B (en) * | 2016-11-21 | 2020-04-03 | 深圳市酷浪云计算有限公司 | Tennis racket action recognition method and device |
| US10737158B2 (en) | 2016-11-21 | 2020-08-11 | Shenzhen Coollang Cloud Computing Co., Ltd | Method and device for recognizing movement of tennis racket |
| CN108735231A (en) * | 2018-04-27 | 2018-11-02 | 大连民族大学 | Theme pitch sequence method of estimation |
| CN108735231B (en) * | 2018-04-27 | 2021-11-12 | 大连民族大学 | Method for estimating pitch sequence of main melody |
| CN109273025A (en) * | 2018-11-02 | 2019-01-25 | 中国地质大学（武汉） | A kind of China National Pentatonic emotion identification method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103412886A (en) | Music melody matching method based on pitch sequence | |
| CN112784130B (en) | Twin network model training and measuring method, device, medium and equipment | |
| Lu et al. | Deep ranking: Triplet MatchNet for music metric learning | |
| CN110110858B (en) | Automatic machine learning method based on reinforcement learning | |
| WO2016184159A1 (en) | Big data calculation method and system | |
| CN106529207A (en) | Method for predicting protein bound with ribonucleic acid | |
| Chang et al. | Audio cover song identification using convolutional neural network | |
| Salman et al. | Fast k-means algorithm clustering | |
| CN105718486A (en) | Online query by humming method and system | |
| CN113128582B (en) | Matrix Profile-based time sequence variable-length die body mining method | |
| CN103247286B (en) | A kind of melody of counterpoint extracting method based on GPU | |
| Bangoria Bhoomi | Enhanced K-Means clustering algorithm to reduce time complexity for numeric values | |
| CN110909785A (en) | Multitask Triplet loss function learning method based on semantic hierarchy | |
| CN106844652A (en) | A kind of product know-how air navigation aid of knowledge based map | |
| CN103605493A (en) | Parallel sorting learning method and system based on graphics processing unit | |
| Vrskova et al. | Hyperparameter tuning of ConvLSTM network models | |
| Lee et al. | Coupled representation learning for domains, intents and slots in spoken language understanding | |
| CN103116646B (en) | A kind of music emotion recognition method based on cloud gene expression programming | |
| Jeon et al. | Efficient search of music pitch contours using wavelet transforms and segmented dynamic time warping | |
| Maršık et al. | Towards a harmonic complexity of musical pieces | |
| Patel et al. | Raag detection in music using supervised machine learning approach | |
| CN105404892B (en) | Penicillin fermentation process stage division method based on orderly fuzzy C-means clustering | |
| Wang et al. | Shapelet classification algorithm based on efficient subsequence matching | |
| Rechkalov et al. | An approach to data mining inside PostgreSQL based on parallel implementation of UDFs | |
| CN104598591A (en) | Model element matching method for type attribute graph model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20131127 |
|
| WD01 | Invention patent application deemed withdrawn after publication |