CN108536871A - Particle filter and the music Melody extraction method and device for limiting Dynamic Programming search range - Google Patents

Particle filter and the music Melody extraction method and device for limiting Dynamic Programming search range Download PDF

Info

Publication number
CN108536871A
CN108536871A CN201810390470.1A CN201810390470A CN108536871A CN 108536871 A CN108536871 A CN 108536871A CN 201810390470 A CN201810390470 A CN 201810390470A CN 108536871 A CN108536871 A CN 108536871A
Authority
CN
China
Prior art keywords
frame
particle filter
pitch
theme
dynamic programming
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201810390470.1A
Other languages
Chinese (zh)
Other versions
CN108536871B (en
Inventor
张维维
陈喆
殷福亮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Dalian Minzu University
Original Assignee
Dalian Nationalities University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dalian Nationalities University filed Critical Dalian Nationalities University
Priority to CN201810390470.1A priority Critical patent/CN108536871B/en
Publication of CN108536871A publication Critical patent/CN108536871A/en
Application granted granted Critical
Publication of CN108536871B publication Critical patent/CN108536871B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Auxiliary Devices For Music (AREA)

Abstract

Particle filter and the music Melody extraction method and device for limiting Dynamic Programming search range,Belong to music information retrieval field,In order to solve to take into account conspicuousness and flatness in music Melody extraction method,And the problem of reducing the influence accompanied by force in short-term,Main points are under particle filter frame,Throw particle at random,The likelihood function of particle filter is modeled as each harmonic amplitude square and product with the smooth sex factor of harmonic wave,With logistic fitting of distribution transition functions,By the prediction of particle filter and renewal equation, recurrence completes theme pitch sequence rough estimate frame by frame,Then it is smoothed,Treated for cunning of making even per the upper of frame pitch,Search range of the lower octave range as the theme pitch of the frame,And it executes dynamic programming algorithm in the range of limitation and is estimated with obtaining final theme pitch sequence.

Description

Particle filter and limit Dynamic Programming search range music Melody extraction method and Device
Technical field
The invention belongs to music information retrieval field, it is related to a kind of particle filter and limits the sound of Dynamic Programming search range Happy Melody extraction method and device.
Background technology
Melody extraction in music information retrieval field estimates theme pitch sequence from music signal, leads It will be according to being the conspicuousness of theme ingredient and sequential continuity in music.Traditional Melody extraction method is mostly theme It extracts PROBLEM DECOMPOSITION and builds two sub-problems at the estimation of multitone height and melody contours, lead to not take into account two characteristics simultaneously, The high estimation stages of multitone do not account for the high correlation of successive frame pitch.The it is proposeds such as Jo are modeled using Bayesian filter frame Melody extraction problem, and estimate parameter by particle filter method, but there is still a need for carry out before particle filter for this method Multitone height is estimated, and needs estimation (S.Jo, C.D.Yoo, and A.Doucet, Melody tracking with more parameter based on sequential Bayesian model,IEEE Journal of Selected Topics in Signal Processing,2011,5(6):1216–1227.)。
In addition, music signal is extremely complex, there are typical non-stationary essential characteristic, i.e., certain accompaniment tone work(strong in short-term Rate can exceed that theme ingredient, and theme pitch frequencies are distributed more widely, including several octave ranges, cause existing scheme to exist The wrong pitch and octave error for coming from accompaniment tone in Melody extraction are widely present.
Invention content
In order to solve to take into account conspicuousness and flatness in music Melody extraction method, and reduces and accompany by force in short-term The problem of influence, the present invention propose following scheme:
A kind of particle filter and the music Melody extraction method for limiting Dynamic Programming search range, in particle filter frame Under, throw particle at random, by the likelihood function of particle filter be modeled as each harmonic amplitude square and with the smooth sex factor of harmonic wave Product, with logistic fitting of distribution transition functions, by the prediction of particle filter and renewal equation, recurrence completes master frame by frame Melody pitch sequence rough estimate, is then smoothed it, the sliding upper and lower octave model that treated per frame pitch of making even The search range of the theme pitch as the frame is enclosed, and dynamic programming algorithm is executed finally to be led in the range of limitation Melody pitch sequence is estimated.
A kind of particle filter and the music Melody extraction device for limiting Dynamic Programming search range, are stored with a plurality of finger It enables, described instruction is loaded and executed suitable for processor:Under particle filter frame, particle is thrown at random, by the likelihood of particle filter Function modelling is each harmonic amplitude square and the product with the smooth sex factor of harmonic wave, with logistic fitting of distribution transition probabilities Function, by the prediction of particle filter and renewal equation, recurrence completes theme pitch sequence rough estimate frame by frame, then to its into Row smoothing processing, the search model for theme pitch of the sliding upper and lower octave range that treated per frame pitch as the frame of making even It encloses, and executes dynamic programming algorithm in the range of being defined and estimated with obtaining final theme pitch sequence.
The conspicuousness of melody and flatness are fused in particle filter frame by the present invention, and conspicuousness passes through in particle filter The each harmonic amplitude square and mode of likelihood function embody, and transition probability is then fitted in particle filter by flatness The mode of logistic distributions realizes that the frame is completed at the same time the estimation of multitone height and melody contours structure in this way, realizes significantly Property and flatness are taken into account.
The present invention obtains the rough estimate of theme pitch sequence using particle filter, is then made smoothing processing, uses To limit the theme pitch search range of Dynamic Programming, then final theme sound is obtained in the range of limiting by Dynamic Programming High sequence estimation, the strong accompaniment reduced outside search range influence;The present invention introduces penalty factor into one in the Dynamic Programming stage Step reduces the influence accompanied by force in short-term.
For the present invention in the likelihood function of particle filter, introducing the smooth sex factor of harmonic wave reduces octave error;The present invention The Dynamic Programming stage the notable angle value of each Frequency point is defined as each harmonic component in theme pitch search range The weighted sum of amplitude further reduced octave error.
Specific implementation mode
This disclosure relates to a kind of music Melody extraction method, for solve traditional Melody extraction method can not take into account it is aobvious Work property and the successional problem of sequential, at the same it is non-stationary caused from the wrong pitch of accompaniment tone for reducing music signal With octave error.
Its scheme is as follows:Under particle filter frame, particle is thrown at random, and the likelihood function of particle filter is modeled as each time The product of harmonic amplitude quadratic sum and the smooth sex factor of harmonic wave is filtered with logistic fitting of distribution transition functions by particle Recurrence completes theme pitch sequence rough estimate frame by frame for the prediction of wave and renewal equation, is then smoothed, takes to it Search range of the upper and lower octave range of every frame pitch after smoothing processing as the theme pitch of the frame, and in being defined In the range of execute dynamic programming algorithm and estimated with obtaining final theme pitch sequence.
Conspicuousness and sequential continuity are fused in same frame by this method;It is mutually tied with Dynamic Programming using particle filter The two benches scheme of conjunction limits the possible range of final theme pitch sequence, reduces in the estimation of theme pitch in short-term Strong accompaniment tone influences;Using the significance function of the smooth sex factor of harmonic wave in particle filter and weighted sum in Dynamic Programming, Reduce the octave error in the estimation of theme pitch.
This method specifically comprises the following steps:
S1, the normal Q transformation for calculating music signal;
Calculate the normal Q transform methods of music signal:Framing is carried out to the music signal of non-stationary, is then converted using normal Q The range value for calculating each Frequency point according to log series model has 36 Frequency points for pressing log series model per octave range.
S2, the rough estimate for obtaining the theme pitch sequence based on particle filter;
The method for obtaining the theme pitch sequence rough estimate based on particle filter:
Initialization:WhereinIt indicates The original pitch of i-th of particleIt obeys and is uniformly distributed in [110Hz, 1200Hz] range, Np=300 be population,Table Show i-th of particle weights.
Iterative solution:
(1) transition probability is pressed, predicts t frame melody pitch frequencies:
According to pitch transition probability statistic analysis result in actual music, pitch transition probability is modeled as logistic points Cloth, probability density function are:
Wherein x=f0,t/f0,t-1, f0,tFor t frame melody pitch frequencies, and μ=1.00003, s=0.0055045.
(2) particle weights are calculated
Wherein P and S has respectively represented the power harmony popin slip factor of i-th of particle of t frames, is respectively defined as:
Wherein Am,tFor i-th of particle pitch frequencies f of t frames0,tThe amplitude of corresponding m order harmonic components, H are maximum Overtone order.
(3) particle weights are normalized
(4) number of effective particles amount is calculated
IfReinitialize particle pitchJump to step (1);Otherwise, it jumps to Step (5), wherein fminAnd fmaxIt is the lower-frequency limit and the upper limit of theme pitch respectively.
(5) current melody pitch estimation is calculated
WhereinIt is the average value of each particle pitch frequencies of maximum weight.
(6) resampling is carried out according to normalized weight to particle, and the particle weights after all resamplings is all set to 1/ Np
(7) t=t+1 is enabled, if t≤Nfrm(NfrmFor the totalframes of audio), jump to step (1);Otherwise, terminate iteration mistake Journey.
After the step of terminating (1)-(7), byConstitute theme pitch sequence rough estimate.
S3, the smooth estimated sequence of theme pitch sequence is obtained;
The method for obtaining the smooth estimated sequence of theme pitch sequence:To the theme pitch in front and back 100 milliseconds of time Sequence rough estimate result is averaged, as the smooth estimated value of theme pitch sequence of present frame, the smooth estimation of each frame Value constitutes the smooth estimated sequence of theme pitch sequence.
S4, final theme pitch sequence estimation is obtained;
The method for obtaining final theme pitch sequence estimation:
(1) using the octave range up and down of every smooth estimated value of frame theme pitch sequence as the theme pitch of the frame Search range.
(2) the notable angle value S of each Frequency point in per frame theme pitch search range is calculatedt′(f):
Wherein Nh=10 be maximum overtone order, | Xt(hf) | it is the h subharmonic amplitudes of frequency f.
(3) to the S ' of every framet(f) it is normalized, obtains St(f):
(4) final theme pitch sequence value is obtained using dynamic programming algorithm, the primary condition of Dynamic Programming is:
D(1,ft,j)=S1(fj) (11)
Wherein D (t, ft,j) it is that t frames take ft,jAs the accumulated costs function of melody pitch, S1(fj) obtained by formula (10) It arrives, is value of the significance function in j-th of frequency point of the 1st frame.Then solve the recurrence formula of theme pitch sequence optimal solution For:
D(t,ft,j)=St(fj)+max{D(t-1,ft-1,k)-λd(ft-1,k,ft,j), t=1 ..., Nfrm (12)
Wherein ft,jFor the probable value of t frame theme pitches, ft-1,kFor t-1 frame theme pitch estimated values, d (ft-1,k,ft,j) it is ft-1,kAnd ft,jHalf beat, λ=0.05 be penalty factor.Wherein, ft,j、ft-1,kIn j,kUse difference Symbolic indication, be the t-1 frame theme pitch estimated values f due to when iterating to t framest-1,kIt has obtained and to determine Value, and the probable value f of t frame theme pitchest,jIt does not determine also, thus to show difference.
Continuous iterative formula (12) obtains final theme pitch sequence estimation.
The conspicuousness of melody and flatness are fused in particle filter frame by the present invention, and conspicuousness passes through in particle filter The each harmonic amplitude square and mode of likelihood function embody, and transition probability is then fitted in particle filter by flatness The mode of logistic distributions realizes that the frame is completed at the same time the estimation of multitone height and melody contours structure in this way.It is short in order to reduce When accompany by force influence, the present invention is solved by two approach:(1) it uses particle filter to obtain the rough of theme pitch sequence to estimate Meter, is then made smoothing processing, to limit the theme pitch search range of Dynamic Programming, then is being limited by Dynamic Programming In the range of obtain the estimation of final theme pitch sequence, reduce the strong accompaniment influence outside search range;(2) in Dynamic Programming Stage introduces penalty factor and is further reduced the influence accompanied by force in short-term.In order to reduce octave error, the present invention passes through two sides Formula solves:(1) in the likelihood function of particle filter, introducing the smooth sex factor of harmonic wave reduces octave error;(2) Dynamic Programming rank The notable angle value of each Frequency point is defined as the weighted sum of each harmonic component amplitude by section in theme pitch search range, Further decrease octave error.Therefore, conspicuousness and flatness have both been fused to particle filter this frame by the present invention simultaneously In, and reduce the influence accompanied by force in short-term and octave error.
Namely by said program, scheme has the advantages that described in the present embodiment:Conspicuousness and flatness are merged Into particle filter frame, the estimation of multitone height and melody contours structure can be completed at the same time;Pass through particle filter and Dynamic Programming phase In conjunction with scheme complete final theme pitch sequence estimation, and theme pitch search range is limited in Dynamic Programming, Penalty factor is also introduced, the influence accompanied by force in short-term is reduced;By introducing the smooth sex factor of harmonic wave in particle filter and moving The significance function of weighted sum reduces the octave error in the estimation of theme pitch in state planning.
Music Melody extraction method described in the present embodiment, disclosed in the Jo that is addressed in disclosure background technology etc. Scheme compares, and difference is as follows:
1. the likelihood function and transitional provavility density of the method for the method of the present invention and Jo etc. are all different, there is different sounds High significance describes method and transitional provavility density expression formula, especially probability density, and method of the invention is in actual music Take statistics analysis on the basis of audio, and pitch transition probability is fitted to logistic distributions, and the method for Jo etc. is with conventional Gaussian density be fitted transition probability;This method also introduces the smooth sex factor of harmonic wave in likelihood function and is missed to reduce octave Difference;
The theme pitch that the method for 2.Jo etc. first estimates to obtain each frame with multitone height is candidate, then uses particle filter Method obtains final theme pitch sequence.In certain theme notes ending, power is likely less than certain strong accompaniments Occasion, the method for Jo etc. still cannot exclude these wrong pitches, and accuracy is poor;
3. the flow of two methods is different, method of the invention is the Dynamic Programming again of first particle filter, and the method for Jo etc. is First multitone height estimates particle filter again;
4. there is this method less parameter, parameter adjustment to be easy;And the method for Jo etc. has more parameter, adjustment tired It is difficult.
By above-mentioned, music has very strong non-stationary property, and strong accompaniment tone may be more than theme within certain short time Power, for example, in certain note endings, possible theme power is accompanied less than some;In addition, octave error is also often deposited , in some note playing procedure, estimation pitch can because each harmonic amplitude proportional variation and different octaves it Between switch.Therefore, if the approximate range of theme can be determined in advance, the accompaniment pitch and octave error outside range will be filtered out.
The melody pitch that the method for the propositions such as Jo first estimates to obtain each frame with multitone height is candidate, then uses particle filter Method obtains final theme pitch sequence.This method cannot still solve the above problems.Method proposed by the present invention is used first Particle filter estimates melody pitch, equally also has the above problem, but the result of particle filter method estimation is most of all It is accurate, therefore make smoothing processing just to estimated sequence to weaken the influence of erroneous estimation, still can determine that melody pitch substantially Range recycles Dynamic Programming to deal in confined range, can filter out the pitch and eight of accompanying by force in short-term outside range Spend error.
The preferable specific implementation mode of the above, only the invention, but the protection domain of the invention is not It is confined to this, any one skilled in the art is in the technical scope that the invention discloses, according to the present invention The technical solution of creation and its inventive concept are subject to equivalent substitution or change, should all cover the invention protection domain it It is interior.

Claims (2)

1. a kind of particle filter and the music Melody extraction method for limiting Dynamic Programming search range, it is characterised in that:In grain Under sub- filter frame, particle is thrown at random, and the likelihood function of particle filter is modeled as each harmonic amplitude square and is put down with harmonic wave The product of the slip factor, with logistic fitting of distribution transition functions, frame by frame by the prediction of particle filter and renewal equation Recurrence completes theme pitch sequence rough estimate, is then smoothed to it, and treated for cunning of making even per frame pitch Upper and lower search range of the octave range as the theme pitch of the frame, and calculated in executing Dynamic Programming in confined range Method is estimated with obtaining final theme pitch sequence.
2. a kind of particle filter and the music Melody extraction device for limiting Dynamic Programming search range, which is characterized in that storage There are a plurality of instruction, described instruction to load and execute suitable for processor:Under particle filter frame, particle is thrown at random, and particle is filtered The likelihood function of wave is modeled as each harmonic amplitude square and the product with the smooth sex factor of harmonic wave, with logistic fittings of distribution Transition function, by the prediction of particle filter and renewal equation, recurrence completes theme pitch sequence rough estimate frame by frame, so It is smoothed afterwards, the sliding theme pitch that treated per the upper and lower octave range of frame pitch as the frame of making even Search range, and estimated with obtaining final theme pitch sequence in executing dynamic programming algorithm in confined range.
CN201810390470.1A 2018-04-27 2018-04-27 Music main melody extraction method and device based on particle filtering and limited dynamic programming search range Expired - Fee Related CN108536871B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810390470.1A CN108536871B (en) 2018-04-27 2018-04-27 Music main melody extraction method and device based on particle filtering and limited dynamic programming search range

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810390470.1A CN108536871B (en) 2018-04-27 2018-04-27 Music main melody extraction method and device based on particle filtering and limited dynamic programming search range

Publications (2)

Publication Number Publication Date
CN108536871A true CN108536871A (en) 2018-09-14
CN108536871B CN108536871B (en) 2022-03-04

Family

ID=63478395

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810390470.1A Expired - Fee Related CN108536871B (en) 2018-04-27 2018-04-27 Music main melody extraction method and device based on particle filtering and limited dynamic programming search range

Country Status (1)

Country Link
CN (1) CN108536871B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111709964A (en) * 2020-06-22 2020-09-25 重庆理工大学 PCBA target edge detection method

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0726559A2 (en) * 1995-02-13 1996-08-14 Yamaha Corporation Audio signal processor selectively deriving harmony part from polyphonic parts
CN1866763A (en) * 2006-05-11 2006-11-22 上海交通大学 Channel estimation method based on particle filtering
US20070076902A1 (en) * 2005-09-30 2007-04-05 Aaron Master Method and Apparatus for Removing or Isolating Voice or Instruments on Stereo Recordings
CN105957538A (en) * 2016-05-09 2016-09-21 大连民族大学 Polyphony melody extraction method based on significance
CN105989609A (en) * 2015-01-27 2016-10-05 国网吉林省电力有限公司延边供电公司 Evolutionary resampling particle filter method and visual target tracking method
CN106528706A (en) * 2016-10-26 2017-03-22 北京邮电大学 Music retrieval method and apparatus
CN106973364A (en) * 2017-05-09 2017-07-21 电子科技大学 A kind of distribution of polynomial parameters likelihood function batch estimated data fusion method
US20180107735A1 (en) * 2015-09-23 2018-04-19 Tencent Technology (Shenzhen) Company Limited Audio generation method, server, and storage medium

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0726559A2 (en) * 1995-02-13 1996-08-14 Yamaha Corporation Audio signal processor selectively deriving harmony part from polyphonic parts
US20070076902A1 (en) * 2005-09-30 2007-04-05 Aaron Master Method and Apparatus for Removing or Isolating Voice or Instruments on Stereo Recordings
CN1866763A (en) * 2006-05-11 2006-11-22 上海交通大学 Channel estimation method based on particle filtering
CN105989609A (en) * 2015-01-27 2016-10-05 国网吉林省电力有限公司延边供电公司 Evolutionary resampling particle filter method and visual target tracking method
US20180107735A1 (en) * 2015-09-23 2018-04-19 Tencent Technology (Shenzhen) Company Limited Audio generation method, server, and storage medium
CN105957538A (en) * 2016-05-09 2016-09-21 大连民族大学 Polyphony melody extraction method based on significance
CN106528706A (en) * 2016-10-26 2017-03-22 北京邮电大学 Music retrieval method and apparatus
CN106973364A (en) * 2017-05-09 2017-07-21 电子科技大学 A kind of distribution of polynomial parameters likelihood function batch estimated data fusion method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
LIMING SONG 等: "Melody Extraction for Vocal Polyphonic Music Based on Bayesian Framework", 《2014 TENTH INTERNATIONAL CONFERENCE ON INTELLIGENT INFORMATION HIDING AND MULTIMEDIA SIGNAL PROCESSING》 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111709964A (en) * 2020-06-22 2020-09-25 重庆理工大学 PCBA target edge detection method
CN111709964B (en) * 2020-06-22 2023-04-25 重庆理工大学 PCBA target edge detection method

Also Published As

Publication number Publication date
CN108536871B (en) 2022-03-04

Similar Documents

Publication Publication Date Title
CN109524020B (en) Speech enhancement processing method
Sun et al. Speech enhancement under low SNR conditions via noise estimation using sparse and low-rank NMF with Kullback–Leibler divergence
RU2459254C2 (en) Method for computer-aided training of one or more neural networks
US20190120719A1 (en) Anomalous sound detection training apparatus, acoustic feature extraction apparatus, anomalous sound sampling apparatus, and methods and programs for the same
US20100246842A1 (en) Information processing apparatus, melody line extraction method, bass line extraction method, and program
JP4268386B2 (en) How to classify songs that contain multiple sounds
CN106971740A (en) Probability and the sound enhancement method of phase estimation are had based on voice
Sigtia et al. RNN-based music language models for improving automatic music transcription
Yoshii et al. A nonparametric Bayesian multipitch analyzer based on infinite latent harmonic allocation
US7076433B2 (en) Apparatus and program for separating a desired sound from a mixed input sound
JP2008058755A (en) Sound analysis apparatus and program
CN111242005B (en) Heart sound classification method based on improved wolf's swarm optimization support vector machine
Rank Application of Bayesian trained RBF networks to nonlinear time-series modeling
CN111931983A (en) Precipitation prediction method and system
CN108735231A (en) Theme pitch sequence method of estimation
CN108647596A (en) Music Melody extraction method
Zhang et al. Multi-task autoencoder for noise-robust speech recognition
CN105046044A (en) Non-stationary wind speed prediction method based on optimum wavelet-packet transformation
Liu et al. A novel pitch extraction based on jointly trained deep BLSTM recurrent neural networks with bottleneck features
CN108536871A (en) Particle filter and the music Melody extraction method and device for limiting Dynamic Programming search range
CN108628982A (en) The rough estimate method of theme pitch sequence based on particle filter
CN108595648A (en) Music Melody extraction system
CN117609702A (en) Pipeline leakage acoustic emission signal denoising method, system, equipment and medium
Gupta et al. Towards Controllable Audio Texture Morphing
Jadda et al. Speech enhancement via adaptive Wiener filtering and optimized deep learning framework

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20220304

CF01 Termination of patent right due to non-payment of annual fee