JP2008209579A - Sound analysis apparatus and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve estimation accuracy of a fundamental frequency as a whole, even when a waveform of an input sound signal is unstable in an attack period. <P>SOLUTION: In an attack detection 1a, attack detection is performed by dividing an input sound signal into frames. In an estimation 41 of a probability density function of a fundamental frequency, the probability density function of the fundamental frequency is estimated for each frame by an Expectation-Multiplication (EM) algorithm. In a continuous tracking 42 of the fundamental frequency by a multi-agent model, the fundamental frequency is estimated from the probability function. At that time, control for switching over calculation modes of the process 41 and 42 for obtaining the fundamental frequency is performed, depending on whether or not, a frame to be processed is in the attack period. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention is directed to a musical sound signal including a singing voice and a plurality of types of instrument sounds recorded on a commercially available CD (compact disc) or the like. The present invention relates to a sound analysis apparatus and a program for estimating the

  It is very difficult to estimate the pitch of a specific sound source from a monaural sound signal in which the sounds of many sound sources are mixed. One of the essential reasons why it is difficult to estimate the pitch of a mixed sound is that, in the time-frequency domain, the frequency component of one sound overlaps with the frequency component of another sound that is playing simultaneously. For example, in typical popular music played on singing voices, keyboard instruments (piano, etc.), guitars, bass guitars, drums, etc., part of the harmonic structure of the singing voice that plays the melody (especially the fundamental frequency component) It frequently overlaps with the harmonic component of the guitar, the higher harmonic component of the bass guitar, the noise component included in the sound of the snare drum, and the like. For this reason, a method of locally tracking each frequency component does not function stably for complex mixed sounds. There is a technique for estimating a harmonic structure on the assumption that a fundamental frequency component exists, but such a technique has a major drawback that it cannot handle a missing fundamental phenomenon. Furthermore, if the frequency components of other sounds that are playing at the same time overlap with the fundamental frequency components, they will not function effectively.

  For the above reasons, there has been a technique for estimating the pitch of a single sound or an acoustic signal that contains a single sound with aperiodic noise, but it is recorded on a commercially available CD. There was no technique for estimating the pitch of a mixed sound signal such as an acoustic signal.

  However, in recent years, a technique for appropriately estimating the pitch of each sound included in the mixed sound by using a statistical method has been proposed. This is the technique of Patent Document 1.

  In the technique of this Patent Document 1, a frequency component belonging to a band considered to be a melody sound and a frequency component belonging to a band considered to be a bass sound are separately extracted from an input acoustic signal by a BPF, and each of those bands is extracted. Based on the frequency components, the fundamental frequencies of the melody sound and the bass sound are estimated.

  More specifically, in the technique of Patent Document 1, a sound model having a probability distribution corresponding to the harmonic structure of a sound is prepared, and each frequency component of the band of the melody sound and each frequency component of the band of the base sound are It is considered to be a mixed distribution obtained by weighting and adding each sound model corresponding to various fundamental frequencies. Then, the weight value of each sound model is estimated using an EM (Expectation-Maximization) algorithm.

This EM algorithm is an iterative algorithm for performing maximum likelihood estimation on a probability model including hidden variables, and a local optimum solution can be obtained. Here, since the probability distribution having the largest weight value can be regarded as the most dominant harmonic structure at that time, the fundamental frequency in the dominant harmonic structure can be obtained as the pitch. . Since this method does not depend on the presence of the fundamental frequency component, the missing fundamental phenomenon can be appropriately handled, and the most dominant harmonic structure can be obtained without depending on the presence of the fundamental frequency component.
Japanese Patent No. 3413634 Patent No. 3660599

  By the way, in the conventional sound analysis apparatus described above, the input acoustic signal is divided into frames having a fixed time length, and the EM algorithm is executed in units of frames to estimate the fundamental frequency of the sound of the sound source. In addition, in each frame, when the weight value for the sound model of various fundamental frequencies is updated and optimized by repeating the EM algorithm, the final value of the weight value estimated in the previous frame is taken over, and this is initialized. The EM algorithm in the frame was executed. However, generally, a musical tone tends to have an unstable waveform in an attack section. For this reason, when the conventional sound analysis apparatus performs the estimation process of the fundamental frequency of the input acoustic signal in the attack period where the waveform is unstable, the estimation process falls into an unstable state, and erroneous estimation of the fundamental frequency continues. There was a problem that it was easy to occur.

  The present invention has been made in view of the circumstances described above, and is a sound that can improve the estimation accuracy of the fundamental frequency as a whole even when the waveform of the input acoustic signal becomes unstable in the attack section. An object is to provide an analysis apparatus and a sound analysis program.

  The present invention divides an input sound signal into frames having a predetermined time length, and an attack detection means for determining whether or not the input sound signal is a signal in an attack section for each frame, and for each frame, the sound of the sound source. Using a sound model that is a probability density function having a structure corresponding to a harmonic structure, a mixed distribution is formed by weighting and adding a plurality of sound models corresponding to various fundamental frequencies. Probability density that sequentially updates and optimizes the weight values for each sound model so as to have a frequency component distribution, and estimates the weight values of each optimized sound model as a probability density function of the fundamental frequency of the sound of the sound source Function estimation means, and fundamental frequency estimation means for estimating and outputting the fundamental frequency of the sound of one or more sound sources included in the input acoustic signal based on the probability density function of the fundamental frequency The calculation for estimating the probability density function of the fundamental frequency in the probability density function estimating means or the basic in the fundamental frequency estimating means depending on whether or not the frame to be processed by the probability density function estimating means is in the attack section There is provided a sound analysis apparatus comprising a calculation control means for switching a calculation mode for frequency estimation, and a computer program for causing a computer to function as the sound analysis apparatus.

  According to this invention, the calculation for estimating the probability density function of the fundamental frequency in the probability density function estimating means or the basic depending on whether the frame to be processed by the probability density function estimating means is in the attack section or not Since the calculation control means for switching the calculation mode for estimation of the fundamental frequency in the frequency estimation means is provided, an appropriate calculation mode suitable for improving the estimation accuracy of the fundamental frequency as a whole is selected, and the probability density The function estimation means or the fundamental frequency estimation means can be executed, and for example, the accuracy of the fundamental frequency as the entire song can be improved.

  Embodiments of the present invention will be described below with reference to the drawings.

<Overall configuration>
FIG. 1 is a diagram showing the processing contents of a sound analysis program according to an embodiment of the present invention. This sound analysis program includes an acoustic signal such as a sound collection function for acquiring an acoustic signal from the natural world, a playback function for reproducing an acoustic signal of music from a recording medium such as a CD, or a communication function for acquiring an acoustic signal of music via a network. The program is installed and executed on a computer such as a personal computer having an acquisition function. The computer that executes the sound analysis program according to the present embodiment functions as the sound analysis device according to the present embodiment.

The sound analysis program according to the present embodiment estimates the pitch of a certain sound source in a monaural music sound signal acquired through the sound signal acquisition function. As the most important example, the melody line and the bass line are estimated here. The melody is a sequence of single notes that can be heard more prominently than others, and the bass is the sequence of the lowest single note in the ensemble. The temporal changes are called the melody line Dm (t) and the base line Db (t), respectively. Assuming that the fundamental frequency F0 at time t is Fi (t) (i = m, b) and the amplitude is Ai (t), these are expressed as follows.

  As a means for obtaining the melody line Dm (t) and the base line Db (t) from the input sound signal, the sound analysis program includes instantaneous frequency calculation 1, attack detection 1a, frequency component candidate extraction 2, frequency band , Restriction 3, melody line estimation 4 a, and baseline estimation 4 b. Each process of the melody line estimation 4a and the baseline estimation 4b includes a fundamental frequency probability density function estimation 41 and a fundamental frequency sequential tracking 42 using a multi-agent model. In this embodiment, the processing contents of instantaneous frequency calculation 1, frequency component candidate extraction 2, and frequency band restriction 3 are basically the same as those disclosed in the above-mentioned Patent Document 1. The feature of the present embodiment is that the attack detection 1a is provided, and the processing contents of the melody line estimation 4a and the baseline estimation 4b that are controlled based on the processing result of the attack detection 1a. Hereinafter, the content of each process which comprises the sound analysis program by this embodiment is demonstrated.

<Instantaneous frequency calculation 1>
This instantaneous frequency calculation 1, attack detection 1 a, frequency component candidate extraction 2, frequency band restriction 3, melody line estimation 4 a, and baseline estimation 4 b estimation of probability density function of fundamental frequency 41 Each of these processes is executed in units of frames of a certain time length obtained by dividing the acoustic signal on the time axis. In the following, time t is specifically a frame number. In the calculation of instantaneous frequency 1, the input acoustic signal is given to a filter bank composed of a plurality of BPFs, and the instantaneous frequency (Flanagan, JL and Golden, RM: Phase Vocoder) that is the time derivative of the phase for each output signal of each BPF , The BellSystem
Technical J., Vol. 45, pp.1493-1509 (1966)). Here, the above-described Flanagan method is used, the short-time Fourier transform (STFT) output is interpreted as the filter bank output, and the instantaneous frequency is efficiently calculated. When the STFT using the window function h (t) for the input acoustic signal x (t) is given by the equations (3) and (4), the instantaneous frequency λ (ω, t) can be obtained by the equation (5). .

  Here, h (t) is a window function that gives the localization of the time frequency (for example, a time window created by convolving a second-order cardinal B-spline function with a Gaussian function that gives the optimum localization of the time frequency. Such).

A wavelet transform may be used to calculate this instantaneous frequency. Here, the STFT is used to reduce the amount of calculation. However, if only a single STFT is used, the time resolution and frequency resolution in a certain frequency band are deteriorated. Therefore, multi-rate filter banks (Vetterli, M .: A Theory of Multirate Filter Banks, IEEE Trans. On ASSP,
Vol.ASSP-35, No.3, pp. 356-372 (1987)), and a reasonable time-frequency resolution is obtained under the restriction that it can be executed in real time.

<Attack detection 1a>
In this processing, it is determined whether or not each frame obtained by dividing the input acoustic signal on the time axis is a frame within the attack section of the input acoustic signal, and whether or not the frame is within the attack section for each frame. The information shown is handed over to the melody line estimation 4a and the baseline estimation 4b. There are various known methods for determining whether or not a frame is in the attack interval of the input acoustic signal. For example, as disclosed in Patent Document 2, each frame is divided into a plurality of shorter time lengths. It is possible to determine whether or not the frame is in the attack section by analyzing the fluctuation of the energy of the acoustic signal in the plurality of analysis sections.

これらの周波数成分のパワーは、Ψ_f ^(t)の各周波数におけるＳＴＦＴパワースペクトルの値として得られるため、周波数成分のパワー分布関数Ψ_p ^(t)(ω)を次のように定義することができる。

<Frequency component candidate extraction 2>
In this process, candidate frequency components are extracted based on the mapping from the center frequency of the filter to its instantaneous frequency (Charpentier, FJ: Pitch detection using the short-termphase
spectrum, Proc. of ICASSP 86, pp. 113-116 (1986)). Consider a mapping from the center frequency ω of an STFT filter to the instantaneous frequency λ (ω, t) of its output. Then, if there is a frequency component of frequency ψ, ψ is located at the fixed point of this mapping, and the value of the instantaneous frequency around it is almost constant. That is, the instantaneous frequency Ψ _f ^(t) of all frequency components can be extracted by the following equation.

Since the power of these frequency components is obtained as the value of the STFT power spectrum at each frequency of ψ _f ^(t) , the power distribution function ψ _p ^(t) (ω) of the frequency component can be defined as follows. it can.

<Frequency band restriction 3>
In this process, the frequency band is limited by weighting the extracted frequency components. Here, two types of BPF are prepared for the melody line and the base line. The melody line BPF can pass most of the main fundamental wave components and harmonic components of a typical melody line, and cuts off a frequency band in which duplication near the fundamental frequency frequently occurs to some extent. On the other hand, the BPF for a bass line can pass many of the main fundamental frequency components and harmonic components of a typical bass line, and the frequency band in which the other performance parts are dominant over the bass line. To some extent.

平均律の半音は１００ｃｅｎｔに、１オクターブは１２００ｃｅｎｔに相当する。 In the present embodiment, the logarithmic scale frequency is expressed in units of cents (originally a scale representing pitch difference (pitch)), and the frequency fHz expressed in Hz is expressed as cents as follows: Convert to fcent.

A semitone of equal temperament corresponds to 100 cent, and one octave corresponds to 1200 cent.

When the frequency response of the BPF at the frequency x cent is BPFi (x) (i = m, b) and the power distribution function of the frequency component is ψ ′ _p ^(t) (x), the frequency component that has passed through the BPF is BPFi. (X) ψ ′ _p ^(t) (x). However, Ψ ′ _p ^(t) (x) is the same function as Ψ _p ^(t) (ω) except that the frequency axis is represented by cent. Here, as a preparation for the next stage, a probability density function p _Ψ ^(t) (x) of a frequency component that has passed through the BPF is defined.

Here, Pow ^(t) is the total power of the frequency components that have passed through the BPF as shown in the following equation.

<Estimation 41 of probability density function of fundamental frequency>
In this process, a probability density function of a fundamental frequency representing how much each harmonic structure is relatively dominant with respect to a frequency component candidate that has passed through the BPF is obtained. Therefore, in the present embodiment, the probability distribution function p _Ψ ^(t) (x) of the frequency component is a mixed distribution model (weighted sum model) of probability distribution (sound model) that models a sound having a harmonic structure. ). When the probability density function of a sound model having a fundamental frequency F is p (x | F), the mixed distribution model p (x; θ (t)) can be defined by the following equation.

Here, Fhi and Fli are the upper and lower limits of the allowable fundamental frequency, and are determined by the pass band of the BPF. W ^(t) (F) is a weight of the sound model p (x | F) that satisfies the following expression.

Since it is impossible to assume the number of sound sources in advance for a real-world acoustic signal such as a CD, it is important to model in consideration of the possibility of all fundamental frequencies at the same time. If the model parameter θ (t) can be estimated as if the observed frequency component p _Ψ ^(t) (x) was generated from the model p (x; θ (t)), then p _Ψ ^(t) (x) is It can be considered that the sound model has been decomposed into individual sound models. As shown in the following equation, the weight w ^(t) (F) for the sound model of each fundamental frequency F is represented by the probability density function p _FO ^{(t ) (F)} and it can be interpreted.

That is, in a mixture distribution, the more dominant a sound model p (x | F) becomes (that is, the larger w ^(t) (F)), the more the model of p _FO (t) (F) The probability of the fundamental frequency F increases.

From the above, it can be seen that the problem of estimating the parameter θ (t) of the model p (x; θ (t)) should be solved when the probability density function p _Ψ ^(t) (x) is observed. The maximum likelihood estimator of θ (t) is obtained by maximizing the average log likelihood defined by the following equation.

Since this maximization problem is difficult to solve analytically, θ (t) is estimated using the aforementioned EM (Expectation-Maximization) algorithm. The EM algorithm performs maximum likelihood estimation from incomplete observation data (in this case, p Ψ (t) (x)) by repeatedly applying an E step (expectation step) and an M step (maximization step) alternately. Iterative algorithm for In this embodiment, by repeating the EM algorithm, the probability density function p Ψ (t) (x) of the frequency component that has passed through the BPF is converted into a plurality of sound models p (x | F) corresponding to various basic frequencies F. Is the most likely weighting parameter θ (t) (= {w (t) (F) | Fli ≦ F ≦ Fhi}, where each iteration of the EM algorithm , Parameter θ (t) (= {w (t) (F) | Fli ≦ F ≦ Fhi}), the old parameter estimate θ old (t) (= {w old (t) (F) | Fli ≦ F ≦ Fhi}) is updated to obtain a new (more likely) parameter estimate θ new (t) (= {w new (t) (F) | Fli ≦ F ≦ Fhi}). value θ old ( Recurrence formula for obtaining the new parameter estimate theta new new (t) from) is as follows. Note that the process derives the recurrence formula is because it is described in detail in Patent Document 1, it is referred to there I want.

FIG. 2 shows a process in which the weight parameter θ ^(t) (= {w ^(t) (F) | Fli ≦ F ≦ Fhi} for the sound model p (x | F) is updated by the EM algorithm in this embodiment. 2 shows an example in which a sound model having four frequency components is used in order to simplify the illustration.

In the EM algorithm in the present embodiment, based on the sound model p (x | F) corresponding to each fundamental frequency F and the weight value w _old ^(t) (F) for each current sound model, the frequency x Each time, a spectrum distribution ratio corresponding to each sound model is obtained.

As shown in the above equation (18), the spectrum distribution ratio (x | F) corresponding to each sound model p (x | F) at a certain frequency x is multiplied by the weight value w _old (F) ^(t) . The sum of the amplitude values w _old (F) ^(t) p (x | F) at the frequency x of each sound model p (x | F) (corresponding to the integral value of the denominator in the equation (18)) is obtained, and the sum _Is obtained by dividing each amplitude value w _old (F) ^(t) p (x | F). As is clear from equation (18), at each frequency x, each spectrum distribution ratio (x | F) corresponding to each sound model p (x | F) is normalized so that the sum is 1. It becomes.

In the present embodiment, at each frequency x, the function value of the probability density function p _Ψ ^(t) (x) at that frequency x is distributed according to the spectrum distribution ratio of each sound model p (x | F) at that frequency x. Then, for each sound model p (x | F), the function values of the probability density function p _Ψ ^(t) (x) distributed in this way are summed up, and a share of each sound model p (x | F) is obtained. And Then, the share of all sound models is summed, the share of each sound model is divided by the sum, and the share of each sound model p (x | F) normalized so that the sum is 1 is a new weighting parameter. Let w _new ^(t) (F). By repeating the above processing, the probability supported by the probability density function p _Ψ ^(t) (x) of the frequency component of the mixed sound among the sound models p (x | F) having different fundamental frequencies F is obtained. The weight parameter w ^(t) (F) for the higher one is gradually emphasized. As a result, the weight parameter w ^(t) (F) represents the probability density function of the fundamental frequency in the mixed sound that has passed through the BPF.

<Frequency tracking 42 of fundamental frequency by multi-agent model (processing as fundamental frequency estimation means)>
The sound analysis program according to the present embodiment estimates and outputs the fundamental frequency of the sound of one or more sound sources included in the input acoustic signal based on the probability density function of the fundamental frequency obtained as described above. Includes processing as estimation means. In this process, in order to determine the most dominant fundamental frequency Fi (t), the probability density function p _F0 ^(t) (F) of the fundamental frequency (Expression (17) ) Is obtained as the final estimated value obtained by iteratively calculating), and the frequency that maximizes) is obtained as the estimated value of the fundamental frequency.

  By the way, in the probability density function of the fundamental frequency, if a plurality of peaks corresponding to the fundamental frequency of the sound that is being played at the same time, those peaks may be selected one after another as the maximum value of the probability density function, The result obtained simply as described above may not be stable. Therefore, in the processing as the fundamental frequency estimation means in the present embodiment, in order to estimate the fundamental frequency from a global viewpoint, the trajectories of a plurality of peaks are tracked continuously in the time change of the probability density function of the fundamental frequency. Select the most dominant and stable fundamental frequency trajectory among them. In order to control such tracking process dynamically and flexibly, a multi-agent model is introduced.

  The multi-agent model is composed of one feature detector and a plurality of agents (see FIG. 3). The feature detector picks up the prominent peaks in the probability density function of the fundamental frequency. The agent basically follows the trajectory driven by those peaks. In other words, the multi-agent model is a general-purpose framework that temporally tracks features that stand out in the input. Specifically, the following processing is performed at each time.

(1) After the probability density function of the fundamental frequency is obtained, the feature detector detects a plurality of conspicuous peaks (peaks exceeding a threshold that dynamically changes according to the maximum peak). Then, for each conspicuous peak, the promising peak is evaluated by considering the total power Pow (t) of the frequency components. This is realized by regarding the current time as a time several frames ahead and prefetching and tracking the peak trajectory up to that time.

(2) When there is an agent already generated, the prominent peak is exclusively assigned to an agent having a locus close to it while interacting with each other. If multiple agents are candidates for assignment, assign them to the agent with the highest reliability.

(3) If the most promising and conspicuous peak has not yet been assigned, a new agent that tracks that peak is generated.

(4) Each agent has a cumulative penalty and disappears when it exceeds a certain threshold.

(5) An agent that has not been assigned a conspicuous peak receives a certain penalty, and tries to find the next peak to be tracked directly from the probability density function of the fundamental frequency. If the peak is not found, a penalty is applied. Otherwise, the penalty is reset.

(6) Each agent self-evaluates the reliability based on the weighted sum of the degree of how promising and conspicuous the peak assigned at present is and the reliability at the previous time.

(7) The fundamental frequency Fi (t) at time t is determined based on an agent having high reliability and a large total power along the track of the peak being tracked. The amplitude Ai (t) is determined by extracting a harmonic component or the like of the fundamental frequency Fi (t) from Ψ _p ^(t) (ω).

<< Improvements of this embodiment over the technique of Patent Document 1 >>
FIG. 4 shows the processing contents of the estimation 41 of the probability density function of the fundamental frequency in this embodiment. As shown in FIG. 4, in the estimation 41 of the probability density function of the fundamental frequency, the E step and M step 411 of the EM algorithm and the convergence determination 412 are repeated.

First, in the E step and the M step 411, according to the recurrence formula of the above equation (17), the probability density function of the fundamental frequency, that is, the weight value θ = θ _new ^{(t) of} the sound model corresponding to various fundamental frequencies F. (= {W _new ^(t) (F) | Fli ≦ F ≦ Fhi}).

Next, in the convergence determination 412, the weight value θ = θ _new ^{(t) of} the sound model corresponding to the various fundamental frequencies F obtained in the current E step and M step 411 and the previous weight value θ = θ _old ^{( t)} to determine whether or not the change in the weight value θ falls within the allowable range. When it is determined that the change in the weight value θ falls within the allowable range, the processing of the estimation 41 of the fundamental frequency probability density function is terminated, and the final value of the probability density function of the fundamental frequency is determined by the multi-agent model. Deliver to the temporal tracking 42 of the fundamental frequency.

  In the sound analysis program according to the present embodiment, based on the information output from the attack detection 1a, the fundamental frequency probability density function is estimated 41 or the fundamental frequency is continuously tracked by the multi-agent model which is a fundamental frequency estimation means. Calculation control means for controlling the calculation mode of the processing of 42 is provided. This is an improvement of the present embodiment over the technique of Patent Document 1. There are the following four modes for controlling the calculation mode of the process of tracking the fundamental frequency over time by the estimation 41 of the probability density function of the fundamental frequency or the multi-agent model as the fundamental frequency estimation means. The user can specify in which mode the calculation control means of the sound analysis program controls the calculation mode by operating an operation unit (not shown).

<<< First Aspect >>>
In the first aspect, when the frame to be processed is in the attack section, the sequential update of the weight value w ^(t) (F) in the frame starts from the predetermined initial value w _flat (F). As shown, if the calculation control of the estimation 41 of the probability density function of the fundamental frequency is performed and the frame to be processed is not in the attack section, the weight value w ^(t) (F) in the frame is sequentially updated. Is a mode in which arithmetic control is performed for the estimation 41 of the probability density function of the fundamental frequency so that the final value of the weight value w ^(t-1) (F) in the previous frame is started as an initial value.

Under the conventional technique, when the probability density function of the fundamental frequency is estimated by repeating the above recurrence formula (17) for each frame, the initial value of w ^(t) (F) is one before. The final value w ^(t-1) (F) of the weight value at time t-1 (the previous frame) is used. However, if the final state of the probability density function of the fundamental frequency in the previous frame is used as the initial value in this way, the estimation process becomes unstable when estimating the fundamental frequency of the frame in the attack period where the waveform is unstable. Therefore, it is easy to fall into an erroneous estimation. Therefore, in the first aspect, when the sequential updating of the weight value w ^(t) (F) is started for each frame, if the frame is other than the attack section, the final value of the weight value in the previous frame If w ^(t-1) (F) is an initial value and is in the attack period, for example, a predetermined initial value w _flat (F) having a flat weight value in the entire frequency band is used as the initial value. It is. More specifically, it is as follows.

First, in this embodiment, when the sequential update of the weight value w ^(t) (F) in each frame is started, as shown in FIG. 4, the final value w ^(t−1) of the weight value in the previous frame. A value obtained by multiplying (F) by a coefficient r and a value obtained by multiplying a predetermined initial value w _flat (F) by a coefficient 1-r are added, and the addition result is obtained as a weight value w ^(t) (F ) Is the initial value.

Then, as shown in FIG. 5, when processing a frame that does not belong to the attack period, the final value w ^(t-1) (F) of the weight value in the previous frame is set to the weight value w by setting the value of r to 1. ^(t) The initial value of (F) is used, and when a frame belonging to the attack section is processed, the value of r is set to 0, so that the predetermined initial value w _flat (F) is set to the weight value w ^(t) (F). The initial value is used.

As described above, according to this embodiment, during processing of the frame of the attack period, during successive updating of the weighting values w in the frame ^{(t) (F),} the final weight value w ^(t in the previous frame ^-1) (F) is not adopted as the initial value. Therefore, even if the waveform of the input acoustic signal becomes unstable during the attack period and the estimation process of the fundamental frequency becomes unstable, it is possible to avoid the erroneous estimation of the fundamental frequency from occurring continuously. The estimation accuracy of the fundamental frequency can be increased.

<<< Second Aspect >>>
When a musical instrument is played with a strong touch, the waveform may remain unstable for a while after the attack period of the acoustic signal ends. In such a case, even if the frame is after the attack period has ended, the final weight value w ^(t−1) (F) of the previous frame is used as the initial value, and the weight value w ^(t) ( When the sequential update of F) is performed, the weight value reaches a peak at the wrong fundamental frequency, and there is a possibility that the fundamental frequency is erroneously estimated.

Therefore, in the second mode, the initial value of the weight value w ^(t) (F) is controlled as follows. First, in the second mode, as in the first mode, when the sequential updating of the weight values w ^(t) (F) in each frame is started, as shown in FIG. The final value w ^(t-1) (F) multiplied by the coefficient r and the predetermined initial value w _flat (F) multiplied by the coefficient 1-r are added, and the result of the addition is added to the frame. The initial value of the weight value w ^(t) (F) at.

Then, as shown in FIG. 6, when processing the frame belonging to the attack section, the value of r is set to 0, so that the predetermined initial value w _flat (F) is the initial value of the weight value w ^(t) (F). And

After the attack period ends, the value of r is gradually increased from 0 to 1 each time the frame is switched. That is, in the second mode, during the processing of each frame that does not belong to the attack section, the sequential update of the weight value w ^(t) (F) in the frame is performed as the final value w ^(t−1) ( F) is controlled to control the estimation 41 of the probability density function of the fundamental frequency so that the weight value obtained by mixing the predetermined initial value w _flat (F) is started as an initial value, and the end of the immediately preceding attack section with the passage time becomes longer from the time, as the final value w of the weight values of the previous frame ^{(t-1) (F)} is stressed, the final value w of the weight values in the previous frame ^{(t-1) (} The mixing ratio between F) and a predetermined initial value w _flat (F) is controlled.

  According to this aspect, even if the waveform of the acoustic signal becomes unstable for a while after the attack period ends, it is possible to avoid the erroneous estimation of the fundamental frequency from occurring continuously, and to estimate the fundamental frequency. Accuracy can be increased.

<<< Third Aspect >>>
In the first and second aspects, the initial value of the weight value w ^(t) (F) for the sound model is controlled according to the information delivered from the attack detection 1a. On the other hand, in the third mode, as shown in FIG. 7, the E step is performed so that the normal sound model is used in the sections other than the attack section, and the sound model for the attack section is used in the attack section. And the sound model used for the M step 411 is switched.

  Here, the sound model for the attack section has a harmonic structure in which there are fewer peaks than the actual harmonic structure of the musical tone, and the amplitude value of each harmonic component changes along a gentle curve on the frequency axis. Use a sound model with By using such a sound model in the attack period, it is possible to estimate a fundamental frequency that is stable with respect to changes in the waveform of the input acoustic signal.

<<< fourth aspect >>>
In the first to third aspects, the estimation 41 of the probability density function of the fundamental frequency is the target of control based on the processing result of the attack detection 1a. On the other hand, in the fourth aspect, the continuous tracking 42 of the fundamental frequency by the multi-agent model which is the fundamental frequency estimation means is a target of control based on the processing result of the attack detection 1a. That is, in this fourth aspect, even if the calculation control means of the sound analysis program obtains the probability density function of the fundamental frequency obtained by the estimation 41 of the probability density function of the fundamental frequency in the attack period, the probability density function The fundamental frequency over time tracking 42 by the multi-agent model is controlled so as not to estimate and output the fundamental frequency based on. In other words, in the attack period in which an erroneous estimation of the fundamental frequency occurs, the estimation and output of the fundamental frequency are not performed, and the estimation accuracy is improved only for the fundamental frequency that is output.

<Other embodiments>
Although one embodiment of the present invention has been described above, the present invention may have other embodiments. For example:
(1) The calculation control means of the sound analysis program may be configured so that one of the first aspect or the second aspect and the third aspect can be used together.
(2) In the third aspect, not only the attack period but also a period from when the attack period ends until a predetermined time elapses, the sound frequency model for the attack period is used to calculate the probability density function of the fundamental frequency. Control 41 may be executed.
(3) In the fourth mode, estimation and output of the fundamental frequency may be stopped not only in the attack period but also in a period from when the attack period ends until a predetermined time elapses.

It is a figure which shows the processing content of the sound analysis program which is one Embodiment of this invention. It is the figure which illustrated the process in which the parameter of the weight with respect to a sound model is updated by EM algorithm in the embodiment. It is a figure which shows time-dependent tracking of the fundamental frequency by the multi agent model comprised by one feature detector and a some agent. It is a figure which shows the processing content of the estimation 41 of the probability density function of the fundamental frequency in the embodiment. It is a time chart which shows the 1st aspect of the calculation control performed by the calculation control means of the sound analysis program by the embodiment. It is a time chart which shows the 2nd aspect of the calculation control performed by the calculation control means. It is a time chart which shows the 3rd aspect of the calculation control performed by the calculation control means.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Calculation of instantaneous frequency, 1a ... Attack detection, 2 ... Extraction of frequency component candidates, 3 ... Frequency band limitation, 4a ... Melody line estimation, 4b ... Baseline estimation, 41 ... ... Estimation of probability density function of fundamental frequency, 42 ... Tracking fundamental frequency over time by multi-agent model.

Claims (6)

Attack detection means for dividing the input acoustic signal into frames of a predetermined time length, and determining whether the input acoustic signal is an attack section signal for each frame;
For each frame, using a sound model that is a probability density function having a structure corresponding to the harmonic structure of the sound of each sound source, a mixed distribution is formed by weighted addition of a plurality of sound models corresponding to various fundamental frequencies. The weight value for each sound model is sequentially updated and optimized so that the mixed distribution becomes the distribution of the frequency components of the input acoustic signal, and the optimized weight value of each sound model is set to the fundamental frequency of the sound of the sound source. A probability density function estimating means for estimating a probability density function of
Fundamental frequency estimation means for estimating and outputting a fundamental frequency of sound of one or a plurality of sound sources included in the input acoustic signal based on a probability density function of the fundamental frequency;
The calculation for estimating the probability density function of the fundamental frequency in the probability density function estimating means or the basic in the fundamental frequency estimating means depending on whether or not the frame to be processed by the probability density function estimating means is in the attack section And a calculation control means for switching a calculation mode for frequency estimation.   When the frame that is the processing target of the probability density function estimating unit is in the attack section, the arithmetic control unit is configured so that the sequential updating of the weight value in the frame starts from a predetermined initial value. When the calculation for estimating the probability density function of the fundamental frequency in the density function estimation means is performed and the frame to be processed by the probability density function estimation means is not in the attack section, the weight value in the frame is 2. The calculation for estimating the probability density function of the fundamental frequency in the probability density function estimating means is controlled so that the sequential update is started with the final value of the weight value in the previous frame as an initial value. The sound analyzer according to 1.   When the frame that is the processing target of the probability density function estimating unit is in the attack section, the arithmetic control unit is configured so that the sequential updating of the weight value in the frame starts from a predetermined initial value. When the calculation for estimating the probability density function of the fundamental frequency in the density function estimation means is performed and the frame to be processed by the probability density function estimation means is not in the attack section, the weight value in the frame is An operation for estimating the probability density function of the fundamental frequency in the probability density function estimating means so that the sequential update is started with a weight value obtained by mixing the final value of the weight value in the previous frame and a predetermined initial value as an initial value. As the elapsed time from the end of the previous attack section becomes longer, the final value of the weight value in the previous frame As is emphasized, the sound analysis apparatus according to claim 1, characterized in that to control the mixing ratio between the final value and the predetermined initial value of the weight values in the previous frame.   The arithmetic and control means switches the sound model used for the estimation of the probability density function of the fundamental frequency depending on whether or not the frame to be processed by the probability density function estimation means is an attack section. The sound analyzer according to claim 1, wherein   The calculation control means causes the fundamental frequency estimation means to output an estimation result of a fundamental frequency for the frame when the frame to be processed by the probability density function estimation means is not in an attack section, and the probability 2. The method according to claim 1, wherein when the frame to be processed by the density function estimation means is in an attack section, the fundamental frequency estimation means is not made to output an estimation result of the fundamental frequency for the frame. The sound analyzer described. Computer
Attack detection means for dividing the input acoustic signal into frames of a predetermined time length, and determining whether the input acoustic signal is an attack section signal for each frame;
For each frame, using a sound model that is a probability density function having a structure corresponding to the harmonic structure of the sound of each sound source, a mixed distribution is formed by weighted addition of a plurality of sound models corresponding to various fundamental frequencies. The weight value for each sound model is sequentially updated and optimized so that the mixed distribution becomes the distribution of the frequency components of the input acoustic signal, and the optimized weight value of each sound model is set to the fundamental frequency of the sound of the sound source. A probability density function estimating means for estimating a probability density function of
Fundamental frequency estimation means for estimating and outputting a fundamental frequency of sound of one or a plurality of sound sources included in the input acoustic signal based on a probability density function of the fundamental frequency;
The calculation for estimating the probability density function of the fundamental frequency in the probability density function estimating means or the basic in the fundamental frequency estimating means depending on whether or not the frame to be processed by the probability density function estimating means is in the attack section A computer program that functions as a calculation control means for switching a calculation mode for frequency estimation.

Images