JP4625934B2 - Sound analyzer and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent wrong estimation of a fundamental frequency when fundamental frequencies of respective sounds are estimated from frequency components of an input sound signal indicating a mixed sound. <P>SOLUTION: When a frequency band is limited 3, frequency components which are possibly fundamental frequency components or harmonic components of the sound of a sound source are extracted from the input sound signal. When a probabilistic density function of the fundamental frequency is estimated 41, it is considered that the extracted fundamental components have harmonic structures respectively and are generated from a mixed distribution of tone models corresponding to different fundamental frequencies, and the probabilistic density function of the fundamental frequency is estimated. In tone model deformation processing 5, deformation of the tone models for reducing the possibility of wrong estimation during the estimation is performed. <P>COPYRIGHT: (C)2008,JPO&amp;INPIT

Description

  The present invention is directed to a musical sound signal including a singing voice and a plurality of types of instrument sounds recorded on a commercially available CD (compact disc) or the like, and a pitch of a melody sound or a bass sound (in this specification, a fundamental frequency). The present invention relates to a sound analysis apparatus and a program for estimating the

  It is very difficult to estimate the pitch of a specific sound source from a monaural sound signal in which the sounds of many sound sources are mixed. One of the essential reasons why it is difficult to estimate the pitch of a mixed sound is that, in the time-frequency domain, the frequency component of one sound overlaps with the frequency component of another sound that is playing simultaneously. For example, in typical popular music played on singing voices, keyboard instruments (piano, etc.), guitars, bass guitars, drums, etc., part of the harmonic structure of the singing voice that plays the melody (especially the fundamental frequency component) It frequently overlaps with the harmonic component of the guitar, the higher harmonic component of the bass guitar, the noise component included in the sound of the snare drum, and the like. For this reason, a method of locally tracking each frequency component does not function stably for complex mixed sounds. There is a technique for estimating a harmonic structure on the assumption that a fundamental frequency component exists, but such a technique has a major drawback that it cannot handle a missing fundamental phenomenon. Furthermore, if the frequency components of other sounds that are playing at the same time overlap with the fundamental frequency components, they will not function effectively.

  For the above reasons, there has been a technique for estimating the pitch of a single sound or an acoustic signal that contains a single sound with aperiodic noise, but it is recorded on a commercially available CD. There was no technique for estimating the pitch of a mixed sound signal such as an acoustic signal.

  However, in recent years, a technique for appropriately estimating the pitch of each sound included in a mixed sound has been proposed by using a statistical method. This is the technique of Patent Document 1.

  In the technique of this Patent Document 1, a frequency component belonging to a band considered to be a melody sound and a frequency component belonging to a band considered to be a bass sound are separately extracted from an input acoustic signal by a BPF, and each of those bands is extracted. Based on the frequency components, the fundamental frequencies of the melody sound and the bass sound are estimated.

  More specifically, in the technique of Patent Document 1, a sound model having a probability distribution corresponding to the harmonic structure of a sound is prepared, and each frequency component of the band of the melody sound and each frequency component of the band of the base sound are It is considered to be a mixed distribution obtained by weighting and adding each sound model corresponding to various fundamental frequencies. Then, the weight value of each sound model is estimated using an EM (Expectation-Maximization) algorithm.

This EM algorithm is an iterative algorithm for performing maximum likelihood estimation on a probability model including hidden variables, and a local optimum solution can be obtained. Here, since the probability distribution having the largest weight value can be regarded as the most dominant harmonic structure at that time, the fundamental frequency in the dominant harmonic structure can be obtained as the pitch. . Since this method does not depend on the presence of the fundamental frequency component, the missing fundamental phenomenon can be appropriately handled, and the most dominant harmonic structure can be obtained without depending on the presence of the fundamental frequency component.
Japanese Patent No. 3413634

  By the way, in order to estimate the fundamental frequency, it is desirable to use a sound model that faithfully simulates the harmonic structure of the sound of the instrument that is actually sounding. However, some types of musical instruments generate a sound with a harmonic structure having a sharp peak at a specific nth harmonic. If the fundamental frequency is estimated using a sound model that simulates such a harmonic structure, the probability density function near the fundamental frequency that does not actually sound increases, and the fundamental frequency may be erroneously estimated. In some cases, it increases. The problem of erroneous estimation will be taken up again in the section “Best Mode for Carrying Out the Invention” for easy understanding.

  The present invention has been made in view of the circumstances described above, and it is an object of the present invention to reduce the possibility of erroneous estimation when estimating the fundamental frequency of each sound from the frequency components of an input acoustic signal indicating a mixed sound. It is said.

  This invention has a structure corresponding to the harmonic structure of the sound of each sound source, and a mixed distribution obtained by weighted addition of a plurality of sound models that are probability density functions corresponding to various fundamental frequencies is a frequency component of an input acoustic signal. A probability density function estimating means for optimizing a weight value for each sound model so as to be a distribution, and estimating a weight value of each optimized sound model as a probability density function of a fundamental frequency of the sound of the sound source; A fundamental frequency estimating means for estimating and outputting a fundamental frequency of sound of one or a plurality of sound sources included in the input acoustic signal based on a probability density function of the fundamental frequency; and a sound model modifying means for transforming the sound model. And a computer program for causing the computer to function as the probability density function estimating means, the fundamental frequency estimating means, and the sound model modifying means. To provide the ram.

  According to this invention, the sound model is deformed by the sound model deforming means, thereby avoiding erroneous estimation of the fundamental frequency due to the harmonic structure of the sound model and reducing the possibility of erroneous estimation.

  Embodiments of the present invention will be described below with reference to the drawings.

<Overall configuration>
FIG. 1 is a diagram showing the processing contents of a sound analysis program according to an embodiment of the present invention. This sound analysis program includes an acoustic signal such as a sound collection function for acquiring an acoustic signal from the natural world, a playback function for reproducing an acoustic signal of music from a recording medium such as a CD, or a communication function for acquiring an acoustic signal of music via a network. The program is installed and executed on a computer such as a personal computer having an acquisition function. The computer that executes the sound analysis program according to the present embodiment functions as the sound analysis device according to the present embodiment.

The sound analysis program according to the present embodiment estimates the pitch of a certain sound source in a monaural music sound signal acquired through the sound signal acquisition function. As the most important example, the melody line and the bass line are estimated here. The melody is a sequence of single notes that can be heard more prominently than the others, and the bass is the sequence of the lowest single note in the ensemble. The temporal changes are called the melody line Dm (t) and the base line Db (t), respectively. Assuming that the fundamental frequency F0 at time t is Fi (t) (i = m, b) and the amplitude is Ai (t), these are expressed as follows.

  As a means for obtaining the melody line Dm (t) and the base line Db (t) from the input sound signal, the sound analysis program includes an instantaneous frequency calculation 1, frequency component candidate extraction 2, frequency band restriction 3, Each process of the melody line estimation 4a and the baseline estimation 4b is included. Each process of the melody line estimation 4a and the baseline estimation 4b includes a fundamental frequency probability density function estimation 41 and a fundamental frequency sequential tracking 42 using a multi-agent model. In this embodiment, instantaneous frequency calculation 1, frequency component candidate extraction 2, frequency band restriction 3, melody line estimation 4 a, and baseline estimation 4 b, tracking of the fundamental frequency over time by the multi-agent model 42. The processing content of is basically the same as that disclosed in the above-mentioned Patent Document 1. The feature of the present embodiment is an improvement added to the estimation 41 of the probability density function of the fundamental frequency among the processes of the sound analysis program. Hereinafter, the content of each process which comprises the sound analysis program by this embodiment is demonstrated.

<Instantaneous frequency calculation 1>
In this process, the input acoustic signal is applied to a filter bank consisting of a plurality of BPFs, and the instantaneous frequency (Flanagan, JL and Golden, RM: Phase Vocoder, The BellSystem)
Technical J., Vol. 45, pp.1493-1509 (1966)). Here, the above-described Flanagan method is used, the short-time Fourier transform (STFT) output is interpreted as the filter bank output, and the instantaneous frequency is efficiently calculated. When the STFT using the window function h (t) for the input acoustic signal x (t) is given by the equations (3) and (4), the instantaneous frequency λ (ω, t) can be obtained by the equation (5). .

  Here, h (t) is a window function that gives the localization of the time frequency (for example, a time window created by convolving a second-order cardinal B-spline function with a Gaussian function that gives the optimum localization of the time frequency. Such).

A wavelet transform may be used to calculate this instantaneous frequency. Here, the STFT is used to reduce the amount of calculation. However, if only a single STFT is used, the time resolution and frequency resolution in a certain frequency band are deteriorated. Therefore, multi-rate filter banks (Vetterli, M .: A Theory of Multirate Filter Banks, IEEE Trans. On
ASSP, Vol. ASSP-35, No. 3, pp. 356-372 (1987)), and obtain a reasonable time-frequency resolution under the restriction that it can be executed in real time.

<Frequency component candidate extraction 2>
In this process, candidate frequency components are extracted based on the mapping from the center frequency of the filter to its instantaneous frequency (Charpentier, FJ: Pitch detection using the short-termphase
spectrum, Proc. of ICASSP 86, pp. 113-116 (1986)). Consider a mapping from the center frequency ω of an STFT filter to the instantaneous frequency λ (ω, t) of its output. Then, if there is a frequency component of frequency ψ, ψ is located at the fixed point of this mapping, and the value of the instantaneous frequency around it is almost constant. That is, the instantaneous frequency Ψ _f ^(t) of all frequency components can be extracted by the following equation.

これらの周波数成分のパワーは、Ψ_f ^(t)の各周波数におけるＳＴＦＴパワースペクトルの値として得られるため、周波数成分のパワー分布関数Ψ_p ^(t)(ω)を次のように定義することができる。

Since the power of these frequency components is obtained as the value of the STFT power spectrum at each frequency of ψ _f ^(t) , the power distribution function ψ _p ^(t) (ω) of the frequency component can be defined as follows. it can.

<Frequency band restriction 3>
In this process, the frequency band is limited by weighting the extracted frequency components. Here, two types of BPF are prepared for the melody line and the base line. The melody line BPF can pass most of the main fundamental wave components and harmonic components of a typical melody line, and cuts off a frequency band in which duplication near the fundamental frequency frequently occurs to some extent. On the other hand, the BPF for a bass line can pass many of the main fundamental frequency components and harmonic components of a typical bass line, and the frequency band in which the other performance parts are dominant over the bass line. To some extent.

平均律の半音は１００ｃｅｎｔに、１オクターブは１２００ｃｅｎｔに相当する。 In the present embodiment, the logarithmic scale frequency is expressed in units of cents (originally a scale representing pitch difference (pitch)), and the frequency fHz expressed in Hz is expressed as cents as follows: Convert to fcent.

A semitone of equal temperament corresponds to 100 cent, and one octave corresponds to 1200 cent.

When the frequency response of the BPF at the frequency x cent is BPFi (x) (i = m, b) and the power distribution function of the frequency component is ψ ′ _p ^(t) (x), the frequency component that has passed through the BPF is BPFi. (X) ψ ′ _p ^(t) (x). However, Ψ ′ _p ^(t) (x) is the same function as Ψ _p ^(t) (ω) except that the frequency axis is represented by cent. Here, as a preparation for the next stage, a probability density function p _Ψ ^(t) (x) of a frequency component that has passed through the BPF is defined.

Here, Pow ^(t) is the total power of the frequency components that have passed through the BPF as shown in the following equation.

<Estimation 41 of probability density function of fundamental frequency>
In this process, a probability density function of a fundamental frequency representing how much each harmonic structure is relatively dominant with respect to a frequency component candidate that has passed through the BPF is obtained. Therefore, in the present embodiment, the probability distribution function p _Ψ ^(t) (x) of the frequency component is a mixed distribution model (weighted sum model) of probability distribution (sound model) that models a sound having a harmonic structure. ). If the probability density function of a sound model having a fundamental frequency F is p (x | F), the mixed distribution model p (x; θ ^(t) ) can be defined by the following equation.

Here, Fhi and Fli are the upper and lower limits of the allowable fundamental frequency, and are determined by the pass band of the BPF. W ^(t) (F) is a weight of the sound model p (x | F) that satisfies the following expression.

Since it is impossible to assume the number of sound sources in advance for a real-world acoustic signal such as a CD, it is important to model in consideration of the possibility of all fundamental frequencies at the same time. If the model parameter θ ^(t) can be estimated as if the observed frequency component p _Ψ ^(t) (x) was generated from the model p (x; θ ^(t) ), then p _Ψ ^(t) (x) is It can be considered that the sound model has been decomposed into individual sound models. As shown in the following equation, the weight w ^(t) (F) for the sound model of each fundamental frequency F is represented by the probability density function p _FO ^{(t ) (F)} and it can be interpreted.

In other words, the more the sound model p (x | F) becomes dominant in the mixed distribution (that is, the larger w ^(t) (F)), the more the model of the model in p _FO ^(t) (F) The probability of the fundamental frequency F increases.

From the above, it can be seen that when the probability density function p _Ψ ^(t) (x) is observed, the problem of estimating the parameter θ ^(t) of the model p (x; θ ^(t) ) should be solved. The maximum likelihood estimator of θ ^(t) is obtained by maximizing the average log likelihood defined by the following equation.

Since this maximization problem is difficult to solve analytically, θ ^(t) is estimated using the aforementioned EM (Expectation-Maximization) algorithm. The EM algorithm performs maximum likelihood estimation from incomplete observation data (in this case, p _Ψ ^(t) (x)) by repeatedly applying an E step (expectation step) and an M step (maximization step) alternately. Iterative algorithm for In this embodiment, by repeating the EM algorithm, the probability density function p _Ψ ^(t) (x) of the frequency component that has passed through the BPF is converted into a plurality of sound models p (x | F) corresponding to various basic frequencies F. Is the most likely weighting parameter θ ^(t) (= {w ^(t) (F) | Fli ≦ F ≦ Fhi}, where each iteration of the EM algorithm , Parameter θ ^(t) (= {w ^(t) (F) | Fli ≦ F ≦ Fhi}), the old parameter estimate θ _old ^(t) (= {w _old ^(t) (F) | Fli ≦ F ≦ Fhi}) is updated to obtain a new (more likely) parameter estimate θ _new ^(t) (= {w _new ^(t) (F) | Fli ≦ F ≦ Fhi}) θ _old ^{(t )} Initial value is 1 The final estimated value at the previous time t−1 is used, and the recurrence formula for obtaining the new parameter estimated value θ _new ^(t) from the old parameter estimated value θ _old ^(t) is as follows. The derivation process of the recurrence formula is described in detail in Patent Document 1, so please refer to that.

FIG. 2 shows a process in which the weight parameter θ ^(t) (= {w ^(t) (F) | Fli ≦ F ≦ Fhi} for the sound model p (x | F) is updated by the EM algorithm in this embodiment. 2 shows an example in which a sound model having four frequency components is used in order to simplify the illustration.

In the EM algorithm in the present embodiment, based on the sound model p (x | F) corresponding to each fundamental frequency F and the weight value w _old ^(t) (F) for each current sound model, the frequency x Each time, a spectrum distribution ratio corresponding to each sound model is obtained.

As shown in the above equation (18), the spectrum distribution ratio (x | F) corresponding to each sound model p (x | F) at a certain frequency x is multiplied by the weight value w _old ^(t) (F). The sum of the amplitude values w _old ^(t) (F) p (x | F) at the frequency x of each sound model p (x | F) (corresponding to the integral value of the denominator in Expression (18)) is obtained, and the sum _Is obtained by dividing each amplitude value w _old ^(t) (F) p (x | F). As is clear from equation (18), at each frequency x, each spectrum distribution ratio (x | F) corresponding to each sound model p (x | F) is normalized so that the sum is 1. It becomes.

In the present embodiment, at each frequency x, the function value of the probability density function p _Ψ ^(t) (x) at that frequency x is distributed according to the spectrum distribution ratio of each sound model p (x | F) at that frequency x. Then, for each sound model p (x | F), the function values of the probability density function p _Ψ ^(t) (x) distributed in this way are summed up, and a share of each sound model p (x | F) is obtained. And Then, the share of all sound models is summed, the share of each sound model is divided by the sum, and the share of each sound model p (x | F) normalized so that the sum is 1 is a new weighting parameter. Let w _new ^(t) (F). By repeating the above processing, the probability supported by the probability density function p _Ψ ^(t) (x) of the frequency component of the mixed sound among the sound models p (x | F) having different fundamental frequencies F is obtained. The weight parameter w ^(t) (F) for the higher one is gradually emphasized. As a result, the weight parameter w ^(t) (F) represents the probability density function of the fundamental frequency in the mixed sound that has passed through the BPF.

こうして得られた周波数を音高とする。 To determine the most dominant fundamental frequency Fi (t), as shown in the following equation, the probability density function p _F0 ^(t) (F) of the fundamental frequency (equation (17) is iteratively calculated from equation (15). Obtained as the final estimated value) may be obtained.

Let the frequency obtained in this way be the pitch.

<Frequency tracking 42 of basic frequency by multi-agent model>
In the probability density function of the fundamental frequency, if multiple peaks corresponding to the fundamental frequency of the sound that is playing at the same time are antagonized, these peaks may be selected one after another as the maximum value of the probability density function. The result obtained simply may not be stable. Therefore, in this embodiment, in order to estimate the fundamental frequency from a global point of view, the trajectory of a plurality of peaks is continuously tracked in the time change of the probability density function of the fundamental frequency, and the most dominant and stable among them. Select the fundamental frequency trajectory. In order to control such tracking process dynamically and flexibly, a multi-agent model is introduced.

  The multi-agent model is composed of one feature detector and a plurality of agents (see FIG. 3). The feature detector picks up the prominent peaks in the probability density function of the fundamental frequency. The agent basically follows the trajectory driven by those peaks. In other words, the multi-agent model is a general-purpose framework that temporally tracks features that stand out in the input. Specifically, the following processing is performed at each time.

(1) After the probability density function of the fundamental frequency is obtained, the feature detector detects a plurality of conspicuous peaks (peaks exceeding a threshold that dynamically changes according to the maximum peak). Then, for each conspicuous peak, the promising peak is evaluated in consideration of the total power Pow ^(t) of frequency components. This is realized by regarding the current time as a time several frames ahead and prefetching and tracking the peak trajectory up to that time.

(2) When there is an agent already generated, the prominent peak is exclusively assigned to an agent having a locus close to it while interacting with each other. If multiple agents are candidates for assignment, assign them to the agent with the highest reliability.

(3) If the most promising and conspicuous peak has not yet been assigned, a new agent that tracks that peak is generated.

(4) Each agent has a cumulative penalty and disappears when it exceeds a certain threshold.

(5) An agent that has not been assigned a conspicuous peak receives a certain penalty, and tries to find the next peak to be tracked directly from the probability density function of the fundamental frequency. If the peak is not found, a penalty is applied. Otherwise, the penalty is reset.

(6) Each agent self-evaluates the reliability based on the weighted sum of the degree of how promising and conspicuous the peak assigned at present is and the reliability at the previous time.

(7) The fundamental frequency Fi (t) at time t is determined based on an agent having high reliability and a large total power along the track of the peak being tracked. The amplitude Ai (t) is determined by extracting a harmonic component or the like of the fundamental frequency Fi (t) from Ψ _p ^(t) (ω).

<Improvements of this embodiment over the technique of Patent Document 1>
In the above-described estimation 41 of the probability density function of the fundamental frequency, in order to calculate the above equation (17), it is necessary to assume the probability density function p (x | F) of the sound model. In this case, in order to perform highly accurate estimation, it is desirable to use a sound model that faithfully simulates the harmonic structure of the sound of the instrument that is actually sounding. However, some types of musical instruments generate a sound with a harmonic structure having a sharp peak at a specific nth harmonic. If the fundamental frequency is estimated using a sound model that simulates such a harmonic structure, the probability density function near the fundamental frequency that does not actually sound increases, and the fundamental frequency may be erroneously estimated. In some cases, it increases. An example thereof will be described with reference to FIG.

FIG. 4 shows a process in which the weight parameter θ ^(t) (= {w ^(t) (F) | Fli ≦ F ≦ Fhi} for the sound model p (x | F) is updated by the EM algorithm, as in FIG. 4, the sound model p (x | F) has a harmonic structure having a sharp peak near the second harmonic frequency. In the probability density function p _Ψ ^(t) (x) of the frequency component of the mixed sound that has passed through the BPF, if the value near the frequency of 200 Hz is high as shown in the figure, this frequency of 200 kHz is changed to the second harmonic in the process of the EM algorithm. The weight w (F) for the sound model p (x | F = 100 Hz) having a fundamental frequency of 100 Hz as the frequency is excessively increased, and as a result, the fundamental frequency is 100 Hz even though it does not actually sound. Wrong guess There is likely to be.

  In order to reduce the possibility of erroneous estimation as described above, the sound analysis program according to the present embodiment includes a sound model deformation process 5 as shown in FIG. In a preferred embodiment, the sound model modification process 5 is a process for modifying the sound model used for the estimation 41 of the probability density function of the fundamental frequency in accordance with, for example, an operation of an operation unit provided in a personal computer that is a sound analyzer. It is. Various forms of the sound model deformation process 5 can be considered, but the sound model deformation process 5 according to the present embodiment has a harmonic structure with a gentle undulation compared to the original sound model that has not been deformed. By mixing the sound models for deformation, the sound model is transformed into a harmonic model with a gentler undulation than the original harmonic structure.

More specifically, in this embodiment, as shown in the following equation, a sound model corresponding to the harmonic structure of the instrument sound obtained by actual measurement (an original sound model that is not deformed) and a Gaussian distribution are obtained. Weighted addition is performed to generate a mixed sound model p (x | F), and this mixed sound model p (x | F) is delivered to the probability density function estimation 41 of the fundamental frequency.

Here, h is a harmonic number indicating the number of harmonics counted from the fundamental frequency component, and Ni is the number of harmonic components constituting the sound model. Also, original (x; F + 1200 log ₂ h) is a sound model created based on the measured harmonic structure of the instrument sound, and is normalized so that the sum of function values at h = 1 to Ni is 1. Has been. Gauss (x; F + 1200 log ₂ h) is a Gaussian distribution normalized so that the sum of function values in h = 1 to Ni is 1, and is given by the following equation. Further, the parameter σ for determining the weight coefficient k and the spread of the Gaussian distribution is set by operating an operation unit (not shown). Further, k is set to a value within the range of 0-1.

FIG. 5 shows a sound model original (x; F + 1200 log ₂ h) based on actual measurement, a Gaussian distribution gauss (x; F + 1200 log ₂ h), and a mixed sound model p (x | F) obtained by weighted addition of these. Are illustrated.

As shown in FIG. 5, in the mixed sound model p (x | F) obtained by weighted addition with the Gaussian distribution, a sharp peak is relaxed compared to the sound model original (x; F + 1200 log ₂ h) based on actual measurement. A harmonic structure. When the sound model is deformed to a harmonic structure in which a sharp peak is relaxed in this way, the possibility of occurrence of erroneous estimation of the fundamental frequency as described above is reduced.

<Other embodiments>
Although one embodiment of the present invention has been described above, the present invention may have other embodiments. For example:

(1) In the above embodiment, the final fundamental frequency is determined by causing the multi-agent to track the fundamental frequency obtained by the fundamental frequency probability density function estimation 41. However, the fundamental frequency probability density function estimation 41 is performed. If the probability of erroneous estimation is low and a highly reliable estimation result is obtained, tracking by a multi-agent may be omitted.

(2) Depending on the type of instrument, the harmonic structure of the sound model that simulates the instrument has a significant peak in the low range (for example, the fundamental component), and the high range component is extremely poor. It may become. In such a case, in the EM algorithm, as illustrated in FIG. 6A, the sound is actually generated in the high frequency part of the probability density function p _Ψ ^(t) (x) of the frequency component that has passed through the BPF. A portion U that cannot be explained only by the sound model corresponding to the sound that is present. Therefore, in the EM algorithm, as illustrated in FIG. 6B, the sound models p (x | Fa) and p (x | Fb) of the sound that is actually sounding are used in order to explain the portion U. In addition to the above weight value, the weight of the sound model p (x | Fc) corresponding to the high-frequency sound that is not actually played may be increased. For this reason, the problem that the fundamental frequency of the sound which is not actually sounding may be output accidentally arises. Therefore, in order to prevent such inconvenience, in the sound model modification process 5, the sound model with a high fundamental frequency F is assumed to have a harmonic structure with an emphasized high range compared to a sound model with a low fundamental frequency F. In addition, the sound model may be modified. For example, in the case where the sound model original (x; F + 1200 log ₂ h) based on the actual measurement is weighted and added according to the above equation 20 and the Gaussian distribution gauss (x; F + 1200 log ₂ h) is generated to generate the mixed sound model p (x | F). If the weighting coefficient k is a function of the fundamental frequency F and the fundamental frequency F is high, the weight of the Gaussian distribution gauss (x; F + 1200 log ₂ h) is increased, and the high frequency of the mixed sound model p (x | F) is increased. A mode of lifting is conceivable. According to this aspect, in the region where the fundamental frequency F is high, the probability density function of the fundamental frequency F is estimated by the harmonic structure sound model in which the harmonic components in the high region are emphasized compared to the region where the fundamental frequency F is low. Therefore, there is an increased possibility that the frequency component of the mixed sound that has passed through the BPF can be explained only by weighted addition of the sound model corresponding to the sound that is actually sounding. Therefore, the possibility of erroneous estimation can be reduced.

(3) A sound model optimization function for automatically optimizing the degree of deformation of the sound model may be provided in the sound analyzer. More specifically with reference to FIG. 7, in this embodiment, the personal computer that is the sound analysis apparatus shows a training acoustic signal whose time transition of the fundamental frequency is known, and a melody line and a base line of the training acoustic signal. Receive each information from outside. Then, the entire sound analysis apparatus is controlled so that the training acoustic signal becomes a processing target, and the evaluation process 6 and the deformation control process 7 are repeated. Here, in the evaluation process 6, the melody line and the base line of the training acoustic signal obtained as an estimation result from the sound analyzer are compared with the melody line and the base line of the training acoustic signal acquired from the outside, and the melody line is compared. For each of the baselines, the degree of coincidence between the estimation results and those given from the outside is evaluated. Then, in the deformation control process 7, the sound model used for estimating the melody line or the sound model used for estimating the baseline is modified so that the degree of coincidence obtained by the evaluation process 6 increases for each of the melody line and the base line. To the sound model deformation process 5. For example, when a mixed sound model obtained by weighting and adding a sound model based on actual measurement and a Gaussian distribution is used for estimation of the probability density function of the fundamental frequency, in the deformation control process 7, the weighting coefficient k in the sound model deformation process 5 is changed variously. A mode in which a weighting factor k that maximizes the degree of coincidence is searched for.

It is a figure which shows the processing content of the sound analysis program which is one Embodiment of this invention. It is the figure which illustrated the process in which the parameter of the weight with respect to a sound model is updated by EM algorithm in the embodiment. It is a figure which shows time-dependent tracking of the fundamental frequency by the multi agent model comprised by one feature detector and a some agent. It is a figure explaining the problem in the process in which the parameter of the weight with respect to a sound model is updated by EM algorithm in the prior art. It is a figure which shows the example of the sound model used in the case of estimation of the probability density function of a fundamental frequency in the embodiment. It is a figure explaining the misestimation of the fundamental frequency generate | occur | produced when a sound model has the harmonic structure which has a peak in a low region. It is a figure explaining the sound model optimization function implement | achieved in other embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Calculation of instantaneous frequency, 2 ... Extraction of frequency component candidates, 3 ... Frequency band limitation, 4a ... Melody line estimation, 4b ... Baseline estimation, 41 ... Fundamental frequency probability density Function estimation, 42 …… Tracking of fundamental frequency by multi-agent model, 5 …… Sound model deformation process, 6 …… Evaluation process, 7 …… Deformation control process.

Claims (6)

Each of the sound components has a structure corresponding to the harmonic structure of the sound source, and a mixed distribution obtained by weighted addition of a plurality of sound models that are probability density functions corresponding to various fundamental frequencies is the distribution of the frequency components of the input acoustic signal. A probability density function estimating means for optimizing a weight value for each sound model and estimating a weight value of each optimized sound model as a probability density function of a fundamental frequency of a sound of a sound source in the input acoustic signal ;
Fundamental frequency estimation means for estimating and outputting a fundamental frequency of one or more sounds included in the input acoustic signal based on a probability density function of the fundamental frequency;
Sound model deformation means for deforming the sound model ,
The sound analysis apparatus according to claim 1, wherein the probability density function estimation means estimates a probability density function of a fundamental frequency of a sound of the sound source in the input acoustic signal using the sound model deformed by the sound model deformation means .   2. The sound according to claim 1, wherein the fundamental frequency estimation means detects a plurality of peaks in the probability density function, and outputs a fundamental frequency having high reliability and high power based on reliability of each peak. Analysis equipment.   The sound model deforming means mixes a sound model for deformation having a gently undulating harmonic structure with the sound model, so that a harmonic structure having a more undulating harmonic structure than the original harmonic structure can be obtained. The sound analysis apparatus according to claim 1, wherein the sound analysis apparatus is transformed into a sound model. The sound model deforming means is configured to provide a plurality of sounds corresponding to the plurality of fundamental frequencies so that a sound model having a high fundamental frequency has a higher harmonic structure than a sound model having a low fundamental frequency. The sound analysis apparatus according to claim 1, wherein the sound analysis apparatus performs deformation on the model . The training acoustic signal whose time transition of the fundamental frequency is known is processed by the probability density function estimating means and the fundamental frequency estimating means. As a result, the time transition of the fundamental frequency obtained from the fundamental frequency estimating means and the training An evaluation process for evaluating the degree of coincidence with the time transition of the fundamental frequency of the acoustic signal;
A sound model optimizing unit that repeatedly executes a deformation control process that causes the sound model deforming unit to deform the sound model so that the degree of coincidence evaluated by the evaluation process increases. The sound analyzer according to any one of claims 1 to 4. Computer
Each of the sound components has a structure corresponding to the harmonic structure of the sound source, and a mixed distribution obtained by weighted addition of a plurality of sound models that are probability density functions corresponding to various fundamental frequencies is the distribution of the frequency components of the input acoustic signal. A probability density function estimating means for optimizing a weight value for each sound model and estimating a weight value of each optimized sound model as a probability density function of a fundamental frequency of a sound of a sound source in the input acoustic signal ;
Fundamental frequency estimation means for estimating and outputting a fundamental frequency of sound of one or a plurality of sound sources included in the input acoustic signal based on a probability density function of the fundamental frequency;
A sound model deforming means for deforming the sound model , wherein the probability density function estimating means uses the sound model deformed by the sound model deforming means and the probability density function of the fundamental frequency of the sound of the sound source in the input acoustic signal A computer program that functions as a sound model deforming means for estimating sound.

Images