JP2010175614A - Clustering calculation device, clustering calculation method, clustering calculation program, and computer readable recording medium recorded with the program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide clustering technology for accurately estimating the number of speakers and a parameter for characterizing each speaker, in dialization. <P>SOLUTION: A clustering calculation device 3 includes: an observation amount creation section 26 which reads a power vector extracted from recorded conversation data, to convert it to an observation amount of the vector corresponding to dynamic Hierarchical Dirichlet Process (dHDP) approximation model; a storage means 10 for accumulating and storing a collection data of the converted observation amount; a variation post-distribution inference section 30 in which a value of post-distribution of a plurality of parameters when the plurality of clusters are created from the collection data of the observation amount by dHDP approximation model, is respectively estimated by an expectation-maximization (EM) algorithm, and sequentially stored and updated in the storage means 10; and an output control section 28 for outputting a latest estimation value of the post-distribution of the plurality of parameters, which are stored in the storage means 10, when a finishing condition set beforehand is satisfied. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a technique for estimating the number of speakers from conversation recording data in which the number of speakers is unknown.

  Conventionally, there has been known a problem of estimating the number of speakers participating in a conversation and the timing of each speaker speaking from recorded data consisting of conversations of a plurality of speakers. This problem and the technology that solves this problem are called diarization. Dialization can be said to be a technology that automatically estimates “when and who spoke”. As an application of this technology, a wide range of uses such as annotation to conference data, search using the same, and speech enhancement are expected.

  Existing dialytical solutions can be roughly divided into two. The first method is a method for estimating speaker-specific voice quality and distinguishing speakers, and the second method is a method for estimating a speaker position. Among these, in the first method (method using speaker voice quality), the speaker who is currently speaking is identified by extracting the voice characteristics of each speaker. This first method has an advantage that the speaker can be identified even if the speaker moves, but has a disadvantage that it becomes difficult to identify the speaker when a plurality of speakers speak at the same time.

  The second method (a method using information related to the speaker position) is a method of estimating the number of speakers and their positions by estimating the positions of the speakers (see, for example, Non-Patent Documents 1 to 3). In the methods of Non-Patent Documents 1 to 3, speaker identification is performed by estimating the position of each speaker using a microphone array. Therefore, although the methods of Non-Patent Documents 1 to 3 have a drawback that they cannot be identified as the same speaker when the speaker moves, the utterance behavior of each speaker is identified even when a plurality of speakers speak at the same time. There is an advantage that you can.

  In dialization, in general, the number of speakers in recorded audio data is unknown and must be estimated. In order to identify each speaker, regardless of whether the first method (method of using speaker voice quality) or the second method (method of using information on the speaker position) is used, The amount (parameter) that characterizes the speaker needs to be estimated. This corresponds to a so-called clustering problem. The clustering problem is a problem of dividing observed data into an appropriate number of clusters and estimating parameters of each cluster. Even if dialization is simply replaced with a clustering problem, the number of clusters and the division of data are unknown, so it is necessary to learn with parameters. In the problem of dialization, the number of clusters represents “the number of speakers”, the parameter of each cluster represents “the feature amount of the speakers”, and “the timing of each speaker's utterance” is suggested from the result of dividing the data obtained. Will be.

  Non-Patent Document 1 uses a clustering technique called a leader-follower algorithm. This is a sequential clustering method, in which clustering is performed while inputting samples little by little. In this clustering method, when a newly input sample is more than a certain distance away from any existing cluster during clustering, a new cluster is generated around that sample. The leader-follower algorithm is described in “R. O. Duda, P. E. Hart and D. G. Stork,“ Pattern Classification ”, John Wiley & Sons, 2001.”.

  Non-Patent Document 3 proposes a method of selecting the number of clusters that maximizes an evaluation value called a BIC criterion. In this method, the evaluation value is calculated by actually performing clustering after setting the number of clusters to K, and then calculating the evaluation value by setting the number of clusters to K + 1. Find the number of clusters that maximizes the value.

  By the way, as a clustering model for data with an unknown number of clusters, a non-parametric Bayes model has recently been used in many situations. For example, Dirichlet Process Mixture (DPM), which is one type of non-parametric Bayesian model, can simultaneously optimize the number of clusters and clustering of each sample at random. Therefore, DPM has a great feature in that optimization of the number of clusters can be easily realized like an existing clustering model. In this DPM, a model called dynamic Hierarchical Dirichlet Process (dHDP) is known as an extended model that models a continuous temporal change of a probability distribution (see Non-Patent Document 4).

S. Araki, M. Fujimoto, K. Ishizuka, H. Sawada and S. Makino, “A DOA Based Speaker Diarization System for Real Meetings”, Proceedings of the Joint Workshop on Hands-Free Speech Communication and Microphone Arrays, p.29 -32, 2008. Akiko Araki, Masayoshi Fujimoto, Kentaro Ishizuka, Hiroshi Sawada, Shoji Makino, “Conference Speech Speaker Identification System Using Speech Interval Detection and Direction Information and Its Evaluation”, Proceedings of the Acoustical Society of Japan, Spring, vol. 1-10 -1, p. 1-4, 2008. JM Pardo, X. Anguera and C. Wooters, “Speaker Diarization for Multi-Microphone Meetings Using Only Between-Channel Differences”, Proceedings of the Third Joint Workshop on Multimodal Interaction and Related machine Learning Algorithms ”, p. 257-264, 2008 . L. Ren, D. B. Dunson and L. Carin, “The Dynamic Hierarchical Dirichlet process”, Proceedings of International Conference on Machine Learning, p. 824-831, 2008.

  In the conventional dialization research, for example, the method described in Non-Patent Document 1 is simple in calculation and can be executed at high speed, but it is necessary to set a distance threshold value for generating a new cluster. By setting this threshold value, the finally obtained clustering and the number of clusters are determined. On the other hand, it is difficult to optimize this threshold in the sense of bringing the number of clusters and the estimated value of the clustering result closer to the true value.

  In addition, for example, the method described in Non-Patent Document 3 needs to perform clustering even under an inappropriate number of clusters in practice, and a large waste occurs from the viewpoint of calculation amount and time in the process. To do.

  Therefore, in the conventional dialization research, there is room for improvement in the problem of clustering, which estimates the number of speakers and speaker characteristics. In addition, reduction of processing load and speeding up of processing are demanded for dialization. Furthermore, in conventional dialization research, nonparametric Bayes models such as dHDP and DPM have not been adopted as clustering models, and their application methods have not been known.

  Therefore, an object of the present invention is to provide a clustering technique for solving the above-described problems and accurately estimating the number of speakers and parameters characterizing each speaker in dialization.

  In order to achieve the above object, the inventors of the present application have made various studies on clustering for estimating the number of speakers and parameters characterizing each speaker in dialization. As a result, we found that the number of speakers can be estimated accurately when the nonparametric Bayes model is adopted as a model that can simultaneously optimize the number of clusters, data division, and parameters in the sense of maximizing the posterior probability.

  Therefore, the clustering calculation apparatus according to the present invention includes a feature amount extraction unit that extracts a feature amount that characterizes each speaker in order to estimate the number of speakers in the conversation from recording data of the conversation in which the number of speakers is unknown. A clustering calculation device for estimating values of a plurality of unknown parameters when generating a plurality of clusters corresponding to each speaker from the extracted feature values, and a conversation calculation method using the estimated plurality of parameter values. A clustering calculation apparatus of a dialization system having identification means for identifying each speaker, the reading means for reading the extracted feature value, and a vector corresponding to the non-parametric Bayes model for the read feature value Observing amount generating means for converting the observed amount into the observed amount, and an observing amount storing means for accumulating and storing the set data of the converted observed amount Posterior distribution inference means for estimating and updating the posterior distribution values of a plurality of parameters by an EM algorithm when generating a plurality of clusters from the aggregated data of the observation amount by a nonparametric Bayes model, and the estimation and update The estimated value storage means for accumulating and storing the values of the posterior distributions of the plurality of parameters, and the latest of the posterior distributions of the plurality of parameters stored in the estimated value storage means when a preset termination condition is satisfied. Output control means for outputting the estimated value.

  According to such a configuration, the clustering calculation apparatus uses the observation amount set data generated by converting the feature amount extracted from the recording data of the conversation into the vector observation amount so that it can be applied to the nonparametric Bayes model. The number of clusters, data division, and parameters are optimized simultaneously in the sense of maximizing posterior probabilities through operations according to the nonparametric Bayes model. In this optimization, the clustering calculation apparatus repeatedly accumulates and stores the posterior distribution estimation and update using the EM algorithm. Here, since the EM algorithm is a calculation algorithm for a local optimum solution, it can be converged to a single solution by repeating the calculation. In the non-parametric Bayes model, the number of clusters prepared in advance converges to the number of effective clusters during the optimization process, and the cluster mixture ratio of other clusters becomes almost zero. . If this clustering result is obtained, the number of samples (number of observations) belonging to each cluster can be calculated. Further, the mixing ratio of each cluster can be calculated using the number of samples belonging to each cluster. Furthermore, the effective number of clusters can be determined by this mixing ratio. Here, since the number of clusters corresponds to the speakers in the recording data of the conversation, the number of speakers can be determined. In addition to the clustering results, estimated values of various variables using the clustering results can be accumulated and stored in the same manner. Then, the clustering calculation apparatus outputs the latest estimated value stored and stored when the end condition is satisfied.

  Further, in the clustering calculation apparatus according to the present invention, the posterior distribution inference means uses a pre-distribution, an observation distribution, and a maximum cluster predetermined in a dHDP (dynamic Hierarchical Dirichlet Process) model as the processing of the E step of the EM algorithm. Number, hyperparameter setting value, aggregated data of observations converted from the past to the estimation target time, and estimated values of the posterior distribution of hidden variables estimated from the past to the latest M steps And the cluster, the mixture ratio, and the weight representing the degree of time variation of the cluster distribution for each calculation target time including the past, for each cluster, and for all data for each calculation target time. As E step calculation means for calculating the value of the posterior distribution of parameters, and M step processing of the EM algorithm, Read the hyper parameter setting values, the aggregated data of the observed quantities converted from the past to the estimation target time, and the estimated values of the posterior distribution of the parameters estimated from the past to the latest E step. Estimate the value of the posterior distribution of two types of hidden variables for each calculation target time including the minute and for all data for each calculation target time, and change the time distribution of the cluster distribution among the two types of hidden variables The value of the posterior distribution of the first hidden variable associated with the weight representing the degree of the calculation is calculated for each cluster, and the value of the posterior distribution of the second hidden variable associated with the first hidden variable and the mixture ratio is calculated. The M step calculation means for calculating each time retroactively from the calculation target time, and the process of the E step and the process of the M step are alternately repeated a predetermined number of times. And a convergence determination means for performing control to execute the return.

  According to such a configuration, the clustering calculation device uses an EM algorithm to simultaneously optimize the number of clusters, data division, and parameters in the sense of posterior probability maximization by calculation according to the dHDP model among nonparametric Bayes models. Is used. Further, in the dHDP model, the parameter distribution at a certain time t is defined as the parameter distribution at the previous time t−1 and the change amount of the parameter distribution at the time t as a change rate (weight) of the change amount at the time t. ). In other words, dHDP is a model in which the distribution changes little by little depending on time. The algorithm of the clustering calculation device was constructed assuming that the parameter distribution at a certain time t constitutes the speaker who actually spoke at time t. Therefore, the dHDP model has become a probabilistic model convenient for the task of dialization in which the distribution of observed quantities (samples) changes with time due to speaker turn-taking. Therefore, it is possible to accurately estimate the number of unknown speakers and the parameters characterizing each speaker from the conversation recording data with the change of speakers.

  Further, in the clustering calculation apparatus according to the present invention, the E-step calculation means is configured such that the observation amount set data and the estimated value of the posterior distribution of the hidden variable are constants set in advance from the estimation target time or immediately before The data of the time retroactively reflected by the number of fluctuations reflecting the estimated value of the M step is read, and the calculation target time is the time from the estimation target time to the past time retroactive by the preset set value. A cluster to be reestimated in advance every time the process of updating the cluster for the calculation of each cluster is performed by calculating the value of the posterior distribution of the parameters related to the cluster, the mixture ratio, and the weight On the basis of the determination criteria, it is determined whether or not the cluster to be estimated is to be re-estimated. The posterior distribution value is calculated, and the M-step calculating means sets a constant set in advance from the estimation target time or the immediately preceding M step for the observation amount set data and the estimated value of the posterior distribution of the parameter. The data of the time retroactively reflected by the number of fluctuations reflecting the estimated value of the first time is read, and the first hidden time is set as the calculation target time from the time of the estimation target to the time of the past retroactively to the set value set in advance. Whether the cluster to be estimated should be re-estimated based on the above judgment criteria every time the posterior distribution values of the variable and the second hidden variable are calculated and the process of updating the cluster for each cluster is executed. If the cluster is to be re-estimated, the value of the posterior distribution of the first hidden variable is calculated.

  According to such a configuration, the clustering calculation apparatus, in the E step and the M step, sets the observation amount set data that has been traced back to a predetermined point in time from the estimation target time, and the estimated value required for the calculation. , And the calculation for obtaining the estimated value is performed using the time up to the past as far as the set value determined in advance from the time of the estimation target as the calculation target time. Therefore, the processing load is reduced and the processing speed is increased compared to the case where the number of variables to be estimated increases with the progress of the time step without having to set a past point in time or set value in advance. it can. In addition, when the cluster is updated for the calculation for each cluster in the E step and the M step, the clustering calculation apparatus performs a re-estimation process of the estimated value in the cluster only when necessary. Therefore, the processing load can be reduced and the processing speed can be increased as compared with the case where the estimation value re-estimation process is executed every time for the maximum number of clusters set in advance in the dHDP model.

  In order to solve the problem, the clustering calculation method according to the present invention includes a storage unit, a reading unit, and a dialing system for estimating the number of speakers of the conversation from the recording data of the conversation whose number of speakers is unknown. An observation amount generating means, a posteriori distribution inference means, and an output control means, and generating a plurality of clusters corresponding to each speaker from the feature quantity characterizing each speaker extracted from the recorded data A clustering calculation method of a clustering calculation apparatus for estimating the values of a plurality of unknown parameters respectively, wherein the reading means reads a feature quantity reading step, and the observation quantity generation means performs the reading Is converted into a vector observable corresponding to the nonparametric Bayes model, and the converted observable An observable amount accumulation step for sequentially accumulating the set data in the storage means; Each value is estimated by an EM algorithm, and the estimated value is sequentially stored and updated in the storage means, and when the end condition preset by the output control means is satisfied in the storage means. And an estimated value output step of outputting a latest estimated value of the posterior distribution of the plurality of parameters stored therein.

  According to such a procedure, in the clustering calculation method, the clustering calculation apparatus first reads the feature amount extracted from the recording data of the conversation, converts it into a vector observation amount so as to be adaptable to the nonparametric Bayes model, and accumulates it. . Then, the clustering calculation apparatus simultaneously optimizes the number of clusters, the data division, and the parameters in the sense of maximizing the posterior probability by the calculation according to the nonparametric Bayes model using the generated aggregate data of the observation amount. In this optimization, the clustering calculation apparatus repeatedly accumulates and stores the posterior distribution estimation and update using the EM algorithm. Then, the clustering calculation apparatus outputs the latest estimated value stored and stored when the end condition is satisfied.

  Further, in the clustering calculation method according to the present invention, the posterior distribution inference means, as the processing of the E step of the EM algorithm in the posterior distribution estimation step, a predistribution, an observation distribution, and a predetermined distribution determined in advance in the dHDP model. The maximum number of clusters, the set value of the hyper parameter, the aggregated data of observations converted from the past to the estimation target time, the estimated value of the posterior distribution of the hidden variables estimated from the past to the latest M steps, and For each calculation target time including the past in the estimation target time, for each cluster, and for each data for each calculation target time, the cluster, the mixture ratio, and the time of the cluster distribution A step of calculating a value of a posterior distribution of a parameter relating to a weight representing a degree of change, and a process of M step of the EM algorithm Then, the setting value of the hyper parameter, the collective data of the observation amount converted from the past to the estimation target time, and the estimated value of the posterior distribution of the parameter estimated from the past to the latest E step are read. Then, the value of the posterior distribution of the two types of hidden variables is estimated for each calculation target time including the past in the estimation target time and for all the data for each calculation target time, and the two types of hidden variables are estimated. Among the posterior distribution values of the first hidden variable associated with the weight representing the degree of temporal change of the cluster distribution is calculated for each cluster, and the second associated with the first hidden variable and the mixture ratio. The value of the posterior distribution of the hidden variable is calculated for each time retroactively from the calculation target time, and the process of the E step and the process of the M step are alternately predicted. Characterized in that it only repeatedly executed the number of times defined.

  According to such a procedure, in the clustering calculation method, the clustering calculation device simultaneously optimizes the number of clusters, the data division, and the parameters in the sense of maximizing the posterior probability by the calculation according to the dHDP model among the nonparametric Bayes models. To do this, the EM algorithm is used. Here, dHDP is a model in which the distribution changes little by little depending on the time, so a dialization task in which the observed quantity (sample) distribution changes with time due to the turn-taking of speakers. It is a good stochastic model. Therefore, it is possible to accurately estimate the number of unknown speakers and the parameters characterizing each speaker from the conversation recording data with the change of speakers.

  Further, in the clustering calculation method according to the present invention, the posterior distribution inference means determines in advance from the time to be estimated about the set data of the observation amount and the estimated value of the posterior distribution of the hidden variable in the E step. The data of the retroactive time is read by the number of fluctuations reflecting the set constant or the estimated value of the previous M step, and in the M step, the aggregated data of the observation amount and the estimated value of the posterior distribution of the parameter are obtained. Is characterized in that it reads data at a time retroactive to the past by the number of fluctuations reflecting a constant set in advance or an estimated value of the immediately preceding M step from the estimation target time.

  According to such a procedure, in the clustering calculation method, in the E step and the M step, the collective data of the observation amount traced back to a predetermined point in time from the time of the estimation target, and the estimated value necessary for the calculation And read. Therefore, the processing load can be reduced and the processing speed can be increased as compared to the case where the number of variables to be estimated is only increased with the progress of the time step without determining one point in the past in advance.

  Further, in the clustering calculation method according to the present invention, the posterior distribution inference means uses, as the calculation target time, a past time that is retroactive by a set value set in advance from the estimation target time in the E step. Calculates the posterior distribution values of the parameters related to the cluster, the mixture ratio, and the weight, and in the M step, calculates from the estimation target time to a past set time retroactive to a preset value. As the target time, the value of the posterior distribution of the first hidden variable and the second hidden variable is calculated.

  According to such a procedure, in the clustering calculation method, in the E step and the M step, the estimated value is calculated using the estimation target time from the estimation target time to a past time traced back in the past by a predetermined set value. Perform the desired calculation. Therefore, the processing load can be reduced and the processing speed can be increased as compared with the case where the number of variables to be estimated is increased with the progress of the time step without setting the set value in advance.

  In addition, the clustering calculation method according to the present invention performs a re-estimation that is set in advance every time the posterior distribution inference means executes a process of updating a cluster for calculation for each cluster in the E step. Based on the judgment criterion of the power cluster, it is determined whether or not the cluster to be estimated is to be re-estimated, and only when the cluster is to be re-estimated, the value of the posterior distribution of the parameter relating to the mixture ratio is calculated, In step M, each time a process for updating a cluster is performed for each cluster operation, it is determined whether or not the estimation target cluster should be re-estimated based on the determination criterion. Only in some cases, the value of the posterior distribution of the first hidden variable is calculated.

  According to such a procedure, in the clustering calculation method, in the E step and the M step, when the cluster is updated for the calculation for each cluster, the estimation value in the cluster is re-estimated only when necessary. Therefore, the processing load can be reduced and the processing speed can be increased as compared with the case where the estimation value re-estimation process is executed every time for the maximum number of clusters set in advance in the dHDP model.

  A clustering calculation program according to the present invention is a program for causing a computer to function as each means constituting one of the clustering calculation apparatuses. By being configured in this way, a computer in which this program is installed can realize each function based on this program.

  The computer-readable recording medium according to the present invention is characterized in that the clustering calculation program is recorded. By being configured in this way, a computer equipped with this recording medium can realize each function based on a program recorded on this recording medium.

  According to the present invention, the non-parametric Bayes model is used for the problem of speaker clustering in dialization, and probabilistic clustering is used, so that it is easy regardless of conventional parameter setting and search. The number of speakers can be estimated. Further, according to the present invention, since the non-parametric Bayes model is adopted, the number of clusters, data division, and parameters can be simultaneously optimized in the sense of maximizing the posterior probability. As a result, the number of speakers and parameters characterizing each speaker can be accurately estimated in dialization.

It is a block diagram which shows the outline | summary of the dialization system containing the clustering calculation apparatus which concerns on embodiment of this invention. It is a flowchart which shows the flow of the whole process of the clustering calculation method which concerns on embodiment of this invention. It is a flowchart which shows the variational posterior distribution inference procedure shown in FIG. It is a flowchart which shows an example of the calculation procedure of E step shown in FIG. It is a flowchart which shows an example of the calculation procedure of M step shown in FIG. It is a flowchart which shows the calculation procedure of E step in the clustering calculation method which concerns on embodiment of this invention. It is a flowchart which shows the calculation procedure of M step in the clustering calculation method which concerns on embodiment of this invention. It is a functional block diagram which shows an example of a structure of the clustering calculation apparatus which concerns on embodiment of this invention. It is a graph which shows the time average power distribution of the artificial speech data used in order to evaluate the clustering performance of the clustering calculation apparatus which concerns on embodiment of this invention. It is a graph which shows the time average power distribution of the real audio | voice data used in order to evaluate the clustering performance of the clustering calculation apparatus which concerns on embodiment of this invention, Comprising: (a) is CP1, (b) is CP2, (c). Represents DC, and (d) represents CN. It is a graph which shows the result of having clustered artificial speech data with the clustering calculation apparatus which concerns on embodiment of this invention, Comprising: (a) has shown DPM and (b) has each shown dHDP. It is a graph which shows the result of having clustered real voice data of CP1 by the clustering calculation apparatus concerning an embodiment of the present invention, and (a) shows DPM and (b) shows dHDP, respectively. It is a graph which shows the result of having clustered real voice data of DC by the clustering calculation device concerning an embodiment of the present invention, and (a) shows DPM and (b) shows dHDP, respectively.

  An embodiment (hereinafter referred to as “embodiment”) for carrying out the clustering calculation apparatus and clustering calculation method of the present invention will be described in detail with reference to the drawings. Below, an outline of the inference principle, an outline of the dialization system, an outline of the clustering calculation method, a calculation algorithm, and a clustering calculation apparatus will be sequentially described.

[Outline of inference principle]
In the present embodiment, for example, dHDP is used as a kind of nonparametric Bayes model. Here, dHDP will be briefly described. Equations (1) to (6) show mathematical models of dHDP. Of course, other non-parametric Bayes models such as DPM may be used. Details of DPM are described in, for example, “Osamu Ueda, Takeshi Yamada,“ Non-parametric Bayes Model ”, Applied Mathematics, Vol. 17, No. 3, pp. 196-214, 2007.”

Here, ~ represents sampling from the probability distribution. DP (·) represents a Dirichlet Process (infinite dimensional distribution), and γ, α ₀ , a ₀ , and b ₀ are hyperparameters set in advance.

In dHDP, first, an infinite number of discrete parameter distributions (clusters) G ₀ are generated using equation (1). In dialization, this distribution G ₀ corresponds to the utterance ratio and composition of each utterer when viewing the entire data. In a non-parametric Bayes model typified by DPM, estimated solutions are automatically aggregated into a distribution consisting of a small number of parameters (speakers).

H _{t in} equation (2) represents the temporal change (distribution change) of the speaker distribution at time t.
W _{t in} equation (3) represents the rate (degree) of temporal change in speaker distribution at time t.
Θ _{ti in} equation (5) represents the parameter of the cluster at time t, and x _{ti in} equation (6) represents the sample distribution at time t. Note that i indicates i-th data at time t.

G _t in equation (4) is a parameter distribution at time t. In dialization, the distribution G _t represents the configuration of the speaker who actually spoke at time t. Expression (4) expresses this G _t as a distribution obtained by superimposing the distribution G _t-1 at time t _-1 and H _t representing the distribution change at time t on w _t . Therefore, the DHDP, variations in the sample distribution at each time (distribution of x _ti) is allowed. On the other hand, since the G ₀ is the time to invariant distribution also to estimate, it has also been learning speaker cluster through the whole.

In addition, in dHDP, w _t, which is the rate of change, is also learned dynamically, so that the distribution changes dramatically when the speaker changes, and otherwise the distribution changes little. It can also support modeling of data whose ratio is not constant. In this way, dHDP is a model in which the distribution changes little by little depending on the time. Therefore, the distribution of observation (sample) changes with time due to the turn-taking of speakers. It is a good probability model for tasks.

[Outline of Dialization System]
FIG. 1 is a configuration diagram showing an outline of a dialization system including a clustering calculation apparatus according to an embodiment of the present invention. The dialization of this embodiment will be described as being based on the second method described above (a method using information related to the speaker position). A conversation by an unknown number of speakers is recorded in advance and used as an input to the dialization system 1. Here, it is assumed that three speakers H _A , H _B , and H _C talk in a fixed position as shown in FIG. The voice data (conversation recording data) 102 is time-series data.

  The dialization system 1 is composed of one large computer or a plurality of computers. Here, the dialization system 1 includes three computers, that is, a feature amount extraction unit 2, a clustering calculation device 3, and an identification unit 4.

  The feature amount extraction unit 2 performs preprocessing such as noise removal and extracts various feature amounts suitable for dialization. For example, the feature amount extraction unit 2 extracts DOA (direction of arrival) information from the recording data acquired from the microphone array and outputs the information to the clustering calculation device 3. The DOA information (sound arrival angle) is an amount obtained by estimating how much sound signal is observed from which direction with respect to the microphone.

  The clustering calculation apparatus 3 estimates the number of speakers and parameters characterizing each speaker by clustering processing based on the DOA information. That is, the clustering calculation device 3 clusters the extracted speech feature values, and estimates the number of clusters and the parameters of each cluster. The clustering calculation apparatus 3 simultaneously optimizes the number of clusters, data division, and parameters in the sense of maximizing the posterior probability by applying a probabilistic clustering model.

  The identification unit 4 identifies the utterance state of the speaker at each time based on the clustering result (clustering estimation value) obtained by the clustering calculation device 3. The identification unit 4 analyzes the clustering estimation value and presents the number of clusters and the position thereof on a screen display that can be identified by the user.

In the graph of the dialization result display 103 illustrated in FIG. 1, the horizontal axis indicates time (seconds) and the vertical axis indicates the direction (speaker position). In this example, rectangular waveforms are displayed in three directions corresponding to the three speakers H _A , H _B and H _C. The part which becomes the peak of each rectangular waveform represents the speech (utterance) of each speaker. First, when the speaker H _C finishes speaking, the speaker H _A speaks, and when the speaker H _C starts speaking again during the speech, the speaker H _A is silent and the speaker H _B speaks. It can be seen that there is a turn-taking of the speaker, such as starting. In addition, it turns out that the timing which two speakers speak simultaneously also arises.

である。なお、ｄ＝−１８０の方向と、ｄ＝１８０の方向とは同じものである。 [Outline of clustering calculation method]
Here, the introduction of the inference principle to the dialization and the clustering model will be described.
<Introduction of inference principle into dialization>
Here, since the non-parametric Bayes model (dHDP) is used in the clustering calculation apparatus 3, the feature quantity extracted by the feature quantity extraction unit 2 in the preceding stage is formulated. Let _ftd be the sound power (DOA information) heard from the direction of angle d (for example, d = −180, −179,..., 0,..., 180) at time t. That is, the audio power vector at each time t is

It is. The direction of d = −180 and the direction of d = 180 are the same.

This power vector is converted into a set of one-dimensional vectors _xti so as to conform to the clustering model used in this embodiment. Here, for each f _td , n _td sample sets having a value g (d) are generated using the threshold constant τ and the parameter μ. Here, the function g (•) indicates an appropriate scale function selected according to the convenience of implementation. For example, as the function g (•), a function that converts [−180: 180] → [−0.5: 0.5] can be used. N _td is defined by equation (7). Note that the present invention is not limited to one dimension, and may be configured to convert into a set of three-dimensional vectors _xti , for example.

  The function h (·) in Expression (7) is a function that returns some positive integer according to the power value, and may be a constant. In this embodiment, for example, h = 1 is used. The above quantization transformation is performed for all angles d, and the observation amount at time t is aggregated into a sample set.

The clustering problem handled in the present embodiment can be regarded as clustering of the sample set X _t = {x ₁ ,..., X _t }. If the human speech power is much larger than the background noise, a large number of sample values reflecting the speaker's position will be observed, so the speaker's position should be able to be estimated as the main cluster. . Therefore, in the clustering calculation method of the present embodiment, while estimating the number of clusters K, the probability distribution of the variable z _ti indicating which cluster belongs to each sample x _{ti is} obtained at the same time, and further, K clusters are obtained. The corresponding parameters Θ = {θ _k } are respectively obtained.

<Clustering model>
In this embodiment, the dHDP approximate model is used in consideration of the calculation amount and the simplicity of the algorithm. The generation model of the dHDP approximate model is expressed by the following formulas (9) to (15). For the dHDP approximation model, see “I. Pruteanu-Malinici, L. Ren, J. Paisley, E. Wang and L. Carin,“ Dynamic Hierarchical Dirichlet Process for Modeling Topics in Time-Stamped Documents ”, IEEE Transactions on Pattern. Analysis and Machine Intelligence, submitted, 2008. "

In the dHDP approximate model, first, the maximum number of clusters K is fixed. The maximum cluster number K may be a value sufficiently larger than the number of speakers to be estimated (for example, several tens to 100). In practice, the number of “effective” clusters K _eff (<K) corresponds to the speaker to be estimated. The “effective” number of clusters K _eff can be determined by utilizing the fact that the weight (mixing ratio) of other clusters that do not correspond to the speaker automatically becomes almost zero as a result of learning. That is, the “effective” number of clusters K _eff is automatically determined in the estimation process.

<< Formula (9), Formula (10) >>
In the dHDP approximate model, parameters corresponding to K clusters are sampled by Equation (9). In Expression (10), sampling of an innovation measure H _t (see Expression (16c) described later) is performed from a finite-dimensional Dirichlet distribution. More specifically, the mixing ratio π _t is sampled. This is because the equation (16a) and its modified equation (16b) are derived from the above equation (4), so that the speaker distribution G _t at time t is H _{1: l} generated up to time t. This is because it can be expressed only by superimposing. Here, “H _{1: l} ” represents H _{1 to} H _l .

<< Formula (11), Formula (12) >>
In dHDP approximate model, followed by the equation (11), by sampling the w _t representing the degree of time change of the speaker distribution G _t, using the w _t, v _tl defined by formula (12) (L = 1,..., T) is calculated. Here, time l represents time t and a past time.

<< Formula (13) >>
The hidden variable d _ti shown in Expression (13) is a t-dimensional {0, 1} vector (t is time, element value is 0 or 1 only). The hidden variable d _ti has a value of 1 in the l-dimensional element d _til among the t-dimensional elements at time t (l ≦ t). The l-dimensional element d _til of the hidden variable d _ti is an element corresponding to the distribution change H _l at time l. Distribution changing H _l at time l is the distribution change for sample i th sample x _ti of the time t.

≪Formula (14) ≫
Similarly, the hidden variable z _ti shown in Equation (14) is a K-dimensional {0, 1} vector.
The hidden variable z _ti has a value of 1 only in the k-th element z _tik corresponding to the cluster (parameter) k that actually samples the sample x _ti .

≪Formula (15) ≫
The observation amount x _ti shown in the equation (15) is generated from the parameter θ corresponding to the given cluster number (k). Expression (15) is the same as Expression (6) and Expression (8) described above, and is another table expression.

≪Observation distribution and prior distribution≫
In the dHDP approximation model, the observation distribution F shown in Expression (15) and the parameter prior distribution H (see Expression (9)) must be determined in advance. In the present embodiment, as an example, a Normal-Gamma distribution in which the observation distribution F is a normal distribution and the parameter prior distribution H is a conjugate prior distribution is used. The Normal-Gamma distribution is described in Reference Document 1 “CM Bishop,“ Pattern Recognition and Machine Learning ”, Springer Japan, 2007.”.

  The reason why the Normal-Gamma distribution is adopted in the present embodiment is that the approximate model solution method of dHDP using these distributions has not been published so far, and is a model that can be applied to many fields, and has the highest practicality. Because I thought. However, it is possible to make these different distributions according to the purpose and actual data.

≪Application of dHDP approximation model to dialization system≫
In the dialization system 1 of FIG. 1, in the dHDP approximation model shown in Expressions (9) to (15), the number of speakers (= number of clusters) for dialization and clustering of each sample are performed, and the parameters (= Speaker position).

Among these, clustering of each sample is equivalent to _obtaining the distribution p (z _ti ) of the hidden variable z _ti shown in Expression (14). Since the hidden variable z _ti is a variable that indicates which cluster belongs to each sample x _ti , if this clustering result is obtained, the number of samples belonging to each cluster (or its expected value) can be calculated. Further, the mixing ratio (β _k ^ described later) of each cluster k can be calculated using the number of samples belonging to each cluster (or its expected value). Further, the number of “effective” clusters K _eff (= the number of speakers K _eff ) can be determined based on the mixing ratio (β _k ^ described later). If the clustering result is obtained, the parameter {θ _k } of each cluster can be easily obtained. In this embodiment, the clustering calculation device 3 performs the determination up to the determination of the “effective” number of clusters K _eff (= the number of speakers K _eff ), but the identification unit 4 may perform this process. That is, the clustering calculation device 3 obtains a clustering result, obtains a parameter {θ _k } of each cluster, obtains a mixture ratio (β _k ^ described later) by the identification unit 4, and obtains the “effective” number of clusters K _eff (= the number of speakers K _eff ) may be determined.

[Calculation algorithm]
The calculation algorithms are as follows: 1) Online estimation method of dHDP model, 2) Variational posterior distribution inference process, 3) Identification of observation model and prior distribution, 4) Determination method of estimation result and number of clusters, 5) dHDP The details will be described separately for each speed-up method.

<1) Online estimation method of dHDP model>
Here, a specific inference algorithm is shown. FIG. 2 is a flowchart showing the overall processing flow of the clustering calculation method according to the embodiment of the present invention. FIG. 2 shows the entire inference process of clustering. Dialization is generally an online calculation process. However, the online estimation method of the dHDP model has not been studied so far. In the present embodiment, an online estimation method for the dHDP model has been developed. The inference process denoted by reference numeral 201 in FIG. 2 shows the online estimation method.

The online estimation method is performed every time T (1 ≦ T ≦ T _total ). Here, the time at a certain point of the estimation target that progresses momentarily is T, and the final time is T _total . In the following, it is assumed that the calculation target time including the past is t. It is determined in advance how far back in the past. For example, when the estimation target time T = 5, the calculation target time t = 1, 2, 3, 4, 5 or t = 3,4, 5 can be set. In this example, t = 1 is considered.

In this inference process, when processing is started, T is first initialized. That is, T = 1 is set (step S1). Then, the estimated values and hyperparameters of hidden variables and parameters (hidden variables / parameters) from time 1 to T are input (step S2). Further, the observation amount (sample) from time 1 to T-1 is input (step S3). Also inputs the voice power f _T of time T (step S4). From the voice power f _{T at} the time T, the observation amount {x _{T i} } at the time T is generated and input (step S5). Also, an unknown number (unknown hidden variable / parameter) corresponding to time T is initialized (step S6).

Then, the variational posterior distribution of hidden variables / parameters is estimated (step S7). After estimation, T is incremented. That is, T ← T + 1 is set (step S8). And it is discriminate | determined whether the input was complete | finished (step S9). If the input has not ended (step S9: No), the process returns to step S2. On the other hand, for example, when the final time T _total is exceeded, it is determined that the input is completed (step S9: Yes), and the estimation result is output (step S10). Note that the processing order of step S2 to step S4 is arbitrary, and may be processed in parallel.
Further, the processing order of step S6 is not limited as long as it is performed before step S7.

By repeating the online estimation method, observation samples {x _ti } are accumulated as the time step progresses, and the variables are re-estimated each time. The estimation result at the time T-1 is used as an initial value for estimation at the next time T.

The purpose of inference is to obtain estimates of all unknown variables ({z _ti }, {d _ti }, w, {π _t }, {θ _k }) when all observation data are given. This is equivalent to obtaining the posterior distribution of all variables from the viewpoint of the probability model. In the present embodiment, a posterior distribution estimation method (variation posterior distribution estimation method) by the variational Bayes method is shown for the dHDP approximation model. The posterior distribution to be actually obtained is Equation (17), but the variation method estimates a distribution (variation posterior distribution) that assumes that all variables are independent as in Equation (18).

Q (·) shown in Equation (18) represents a variational posterior distribution. Here, it is assumed that the observation amount up to time T is obtained. At this time, the variation posterior distribution estimated value q ^* of the variable relating to the time t (≦ T) including the past from the current time T is expressed as in Expression (19) to Expression (23). The distribution of each of these variables is corrected by adding data information from the original distribution (formula (9), formula (10), formula (11), formula (13), formula (14)). . However, Expression (19) depends on the observation model F and the parameter prior distribution H. Therefore, in the present embodiment, the following formula (36) is used based on the formula (19).

  However, since it is difficult to optimize the equations (19) to (23) at the same time, in the variational method, each variable is optimized individually by estimation by iterative calculation according to the EM algorithm (Expectation-Maximization algorithm) format. Turn into. The EM algorithm is a calculation method that simultaneously maximizes a plurality of variables, and performs overall optimization by repeatedly calculating calculation steps including an E step (Expectation step) and an M step (Maximization step). is there. The EM algorithm is described in Reference Document 1 described above.

<2) Variational posterior distribution inference process>
FIG. 3 is a flowchart showing the variational post-distribution inference procedure shown in FIG. 2 and shows an inference process including the EM algorithm. In the inference process shown in FIG. 3, first, samples {x _{1: T} } from time 1 to T are input (step S21). Further, the estimated values of the hidden variables / parameters and the hyper parameters from time 1 to T are input (step S22). Note that the processing order of steps S21 and S22 is arbitrary.

  Then, an identifier j representing the number of repetitions of the EM step is initialized. That is, j = 1 is set (step S23). Then, the E step is calculated (step S24). In step E, the estimated parameter values from time 1 to T are updated. Subsequently, the M step is calculated (step S25). In the M step, the estimated value of the hidden variable from time 1 to T is updated.

Then, the number of repetitions j of the EM step is incremented. That is, j ← j + 1 is set (step S26). Then, it is determined whether or not the current number of repetitions j has exceeded a preset threshold value (j _max ). That is, it is determined whether j> j _max is satisfied (step S27). If j ≦ j _max (step S27: No), the process returns to step S24. On the other hand, if j> j _max is satisfied (step S27: Yes), it is determined whether or not the time T has exceeded a preset setting value (a multiple of an appropriate positive integer t _update ). That is, it is determined whether or not T is “a multiple of t _update ” (step S28).

When T becomes “a multiple of t _update ” (step S28: Yes), the hyperparameter is updated (step S29), and the estimation result is stored (step S30). When T is not “a multiple of t _update ” (step S28: No), step S29 is skipped and the estimation result is stored (step S30).

In this way, the E step and the M step are repeatedly calculated over j _max times to obtain the variational posterior distribution equations (19) to (23). These are calculated in M steps.

In the present embodiment, as shown in FIG. 3, the hyper parameter is estimated every time the time step T is incremented by an appropriate positive integer t _update times by performing the processing of steps S <b> 28 to S <b> 30. Usually, the hyper parameter is a constant given in advance, but by executing the processing of steps S28 to S30, the accuracy can be improved as compared with the case where the hyper parameter is a fixed value. As a hyper parameter update method, various known methods such as a sampling method from a posterior distribution can be used. The hyperparameter itself can also be estimated by the EM algorithm.

≪E step≫
A specific calculation formula of the E step is expressed by the following formulas (24) to (28). Here, ψ (·) is a psi function (or digamma function). Expression (27) depends on the observation model F. Therefore, in the present embodiment, the following formula (37) is used based on the formula (27).

In step E, the estimated values related to the unknown parameters (θ, π, w) are recalculated. Here, E _x [f (x)] indicates an expected value on the variation distribution. The definition is shown in Equation (28).

FIG. 4 is a flowchart showing an example of a calculation procedure of the E step shown in FIG. In step E, recalculation is performed for all variables related to time T. Specifically, in step E, first, samples {x _{1: T} } from time 1 to T are input (step S31). Also, the estimated values of the hidden variables / parameters and the hyper parameters from time 1 to T, and the latest M step calculation result are input (step S32). Note that the processing order of steps S31 and S32 is arbitrary.

  Then, the calculation target time t including the past is initialized. That is, t = 1 is set (step S33). Then, the equations (24) and (25) are calculated for the time t to be calculated (step S34). Next, the cluster identifier k is initialized. That is, k = 1 is set (step S35). Then, the equation (26) is calculated for the time t and cluster k to be calculated (step S36). Further, the data identifier i at time t is initialized. That is, i = 1 is set (step S37). Then, the equation (27) is calculated for the i-th data of the cluster k at time t (step S38).

Then, the data identifier i at time t is incremented. That is, i ← i + 1 is set (step S39). Subsequently, it is determined whether or not a i> n _t (step S40). Here, n _t is the number shown in Expression (8). If a i ≦ n _t (step S40: No), the flow returns to step S38. On the other hand, when it is i> n _t (step S40: Yes), and updates the next cluster. That is, k ← k + 1 is set (step S41). And it is discriminate | determined whether it calculated about all the clusters. That is, it is determined whether or not k> K is satisfied (step S42). If k ≦ K (step S42: No), the process returns to step S36. On the other hand, when k> K is satisfied (step S42: Yes), the calculation target time t is updated. That is, t ← t + 1 is set (step S41). And it is discriminate | determined whether the time t of calculation object became the time T of estimation object. That is, it is determined whether or not t> T is satisfied (step S44). If t ≦ T (step S44: No), the process returns to step S34. On the other hand, when t> T is satisfied (step S44: Yes), the estimation result of the E step at the estimation target time T is stored (step S45).

≪M step≫
The specific calculation formula of M tep is expressed by the following formulas (29) to (32). In the M step, the estimated values related to the hidden variables (z _t , d _t ) are recalculated.

FIG. 5 is a flowchart showing an example of the calculation procedure of the M step shown in FIG. In the M step, as in the E step, recalculation is performed for all variables related to the time T. Specifically, in the M step, first, samples {x _{1: T} } from time 1 to T are input (step S51). Also, the estimated values of the hidden variables / parameters from time 1 to T, the hyper parameters, and the latest calculation result of E step are input (step S52). Note that the processing order of steps S51 and S52 is arbitrary.

  Hereinafter, description of processes similar to those in step E will be omitted as appropriate. In the M step, first, t = 1 (step S53), i = 1 (step S54), k = 1 (step S55), and then the above equations (29) and (30) are calculated (steps). S56). Thereafter, k ← k + 1 is set (step S57), and the process of returning to step S56 is repeated until k> K. When k> K is satisfied (step S58: Yes), the time identifier l is initialized. That is, l = 1 is set (step S59). Then, equations (31) and (32) are calculated for the i-th data of time t, time l, and cluster k (step S60).

Then, the time l is incremented. That is, l ← l + 1 is set (step S61). The process of returning to step S60 is repeated until all calculations for time l indicating the time from the past to time t are completed. When l> t is satisfied (step S62: Yes), i ← i + 1 is set (step S63). And the process which returns to step S55 is repeated until all the calculations about i are completed. When it becomes i> n _t (step S64: Yes), the t ← t + 1 (step S65). Further, the process of returning to step S54 is repeated until all calculations for t are completed. When t> T is satisfied (step S66: Yes), the estimation result is stored (step S67).

<3) Identification of observation model and prior distribution>
In the variational posterior distribution inference process described with reference to FIGS. 3 to 5 and equations (19) to (27), the observation model and the prior distribution are generalized. That is, the above-described equations (19) and (27) depend on the observation model F = p (x _ti | θ _k ) and the prior distribution H of the parameter θ _k . In the present embodiment, the observation model F is assumed to be a normal distribution, and a Normal-Gamma distribution is assumed as the prior distribution H. The Normal-Gamma distribution is a model represented by Expression (33) to Expression (35).

  In the case of this model, the above equations (19) and (27) are represented by equations (36) and (37), respectively. Moreover, the hyper parameter in Formula (36) and Formula (37) is represented by Formula (38)-Formula (41). Furthermore, the variables in the expressions (38) to (41) are expressed by the expressions (42a) to (42c). The calculations of equations (38) to (42) are completed in the E step.

<4) Estimation method and number of clusters determination method>
≪How to determine the number of clusters≫
In the EM algorithm, K clusters are always held, but as the estimation proceeds, only a small number of clusters have a large mixing ratio, and the size of other clusters is almost zero. The expected value of the number of samples distributed to cluster k at time t can be calculated by the definition of equation (43).

In the present embodiment, the “effective” number of clusters K _eff is determined by the ratio of the sum total of time ｔz _{t, k}時刻 defined by Expression (43). For example, the mixing ratio of each cluster k can be estimated by Expression (44a). In this rule, if the condition of Expression (44b) is satisfied, it is determined that the cluster k is an “effective” cluster.

In this rule, let the total number of such clusters k be the “effective” number of clusters K _eff . This guarantees that the “effective” number of clusters is smaller than the maximum number of clusters K, that is, K _eff <K.

≪Estimated results to be saved≫
First, the estimation result to be stored is a variational posterior distribution determined by each variable estimated by the EM algorithm. Second, the number of “effective” clusters K _eff obtained using the clustering result, ‖z _{t, k}に shown in Equation (43), β _k ＾ shown in Equation (44a), and the like. Here, ^ means a symbol added on the character β.
In particular, the estimated amount “number of effective” clusters K _eff obtained using the second clustering result as an estimation result to be stored, ‖z _{t, k} ‖ shown in Equation (43), and Equation (44a) Β _k ^ ”shown is an important estimation amount that can be used by the identification unit 4 in FIG. 1. Note that the operations of Equation (43), Equation (44a), and Equation (44b) are performed in M steps. To do in.

<5) dHDP speedup method>
The on-line estimation method in the clustering calculation method of the present embodiment is characterized in that the number of variables to be estimated increases as the time step progresses, as is apparent from the processing flow of FIG. 4 and FIG. Therefore, we developed a technique for labor saving in calculation considering real time characteristics. In this embodiment, three types of speed-up methods (speed-up 1, speed-up 2, speed-up 3) are roughly used as a speed-up method for online speaker clustering using dHDP.

≪Speedup 1: Forgetting data≫
The above equation (16a) means that the distribution change H _{l at} time 1 ≦ l ≦ t is necessary to calculate the speaker distribution G _t at time t. Therefore, since it is necessary to keep the information from time 1, the amount of inference calculation increases as time step t advances. Here, the following assumptions are introduced. That is, it is assumed that a speaker change occurs at time l <t. Then, the speaker distribution H is expected to change greatly at this time. This means that w ₁ ≈1 in the above-described equation (4), and the influence of G _l−1 is almost eliminated. When this is compared with the above-described equations (16a) and (16b), v _t1 ≈... ≈v _{l (t−1)} = 0. Therefore, in practice, it can be seen that only the distribution after the distribution change H _{l at} time l is involved in the inference of G _t . From this, it is expected that s _til (see equation (30)) corresponding to the posterior distribution of the variable {d _ti } representing the distribution selection is mostly zero.

The part where s exists in the EM algorithm is a multiplication of s and a constant or another variable, so it is not necessary to calculate the part where s _til = 0. Therefore, it is not necessary to access the variable corresponding to s. From this discussion, for the appropriate time past variables or constants than the step width W ₁ that is not accessible in EM update equation (forgetting) can reduce the computational time by.

The method of determining the time step width W _1, a method of previously have determined the appropriate constants is most convenient. Alternatively, the following method can be considered from the suggestion from this consideration. That is, there is a method of finding l where s _{ti 1} =... = S _til = 0 from the estimation result of s _ti at each time and setting W ₁ = t−l. As a method of finding such l, it is only necessary to find the maximum l satisfying the relationship of Expression (45) with respect to an appropriate threshold th (<1.0). In this case, the time width W ₁ of forgetting the data is to dynamically vary according to the inference result.

≪Speedup 2: Estimated time width limit≫
In the online estimation method shown in FIG. 2, the EM algorithm is estimated for all variables at each time T. This means that the EM re-estimation is performed many times for the variable related to the early time step. Since the EM algorithm is a calculation algorithm for a local optimum solution, when the calculation is repeated, the EM algorithm converges to one solution. Therefore, there is a high possibility that the values related to the early time step have converged without re-estimation.

From this consideration, it can be seen that the variable re-estimated by the EM algorithm can be limited to a variable in the range of T−W ₂ ≦ t ≦ T by using an appropriate time width W ₂ . This makes it possible to keep the number of variables to be constant so that the number of variables to be estimated does not increase linearly with respect to time step T. The method for determining the time width W _2, a method of previously have determined the appropriate constants is most convenient.

≪Speedup 3: Limitation of cluster variable re-estimation≫
Since dHDP is a non-parametric Bayes model, even if the maximum number of clusters (K) is prepared in the model in advance, the number of “effective” clusters K _eff (<K) is actually the number of clusters. Only the cluster becomes a substantial cluster, and the other clusters are erased because the mixing ratio is almost zero. Since there is no valid information in the deleted cluster, it is only useless to perform estimation calculation of parameters and mixing ratios using such a cluster.

  From this, in the arithmetic processing that actually performs the variable re-estimation for each cluster using the variable (k) for estimating for each cluster such as the above-mentioned formula (26), formula (27), and formula (29), It is conceivable to reduce the number of variable re-estimations stochastically (based on probability theory). The method can mention three types, for example.

As a simple method, the first reduction method is a method for determining whether to re-estimate all clusters at random with a probability c (≦ 1.0) each time.
The second reduction method is a method of increasing or decreasing the number of re-estimations by the EM algorithm in accordance with the mixture ratio β _k ^ (see equation (44)) of each cluster k.
The third reduction method re-estimates every time when the condition of the above-described equation (44b) is satisfied, that is, when the target cluster is an “effective” cluster, but in other cases, the cluster k Is a method of updating the probability.
In these reduction methods, if the probability of updating each cluster k is set as p _update (k), the first to third reduction methods can be expressed as Equations (46) to (48), respectively. In particular, the third reduction method has an effect of reducing the calculation amount to about K _eff / K without sacrificing estimation accuracy.

  An example of the EM algorithm in the case of using all the three speed-up methods (speed-up 1, speed-up 2, speed-up 3) is shown in FIGS. 6 and 7, respectively. These three methods can be used independently of each other.

FIG. 6 is a flowchart showing a calculation procedure of the E step in the clustering calculation method according to the embodiment of the present invention. In addition, a different part is shown with a thick line and a broken line compared with the flowchart of FIG. 4, description is abbreviate | omitted suitably and a different process is demonstrated. In FIG. 6, the high speed 1 includes steps S <b> 31 </ b> A and S <b> 32 </ b> A as indicated by reference numeral 301. Here, samples {x _{T-W1: T} } from time T-W ₁ to T are input (step S31A). Also, the estimated values of the hidden variables / parameters and the hyper parameters from time T-W ₁ to T, and the latest M step calculation result are input (step S32A). Next, the speed-up 2 includes step S33A as indicated by reference numeral 302. Here, the initial value of t is set to t = TW ₂ instead of 1 (step S33A).

Furthermore, the speed-up 3 is processing performed subsequent to step S35, and includes steps S71 and S72 as indicated by reference numeral 303. Here, a uniform random number u of [0, 1] is generated (step S71), and it is determined whether or not p _update (k)> u is satisfied (step S72). When p _update (k)> u is satisfied (step S72: Yes), the above-described steps S36 to S45 are executed. That is, the above-described equations (26) and (27) are calculated. However, if k ≦ K in step S42 (step S42: No), the process returns to step S71. On the other hand, when p _update (k) ≦ u is satisfied in step S72 (step S72: No), the above-described steps S36 to S40 are skipped and the process proceeds to step S41. In other words, the above-described equations (26) and (27) are not calculated.

FIG. 7 is a flowchart showing a calculation procedure of M steps in the clustering calculation method according to the embodiment of the present invention. Note that different parts compared to the flowchart of FIG. 5 are indicated by thick lines and broken lines, and description thereof will be omitted as appropriate. In FIG. 7, the high speed 1 includes steps S <b> 51 </ b> A and S <b> 52 </ b> A as indicated by reference numeral 401. Here, samples {x _{T-W1: T} } from time T-W ₁ to T are input (step S51A). Further, the estimated values of the hidden variables / parameters and the hyper parameters from the time T-W ₁ to T, and the latest calculation result of the E step are input (step S52A). Next, the speed-up 2 includes step S53A as indicated by reference numeral 402. Here, the initial value of t is set to t = TW ₂ instead of 1 (step S53A).

Further, the speedup 3 is a process performed subsequent to step S55, and includes steps S81 and S82 as indicated by reference numeral 403. Here, a uniform random number u of [0, 1] is generated (step S81), and it is determined whether or not p _update (k)> u is satisfied (step S82). When p _update (k)> u (step S82: Yes), the above-described steps S56 to S67 are executed. That is, the above-described equations (29) to (32) are calculated. However, if k ≦ K in step S58 (step S58: No), the process returns to step S81. On the other hand, when p _update (k) ≦ u is satisfied in step S82 (step S82: No), the above-described step S56 is skipped and the process proceeds to step S57. In other words, the above-described equations (29) and (30) are not calculated.

[Clustering calculator]
FIG. 8 is a functional block diagram showing an example of the configuration of the clustering calculation apparatus according to the embodiment of the present invention. The clustering calculation device 3 implements, for example, the inference process of FIG. 2 and the like, and includes information from the outside such as a calculation device such as a CPU, a storage device (storage means) such as a memory and a hard disk, and a mouse and a keyboard. An input device for detecting the input of a computer, an interface device for transmitting / receiving various information to / from the outside, a computer having a display device such as an LCD (Liquid Crystal Display), and a program installed in the computer .

  The clustering calculation device 3 is realized by the hardware device and software cooperating to control the above-described hardware resources by a program. As shown in FIG. As functions, a variational posterior distribution inference unit 30, a power vector reading unit 21, an input control unit 22, an allocation unit 23, an unknown number initialization unit 24, a power vector writing unit 25, and an observation amount generation unit 26 And an end determination unit 27 and an output control unit 28.

<Storage means>
The storage unit 10 includes a ROM, a RAM, an HDD, and the like. The storage means 10 is divided into a program storage area, a setting data storage area, a calculation data temporary storage area, an estimation result storage area, and the like, and stores various information such as commands, data, programs, and the like. For example, as shown in FIG. 9, in the estimation result storage area, as shown in FIG. 9, unknown posterior distribution estimated value 11, hyperparameter 12, estimated value in E step (collectively expressed as E step 13), in M step An estimated value (collectively expressed as M step 14), a power vector 15, and an observation amount (sample) 16 are stored.

<Variational posterior distribution reasoning section>
The main variational posterior distribution inference unit 30 implements, for example, the inference process of FIGS. 3, 6, and 7. Here, the E step calculation unit 31, the M step calculation unit 32, The EM convergence determination unit 33, the parameter update condition determination unit 34, and the hyper parameter update unit 35 are provided. Details will be described later.

<Power vector reading part>
The power vector reading unit 21 sequentially reads the power vector f _T according to the estimation target time T and passes it to the input control unit 22.

<Input control unit>
When acquiring the power vector f _T , the input control unit 22 sends the power vector f _T to the power vector writing unit 25 as it is. Output each necessary command.

<Allocation unit>
According to the command from the input control unit 22, the allocating unit 23 calculates the above-described equation according to T (= 1, 2,...) And i corresponding to the angle d (180, −179,...). The unknown numbers (parameters / hidden variables) of (9) to (14) are sequentially generated (initial value is 0, for example), and stored in the storage means 10 as the posterior distribution estimated value 11 of the unknown number. That is, the assigning unit 23 assigns T and i to the unknown to be estimated. In this sense, in FIG. 9, the assignment unit 23 of T and i is indicated. The storage unit 10 stores in advance initial values of the hyper parameters 12 used in the dHDP approximate model.

<Unknown number initialization part>
The unknown number initialization unit 24 uses the above-described equations (24) to (26) for use in the E step calculation according to the command from the input control unit 22 and i corresponding to T and the angle d at that time. , (37) are sequentially generated (the initial value is 0, for example), and stored in the storage means 10 as the initial value of E step 13. The unknown number initialization unit 24 sequentially generates the left side parameters of the equations (29) to (32) to be used for the M step calculation (initial value is 0, for example). Save as value. Further, the unknown number initialization unit 24 sequentially generates the left side parameters of the expressions (36) and (20) to (23) to be used for the calculation of the EM estimated value (the initial value is 0, for example). The posterior distribution estimated value 11 of unknown number is overwritten and saved (updated).

<Power vector writing unit>
The power vector writing unit 25 sequentially stores the power vector f _T acquired from the input control unit 22 as the power vector 15 in the storage unit 10.

<Observation generator>
In accordance with a command from the input control unit 22, the observation amount generation unit 26 determines the power value for each angle d of the data of the power vector 15 read from the storage means 10 according to a predetermined rule according to T at that time. Then, an observation amount (sample) is generated by converting the data into identifier i (number n _t ), and sequentially stored as the observation amount 16 in the storage unit 10. In the present embodiment, the above-described formulas (7) and (8) are used as the predetermined rules.

<End determination unit>
End determining unit 27, when the input signal from the input control unit 22 (command) is interrupted a predetermined period, it is determined that the input of the power vector f _T is completed, it notifies the output control unit 28. In the present embodiment, it is determined that the input is completed when the final time T _total is reached.

<Output control unit>
When the output control unit 28 receives the end notification, the output control unit 28 obtains an unknown posterior distribution estimated value 11 from the storage unit 10 as a final estimated value, and outputs it to the identifying unit 4.

[Details of variational posterior distribution reasoning section]
≪E step calculation section≫
The E step calculation unit 31 reads the sample, the hyper parameter, the calculation result of the M step, and the like from the storage unit 10 regarding the calculation target time t (t ≦ T) including the past according to T at the time of processing. Then, the E-step calculation unit 31 calculates the expressions (24) to (26) and (37) with respect to all the calculation target times t (t ≦ T) including the past according to T at the processing time point, The calculation result is stored as E step 13 in the storage means 10.

≪M step calculation part≫
The calculation unit for M step 32 reads the sample, the hyper parameter, the calculation result of the E step, and the like from the storage unit 10 regarding the calculation target time t (t ≦ T) including the past according to T at the time of processing. Then, the M step calculation unit 32 calculates equations (29) to (32) for all the calculation target times t (t ≦ T) including the past according to T at the time of processing, and calculates the calculation results. Then, it is stored in the storage means 10 as M step 14. The M-step calculating unit 32 uses all of the calculation results stored in the storage unit 10 and uses all the calculation target times t (1 ≦ t ≦ T) including the past according to T at the time of processing. ), The formulas (20) to (23) and (36) are calculated, and the calculation results are overwritten and saved (updated) in the storage means 10 as the unknown posterior distribution estimated value 11.

≪EM convergence judgment part≫
The EM convergence determination unit 33 determines whether or not the EM algorithm has converged by determining whether or not the threshold (j _max ) of the number j of repetitions of a set of processing including the E step and the M step has been reached. Is determined. The threshold value (j _max ) is set in advance. When the EM algorithm has not converged, the EM convergence determination unit 33 performs control to repeat the E step and the M step. When the EM algorithm has converged, the EM convergence determination unit 33 performs control to stop the processing of the E step and the M step. In the present embodiment, the EM convergence determination unit 33 notifies the parameter update condition determination unit 34 of the value of T at that time regardless of the convergence.

≪Parameter update condition judgment unit≫
The parameter update condition determination unit 34 determines whether or not T at the received processing time is the same as a preset setting value (a multiple of an appropriate positive integer t _update ). If they are the same, the parameter update condition determination unit 34 notifies the hyper parameter update unit 35.

≪Hyper parameter update part≫
When receiving the notification, the hyper parameter update unit 35 updates the hyper parameter 12 stored in the storage unit 10 by an arbitrary method. As a result, the E-step calculation unit 31 and the M-step calculation unit 32 use the updated hyperparameter from the next timing when T at the time of processing is the same as a preset appropriate multiple of a positive integer. be able to. The parameter update condition determination unit 34 and the hyper parameter update unit 35 correspond to the processes in steps S28 to S30 described above. However, these configurations are not necessarily provided.

  Note that the clustering calculation device 3 can be realized by operating a general computer by a program (clustering calculation program) that functions as each of the above-described means constituting the clustering calculation device 3. This program can be provided via a communication line, or can be written on a recording medium such as a CD-ROM and distributed. The computer in which this program is installed can achieve the same effect as the clustering calculation apparatus 3 by the CPU developing this program stored in the ROM or the like in the RAM.

According to this embodiment, the speakers in dialization that estimate the number of speakers who participated in the conversation, the positions of each speaker, and the timing of each speaker's utterance behavior from recorded data consisting of conversations of a plurality of speakers. Since probabilistic clustering is used for the clustering problem, the number of speakers can be easily estimated without using conventional parameter setting or searching.
Further, according to the present embodiment, by adopting the dHDP approximate model, it is possible to appropriately model the situation in which the speakers participating in the utterance change with time. As a result, more accurate speaker clustering can be realized.
Furthermore, according to the present embodiment, it is possible to perform inference at high speed by using the online estimation method of dHDP and its speed-up method. Real-time inference is also possible by a trade-off between accuracy and time.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to this, It can implement in the range which does not change the meaning. For example, the clustering calculation apparatus 3 assumes the Normal-Gamma distribution in the dHDP approximate model, but may assume another distribution. In this case, a calculation formula similar to the above-described formula (36), formula (37), and related relational formula may be derived again with another assumed distribution.

  In the present embodiment, the clustering calculation device 3 is inferred from the dHDP approximation model, but may be a dHDP model or other nonparametric Bayes model, for example. In the case of other non-parametric Bayes models, a distribution obtained by modeling a continuous temporal change of a probability distribution is preferable.

  In the present embodiment, in order to perform clustering based on the position of the speaker, the feature amount extraction unit 2 extracts DOA information (speech arrival angle) using a microphone array as an example. If it is suitable for dialization, various other feature quantities can be extracted. For example, in the case of clustering based on the voice quality feature for each speaker, an MFCC feature amount (Mel Frequency Cepstrum Coefficient) or the like can be extracted. Even such a feature amount can be applied to the algorithm described above.

  In order to confirm the effect of the present invention, the performance of the clustering calculation apparatus according to the present embodiment was measured. First, as a first stage experiment, an experiment (clustering verification experiment) was performed to verify whether or not clustering having the number of clusters as designed and appropriate parameters can be realized by dHDP. Next, as a second stage experiment, an experiment (evaluation of dialization accuracy by DER) for evaluating the dialization accuracy using the obtained cluster result was performed. In the two types of experiments, the performance of the clustering calculation apparatus was confirmed using artificial voice data and real voice data.

<Experimental data>
≪Artificial voice data≫
Artificial voice data is data that simulates a situation in which three speakers switch between speaking and non-speaking in turn. This artificial voice data is composed of DOA data (voice arrival angle data) calculated at a time step of 64 [msec] and voice / non-voice judgment results by VAD (voice section detector). Artificial voice data is data on which almost no noise is superimposed. In each experiment, the non-speech section was excluded by threshold processing using the determination result by VAD, and artificial speech data of a continuous sequence of 422 steps was created. In addition, in order to stabilize the sample distribution for each time step to some extent, the 422 steps are combined every few steps without duplication to form one long meta step. In each experiment, 5 steps of data were combined into 1 metastep, and the number of metasteps corresponded to t and T. Therefore, the number of steps is _{T total = ceil (422/5) =} 85. Here, ceil indicates rounding up. In the artificial voice data, there is a section where a plurality of speakers speak simultaneously.

  FIG. 9 shows the distribution of the average power vector at all times for the artificial voice data. In FIG. 9, the horizontal axis indicates DOA data, that is, the voice arrival direction angle [deg], and the vertical axis indicates time average power [dB]. As shown in FIG. 9, three peaks of the power distribution corresponding to each speaker can be observed.

≪Real voice data≫
The actual voice data is data that records the actual conversations of multiple speakers. Four data described in Non-Patent Document 1 were used as actual voice data. Details of the four data are shown in Table 1. In Table 1, CP represents crossword puzzle data, DC represents discussion data, and CN represents conversation data.

  The actual voice data is 300 seconds of voice data. The distribution of the average power vector for these actual voice data is shown in FIG. The horizontal and vertical axes in FIG. 10 are the same as those in the graph of FIG. However, the order of time average power is low.

  Regarding CP1 shown in FIG. 10A and CP2 shown in FIG. 10B, the number of peaks of the power distribution corresponding to the number of speakers “4” shown in Table 1 is also “4”. Good clustering results are expected. On the other hand, for the DC shown in FIG. 10C and the CN shown in FIG. 10D, the three peaks of the power distribution corresponding to the number of speakers “3” shown in Table 1 cannot be clearly observed. The estimation of the correct number of clusters is expected to be difficult.

[Clustering verification experiment]
First, as a first step, the performance of clustering was confirmed. Here, speaker clusters were estimated online using DPM (Reference Example 1) and dHDP (Example 1). In this clustering verification experiment, the final clustering result was obtained by counting only the clusters whose final mixing ratio exceeded the chance level (1 / K) at the final time T _total as valid clusters. Then, based on the obtained clustering result, DPM (Reference Example 1) and dHDP (Example 1) are compared to determine whether or not a clustering result corresponding to the distribution of speakers and the number of speakers is obtained. . The chance level is the probability that a coincidence will occur.

≪For artificial voice data≫
FIG. 11 shows the result of applying DPM (Reference Example 1) and dHDP (Example 1) online to the artificial voice data shown in FIG. In FIG. 11, the horizontal axis represents an angle normalized to a dimensionless amount using a function for converting DOA data from [−180: 180] → [−0.5: 0.5], that is, a normalized angle. Indicates. The vertical axis represents the probability density function (probabilistic density function: pd f) value (dimensionless number).

  As a result of comparison between the result of dHDP (Example 1) shown in FIG. 11B and the graph of FIG. 9, it can be seen that the correct number of clusters and parameters can be obtained in dHDP. On the other hand, in the case of DPM (Reference Example 1) shown in FIG. 11A, the number of clusters was “1”, and it was confirmed that an inappropriate result was obtained for the artificial voice data shown in FIG.

<< In case of real voice data (CP1, CP2) >>
FIG. 12 shows the result of applying DPM (Reference Example 2) and dHDP (Example 2) online to the actual voice data (CP1) shown in FIG. Each axis in FIG. 12 is the same as the graph in FIG. As a result of comparison between the result of dHDP (Example 2) shown in FIG. 12B and the graph of FIG. 10A, it can be seen that the correct number of clusters and parameters can be obtained in dHDP. On the other hand, in the case of DPM (Reference Example 2) shown in FIG. 12A, it was divided into a large number of clusters, and it was confirmed that an inappropriate result was obtained for CP1 shown in FIG. . A similar trend was confirmed for CP2 data. The description of the results of DPM (Reference Example 3) and dHDP (Example 3) at this time was omitted.

≪In case of real voice data (DC) ≫
FIG. 13 shows the result of applying DPM (Reference Example 4) and dHDP (Example 4) online to the actual voice data (DC) shown in FIG. Each axis in FIG. 13 is the same as the graph in FIG. As a result of comparison between the result of dHDP (Example 4) shown in FIG. 13B and the number of speakers in the graph of FIG. 10C and Table 1, three clusters that are the number of speakers are obtained in dHDP. I couldn't. However, among the clusters, the positions of the normalized angles of “cluster 4”, “cluster 6”, and “cluster 14”, which are the top three of the number of sizes (cluster size in the figure), are the speakers in the DC data. It was possible to correspond to the position of. Here, the size number (cluster size in the figure) was defined by a numerical value indicated by the denominator on the right side of the above-described equation (44). Note that the lower two of the size numbers were noise clusters.

  On the other hand, in the case of the DPM (Reference Example 4) shown in FIG. 13A, the cluster is divided into a larger number of clusters, and the top three clusters in the number of sizes cannot correspond to the positions of the speakers. From this point, it is considered that dHDP (Example 4) can realize more accurate clustering compared to clustering by DPM (Reference Example 4). In the case of DPM (Reference Example 4), the positions of the normalized angles of “cluster 2”, “cluster 4”, and “cluster 5”, which are the top three sizes in each cluster, Only two of the speaker locations could be handled. In addition, the 4th to 6th positions could not correspond to another speaker position. Also, CN data is considered to have the same tendency as DC data.

[Evaluation of dialization accuracy by DER]
Following the clustering verification experiment in the first stage, in the second stage, an evaluation by DER (diarization error ratio) was attempted in order to evaluate the performance as clustering for dialization. DER is an index of speaker identification ability proposed by NIST. Specifically, the DER indicates the percentage of the following three types of misidentification sections (1) to (3) with respect to the total speech section length. The smaller the DER value, the better the dialization.

(1) false alarm speaker time: the length of the section that was falsely detected that someone was speaking when no one was speaking (2) missed speaker time: the section where no one was speaking when someone was speaking Long (3) speaker error time: It is detected correctly that someone is speaking, but the speaker is wrong

Details of DER are described in the following URL.
“NIST Speech group,“ Spring2007 (RT-07) Rich Transcription Meeting Recognition Evaluation Plan ”, [online], [searched on January 21, 2009], Internet <URL: http://www.nist.gov/speech /tests/rt/2007/index.html>"

Clustering by dHDP in the first stage is only a sub-problem of dialization, and speaker identification cannot be performed as it is. However, in the clustering by dHDP, the samples at each time (= directed sound power data) are clustered. Therefore, if the number of samples assigned to each cluster in each frame (time step) is counted, a predetermined threshold value is set. By using it, it is possible to determine utterance or non-voicing for each speaker. Therefore, the utterance or non-utterance of the speaker k at each time is determined according to the rules shown in Expression (49) and Expression (50) using ‖z _{t, k}定義 defined in Expression (43). Note that τ _DER shown in Expression (49) as a predetermined threshold was set to an appropriate value. Table 2 shows the DER results calculated for dHDP (Example 5 and Example 6) for each real voice data shown in Table 1. This DER value was compared with the result in Non-Patent Document 1 (comparative example).

In Table 2, the comparative example shows the best result reported in Non-Patent Document 1, which is an existing method. Of the results (Examples 5 and 6) of the dHDP clustering of the present invention, Example 5 (naive) shows a value when DER is calculated only by the rules shown in Expressions (49) and (50). . In addition, in Example 6 (heuristic), in addition to the rules (identification rules) shown in Equation (49) and Equation (50), an upper limit is assumed for the number of simultaneous utterances within one frame (within one hour step). In this embodiment, false alarms in non-speech intervals are reduced. The method employed in Example 6 is in common with the discrimination rule obtained by the best method in Non-Patent Document 1. That is, Example 6 is the result of searching based on the sample number threshold τ _DER shown in Expression (49) and the assumption of the upper limit of the number of simultaneous speakers. Therefore, Example 6 gave better results than Example 5. Furthermore, it can be seen that Example 6 clearly shows a good value as the DER value even when compared with the comparative example in which the upper limit of the number of simultaneous speakers is assumed.

  Summarizing the above verification experiment and evaluation experiment, it is clustering that is a sub-problem (first stage) that is solved by dHDP, but in the second stage, sample assignment z calculated in the clustering process is used. Thus, it is possible to improve the DER index indicating the accuracy of dialization. That is, according to the present invention, it can be concluded that a good dialization can be naturally achieved by solving the clustering problem with dHDP.

1 Dialization System 2 Feature Extraction Unit (Feature Extraction Means)
3 Clustering calculation device 4 Identification part (identification means)
10 storage means (estimated value storage means, observation amount storage means)
21 Power vector reading part (reading means)
22 Input control unit 23 Allocation unit 24 Unknown number initialization unit 25 Power vector writing unit 26 Observation amount generation unit (observation amount generation means)
27 End determination unit 28 Output control unit (output control means)
30 Variational post-distribution reasoning section
31 E step calculation unit (E step calculation means)
32 M step calculation unit (M step calculation means)
33 EM convergence determination unit (convergence determination means)
34 Hyper parameter update condition determination unit 35 Hyper parameter update unit

Claims (10)

In order to estimate the number of speakers of the conversation from the recording data of the conversation whose number of speakers is unknown, feature amount extraction means for extracting a feature amount that characterizes each speaker, and each story from the extracted feature amount A clustering calculation device for estimating a plurality of unknown parameter values when generating a plurality of clusters corresponding to a speaker, and an identification means for identifying each speaker of the conversation based on the estimated plurality of parameter values. The clustering calculation device of the dialization system,
Reading means for reading the extracted feature value;
Observation amount generating means for converting the read feature amount into an observation amount of a vector corresponding to a nonparametric Bayes model;
Observation amount storage means for accumulating and storing the set of converted observation amounts;
Posterior distribution inference means for estimating and updating the posterior distribution values of a plurality of parameters when generating a plurality of clusters from the aggregated data of the observation amount by a nonparametric Bayes model, respectively, using an EM algorithm;
Estimated value storage means for accumulating and storing values of posterior distributions of the estimated and updated parameters;
Output control means for outputting a latest estimated value of the posterior distribution of the plurality of parameters stored in the estimated value storage means when a preset termination condition is satisfied;
A clustering calculation apparatus comprising: The posterior distribution inference means is
As the processing of the E step of the EM algorithm, conversion is performed from the past to the estimation target time, the pre-distribution, observation distribution and maximum number of clusters determined in advance in the dHDP (dynamic Hierarchical Dirichlet Process) model, the set value of the hyperparameter Read the aggregate data of the observed quantity and the estimated value of the posterior distribution of the hidden variable estimated from the past to the latest M steps, and for each calculation target time including the past, for each cluster, and E step calculation means for calculating a value of a posterior distribution of a parameter related to the cluster, a mixture ratio, and a weight indicating a degree of temporal change of the cluster distribution, for all data for each calculation target time;
As the processing of the M step of the EM algorithm, the set value of the hyper parameter, the collective data of the observation amount converted from the past to the estimation target time, and the posterior of the parameter estimated from the past to the latest E step The estimated values of the distribution are read in, and the values of the posterior distributions of the two types of hidden variables are estimated for each calculation target time including the past and for every data for each calculation target time. Among the variables, the value of the posterior distribution of the first hidden variable associated with the weight representing the degree of temporal change of the cluster distribution is calculated for each cluster, and the first hidden variable and the mixture ratio associated with the mixture ratio are calculated. For the value of the posterior distribution of the two hidden variables, M step calculation means for calculating each time retroactively from the calculation target time,
Convergence determining means for performing control to repeatedly execute the process of the E step and the process of the M step alternately a predetermined number of times;
The clustering calculation apparatus according to claim 1, further comprising: The calculation means for the E step is:
For the collective data of observations and the estimated value of the posterior distribution of hidden variables, the time of retroactively reflecting the number of fluctuations reflecting the previously set constant or the estimated value of the previous M step from the estimation target time The data is read, and the posterior distribution values of the parameters related to the cluster, the mixture ratio, and the weight are calculated as the calculation target time from the estimation target time to the past time set in advance by the set value set in advance. Whether to re-estimate the target cluster for estimation based on the preset criteria for re-estimating the cluster each time it performs a calculation and updates the cluster for each cluster operation If the cluster is to be discriminated and re-estimated, the value of the posterior distribution of the parameter related to the mixture ratio is calculated,
The M step calculating means includes:
For the aggregated data of the observed quantities and the estimated values of the posterior distribution of parameters, the data of the time retroactively reflected by the number of fluctuations reflecting the constants set in advance from the estimated time or the estimated value of the previous M step And calculating the values of the posterior distributions of the first hidden variable and the second hidden variable from the estimation target time to the past set time set in advance as the calculation target time, for each cluster Each time the process of updating the cluster for the calculation of is performed, it is determined whether or not the estimation target cluster is to be reestimated based on the determination criterion. The clustering calculation apparatus according to claim 2, wherein a value of a posterior distribution of one hidden variable is calculated. In a dialization system for estimating the number of speakers of the conversation from conversation recording data whose number of speakers is unknown, storage means, reading means, observation generation means, posterior distribution inference means, output control means, And a clustering calculation of a clustering calculation device for estimating values of a plurality of unknown parameters when generating a plurality of clusters corresponding to each speaker from a feature amount characterizing each speaker extracted from the recording data A method,
A feature amount reading step of reading the extracted feature amount by the reading means;
An observation amount accumulation step of converting the read feature amount into a vector observation amount corresponding to a non-parametric Bayes model by the observation amount generation unit, and sequentially storing the converted set of observation amounts in a storage unit;
The posterior distribution inference means estimates the posterior distribution values of a plurality of parameters when generating a plurality of clusters from the observation data set by a non-parametric Bayes model using an EM algorithm, and stores the estimated values in the memory A posteriori distribution estimation step for sequentially storing and updating the means;
An estimated value output step for outputting a latest estimated value of the posterior distribution of the plurality of parameters stored in the storage means when a preset end condition is satisfied by the output control means;
A clustering calculation method comprising: The posterior distribution inference means is
In the posterior distribution estimation step,
As the processing of the E step of the EM algorithm, a set of a prior distribution, observation distribution and maximum number of clusters determined in advance in the dHDP model, a set value of the hyperparameter, and an observation amount converted from the past to the estimation target time Read the data and the estimated value of the posterior distribution of the hidden variable estimated from the past to the latest M step, for each calculation target time including the past in the estimation target time, for each cluster, and Calculating the value of the posterior distribution of the parameter related to the cluster, the mixture ratio, and the weight representing the degree of temporal change of the cluster distribution for every data for each calculation target time;
As the processing of the M step of the EM algorithm, the set value of the hyper parameter, the collective data of the observation amount converted from the past to the estimation target time, and the posterior of the parameter estimated from the past to the latest E step The estimated value of the distribution is read, and the value of the posterior distribution of the two types of hidden variables is estimated for each calculation target time including the past in the estimation target time and for each data for each calculation target time. Of the two types of hidden variables, the value of the posterior distribution of the first hidden variable associated with the weight representing the degree of temporal change of the cluster distribution is calculated for each cluster, and the first hidden variable and the mixture The value of the posterior distribution of the second hidden variable associated with the ratio includes calculating each time retroactively from the calculation target time,
The clustering calculation method according to claim 4, wherein the processing of the E step and the processing of the M step are alternately and repeatedly executed a predetermined number of times. The posterior distribution inference means is
In the E step, the observation amount set data and the estimated value of the posterior distribution of the hidden variable are the number of fluctuations reflecting the constant set in advance from the estimation target time or the estimated value of the immediately preceding M step. Read the data of retroactive time,
In the M step, the observation amount set data and the estimated value of the posterior distribution of the parameter are past by the number of fluctuations reflecting the constant set in advance from the estimation target time or the estimated value of the immediately preceding M step. The clustering calculation method according to claim 5, wherein data at a time retroactively read is read. The posterior distribution inference means is
In the E step, the value of the posterior distribution of the parameters relating to the cluster, the mixture ratio, and the weight is calculated from the estimation target time to a past time that is retroactive by a set value set in advance. And
In the M step, calculating the values of the posterior distributions of the first hidden variable and the second hidden variable with the calculation target time from the estimation target time to the past set time set in advance. The clustering calculation method according to claim 5, wherein: The posterior distribution inference means is
Whether the estimation target cluster should be re-estimated based on a preset criterion for re-estimation of the cluster every time the process of updating the cluster for the computation for each cluster is executed in the E step. Only if it is a cluster to be re-estimated, calculate the value of the posterior distribution of the parameter related to the mixture ratio,
In the M step, each time a process for updating a cluster is performed for computation for each cluster, it is determined whether or not a cluster to be estimated is to be reestimated based on the determination criterion, and a cluster to be reestimated The clustering calculation method according to claim 5, wherein the value of the posterior distribution of the first hidden variable is calculated only when   The clustering calculation program for functioning a computer as each means which comprises the clustering calculation apparatus as described in any one of Claims 1 thru | or 3.   A computer-readable recording medium on which the clustering calculation program according to claim 9 is recorded.

Images