JP6538624B2 - Signal processing apparatus, signal processing method and signal processing program - Google Patents

Description

  The present invention relates to a signal processing device, a signal processing method, and a signal processing program.

  2. Description of the Related Art Conventionally, a sound source localization technique is known which estimates the position of a sound source generating the sound based on the recorded sound observed by a plurality of microphones or the like. As a sound source localization technique, for example, there is known a method of estimating a sound source position using a covariance matrix estimated by applying time-frequency analysis to an observation signal, assuming that the number of sound sources is known.

R. O. Schmidt, "Multiple emitter location and signal parameter estimation," IEEE Transactions on Antennas and Propagation, March 1986, vol. AP-34, No. 3, p. N. Ito, E. Vincent, N. Ono, R. Gribonval, and S. Sagayama, "Crystal-MUSIC: Accurate localization of multiple sources in diffuse noise environments using crystal-shaped microphone arrays," September, 2010, Proceedings of 9th International Conference on Latent Variable Analysis and Signal Separation (LVA / ICA), p. 81-88.

  However, the conventional sound source localization technology has a problem that when the observation signal length is short, the sound source localization may not be performed accurately. For example, when the observation signal length is short, it may not be possible to obtain sufficient samples for estimation of the covariance matrix, and the sound source localization can not be performed accurately.

  A signal processing apparatus according to the present invention applies time-frequency analysis to the recorded sound acquired at a plurality of different positions, and calculates an observation signal vector which is an M-dimensional vector by the time-frequency analysis unit; A feature vector calculation unit that calculates, for each time frequency point, a feature vector that is a vector including information on the direction of the observed signal vector y (t, f), and a plurality of sound source position candidates in which the state representing the sound source position is A parameter storage unit for storing model parameters of the conditional probability distribution of the feature vector under conditions corresponding to the respective conditions; and the parameter storage unit using an a priori probability distribution of a state representing the sound source position as a load The weight of the conditional probability distribution of the feature vector under conditions where the state representing the sound source position is known, based on the model parameters stored in To a prior probability distribution calculating unit that applies the mixed model that is the feature vector to the feature vector calculated by the feature vector calculating unit, and calculates the prior probability distribution, and the prior probability distribution calculated by the prior probability distribution calculating unit And a sound source position calculating unit that calculates a sound source position corresponding to the feature vector.

  A signal processing method according to the present invention is a signal processing method executed by a signal processing apparatus, which applies time-frequency analysis to recorded sounds acquired at a plurality of different positions, and calculates an observation signal vector which is an M-dimensional vector. Time-frequency analysis process, and a feature vector calculation process for calculating a feature vector which is a vector including information on the direction of the observed signal vector y (t, f) calculated by the time-frequency analysis process for each time frequency point And the model parameter stored in the parameter storage unit storing the model parameter of the conditional probability distribution of the feature vector under the condition that the state representing the sound source position corresponds to each of the plurality of sound source position candidates. A state representing the sound source position obtained based on the model parameter, which is obtained and obtained as a load is the prior probability distribution of the state representing the sound source position A prior probability distribution calculation for calculating the prior probability distribution by fitting a mixed model which is a weighted sum of the conditional probability distributions of the feature vectors under knowledge conditions to the feature vectors calculated by the feature vector calculating step And sound source position calculating step of calculating a sound source position corresponding to the feature vector based on the step and the prior probability distribution calculated by the prior probability distribution calculating step.

  According to the present invention, even when the observation signal length is short, sound source localization can be performed accurately.

FIG. 1 is a diagram for explaining sound source localization in the present invention. FIG. 2 is a diagram showing an example of the configuration of the signal processing device according to the first embodiment. FIG. 3 is a flowchart showing the flow of processing of the signal processing device according to the first embodiment. FIG. 4 is a diagram showing an example of the configuration of a signal processing device according to the eighth embodiment. FIG. 5 is a view showing an example of the configuration of a signal processing apparatus according to the ninth embodiment. FIG. 6 is a diagram showing an example of the configuration of a signal processing apparatus according to the tenth embodiment. FIG. 7 is a diagram showing an example of the configuration of a signal processing apparatus according to the eleventh embodiment. FIG. 8 is a diagram illustrating an example of a computer in which a signal processing apparatus is realized by executing a program.

  Hereinafter, embodiments of a signal processing device, a signal processing method, and a signal processing program according to the present application will be described in detail based on the drawings. The present invention is not limited by this embodiment.

[About sound source localization in the present invention]
Since the sound source signal is usually sparse with a large power only at a sparse point on the time frequency plane, the observation signal is one of the sound source signals at each time frequency point even in a situation where multiple sound source signals are sounding simultaneously. It can be regarded as including at most one. Therefore, for example, an observation signal vector y (t, f) (t is a frame number (t = t), which is an M-dimensional longitudinal vector consisting of time-frequency transformation of observation signals acquired at M (M> 1) different positions. 1 to T) and f are the frequency bin numbers (f = 1 to F) are directed in the specific direction determined by the sound source position of the sound source signal included in the observation signal at the time frequency point (t, f) It can be regarded as To be precise, due to the influence of noise and reverberation, the direction of the observation signal vector y (t, f) is distributed with some spread around the specific direction determined by the above sound source position. By utilizing the above-described properties of the observation signal, the sound source position can be estimated based on the direction of the observation signal vector y (t, f).

  In the embodiment of the present invention, as for sound source localization, a problem of specifying one of a plurality of (L) sound source position candidates that actually emits a sound, that is, (number of sound source position candidates that actually emits a sound) Formulated as a problem of estimating the set of As this sound source position candidate, for example, a plurality of places in the room where sound source localization is performed (for example, places corresponding to respective grid points when the inside of the room is finely divided in a grid shape) may be used as the sound source position candidate. it can. Further, when the area where the sound source may exist is known, the sound source position candidate can set a plurality of places in the area as the sound source position candidate. For example, when performing sound source localization for the recorded sound of a plurality of conversations sitting around a table, it can be considered that a speaker who is a sound source can exist only near the outer periphery of the table, so multiple locations near the outer periphery of the table As a sound source position candidate (see FIG. 1). Therefore, based on the above-described properties of the observation signal, in the embodiment of the present invention, the sound source of the feature vector z (t, f) which is a vector including information on the direction of the observation signal vector y (t, f) A model parameter of conditional probability distribution under a condition that takes a state corresponding to each of a plurality (L) of sound source position candidates representing a position is stored, and the sound source position is estimated using the model parameter as prior information Use for As described above, since the direction of the observed signal vector y (t, f) can be considered to be determined by the sound source position, the feature vector z which is a vector including information on the direction of the observed signal vector y (t, f) (T, f) has an inherent probability distribution determined by the sound source position. The conditional probability distribution represents the probability distribution of the feature vector z (t, f) under the condition that the state representing the sound source position takes a state corresponding to each of a plurality of (L) sound source position candidates.

  The direction of the observation signal vector y (t, f) mathematically means that the element ratio y (1, t, f) for all the microphones of the observation signal vector y (t, f): y (2, t, f) f): ...: y (M, t, f) (in other words, a space obtained by identifying vectors in a scalar multiple relationship in an M-dimensional vector space on a complex number field, Refers to an element of M-1 dimensional projection space on a complex number field). Here, y (m, t, f) represents the m-th element of the vector y (t, f). Therefore, the feature vector z (t, f) is a vector including information on the direction of the observation signal vector y (t, f) when the feature vector z (t, f) is given. Element ratio y (1, t, f): y (2, t, f): ... for all microphones of y (t, f): ... means that y (M, t, f) is uniquely determined Do. As a feature vector which is a vector including information on the direction of the observation signal vector y (t, f), for example, a unit vector parallel to the observation signal vector can be used. Also, since the observation signal vector y (t, f) itself naturally contains information on the direction of the observation signal vector y (t, f), this information can be used as the information on the direction of the observation signal vector y (t, f) It can also be used as a feature vector that is a contained vector. A feature vector that is a vector including information on the direction of the observation signal vector y (t, f) includes information on both phase difference and amplitude ratio as information on the sound source position. This is largely different from the conventional feature quantity (for example, Time Difference of Arrival (TDOA) or Direction Of Arrival (DOA)) using only the phase difference without using the amplitude ratio. Therefore, a feature vector that is a vector including information on the direction of the observed signal vector y (t, f) relates to more sound source positions as compared to a conventional feature using only a phase difference without using an amplitude ratio. Using information, more accurate sound source localization is possible. Further, since information on the sound source position can be extracted to a maximum extent from the limited data length, in the embodiment of the present invention, even if the observation signal length is short, it is possible to accurately perform the sound source localization. It contributes to the feature. By using a feature vector that is a vector including information on the direction of the observed signal vector y (t, f), more effective signal processing (for example, sound source separation or noise compared to the case where only the phase difference is used) It has been shown that removal is possible (see H. Sawada, S. Araki, and S. Makino, “Underdetermined Convolutive Blind Source Separation via Frequency Bin-Wise Clustering and Permutation Alignment,” IEEE Transactions on Audio. , Speech, and Language Processing, vol. 19, no. 3, pp. 516-527, Mar. 2011. "). Note that the feature vector, which is a vector including information on the direction of the observed signal vector y (t, f), includes information on both the phase difference and the amplitude ratio as information on the sound source position, as described below. It can be explained as follows. As described above, the direction of the observation signal vector y (t, f) is the element ratio y (1, t, f) for all microphones of the observation signal vector y (t, f): y (2, t, f) ): ...: y (M, t, f), which is an element ratio y (m, t, f) for two microphones (m, n) for all microphone pairs (m, n) ): Equivalent to y (n, t, f) as information. Furthermore, noting that the ratio of complex numbers is equivalent to the ratio of phase difference and absolute value ratio (amplitude ratio) as information, the element ratio to two microphones (m, n) for all microphone pairs (m, n) y (m, t, f): y (n, t, f) is equivalent as information on phase difference and amplitude ratio for two microphones (m, n) for all microphone pairs (m, n) . Therefore, the direction of the observation signal vector y (t, f) is equivalent to information on phase difference and amplitude ratio for two microphones (m, n) for all microphone pairs (m, n). That is, a feature vector that is a vector including information on the direction of the observed signal vector y (t, f) includes information on both the phase difference and the amplitude ratio as information on the sound source position.

  An example in the case of performing sound source localization with respect to recording sound of conversations of a plurality of persons sitting around a table will be described using FIG. 1. FIG. 1 is a diagram for explaining sound source localization in the present invention. First, as shown in FIG. 1, the signal processing apparatus can set L points obtained by finely dividing the area around the table 100 at equal intervals as the sound source position candidate 110. In the example of FIG. 1, L = 8. Also, on the table 100, three microphones 120 are placed. In this example, since the sound source can be considered to exist only at the outer periphery of the table, and the seat height can be regarded as substantially constant regardless of the individual, the sound source position can be specified by the direction (azimuth angle) viewed from the microphone 120 .

  Based on the observation signal observed by the microphone 120, the signal processing device is a feature vector z (vector) including information on the direction of the observation signal vector y (t, f) and the observation signal vector y (t, f). Calculate t, f). Then, based on the model parameters of the conditional probability distribution, the signal processing device is a condition of the feature vector z (t, f) under the condition that the state representing the sound source position takes a state corresponding to each of a plurality of sound source position candidates. By applying a mixed model, which is a weighted sum of the probability distribution, to the feature vector z (t, f), the prior probability distribution, which is the weight in the weighted sum, is calculated. At this time, since the calculated prior probability distribution takes a large value at the sound source position, the sound source position can be estimated based on the prior probability distribution. At this time, for example, in the case where the prior probability distribution takes the largest value in the sound source position candidate 110 where l = 2, the sound source position can be regarded as the direction indicated by the arrow 130.

First Embodiment
The signal processing apparatus according to the first embodiment estimates a set of sound source positions under conditions where the number of sound sources N is unknown. Here, the number of sound sources N may be N = 0 (corresponding to the case where the set of sound source positions is an empty set). In the present embodiment, the signal processing apparatus determines the direction vector of the observed signal vector y (t, f) as a feature vector z (t, f) that is a vector including information on the direction of the observed signal vector y (t, f). The feature vector z under the condition that the state representing the sound source position corresponds to each of the plurality of sound source position candidates using the state corresponding to each of the plurality of sound source position candidates as the state representing the sound source position using The complex Watson distribution is used as the conditional probability distribution of t, f), and model parameters of the complex Watson distribution are calculated and stored based on the assumption that the target signal propagates as a spherical wave, and time-variant prior is Use probability distribution.

  The configuration of the signal processing apparatus according to the first embodiment will be described with reference to FIG. FIG. 2 is a diagram showing an example of the configuration of the signal processing device according to the first embodiment. As shown in FIG. 2, the signal processing device 1 includes a time frequency analysis unit 10, a feature vector calculation unit 20, a parameter storage unit 30, an a priori probability distribution calculation unit 40, and a sound source position calculation unit 50.

  The time-frequency analysis unit 10 uses observation signals y (m, τ) (m is a microphone number (m = 1 to M), τ is time) of M microphones that are recording sounds acquired at a plurality of different positions. Applying time-frequency analysis to the numbers, time-to-frequency conversion y (m, t, f) of the observation signal (t is the frame number (t = 1 to T), f is the frequency bin number (f = 1 to F) )) To create an observed signal vector y (t, f) which is an M-dimensional longitudinal vector consisting of y (m, t, f) (m = 1 to M). The recorded sound acquired at the plurality of different positions may be a recorded sound that has been subjected to some pre-processing (for example, dereverberation processing, spatial whitening processing, etc.) after being acquired at a plurality of different positions (reference document) "T. Yoshioka, T. Nakatani, M. Miyoshi, and HG Okuno," Blind separation and dereverberation of speech mixtures by joint optimization, "IEEE Trans. Audio, Speech, Language Process., Vol. 19, no. 1, pp. 69-84, 2011., “H. Sawada, S. Araki, and S. Makino,“ Underdetermined Convolutive Blind Source Separation via Frequency Bin-Wise Clustering and Permutation Alignment, ”IEEE Transactions on Audio, Speech, and Language Processing, vol. 19, no. 3, pp. 516-527, Mar. 2011.

  The feature vector calculation unit 20 receives the observed signal vector y (t, f) from the time frequency analysis unit 10, and the feature vector z (t (t, f) is a vector including information on the direction of the observed signal vector y (t, f). , F) are calculated by equation (1).

  Here, || · || is the Euclidean norm, and the arrow ← indicates that the right side is substituted for the left side. In the modeling in this embodiment, it is assumed that the observation signal vector y (t, f) consists of N (N may be unknown or N may be 0) target signals and does not include background noise. Further, in the modeling in the embodiment of the present invention, it is assumed that each target signal has the sparsity of having large power only at the sparse point of the time frequency plane. Based on these assumptions, in the present embodiment, it is assumed that the observed signal vector y (t, f) contains only one target signal at each time frequency point. That is, the observed signal vector y (t, f) is modeled by equation (2).

  Here, s (n, t, f) is time-frequency conversion of the n-th target signal, and n is the number (n = 1 to N) of the target signal. The vector h (n, f) is a steering vector representing the space transfer characteristic of the nth target signal, and takes a unique value depending on the sound source position of the nth target signal. Equation (2) indicates that the observed signal vector y (t, f) consists only of the n-th target signal (n varies with time frequency points (t, f)).

  The direction of the observed signal vector y (t, f) in the M-dimensional complex vector space (that is, a one-dimensional subspace in which the observed signal vector y (t, f) spans in the M-dimensional complex vector space) is the time frequency point (t , F) become a unique direction (specifically, the direction of the steering vector h (n, f)) determined by the sound source position of the sound source signal included in the observation signal. More precisely, due to noise and reverberation, the direction of the observation signal vector y (t, f) is distributed with some spread around the specific direction determined by the above-mentioned sound source position. In the present embodiment, the feature vector of Expression (1), which is a direction vector of the observation signal vector y (t, f), is used as a feature vector that is a vector including information on the direction of the observation signal vector y (t, f). Use.

  In the present embodiment, the conditional probability distribution of the feature vector z (t, f) under the condition that the state representing the sound source position corresponds to each of a plurality of (L) sound source position candidates is a complex Watson distribution. Model (other complex Bingham distribution, complex angular central Gaussian distribution, complex Gaussian distribution, mixed complex Watson distribution, mixed complex Bingham distribution, mixed complex angular center Gaussian distribution, mixed complex Gaussian distribution etc. Can be modeled by the probability distribution of That is, the feature vector z (t, f) is modeled by equation (3).

  Here, g (t, f) is a state representing the sound source position at the time frequency point (t, f). In the present embodiment, it is assumed that the state representing the sound source position takes any value of states 1 to L corresponding to each of a plurality of (L) sound source position candidates. Here, the state l is defined as a state in which the sound source position of the sound source signal included in the observed signal vector y (t, f) at the time frequency point (t, f) is the l-th sound source position candidate. p (z (t, f) | g (t, f) = 1) is a conditional probability distribution of the feature vector z (t, f) under the condition of g (t, f) = 1. The vector a (l, f) is a model parameter that determines the average direction of the feature vector z (t, f) with respect to the l-th sound source position candidate, is called an average direction vector, and satisfies equation (4). κ (l, f) is a model parameter that defines the degree of concentration around the mean direction vector a (l, f) of the probability distribution of the feature vector z (t, f) for the l-th sound source position candidate It is called.

  W (z; a,)) is a complex Watson distribution of vector z whose average direction vector is a and whose concentration parameter is κ, and is expressed by equation (5).

  At this time, K is a Kummer function (a first-order combined hypergeometric function) defined by the infinite series of Equation (6), and the superscript H is Hermite transposition. However, when i = 0, it is determined that ・ ・ ・ (ξ + 1) ... (ξ + i-1) / [η (η + 1) ... (・ ・ ・ + i-1)] = 1.

  The parameter storage unit 30 stores model parameters of the conditional probability distribution of the feature vector under the condition that the state representing the sound source position corresponds to each of the plurality of sound source position candidates. Specifically, the parameter storage unit 30 is an average direction vector a (l, f) (l = 1 to L, f = 1 to 5) which is a model parameter for modeling the sound source position of the conditional probability distribution of Expression (3). F) and store concentration parameters ((l, f) (l = 1 to L, f = 1 to F). In the present embodiment, these model parameters are calculated as follows. That is, based on the assumption that the target signal propagates as a spherical wave, the m-th element of the average direction vector a (l, f) is calculated by equation (7).

  Here, the vector q (m) is a three-dimensional real vector (which is assumed to be known in this embodiment) which is the orthogonal coordinates of the m-th microphone, and the vector r (l) is the orthogonal coordinates of the l-th sound source position candidate A real vector of dimension (known), j is an imaginary unit, ω (f) is an angular frequency of an f-th frequency bin, c is a velocity of sound, and a subscript m on the left side is an m-th element. The term of the square root of the denominator is a normalization coefficient for making the mean direction vector a (l, f) satisfy the constraint of equation (4).

  On the other hand, it is assumed that the concentration parameter ((l, f) is proportional to, for example, the negative square of the frequency (ω (f) / 2π), and is calculated by equation (8). Expression (8) is based on the property that the direction of the observation signal vector y (t, f) has a smaller dispersion (larger degree of concentration) as the frequency is lower. Thus, estimation of the prior probability distribution and sound source localization based thereon can be accurately performed by properly considering the above-mentioned properties. Although the proportional constant β may be determined in any way, it may be determined, for example, as β = 6.4 × 10 7 Hz 2.

  Next, modeling of the marginal probability distribution of the feature vector z (t, f) in the present embodiment will be described. In this embodiment, the condition that the peripheral probability distribution of the feature vector z (t, f) is a load with the prior probability distribution P (g (t, f) = 1) of the state g (t, f) representing the sound source position. It models by the mixed model of Formula (9) which is a weighted sum of attached probability distribution p (z (t, f) | g (t, f) = 1).

  The conditional probability distribution p (z (t, f) | g (t, f) = 1) is the probability distribution of the feature vector z (t, f) when the state representing the sound source position is known, The marginal probability distribution p (z (t, f)) of Expression (9) is a probability distribution of the feature vector z (t, f) when the state representing the sound source position is unknown. The prior probability distribution P (g (t, f) = 1) may be "time-variant" or "time-invariant". In the former case, the prior probability distribution P (g (t, f) = 1) may take different values for each time interval (eg, frame). In the latter case, the prior probability distribution P (g (t, f) = 1) takes the same value regardless of the time interval (eg, frame).

  When the prior probability distribution P (g (t, f) = 1) is time-invariant, the prior probability distribution taking a large value at the sound source position is estimated using all time intervals (eg, frames). Since data longer than that in the case can be used for estimation, there is an effect that the sound source position can be estimated more accurately in a situation where there is no movement of the sound source or alternate speech. On the other hand, sound source position estimation can not be performed for each time interval (for example, a frame), and therefore, in the case of time change, tracking and diarization in a dynamic situation with movement of sound source and speech alternation And so on.

  On the other hand, when the prior probability distribution P (g (t, f) = 1) is time-variant, in order to estimate the prior probability distribution which is a function taking a large value at the sound source position for each time interval (for example, frame) In addition to the effect that estimation can be performed for each time interval (for example, frame), there is an effect that tracking and dialing can be performed based on sound source position estimation for each time interval (for example, frame). For example, in speech recognition in a multi-person conversation, it is necessary to cut out a section to which speech recognition should be applied by performing a dialy to estimate "when did you talk" in order to prevent false recognition of noise as speech. Although there is a "time-variant" case, it is applicable to such a case.

  In this embodiment, it is assumed that the prior probability distribution P (g (t, f) = 1) is time-invariant. Further, in the present embodiment, it is assumed that the prior probability distribution P (g (t, f) = 1) does not depend on the frequency. That is, in the present embodiment, it is assumed that the prior probability distribution P (g (t, f) = 1) does not depend on the frame and frequency bin, and is represented by α (1). However, α (l) satisfies the constraint condition α (1) +... + Α (L) = 1. Since the prior probability distribution can be estimated using information of the feature vector z (t, f) observed at all frequencies by using the prior probability distribution independent of frequency, the frequency dependent prior probability distribution More information can be used for estimation of the prior probability distribution compared to the case of using, and more accurate estimation of the prior probability distribution and sound source localization based on it can be realized, and more accurate even when the observation signal length is short Estimation of the prior probability distribution and sound source localization based thereon can be realized.

  The prior probability distribution calculating unit 40 uses the prior probability distribution α (l) (l = 1 to L) of the state representing the sound source position as a load, and is an average direction vector a (model parameters stored in the parameter storage unit 30). a mixed model which is a weighted sum of conditional probability distributions of feature vectors under conditions where the state representing the sound source position is known, based on l, f) and the concentration parameter ((l, f), The prior probability distribution α (l) (l = 1 to L) is calculated by fitting to the feature vector calculated by.

  There are various methods for applying the mixed model of equation (9) to the feature vector z (t, f). For example, let the likelihood for equation (9) be an objective function (in addition, the posterior probability etc. be an objective function) Max) with respect to the prior probability distribution α (l) (l = 1 to L) by the gradient method (others can be maximized by the Expectation-Maximization (EM) algorithm etc.).

  The method based on the gradient method is advantageous in terms of complexity compared to the method based on the EM algorithm. In the method based on the EM algorithm, it is necessary to calculate the contribution rate of each sound source position candidate for each time frequency point in addition to the prior probability distribution α (l) (l = 1 to L) for each iteration. On the other hand, in the gradient method, only the prior probability distribution α (l) (l = 1 to L) needs to be calculated for each iteration, so the amount of calculation can be significantly reduced compared to the EM algorithm. The processing in the prior probability distribution calculation unit 40 is, for example, as follows.

  First, α (l) is initialized by α (l) ← 1 / L (l = 1 to L). Next, the processing of α (l) (1 = 1 to L) according to the following formulas (10) and (11) is alternately repeated a predetermined number of times (for example, 10 times).

  Then, α (l) (l = 1 to L) is output. Here, vector α is an L-dimensional vertical vector consisting of α (l) (l = 1 to L), and vector w (t, f) is W (z (t, f); a (l, f), κ (l) , F)) (L = 1 to L), superscript T is transposition, and λ is a predetermined positive constant (eg, λ = 1).

  Here, the derivation of the equations (10) and (11) will be described. The likelihood of being an objective function is the probability that z (t, f) (t = 1 to T, f = 1 to F) is observed, and is expressed by equation (12).

  The maximization of equation (12) is equivalent to the maximization of equation (13) taking the natural logarithm.

  Here, ln represents a natural logarithm, and Δ above = represents a definition. Taking the slope of equation (13) yields equation (14), from which equation (10) follows. On the other hand, equation (11) is a process for making α (l) satisfy the constraint condition α (1) +... + Α (L) = 1. In Equation (13), an objective function may be used which is changed so as to take the load sum using the load based on the reliability, instead of taking the sum without using the load. As a result, it is possible to give greater weight to feature vectors at highly reliable time frequency points, and to improve the accuracy of prior probability distribution estimation and sound source localization based thereon. For example, based on the assumption that the time frequency point where the norm of the observation signal vector y (t, f) is small corresponds to noise, and the time frequency point where the norm is large corresponds to the target signal It can be used as

  The sound source position calculating unit 50 receives the prior probability distribution α (l) (l = 1 to L) from the prior probability distribution calculating unit 40, and detects the peak position of the prior probability distribution α (l) (l = 1 to L). A set J is calculated, and a set G of sound source positions is calculated and output based on the set J of peak positions.

  The set J of peak positions can be calculated, for example, as follows. It is assumed that for each number l = 1 to L, a set of sound source position candidate numbers adjacent to the l-th sound source position candidate is known. At this time, “the number l being a peak position means that α (l)> α (l ′) holds for the number l ′ of all sound source position candidates adjacent to the l-th sound source position candidate. The peak position set J can be calculated by determining whether or not the number l is a peak position for each of the numbers l = 1 to L. Based on this set J of peak positions, it is possible to calculate a set G of detected sound source positions which is a set of number l specifying the sound source position or a set of coordinates (orthogonal coordinates, polar coordinates, spherical coordinates, etc.) as follows. .

  For example, the set J of peak positions may be used as the set G of sound source positions detected as it is, or a set of peak positions l where the peak value α (l) exceeds a predetermined threshold S among peak positions l {lεJ | A set G of detected sound source positions may be used as α (l)> S}. The threshold value S may be determined in any manner, for example, S = 1 / L. Alternatively, a set {r (l) | lεJ} of vectors r (l), which are coordinates of a sound source position candidate corresponding to the peak position l, may be used as the set G of detected sound source positions. A set {r (l) | lεJ, α (l)> S} of vectors r (l) which are coordinates of the sound source position candidate corresponding to the peak position l where l) exceeds the predetermined threshold S is detected It may be a set G of sound source positions.

Processing of the First Embodiment
The flow of processing of the signal processing device 1 will be described with reference to FIG. FIG. 3 is a flowchart showing the flow of processing of the signal processing device according to the first embodiment. As shown in FIG. 3, first, the time-frequency analysis unit 10 performs time-frequency analysis on the observation signal to calculate an observation signal vector (step S11).

  Next, the feature vector calculation unit 20 calculates a feature vector that is a vector including information on the direction of the observation signal vector y (t, f) (step S12). Then, from the parameter storage unit 30, the a priori probability distribution calculation unit 40 generates, from the parameter storage unit 30, the parameters of the conditional probability distribution model of the feature vector under the condition of taking the state corresponding to each of the plurality of sound source position candidates. It acquires (step S13).

  Next, the prior probability distribution calculating unit 40 initializes the prior probability distribution of the state representing each sound source position (step S14). Then, the prior probability distribution calculation unit 40 updates the prior probability distribution (step S15).

  At this time, the prior probability distribution calculation unit 40 uses, for example, a mixed model in which the conditional probability distribution of the feature vector represented by the model parameter acquired from the parameter storage unit is loaded by the prior probability distribution. Model the distribution. Then, the prior probability distribution calculation unit 40 uses the gradient method to update the prior probability distribution so that the likelihood when the likelihood of the surrounding probability distribution is the objective function is maximized. Then, when the prior probability distribution is not repeatedly updated a predetermined number of times (step S16, No), the prior probability distribution calculation unit 40 further updates the prior probability distribution (step S15).

  On the other hand, when the prior probability distribution is repeatedly updated a predetermined number of times (step S16, Yes), the sound source position calculation unit 50 calculates the sound source position based on the prior probability calculated by the prior probability distribution calculation unit 40. (Step S17). At this time, the sound source position calculation unit 50 can use, for example, the sound source position at which the prior probability reaches a peak as the calculation result.

[Effect of First Embodiment]
The time frequency analysis unit 10 applies time frequency conversion to the recorded sound acquired at M different positions, and calculates an observation signal vector which is an M-dimensional vector. Then, the feature vector calculation unit 20 calculates, for each time frequency point, a feature vector that is a vector including information on the direction of the observation signal vector y (t, f) calculated by the time frequency analysis unit 10. The parameter storage unit 30 also stores model parameters of the conditional probability distribution of the feature vector under the condition that the state representing the sound source position corresponds to each of the plurality of sound source position candidates.

  Here, the prior probability distribution calculation unit 40 uses the prior probability distribution of the state representing the sound source position as a load, based on the model parameters stored in the parameter storage unit 30, under conditions where the state representing the sound source position is known. A mixed model, which is a weighted sum of conditional probability distributions of feature vectors, is applied to the feature vectors calculated by the feature vector calculation unit 20 to calculate a prior probability distribution. Then, the sound source position calculation unit 50 calculates the sound source position corresponding to the feature vector based on the prior probability distribution calculated by the prior probability distribution calculation unit 40.

  As described above, according to the first embodiment, it is possible to calculate the prior probability distribution that can be regarded as a space spectrum that is a function that takes a large value at the sound source position without using the covariance matrix of the observation signal vector. Even when the observation signal length is short, sound source localization can be performed accurately. Therefore, when the observation signal length is short, accurate sound source localization is difficult, compared with the conventional sound source localization method such as the Caporn method or MUSIC method, the situation where the sound source position changes temporally or the conversation with utterance substitution It is advantageous under dynamic situations such as situations. Further, according to the first embodiment, even in a situation where sound source signals from a plurality of sound sources are mixed, it is possible to estimate the sound source position of each sound source. Therefore, in the situation where there are multiple sound sources such as a conversational situation where there are overlapping of utterances, as compared with the conventional sound source localization methods such as delay-and-sum array and generalized cross correlation function method in which estimation of multiple sound source positions is difficult Is advantageous. In addition, sound source localization can be performed even in a situation where the number of sound sources is unknown. Therefore, in many practical applications, the number of sound sources is often unknown in advance, but even under such circumstances, sound source localization is possible according to the present embodiment. This is advantageous over conventional sound source localization methods such as the MUSIC method that requires advance information on the number of sound sources. Furthermore, the prior probability distribution obtained by the method of the first embodiment can be used in various applications such as tracking, dialing, mask estimation, speech enhancement, and speech recognition. Furthermore, according to the first embodiment, the prior probability distribution is estimated using information of feature vectors z (t, f) observed at all frequencies by using the prior probability distribution not depending on the frequency. (It can also be understood from equation (10) that vector α is updated using vector w (t, f) at all frequencies), so we use a frequency-dependent prior probability distribution Compared to the case, more information can be used to estimate the prior probability distribution, more accurate estimation of the prior probability distribution and sound source localization based on it can be realized, and the more accurate prior can be realized even when the observation signal length is short Estimation of probability distribution and sound source localization based on it can be realized. Although the above describes the batch processing for processing observation signals in all frames at one time, block batch processing (or processing for processing the observation signals for each frame (or every several frames) to estimate the sound source position It can also be an online process.

Second Embodiment
Next, the configuration of the second embodiment will be described. The second embodiment is an example of estimating a sound source position based on the present invention, and is a modification based on the first embodiment, in which a time-varying prior probability distribution is used as the prior probability distribution. is there. That is, in the second embodiment, the prior probability distribution is estimated for each time interval (for example, frame). As a result, in addition to the effect that the sound source position estimation can be performed for each time interval (for example, frame), the effect that tracking and dialing can be performed based on the sound source position estimation for each time interval (for example, frame) Is obtained.

  An example of the configuration of the signal processing device according to the second embodiment is shown in FIG. 2 as in the signal processing device 1 according to the first embodiment. The signal processing device 1 according to the second embodiment includes a time frequency analysis unit 10, a feature vector calculation unit 20, a parameter storage unit 30, an a priori probability distribution calculation unit 40, and a sound source position calculation unit 50. The time-frequency analysis unit 10, the feature vector calculation unit 20, and the parameter storage unit 30 are the same as in the first embodiment, and hence the prior probability distribution calculation unit 40 and the sound source position calculation unit 50 which are differences below. explain in detail. The main differences between the first embodiment and the present embodiment are as follows. In the first embodiment, the prior probability distribution calculating unit 40 calculates the prior probability distribution not depending on the time interval, and the sound source position calculating unit 50 calculates the sound source position independent of the time interval based on the prior probability distribution. On the other hand, in the present embodiment, the prior probability distribution calculating unit 40 calculates the prior probability distribution for each time interval, and the sound source position calculating unit 50 calculates the sound source position for each time interval based on the prior probability distribution.

  A model stored in the parameter storage unit 30 in which the prior probability distribution calculation unit 40 uses the prior probability distribution α (l, t) (l = 1 to L, t = 1 to T) of the state indicating the sound source position as a load Conditional probability distribution of the feature vector z (t, f) under conditions where the state representing the sound source position is known, based on the parameters average direction vector a (l, f) and concentration parameter ((l, f) (15) is applied to the feature vector z (t, f) calculated by the feature vector calculation unit 20, and the prior probability distribution α (l, t) (l = 1 to L, Calculate t = 1 to T). However, α (l, t) satisfies the constraint condition α (1, t) +... + Α (L, t) = 1.

  Here, it should be noted that the load in equation (15) is not time-invariant α (l) but time-varying α (l, t) unlike the first embodiment. There are various methods for fitting the mixed model of Equation (15) to the feature vector z (t, f). For example, the likelihood with respect to Equation (15) is maximized by the gradient method.

The processing in the prior probability distribution calculation unit 40 is, for example, as follows.
1. The prior probability distribution α (l, t) is initialized by α (l, t) ← 1 / L (l = 1 to L, t = 1 to T).
2. The update of the prior probability distribution α (l, t) (l = 1 to L, t = 1 to T) according to the following Equation (16) and Equation (17) is alternately repeated a predetermined number of times (for example, 10 times).

  3. The prior probability distribution α (l, t) (l = 1 to L, t = 1 to T) is output.

  However, the vector α (t) is an L-dimensional vertical vector consisting of α (l, t) (l = 1 to L). The derivation of equations (16) and (17) is omitted as it is similar to the derivation of equations (10) and (11).

  The sound source position calculating unit 50 receives the prior probability distribution α (l, t) (l = 1 to L, t = 1 to T) from the prior probability distribution calculating unit 40, and the prior probability distribution α (l, t) ( A set J (t) of peak positions of l = 1 to L, t = 1 to T) is calculated for each frame, and a set G (t) of sound source positions detected based on the set J (t) of peak positions Is calculated and output for each frame.

  The set of peak positions J (t) can be calculated, for example, as follows. A set (assumed to be known) of the numbers of sound source position candidates adjacent to the l-th (l = 1 to L) sound source position candidate is represented by a set A (l). At this time, the set J (t) of peak positions is a set J of numbers l where α (l, t)> α (l ′, t) for all numbers l ′ belonging to set A (l). It can be calculated as (t) = {l | ∀l'∈A (l), α (l, t)> α (l ', t)}. Based on the set J (t) of peak positions, a set G (t) of detected sound source positions is a set of numbers l specifying the sound source position or a set of coordinates (orthogonal coordinates, polar coordinates, spherical coordinates, etc.) It can be calculated as follows.

  For example, the set J (t) of peak positions can be set as the set G (t) of sound source positions detected as it is. Also, among the peak positions l, a set of sound source positions for which a set {lεJ (t) | α (l, t)> S} of corresponding peak values α (l, t) exceeding a predetermined threshold value S is detected It can also be set G (t). Here, the threshold value S may be determined in any way, but for example, it may be S = 1 / L. Also, let a set {r (l) | lεJ (t)} of vectors r (l), which are coordinates of sound source position candidates corresponding to the peak position l, be a set G (t) of detected sound source positions You can also. In addition, a set of vectors r (l) which are coordinates of sound source position candidates corresponding to those of the peak positions l where the peak value α (l, t) exceeds the predetermined threshold S {r (l) | lεJ ( A set G (t) of detected sound source positions may be used as t) and α (l, t)> S}.

Third Embodiment
Next, the configuration of the third embodiment will be described. The third embodiment is an example of estimating a sound source position based on the present invention, and based on the first embodiment, each of a plurality of (L) sound source position candidates is a state representing a sound source position. In addition to the corresponding states (states 1 to L), the condition corresponding to background noise (state 0) is also considered, and the condition of the feature vector under the condition that the state representing the sound source position takes state 0 The modified probability distribution is modeled as a uniform distribution on the hypersphere. This has an advantage that it is possible to appropriately model an observation signal including background noise and perform source localization with high accuracy even under background noise.

  An example of the configuration of the signal processing device according to the third embodiment is shown in FIG. 2 as in the signal processing device 1 according to the first embodiment. The signal processing device 1 according to the third embodiment includes a time frequency analysis unit 10, a feature vector calculation unit 20, a parameter storage unit 30, an a priori probability distribution calculation unit 40, and a sound source position calculation unit 50. The time-frequency analysis unit 10 and the feature vector calculation unit 20 are the same as in the first embodiment, and hence the parameter storage unit 30, the prior probability distribution calculation unit 40, and the sound source position calculation unit 50 which are differences below. explain in detail. The main differences between the first embodiment and the present embodiment are as follows. In the first embodiment, the parameter storage unit 30 stores model parameters of conditional probability distribution under the condition that the state representing the sound source position corresponds to each of a plurality of sound source position candidates, and the a priori probability distribution The calculation unit 40 calculates a priori probability distribution for a state corresponding to a plurality of sound source position candidates, and the sound source position calculation unit 50 calculates a sound source position based on the a priori probability distribution. On the other hand, in the present embodiment, the parameter storage unit 30 further stores model parameters of conditional probability distribution under the condition that the state representing the sound source position corresponds to the background noise, and the prior probability distribution calculation unit At 40, the a priori probability distribution of the state corresponding to the plurality of sound source position candidates and the background noise is calculated, and the sound source position calculating unit 50 calculates the sound source position based on the a priori probability distribution.

  First, modeling of the observed signal vector y (t, f) in the present embodiment will be described. In the modeling in this embodiment, it is assumed that the observation signal vector y (t, f) includes background noise as well as N target signals (N may be unknown; N may be 0). Further, in the present embodiment, it is assumed that the observed signal vector y (t, f) includes at most one target signal of the target signals at each time frequency point, and the background noise is observed signal vectors at all time frequency points. Suppose that it is included in y (t, f). At this time, the observed signal vector y (t, f) is modeled by either equation (18) or (19).

  Here, in the time frequency point (t, f), only the target signal of the n-th (n can vary depending on the time frequency point (t, f)) target signal at time frequency point (t, f) observed signal vector y (t , F), equation (19) represents the case where no target signal is included in the observed signal vector y (t, f) at the time frequency point (t, f), and the vector s (n) , T, f) is the n-th target signal, and vector v (t, f) is background noise.

  Unlike the case of the first embodiment, in the present embodiment, the case where only the background noise is included in the observation signal vector y (t, f) without any target signal as in equation (19) is also taken into consideration. The modeling is performed, and the observation signal under background noise can be modeled more accurately.

  As described above, in the present embodiment, the case where no target signal is included in the observed signal vector y (t, f) as in Expression (19) is also considered. In this embodiment, in order to be able to model appropriately also in such a case, in addition to the state corresponding to a plurality of sound source position candidates, the state representing the sound source position obtainable by the observation signal vector at each time frequency point Further consider the condition corresponding to the noise. The former corresponds to equation (18) and the latter corresponds to equation (19).

  Hereinafter, a state representing the sound source position at a time frequency point (t, f) will be represented by g (t, f). The conditional probability distribution of the feature vector z (t, f) under the condition of g (t, f) = l (l = 1 to L) is the same as in the first embodiment. Other probability models such as complex Bingham distribution, complex angular center Gaussian distribution, complex Gaussian distribution, mixed complex Watson distribution, mixed complex Bingham distribution, mixed complex angular center Gaussian distribution, mixed complex Gaussian distribution etc. Can be modeled by distribution).

  On the other hand, the conditional probability distribution of the feature vector z (t, f) under the condition of g (t, f) = 0 is, as shown in equation (20), one of the units on the unit sphere in the M-dimensional complex vector space. It is modeled by uniform distribution.

  Equation (20) is based on the assumption that background noise comes uniformly from all directions. In the present embodiment, by introducing equation (20), it is possible to appropriately model the state corresponding to background noise as in equation (19), and the source position can be accurately estimated even under background noise. it can.

  Next, modeling of the marginal probability distribution of the feature vector z (t, f) in the present embodiment will be described. In the present embodiment, conditional probability distribution p (p (g, t, f) = 1), which is a prior probability distribution P (g (t, f) = 1) of the state representing the sound source position, is used as the peripheral probability distribution of feature vector z (t, f). It models by the mixed model of Formula (21) which is a load sum of z (t, f) | g (t, f) = l.

  In this embodiment, it is assumed that the prior probability distribution P (g (t, f) = 1) does not depend on frames and frequency bins, and is represented by α (1) (1 = 0 to L). However, α (l) satisfies the constraint condition α (0) +... + Α (L) = 1. If 式 = 0 and a is an arbitrary unit vector, note that the complex Watson distribution W (z; a,)) matches the uniform distribution of equation (20), equation (21) It can also be rewritten as (22). However, κ (0, f) = 0 and vector a (0, f) is an arbitrary unit vector. Since the prior probability distribution can be estimated using information of the feature vector z (t, f) observed at all frequencies by using the prior probability distribution independent of frequency, the frequency dependent prior probability distribution More information can be used for estimation of the prior probability distribution compared to the case of using, and more accurate estimation of the prior probability distribution and sound source localization based on it can be realized, and more accurate even when the observation signal length is short Estimation of the prior probability distribution and sound source localization based thereon can be realized. Furthermore, since the prior probability distribution can be estimated using information of the feature vector z (t, f) observed at all frequencies, the feature vector z (observed at one frequency due to the influence of noise and reverberation) Even when the sound source position can not be determined with certainty using only t and f), sound source localization can be performed more accurately, and it is more robust against noise and reverberation than using a frequency-dependent prior probability distribution. It is possible to improve the quality.

  The parameter storage unit 30 is a model parameter of the conditional probability distribution under the condition that the state representing the sound source position corresponds to each of a plurality of sound source position candidates, and the state representing the sound source position corresponds to the background noise Store the model parameters of the conditional probability distribution under the condition The former can be calculated by, for example, the method described in the first embodiment, and the latter can be, for example, κ (0, f) ← 0, and the vector a (0, f) can be any unit vector.

  The prior probability distribution calculating unit 40 uses the prior probability distribution α (l) (l = 0 to L) of the state representing the sound source position as a load, and the average direction vector a (model parameters stored in the parameter storage unit 30) The state representing the sound source position is known based on l, f) (l = 0 to L, f = 1 to F) and concentration parameters κ (l, f) (l = 0 to L, f = 1 to F) (21) is applied to the feature vector z (t, f) calculated by the feature vector calculation unit 20, and the prior probability distribution α (L) Calculate (l = 0 to L).

  There are various methods for applying the mixed model of Equation (21) to the feature vector z (t, f). For example, let the likelihood for Equation (21) be an objective function (in addition, the posterior probability etc. be an objective function) This can be maximized with respect to the prior probability distribution α (l) (l = 0 to L) by the gradient method (in addition, it can be maximized by the EM algorithm etc.).

The processing in the prior probability distribution calculation unit 40 is, for example, as follows.
1. The prior probability distribution α (l) (l = 0 to L) is initialized by α (l) ← 1 / (L + 1).
2. The updating of the prior probability distribution α (l) (l = 0 to L) according to the following equations (23) and (24) is alternately repeated a predetermined number of times (for example, 10 times).

  3. The prior probability distribution α (l) (l = 0 to L) is output.

  Here, a vector (α (the symbol “〜” before α represents that the symbol “〜” is attached on α) is an (L + 1) -dimensional vertical pattern consisting of α (l) (l = 0 to L) A vector, and the vector ~ w (t, f) is W (z (t, f); a (l, f), κ (l, f)) (l = 0 to L) (L + 1) dimensional vertical It is a vector. The derivation of the equations (23) and (24) is the same as that of the first embodiment, and is therefore omitted.

  The sound source position calculation unit 50 calculates and outputs a set G of detected sound source positions based on the prior probability distribution α (l) (l = 0 to L) received from the prior probability distribution calculation unit 40. Specifically, the process described in the first embodiment is applied to α (l) (l = 1 to L) in which the domain of l is limited to l = 1 to L corresponding to the target sound source. Thus, a set G of detected sound source positions is calculated.

Fourth Embodiment
Next, the configuration of the fourth embodiment will be described. The fourth embodiment is an example of estimating the sound source position based on the present invention, and based on the first embodiment, assuming that the target signal propagates as a spherical wave as a model parameter of conditional probability distribution. Instead of calculating based on the change, a change is made such that actual data is used as learning data and learning is performed in advance. The above assumption that the target signal propagates as a spherical wave assumes an ideal environment without reflection, reverberation, diffraction, etc., such as an anechoic chamber. Therefore, in the first embodiment, in an environment with reflection, reverberation, diffraction, etc., there is a mismatch between the assumed environment and the environment for performing sound source localization, so the problem of the performance of the sound source localization is degraded. is there. On the other hand, in the present embodiment, such mismatch is eliminated by pre-learning model parameters of conditional probability distribution using measured data in an environment where sound source localization is performed, and there are reflections, reverberations, diffractions, etc. However, there is an advantage that it becomes possible to estimate the sound source position accurately. On the contrary, the first embodiment has an advantage that the time and effort for acquiring the above-mentioned actual measurement data can be saved unlike the present embodiment.

  An example of the configuration of the signal processing device according to the fourth embodiment is shown in FIG. 2 as in the signal processing device 1 according to the first embodiment. The signal processing device 1 according to the fourth embodiment includes a time frequency analysis unit 10, a feature vector calculation unit 20, a parameter storage unit 30, an a priori probability distribution calculation unit 40, and a sound source position calculation unit 50. The time frequency analysis unit 10, the feature vector calculation unit 20, the a priori probability distribution calculation unit 40, and the sound source position calculation unit 50 are the same as in the first embodiment, so explain in detail. The main differences between the first embodiment and the present embodiment are as follows. The parameter storage unit 30 in the first embodiment stores model parameters of the conditional probability distribution, which are calculated based on the assumption that the target signal propagates as a spherical wave. On the other hand, the parameter storage unit 30 in the present embodiment stores the model parameters of the conditional probability distribution learned using the learning data acquired under reverberation.

  The parameter storage unit 30 is a model parameter learned using learning data acquired under reverberation, and is a condition that takes a state corresponding to each of a plurality (L) of sound source position candidates. And a directional vector a (l, f) (l = 1 to L, f = 1 to F) which is a model parameter of a complex Watson distribution that is a conditional probability distribution of the feature vector z (t, f) under The concentration parameters κ (l, f) (l = 1 to L, f = 1 to F) are stored. As the learning data acquired under the reverberation, for example, an observation signal x (l, m, τ) in the case where a sound is emitted only from each of a plurality of sound source position candidates in the absence of background noise is used. Can.

The above-mentioned prior learning can be performed, for example, in the following procedure.
1. Generate an observation signal x (l, m, τ) when a sound is emitted from only one sound source position candidate. For example, x (l, m, τ) can be generated by performing recording in a situation where sound is emitted only from the sound source position candidate for each of the L sound source position candidates. Alternatively, for each of the L sound source position candidates, an impulse response from the sound source position candidate to each microphone position is measured, and x (l, m, τ) is generated by convoluting this impulse response into the target signal. it can.
2. An M-dimensional vector x (l, t, f) consisting of time-frequency transforms x (l, m, t, f) (m = 1 to M) of x (l, m, τ) is calculated.
3. The feature vector ζ (l, t, f) is calculated by the following equation (25).

  4. The feature covariance matrix R (l, f) is calculated by the following equation (26).

5. Eigenvalue decomposition of the feature covariance matrix R (l, f) is performed to obtain the largest eigenvalue μ (l, f) and the eigenvector e (l, f) of the norm 1 corresponding to the largest eigenvalue.
6. An average direction vector a (l, f) is set as a (l, f) ee (l, f).
7. The concentration parameter κ (l, f) is calculated by the following equation (27).

  The derivation of the above process will be described. The above process averages the direction vector a (l) of equation (29), which is the log likelihood for equation (28), under the assumption that the feature vector ζ (l, t, f) is generated according to equation (28) , F) and maximization with respect to the lumped parameter ((l, f).

  In equation (29), average direction vector a (l, f) (l = 1 to L, f = 1 to F) and concentration parameter ((l, f) (l = 1 to L, f = 1 to F) Ignoring a constant term that does not depend on any of, can be rewritten as equation (30).

  Here, the matrix R (l, f) is defined by equation (26). The term depending on the vector a (l, f) in the equation (27) is the equation (31).

  According to the Courant-Fisher theorem, the vector a (l, f) which maximizes the equation (31) under the constraint of the equation (4) is the maximum eigenvalue μ (l, l) of the feature covariance matrix R (l, f) It is an eigenvector e (l, f) of norm 1 corresponding to f). The term depending on the concentration parameter κ (l, f) in equation (30) is equation (32).

  Here, when the partial derivative of the concentration parameter 0 (l, f) is set to 0, equation (33) is obtained.

  Reference 1 in S. Sra and D. Karp, "The multivariate Watson distribution: Maximum-likelihood estimation and other aspects," Journal of Multivariate Analysis, February 2013, vol. 114, p. 256-269. Based on Equation (3.8), Equation (33) is approximately solved for the concentration parameter κ (l, f) to obtain Equation (27). In the present embodiment, in order to learn concentration parameters from learning data, as in the first embodiment, the direction of the observation signal vector y (t, f) described above decreases dispersion (larger degree of concentration) as the frequency decreases. It is possible to properly take into consideration the nature of having, and to estimate the prior probability distribution and the sound source localization based on it accurately.

Fifth Embodiment
Next, the configuration of the fifth embodiment will be described. The fifth embodiment is an example of estimating the sound source position based on the present invention, and based on the third embodiment, the measurement data is not used as the conditional probability distribution for background noise, but the uniform distribution is used. Is modified to use the conditional probability distribution previously learned using.

  The above uniform distribution assumption in the third embodiment assumes an ideal environment in which noise arrives uniformly from all directions. Therefore, in the third embodiment, in an environment where there is a bias in the noise arrival direction, a mismatch exists between the assumed environment and the environment for performing sound source localization, and there is a risk that the performance of sound source localization may be degraded. . On the other hand, in the present embodiment, the above mismatch is eliminated by pre-learning model parameters of conditional probability distribution using measured data in an environment where sound source localization is performed, and there is a bias in the noise arrival direction. However, there is an advantage that it is possible to accurately estimate the sound source position.

  An example of the configuration of the signal processing device according to the fifth embodiment is shown in FIG. 2 as in the signal processing device 1 according to the third embodiment. The signal processing device 1 according to the fifth embodiment includes a time frequency analysis unit 10, a feature vector calculation unit 20, a parameter storage unit 30, an a priori probability distribution calculation unit 40, and a sound source position calculation unit 50. The time frequency analysis unit 10, the feature vector calculation unit 20, the a priori probability distribution calculation unit 40, and the sound source position calculation unit 50 are the same as in the third embodiment, so explain in detail. The main differences between the third embodiment and the present embodiment are as follows. The parameter storage unit 30 according to the third embodiment stores model parameters corresponding to uniform distribution as model parameters of conditional probability distribution under the condition that the state representing the sound source position corresponds to the background noise. . On the other hand, in the parameter storage unit 30 in the present embodiment, a model parameter learned using learning data as a model parameter of conditional probability distribution under the condition that the state representing the sound source position corresponds to the background noise. Remember.

  In the present embodiment, the feature vector z under the condition that the state representing the sound source position of the observed signal vector y (t, f) at each time frequency point is g (t, f) = 1 (1 = 0 to L). The conditional probability distribution of (t, f) is modeled by the complex Watson distribution of Equation (3) (Others: complex Bingham distribution, complex angular center Gaussian distribution, complex Gaussian distribution, mixed complex Watson distribution, mixed complex Bingham It can be modeled by a probability distribution such as distribution, mixed complex angular center Gaussian distribution, mixed complex Gaussian distribution, etc.).

  The parameter storage unit 30 has a model parameter of conditional probability distribution under a condition in which the state representing the sound source position corresponds to each of a plurality of sound source position candidates and the state representing the sound source position. The model parameters of the conditional probability distribution under the condition of taking a state (state 0) corresponding to background noise are stored. The model parameters of the conditional probability distribution under the condition that the state representing the sound source position corresponds to each of the plurality of sound source position candidates may be calculated by, for example, the method described in the first or fourth embodiment. it can.

On the other hand, model parameters of the conditional probability distribution under the condition that the state representing the sound source position corresponds to the background noise are pre-learned as follows, for example.
1. Create an M-dimensional longitudinal vector x (0, t, f) consisting of time-frequency transformation x (0, m, t, f) (m = 1 to M) of measured background noise x (0, m, τ) .
2. The feature vector ζ (0, t, f) is calculated by the following equation (34).

  3. The feature covariance matrix R (0, f) is calculated by the following equation (35).

4. Eigenvalue decomposition of the feature covariance matrix R (0, f) is performed to obtain the largest eigenvalue μ (0, f) and the eigenvector e (0, f) of the norm 1 corresponding to the largest eigenvalue.
5. The average direction vector a (0, f) is set as a (0, f) (e (0, f).
6. The concentration parameter κ (0, f) is calculated by the following equation (36).

  The derivation of the above process is the same as that of the fourth embodiment and thus will not be described.

Sixth Embodiment
Next, the configuration of the sixth embodiment will be described. The sixth embodiment is an example of estimating a sound source position based on the present invention, and based on the fourth embodiment, a state representing a sound source position takes a state corresponding to each of a plurality of sound source position candidates. This is a modification in which not a complex Watson distribution but a complex angularly centered Gaussian distribution is used as the conditional probability distribution of the feature vector z (t, f) under the conditions. The complex Watson distribution can only represent the case where the conditional probability distribution of the feature vector of Equation (1), which is the direction of the observation signal vector, is rotationally symmetric, while the complex angular center Gaussian distribution rotates this conditional probability distribution. Not only symmetric cases but also elliptical cases can be represented. Since the distribution of feature vectors in equation (1) is not necessarily rotationally symmetric, this embodiment makes it possible to model the distribution of feature vectors in equation (1) more accurately than the fourth embodiment. As a result, the sound source position can be estimated more accurately.

  An example of the configuration of the signal processing device according to the sixth embodiment is shown in FIG. 2 as in the signal processing device 1 according to the fourth embodiment. The signal processing device 1 according to the sixth embodiment includes a time frequency analysis unit 10, a feature vector calculation unit 20, a parameter storage unit 30, an a priori probability distribution calculation unit 40, and a sound source position calculation unit 50. The time-frequency analysis unit 10, the feature vector calculation unit 20, and the sound source position calculation unit 50 are the same as those in the fourth embodiment, and therefore, about the parameter storage unit 30 and the prior probability distribution calculation unit 40 which are differences below. explain in detail. The main differences between the fourth embodiment and the present embodiment are as follows. In the fourth embodiment, the parameter storage unit 30 stores model parameters of a complex Watson distribution for modeling a conditional probability distribution, and the prior probability distribution calculating unit 40 performs advance in advance based on the model parameters of the complex Watson distribution. Calculate the probability distribution. On the other hand, in the present embodiment, the parameter storage unit 30 stores model parameters of complex angular center Gaussian distribution for modeling conditional probability distribution, and the prior probability distribution calculation unit 40 stores the complex angular center Gaussian distribution. Calculate the prior probability distribution based on the model parameters.

  In the present embodiment, L conditional probability distributions for L source position candidates are modeled by a complex angular center Gaussian distribution. That is, the conditional probability distribution p (z (t, f) | g (t, f) = 1) is modeled by equation (37).

  Here, the matrix Σ (l, f) is a positive definite Hermite matrix which is a model parameter for determining the position, the spread, the direction, and the shape of the distribution of the feature vector z (t, f) for the l-th sound source position candidate. Called a matrix, A (z;)) is a complex angular center Gaussian distribution of vector z whose parameter matrix is matrix 、, and is expressed by equation (38).

  The parameter storage unit 30 is a complex angular center Gaussian distribution that is a conditional probability distribution of the feature vector z (t, f) under the condition that the state representing the sound source position corresponds to each of a plurality of sound source position candidates. The parameter matrix Σ (l, f) (l = 1 to L, f = 1 to F) which is a model parameter of The parameter matrix Σ (l, f) is pre-learned using the observation signal x (l, m, τ) when sound is emitted from only the sound source position candidate for each of the L sound source position candidates. Ru. In this embodiment, since the parameter matrix ((l, f) for determining the position, the spread, the direction, and the shape of the conditional probability distribution of the feature quantity z (t, f) is learned from the learning data, the first embodiment Similarly, the above-mentioned property that the direction of the observation signal vector y (t, f) has a smaller dispersion (corresponding to the spread) as the frequency is lower can be appropriately taken into consideration, estimation of the prior probability distribution, and It is possible to accurately perform sound source localization based on

This prior learning can be performed, for example, by the following procedure.
1. A feature vector ζ (l, t, f) (l = 1 to L, t = 1 to T, f = 1 to F) is calculated in the same manner as the fourth embodiment.
2. The parameter matrix Σ (l, f) (l = 1 to L, f = 1 to F) is initialized with an M × M identity matrix.
3. The update of the parameter matrix Σ (l, f) (l = 1 to L, f = 1 to F) according to the following equation (39) is repeated a predetermined number of times (for example, ten times).

  4. The parameter matrix Σ (l, f) (l = 1 to L, f = 1 to F) is stored in the parameter storage unit 30.

  The derivation of equation (39) will be described. Equation (39) gives equation (40), which is the log likelihood for equation (37), under the assumption that the feature vector ζ (l, t, f) is generated according to the conditional probability distribution of equation (37) It is derived by maximizing the parameter matrix パ ラ メ ー タ (l, f).

  Ignoring constant terms not based on the parameter matrix 無視 (l, f) in equation (40), equation (40) can be rewritten as equation (41).

  Equation (39) can be obtained by rearranging the partial derivatives of the parameter matrix Σ (l, f) of equation (41) with 0.

  The prior probability distribution calculation unit 40 uses, as a load, the prior probability distribution of the state representing the sound source position, a parameter matrix パ ラ メ ー タ (l, f) which is a model parameter stored in the parameter storage unit 30 (l = 1 to L, f A mixed model which is a weighted sum of complex angular center Gaussian distributions, which is a conditional probability distribution of feature vectors z (t, f) under conditions where the state representing the sound source position is known, based on The prior probability distribution is calculated by applying to the feature vector z (t, f) calculated by the feature vector calculation unit 20. In this embodiment, a time-invariant prior probability distribution α (l) (l = 1 to L) is considered as the prior probability distribution.

  The calculation of the prior probability distribution in the prior probability distribution calculation unit 40 may be performed, for example, as follows. That is, let the vector w (t, f) be an L-dimensional longitudinal vector consisting of a complex angular center Gaussian distribution A (z (t, f); ((l, f)) (l = 1 to L) which is a conditional probability. The processing in the prior probability distribution calculation unit 40 of the first embodiment is applied to the vectors w (t, f). However, it should be noted that the definition of the vector w (t, f) is different from that of the first embodiment. Note that the derivation of the above process is the same as that of the first embodiment and thus will not be described.

Seventh Embodiment
Next, the configuration of the seventh embodiment will be described. The seventh embodiment is an example of estimating the sound source position based on the present invention, and is a vector including information on the direction of the observed signal vector y (t, f) based on the fourth embodiment. The observation signal vector y (t, f) itself is used as the feature vector z (t, f) instead of the direction vector of equation (1), and the state representing the sound source position corresponds to each of a plurality of sound source position candidates As a conditional probability distribution of the feature vector z (t, f) under the condition of taking, not using complex Watson's distribution but using complex time-varying Gaussian distribution, the space covariance matrix which is a model parameter of complex time-varying Gaussian distribution A change has been made to pre-learn and store.

  The complex Watson's distribution can be expressed only when the distribution in the direction of the observed signal vector is rotationally symmetric, while in the complex time-varying Gaussian distribution, the distribution in the direction of the observed signal vector is elliptical as well as the rotationally symmetric distribution. Can also be represented. Since the distribution of the direction of the observed signal vector is not necessarily rotationally symmetric, this embodiment makes it possible to model the distribution of the direction of the observed signal vector characterizing the sound source position more accurately than the fourth embodiment. The source position can be more accurately estimated based on this modeling.

  An example of the configuration of the signal processing device according to the seventh embodiment is shown in FIG. 2 as in the signal processing device 1 according to the fourth embodiment. The signal processing device 1 according to the seventh embodiment includes a time frequency analysis unit 10, a feature vector calculation unit 20, a parameter storage unit 30, an a priori probability distribution calculation unit 40, and a sound source position calculation unit 50. The time-frequency analysis unit 10 and the sound source position calculation unit 50 are the same as in the fourth embodiment, so the feature vector calculation unit 20, the parameter storage unit 30, and the prior probability distribution calculation unit 40 which are differences will be described in detail below. Do. The main differences between the fourth embodiment and the present embodiment are as follows. In the fourth embodiment, the feature vector calculation unit 20 calculates the feature vector of equation (1), and the parameter storage unit 30 stores model parameters of complex Watson distribution for modeling the conditional probability distribution of the feature vector. The prior probability distribution calculating unit 40 applies a mixed model, which is a weighted sum of complex Watson distributions for modeling conditional probability distributions, to the feature vector, with the prior probability distribution of the state representing the sound source position as a load. , Calculate the prior probability distribution. On the other hand, in the present embodiment, the feature vector calculation unit 20 outputs the observation signal vector from the time frequency analysis unit 10 as a feature vector, and the parameter storage unit 30 outputs the conditional probability of the observation signal vector which is the feature vector. A conditional probability distribution that stores a spatial covariance matrix, which is a model parameter of a complex time-varying Gaussian distribution that models a distribution, and uses the prior probability distribution of a state representing a sound source position as a load in the prior probability distribution calculating unit 40 The prior probability distribution is calculated by applying a mixed model, which is a weighted sum of complex time-variant Gaussian distributions to be modeled, to an observation signal vector which is a feature vector.

  The feature vector calculation unit 20 receives the observation signal vector y (t, f) from the time frequency analysis unit 10, and outputs the observation signal vector y (t, f) as a feature vector z (t, f).

  In this embodiment, a complex time-varying Gaussian distribution is used as L conditional probability distributions for L source position candidates. That is, the conditional probability distribution p (z (t, f) | g (t, f) = 1) is modeled by equation (42).

  In the equation (42), φ (l, t, f) is a positive parameter that controls the distribution of the “size (norm)” of the feature vector z (t, f). On the other hand, the matrix B (l, f) in equation (42) controls the distribution of the “direction” of the feature vector z (t, f) (specifically, the direction of the feature vector z (t, f) Control the position, spread, direction and shape of the distribution). The matrix B (l, f) is a positive definite Hermitian matrix and is called a space covariance matrix. N (z; 0,)) is a complex Gaussian distribution of the vector z whose mean is the vector 0 and the covariance matrix is the matrix 、, and is expressed by equation (43).

  (42) has a time-varying covariance matrix φ (l, t, f) B (l, f), so it is called a complex time-varying Gaussian distribution here.

  The parameter storage unit 30 generates an observation signal vector y (t, f) which is a feature vector z (t, f) under the condition that the state representing the sound source position corresponds to each of a plurality of sound source position candidates. The space covariance matrix B (l, f) (l = 1 to L, f = 1 to F) which is a model parameter of conditional probability distribution is stored. In the present embodiment, the parameter storage unit 30 determines the spatial covariance associated with the sound source position among the spatial covariance matrices B (l, f) and φ (l, t, f) which are model parameters of the conditional probability distribution. Only the variance matrix B (l, f) is stored. On the other hand, since φ (l, t, f) depends on the power of the signal, it is not stored in the parameter storage unit 30, but the feature vector from the feature vector calculation unit 20 in the prior probability distribution calculation unit 40 as described later. Estimate using In this embodiment, since the parameter matrix B (l, f) for determining the position, the spread, the direction, and the shape of the distribution in the direction of the observation signal vector y (t, f) is learned from the learning data, Similarly, the property that the direction of the above-mentioned observation signal vector y (t, f) has a smaller dispersion (corresponding to the spread) as the lower frequency can be appropriately taken into consideration, estimation of the prior probability distribution, and based thereon Sound source localization can be performed accurately.

The spatial covariance matrix B (l, f) is generated, for example, using the observation signal x (l, m, τ) when a sound is emitted from only one of the L source position candidates. It is learned in advance by the following procedure.
1. M-dimensional longitudinal vector x (l, t, f) (l = 1 to L) consisting of time-frequency transformation x (l, m, t, f) (m = 1 to M) of x (l, m, τ) Create t = 1 to T, f = 1 to F). The feature vector ζ (l, t, f) is, (l, t, f) ← x (l, t, f). Here, it should be noted that the method of calculating the feature vector ζ (l, t, f) is different from that of the fourth embodiment.
2. A space covariance matrix B (l, f) (l = 1 to L, f = 1 to F) is initialized with an M × M identity matrix.
3. The update of the spatial covariance matrix B (l, f) (l = 1 to L, f = 1 to F) according to the following equation (44) is repeated a predetermined number of times (for example, ten times).

  4. The space covariance matrix B (l, f) (l = 1 to L, f = 1 to F) is stored in the parameter storage unit 30.

  The derivation of equation (44) will be described. Eq. (44) is a space of Eq. (45) which is the log likelihood for Eq. (42) under the assumption that the vector ζ (l, t, f) is generated according to the conditional probability distribution of Eq. It is derived by maximizing on the correlation matrices B (l, f) and φ (l, t, f).

  Ignoring the constant terms not based on the spatial correlation matrices B (l, f) and φ (l, t, f) in equation (45), equation (45) can be rewritten as equation (46).

  Equation (47) is obtained by putting 0 as the partial differential of φ (l, t, f) in equation (46).

  Further, when the partial differential of B (l, f) in equation (46) is set to 0, equation (48) is obtained, and equation (47) is substituted in equation (48) to obtain equation (44).

  Next, modeling of the marginal probability distribution of the feature vector z (t, f) in the present embodiment will be described. In this embodiment, the peripheral probability distribution of the feature vector z (t, f) is set to the prior probability distribution P (g (t, f) = 1) of the state g (t, f) representing the sound source position as a load. The conditional probability distribution p (z (t, f) | g (t, f) = 1) is modeled by a mixed model of equation (49) which is a weighted sum.

  The prior probability distribution calculation unit 40 is a spatial correlation matrix B, which is a model parameter stored in the parameter storage unit 30, with the prior probability distribution α (l) (l = 1 to L) of the state representing the sound source position as a load. The weighted sum of the conditional probability distributions of the feature vector z (t, f) under conditions where the state representing the sound source position is known based on l, f) (l = 1 to L, f = 1 to F) The mixed model of equation (49) is applied to the feature vector z (t, f) calculated by the feature vector calculation unit 20 to calculate the prior probability distribution α (l) (l = 1 to L).

  There are various methods for applying the mixed model of equation (49) to the feature vector z (t, f). For example, the likelihood of equation (49) is taken as an objective function (in addition, the posterior probability etc. is taken as an objective function) Can be maximized based on the gradient method (others can also be maximized based on the EM algorithm etc.).

  The estimation of the prior probability distribution α (l) (l = 1 to L) in the prior probability distribution calculation unit 40 can be performed in the same manner as in the first embodiment. However, unlike the first embodiment, the vector w (t, f) can be expressed as N (z (t, f), 0, 0 (l, t, f) B (l, f)) (l = 1 Let L be an L-dimensional vertical vector consisting of Here, φ (l, t, f) can be calculated by the following equation.

  The derivation of the above process will be described. The likelihood that is the objective function is the probability that the feature vector z (t, f) (t = 1 to T, f = 1 to F) is observed, and is expressed by equation (51).

  Equation (50) is derived by maximization with respect to φ (l, t, f) of equation (51). The maximization of φ (l, t, f) in equation (51) is given by φ (ln (z (t, f), 0, φ (l, t, f) B (l, f))] It is equivalent to maximization with respect to l, t, f). Therefore, assuming that the partial derivative of ln [N (z (t, f), 0, φ (l, t, f) B (l, f))] with respect to φ (l, t, f) is 0, Get (50). After that, Equations (10) and (11), which are update equations of the prior probability distribution α (l) (l = 1 to L), can be derived as in the first embodiment.

Eighth Embodiment
Next, the configuration of the eighth embodiment will be described. In the eighth embodiment, the sound source position is tracked using the set G (t) of sound source positions detected by the signal processing device 1 according to the second embodiment, and the sound source positions ρ (for each sound source) This is an example of calculating n, t) (n = 1 to N, t = 1 to T, N is the number of sound sources). In the present embodiment, it is assumed that the sound source position is designated only by the azimuth, G (t) is a set of azimuths, and ρ (n, t) is an azimuth. Such a situation may include, for example, a situation in which some people are having a conversation around a table on which a microphone is mounted.

  The configuration of the signal processing apparatus according to the eighth embodiment will be described with reference to FIG. FIG. 4 is a diagram showing an example of the configuration of a signal processing device according to the eighth embodiment. As shown in FIG. 4, the signal processing device 2 includes a time frequency analysis unit 10, a feature vector calculation unit 20, a parameter storage unit 30, an a priori probability distribution calculation unit 40, a sound source position calculation unit 50, and a tracking unit 51. The time frequency analysis unit 10, the feature vector calculation unit 20, the parameter storage unit 30, the a priori probability distribution calculation unit 40, and the sound source position calculation unit 50 are the same as those in the second embodiment. The section 51 will be described in detail.

  The tracking unit 51 receives the set G (t) (t = 1 to T) of the detected sound source positions (azimuth angles) from the sound source position calculation unit 50, performs tracking of the sound source position, and sets the sound source for each frame. The sound source position (azimuth angle) ((n, t) (n = 1 to N, t = 1 to T) is calculated and output. This tracking can be done in various ways. In the following, as an example, it is assumed that the rough sound source position (azimuth angle) of each sound source is known, and tracking is performed using this. An example of a situation in which the rough sound source position (azimuth angle) of each sound source is known is a situation in which a plurality of people are sitting in a chair and having a meeting, surrounding a desk on which a microphone is placed. In this case, if the chair is substantially fixed at a known position and there is no seat movement of the speaker during conversation, use the chair position (known) as the rough sound source position of each sound source (speaker) Can.

  First, a rough sound source position of each sound source is set as an initial value ρ (n, 0) of the sound source position (azimuth angle) ((n, t).

Assuming that the sound source position (azimuth angle) ((n, t-1) at frame t-1 is obtained, the sound source position (azimuth angle) ((n, t) at frame t is processed as follows It can be determined by
1. Initialize ρ (n, t) by ρ (n, t) ← ρ (n, t−1).
2. The following processing of 2-1 and 2-2 is performed on each of the detected sound source positions (azimuth angles) rεG (t) (0 ≦ r <2π).
2-1. The number (of the sound source closest to the detected sound source position (azimuth angle) r is calculated according to the following equation (52).

    2-2. The sound source position (azimuth angle) ((ν, t) of the 番 目 -th sound source is updated by equation (53).

  In equation (53), d (式,)) is a circumferential distance defined by equation (54).

  Also, in equation (53), the symbol with subscript [0, 2π) attached to ∠ is an operator for calculating the argument of the range of [0, 2π) with respect to a nonzero complex number. The subscripted [-π, π) symbol is an operator for calculating the argument of the range of [-π, π) with respect to a nonzero complex number, and δ is a constant satisfying 0 <δ <1. (For example, δ = 0.005).

The ninth embodiment
Next, the configuration of the ninth embodiment will be described. The ninth embodiment is an example in which diarization is performed based on the processing result by the signal processing device 2 according to the eighth embodiment. This dialing may be performed by determining whether each sound source is present or absent for each frame (hard decision), or calculated by calculating the existence probability of each sound source for each frame (soft decision). May be Here, an example of the former case is shown.

  The configuration of the signal processing apparatus according to the ninth embodiment will be described with reference to FIG. FIG. 5 is a view showing an example of the configuration of a signal processing apparatus according to the ninth embodiment. As shown in FIG. 5, the signal processing device 3 includes a time frequency analysis unit 10, a feature vector calculation unit 20, a parameter storage unit 30, an a priori probability distribution calculation unit 40, a sound source position calculation unit 50, a tracking unit 51, and a dilation unit. Have 60. The time-frequency analysis unit 10, the feature vector calculation unit 20, the parameter storage unit 30, the prior probability distribution calculation unit 40, the sound source position calculation unit 50, and the tracking unit 51 are the same as the signal processing device 2, and the differences are described below. The dialing unit 60, which is

  The dilation unit 60 sets the detected sound source position G (t) (t = 1 to T) from the sound source position calculation unit 50 and the sound source position (azimuth angle) rho for each sound source from the tracking unit 51. (N, t) (n = 1 to N, t = 1 to T) are received, and a dialation result d (n, t) for each frame is calculated and output for each sound source. However, it is determined that d (n, t) = 1 when the sound source n is present in the frame t, and d (n, t) = 0 when the sound source n is not present in the frame t.

Various methods can be considered as a method of calculating the dilation result d (n, t). For example, calculation may be performed as follows.
1. Initialize d (n, t) (n = 1 to N, t = 1 to T) by d (n, t)) 0.
2. The following process is performed for t = 1 to T: the distance d (r, ((n, t)) is minimized for each of the detected sound source positions (azimuth angles) r ∈ G (t) Find ν, which is the sound source number n, and let d (ν, t) ← 1.
3. Let d (n, t) (n = 1 to N, t = 1 to T) be the dilation result.

  In the ninth embodiment, when the exact sound source position (azimuth angle) of each sound source is known, instead of using the sound source position (azimuth angle) calculated by the tracking unit 51, the known sound source position (azimuth) The angle) may be used as the sound source position (azimuth angle) ((n, t) for each sound source and frame. As such a situation, for example, there is a situation where a speaker is sitting in a fixed chair and has a conversation, a situation where a sound source position (azimuth angle) is known by an image of a video camera, and the like.

Tenth Embodiment
Next, the configuration of the tenth embodiment will be described. The tenth embodiment is an example of estimating the waveform of each target signal based on the sound source position estimated by the present invention in a situation where N (N> 0) target signals are mixed under background noise. According to this embodiment, it is possible to separate mixed target signals into individual target signals and to remove background noise.

  The configuration of a signal processing apparatus according to the tenth embodiment will be described with reference to FIG. FIG. 6 is a diagram showing an example of the configuration of a signal processing apparatus according to the tenth embodiment. As shown in FIG. 6, the signal processing device 4 includes a time frequency analysis unit 10, a feature vector calculation unit 20, a parameter storage unit 30, an a priori probability distribution calculation unit 40, a sound source position calculation unit 50, a tracking unit 51, and a dilation unit. 60 includes a mask estimation unit 70 and a signal enhancement unit 80. The time frequency analysis unit 10, the feature vector calculation unit 20, the tracking unit 51, and the dilation unit 60 are the same as those of the signal processing device 3, and hence the parameter storage unit 30 and the a priori probability distribution calculation unit 40 which are differences below. The sound source position calculation unit 50, the mask estimation unit 70, and the signal enhancement unit 80 will be described in detail. The main differences between the signal processing device 3 and the signal processing device 4 are as follows. The signal processing device 3 stores, in the parameter storage unit 30, model parameters of the conditional probability distribution under the condition that the state representing the sound source position corresponds to each of the plurality of sound source position candidates, and calculates the prior probability distribution. In the unit 40, a prior probability distribution of states corresponding to a plurality of sound source position candidates is calculated based on the model parameters, and in the sound source position calculation unit 50, a sound source position is calculated based on the prior probability distribution. On the other hand, in the signal processing device 4, the parameter storage unit 30 further stores model parameters of conditional probability distribution under the condition that the state representing the sound source position corresponds to the background noise, and calculates the prior probability distribution In part 40, a prior probability distribution of states corresponding to a plurality of sound source position candidates and background noise is calculated based on the model parameters, and in sound source position calculation part 50, a sound source position is calculated based on the prior probability distribution. . In the signal processing device 4, the mask estimation unit 70 further estimates a mask that is the degree of contribution (posterior probability) for each time frequency point of each target signal and background noise, and the signal enhancement unit 80 determines each mask based on the mask. Calculate the waveform of the target signal.

  The parameter storage unit 30 is a model of a complex Watson distribution that is a conditional probability distribution of the feature vector z (t, f) under the condition that the state representing the sound source position corresponds to each of a plurality of sound source position candidates. Parameters of the average direction vector a (l, f) (l = 1 to L, f = 1 to F) and the concentration parameter パ ラ メ ー タ (l, f) (l = 1 to L, f = 1 to F), and the sound source Average direction vector a (0, f) which is a model parameter of complex Watson distribution which is conditional probability distribution of feature vector z (t, f) under the condition that the state representing position corresponds to background noise. (F = 1 to F) and concentration parameters パ ラ メ ー タ (0, f) (f = 1 to F) are stored. These model parameters can be calculated, for example, by the method described in the fourth embodiment for the state corresponding to each of the sound source position candidates, and for the state corresponding to the background noise, for example, in the third embodiment. It can be calculated by the method described.

  The prior probability distribution calculation unit 40 uses, as a load, the prior probability distribution of the state representing the sound source position, and is an average direction vector a (l, f) which is a model parameter stored in the parameter storage unit 30 (l = 0 to L, A feature vector z (t, t) based on a condition representing a sound source position based on f = 1 to F and a concentration parameter ((l, f) (l = 0 to L, f = 1 to F) The mixed model of Formula (21) which is a weighted sum of the conditional probability distribution of f) is applied to the feature vector z (t, f) calculated by the feature vector calculation unit 20 to calculate the prior probability distribution. In this embodiment, it is assumed that the prior probability distribution P (g (t, f) = 1) depends on the frame, and is represented by α (l, t) (l = 0 to L, t = 1 to T). α (l, t) satisfies the constraint condition α (0, t) +... + α (L, t) = 1. There are various methods for applying the mixed model of Equation (21) to the feature vector z (t, f), but in the present embodiment, the likelihood of Equation (21) is a priori probability distribution α (l, t) by maximizing with respect to (l = 0 to L, t = 1 to T).

The processing in the prior probability distribution calculation unit 40 is, for example, as follows.
1. The prior probability distribution α (l, t) (l = 0 to L, t = 1 to T) is initialized by α (l, t) ← 1 / (L + 1).
2. Updating of the prior probability distribution α (l, t) (l = 0 to L, t = 1 to T) according to the following equation (55) and equation (56) is alternately repeated a predetermined number of times (for example, 10 times).

  3. The prior probability distribution α (l, t) (l = 0 to L, t = 1 to T) is output.

  Here, the vector ~ α (t) (the symbol "~" in front of α represents adding the symbol "~" on α) is composed of α (l, t) (l = 0 to L) (L + 1) -dimensional vertical vector, vector ~ w (t, f) consists of W (z (t, f); a (l, f), ((l, f)) (l = 0 to L) (L + 1) -dimensional vertical vector. The derivation of the equation (55) and the equation (56) is the same as that of the first embodiment, and is therefore omitted.

  The sound source position calculating unit 50 detects the set G of sound source positions detected based on the prior probability distribution α (l, t) (l = 0 to L, t = 1 to T) received from the prior probability distribution calculating unit 40. (T) Calculate (t = 1 to T) and output. Specifically, α (l, t) in which the domain of the prior probability distribution α (l, t) (l = 0 to L, t = 1 to T) is limited to l = 1 to L corresponding to the target sound source A set G (t) of sound source positions detected by applying the processing in the sound source position calculation unit 50 of the second embodiment to (l = 1 to L, t = 1 to T) Calculate 1 to T).

  The mask estimation unit 70 calculates the average direction vector a (l, f) (l = 0 to L, f = 1 to F) from the parameter storage unit 30 and the concentration parameter パ ラ メ ー タ (l, f) (l = 0 to L, f = 1 to F), the prior probability distribution α (l, t) (l = 0 to L, t = 1 to T) from the prior probability distribution calculating unit 40, and the sound source for each sound source from the tracking unit 51 for each frame Receiving position (azimuth angle) ((n, t) (n = 1 to N, t = 1 to T), background noise to feature vector z (t, f) and contribution of each target signal at each time frequency point Calculate and output a mask γ (n, t, f) (n = 0 to N, t = 1 to T, f = 1 to F) which is a degree (a posterior probability). Here, γ (0, t, f) is a mask corresponding to background noise, and γ (n, t, f) (n = 1 to N) is a mask corresponding to the target signal n.

The mask γ (n, t, f) can be calculated by various methods, for example, as follows.
1. A posteriori probability P (g (t, f) = l | z (t, f)) (l = 0 to g (t, f) = 1 under the condition that the feature vector z (t, f) is given L, t = 1 to T, f = 1 to F) are calculated by the following equations (57) and (58).

  2. The mask γ (0, t, f) (t = 1 to T, f = 1 to F) corresponding to the background noise is calculated by the following equation (59).

  3. A set J (n, t) (n = 1 to N, t = 1 to T) of the number l of sound source position candidates corresponding to each target signal n in the frame t is calculated by the following equation (60).

  4. A mask γ (n, t, f) (n = 1 to N, t = 1 to T, f = 1 to F) corresponding to the target signal is calculated by the following equation (61).

  5. The mask γ (n, t, f) (n = 0 to N, t = 1 to T, f = 1 to F) is output.

  The signal emphasizing unit 80 obtains the observed signal vector y (t, f) from the time frequency analysis unit 10, and the dilation result d (n, t) (n) taking any value of 0 or 1 from the dilation unit 60. = 1 to N, t = 1 to T), background noise from the mask estimation unit 70, and masks γ (n, t, f) of target signals (n = 0 to N, t = 1 to T, f = 1 to F) to estimate each target signal s (n, τ).

An example of specific processing in the signal emphasizing unit 80 is as follows.
1. The covariance matrix ((f) of the observed signal is calculated by the following equation (62).

  2. A mask corrected by using the dilation result d (n, t) (n = 1 to N, t = 1 to T) ~ γ (n, t, f) (n = 0 to N, t = 1 to T, f = 1 to F) is calculated by the following equation (63) and equation (64). Equation (63) means that the mask of sound source n in frame t is replaced with 0 when d (n, t) = 0. Further, equation (64) is a process for causing the sum of n of the masks ̃γ (n, t, f) to be one.

  3. The covariance matrix Ψ (n, f) (n = 0 to N, f = 1 to F) is calculated by the following equation (65). Here, the matrix Ψ (0, f) is a covariance matrix corresponding to the background noise, and the matrix Ψ (n, f) (n = 1 to N) corresponds to the sum of the n-th target signal and the background noise It is a covariance matrix.

  4. Corresponds to the nth target signal by subtracting the covariance matrix Ψ (0, f) corresponding to the background noise from the covariance matrix Ψ (n, f) corresponding to the sum of the nth target signal and the background noise The covariance matrix Ψ (n, f) (n = 1 to N, f = 1 to F) to be calculated is obtained. Next, steering vectors h (n, f) (n = 1 to N, f = 1 to F) of the respective target signals are determined as eigenvectors corresponding to the maximum eigenvalues of the matrix Ψ (n, f). Then, h (n, f) に よ り h (n, f) / h (1, n, f) causes vector h (n, f) such that the first element of vector h (n, f) is equal to 1. Normalize). Here, h (1, n, f) represents the first element of the vector h (n, f).

  5. Based on the minimum dispersion beamformer, time frequency conversion s (n, t, f) (n = 1 to N, t = 1 to T, f = 1 to F) of each target signal is calculated by the following equation (67) Do.

  6. Each target signal is obtained by applying the inverse transform of the time frequency conversion to the time frequency conversion s (n, t, f) (n = 1 to N, t = 1 to T, f = 1 to F) of each target signal. Calculate s (n, τ).

Eleventh Embodiment
Next, the configuration of the eleventh embodiment will be described. In the eleventh embodiment, in a situation where N (N> 0) target voices are present under background noise, the waveform of each target voice is estimated based on the sound source position estimated by the present invention, and each target voice is In contrast, this is an example of speech recognition of each target speech by applying the existing speech recognition technology. According to the present invention, even in the situation where background noise and speech from a plurality of speakers are mixed, it is possible to separate mixed target signals into individual target signals and remove background noise to realize highly accurate speech recognition. An application example is, for example, automatic transcription of a conference held at a corner of an office where various sounds are sounding.

  The configuration of a signal processing apparatus according to the eleventh embodiment will be described with reference to FIG. FIG. 7 is a diagram showing an example of the configuration of a signal processing apparatus according to the eleventh embodiment. As shown in FIG. 7, the signal processing device 5 includes a time frequency analysis unit 10, a feature vector calculation unit 20, a parameter storage unit 30, an a priori probability distribution calculation unit 40, a sound source position calculation unit 50, a tracking unit 51, and a dilation unit. 60 includes a mask estimation unit 70, a signal enhancement unit 80, and a speech recognition unit 90. The time frequency analysis unit 10, the feature vector calculation unit 20, the parameter storage unit 30, the prior probability distribution calculation unit 40, the sound source position calculation unit 50, the tracking unit 51, the dilation unit 60, the mask estimation unit 70, and the signal enhancement unit 80 This is the same as the tenth embodiment. The speech recognition unit 90 receives the waveforms of the respective target signals from the signal enhancement unit 80 and applies existing speech recognition technology thereto to output a recognition result for each target signal.

[System configuration etc.]
Further, each component of each device illustrated is functionally conceptual, and does not necessarily have to be physically configured as illustrated. That is, the specific form of the distribution and integration of each device is not limited to the illustrated one, and all or a part thereof may be functionally or physically dispersed in any unit depending on various loads, usage conditions, etc. It can be integrated and configured. Furthermore, all or any part of each processing function performed by each device may be realized by a CPU and a program analyzed and executed by the CPU, or may be realized as wired logic hardware.

  Further, among the processes described in the present embodiment, all or part of the process described as being automatically performed may be manually performed, or the process described as being manually performed. All or part of them can be performed automatically by known methods. In addition to the above, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified.

[program]
The signal processing apparatuses 1 to 5 of the embodiment can be implemented by installing a signal processing program for executing the above-described sound source localization, tracking, dilation, voice emphasis, and voice recognition as package software or online software in a desired computer. For example, by causing the information processing apparatus to execute the above signal processing program, the information processing apparatus can be functioned as the signal processing apparatuses 1 to 5. The information processing apparatus referred to here includes a desktop or laptop personal computer. In addition, the information processing apparatus also includes mobile communication terminals such as smartphones, cellular phones and PHS (Personal Handyphone System), and slate terminals such as PDA (Personal Digital Assistant).

  The signal processing devices 1 to 5 can also be implemented as a signal processing server device that uses a terminal device used by a user as a client and provides the client with a service related to the above signal processing. For example, the signal processing server device is implemented as a server device that provides a sound source localization service in which an observation signal is input and a position of a sound source is output. In this case, the signal processing server apparatus may be implemented as a Web server, or may be implemented as a cloud that provides the above-mentioned signal processing service by outsourcing.

  FIG. 8 is a diagram illustrating an example of a computer in which a signal processing apparatus is realized by executing a program. The computer 1000 includes, for example, a memory 1010 and a CPU 1020. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These units are connected by a bus 1080.

  The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012. The ROM 1011 stores, for example, a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. Disk drive interface 1040 is connected to disk drive 1100. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to, for example, a mouse 1110 and a keyboard 1120. The video adapter 1060 is connected to, for example, the display 1130.

  The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, a program defining each process of the signal processing devices 1 to 5 is implemented as a program module 1093 in which a computer-executable code is described. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, a program module 1093 for executing the same processing as the functional configuration of the signal processing devices 1 to 5 is stored in the hard disk drive 1090. The hard disk drive 1090 may be replaced by an SSD.

  The setting data used in the process of the above-described embodiment is stored as program data 1094 in, for example, the memory 1010 or the hard disk drive 1090. Then, the CPU 1020 reads out the program module 1093 and the program data 1094 stored in the memory 1010 and the hard disk drive 1090 to the RAM 1012 as needed, and executes them.

  The program module 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1090, and may be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and the program data 1094 may be stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.). The program module 1093 and the program data 1094 may be read by the CPU 1020 from another computer via the network interface 1070.

1, 2, 3, 4, 5 Signal processing device 10 Time-frequency analysis unit 20 Feature vector calculation unit 30 Parameter storage unit 40 Prior probability distribution calculation unit 50 Sound source position calculation unit 51 Tracking unit 60 Dialing unit 70 Mask estimation unit 80 Signal Highlighter 90 Speech recognition unit

Claims (7)

A time-frequency analysis unit that applies time-frequency analysis to the recorded sound acquired at a plurality of different positions and calculates an observation signal vector that is an M-dimensional vector;
A feature vector calculation unit that calculates, for each time frequency point, a feature vector that is a vector including information on the direction of the observed signal vector calculated by the time frequency analysis unit;
A parameter storage unit storing model parameters of conditional probability distribution of the feature vector under a condition that a state representing a sound source position corresponds to each of a plurality of sound source position candidates;
Conditional probability distribution of the feature vector under conditions where the state representing the sound source position is known, based on the model parameters stored in the parameter storage unit, with the prior probability distribution of the state representing the sound source position as a load A prior probability distribution calculation unit which applies the mixed model, which is a weighted sum of the above, to the feature vector calculated by the feature vector calculation unit, and calculates the prior probability distribution;
A sound source position calculating unit that calculates a sound source position corresponding to the feature vector based on the prior probability distribution calculated by the prior probability distribution calculating unit;
A signal processing apparatus characterized by comprising: The prior probability distribution calculating unit uses, as a load, an a priori probability distribution for each time section of the state representing the sound source position, and the state representing the sound source position based on the model parameter stored in the parameter storage unit is known. Fitting a mixed model, which is a weighted sum of the conditional probability distributions of the feature vectors under the conditions, to the feature vectors calculated by the feature vector calculation unit, and calculating a prior probability distribution for each of the time intervals;
The sound source position calculation unit is characterized by calculating a sound source position for each of the time intervals corresponding to the feature vector based on the a priori probability distribution for each of the time intervals calculated by the a priori probability distribution calculation unit. The signal processing device according to claim 1.   The parameter storage unit is a model parameter learned using learning data acquired under reverberation, and a condition representing a sound source position takes a state corresponding to each of a plurality of sound source position candidates, The signal processing apparatus according to claim 1, wherein model parameters of the conditional probability distribution of the feature vector are stored.   The said parameter memory | storage part further stores the model parameter of conditional probability distribution in the condition which takes the state which respond | corresponds to a background noise the state showing the said sound source position, The conditions any one of Claim 1 to 3 characterized by the above-mentioned. The signal processing device according to item 1.   The signal processing apparatus according to any one of claims 1 to 4, wherein the prior probability distribution calculating unit calculates the prior probability distribution based on a gradient method. A signal processing method to be executed by a signal processing device, comprising:
Applying time-frequency analysis to the recorded sound acquired at a plurality of different positions, and calculating an observation signal vector which is an M-dimensional vector;
Calculating a feature vector, which is a vector including information on the direction of the observed signal vector calculated by the time frequency analysis step, for each time frequency point;
Obtaining a model parameter stored in a parameter storage unit storing a model parameter of the conditional probability distribution of the feature vector under a condition that the state indicating the sound source position corresponds to each of a plurality of sound source position candidates; A mixture based on the model parameters, a load sum of conditional probability distributions of the feature vectors under a condition in which the state representing the sound source position is known, wherein a load is a prior probability distribution of the state representing the sound source position; A prior probability distribution calculating step of fitting a model to the feature vector calculated by the feature vector calculating step, and calculating the prior probability distribution;
A sound source position calculating step of calculating a sound source position corresponding to the feature vector based on the prior probability distribution calculated by the prior probability distribution calculating step;
A signal processing method comprising:   A signal processing program for causing a computer to function as the signal processing device according to any one of claims 1 to 5.

Images