JP2018032001A - Signal processing device, signal processing method and signal processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately perform sound source localization even in the case that observation signal length is short.SOLUTION: A time frequency analysis part 10 applies time frequency conversion to recorded sound acquired at different positions, and calculates an observation signal vector. A feature vector calculation part 20 calculates a feature vector including information of the direction of the observation signal vector at each time frequency point. A parameter storage part 30 stores a model parameter of a conditional probability distribution of the feature vector under a condition that a state representing a sound source position takes a state corresponding to each of a plurality of sound source position candidates. A prior probability distribution calculation part 40 adapts a mixture model in which the state representing the sound source position based on the model parameter of the parameter storage part 30 with a prior probability distribution of the state as a load is a load sum of the conditional probability distribution of the feature vector under a known condition to the feature vector, and calculates a prior probability distribution. A sound source position calculation part 50 calculates a sound source position on the basis of the prior probability distribution.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a signal processing device, a signal processing method, and a signal processing program.

  Conventionally, a sound source localization technique for estimating the position of a sound source that generates sound based on recorded sound observed with a plurality of microphones or the like is known. As a sound source localization technique, for example, a method of estimating a sound source position using a covariance matrix estimated by applying time-frequency analysis to an observation signal on the assumption that the number of sound sources is known is known.

R. O. Schmidt, "Multiple emitter location and signal parameter estimation," IEEE Transactions on Antennas and Propagation, March 1986, vol.AP-34, No.3, p.276-280. N. Ito, E. Vincent, N. Ono, R. Gribonval, and S. Sagayama, "Crystal-MUSIC: Accurate localization of multiple sources in diffuse noise environments using crystal-shaped microphone arrays," September 2010, Proceedings of 9th International Conference on Latent Variable Analysis and Signal Separation (LVA / ICA), p.81-88.

  However, the conventional sound source localization technique has a problem that sound source localization may not be performed accurately when the observation signal length is short. For example, when the observation signal length is short, sufficient samples for estimation of the covariance matrix cannot be obtained, and sound source localization may not be performed accurately.

  The signal processing apparatus according to the present invention applies time-frequency analysis to recorded sound acquired at a plurality of different positions, calculates an observed signal vector that is an M-dimensional vector, and is calculated by the time-frequency analysis unit. A feature vector calculation unit that calculates, for each time frequency point, a feature vector that is a vector including information on the direction of the observed signal vector y (t, f), and a state that represents the sound source position is a plurality of sound source position candidates. A parameter storage unit that stores model parameters of the conditional probability distribution of the feature vector under conditions corresponding to the respective states, and the parameter storage unit that uses the prior probability distribution of the state representing the sound source position as a load The load of the conditional probability distribution of the feature vector under the condition that the state representing the sound source position is known based on the model parameter stored in Is applied to the feature vector calculated by the feature vector calculation unit, the prior probability distribution calculation unit for calculating the prior probability distribution, and the prior probability distribution calculated by the prior probability distribution calculation unit. And a sound source position calculation unit that calculates a sound source position corresponding to the feature vector.

  The signal processing method of the present invention is a signal processing method executed by a signal processing apparatus, and applies an observation frequency vector that is an M-dimensional vector by applying time-frequency analysis to recorded sound obtained at a plurality of different positions. And a feature vector calculation step of calculating, for each time frequency point, a feature vector which is a vector including information on the direction of the observed signal vector y (t, f) calculated by the time frequency analysis step. And a model parameter stored in a parameter storage unit that stores a model parameter of the conditional probability distribution of the feature vector under a condition that the state representing the sound source position corresponds to each of a plurality of sound source position candidates. The state representing the sound source position based on the model parameter is obtained, with the prior probability distribution of the state representing the sound source position as a load. Prior probability distribution calculation for applying the mixed model, which is a weighted sum of conditional probability distributions of the feature vector under knowledge conditions, to the feature vector calculated by the feature vector calculation step and calculating the prior probability distribution And a sound source position calculating step of calculating a sound source position corresponding to the feature vector based on the prior probability distribution calculated by the prior probability distribution calculating step.

  According to the present invention, sound source localization can be performed accurately even when the observation signal length is short.

FIG. 1 is a diagram for explaining sound source localization in the present invention. FIG. 2 is a diagram illustrating an example of the configuration of the signal processing device according to the first embodiment. FIG. 3 is a flowchart showing a processing flow of the signal processing apparatus according to the first embodiment. FIG. 4 is a diagram illustrating an example of a configuration of a signal processing device according to the eighth embodiment. FIG. 5 is a diagram illustrating an example of a configuration of a signal processing device according to the ninth embodiment. FIG. 6 is a diagram illustrating an example of a configuration of a signal processing device according to the tenth embodiment. FIG. 7 is a diagram illustrating an example of the configuration of the signal processing device according to the eleventh embodiment. FIG. 8 is a diagram illustrating an example of a computer in which a signal processing apparatus is realized by executing a program.

  Hereinafter, embodiments of a signal processing device, a signal processing method, and a signal processing program according to the present application will be described in detail with reference to the drawings. In addition, this invention is not limited by this embodiment.

[Sound source localization in the present invention]
The sound source signal usually has sparseness that has a large power only at sparse points on the time-frequency plane, so even if multiple sound source signals are sounding at the same time, the observation signal is not included in the sound source signal at each time-frequency point. It can be considered that only one is included at most. Therefore, for example, an observation signal vector y (t, f) (t is a frame number (t = t = f)) which is an M-dimensional vertical vector formed by time-frequency conversion of observation signals acquired at M (M> 1) different positions. 1 to T), f is a frequency bin number (f = 1 to F)) is directed to a specific direction determined by the sound source position of the sound source signal included in the observation signal at the time frequency point (t, f). Can be considered. Precisely, due to the influence of noise and reverberation, the direction of the observation signal vector y (t, f) is distributed with a certain extent around the unique direction determined by the above-mentioned sound source position. If the above-mentioned property of the observation signal is used, the sound source position can be estimated based on the direction of the observation signal vector y (t, f).

  In the embodiment of the present invention, the sound source localization is a problem of identifying a sound source that actually emits sound among a plurality (L) of sound source position candidates, that is, a sound source position candidate that actually emits a sound (number) ) Is formulated as a problem of estimating the set. As the sound source position candidates, for example, a plurality of locations in a room where sound source localization is performed (for example, locations corresponding to lattice points when the room is finely divided into a lattice shape) may be used as sound source position candidates. it can. In addition, when a region where a sound source can exist is known, a plurality of locations in the region can be set as sound source position candidates. For example, when sound source localization is performed on the recorded sound of conversations of multiple people sitting around the table, it can be considered that the speaker who is the sound source can exist only near the outer periphery of the table, so multiple locations near the outer periphery of the table Can be set as sound source position candidates (see FIG. 1). Therefore, based on the above properties of the observation signal, in the embodiment of the present invention, the sound source of the feature vector z (t, f), which is a vector including information on the direction of the observation signal vector y (t, f). A model parameter of a conditional probability distribution under a condition in which a state representing a position corresponds to each of a plurality (L) of sound source position candidates is stored, and the sound source position is estimated using the model parameter as prior information. To use. As described above, since the direction of the observation signal vector y (t, f) can be considered to be determined by the sound source position, the feature vector z which is a vector including information on the direction of the observation signal vector y (t, f). (T, f) has a unique probability distribution determined by the sound source position. The conditional probability distribution represents the probability distribution of the feature vector z (t, f) under the condition that the state representing the sound source position takes a state corresponding to each of a plurality (L) sound source position candidates.

  The direction of the observation signal vector y (t, f) is mathematically the element ratio y (1, t, f) for all microphones of the observation signal vector y (t, f): y (2, t, f, f):...: points to y (M, t, f) (in other words, a space obtained by equating vectors having a scalar multiple relationship to each other in an M-dimensional vector space on a complex number field, This refers to the element of the M-1 dimensional projective space on the complex number field). Here, y (m, t, f) represents the m-th element of the vector y (t, f). Therefore, the feature vector z (t, f) is a vector including information on the direction of the observation signal vector y (t, f). When the feature vector z (t, f) is given, the observation signal vector Element ratio y (1, t, f) for all microphones of y (t, f): y (2, t, f): ... means that y (M, t, f) is uniquely determined. To do. For example, a unit vector parallel to the observation signal vector can be used as the feature vector that is a vector including information on the direction of the observation signal vector y (t, f). Further, since the observation signal vector y (t, f) itself naturally includes information on the direction of the observation signal vector y (t, f), this is used as information on the direction of the observation signal vector y (t, f). It can also be used as a feature vector that is an included vector. A feature vector, which is a vector including information on the direction of the observation signal vector y (t, f), includes information on both the phase difference and the amplitude ratio as information on the sound source position. This is greatly different from conventional feature quantities (for example, Time Difference of Arrival (TDOA) and Direction Of Arrival (DOA)) that use only the phase difference without using the amplitude ratio. Therefore, the feature vector which is a vector including information on the direction of the observation signal vector y (t, f) relates to more sound source positions compared to the conventional feature amount using only the phase difference without using the amplitude ratio. Information is used, and more accurate sound source localization is possible. In addition, since information on the sound source position can be extracted to the maximum from the limited data length, it can be said that sound source localization can be performed accurately even when the observation signal length is short in the embodiment of the present invention. Contributes to the features. By using a feature vector that is a vector including information on the direction of the observed signal vector y (t, f), more effective signal processing (for example, sound source separation and noise) than in the case of using only the phase difference. (H. Sawada, S. Araki, and S. Makino, “Underdetermined Convolutive Blind Source Separation via Frequency Bin-Wise Clustering and Permutation Alignment,” IEEE Transactions on Audio , Speech, and Language Processing, vol. 19, no. 3, pp. 516-527, Mar. 2011. Note that the feature vector, which is a vector including information on the direction of the observed signal vector y (t, f), includes information on both the phase difference and the amplitude ratio as information on the sound source position. Can be explained as follows. As described above, the direction of the observation signal vector y (t, f) is the element ratio y (1, t, f) for all microphones of the observation signal vector y (t, f): y (2, t, f). ):...: Y (M, t, f), which is the element ratio y (m, t, f) for two microphones (m, n) for all microphone pairs (m, n). ): Equivalent to y (n, t, f) as information. Furthermore, note that the complex ratio is informationally equivalent to the phase difference and absolute value ratio (amplitude ratio), and the element ratio for two microphones (m, n) for all microphone pairs (m, n). y (m, t, f): y (n, t, f) is equivalent as information to the phase difference and amplitude ratio for two microphones (m, n) for all microphone pairs (m, n). . Therefore, the direction of the observed signal vector y (t, f) is equivalent as information with respect to the phase difference and amplitude ratio for the two microphones (m, n) for all microphone pairs (m, n). In other words, the feature vector, which is a vector including information on the direction of the observed signal vector y (t, f), includes information on both the phase difference and the amplitude ratio as information on the sound source position.

  An example in which sound source localization is performed on recorded sounds of conversations of a plurality of people sitting around a table will be described with reference to FIG. FIG. 1 is a diagram for explaining sound source localization in the present invention. First, as shown in FIG. 1, the signal processing apparatus can set the L point obtained by finely dividing the area around the table 100 at equal intervals as the sound source position candidate 110. In the example of FIG. 1, L = 8. In addition, three microphones 120 are placed on the table 100. In this example, it can be considered that the sound source can exist only on the outer periphery of the table, and the sitting height can be considered to be almost constant regardless of the individual, and therefore the sound source position can be designated by the direction (azimuth angle) seen from the microphone 120. .

  The signal processing device, based on the observation signal observed by the microphone 120, the feature vector z (which is a vector including information on the direction of the observation signal vector y (t, f) and the observation signal vector y (t, f). t, f) is calculated. Then, the signal processing device, based on the model parameter of the conditional probability distribution, satisfies the condition of the feature vector z (t, f) under the condition that the state representing the sound source position corresponds to each of a plurality of sound source position candidates. By applying the mixed model, which is the load sum of the attached probability distribution, to the feature vector z (t, f), the prior probability distribution that is the load in the load sum is calculated. At this time, since the calculated prior probability distribution takes a large value at the sound source position, the sound source position can be estimated based on the prior probability distribution. At this time, for example, when the prior probability distribution has the largest value among the sound source position candidates 110 with l = 2, the sound source position can be regarded as the direction indicated by the arrow 130.

[First Embodiment]
The signal processing apparatus according to the first embodiment estimates a set of sound source positions under a condition where the number N of sound sources is unknown. Here, the number N of sound sources may be N = 0 (corresponding to the case where the set of sound source positions is an empty set). In the present embodiment, the signal processing apparatus uses the direction vector of the observation signal vector y (t, f) as the feature vector z (t, f) that is a vector including information on the direction of the observation signal vector y (t, f). , And a state corresponding to each of the plurality of sound source position candidates as a state representing the sound source position, and a feature vector z () under a condition that the state representing the sound source position corresponds to each of the plurality of sound source position candidates. The complex Watson distribution is used as the conditional probability distribution of t, f), and the model parameters of the complex Watson distribution are calculated and stored based on the assumption that the target signal propagates as a spherical wave. Use a probability distribution.

  The configuration of the signal processing apparatus according to the first embodiment will be described with reference to FIG. FIG. 2 is a diagram illustrating an example of the configuration of the signal processing device according to the first embodiment. As shown in FIG. 2, the signal processing device 1 includes a time-frequency analysis unit 10, a feature vector calculation unit 20, a parameter storage unit 30, a prior probability distribution calculation unit 40, and a sound source position calculation unit 50.

  The time frequency analysis unit 10 is an observation signal y (m, τ) (m is a microphone number (m = 1 to M), and τ is a time of the recorded sound obtained at a plurality of different positions. Time frequency analysis is applied to the number), and the time frequency conversion y (m, t, f) of the observation signal (t is the frame number (t = 1 to T), and f is the frequency bin number (f = 1 to F). )) Is calculated, and an observation signal vector y (t, f), which is an M-dimensional vertical vector composed of y (m, t, f) (m = 1 to M), is created. The recorded sounds acquired at the plurality of different positions may be recorded sounds obtained at a plurality of different positions and then subjected to some preprocessing (for example, dereverberation processing, spatial whitening processing, etc.) (references). “T. Yoshioka, T. Nakatani, M. Miyoshi, and HG Okuno,“ Blind separation and dereverberation of speech mixture by joint optimization, ”IEEE Trans. Audio, Speech, Language Process., Vol. 19, no. 1, pp 69-84, 2011. ”and references“ H. Sawada, S. Araki, and S. Makino, “Underdetermined Convolutive Blind Source Separation via Frequency Bin-Wise Clustering and Permutation Alignment,” IEEE Transactions on Audio, Speech, and. Language Processing, vol. 19, no. 3, pp. 516-527, Mar. 2011.)).

  The feature vector calculation unit 20 receives the observation signal vector y (t, f) from the time-frequency analysis unit 10, and the feature vector z (t, which is a vector including information on the direction of the observation signal vector y (t, f). , F) is calculated by equation (1).

  Here, || · || is an Euclidean norm, and an arrow ← indicates that the right side is assigned to the left side. In the modeling in this embodiment, it is assumed that the observed signal vector y (t, f) is composed of N target signals (N may be unknown or N = 0) and does not include background noise. In the modeling in the embodiment of the present invention, it is assumed that each target signal has sparsity that has a large power only at a sparse point on the time-frequency plane. Based on these assumptions, in the present embodiment, it is assumed that the observed signal vector y (t, f) includes only one target signal at each time frequency point. That is, the observation signal vector y (t, f) is modeled by the equation (2).

  Here, s (n, t, f) is the time-frequency conversion of the nth target signal, and n is the number of the target signal (n = 1 to N). The vector h (n, f) is a steering vector representing the spatial transfer characteristic of the nth target signal, and takes a specific value depending on the sound source position of the nth target signal. Expression (2) indicates that the observed signal vector y (t, f) is composed of only the nth target signal (n varies depending on the time frequency point (t, f)).

  The direction of the observed signal vector y (t, f) in the M-dimensional complex vector space (that is, the one-dimensional subspace spanned by the observed signal vector y (t, f) in the M-dimensional complex vector space) is the time frequency point (t , F) is a specific direction (specifically, the direction of the steering vector h (n, f)) determined by the sound source position of the sound source signal included in the observation signal. More precisely, due to the influence of noise and reverberation, the direction of the observation signal vector y (t, f) is distributed with a certain extent around the inherent direction determined by the sound source position. In the present embodiment, the feature vector of Expression (1), which is the direction vector of the observation signal vector y (t, f), is used as the feature vector that includes information on the direction of the observation signal vector y (t, f). Use.

  In the present embodiment, the conditional probability distribution of the feature vector z (t, f) under the condition that the state representing the sound source position corresponds to each of a plurality (L) of sound source position candidates is represented by a complex Watson distribution. Model (other complex Bingham distribution, complex angular central Gaussian distribution, complex Gaussian distribution, mixed complex Watson distribution, mixed complex Bingham distribution, mixed complex angular central Gaussian distribution, mixed complex Gaussian distribution, etc. Can be modeled by the probability distribution). That is, the feature vector z (t, f) is modeled by Equation (3).

  Here, g (t, f) is a state representing the sound source position at the time frequency point (t, f). In the present embodiment, the state representing the sound source position is assumed to take any one of the states 1 to L corresponding to each of a plurality (L) of sound source position candidates. Here, the state l is defined as a state in which the sound source position of the sound source signal included in the observation signal vector y (t, f) at the time frequency point (t, f) is the lth sound source position candidate. p (z (t, f) | g (t, f) = l) is a conditional probability distribution of the feature vector z (t, f) under the condition of g (t, f) = 1. The vector a (l, f) is a model parameter that determines the average direction of the feature vector z (t, f) for the l-th sound source position candidate, and is referred to as an average direction vector and satisfies the equation (4). κ (l, f) is a model parameter that determines the degree of concentration around the mean direction vector a (l, f) of the probability distribution of the feature vector z (t, f) for the l-th sound source position candidate. Called.

  W (z; a, κ) is a complex Watson distribution of a vector z having an average direction vector a and a concentration parameter κ, and is represented by Expression (5).

  At this time, K is a Kummer function (first-type confluent hypergeometric function) defined by the infinite series of Equation (6), and the superscript H is Hermitian transpose. However, when i = 0, ξ (ξ + 1)... (Ξ + i−1) / [η (η + 1)... (Η + i−1)] = 1.

  The parameter storage unit 30 stores model parameters of a conditional probability distribution of feature vectors under a condition in which a state representing a sound source position corresponds to each of a plurality of sound source position candidates. Specifically, the parameter storage unit 30 is an average direction vector a (l, f) (l = 1 to L, f = 1 to 1) which is a model parameter for modeling the sound source position of the conditional probability distribution of Expression (3). F) and the concentration parameter κ (l, f) (l = 1 to L, f = 1 to F) are stored. In the present embodiment, these model parameters are calculated as follows. That is, based on the assumption that the target signal propagates as a spherical wave, the m-th element of the average direction vector a (l, f) is calculated by Expression (7).

  Here, the vector q (m) is a three-dimensional real vector (assumed to be known in the present embodiment) that is an orthogonal coordinate of the mth microphone, and the vector r (l) is an orthogonal coordinate of the lth sound source position candidate 3. Dimensional real vector (known), j is an imaginary unit, ω (f) is the angular frequency of the f th frequency bin, c is the speed of sound, subscript m on the left side represents the m th element, The term of the square root of the denominator is a normalization coefficient for making the average direction vector a (l, f) satisfy the constraint condition of the equation (4).

  On the other hand, the concentration parameter κ (l, f) is calculated by the equation (8) assuming that it is proportional to the minus square of the frequency (ω (f) / 2π), for example. Expression (8) is based on the property that the direction of the observation signal vector y (t, f) has a smaller dispersion (large concentration) as the frequency is lower. As described above, by appropriately considering the property, the estimation of the prior probability distribution and the sound source localization based on the prior probability distribution can be accurately performed. The proportionality constant β may be determined in any way, for example, β = 6.4 × 10 ^ 7 Hz ^ 2.

  Next, the modeling of the peripheral probability distribution of the feature vector z (t, f) in the present embodiment will be described. In the present embodiment, the peripheral probability distribution of the feature vector z (t, f) is a condition with the prior probability distribution P (g (t, f) = 1) of the state g (t, f) representing the sound source position as a load. Modeled by the mixed model of Equation (9), which is the load sum of the attached probability distribution p (z (t, f) | g (t, f) = l).

  The conditional probability distribution p (z (t, f) | g (t, f) = 1) is the probability distribution of the feature vector z (t, f) when the state representing the sound source position is known. The peripheral probability distribution p (z (t, f)) in the equation (9) is a probability distribution of the feature vector z (t, f) when the state representing the sound source position is unknown. Prior probability distribution P (g (t, f) = 1) may be “time-varying” or “time-invariant”. In the former case, the prior probability distribution P (g (t, f) = 1) can take a different value for each time interval (for example, frame). In the latter case, the prior probability distribution P (g (t, f) = 1) takes the same value regardless of the time interval (for example, frame).

  When the prior probability distribution P (g (t, f) = 1) is time-invariant, the prior probability distribution that takes a large value at the sound source position is estimated using all time intervals (for example, frames). Since data longer than the case can be used for estimation, there is an effect that the position of the sound source can be estimated more accurately in a situation where there is no movement of the sound source or utterance change. On the other hand, sound source position estimation cannot be performed for each time interval (for example, frame), and therefore tracking and dialization in a dynamic situation with sound source movement and utterance change is more time-varying. Suitable for etc.

  On the other hand, when the prior probability distribution P (g (t, f) = 1) is time-varying, the prior probability distribution, which is a function that takes a large value in the sound source position, is estimated for each time interval (for example, frame). In addition to the effect that estimation can be performed for each time interval (for example, frame), there is an effect that tracking and dialization can be performed based on sound source position estimation for each time interval (for example, frame). For example, in speech recognition of multi-person conversation, it is necessary to cut out the section to which speech recognition should be applied by performing dialization that estimates "when and who spoke" in order to prevent misrecognition by regarding noise as speech. However, “time-varying” can be applied to such a case.

  In the present embodiment, it is assumed that the prior probability distribution P (g (t, f) = 1) is time invariant. In the present embodiment, it is further assumed that the prior probability distribution P (g (t, f) = 1) does not depend on the frequency. That is, in the present embodiment, it is assumed that the prior probability distribution P (g (t, f) = 1) does not depend on the frame and the frequency bin, and is represented by α (l). However, α (l) satisfies the constraint condition α (1) +... + Α (L) = 1. By using the prior probability distribution that does not depend on the frequency, it is possible to estimate the prior probability distribution using information of the feature vector z (t, f) observed at all frequencies, and thus the prior probability distribution that depends on the frequency. Compared to the case of using, more information can be used to estimate the prior probability distribution, more accurate prior probability distribution estimation and sound source localization can be realized, and more accurate even when the observation signal length is short It is possible to estimate a prior probability distribution and sound source localization based on it.

  The prior probability distribution calculation unit 40 uses an average direction vector a (model parameter stored in the parameter storage unit 30 with the prior probability distribution α (l) (l = 1 to L) in a state representing the sound source position as a load. l, f) and a concentration model κ (l, f) based on a known model representing a sound source position, a mixed model that is a weighted sum of conditional probability distributions of feature vectors is represented by a feature vector calculator 20. The prior probability distribution α (l) (l = 1 to L) is calculated by applying to the feature vector calculated by (1).

  There are various methods for applying the mixed model of Equation (9) to the feature vector z (t, f). For example, the likelihood related to Equation (9) is used as an objective function (the posterior probability is also used as the objective function). This is maximized with respect to the prior probability distribution α (l) (l = 1 to L) by the gradient method (in addition, it can be maximized by an Expectation-Maximization (EM) algorithm or the like).

  The method based on the gradient method is advantageous in terms of calculation amount compared with the method based on the EM algorithm. In the method based on the EM algorithm, it is necessary to calculate the contribution ratio of each sound source position candidate for each time frequency point in addition to the prior probability distribution α (l) (l = 1 to L) for each iteration. On the other hand, in the gradient method, since only the prior probability distribution α (l) (l = 1 to L) needs to be calculated for each iteration, the calculation amount can be greatly reduced as compared with the EM algorithm. The processing in the prior probability distribution calculation unit 40 is, for example, as follows.

  First, α (l) is initialized by α (l) ← 1 / L (l = 1 to L). Next, the processing of α (l) (l = 1 to L) by the following formulas (10) and (11) is alternately repeated a predetermined number of times (for example, 10 times).

  Then, α (l) (l = 1 to L) is output. However, the vector α is an L-dimensional vertical vector composed of α (l) (l = 1 to L), and the vector w (t, f) is W (z (t, f); a (l, f), κ (l , F)) (L = 1 to L), an L-dimensional vertical vector, superscript T is transposed, and λ is a predetermined positive constant (for example, λ = 1).

  Here, the derivation of the equations (10) and (11) will be described. The likelihood, which is an objective function, is the probability that z (t, f) (t = 1 to T, f = 1 to F) is observed, and is represented by equation (12).

  Maximization of equation (12) is equivalent to maximization of equation (13) taking the natural logarithm.

  Here, ln represents a natural logarithm, and Δ above = represents a definition. Taking the slope of equation (13) gives equation (14), from which equation (10) follows. On the other hand, Expression (11) is a process for making α (l) satisfy the constraint condition α (1) +... + Α (L) = 1. In Equation (13), instead of taking a sum without using a load, an objective function that is changed so as to take a load sum using a load based on reliability may be used. Thereby, a greater weight can be given to the feature vector at the time frequency point with high reliability, and the accuracy of the sound source localization based on the prior probability distribution estimation and it can be improved. For example, based on the assumption that a time frequency point with a small norm of the observed signal vector y (t, f) corresponds to noise and a time frequency point with a large norm corresponds to a target signal, the norm is a load based on reliability. Can be used as

  The sound source position calculation unit 50 receives the prior probability distribution α (l) (l = 1 to L) from the prior probability distribution calculation unit 40, and determines the peak position of the prior probability distribution α (l) (l = 1 to L). A set J is calculated, and a set G of sound source positions is calculated and output based on the set J of peak positions.

  The set J of peak positions can be calculated as follows, for example. Assume that for each number l = 1 to L, a set of numbers of sound source position candidates adjacent to the l-th sound source position candidate is known. At this time, “the number l is the peak position means that α (l)> α (l ′) holds for the numbers l ′ of all sound source position candidates adjacent to the l-th sound source position candidate.” And a set J of peak positions can be calculated by determining whether or not the number 1 is a peak position for each of the numbers l = 1 to L. Based on the set J of peak positions, a set G of detected sound source positions, which is a set of numbers l or a set of coordinates (orthogonal coordinates, polar coordinates, spherical coordinates, etc.) for specifying a sound source position, can be calculated as follows. .

  For example, the set J of peak positions may be used as the set G of sound source positions detected as they are, or the set {l∈J | of the peak positions 1 in which the peak value α (l) exceeds a predetermined threshold S among the peak positions l. α (l)> S} may be a set G of detected sound source positions. The threshold value S may be determined in any way, for example, S = 1 / L. Further, a set {r (l) | lεJ} of vectors r (l) that are coordinates of sound source position candidates corresponding to the peak position l may be set as a detected sound source position set G, or a peak value α ( A set {r (l) | lεJ, α (l)> S} of vectors r (l), which are coordinates of a sound source position candidate corresponding to a peak position l where l) exceeds a predetermined threshold S, has been detected. A set G of sound source positions may be used.

[Process of First Embodiment]
A processing flow of the signal processing apparatus 1 will be described with reference to FIG. FIG. 3 is a flowchart showing a processing flow of the signal processing apparatus according to the first embodiment. As shown in FIG. 3, first, the time-frequency analysis unit 10 performs time-frequency analysis on the observation signal and calculates an observation signal vector (step S11).

  Next, the feature vector calculation unit 20 calculates a feature vector which is a vector including information on the direction of the observation signal vector y (t, f) (step S12). Then, the prior probability distribution calculation unit 40 obtains, from the parameter storage unit 30, the parameters of the conditional probability distribution model of the feature vector under the condition that the state representing the sound source position corresponds to each of a plurality of sound source position candidates. Obtain (step S13).

  Next, the prior probability distribution calculation unit 40 initializes a prior probability distribution in a state representing each sound source position (step S14). Then, the prior probability distribution calculation unit 40 updates the prior probability distribution (step S15).

  At this time, the prior probability distribution calculation unit 40 uses, for example, a mixture model in which the conditional probability distribution of the feature vector represented by the model parameter acquired from the parameter storage unit is loaded with the prior probability distribution. Model the distribution. Then, the prior probability distribution calculation unit 40 uses the gradient method to update the prior probability distribution so that the likelihood when the likelihood of the peripheral probability distribution is used as an objective function is maximized. If the prior probability distribution has not been updated a predetermined number of times (step S16, No), the prior probability distribution calculation unit 40 further updates the prior probability distribution (step S15).

  On the other hand, when the prior probability distribution is updated a predetermined number of times (step S16, Yes), the sound source position calculation unit 50 calculates the sound source position based on the prior probability calculated by the prior probability distribution calculation unit 40. (Step S17). At this time, the sound source position calculation unit 50 can use, for example, a sound source position at which the prior probability reaches a peak as a calculation result.

[Effect of the first embodiment]
The time-frequency analysis unit 10 applies time-frequency conversion to recorded sounds acquired at M different positions, and calculates an observation signal vector that is an M-dimensional vector. Then, the feature vector calculation unit 20 calculates a feature vector that is a vector including information on the direction of the observed signal vector y (t, f) calculated by the time frequency analysis unit 10 for each time frequency point. Further, the parameter storage unit 30 stores model parameters of a conditional probability distribution of feature vectors under a condition in which a state representing a sound source position takes a state corresponding to each of a plurality of sound source position candidates.

  Here, the prior probability distribution calculation unit 40 uses the prior probability distribution of the state representing the sound source position as a load, and is based on the model parameter stored in the parameter storage unit 30 under a condition where the state representing the sound source position is known. A mixed model, which is a weighted sum of conditional probability distributions of feature vectors, is applied to the feature vectors calculated by the feature vector calculation unit 20 to calculate a prior probability distribution. Then, the sound source position calculation unit 50 calculates the sound source position corresponding to the feature vector based on the prior probability distribution calculated by the prior probability distribution calculation unit 40.

  Thus, according to the first embodiment, it is possible to calculate a prior probability distribution that can be regarded as a spatial spectrum that is a function that takes a large value at the sound source position without using the covariance matrix of the observation signal vector. Sound source localization can be performed accurately even when the observation signal length is short. Therefore, compared to conventional sound source localization methods such as the Capon method and MUSIC method, where accurate sound source localization is difficult when the observation signal length is short, the situation where the sound source position changes over time and conversations with utterance changes It is advantageous in dynamic situations such as situations. Further, according to the first embodiment, the sound source position of each sound source can be estimated even in a situation where sound source signals from a plurality of sound sources are mixed. Therefore, compared to conventional sound source localization methods such as delay-and-sum arrays and generalized cross-correlation function methods where it was difficult to estimate multiple sound source positions, there are multiple sound sources such as conversation situations with overlapping utterances. Is advantageous. Further, sound source localization can be performed even in a situation where the number of sound sources is unknown. Therefore, in actual applications, the number of sound sources is often not known in advance, but sound source localization can be performed according to this embodiment even under such circumstances. This is advantageous compared to a conventional sound source localization method such as the MUSIC method that requires prior information on the number of sound sources. Furthermore, the prior probability distribution obtained by the method of the first embodiment can be used for various applications such as tracking, dialization, mask estimation, speech enhancement, and speech recognition. Furthermore, according to the first embodiment, the prior probability distribution is estimated using information on the feature vector z (t, f) observed at all frequencies by using the prior probability distribution that does not depend on the frequency. (This is also known from the fact that the vector α is updated using the vector w (t, f) at all frequencies in the equation (10).) Therefore, the prior probability distribution depending on the frequency is used. Compared to the case, more information can be used to estimate the prior probability distribution, more accurate prior probability distribution estimation and sound source localization can be realized, and more accurate prior estimation even when the observation signal length is short. Probability distribution estimation and sound source localization based on it can be realized. In the above, the batch processing for processing the observation signals in all the frames at a time has been described. However, the block batch processing for processing the observation signals for each frame (or every several frames) and estimating the sound source position (or Online processing).

[Second Embodiment]
Next, the configuration of the second embodiment will be described. The second embodiment is an example of estimating a sound source position based on the present invention, and is based on the first embodiment with a change that a time-varying prior probability distribution is used as the prior probability distribution. is there. That is, in the second embodiment, the prior probability distribution is estimated for each time interval (for example, frame). Thus, in addition to the effect that sound source position estimation can be performed for each time interval (for example, frame), tracking and dialization can be performed based on sound source position estimation for each time interval (for example, frame). Is obtained.

  An example of the configuration of the signal processing device according to the second embodiment is illustrated in FIG. 2 as in the case of the signal processing device 1 according to the first embodiment. The signal processing apparatus 1 according to the second embodiment includes a time-frequency analysis unit 10, a feature vector calculation unit 20, a parameter storage unit 30, a prior probability distribution calculation unit 40, and a sound source position calculation unit 50. Since the time frequency analysis unit 10, the feature vector calculation unit 20, and the parameter storage unit 30 are the same as those in the first embodiment, the prior probability distribution calculation unit 40 and the sound source position calculation unit 50, which are different points, are described below. explain in detail. The main differences between the first embodiment and this embodiment are as follows. In the first embodiment, the prior probability distribution calculation unit 40 calculates a prior probability distribution that does not depend on the time interval, and based on the prior probability distribution, the sound source position calculation unit 50 calculates a sound source position that does not depend on the time interval. On the other hand, in this embodiment, the prior probability distribution calculation unit 40 calculates the prior probability distribution for each time interval, and the sound source position calculation unit 50 calculates the sound source position for each time interval based on the prior probability distribution.

  The prior probability distribution calculation unit 40 is a model stored in the parameter storage unit 30 with the prior probability distribution α (l, t) (l = 1 to L, t = 1 to T) in a state representing the sound source position as a load. Conditional probability distribution of the feature vector z (t, f) under the condition that the state representing the sound source position is known based on the average direction vector a (l, f) and the concentration parameter κ (l, f) as parameters. Is applied to the feature vector z (t, f) calculated by the feature vector calculation unit 20, and a prior probability distribution α (l, t) (l = 1 to L, t = 1 to T) is calculated. However, α (l, t) satisfies the constraint condition α (1, t) +... + Α (L, t) = 1.

  Note that, unlike the first embodiment, the load in equation (15) is not time-invariant α (l) but time-variant α (l, t). There are various methods for applying the mixed model of Expression (15) to the feature vector z (t, f). For example, the likelihood related to Expression (15) is maximized by the gradient method.

The processing in the prior probability distribution calculation unit 40 is, for example, as follows.
1. Prior probability distribution α (l, t) is initialized by α (l, t) ← 1 / L (l = 1 to L, t = 1 to T).
2. The update of the prior probability distribution α (l, t) (l = 1 to L, t = 1 to T) according to the following equations (16) and (17) is alternately repeated a predetermined number of times (for example, 10 times).

  3. Prior probability distribution α (l, t) (l = 1 to L, t = 1 to T) is output.

  However, the vector α (t) is an L-dimensional vertical vector composed of α (l, t) (l = 1 to L). Since the derivations of the equations (16) and (17) are the same as the derivations of the equations (10) and (11), they are omitted.

  The sound source position calculation unit 50 receives the prior probability distribution α (l, t) (l = 1 to L, t = 1 to T) from the prior probability distribution calculation unit 40 and receives the prior probability distribution α (l, t) ( A set J (t) of peak positions of l = 1 to L, t = 1 to T) is calculated for each frame, and a set G (t) of sound source positions detected based on the peak position set J (t) Is calculated and output for each frame.

  The set of peak positions J (t) can be calculated as follows, for example. A set (assumed to be known) of numbers of sound source position candidates adjacent to the l-th (l = 1 to L) sound source position candidates is represented by a set A (l). At this time, the set J (t) of peak positions is a set J of number l such that “α (l, t)> α (l ′, t) for all numbers l ′ belonging to set A (l)”. (T) = {l | ∀l′∈A (l), α (l, t)> α (l ′, t)}. Based on this set of peak positions J (t), a set G (t) of detected sound source positions, which is a set of number l or a set of coordinates (orthogonal coordinates, polar coordinates, spherical coordinates, etc.) specifying the sound source position, is used. It can be calculated as follows:

  For example, the peak position set J (t) can be used as the detected sound source position set G (t). In addition, among the peak positions l, the sound source positions where the set {lεJ (t) | α (l, t)> S} in which the corresponding peak value α (l, t) exceeds a predetermined threshold S is detected. It can also be set G (t). Here, the threshold value S may be determined in any way, but for example, S = 1 / L. Further, a set {r (l) | l∈J (t)} of vectors r (l) which are coordinates of a sound source position candidate corresponding to the peak position l is set as a detected sound source position set G (t). You can also. Further, a set {r (l) | l∈J () of the vector r (l) corresponding to the coordinates of the sound source position candidate corresponding to the peak position l of which the peak value α (l, t) exceeds the predetermined threshold S. t), α (l, t)> S} may be a set G (t) of detected sound source positions.

[Third Embodiment]
Next, the configuration of the third embodiment will be described. The third embodiment is an example of estimating the sound source position based on the present invention. Based on the first embodiment, each of a plurality (L) of sound source position candidates is represented as a state representing the sound source position. In addition to the corresponding states (states 1 to L), the state corresponding to the background noise (state 0) is considered, and the condition of the feature vector under the condition that the state representing the sound source takes state 0 The attached probability distribution is modeled by a uniform distribution on the hypersphere. Thus, there is an advantage that an observation signal including background noise can be appropriately modeled, and sound source localization can be performed with high accuracy even under background noise.

  An example of the configuration of the signal processing device according to the third embodiment is illustrated in FIG. 2 as in the case of the signal processing device 1 according to the first embodiment. The signal processing apparatus 1 according to the third embodiment includes a time-frequency analysis unit 10, a feature vector calculation unit 20, a parameter storage unit 30, a prior probability distribution calculation unit 40, and a sound source position calculation unit 50. Since the time-frequency analysis unit 10 and the feature vector calculation unit 20 are the same as those in the first embodiment, the parameter storage unit 30, the prior probability distribution calculation unit 40, and the sound source position calculation unit 50, which are differences, will be described below. explain in detail. The main differences between the first embodiment and this embodiment are as follows. In the first embodiment, the parameter storage unit 30 stores the model parameters of the conditional probability distribution under the condition that the state representing the sound source position corresponds to each of a plurality of sound source position candidates, and the prior probability distribution The calculation unit 40 calculates a prior probability distribution for states corresponding to a plurality of sound source position candidates, and the sound source position calculation unit 50 calculates a sound source position based on the prior probability distribution. On the other hand, in the present embodiment, the parameter storage unit 30 further stores a model parameter of the conditional probability distribution under the condition that the state representing the sound source position corresponds to the background noise, and the prior probability distribution calculation unit In 40, a prior probability distribution in a state corresponding to a plurality of sound source position candidates and background noise is calculated, and a sound source position calculation unit 50 calculates a sound source position based on the prior probability distribution.

  First, modeling of the observation signal vector y (t, f) in the present embodiment will be described. In the modeling in this embodiment, it is assumed that the observed signal vector y (t, f) includes background noise in addition to N target signals (N may be unknown or N = 0). In the present embodiment, it is further assumed that the observed signal vector y (t, f) includes at most one target signal among the target signals at each time frequency point, and the background noise is observed signal vectors at all time frequency points. Assume that y (t, f) is included. At this time, the observation signal vector y (t, f) is modeled by any one of the equations (18) and (19).

  Here, the equation (18) indicates that only the n-th target signal (where n may vary depending on the time frequency point (t, f)) of the target signals at the time frequency point (t, f) is the observed signal vector y (t , F), Equation (19) represents a case where no observed signal is included in the observed signal vector y (t, f) at the time frequency point (t, f), and the vector s (n , T, f) is the nth target signal, and the vector v (t, f) is background noise.

  Unlike the case of the first embodiment, the present embodiment takes into account the case where the observation signal vector y (t, f) does not include any target signal and only the background noise is included as in the equation (19). Modeling has been done, and the observation signal under background noise can be modeled more accurately.

  As described above, in the present embodiment, a case where no observation signal is included in the observation signal vector y (t, f) as in Expression (19) is also considered. In the present embodiment, in order to appropriately model in such a case, the state representing the sound source position that can be taken by the observation signal vector at each time frequency point, in addition to the state corresponding to a plurality of sound source position candidates, Consider further the state corresponding to noise. The former corresponds to equation (18) and the latter corresponds to equation (19).

  Hereinafter, a state representing the sound source position at the time frequency point (t, f) is represented by g (t, f). The conditional probability distribution of the feature vector z (t, f) under the condition of g (t, f) = l (l = 1 to L) is expressed by the equation (3) as in the first embodiment. Modeled by complex Watson distribution (other probabilities such as complex Bingham distribution, complex angular center Gaussian distribution, complex Gaussian distribution, mixed complex Watson distribution, mixed complex Bingham distribution, mixed complex angular center Gaussian distribution, mixed complex Gaussian distribution, etc. Can be modeled by distribution).

  On the other hand, the conditional probability distribution of the feature vector z (t, f) under the condition of g (t, f) = 0 is one on the unit sphere in the M-dimensional complex vector space as shown in the equation (20). Modeled by a uniform distribution.

  Equation (20) is based on the assumption that background noise comes uniformly from all directions. In this embodiment, by introducing the equation (20), it becomes possible to appropriately model the state corresponding to the background noise as in the equation (19), and the sound source position is accurately estimated even under the background noise. it can.

  Next, the modeling of the peripheral probability distribution of the feature vector z (t, f) in the present embodiment will be described. In the present embodiment, the peripheral probability distribution of the feature vector z (t, f) is a conditional probability distribution p () with a prior probability distribution P (g (t, f) = 1) representing a sound source position as a load. Modeling is performed by the mixed model of Expression (21) which is a load sum of z (t, f) | g (t, f) = l).

  In this embodiment, it is assumed that the prior probability distribution P (g (t, f) = 1) does not depend on the frame and the frequency bin, and is expressed as α (1) (1 = 0 to L). However, α (l) satisfies the constraint condition α (0) +... + Α (L) = 1. Note that when κ = 0 and a is an arbitrary unit vector, the complex Watson distribution W (z; a, κ) matches the uniform distribution of Equation (20). It can also be rewritten as in (22). However, κ (0, f) = 0 and the vector a (0, f) is an arbitrary unit vector. By using the prior probability distribution that does not depend on the frequency, it is possible to estimate the prior probability distribution using information of the feature vector z (t, f) observed at all frequencies, and thus the prior probability distribution that depends on the frequency. Compared to the case of using, more information can be used to estimate the prior probability distribution, more accurate prior probability distribution estimation and sound source localization can be realized, and more accurate even when the observation signal length is short It is possible to estimate a prior probability distribution and sound source localization based on it. Furthermore, since the prior probability distribution can be estimated using the information of the feature vectors z (t, f) observed at all frequencies, the feature vector z (( Even when the sound source position is not known with t, f) alone, sound source localization can be performed more accurately, and it is more robust against noise and reverberation than using a frequency-dependent prior probability distribution. Can be improved.

  The parameter storage unit 30 includes a model parameter of a conditional probability distribution under a condition in which a state representing the sound source position corresponds to each of a plurality of sound source position candidates, and a state in which the state representing the sound source position corresponds to background noise The model parameter of the conditional probability distribution under the condition of taking The former can be calculated, for example, by the method described in the first embodiment, the latter can be, for example, κ (0, f) ← 0, and the vector a (0, f) can be an arbitrary unit vector.

  The prior probability distribution calculation unit 40 uses the average direction vector a (model parameter stored in the parameter storage unit 30 with the prior probability distribution α (l) (l = 0 to L) in a state representing the sound source position as a load. l, f) (l = 0 to L, f = 1 to F) and the state representing the sound source position based on the concentration parameter κ (l, f) (l = 0 to L, f = 1 to F) are known. The mixed model of Expression (21), which is the load sum of the conditional probability distribution of the feature vector under the condition, is applied to the feature vector z (t, f) calculated by the feature vector calculation unit 20, and the prior probability distribution α (L) Calculate (l = 0 to L).

  There are various methods for applying the mixed model of Equation (21) to the feature vector z (t, f). For example, the likelihood related to Equation (21) is used as an objective function (the posterior probability or the like is also used as the objective function). This is maximized with respect to the prior probability distribution α (l) (l = 0 to L) by the gradient method (otherwise, it can be maximized by the EM algorithm or the like).

The processing in the prior probability distribution calculation unit 40 is, for example, as follows.
1. Prior probability distribution α (l) (l = 0 to L) is initialized by α (l) ← 1 / (L + 1).
2. The update of the prior probability distribution α (l) (l = 0 to L) by the following equations (23) and (24) is alternately repeated a predetermined number of times (for example, 10 times).

  3. Prior probability distribution α (l) (l = 0 to L) is output.

  Here, the vector .about.α (the symbol “˜” in front of α indicates that the symbol “˜” is added on α.) Is a (L + 1) -dimensional vertical length composed of α (l) (l = 0 to L). The vector ~ w (t, f) is W (z (t, f); a (l, f), κ (l, f)) (l = 0 to L) and is an (L + 1) -dimensional vertical Is a vector. In addition, since derivation | leading-out of Formula (23) and Formula (24) is the same as that of 1st Embodiment, it abbreviate | omits.

  The sound source position calculation unit 50 calculates and outputs a set G of detected sound source positions based on the prior probability distribution α (l) (l = 0 to L) received from the prior probability distribution calculation unit 40. Specifically, the process described in the first embodiment is applied to α (l) (l = 1 to L) in which the domain of l is limited to l = 1 to L corresponding to the target sound source. Thus, a set G of detected sound source positions is calculated.

[Fourth Embodiment]
Next, the configuration of the fourth embodiment will be described. The fourth embodiment is an example of estimating the sound source position based on the present invention. Based on the first embodiment, the model signal of the conditional probability distribution is assumed to propagate as a spherical wave. Instead of calculating based on this, a change is made such that actual measurement data is used as learning data and pre-learning is performed. The above assumption that the target signal propagates as a spherical wave assumes an ideal environment where there is no reflection, reverberation, diffraction, etc., as in an anechoic chamber. Therefore, in the first embodiment, in an environment where there is reflection, reverberation, diffraction, etc., there is a mismatch between the assumed environment and the environment where sound source localization is performed. is there. On the other hand, in the present embodiment, when the model parameters of the conditional probability distribution are pre-learned using measured data in an environment where sound source localization is performed, such mismatch is eliminated, and there is reflection, reverberation, diffraction, etc. However, there is an advantage that the sound source position can be accurately estimated. On the other hand, unlike the present embodiment, the first embodiment has the advantage of saving the trouble of acquiring the actual measurement data.

  An example of the configuration of the signal processing device according to the fourth embodiment is illustrated in FIG. 2, similarly to the signal processing device 1 according to the first embodiment. The signal processing apparatus 1 according to the fourth embodiment includes a time-frequency analysis unit 10, a feature vector calculation unit 20, a parameter storage unit 30, a prior probability distribution calculation unit 40, and a sound source position calculation unit 50. Since the time-frequency analysis unit 10, the feature vector calculation unit 20, the prior probability distribution calculation unit 40, and the sound source position calculation unit 50 are the same as those in the first embodiment, the parameter storage unit 30 that is different from the first embodiment will be described below. explain in detail. The main differences between the first embodiment and this embodiment are as follows. The parameter storage unit 30 in the first embodiment stores a model parameter of a conditional probability distribution calculated based on the assumption that the target signal propagates as a spherical wave. On the other hand, the parameter storage unit 30 in the present embodiment stores the model parameters of the conditional probability distribution learned using the learning data acquired under reverberation.

  The parameter storage unit 30 is a model parameter learned using learning data acquired under reverberation, and the condition representing the sound source position corresponds to each of a plurality (L) of sound source position candidates. An average direction vector a (l, f) (l = 1 to L, f = 1 to F) which is a model parameter of a complex Watson distribution which is a conditional probability distribution of the feature vector z (t, f) below. The concentration parameter κ (l, f) (l = 1 to L, f = 1 to F) is stored. As the learning data acquired under reverberation, for example, an observation signal x (l, m, τ) when sound is emitted only from each of a plurality of sound source position candidates in a situation where no background noise exists is used. Can do.

The prior learning can be performed, for example, by the following procedure.
1. Generate observation signal x (l, m, τ) when sound is emitted from only one sound source position candidate. For example, x (l, m, τ) can be generated by performing recording for each of the L sound source position candidates in a situation where sound is emitted only from the sound source position candidates. Alternatively, for each of the L sound source position candidates, an impulse response from the sound source position candidate to each microphone position is measured, and x (l, m, τ) is generated by convolving the impulse response with a target signal. it can.
2. An M-dimensional vector x (l, t, f) consisting of a time frequency transformation x (l, m, t, f) (m = 1 to M) of x (l, m, τ) is calculated.
3. The feature vector ζ (l, t, f) is calculated by the following equation (25).

  4). The feature covariance matrix R (l, f) is calculated by the following equation (26).

5). The eigenvalue decomposition of the feature covariance matrix R (l, f) is performed to obtain the maximum eigenvalue μ (l, f) and the eigenvector e (l, f) of norm 1 corresponding to the maximum eigenvalue.
6). Let the average direction vector a (l, f) be a (l, f) ← e (l, f).
7). The concentration parameter κ (l, f) is calculated by the following equation (27).

  Derivation of the above processing will be described. In the above processing, under the assumption that the feature vector ζ (l, t, f) is generated according to the equation (28), the equation (29) which is the logarithmic likelihood with respect to the equation (28) is converted into the average direction vector a (l , F) and the concentration parameter κ (l, f).

  In the equation (29), the average direction vector a (l, f) (l = 1 to L, f = 1 to F) and the concentration parameter κ (l, f) (l = 1 to L, f = 1 to F) If a constant term that does not depend on any of the above is ignored, it can be rewritten as shown in Equation (30).

  Here, the matrix R (l, f) is defined by Equation (26). The term depending on the vector a (l, f) in the equation (27) is the equation (31).

  From the Courant-Fisher theorem, the vector a (l, f) that maximizes the equation (31) under the constraint of the equation (4) is the maximum eigenvalue μ (l, f) of the feature covariance matrix R (l, f). The eigenvector e (l, f) of norm 1 corresponding to f). Further, the term depending on the concentration parameter κ (l, f) in the equation (30) is the equation (32).

  Here, when the partial differential with respect to the concentration parameter κ (l, f) is set to 0, the equation (33) is obtained.

  Reference 1 "S. Sra and D. Karp," The multivariate Watson distribution: Maximum-likelihood estimation and other aspects, "Journal of Multivariate Analysis, February 2013, vol. 114, p. 256-269. Based on equation (3.8), equation (33) is approximately solved with respect to the concentrated parameter κ (l, f) to obtain equation (27). In the present embodiment, in order to learn the concentration parameter from the learning data, as in the first embodiment, the observed signal vector y (t, f) has a smaller variance (large concentration degree) as the frequency of the observed signal becomes lower. Therefore, it is possible to appropriately consider the property of possessing, and to accurately estimate the prior probability distribution and sound source localization based thereon.

[Fifth Embodiment]
Next, the configuration of the fifth embodiment will be described. The fifth embodiment is an example of estimating a sound source position based on the present invention, and based on the third embodiment, instead of using a uniform distribution as a conditional probability distribution for background noise, actual measurement data is used. A change is made to use a conditional probability distribution pre-learned using.

  The above assumption of the uniform distribution in the third embodiment assumes an ideal environment in which noise uniformly arrives from all directions. Therefore, in the third embodiment, in an environment where the noise arrival direction is biased, there is a possibility that a mismatch exists between the assumed environment and the sound source localization environment, and the sound source localization performance may be degraded. . On the other hand, in the present embodiment, when the model parameters of the conditional probability distribution are pre-learned using measured data in an environment where sound source localization is performed, the above mismatch is eliminated, and the noise arrival direction is biased However, there is an advantage that it is possible to accurately estimate the sound source position.

  An example of the configuration of the signal processing device according to the fifth embodiment is illustrated in FIG. 2, similarly to the signal processing device 1 according to the third embodiment. The signal processing apparatus 1 according to the fifth embodiment includes a time-frequency analysis unit 10, a feature vector calculation unit 20, a parameter storage unit 30, a prior probability distribution calculation unit 40, and a sound source position calculation unit 50. Since the time-frequency analysis unit 10, the feature vector calculation unit 20, the prior probability distribution calculation unit 40, and the sound source position calculation unit 50 are the same as those in the third embodiment, the parameter storage unit 30, which is a difference, will be described below. explain in detail. The main differences between the third embodiment and this embodiment are as follows. The parameter storage unit 30 according to the third embodiment stores a model parameter corresponding to a uniform distribution as a model parameter of a conditional probability distribution under a condition where a state representing a sound source position corresponds to a background noise. . In contrast, in the parameter storage unit 30 according to the present embodiment, model parameters learned using learning data as model parameters of a conditional probability distribution under conditions where the state representing the sound source position takes a state corresponding to background noise. Remember.

  In the present embodiment, the feature vector z under the condition that the state representing the sound source position of the observation signal vector y (t, f) at each time frequency point is g (t, f) = 1 (l = 0 to L). The conditional probability distribution of (t, f) is modeled by the complex Watson distribution of Equation (3) (in addition, complex Bingham distribution, complex angular center Gaussian distribution, complex Gaussian distribution, mixed complex Watson distribution, mixed complex Bingham Distribution, mixed complex angular center Gaussian distribution, mixed complex Gaussian distribution, and other probability distributions).

  The parameter storage unit 30 includes a model parameter of a conditional probability distribution under a condition in which a state representing the sound source position corresponds to each of a plurality of sound source position candidates (states 1 to L), and a state representing the sound source position. Stores model parameters of a conditional probability distribution under conditions that take a state corresponding to background noise (state 0). The model parameter of the conditional probability distribution under the condition that the state representing the sound source position corresponds to each of the plurality of sound source position candidates can be calculated by the method described in the first or fourth embodiment, for example. it can.

On the other hand, the model parameter of the conditional probability distribution under the condition that the state representing the sound source position corresponds to the background noise is pre-learned as follows, for example.
1. An M-dimensional vertical vector x (0, t, f) composed of time-frequency conversion x (0, m, t, f) (m = 1 to M) of the measured background noise x (0, m, τ) is created. .
2. The feature vector ζ (0, t, f) is calculated by the following equation (34).

  3. The feature covariance matrix R (0, f) is calculated by the following equation (35).

4). The eigenvalue decomposition of the feature covariance matrix R (0, f) is performed to obtain the maximum eigenvalue μ (0, f) and the eigenvector e (0, f) of the norm 1 corresponding to the maximum eigenvalue.
5). Let the average direction vector a (0, f) be a (0, f) ← e (0, f).
6). The concentration parameter κ (0, f) is calculated by the following equation (36).

  Note that the derivation of the above processing is the same as in the case of the fourth embodiment, and will be omitted.

[Sixth Embodiment]
Next, the configuration of the sixth embodiment will be described. The sixth embodiment is an example of estimating a sound source position based on the present invention. Based on the fourth embodiment, the state representing the sound source position takes a state corresponding to each of a plurality of sound source position candidates. This is a modification in which a complex angular center Gaussian distribution is used instead of a complex Watson distribution as a conditional probability distribution of the feature vector z (t, f) under the conditions. In the complex Watson distribution, the conditional probability distribution of the feature vector of Equation (1), which is the direction of the observed signal vector, can be expressed only when it is rotationally symmetric, whereas in the complex angular center Gaussian distribution, this conditional probability distribution is rotated. Not only a symmetrical case but also an elliptical distribution can be expressed. Since the distribution of the feature vector of the formula (1) is not necessarily rotationally symmetric, the present embodiment can model the distribution of the feature vector of the formula (1) more accurately than the fourth embodiment. As a result, the sound source position can be estimated more accurately.

  An example of the configuration of the signal processing device according to the sixth embodiment is illustrated in FIG. 2, similarly to the signal processing device 1 according to the fourth embodiment. The signal processing apparatus 1 according to the sixth embodiment includes a time-frequency analysis unit 10, a feature vector calculation unit 20, a parameter storage unit 30, a prior probability distribution calculation unit 40, and a sound source position calculation unit 50. Since the time-frequency analysis unit 10, the feature vector calculation unit 20, and the sound source position calculation unit 50 are the same as those in the fourth embodiment, the parameter storage unit 30 and the prior probability distribution calculation unit 40, which are differences, are described below. explain in detail. The main differences between the fourth embodiment and this embodiment are as follows. In the fourth embodiment, the parameter storage unit 30 stores the model parameters of the complex Watson distribution that models the conditional probability distribution, and the prior probability distribution calculation unit 40 stores the prior parameters based on the model parameters of the complex Watson distribution. Calculate the probability distribution. On the other hand, in the present embodiment, the parameter storage unit 30 stores model parameters of a complex angular center Gaussian distribution that models a conditional probability distribution, and the prior probability distribution calculation unit 40 stores the complex angular center Gaussian distribution. Calculate prior probability distribution based on model parameters.

  In the present embodiment, L conditional probability distributions for L sound source position candidates are modeled by a complex angle center Gaussian distribution. That is, the conditional probability distribution p (z (t, f) | g (t, f) = 1) is modeled by the equation (37).

  Here, the matrix Σ (l, f) is a positive definite Hermitian matrix that is a model parameter that determines the position, spread, direction, and shape of the distribution of the feature vector z (t, f) for the l-th sound source position candidate. It is called a matrix, and A (z; Σ) is a complex angle center Gaussian distribution of a vector z whose parameter matrix is a matrix Σ and is expressed by Expression (38).

  The parameter storage unit 30 is a complex angular center Gaussian distribution that is a conditional probability distribution of the feature vector z (t, f) under the condition that the state representing the sound source position corresponds to each of a plurality of sound source position candidates. The parameter matrix Σ (l, f) (l = 1 to L, f = 1 to F), which is the model parameter, is stored. The parameter matrix Σ (l, f) is pre-trained for each of the L sound source position candidates using the observation signal x (l, m, τ) when sound is emitted from only the sound source position candidate. The In the present embodiment, the parameter matrix Σ (l, f) that determines the position, spread, direction, and shape of the conditional probability distribution of the feature quantity z (t, f) is learned from the learning data. Similarly, the above-mentioned property that the direction of the observed signal vector y (t, f) has a smaller variance (corresponding to the spread) at a lower frequency can be appropriately taken into consideration, and estimation of the prior probability distribution and Sound source localization based on this can be performed accurately.

This pre-learning can be performed, for example, by the following procedure.
1. The feature vector ζ (l, t, f) (l = 1 to L, t = 1 to T, f = 1 to F) is calculated in the same manner as in the fourth embodiment.
2. The parameter matrix Σ (l, f) (l = 1 to L, f = 1 to F) is initialized with an M × M unit matrix.
3. The update of the parameter matrix Σ (l, f) (l = 1 to L, f = 1 to F) by the following equation (39) is repeated a predetermined number of times (for example, 10 times).

  4). The parameter matrix Σ (l, f) (l = 1 to L, f = 1 to F) is stored in the parameter storage unit 30.

  Derivation of Expression (39) will be described. Equation (39) is obtained by changing Equation (40), which is a logarithmic likelihood with respect to Equation (37) under the assumption that the feature vector ζ (l, t, f) is generated according to the conditional probability distribution of Equation (37). Derived by maximizing the parameter matrix Σ (l, f).

  If a constant term that does not depend on the parameter matrix Σ (l, f) in Expression (40) is ignored, Expression (40) can be rewritten as Expression (41).

  When the partial differentiation with respect to the parameter matrix Σ (l, f) in Expression (41) is set to 0 and rearranged, Expression (39) is obtained.

  The prior probability distribution calculation unit 40 is a parameter matrix Σ (l, f) (l = 1 to L, f) that is a model parameter stored in the parameter storage unit 30 with the prior probability distribution in a state representing the sound source position as a load. = 1 to F), a mixed model that is a load sum of a complex angular center Gaussian distribution that is a conditional probability distribution of the feature vector z (t, f) under a condition in which the state representing the sound source position is known, A prior probability distribution is calculated by applying to the feature vector z (t, f) calculated by the feature vector calculator 20. In the present embodiment, a time-invariant prior probability distribution α (l) (l = 1 to L) is considered as the prior probability distribution.

  The calculation of the prior probability distribution in the prior probability distribution calculation unit 40 may be performed as follows, for example. That is, the vector w (t, f) is an L-dimensional vertical vector composed of a complex angular center Gaussian distribution A (z (t, f); Σ (l, f)) (l = 1 to L) that is a conditional probability. , The process in the prior probability distribution calculation unit 40 of the first embodiment is applied to the vector w (t, f). However, it should be noted that the definition of the vector w (t, f) is different from that in the first embodiment. Note that derivation of the above processing is the same as in the case of the first embodiment, and will be omitted.

[Seventh Embodiment]
Next, the configuration of the seventh embodiment will be described. The seventh embodiment is an example of estimating the sound source position based on the present invention, and is a vector including information on the direction of the observation signal vector y (t, f) based on the fourth embodiment. The observation signal vector y (t, f) itself is used as the feature vector z (t, f) instead of the direction vector of the expression (1), and the state representing the sound source position corresponds to each of a plurality of sound source position candidates. Is used as the conditional probability distribution of the feature vector z (t, f) under the condition of ## EQU2 ## instead of the complex Watson distribution, and the spatial covariance matrix, which is a model parameter of the complex time-varying Gaussian distribution, is used. It is a modification to make it memorize and memorize in advance.

  The complex Watson distribution can be represented only when the observed signal vector direction distribution is rotationally symmetric, whereas the complex time-varying Gaussian distribution is not only when the observed signal vector direction distribution is rotationally symmetric, but also an elliptical distribution. It can also be expressed. Since the observation signal vector direction distribution is not necessarily rotationally symmetric, this embodiment can model the observation signal vector direction distribution characterizing the sound source position more accurately than the fourth embodiment. Based on this modeling, the sound source position can be estimated more accurately.

  An example of the configuration of the signal processing device according to the seventh embodiment is illustrated in FIG. 2 as in the case of the signal processing device 1 according to the fourth embodiment. The signal processing apparatus 1 according to the seventh embodiment includes a time-frequency analysis unit 10, a feature vector calculation unit 20, a parameter storage unit 30, a prior probability distribution calculation unit 40, and a sound source position calculation unit 50. Since the time frequency analysis unit 10 and the sound source position calculation unit 50 are the same as those in the fourth embodiment, the feature vector calculation unit 20, the parameter storage unit 30, and the prior probability distribution calculation unit 40 which are different points will be described in detail below. To do. The main differences between the fourth embodiment and this embodiment are as follows. In the fourth embodiment, the feature vector calculation unit 20 calculates the feature vector of Expression (1), and the parameter storage unit 30 stores the model parameters of the complex Watson distribution that models the conditional probability distribution of the feature vector. Then, in the prior probability distribution calculation unit 40, by applying a mixed model, which is a weighted sum of complex Watson distributions that models the conditional probability distribution, with the prior probability distribution of the state representing the sound source position as a load, to the feature vector The prior probability distribution is calculated. On the other hand, in the present embodiment, the feature vector calculation unit 20 outputs the observation signal vector from the time-frequency analysis unit 10 as a feature vector, and the parameter storage unit 30 has a conditional probability of the observation signal vector that is the feature vector. A conditional probability distribution is stored in which a spatial covariance matrix, which is a model parameter of a complex time-varying Gaussian distribution that models the distribution, is stored, and the prior probability distribution calculation unit 40 uses the prior probability distribution of the state representing the sound source position as a load The prior probability distribution is calculated by applying a mixed model, which is a weighted sum of complex time-varying Gaussian distributions to be modeled, to an observed signal vector that is a feature vector.

  The feature vector calculation unit 20 receives the observation signal vector y (t, f) from the time-frequency analysis unit 10 and outputs the observation signal vector y (t, f) as the feature vector z (t, f).

  In the present embodiment, a complex time-varying Gaussian distribution is used as the L conditional probability distributions for the L sound source position candidates. That is, the conditional probability distribution p (z (t, f) | g (t, f) = 1) is modeled by the equation (42).

  Φ (l, t, f) in equation (42) is a positive parameter that controls the distribution of the “magnitude (norm)” of the feature vector z (t, f). On the other hand, the matrix B (l, f) in the equation (42) controls the distribution of the “direction” of the feature vector z (t, f) (specifically, the direction of the feature vector z (t, f)). This parameter controls the position, spread, direction, and shape of the distribution. The matrix B (l, f) is a positive definite Hermitian matrix and is called a spatial covariance matrix. N (z; 0, Φ) is a complex Gaussian distribution of a vector z whose average is the vector 0 and whose covariance matrix is the matrix Φ, and is represented by Expression (43).

  Since equation (42) has a time-varying covariance matrix φ (l, t, f) B (l, f), it is referred to herein as a complex time-varying Gaussian distribution.

  The parameter storage unit 30 stores the observation signal vector y (t, f), which is the feature vector z (t, f), under the condition that the state representing the sound source position corresponds to each of the plurality of sound source position candidates. A spatial covariance matrix B (l, f) (l = 1 to L, f = 1 to F), which is a model parameter of the conditional probability distribution, is stored. In the present embodiment, the parameter storage unit 30 includes the spatial covariance matrix B (l, f) and φ (l, t, f), which are model parameters of the conditional probability distribution, related to the sound source position. Only the variance matrix B (l, f) is stored. On the other hand, since φ (l, t, f) depends on the signal power, it is not stored in the parameter storage unit 30, and the feature vector from the feature vector calculation unit 20 in the prior probability distribution calculation unit 40 as described later. Estimate using. In the present embodiment, the parameter matrix B (l, f) that determines the position, spread, direction, and shape of the distribution in the direction of the observation signal vector y (t, f) is learned from the learning data. Similarly, the property that the direction of the observed signal vector y (t, f) has a smaller variance (corresponding to the spread) at a lower frequency can be appropriately taken into account, and the prior probability distribution is estimated and based thereon. Sound source localization can be performed accurately.

The spatial covariance matrix B (l, f) uses, for example, an observation signal x (l, m, τ) when sound is emitted from only one sound source position candidate among L sound source position candidates. Pre-learning is performed according to the following procedure.
1. M-dimensional vertical vector x (l, t, f) (l = 1 to L) consisting of time frequency transformation x (l, m, t, f) (m = 1 to M) of x (l, m, τ) t = 1 to T, f = 1 to F). The feature vector ζ (l, t, f) is set to ζ (l, t, f) ← x (l, t, f). Here, it should be noted that the calculation method of the feature vector ζ (l, t, f) is different from that in the fourth embodiment.
2. The spatial covariance matrix B (l, f) (l = 1 to L, f = 1 to F) is initialized with an M × M unit matrix.
3. The update of the spatial covariance matrix B (l, f) (l = 1 to L, f = 1 to F) by the following equation (44) is repeated a predetermined number of times (for example, 10 times).

  4). The spatial covariance matrix B (l, f) (l = 1 to L, f = 1 to F) is stored in the parameter storage unit 30.

  Derivation of Expression (44) will be described. Equation (44) is obtained by substituting Equation (45), which is a logarithmic likelihood with respect to Equation (42), under the assumption that the vector ζ (l, t, f) is generated according to the conditional probability distribution of Equation (42). Derived by maximizing the correlation matrices B (l, f) and φ (l, t, f).

  When the constant term not depending on the spatial correlation matrix B (l, f) and φ (l, t, f) in the equation (45) is ignored, the equation (45) is rewritten into the equation (46).

  When the partial differentiation with respect to φ (l, t, f) in the equation (46) is set to 0, the equation (47) is obtained.

  Further, when the partial differentiation with respect to B (l, f) in equation (46) is set to 0, equation (48) is obtained, and equation (44) is obtained by substituting equation (47) into equation (48).

  Next, the modeling of the peripheral probability distribution of the feature vector z (t, f) in the present embodiment will be described. In the present embodiment, the surrounding probability distribution of the feature vector z (t, f) is set to the prior probability distribution P (g (t, f) = l) of the state g (t, f) representing the sound source position as a load. Modeling is performed by a mixed model of Expression (49), which is a load sum of the conditional probability distribution p (z (t, f) | g (t, f) = l).

  The prior probability distribution calculation unit 40 uses the spatial probability matrix B (model parameter stored in the parameter storage unit 30 with the prior probability distribution α (l) (l = 1 to L) in a state representing the sound source position as a load. l, f) (1 = 1 to L, f = 1 to F), based on a condition that represents a sound source position under a known condition, a weighted sum of conditional probability distributions of feature vectors z (t, f) A mixture model of a certain formula (49) is applied to the feature vector z (t, f) calculated by the feature vector calculation unit 20 to calculate the prior probability distribution α (l) (l = 1 to L).

  There are various methods for applying the mixed model of Expression (49) to the feature vector z (t, f). For example, the likelihood related to Expression (49) is used as an objective function (the posterior probability is also used as the objective function). This is maximized based on the gradient method (or can be maximized based on the EM algorithm or the like).

  The estimation of the prior probability distribution α (l) (l = 1 to L) in the prior probability distribution calculation unit 40 can be performed in the same manner as in the first embodiment. However, unlike the first embodiment, the vector w (t, f) is changed to N (z (t, f), 0, φ (l, t, f) B (l, f)) (l = 1). To L). Here, φ (l, t, f) can be calculated by the following equation.

  Derivation of the above processing will be described. The likelihood, which is an objective function, is the probability that a feature vector z (t, f) (t = 1 to T, f = 1 to F) will be observed, and is represented by equation (51).

  Equation (50) is derived by maximization of Equation (51) with respect to φ (l, t, f). The maximization with respect to φ (l, t, f) in the equation (51) is expressed as φ (n [N (z (t, f), 0, φ (l, t, f) B (l, f))]] It is equivalent to maximization with respect to l, t, f). Therefore, when the partial differential with respect to φ (l, t, f) of ln [N (z (t, f), 0, φ (l, t, f) B (l, f))] is set to 0, (50) is obtained. After that, similarly to the first embodiment, Expressions (10) and (11), which are update expressions of the prior probability distribution α (l) (l = 1 to L), can be derived.

[Eighth Embodiment]
Next, the configuration of the eighth embodiment will be described. In the eighth embodiment, sound source position tracking is performed using a set G (t) of sound source positions detected by the signal processing apparatus 1 according to the second embodiment, and the sound source position ρ ( n, t) (n = 1 to N, t = 1 to T, N is the number of sound sources). In this embodiment, it is assumed that the sound source position is designated only by the azimuth, G (t) is a set of azimuths, and ρ (n, t) is the azimuth. As such a situation, for example, there is a situation where several people are having a conversation around a table on which a microphone is placed.

  The configuration of the signal processing device according to the eighth embodiment will be described with reference to FIG. FIG. 4 is a diagram illustrating an example of a configuration of a signal processing device according to the eighth embodiment. As illustrated in FIG. 4, the signal processing device 2 includes a time-frequency analysis unit 10, a feature vector calculation unit 20, a parameter storage unit 30, a prior probability distribution calculation unit 40, a sound source position calculation unit 50, and a tracking unit 51. Since the time-frequency analysis unit 10, the feature vector calculation unit 20, the parameter storage unit 30, the prior probability distribution calculation unit 40, and the sound source position calculation unit 50 are the same as those in the second embodiment, tracking is the difference in the following. The unit 51 will be described in detail.

  The tracking unit 51 receives the set G (t) (t = 1 to T) of the detected sound source positions (azimuth angles) from the sound source position calculation unit 50, performs tracking of the sound source position, and performs each sound source for each frame. The sound source position (azimuth angle) ρ (n, t) (n = 1 to N, t = 1 to T) is calculated and output. This tracking can be performed by various methods. In the following, as an example, it is assumed that the rough sound source position (azimuth angle) of each sound source is known, and tracking is performed using this. As an example of the situation where the rough sound source position (azimuth angle) of each sound source is known, there is a situation in which a desk is placed around a microphone and a plurality of people are sitting on a chair and having a meeting. In this case, if the chair is almost fixed at a known position and there is no seat movement of the speaker during the conversation, the chair position (known) should be used as a rough sound source position of each sound source (speaker). Can do.

  First, the rough sound source position of each sound source is set as the initial value ρ (n, 0) of the sound source position (azimuth angle) ρ (n, t).

Assuming that the sound source position (azimuth angle) ρ (n, t−1) at frame t−1 is obtained, the sound source position (azimuth angle) ρ (n, t) at frame t is It can ask for.
1. ρ (n, t) is initialized by ρ (n, t) ← ρ (n, t−1).
2. The following processes 2-1 and 2-2 are performed on each detected sound source position (azimuth angle) rεG (t) (0 ≦ r <2π).
2-1. The sound source number ν closest to the detected sound source position (azimuth angle) r is calculated by the following equation (52).

    2-2. The sound source position (azimuth angle) ρ (ν, t) of the νth sound source is updated by the equation (53).

  In the equation (53), d (ξ, η) is a distance on the circumference defined by the equation (54).

  In addition, in Equation (53), the symbol with [0, 2π) subscripted to ∠ is an operator that calculates the declination in the range [0, 2π) for a non-zero complex number. The subscript symbol [-π, π) is an operator for calculating the declination in the range [−π, π) for a non-zero complex number, and δ is a constant satisfying 0 <δ <1. (For example, δ = 0.005).

[Ninth Embodiment]
Next, the configuration of the ninth embodiment will be described. The ninth embodiment is an example in which dialization is performed based on a processing result by the signal processing device 2 according to the eighth embodiment. This dialization may be performed by determining whether each sound source exists or does not exist for each frame (hard decision), or by calculating the existence probability of each sound source for each frame (soft decision). May be. Here, an example of the former case is shown.

  The configuration of the signal processing apparatus according to the ninth embodiment will be described with reference to FIG. FIG. 5 is a diagram illustrating an example of a configuration of a signal processing device according to the ninth embodiment. As shown in FIG. 5, the signal processing device 3 includes a time frequency analysis unit 10, a feature vector calculation unit 20, a parameter storage unit 30, a prior probability distribution calculation unit 40, a sound source position calculation unit 50, a tracking unit 51, and a dialization unit. 60. Since the time-frequency analysis unit 10, the feature vector calculation unit 20, the parameter storage unit 30, the prior probability distribution calculation unit 40, the sound source position calculation unit 50, and the tracking unit 51 are the same as those of the signal processing device 2, the differences are as follows. The dialization unit 60 is described in detail.

  The dialization unit 60 includes a set G (t) (t = 1 to T) of the detected sound source positions from the sound source position calculation unit 50, and a sound source position (azimuth angle) ρ for each sound source from the tracking unit 51. (N, t) (n = 1 to N, t = 1 to T) are received, and the dialization result d (n, t) for each frame for each sound source is calculated and output. However, it is determined that d (n, t) = 1 when the sound source n exists in the frame t and d (n, t) = 0 when the sound source n does not exist in the frame t.

Various methods are conceivable as a method of calculating the dialization result d (n, t). For example, the calculation may be performed as follows.
1. d (n, t) (n = 1 to N, t = 1 to T) is initialized by d (n, t) ← 0.
2. The following processing is performed for t = 1 to T: For each detected sound source position (azimuth angle) rεG (t), the distance d (r, ρ (n, t)) is minimized. The sound source number n is obtained, and d (ν, t) ← 1 is set.
3. Let d (n, t) (n = 1 to N, t = 1 to T) be a dialization result.

  In the ninth embodiment, when the accurate sound source position (azimuth angle) of each sound source is known, instead of using the sound source position (azimuth angle) calculated by the tracking unit 51, the known sound source position (azimuth angle) is used. (Angle) may be used as the sound source position (azimuth angle) ρ (n, t) for each sound source and each frame. Examples of such a situation include a situation where a speaker is sitting on a fixed chair and talking, a situation where a sound source position (azimuth angle) is known from a video camera image, and the like.

[Tenth embodiment]
Next, the configuration of the tenth embodiment will be described. The tenth embodiment is an example in which the waveform of each target signal is estimated based on the sound source position estimated by the present invention in a situation where N (N> 0) target signals are mixed under background noise. According to the present embodiment, the mixed target signal can be separated into individual target signals and background noise can be removed.

  The configuration of the signal processing apparatus according to the tenth embodiment will be described with reference to FIG. FIG. 6 is a diagram illustrating an example of a configuration of a signal processing device according to the tenth embodiment. As shown in FIG. 6, the signal processing device 4 includes a time frequency analysis unit 10, a feature vector calculation unit 20, a parameter storage unit 30, a prior probability distribution calculation unit 40, a sound source position calculation unit 50, a tracking unit 51, and a dialization unit. 60, a mask estimation unit 70, and a signal enhancement unit 80. Since the time-frequency analysis unit 10, the feature vector calculation unit 20, the tracking unit 51, and the dialization unit 60 are the same as those of the signal processing device 3, the parameter storage unit 30 and the prior probability distribution calculation unit 40, which are the differences, are described below. The sound source position calculation unit 50, the mask estimation unit 70, and the signal enhancement unit 80 will be described in detail. The main differences between the signal processing device 3 and the signal processing device 4 are as follows. In the signal processing device 3, the parameter storage unit 30 stores the model parameters of the conditional probability distribution under the condition that the state representing the sound source position corresponds to each of the plurality of sound source position candidates, and calculates the prior probability distribution. The unit 40 calculates a prior probability distribution in a state corresponding to a plurality of sound source position candidates based on the model parameter, and the sound source position calculating unit 50 calculates a sound source position based on the prior probability distribution. On the other hand, in the signal processing device 4, the parameter storage unit 30 further stores a model parameter of a conditional probability distribution under the condition that the state representing the sound source position corresponds to the background noise, and calculates the prior probability distribution calculation. The unit 40 calculates a prior probability distribution in a state corresponding to a plurality of sound source position candidates and background noise based on the model parameter, and the sound source position calculation unit 50 calculates a sound source position based on the prior probability distribution. . In the signal processing device 4, the mask estimation unit 70 further estimates a mask that is a contribution (posterior probability) for each time frequency point of each target signal and background noise, and the signal enhancement unit 80 determines each mask based on the mask. Calculate the waveform of the target signal.

  The parameter storage unit 30 is a model of a complex Watson distribution that is a conditional probability distribution of the feature vector z (t, f) under the condition that the state representing the sound source position takes a state corresponding to each of a plurality of sound source position candidates. Parameters, average direction vector a (l, f) (l = 1 to L, f = 1 to F), concentrated parameter κ (l, f) (l = 1 to L, f = 1 to F), and sound source An average direction vector a (0, f), which is a model parameter of a complex Watson distribution, which is a conditional probability distribution of the feature vector z (t, f) under the condition that the state representing the position corresponds to the background noise. (F = 1 to F) and the concentration parameter κ (0, f) (f = 1 to F) are stored. These model parameters can be calculated by the method described in the fourth embodiment for the state corresponding to each of the sound source position candidates, and for the state corresponding to the background noise, for example, in the third embodiment. It can be calculated by the method described.

  The prior probability distribution calculation unit 40 uses the average probability vector a (l, f) (l = 0 to L, which is a model parameter stored in the parameter storage unit 30 with the prior probability distribution in a state representing the sound source position as a load. feature vector z (t, t) under a condition in which the sound source position is known based on f = 1 to F) and the concentration parameter κ (l, f) (l = 0 to L, f = 1 to F). The mixed model of Formula (21), which is the load sum of the conditional probability distribution of f), is applied to the feature vector z (t, f) calculated by the feature vector calculation unit 20 to calculate the prior probability distribution. In the present embodiment, it is assumed that the prior probability distribution P (g (t, f) = 1) depends on the frame, and is represented by α (l, t) (1 = 0 to L, t = 1 to T). α (l, t) satisfies the constraint condition α (0, t) +... + α (L, t) = 1. There are various methods for applying the mixed model of Expression (21) to the feature vector z (t, f). In this embodiment, the likelihood related to Expression (21) is calculated by the gradient method using the prior probability distribution α (l, t) by maximizing with respect to (l = 0 to L, t = 1 to T).

The processing in the prior probability distribution calculation unit 40 is, for example, as follows.
1. Prior probability distribution α (l, t) (l = 0 to L, t = 1 to T) is initialized by α (l, t) ← 1 / (L + 1).
2. The update of the prior probability distribution α (l, t) (l = 0 to L, t = 1 to T) according to the following equations (55) and (56) is alternately repeated a predetermined number of times (for example, 10 times).

  3. Prior probability distribution α (l, t) (l = 0 to L, t = 1 to T) is output.

  Here, the vector .about..alpha. (T) (the symbol ".about." In front of .alpha. Indicates that the symbol ".about." Is added above .alpha.) Consists of .alpha. (L, t) (l = 0 to L). (L + 1) dimensional vertical vector, vector ~ w (t, f) consists of W (z (t, f); a (l, f), κ (l, f)) (l = 0 ~ L) (L + 1) dimensional vertical vector. In addition, since derivation | leading-out of Formula (55) and Formula (56) is the same as that of the case of 1st Embodiment, it abbreviate | omits.

  The sound source position calculation unit 50 detects a set G of sound source positions detected based on the prior probability distribution α (l, t) (l = 0 to L, t = 1 to T) received from the prior probability distribution calculation unit 40. (T) Calculate (t = 1 to T) and output. Specifically, α (l, t) in which the domain of the prior probability distribution α (l, t) (l = 0 to L, t = 1 to T) is limited to l = 1 to L corresponding to the target sound source. By applying the processing in the sound source position calculation unit 50 of the second embodiment to (l = 1 to L, t = 1 to T), a set G (t) of detected sound source positions (t = 1-T).

  The mask estimation unit 70 includes an average direction vector a (l, f) (l = 0 to L, f = 1 to F) from the parameter storage unit 30 and a concentrated parameter κ (l, f) (l = 0 to L, f = 1 to F), prior probability distribution α (l, t) (1 = 0 to L, t = 1 to T) from the prior probability distribution calculation unit 40, and sound source for each sound source from the tracking unit 51 for each frame. The position (azimuth) ρ (n, t) (n = 1 to N, t = 1 to T) is received, and the background noise to the feature vector z (t, f) and the contribution of each target signal for each time frequency point A mask γ (n, t, f) (n = 0 to N, t = 1 to T, f = 1 to F) which is a degree (a posteriori probability) is calculated and output. Here, γ (0, t, f) is a mask corresponding to the background noise, and γ (n, t, f) (n = 1 to N) is a mask corresponding to the target signal n.

The mask γ (n, t, f) can be calculated by various methods. For example, it is calculated as follows.
1. A posteriori probability P (g (t, f) = l | z (t, f)) (1 = 0 to g (t, f) = 1 under the given condition of the feature vector z (t, f) L, t = 1 to T, f = 1 to F) are calculated by the following equations (57) and (58).

  2. A mask γ (0, t, f) (t = 1 to T, f = 1 to F) corresponding to the background noise is calculated by the following equation (59).

  3. A set J (n, t) (n = 1 to N, t = 1 to T) of sound source position candidates corresponding to each target signal n in the frame t is calculated by the following equation (60).

  4). A mask γ (n, t, f) (n = 1 to N, t = 1 to T, f = 1 to F) corresponding to the target signal is calculated by the following equation (61).

  5. A mask γ (n, t, f) (n = 0 to N, t = 1 to T, f = 1 to F) is output.

  The signal emphasizing unit 80 is an observation signal vector y (t, f) from the time-frequency analysis unit 10 and a dialization result d (n, t) (n) that takes either 0 or 1 from the dialization unit 60. = 1 to N, t = 1 to T), and background noise from the mask estimation unit 70 and mask γ (n, t, f) of each target signal (n = 0 to N, t = 1 to T, f = 1-F) is received and each target signal s (n, τ) is estimated.

A specific example of processing in the signal enhancement unit 80 is as follows.
1. The covariance matrix Φ (f) of the observation signal is calculated by the following equation (62).

  2. Masks corrected using the dialyzation results d (n, t) (n = 1 to N, t = 1 to T) to γ (n, t, f) (n = 0 to N, t = 1 to T, f = 1 to F) is calculated by the following equations (63) and (64). Equation (63) means that the mask of the sound source n in the frame t is replaced with 0 when d (n, t) = 0. Expression (64) is a process for making the sum of masks to γ (n, t, f) related to n equal to 1.

  3. The covariance matrix Ψ (n, f) (n = 0 to N, f = 1 to F) is calculated by the following equation (65). Here, the matrix Ψ (0, f) is a covariance matrix corresponding to the background noise, and the matrix Ψ (n, f) (n = 1 to N) corresponds to the sum of the nth target signal and the background noise. It is a covariance matrix.

  4). Corresponds to the nth target signal by subtracting the covariance matrix Ψ (0, f) corresponding to the background noise from the covariance matrix Ψ (n, f) corresponding to the sum of the nth target signal and the background noise. The covariance matrix to Ψ (n, f) (n = 1 to N, f = 1 to F) is obtained. Next, the steering vector h (n, f) (n = 1 to N, f = 1 to F) of each target signal is obtained as an eigenvector corresponding to the maximum eigenvalue of the matrix to Ψ (n, f). Then, the vector h (n, f) is expressed by h (n, f) ← h (n, f) / h (1, n, f) so that the first element of the vector h (n, f) is equal to 1. ) Is normalized. Here, h (1, n, f) represents the first element of the vector h (n, f).

  5. Based on the minimum dispersion beamformer, the time-frequency conversion s (n, t, f) (n = 1 to N, t = 1 to T, f = 1 to F) of each target signal is calculated by the following equation (67). To do.

  6). By applying the inverse transformation of the time frequency conversion to the time frequency conversion s (n, t, f) (n = 1 to N, t = 1 to T, f = 1 to F) of each target signal, s (n, τ) is calculated.

[Eleventh embodiment]
Next, the configuration of the eleventh embodiment will be described. In the eleventh embodiment, in a situation where N (N> 0) target sounds exist under background noise, the waveform of each target sound is estimated based on the sound source position estimated by the present invention. This is an example in which each target speech is recognized by applying an existing speech recognition technology. According to the present invention, even in a situation where background noise and voices from a plurality of speakers are mixed, it is possible to separate mixed target signals into individual target signals and remove background noise to realize highly accurate speech recognition. As an application example, for example, there is an automatic transcription of a conference held in one corner of an office where various sounds are being made.

  The configuration of the signal processing apparatus according to the eleventh embodiment will be described with reference to FIG. FIG. 7 is a diagram illustrating an example of the configuration of the signal processing device according to the eleventh embodiment. As shown in FIG. 7, the signal processing device 5 includes a time frequency analysis unit 10, a feature vector calculation unit 20, a parameter storage unit 30, a prior probability distribution calculation unit 40, a sound source position calculation unit 50, a tracking unit 51, and a dialization unit. 60, a mask estimation unit 70, a signal enhancement unit 80, and a speech recognition unit 90. About the time-frequency analysis unit 10, the feature vector calculation unit 20, the parameter storage unit 30, the prior probability distribution calculation unit 40, the sound source position calculation unit 50, the tracking unit 51, the dialization unit 60, the mask estimation unit 70, and the signal enhancement unit 80 This is the same as in the tenth embodiment. The speech recognition unit 90 receives the waveform of each target signal from the signal enhancement unit 80, and outputs a recognition result for each target signal by applying an existing speech recognition technology to the waveform.

[System configuration, etc.]
Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. Furthermore, all or a part of each processing function performed in each device may be realized by a CPU and a program that is analyzed and executed by the CPU, or may be realized as hardware by wired logic.

  Also, among the processes described in this embodiment, all or part of the processes described as being performed automatically can be performed manually, or the processes described as being performed manually can be performed. All or a part can be automatically performed by a known method. In addition, the processing procedure, control procedure, specific name, and information including various data and parameters shown in the above-described document and drawings can be arbitrarily changed unless otherwise specified.

[program]
The signal processing apparatuses 1 to 5 according to the embodiment can be implemented by installing a signal processing program for executing the sound source localization, tracking, dialization, speech enhancement, and speech recognition as package software or online software in a desired computer. For example, by causing the information processing apparatus to execute the signal processing program, the information processing apparatus can function as the signal processing apparatuses 1 to 5. The information processing apparatus referred to here includes a desktop or notebook personal computer. In addition, the information processing apparatus includes mobile communication terminals such as smartphones, mobile phones and PHS (Personal Handyphone System), and slate terminals such as PDA (Personal Digital Assistant).

  Further, the signal processing devices 1 to 5 can be implemented as a signal processing server device that uses a terminal device used by a user as a client and provides the client with services related to the above signal processing. For example, the signal processing server apparatus is implemented as a server apparatus that provides a sound source localization service that receives an observation signal and outputs a sound source position. In this case, the signal processing server device may be implemented as a Web server, or may be implemented as a cloud that provides the above-described signal processing services by outsourcing.

  FIG. 8 is a diagram illustrating an example of a computer in which a signal processing apparatus is realized by executing a program. The computer 1000 includes a memory 1010 and a CPU 1020, for example. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These units are connected by a bus 1080.

  The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012. The ROM 1011 stores a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1100. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to the display 1130, for example.

  The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, a program that defines each process of the signal processing devices 1 to 5 is implemented as a program module 1093 in which a code executable by a computer is described. The program module 1093 is stored in the hard disk drive 1090, for example. For example, a program module 1093 for executing processing similar to the functional configuration in the signal processing devices 1 to 5 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced by an SSD.

  The setting data used in the processing of the above-described embodiment is stored as program data 1094 in, for example, the memory 1010 or the hard disk drive 1090. Then, the CPU 1020 reads the program module 1093 and the program data 1094 stored in the memory 1010 and the hard disk drive 1090 to the RAM 1012 and executes them as necessary.

  The program module 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1090, but may be stored in, for example, a removable storage medium and read out by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and the program data 1094 may be stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.). The program module 1093 and the program data 1094 may be read by the CPU 1020 from another computer via the network interface 1070.

1, 2, 3, 4, 5 Signal processing apparatus 10 Time frequency analysis unit 20 Feature vector calculation unit 30 Parameter storage unit 40 Prior probability distribution calculation unit 50 Sound source position calculation unit 51 Tracking unit 60 Dialization unit 70 Mask estimation unit 80 Signal Emphasis unit 90 Speech recognition unit

Claims (7)

A time-frequency analysis unit that applies time-frequency analysis to recorded sound obtained at a plurality of different positions and calculates an observation signal vector that is an M-dimensional vector;
A feature vector calculation unit that calculates a feature vector, which is a vector including information on the direction of the observed signal vector calculated by the time frequency analysis unit, for each time frequency point;
A parameter storage unit for storing a model parameter of the conditional probability distribution of the feature vector under a condition in which a state representing a sound source position corresponds to each of a plurality of sound source position candidates;
Conditional probability distribution of the feature vector under a condition where the state representing the sound source position is known, based on the model parameter stored in the parameter storage unit, with the prior probability distribution of the state representing the sound source position as a load A prior model that calculates the prior probability distribution by applying a mixed model that is a load sum of the above to the feature vector calculated by the feature vector calculator;
A sound source position calculation unit that calculates a sound source position corresponding to the feature vector based on the prior probability distribution calculated by the prior probability distribution calculation unit;
A signal processing apparatus comprising: The prior probability distribution calculation unit uses a prior probability distribution for each time interval of the state representing the sound source position as a load, and the state representing the sound source position based on the model parameter stored in the parameter storage unit is known. Applying a mixed model, which is a weighted sum of conditional probability distributions of the feature vectors under the conditions, to the feature vectors calculated by the feature vector calculation unit, and calculating a prior probability distribution for each time interval;
The sound source position calculation unit calculates a sound source position for each time interval corresponding to the feature vector based on the prior probability distribution for each time interval calculated by the prior probability distribution calculation unit. The signal processing apparatus according to claim 1.   The parameter storage unit is a model parameter learned using learning data acquired under reverberation, and under a condition that a state representing a sound source position takes a state corresponding to each of a plurality of sound source position candidates. The signal processing apparatus according to claim 1, wherein a model parameter of a conditional probability distribution of the feature vector is stored.   The parameter storage unit further stores a model parameter of a conditional probability distribution under a condition in which a state representing the sound source position takes a state corresponding to background noise. 2. The signal processing device according to item 1.   5. The signal processing apparatus according to claim 1, wherein the prior probability distribution calculating unit calculates the prior probability distribution based on a gradient method. 6. A signal processing method executed by a signal processing device,
Applying a time-frequency analysis to recorded sound obtained at a plurality of different positions and calculating an observation signal vector which is an M-dimensional vector;
A feature vector calculation step of calculating, for each time frequency point, a feature vector that is a vector including information on the direction of the observed signal vector calculated by the time frequency analysis step;
A model parameter stored in a parameter storage unit that stores a model parameter of a conditional probability distribution of the feature vector under a condition that a state representing a sound source position corresponds to each of a plurality of sound source position candidates is acquired. , A load that is a weighted prior probability distribution of the state representing the sound source position, and a load sum of the conditional probability distribution of the feature vector based on the model parameter under a condition where the state representing the sound source position is known Applying a model to the feature vector calculated by the feature vector calculating step to calculate the prior probability distribution; and
A sound source position calculating step for calculating a sound source position corresponding to the feature vector based on the prior probability distribution calculated by the prior probability distribution calculating step;
A signal processing method comprising:   A signal processing program for causing a computer to function as the signal processing device according to any one of claims 1 to 5.

Images