EP1605440B1 - Method for signal source separation from a mixture signal - Google Patents

Description

The present invention relates to a method for determining the separation signals respectively relating to sound sources from a signal from the mixture of these signals.

The field of the present invention is that of the digital processing of signals relating to sound sources, also called simply sound, audio or audio signals. In this particular field, the processing performed on the sound signals is not in the time domain but in the frequency domain. Also, we use frequently before any processing a short-term Fourier transform which is a linear transform associating with a signal in the sampled time domain {x (t ₁ ), ..., x (t _N ) } a two-dimensional time-frequency signal noted here x (t _k , f), where t _k is a frame index of the sampled digital signal and f is a generally discrete frequency index. The signal x (t _k , f) is therefore a signal of the frequency domain and is in the form of frames indexed at t _k .

où b(t) est un bruit blanc gaussien de variance σ² _b et S(t,f) est le vecteur dont chaque composante est associée à une source : $$S\left(t,f\right)=\left(\begin{array}{c}{s}_1\left(t,f\right)\\ ⋮\\ s_N\left(t,f\right)\end{array}\right)$$

In the present description, all the quantities in question are described by means of multidimensional Gaussian random variables. The mixture observed at time t is expressed in the form: $${\mathrm{S}}_{\mathrm{Obs}}\left(\mathrm{t},\mathrm{f}\right)\mathrm{=}\mathrm{S}\left(\mathrm{t},\mathrm{f}\right)\mathrm{+}\mathrm{b}\left(\mathrm{t},\mathrm{f}\right)$$

where b (t) is a gaussian white noise of variance σ ² _b and S (t, f) is the vector of which each component is associated with a source: $$S\left(t,f\right)=\left(\begin{array}{c}{s}_1\left(t,f\right)\\ ⋮\\ s_{\mathrm{NOT}}\left(t,f\right)\end{array}\right)$$

(f)For each frequency f and for each source i, s ₁ (t, f) follows a Gaussian law centered and of variance ${\sigma }_i^2\left(f\right)$

( f )

To designate variables as a vector or matrix, uppercase letters are used.

Moreover, in the present application, the notion of signal is often confused with that of the random variable which represents it.

With regard to the separation of audio signals, a method is known which is based on a filter, called a Wiener filter, which ultimately performs a estimation of the separation signal Ŝ _W ( t, f ) under the assumption of the overall stationarity of the mixing signals. If we call x (t _k , f) the random variable which describes the signal of the frequency domain resulting from the mixing of the signals of the sources, and which is applied to the input of the filter, the expectation of the random variable describing the output signal of the filter is conditioned to the signals x (t _k , f). We can write: $${\hat{S}}_W\left(t_k,f\right)=E\left[S\left(t_k,f\right)|x\left(t_k,f\right)\right]$$

où e_i(f) est la fraction d'énergie de la source i contenue a priori dans le signal de mélange, à la fréquence d'indice f, N étant le nombre totale de sources et x(t_k ,f) étant le signal de mélange.In the case of the Wiener filter, each component of the vector Ŝ _W ( t _k , f ) can be obtained by the following relation: $${\hat{S}}_W\left(t,f\right)=\left(\begin{array}{c}{\hat{S}}_{W,1}\left(t,f\right)\\ ⋮\\ {\hat{S}}_{W,\mathrm{NOT}}\left(t,f\right)\end{array}\right)\mathrm{with}{\hat{s}}_{W,i}\left(t,f\right)=\frac{e_i\left(f\right)}{{\displaystyle \underset{j}{\Sigma }}{e}_j\left(f\right)+{\sigma }_b^2}x\left(t,f\right)$$

where e _i (f) is the energy fraction of the source i contained a priori in the mixing signal, at the index frequency f, where N is the total number of sources and x ( t _k , f ) is the mixing signal.

It is considered, by way of illustration only, the particular case of two sources delivering signals respectively noted in the time domain, s ₁ (t) and s ₂ (t). Initially, we have a sound signal, noted in the time domain x (t) representative of the mixture of these sound signals: $$\mathrm{x}\left(\mathrm{t}\right)\mathrm{=}{\mathrm{s}}_{\mathrm{1}}\left(\mathrm{t}\right)\mathrm{+}{\mathrm{s}}_{\mathrm{2}}\left(\mathrm{t}\right)\mathrm{.}$$

(f) et ${\mathrm{\sigma }}_2^{\mathrm{2}}$

(f), qui représentent, en définitive comme il est connu, leurs répartitions énergétiques en fonction de la fréquence. Si l'on considère que les signaux dans le domaine fréquentiel relatifs à ces deux sources s ₁(t,f) et s ₂(t,f) sont des variables aléatoires gaussiennes, non stationnaires, ${\mathrm{\sigma }}_1^{\mathrm{2}}$

(f) et ${\mathrm{\sigma }}_2^{\mathrm{2}}$

(f) représentent respectivement leur variance. Le filtre de Wiener délivre une estimation du signal de son de chaque source et, ce dans le domaine fréquentiel, en accord avec les relations suivantes : $${\hat{s}}_{W,1}\left(t,f\right)=\frac{{\mathrm{\sigma }}_1^{\mathrm{2}}\left(f\right)}{{\mathrm{\sigma }}_1^{\mathrm{2}}\left(f\right)+{\mathrm{\sigma }}_2^{\mathrm{2}}}x\left(t,f\right)$$

$${\hat{s}}_{W,2}\left(t,f\right)=\frac{{\mathrm{\sigma }}_2^{\mathrm{2}}\left(f\right)}{{\mathrm{\sigma }}_1^{\mathrm{2}}\left(f\right)+{\mathrm{\sigma }}_2^{\mathrm{2}}}x\left(t,f\right)$$

qui peuvent s'écrire sous forme matricielle de la manière suivante : $$\mathrm{S}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{=}\mathrm{P}\mathrm{.}\mathrm{x}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)$$

In a preliminary learning phase, the two sound sources were evaluated and their respective characteristic spectral shapes were more accurately estimated. ${\mathrm{\sigma }}_{\mathrm{1}}^{\mathrm{2}}$

(f) and ${\mathrm{\sigma }}_2^{\mathrm{2}}$

(f), which ultimately represent, as is known, their energy distributions as a function of frequency. If we consider that the signals in the frequency domain relating to these two sources s ₁ ( t, f ) and s ₂ ( t , f ) are gaussian random variables, non-stationary, ${\mathrm{\sigma }}_1^{\mathrm{2}}$

(f) and ${\mathrm{\sigma }}_2^{\mathrm{2}}$

(f) represent their variance, respectively. The Wiener filter delivers an estimate of the sound signal from each source and, in the frequency domain, according to the following relationships: $${\hat{s}}_{W,1}\left(t,f\right)=\frac{{\mathrm{\sigma }}_1^{\mathrm{2}}\left(f\right)}{{\mathrm{\sigma }}_1^{\mathrm{2}}\left(f\right)+{\mathrm{\sigma }}_2^{\mathrm{2}}}x\left(t,f\right)$$

$${\hat{s}}_{W,2}\left(t,f\right)=\frac{{\mathrm{\sigma }}_2^{\mathrm{2}}\left(f\right)}{{\mathrm{\sigma }}_1^{\mathrm{2}}\left(f\right)+{\mathrm{\sigma }}_2^{\mathrm{2}}}x\left(t,f\right)$$

which can be written in matrix form as follows: $$\mathrm{S}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{=}\mathrm{P}\mathrm{.}\mathrm{x}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)$$

Where P is a matrix that describes the weights and is given below for N sources: $$\mathrm{P}\mathrm{=}\left[\frac{{\mathrm{\sigma }}_{\mathrm{1}}^{\mathrm{2}}\left(\mathrm{f}\right)}{{\displaystyle \underset{\mathrm{i}\mathrm{=}\mathrm{1}}{\overset{\mathrm{NOT}}{\mathrm{\Sigma }}}}{\mathrm{\sigma }}_{\mathrm{i}}^{\mathrm{2}}\left(\mathrm{f}\right)},\mathrm{...},\frac{{\mathrm{\sigma }}_{\mathrm{NOT}}^{\mathrm{2}}\left(\mathrm{f}\right)}{{\displaystyle \underset{\mathrm{i}\mathrm{=}\mathrm{1}}{\overset{\mathrm{NOT}}{\mathrm{\Sigma }}}}{\mathrm{\sigma }}_{\mathrm{i}}^{\mathrm{2}}\left(\mathrm{f}\right)}\right]$$

In the context of the separation of sound signals, the Wiener filter has the following main disadvantages. It operates identically on all the frames of the mixing sound signal and therefore does not take changes in the sound energy content from one frame to another. In the end, it is not an adaptive filter. Another disadvantage lies in the fact that it only takes into account a characteristic spectral form by sound source even though the sound sources have a great spectral variety in terms of timbre, pitch, intensity, etc.

Improvements to the Wiener filter have been proposed to take these disadvantages into account and have resulted in two methods which are essentially based on the use of multiple spectral forms to describe each of the sources involved.

(f), k_i ∈ [1,...,K_i]. Si l'on considère N sources, leur mélange est caractérisé par un ensemble de K₁ x K₂ x ... x K_N N-uplets de formes spectrales caractéristiques ( ${\mathrm{\sigma }}_{{\mathrm{k}}_1}^{\mathrm{2}}$

(f) ,..., ${\mathrm{\sigma }}_{{\mathrm{k}}_{\mathrm{N}}}^{\mathrm{2}}$

(f)). Pour chaque trame d'indice t_k, la méthode consiste à d'abord choisir le N-uplet de formes spectrales qui correspond le mieux au signal de son du mélange. Par exemple, elle peut consister à maximaliser la probabilité de correspondance entre le spectrogramme du mélange |x(t_k ,f)|² et la variance résultant du couple de formes spectrales. Ensuite, elle consiste à filtrer par un filtrage de Wiener classique le mélange en utilisant le N-uplet de formes spectrales ainsi sélectionné. On peut constater que cette méthode est adaptative puisque le choix des paramètres du filtre dépend de l'indice de trame t_k considéré.The first of these methods was introduced as part of speech recognition and was then used in audio. According to this method, the sound signal of each source s _i (t) is characterized by a set of K _i spectral shapes ${\mathrm{\sigma }}_{{\mathrm{k}}_{\mathrm{i}}}^{\mathrm{2}}$

(f), k _i ∈ [1, ..., K _i ]. If we consider N sources, their mixture is characterized by a set of K ₁ x K ₂ x ... x K _N N-tuples of characteristic spectral forms ( ${\mathrm{\sigma }}_{{\mathrm{k}}_1}^{\mathrm{2}}$

(f), ..., ${\mathrm{\sigma }}_{{\mathrm{k}}_{\mathrm{NOT}}}^{\mathrm{2}}$

(F)). For each frame of index t _k , the method consists in first choosing the N-tuple of spectral shapes that best corresponds to the sound signal of the mixture. For example, it can consist in maximizing the probability of correspondence between the spectrogram of the mixture | x ( t _k , f ) | ² and the resulting variance of the pair of spectral shapes. Next, it consists of filtering the mixture by conventional Wiener filtering using the N-tuple of spectral shapes thus selected. We can note that this method is adaptive since the choice of the parameters of the filter depends on the frame index t _k considered.

The main disadvantage of this method lies in its algorithmic complexity. Indeed, if K characteristic spectral forms by source i and N sources i are considered in the mixture, K ^N N-tuples of characteristic spectral forms must be tested for each frame so that the complexity is in O (K ⁿ x T) if T is the number of frames of the mix signal to be analyzed. This disadvantage of complexity can make this method unacceptable, especially when the number of characteristic spectral forms per source is relatively large.

(f) mais qui sont là regroupées dans un dictionnaire de formes spectrales. Ainsi, le spectrogramme du mélange |x(t_k ,f)|² est décomposé sur l'union des dictionnaires en présence et il est donc possible d'écrire : $${\left|x\left(t_k,f\right)\right|}^2\approx \sum _{k_1=1}^{K_1}{a}_{k_1}\left(t_k\right){\sigma }_{k_1}^2\left(f\right)+\dots +\sum _{k_2=1}^{K_N}{a}_{k_N}\left(t_k\right){\sigma }_{k_N}^2\left(f\right)$$

où les coefficients a_ki (t), sont nommés "facteurs d'amplitude", sont les inconnues à résoudre.Another method has also been proposed to make the separation process adaptive. As before, the sound signal of each source s _i (t) is characterized by a set of K _i characteristic spectral forms ${\mathrm{\sigma }}_{{\mathrm{k}}_{\mathrm{i}}}^{\mathrm{2}}$

(f) but which are there grouped in a dictionary of spectral forms. Thus, the spectrogram of the mixture | x ( t _k , f ) | ² is decomposed on the union of the dictionaries in presence and it is thus possible to write: $${\left|x\left(t_k,f\right)\right|}^2\approx \underset{k_1=1}{\overset{K_1}{\Sigma }}{\mathrm{at}}_{k_1}\left(t_k\right){\sigma }_{k_1}^2\left(f\right)+...+\underset{k_2=1}{\overset{K_{\mathrm{NOT}}}{\Sigma }}{\mathrm{at}}_{k_{\mathrm{NOT}}}\left(t_k\right){\sigma }_{k_{\mathrm{NOT}}}^2\left(f\right)$$

where the coefficients a _{k i} (t), are called "amplitude factors", are the unknowns to solve.

(f) et qui se mélangent entre elles avec des facteurs d'amplitude respectifs a_ki (t) fonction du temps. On notera que chaque facteur d'amplitude a_ki (t) d'une source élémentaire est caractéristique de l'enveloppe de cette source. Il est donc un nombre positif.Note that the equation above can be interpreted as if there were K ₁ + ... + K _N stationary stationary sources which are each characterized by a spectral form ${\mathrm{\sigma }}_{{\mathrm{k}}_{\mathrm{i}}}^{\mathrm{2}}$

(f) and which mix with each other with respective amplitude factors a _{k i} (t) function of time. Note that each amplitude factor has _{k i} (t) an elemental source is characteristic of the envelope of this source. It is therefore a positive number.

   e_i(t_k,f) représente la fraction d'énergie de la source i contenue dans le mélange à analyser.The equation above can be rewritten as follows: $${\left|x\left(t_k,f\right)\right|}^2\approx \underset{i=1}{\overset{K_1}{\Sigma }}{e}_i\left(t_k,f\right)\mathrm{with}{e}_i\left(t_k,f\right)=\underset{k=1}{\overset{K_i}{\Sigma }}{\mathrm{at}}_k\left(t_k\right){\sigma }_{k,i}^2\left(f\right)$$

e _i (t _k , f) represents the fraction of energy of the source i contained in the mixture to be analyzed.

A first method for estimating the sound signals from sources 1 to N is to implement Wiener time-frequency filtering, which is nevertheless adaptive since it depends on the frame index t. This filter is called a generalized Wiener filter. So for the source i, the estimate ŝ _{i, W boy Wut} ( t _k , f ): $${\hat{s}}_{i,W_s}\left(t_k,f\right)=\frac{e_i\left(t_k,f\right)}{{\displaystyle \underset{i=1}{\overset{\mathrm{NOT}}{\Sigma }}}{e}_i\left(t_k,f\right)}x\left(t_k,f\right)$$

et sa phase comme étant estimée par celle du mélange. Il est donc possible d'écrire pour la source i : $${\tilde{\mathrm{s}}}_{\mathrm{i}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{=}\sqrt{{\mathrm{e}}_{\mathrm{i}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)}\mathrm{.}\mathrm{sign}\left[\tilde{\mathrm{x}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\right]$$

où $\mathrm{sign}\left[\mathrm{x}\right]\mathrm{=}\frac{\mathrm{x}}{\left|\mathrm{x}\right|}$

correspond à la phase de x.Another method, called resynthesis, considers the amplitude of the sound signal of each source i to be equal to $\sqrt{{\mathrm{e}}_{\mathrm{i}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)}$

and its phase as estimated by that of the mixture. It is therefore possible to write for the source i: $${\stackrel{\mathrm{~}}{\mathrm{s}}}_{\mathrm{i}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{=}\sqrt{{\mathrm{e}}_{\mathrm{i}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)}\mathrm{.}\mathrm{sign}\left[\stackrel{\mathrm{~}}{\mathrm{x}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\right]$$

or $\mathrm{sign}\left[\mathrm{x}\right]\mathrm{=}\frac{\mathrm{x}}{\left|\mathrm{x}\right|}$

corresponds to the phase of x.

This second method by the use of a dictionary of characteristic spectral shapes has the advantage over the previous method of reducing the algorithmic complexity. Indeed, for n sources each having K spectral forms, the algorithmic complexity is in O (nx K x T) where T is the number of frames to be analyzed, therefore lower than that of the previous method which was in O (K ⁿ x T).

The three methods that have just been presented nevertheless have the major disadvantage that the phase of each of the sources involved (or elementary sources involved according to the method used) is strictly equal to the mixing phase. However, in general, the sources that add up do not all have the same phase so that, in the methods presented above, during the separation, there is destruction of the phase structure of the sources, which can cause annoying effects for listening to sound signals from recovered sources. The human auditory system is indeed very sensitive to phase coherences in the audio signals, in particular inter-frame coherences for fixed f (coherent phase between s ( t _{k +1} , f ) and s ( t _k , f )) and the phase coherences for the same frame but for different values of the frequency f (phase of s ( t _k , f ) for different values of f). These phase coherence effects are notably very sensitive on harmonic sounds, such as of a musical instrument, or voiced sounds, while they are less important on white, pink, and so on. or the sounds of percussion instruments.

The STONE JV publication "Blind source separation using temporal predictability" and MANDIC DP and AL's publication "An online algorithm for blind source extraction based on non-linear prediction approach" describe methods for determining separation signals relative to sound sources from a signal from the mixing of these signals.

The thesis entitled "Separation of several sound sources with a single microphone" by Elie Laurent BENAROYA describes the study of the separation of sound sources with a single sensor from an extension of the Wiener filtering to Gaussian mixing models for the sources as well as from a non-negative decomposition of the spectrum of the mixture on a dictionary of spectral form characteristic of the sources.

The purpose of the present is to propose a method of separating the signals relating to sound sources from a signal resulting from mixing these signals which does not exhibit the phase inconsistencies of the methods mentioned above.

• une étape de détermination d'un signal d'estimée ;
• une étape de prédiction (E40) d'un signal prédit pour la trame présente basée sur le signal de séparation pour la trame précédente ; et
• une étape de détermination du signal de séparation (E50) pour la trame présente sur la base dudit signal prédit et dudit signal d'estimée, caractérisé en ce que ladite étape de détermination du signal de séparation consiste à sommer de manière pondérée le signal d'estimée et le signal prédit, le signal d'estimée étant pondéré par un premier coefficient matriciel déterminé de manière à minimiser la covariance du signal de séparation,

et en ce que la valeur dudit premier coefficient matriciel est calculée au moyen de la relation suivante de la covariance du signal prédit Cov^p(t_k,f) et de la somme de la covariance du signal prédit Cov^p(t_k,f) et de la covariance du signal d'estimée Cov^e (t_k,f), soit : $$\mathrm{\alpha }\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{=}{\left[{\mathrm{Cov}}^{\mathrm{e}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{+}{\mathrm{Cov}}^{\mathrm{p}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\right]}^{\mathrm{-}\mathrm{1}}\mathrm{\cdot }{\mathrm{Cov}}^{\mathrm{p}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)$$

To do this, a method for determining the separation signals respectively relating to sound sources from a signal derived from the mixing of these signals is defined in claim 1, said signals being in the form of successive frames, said method including for each of said sources:

• a step of determining an estimate signal;
• a prediction step (E40) of a predicted signal for the present frame based on the separation signal for the previous frame; and
• a step of determining the separation signal (E50) for the frame present on the basis of said predicted signal and said estimation signal, characterized in that said step of determining the separation signal comprises summing in a weighted manner the signal of estimated and the signal predicted, the estimated signal being weighted by a first matrix coefficient determined so as to minimize the covariance of the separation signal,

and in that the value of said first matrix coefficient is calculated by means of the following covariance relation of the predicted Cov ^p (t _k , f) signal and the sum of the covariance of the predicted signal Cov ^p (t _k , f) and the covariance of the estimation signal Cov ^e (t _k , f), that is: $$\mathrm{\alpha }\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{=}{\left[{\mathrm{Cov}}^{\mathrm{e}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{+}{\mathrm{Cov}}^{\mathrm{p}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\right]}^{\mathrm{-}\mathrm{1}}\mathrm{\cdot }{\mathrm{Cov}}^{\mathrm{p}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)$$

This method also applies to non-sound signals such as any digital signals from the sampling of a transducer for transforming a physical quantity into an electrical signal.

• une étape de détermination d'un signal d'estimée ;
• une étape de prédiction (E40) d'un signal prédit pour la trame présente basée sur le signal de séparation pour la trame précédente ;et
• une étape de détermination du signal de séparation (E50) pour la trame présente sur la base dudit signal prédit et dudit signal d'estimée, caractérisé en ce que ladite étape de détermination du signal de séparation consiste à sommer de manière pondérée le signal d'estimée et le signal prédit, le signal d'estimée étant pondéré par un premier coefficient matriciel déterminé de manière à minimiser la covariance du signal de séparation,

la valeur dudit premier coefficient matriciel est calculée au moyen de la relation suivante de la covariance du signal prédit Cov^p(t_k,f) et de la somme de la covariance du signal prédit Cov^p(t_k,f) et de la covariance du signal d'estimée Cov^e(t_k,f), soit : $$\mathrm{\alpha }\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{=}{\left[{\mathrm{Cov}}^{\mathrm{e}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{+}{\mathrm{Cov}}^{\mathrm{p}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\right]}^{\mathrm{-}\mathrm{1}}\mathrm{\cdot }{\mathrm{Cov}}^{\mathrm{p}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)$$

For this purpose, the subject of the invention is a method for determining separation signals respectively relating to non-sonic sources from a signal resulting from the mixing of these signals as defined in claim 2, said signals being in the form of successive frames, said method including for each of said sources:

• a step of determining an estimate signal;
• a prediction step (E40) of a predicted signal for the present frame based on the separation signal for the previous frame, and
• a step of determining the separation signal (E50) for the frame present on the basis of said predicted signal and said estimation signal, characterized in that said step of determining the separation signal comprises summing in a weighted manner the signal of estimated and the signal predicted, the estimated signal being weighted by a first matrix coefficient determined so as to minimize the covariance of the separation signal,

the value of said first matrix coefficient is calculated using the following covariance covariance cov covence ratio ^p (t _k , f) and the covariance sum of the Cov ^p (t _k , f) predicted signal and the covariance the estimation signal Cov ^e (t _k , f), that is: $$\mathrm{\alpha }\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{=}{\left[{\mathrm{Cov}}^{\mathrm{e}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{+}{\mathrm{Cov}}^{\mathrm{p}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\right]}^{\mathrm{-}\mathrm{1}}\mathrm{\cdot }{\mathrm{Cov}}^{\mathrm{p}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)$$

• La Fig. 1 est un schéma synoptique d'un système de séparation des signaux relatifs à des sources sonores à partir d'un signal issu de mélange de ces signaux selon la présente invention, et
• La Fig. 2 est un diagramme montrant les différentes étapes mises en oeuvre par un procédé de séparation de signaux selon la présente invention.

The characteristics of the invention mentioned above, as well as others, will appear more clearly on reading the following description of an exemplary embodiment, said description being given in relation to the attached drawings, among which:

• The Fig. 1 is a block diagram of a system for separating signals relating to sound sources from a signal derived from mixing these signals according to the present invention, and
• The Fig. 2 is a diagram showing the different steps implemented by a signal separation method according to the present invention.

In the remainder of the description, we will consider sound sources which are in themselves elementary, that is to say which are each characterized by a given characteristic spectral shape. However, sound sources whose spectral shape characteristic is a characteristic among several possible spectral shape characteristics, for example belonging to a dictionary of characteristic spectral shapes (see the preamble of the present description), will also be considered. As mentioned in the preamble of the description, it is then possible to consider a sound source as being a weighted combination of a plurality of elementary sound sources each of which has a given spectral shape characteristic (for example derived from a dictionary or determined ).

In order to solve the problem of phase inconsistencies of the methods of the state of the art mentioned in the preamble of the description, the present invention provides means for connecting adjacent frames. In other words, each elementary sound source is determined in a recursive and iterative manner.

(t_k,f) est le signal d'estimée pour une source du mélange d'indice i. Si l'on dispose de N sources élémentaires, le signal de d'estimée est représenté par un vecteur dont chaque composante est relative à une source : $${\mathrm{S}}^{\mathrm{e}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{=}\left(\begin{array}{c}{\mathrm{s}}_{\mathrm{1}}^{\mathrm{e}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\\ \mathrm{\cdot }\\ {\mathrm{s}}_{\mathrm{N}}^{\mathrm{e}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\end{array}\right)$$

We have shown Fig. 1 , a system for separating sound signals from sound sources according to an embodiment of the present invention which comprises these connecting means between adjacent frames. This system essentially consists of an estimation unit 10 which, on the basis of a signal of frequency domain mixing noted x (t _k , f) obtained for example by a short-term Fourier transform of the signal x (t) in the sampled time domain, delivers an estimation signal represented by the random variable S ^e (t _k , f) of which each component ${\mathrm{s}}_{\mathrm{i}}^{\mathrm{e}}$

(t _k , f) is the estimated signal for a source of the index mixture i. If we have N elementary sources, the estimated signal is represented by a vector of which each component is relative to a source: $${\mathrm{S}}^{\mathrm{e}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{=}\left(\begin{array}{c}{\mathrm{s}}_{\mathrm{1}}^{\mathrm{e}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\\ \mathrm{\cdot }\\ {\mathrm{s}}_{\mathrm{NOT}}^{\mathrm{e}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\end{array}\right)$$

The estimation unit 10 is such that the expectation of the signal at its output is conditioned by the signals x (t _k , f) which are actually observed. We can write: $$S^e\left(t_k,f\right)=E\left[S\left(t_k,f\right)|x\left(t_k,f\right)\right]$$

où e_i(t_k,f) est la fraction d'énergie de la source i contenue dans le signal de mélange, dans la trame d'indice t_k et de fréquence d'indice f, N étant le nombre totale de sources et x̃(t_k,f) étant le signal de mélange.The estimation unit 10 is for example a Wiener filter (see the different forms of this type of filter given in the preamble of the present description), a unit operating by a time-frequency thresholding method, or by a method said Ephraim and Malah, etc. For example, in the case of a Wiener filter, each component of the vector S ^e (t _k , f) can be obtained by the following relation: $$S^e\left(t_k,f\right)=\left(\begin{array}{c}{\hat{S}}_{1,W_{\mathrm{boy\; Wut}}}\left(t_k,f\right)\\ ⋮\\ {\hat{S}}_{\mathrm{NOT},W}\left(t_k,f\right)\end{array}\right)\mathrm{with}{\hat{s}}_{i,W_{\mathrm{boy\; Wut}}}\left(t_k,f\right)=\frac{e_i\left(t_k,f\right)}{{\displaystyle \underset{i=1}{\overset{\mathrm{NOT}}{\Sigma }}}{e}_i\left(t_k,f\right)}x\left(t_k,f\right)$$

where e _i (t _k , f) is the energy fraction of the source i contained in the mixing signal, in the frame of index t _k and frequency of index f, where N is the total number of sources and x (t _k , f) being the mixing signal.

où K_i représente le nombre de sources élémentaires considérés pour la source i, a_ki (t_k) représente le facteur d'amplitude de la source élémentaire d'indice k_i et ${\mathrm{\sigma }}_{{\mathrm{k}}_{\mathrm{i}}}^{\mathrm{2}}$

(f) la variance de cette source élémentaire d'indice k_i.We recall here that for an elementary source i, we can write: $${\mathrm{e}}_{\mathrm{i}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{=}\underset{{\mathrm{k}}_{\mathrm{i}}\mathrm{=}\mathrm{1}}{\overset{{\mathrm{K}}_{\mathrm{i}}}{\mathrm{\Sigma }}}{\mathrm{at}}_{{\mathrm{k}}_{\mathrm{i}}}\left({\mathrm{t}}_{\mathrm{k}}\right){\mathrm{\sigma }}_{{\mathrm{k}}_{\mathrm{i}}}^{\mathrm{2}}\left(\mathrm{f}\right)$$

where K _i represents the number of elementary sources considered for the source i, a _{k i} (t _k ) represents the amplitude factor of the elementary source of index k _i and ${\mathrm{\sigma }}_{{\mathrm{k}}_{\mathrm{i}}}^{\mathrm{2}}$

(f) the variance of this elementary source of index k _i .

The system for separating sound signals from sound sources represented in Fig. 1 further comprises an update unit 20 and a prediction unit 30. It is these units 20 and 30 which constitute the inter-frame link means which are mentioned above.

The prediction unit 30 is provided to deliver a prediction signal considered as a corresponding random variable S ^p (t _k , f)

We recall here that if we have N elementary sources, the prediction signal is a vector whose each component is relative to a source: $${\mathrm{S}}^{\mathrm{p}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{=}\left(\begin{array}{c}{\mathrm{s}}_{\mathrm{1}}^{\mathrm{p}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\\ \mathrm{⋮}\\ {\mathrm{s}}_{\mathrm{NOT}}^{\mathrm{p}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\end{array}\right)$$

As can be seen from the Fig. 1 the updating unit 20, on the basis of the prediction signal S ^p (t _k , f) delivered by the prediction unit 30 and the estimation signal S ^e (t _k , f) delivered by the estimation unit 10 delivers, as for it, the separation signal whose random variable is noted S ^tot (t _k , f).

If we have N elementary sources, the separation signal is represented by a vector whose each component is relative to a source: $${\mathrm{S}}^{\mathrm{early}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{=}\left(\begin{array}{c}{\mathrm{s}}_{\mathrm{1}}^{\mathrm{early}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\\ \mathrm{⋮}\\ {\mathrm{s}}_{\mathrm{NOT}}^{\mathrm{early}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\end{array}\right)$$

Concerning the prediction unit 30, in the simplest case it can return to introduce an offset term between two successive frames, by its unit 32, and one can write: $$S^p\left(t_k,f\right)=H\left(f\right)\cdot S^{\mathit{early}}\left(t_{k-1},f\right)$$

The predicted signal for the present frame is based on the separation signal for the previous frame.

où H(f) est un terme qui, dans le domaine fréquentiel, est représentatif du décalage entre deux trames successives et qui, du fait que les signaux considérés sont des signaux stationnaires, peut s'écrire : $$H\left(f\right)=\exp \left[2\mathit{\pi i}\frac{f\mathrm{.}M}{T}\right]$$

où T est la longueur d'une trame, M le décalage considéré, et i le nombre complexe tel que i² = -1. Généralement, le décalage M entre trame est inférieur à la longueur T d'une trame et, même, il est souvent moitié de la longueur d'une trame : $$\mathrm{M}\mathrm{=}\mathrm{T}\mathrm{/}\mathrm{2}$$

The expectation of the prediction signal is given by the following relation: $${\hat{S}}^p\left(t_k,f\right)=H\left(f\right)\cdot {\hat{S}}^{\mathit{early}}\left(t_{k-1},f\right)$$

where H (f) is a term which, in the frequency domain, is representative of the offset between two successive frames and which, because the signals considered are stationary signals, can be written as: $$H\left(f\right)=\exp \left[2\mathit{\pi i}\frac{f\mathrm{.}M}{T}\right]$$

where T is the length of a frame, M is the shift considered, and i is the complex number such that i ² = -1. Generally, the M difference between the frame is less than the length T of a frame and, even, it is often half the length of a frame: $$\mathrm{M}\mathrm{=}\mathrm{T}\mathrm{/}\mathrm{2}$$

As for the updating unit 20, it is intended to determine the separation signal S ^tot (t _k , f) by summing the estimation signal S ^e (t _k , f) in a weighted manner and the predicted signal S ^p (t _k , f). In the embodiment shown, the estimated signal S ^e (t _k , f) is weighted by a matrix coefficient α (tk, f) while the predicted signal is weighted by a coefficient I-α (tk, f) , I being the unit matrix.

For example, this is done by adding, in an adder 21, to the predicted signal S ^p (t _k , f) an error signal calculated as the difference between the predicted signal S ^p (t _k , f) and the signal d estimated S ^e (t _k , f), said error signal being weighted by a coefficient α (tk, f), the weighting being performed by a weighting unit 23. It is therefore possible to write the relation: $${\mathrm{S}}^{\mathrm{early}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{=}{\mathrm{S}}^{\mathrm{p}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{+}\mathrm{\alpha }\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{.}\left({\mathrm{S}}^{\mathrm{e}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{-}{\mathrm{S}}^{\mathrm{p}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\right)$$

The separation system shown in Fig. 1 is provided for determining the optimal coefficient matrix α (tk, f) for minimizing the variance of the estimate of the separation signal S ^tot (t _k , f). It can be shown that this optimum value of the weighting factor is given by the following covariance ratio of the predicted Cov ^p signal (t _k , f) and the sum of covariance of the predicted Cov ^p signal (t _k , f) and the covariance of the estimation signal Cov ^e (t _k , f), that is: $$\mathrm{\alpha }\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{=}{\left[{\mathrm{Cov}}^{\mathrm{e}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{+}{\mathrm{Cov}}^{\mathrm{p}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\right]}^{\mathrm{-}\mathrm{1}}\mathrm{\cdot }{\mathrm{Cov}}^{\mathrm{p}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)$$

(t_k,f) qui constitue alors la sortie de l'unité de mise à jour 20 : $${\mathrm{S}}_{\mathrm{0}}^{\mathrm{tot}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{=}{\mathrm{S}}_{\mathrm{0}}^{\mathrm{p}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{+}\mathrm{\alpha }\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{.}\left({\mathrm{S}}_{\mathrm{0}}^{\mathrm{e}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{-}{\mathrm{S}}_{\mathrm{0}}^{\mathrm{p}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\right)$$

Since the value of the weighting coefficient α (t _k , f) is known, it is possible to determine the expectation of the separation signal. ${\mathrm{S}}_{\mathrm{0}}^{\mathrm{early}}$

(t _k , f) which then constitutes the output of the update unit 20: $${\mathrm{S}}_{\mathrm{0}}^{\mathrm{early}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{=}{\mathrm{S}}_{\mathrm{0}}^{\mathrm{p}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{+}\mathrm{\alpha }\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{.}\left({\mathrm{S}}_{\mathrm{0}}^{\mathrm{e}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{-}{\mathrm{S}}_{\mathrm{0}}^{\mathrm{p}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\right)$$

We will proceed according to the diagram of the Fig. 2 . In this diagram, it can be seen that it has two branches I and II: the first I groups steps E10, E20 and E30 and corresponds to the covariance calculations. different random variables leading essentially to the calculation of the optimal coefficient matrix α (t _k , f) while the second II which groups together the steps E40 and E50 corresponds to the calculations of the expectations of these random variables leading to the calculation of the expectation of the separation signal according to the estimation signal delivered by the estimation unit 10.

More precisely, in step E10, the updating of the covariance of the predicted signal represented, it is recalled, by the random variable S ^p (t _{k + 1} , f) is carried out.

avec $$\mathrm{var}\left(b^p\left(t_k,f\right)\right)$$

variance du bruit de prédiction.Due to the unit 32 which links two successive frames together, it can easily be shown that the covariance of the predicted signal is given by the following relation: $${\mathit{Cov}}^p\left(t_k,f\right)={\mathit{Cov}}^{\mathit{early}}\left(t_{k-1},f\right)+\mathrm{var}\left(b^p\left(t_k,f\right)\right)$$

with $$\mathrm{var}\left(b^p\left(t_k,f\right)\right)$$

variance of the prediction noise.

The module of the function H (f) is indeed equal to 1.

The variance of the prediction noise var (b ^p (t _k , f)) depends on the sources or sub-sources considered and on the frequency f. It does not depend on the frame considered, so it can also be written: $$\mathrm{var}\left({\mathrm{b}}^{\mathrm{p}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\right)\mathrm{=}\mathrm{var}\left({\mathrm{b}}^{\mathrm{p}}\left(\mathrm{f}\right)\right)$$

This variance is advantageously estimated in a learning phase. In the end, we have: $${\mathrm{Cov}}^{\mathrm{p}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{=}{\mathrm{Cov}}^{\mathrm{early}}\left({\mathrm{t}}_{\mathrm{k}\mathrm{-}\mathrm{1}},\mathrm{f}\right)\mathrm{+}\mathrm{var}\left({\mathrm{b}}^{\mathrm{p}}\left(\mathrm{f}\right)\right)$$

Cov ^tot (t _k-1 , f) is a quantity that was calculated at the previous iteration (see step E30 below).

In step E20, the optimal coefficient matrix α (t _k , f) is determined. To do this, we use the expression above: $$\mathrm{\alpha }\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{=}{\left[{\mathrm{Cov}}^{\mathrm{e}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{+}{\mathrm{Cov}}^{\mathrm{p}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\right]}^{\mathrm{-}\mathrm{1}}\mathrm{\cdot }{\mathrm{Cov}}^{\mathrm{p}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)$$

(f) et les facteurs d'amplitude a_ki (t_k) des sources ou sources élémentaires considérées.The covariance of the predicted separation signal Cov ^p (t _k , f) is given by the calculation performed in step E10. As for the covariance of the estimation signal Cov ^e (t _k , f), it is determined by the characteristic spectral forms ${\mathrm{\sigma }}_{{\mathrm{k}}_{\mathrm{i}}}^{\mathrm{2}}$

(f) and amplitude factors a _{k i} (t _k ) sources or elementary sources considered.

où b(t,f) représente l'expression d'un bruit blanc gaussien stationnaire de variance ${\mathrm{\sigma }}_{\mathrm{b}}^{\mathrm{2}}$

Quant aux sources élémentaires s_i (t,f), elles sont considérées a priori comme des sources gaussiennes non stationnaires de variance a_i(t,f) ${\mathrm{\sigma }}_{\mathrm{i}}^{\mathrm{2}}$

(f) mais comme stationnaires conditionnellement à a_i(t).We recall that the equation of the mixture is as follows: $$\mathit{x}\left(\mathit{t},\mathit{f}\right)={\displaystyle \underset{j}{\Sigma }{s}_j}\left(t,f\right)+b\left(t,f\right)$$

where b ( t , f ) represents the expression of a stationary Gaussian white noise of variance ${\mathrm{\sigma }}_{\mathrm{b}}^{\mathrm{2}}$

As for the elementary sources s _i ( t, f ), they are considered a priori as non-stationary Gaussian sources of variance a _i (t, f) ${\mathrm{\sigma }}_{\mathrm{i}}^{\mathrm{2}}$

(f) but as conditionally stationary at a _i (t).

The estimation signal S ^e (t, f) of the mixture of the set of elementary sources is a Gaussian random variable of variance Cov ^e (t, f):

expression dans laquelle :

• a_j(t_k,f) est le facteur d'amplitude de la source ou de la source élémentaire d'indice j, pour la trame d'indice t_k et pour la fréquence d'indice f,
• σ_j(f) est la forme spectrale caractéristique de la source ou de la source élémentaire d'indice j et pour la fréquence f,
• σ_b est la variance d'un bruit blanc gaussien, et
• N est le nombre total de sources élémentaires considérées.

It has been shown that this covariance of the estimation signal Cov ^e (t _k , f) could be expressed as follows: $${\mathit{Cov}}^e\left(t_k,f\right)=\left(\begin{array}{ccc}{\mathrm{at}}_1\left(t_k\right){\mathit{\sigma }}_1^{\mathrm{2}}\left(f\right)& 0& 0\\ 0& \ddots & 0\\ 0& 0& {\mathrm{at}}_{\mathrm{NOT}}\left(t_k\right){\mathit{\sigma }}_{\mathrm{NOT}}^{\mathrm{2}}\left(f\right)\end{array}\right)-\frac{1}{{\displaystyle \underset{j=1}{\overset{\mathrm{NOT}}{\Sigma }}}\begin{array}{c}{\mathrm{at}}_j\left(t_k\right){\mathit{\sigma }}_j^{\mathrm{2}}\left(f\right)\end{array}+{\mathit{\sigma }}_b^{\mathrm{2}}}\left(\begin{array}{c}{\mathrm{at}}_1\left(t_k\right){\sigma }_1^{\mathrm{2}}\left(f\right)\\ ⋮\\ {\mathrm{at}}_{\mathrm{NOT}}\left(t_k\right){\mathrm{\sigma }}_{\mathrm{NOT}}^{\mathrm{2}}\left(f\right)\end{array}\right)\left(\begin{array}{ccc}{\mathrm{at}}_1\left(t_k\right){\mathit{\sigma }}_1^{\mathrm{2}}\left(f\right)& \cdots & {\mathrm{at}}_{\mathrm{NOT}}\left(t_k\right){\mathit{\sigma }}_{\mathrm{NOT}}^{\mathrm{2}}\left(f\right)\end{array}\right)$$

expression in which:

• a _j (t _k , f) is the amplitude factor of the source or elementary source of index j, for the frame of index t _k and for the frequency of index f,
• σ _j (f) is the spectral form characteristic of the source or the elementary source of index j and for the frequency f,
• σ _b is the variance of a white Gaussian noise, and
• N is the total number of elementary sources considered.

expression dans laquelle :

• I est la matrice identité,
• α (t_k,f) est la matrice de coefficients telle que déterminée à l'étape E20 ci-dessus,
• Cov^p(t_k,f) est la covariance du signal de séparation prédit telle que calculée à l'étape E10.

In step E30, the covariance matrix of the separation signal is updated using the following expression: $${\mathrm{Cov}}^{\mathrm{early}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{=}\left[\mathrm{I}\mathrm{-}\mathrm{\alpha }\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\right]\mathrm{.}{\mathrm{Cov}}^{\mathrm{p}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)$$

expression in which:

• I is the identity matrix,
• α (t _k , f) is the coefficient matrix as determined in step E20 above,
• Cov ^p (t _k , f) is the covariance of the predicted separation signal as calculated in step E10.

After step E30, for covariance related calculations, the next frame is considered and the process is resumed in step E10.

(t_k,f) laquelle est donnée par la relation suivante en fonction de l'espérance du signal de séparation ${\mathrm{S}}_{\mathrm{0}}^{\mathrm{tot}}$

(t_k-1,f) déterminée à la trame précédente : $$S_{\mathrm{0}}^p\left(t_k,f\right)=H\left(f\right)\cdot S_{\mathrm{0}}^{\mathit{tot}}\left(t_{k-1},f\right)$$

We now consider the steps E40 and E50 related to the calculations of the expectations. In step E40, the expectation of the predicted signal is determined. ${\mathrm{S}}_{\mathrm{0}}^{\mathrm{p}}$

(t _k , f) which is given by the following relation as a function of the expectation of the separation signal ${\mathrm{S}}_{\mathrm{0}}^{\mathrm{early}}$

(t _k-1 , f) determined at the previous frame: $$S_{\mathrm{0}}^p\left(t_k,f\right)=H\left(f\right)\cdot S_{\mathrm{0}}^{\mathit{early}}\left(t_{k-1},f\right)$$

expression dans laquelle :

• ${\mathrm{S}}_{\mathrm{0}}^{\mathrm{p}}$
  
  (t_k,f) est l'espérance du signal de séparation prédit déterminé à l'étape E10 ci-dessus,
• ${\mathrm{S}}_{\mathrm{0}}^{\mathrm{e}}$
  
  (t_k,f) est l'espérance du signal d'estimée telle qu'il apparaît à la sortie du l'unité d'estimation 10, et
• α(t_k,f) est la matrice de coefficients telle que déterminée à l'étape E20 ci-dessus.

In step E50, the expectation of the separation signal is calculated by means of the following expression: $${\mathrm{S}}_{\mathrm{0}}^{\mathrm{early}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{=}{\mathrm{S}}_{\mathrm{0}}^{\mathrm{p}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{+}\mathrm{\alpha }\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{.}\left({\mathrm{S}}_{\mathrm{0}}^{\mathrm{e}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\mathrm{-}{\mathrm{S}}_{\mathrm{0}}^{\mathrm{p}}\left({\mathrm{t}}_{\mathrm{k}},\mathrm{f}\right)\right)$$

expression in which:

• ${\mathrm{S}}_{\mathrm{0}}^{\mathrm{p}}$
  
  (t _k , f) is the expectation of the predicted separation signal determined in step E10 above,
• ${\mathrm{S}}_{\mathrm{0}}^{\mathrm{e}}$
  
  (t _k , f) is the expectation of the estimate signal as it appears at the output of the estimation unit 10, and
• α (t _k , f) is the coefficient matrix as determined in step E20 above.

(t_k,f) est le signal de sortie du système. Ses composantes sont les signaux de séparation de chacune des sources ou des sources élémentaires considérées.The expectation of the separation signal ${\mathrm{S}}_{\mathrm{0}}^{\mathrm{early}}$

(t _k , f) is the output signal of the system. Its components are the signals of separation of each of the sources or elementary sources considered.

(t_k,f) est décalée d'une trame pour obtenir l'espérance du signal de séparation de la trame t _k-1 et cette dernière espérance est utilisée au cours de l'étape E40.In step E60, the expectation of the separation signal of the frame Tr, $S_o^{\mathit{early}}$

( t _k , f ) is shifted by one frame to obtain the expectation of the separation signal of the frame t _{k -1} and this latter expectation is used during step E40.

After the steps E50 and E60, the next frame is considered and the process is resumed in step E40 for the steps related to the expectation calculations.

The steps E10 and E40 are implemented by the prediction unit 30 while the steps E20, E30 and E50 are implemented by the update unit 20.

It will be noted that at the initialization of the method, the expectation and the covariance of the random variable representing the separation signal are set to zero and then the steps E10 and E40 are implemented.

Claims (7)

1. Process of determination of the separation signals relative respectively to sound sources from a signal coming from a mix of these signals. The signals are represented as consecutive frames, the aforesaid process including for each of the aforesaid sources:

   • a step of determination of the estimated signal of the sources

   • a step of prediction (E40) of predicted signal for the current frame based on the separation signal for the previous frame; and,

   • a step of determination of the separation signal (E50) for the previous frame on the basis of the predicted signal and the estimated signal, characterized by the fact that the step of determination of the separation signal consists in summing in a pondered way the estimated signal and the predicted signal, the estimated signal being pondered by a first coefficient matrix α(t_k,f) ant the predicted signal being pondered by a second coefficient matrix equal to the unit matrix minus the first matrix coefficient, the first coefficient matrix being determined by means of the following relation of the covariance of the predicted signal Cov ^p (t_k , f) and of the sum of the covariance of the predicted signal Cov ^p(t_k , f) and of the covariance of the estimated signal Cov ^e (t_k , f) namely: $$\alpha \left({\mathit{t}}_{\mathit{k}},\mathit{f}\right)\mathrm{=}{\left[{\mathrm{Cov}}^e\left({\mathit{t}}_{\mathit{k}},\mathit{f}\right)\mathrm{+}{\mathrm{Cov}}^p\left({\mathit{t}}_{\mathit{k}},\mathit{f}\right)\right]}^{\mathrm{-}\mathrm{1}}{\mathrm{Cov}}^p\left({\mathit{t}}_{\mathit{k}},\mathit{f}\right)$$

   
2. Process of determination of the separation signals respectively relative to non-sound sources from a signal coming from the mix of these signals, these signals showing as consecutive frames, the aforesaid process including for each of the aforesaid sources:

   • a step of determination of the estimated signal of the sources

   • a step of prediction (E40) of predicted signal for the current frame based on the separation signal for the previous frame; and,

   • a step of determination of the separation signal (E50) for the previous frame on the basis of the predicted signal and the estimated signal, characterized by the fact that the step of determination of the separation signal consists in summing in a pondered way the estimated signal and the predicted signal, the estimated signal being pondered by a first coefficient matrix α(t_k, f) ant the predicted signal being pondered by a second coefficient matrix equal to the unit matrix minus the first matrix coefficient, the first coefficient matrix being determined by means of the following relation of the covariance of the predicted signal Cov ^p (t_k , f) and of the sum of the covariance of the predicted signal Cov ^p (t_k , f) and of the covariance of the estimated signal Cov ^e (t_k, f) namely: $$\alpha \left({\mathit{t}}_{\mathit{k}},\mathit{f}\right)\mathrm{=}{\left[{\mathrm{Cov}}^e\left({\mathit{t}}_{\mathit{k}},\mathit{f}\right)\mathrm{+}{\mathrm{Cov}}^p\left({\mathit{t}}_{\mathit{k}},\mathit{f}\right)\right]}^{\mathrm{-}\mathrm{1}}{\mathrm{Cov}}^p\left({\mathit{t}}_{\mathit{k}},\mathit{f}\right)$$

   
3. Process of separation according to claim 1 or claim 2, characterized by the fact that the covariance of the predicted signal Cov ^p (t_k , f) is determined as a function of the covariance of the separation signal Cov^tot (t_k-1 , f) for the previous frame by means of the following relation: $${\mathrm{Cov}}^p\left({\mathit{t}}_{\mathit{k}},\mathit{f}\right)={\mathrm{Cov}}^{\mathrm{tot}}\left({\mathit{t}}_{\mathit{k}-1},\mathit{f}\right)+\mathrm{var}\left(b^p\left({\mathit{t}}_{\mathit{k}},\mathit{f}\right)\right)$$

   

   
   var (b^p (t_k, f)) being the variance of the prediction noise that depends on the considered sources or sub-sources.
4. Process of separation according to claim 3, characterized by the fact that the aforesaid variance of the prediction noise var (b^p (t_k, f)) is estimated in a learning phase.
5. Process of separation according to any of the claims 1 to 4, characterized by the fact that the aforesaid covariance of the estimated signal Cov^e(t_k, f) is determined by means of the following relation: $${\mathrm{Cov}}^e\left(t_k,f\right)=\left(\begin{array}{ccc}{a}_1\left(t_k\right){\sigma }_1^2\left(f\right)& 0& 0\\ 0& \ddots & 0\\ 0& 0& a_N\left(t_k\right){\sigma }_N^2\left(f\right)\end{array}\right)\frac{1}{\sum _{j=1}^N{a}_j\left(t_k\right){\sigma }_j^2\left(f\right)+{\sigma }_b^2)}\left(\begin{array}{c}{a}_1\left(t_k\right){\sigma }_1^2\left(f\right)\\ ⋮\\ a_N\left(t_k\right){\sigma }_N^2\left(f\right)\end{array}\right)\left(\begin{array}{c}{a}_1\left(t_k\right){\sigma }_1^2\left(f\right)\end{array}\dots \begin{array}{c}{a}_N\left(t_k\right){\sigma }_N^2\left(f\right)\end{array}\right)$$

   

   
   expression in which:

   • a_j (t_k, f) is the amplitude factor of the source or of the elementary source indexed by j for the frame indexed by t_k and for the frequency indexed by f,

   • σ _j (f) is the spectral shape that characterizes the source of the elementary source indexed by j and for the frequency f,

   • σ _b is the variance of a Gaussian white noise, and

   • N is the total number of considered sources or elementary sources.
6. Process of separation according to any of the claims 1 to 5, characterized by the fact that the covariance matrix of the separation signal is updated by using the following expression: $${\mathrm{Cov}}^{\mathrm{tot}}\left({\mathit{t}}_{\mathit{k}},\mathit{f}\right)\mathrm{=}\left[\mathrm{I}\mathrm{-}\alpha \left({\mathit{t}}_{\mathit{k}},\mathit{f}\right)\right]{\mathrm{Cov}}^p\left({\mathit{t}}_{\mathit{k}},\mathit{f}\right)$$

   

   
   Expression in which :

   • I is the identity matrix

   • α(t_k,f) is the matrix of the first weighting coefficient; and

   • Cov ^p (t_k, f) is the covariance of the predicted signal.
7. Process of separation according to any of the claims 1 to 6, characterized by the fact that it contains one determination step of the input signal S^e (t_k, f), each component ${\hat{s}}_i^e\left(t_k,f\right)$

   

   corresponding to the estimated of an elementary source i of the aforesaid signal S^e (t_k, f), being obtained from the following formulas: $${\hat{s}}_i^e\left(t_k,f\right)\mathrm{=}\frac{e_i\left(t_k,f\right)}{\sum _{j=1}^N{e}_j\left(t_k,f\right)}x\left(t_k,f\right)$$

   

   $$e_i\left(t_k,f\right)={\displaystyle \sum _{k_i=1}^{K_i}}{a}_{k_i}\left(t_k\right){\sigma }_{k_i}^2\left(f\right)$$

   

   
   In which:

   • e_i (t_k, f) is the energy fraction of the source i contained in the signal coming from the mix of signals, in the frame indexed by t_k and of frequency indexed by f, N being the total number of sources;

   • x(t_k, f) is the signal of the mix of signals;

   • K_i is the number of elementary sources considered for the source i;

   • a_ki (t_k ) is the amplitude factor of the elementary source indexed by k_i and,

   • ${\sigma }_{k_i}^2\left(f\right)$

   

   is the variance of the elementary source indexed by k_i .

Images