EP2901447B1 - Method and device for separating signals by minimum variance spatial filtering under linear constraint - Google Patents

Description

The present invention relates to a method for separating some of the source signals comprising a digital audio overall signal. The invention also relates to a device for implementing this method.

Signal mixing consists of summing several signals, called source signals, to obtain one or more composite signals, called mixed signals. In audio applications in particular, the mixing may consist of a simple step of adding the source signals or may also include signal filtering steps before and / or after the addition. On the other hand, for some applications such as compact disc audio, the source signals can be mixed differently to form two mixed signals corresponding to the two channels or channels (left and right) of a stereo signal.

Separation of sources consists of estimating source signals from the observation of a certain number of different mixed signals formed from these same source signals. The objective is generally to enhance, if possible to completely extract one or more target source signals. The separation of sources is particularly difficult in so-called "under-determined" cases in which there is a number of mixed signals less than the number of source signals present in the mixed signals. The extraction is in this case very difficult or impossible because of the small amount of information available in these mixed signals compared to that present in the source signals. Music signals on compact-disc audio are a particularly representative example because only two stereo channels (ie two mixed left and right signals), which are generally very redundant, are available for a large number of potential speakers. source signals.

There are several types of approaches in the separation of source signals: among them the blind separation, the analysis of scenes Computational auditory, and model-based separation. Blind separation is the most general form, in which no information on the source signals nor on the nature of the mixed signals is known a priori. We then make a number of assumptions about these source signals and the mixed signals (for example that the source signals are statistically independent) and we estimate the parameters of a separation system by maximizing a criterion based on these hypotheses (for example maximizing the independence of the signals obtained by the separation device). However, this method is generally used in cases where there are many mixed signals (at least as much as source signals) and is therefore not applicable to under-determined cases in which the number of mixed signals is less than number of source signals.

Computational auditory scene analysis usually consists of modeling partial source signals, but the mixed signal is not explicitly decomposed. This method is based on the mechanisms of the human auditory system to separate the source signals in the same way that our ear does. These include: DPW Ellis, Using knowledge to organize sound: The prediction-driven approach to computational auditory scene analysis, and its application to speechlnon-speech mixture (Speech Communication, 27 (3), pp. 281-298, 1999 ) D. Godsmark and GJBrown, A blackboard architecture for computational auditory scene analysis (Speech Communication, 27 (3), pp. 351-366, 1999 ), in the same way T. Kinoshita, S. Sakai, and H. Tanaka, Musical sound source identification based on frequency adaptation (In Proc. IJCAI Workshop on CASA, pp. 18-24, 1999 ). However, computational auditory scene analysis currently leads to insufficient results in terms of the quality of the separate source signals.

Another form of separation relies on a decomposition of the mixture on the basis of suitable functions. There are two main categories: the temporary parsimonious decomposition and the parsimonious decomposition in frequency.

For the first, it is a question of decomposing the waveform of the mixture, and for the other it is a question of decomposing its spectral representation, in a sum of elementary functions called "atoms" elements of a dictionary. Various algorithms allow to choose the type of dictionary and the corresponding decomposition most likely. For the time domain, mention may in particular be made of: L. Benaroya, Sparse representations for source separation with a single sensor (GRETSI Proceedings, 2001) ), or PJ Wolfe and SJ Godsill, A Gabor Regression scheme for audio signal analysis (IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, pp. 103-106, 2003). ). In the method proposed by Gribonval ( R. Gribonval and E. Bacry, Harmonic Decomposition of Audio Signals with Matching Pursuit, IEEE Trans. Signal Proc., 51 (1), pp. 101-112, 2003 ), we classify the atoms of decomposition in independent subspaces, which makes it possible to extract groups of harmonic partials. One of the restrictions of this method is that generic dictionaries of atoms such as the Gabor atoms, for example, not adapted to the signals, do not give good results. Moreover, for these decompositions to be effective, the dictionary must contain all the translated forms of the waveforms of each type of instrument. The decomposition dictionaries must then be extremely large for projection and thus separation to be effective.

To overcome this problem of invariance by translation which appears in the temporal case, there are approaches of parsimonious decomposition in frequency. We can mention in particular MA Casey and A. Westner (Separation of mixed audio sources by independant subspace analysis, Proc. Int. Computer Music Conf., 2000 ) who introduced the analysis in independent subspace (ISA). This analysis consists of breaking down the short-term amplitude spectrum of the mixed signal (calculated by short-term Fourier transform (TFCT)) on an atomic basis, and then grouping the atoms into independent subspaces, each sub-space. space being specific to a source, to then resynthesize the sources separately. However, this approach is generally limited by several factors: the resolution of the spectral analysis by TFCT, the superposition of the sources in this spectral domain, and the restriction of the spectral separation to the amplitude (the phase of the resynthesized signals being that of the mixed signal ). It is thus generally difficult to represent the mixed signal as a sum of independent subspaces because of the complexity of the sound scene in the spectral domain (strong interweaving of the different components) and because of the evolution as a function of time , the contribution of each component in the mixed signal. In fact, the methods are often evaluated on well-controlled "simplified" mixed signals (the source signals are MIDI instruments or are relatively well separable instruments, in limited numbers).

Another method of source separation is "informed" source separation: information relating to one or more source signals is transmitted with the mixed signal to the decoder. The decoder is then able, from algorithms and information, to at least partially separate at least one source signal of the mixed signal. An example of informed source separation is described by M. Parvaix and L. Girin (IEEE trans. Audio Speech Lang. Process., Volume 19, pages 1721-1733, August 2011). ). The information transmitted to the decoder indicates in particular the two predominant source signals in the mixed signal, for different frequency zones. However, such a method is not always suitable when there are more than two source signals contributing simultaneously in the same frequency zone of the mixed signal: in this case, at least one source signal is neglected, thus creating a "spectral hole" in the reconstitution of said source signal.

It is also known, particularly in the field of telecommunications, to filter signals picked up by a plurality of sensors, as a function of the position in the space of said signals relative to said sensors. This is a spatial filtering (or Beamforming) which makes it possible to privilege the signal in a direction of the given space and to filter the signals coming from other directions. An example of such filters are the linear minimum constrained variance (LMCV) spatial filters. An example of such a filter is disclosed in particular in the document EP 1 633 121 .

An object of the present invention is therefore to provide a method for separating source signals included in one or more mixed signals, more effectively.

• on détermine le module de l'amplitude ou la puissance normalisée du ou des signaux sources particuliers à partir des valeurs représentatives dudit ou desdits signaux sources particuliers contenues dans le signal mixé, puis
• on effectue un filtrage spatial à variance minimale sous contrainte linéaire du signal mixé pour obtenir au moins partiellement chaque signal source particulier, ledit filtrage étant basé sur la répartition dudit signal source particulier entre au moins deux canaux du signal mixé, et le module de l'amplitude ou la puissance normalisée dudit signal source particulier étant utilisé comme contrainte linéaire du filtre.

For this purpose, in one embodiment, there is provided a method for separating, at least partially, one or more particular digital audio source signals contained in a multichannel digital audio mix signal (i.e. comprising at least two channels), for example stereo. The mixed signal is obtained by mixing several digital audio source signals and includes values representative of the particular source signal or signals. According to the method:

• the modulus of the amplitude or the normalized power of the particular source signal or signals is determined from the values representative of the at least one particular source signal contained in the mixed signal, and then
• linearly minimally space-constrained spatial filtering of the mixed signal to at least partially obtain each particular source signal, said filtering being based on the distribution of said particular source signal between at least two channels of the mixed signal, and the module of the amplitude or the normalized power of said particular source signal being used as a linear constraint of the filter.

The representative values may be the temporal, spectral or spectro-temporal distribution of the particular source signal, or the temporal, spectral or spectro-temporal contribution of the particular source signal in the mixed signal. The representative values of the source signals can thus be in amplitude modulus or in normalized power (that is to say in energy, which corresponds to the square of the modulus of the amplitude): the representative values can thus be the values in modulus of the amplitude or the values of normalized power (or of energy).

Representative values may for example be the temporal, spectral or spectro-temporal distribution of the particular source signal, or the temporal, spectral or spectro-temporal contribution of the particular source signal in the mixed signal, for several zones (or points) of a particular source signal. time-frequency plan. In this case, the determination of the amplitude or normalized power module of the particular source signal or signals can be done in the time-frequency plane: the amplitude modules and the normalized powers are spectro-temporal values.

A transformation or representation in the time-frequency plane consists in representing, in energy (or normalized power) or in modulus of the amplitude (that is to say the square root of the energy), the source signal in function of two parameters, time and frequency. This corresponds to the evolution, in energy or in module, of the frequency content of the source signal as a function of time. Thus, for a given instant and a given frequency, a real positive value corresponding to the signal components at this frequency and at this instant is obtained. Examples of theoretical formulations and practical implementations of time-frequency representations are already described ( L. Cohen: Time-Frequency Distributions, Review, Proceedings of the IEEE, vol. 77, No. 7, 1989 ; F. Hlawatsch, F. Auger: Time-frequency, concepts and tools, Hermès Science, Lavoisier 2005 ; P. Flandrin: Time Frequency, Hermès Science, 1998 ).

Thus, thanks to the method described, it is possible to effectively separate, with spatial filtering improved by the information contained in the mixed signal, the particular source signals, without making any assumption on these different signals (with the exception of the statistical hypotheses classics, ie independence of the source signals, zero mean of these source signals, Gaussian distribution). In particular, the method is based on the distribution of each source signal between the different channels of the mixed signal to isolate said source signals (spatial filtering). The use of a linear variance minimum variance filter makes it possible to obtain a high-performance spatial separation, by using the modulus of amplitude or the normalized power of the source signal as a constraint. It is thus possible to spatially decorrelate a particular source signal of the mixed signal, and to adjust the amplitude of the separated signal to the desired level at the same time. The spatial filtering step is therefore improved by taking into consideration the representative value of the particular source signal of which we know.

It is in particular possible to simultaneously isolate the different particular source signals present in the mixed signal, for example by using as many spatial filters as there are source signals to be separated.

Preferably, the filtering is also based on the amplitude module or the normalized power of the particular source signals. More specifically, the spatial filtering step may comprise modeling a spatial correlation matrix using the amplitude module or the normalized power of the particular source signals and the distribution of said particular source signal between at least two channels of the mixed signal. .

Preferably, the mixed signal comprises values representative of the particular source signal or signals for at least two channels of the mixed signal, and, before performing the spatial filtering, from the mixed signal and from said representative values, particular source signals are determined. , the distribution of each particular source signal between said at least two channels of the mixed signal.

Alternatively, it is possible to receive as input, for example with the mixed signal, the distribution of the particular source signal or signals between at least two channels of the mixed signal.

In other words, the distribution of the particular source signals between the different channels of the mixed signal can be provided during the implementation of the separation method, for example at the same time as the representative values of said particular source signals, or Well can be determined during the separation process from the multichannel mixed signal and the representative values of the particular source signals.

According to one embodiment, the determination of the modulus of the amplitude or of the normalized power of the particular source signal or signals comprises the extraction of the values representative of the particular source signal or signals which have been inserted in the mixed signal, by example by tattoo. The extraction of the representative values results from the transmission of the representative values of the particular source signals, which can be done with the mixed signal, for example when the information is tattooed or inserted in an inaudible manner, in the mixed signal, or by a channel particular of the mixed signal which is dedicated to the transmission of said representative values.

• un moyen de détermination du module de l'amplitude ou de la puissance normalisée du ou des signaux sources particuliers à partir des valeurs représentatives dudit ou desdits signaux sources particuliers contenues dans le signal mixé, et
• un filtre spatial à variance minimale sous contrainte linéaire apte à isoler au moins partiellement, à partir du signal mixé, chaque signal source particulier, ledit filtre étant basé sur la répartition dudit signal source particulier entre au moins deux canaux du signal mixé, et le module de l'amplitude ou la puissance normalisée dudit signal source particulier étant utilisé comme contrainte linéaire.

In another aspect, there is provided a device for separating, at least partially, one or more particular digital audio source signals contained in a mixed digital audio multichannel signal. The mixed signal is obtained by mixing several digital audio source signals and includes values representative of the particular source signal or signals. The device comprises:

• means for determining the modulus of the amplitude or the normalized power of the particular source signal or signals from the values representative of the at least one particular source signal contained in the mixed signal, and
• a linear constrained minimum variance spatial filter adapted to at least partially isolate, from the mixed signal, each particular source signal, said filter being based on the distribution of said particular source signal between at least two channels of the mixed signal, and the module amplitude or normalized power of said particular source signal being used as a linear constraint.

Preferably, the mixed signal is a stereo signal.

Preferably, the mixed signal comprises values representative of the particular source signal or signals for at least two channels of the mixed signal, and the device comprises a means of determining, from the mixed signal and said representative values of the particular source signals, the distribution of each particular source signal between said at least two channels of the mixed signal.

Preferably, the means for determining the amplitude or the normalized power module comprises means for extracting the values representative of the particular source signal or signals which have inserted into the mixed signal, for example by tattooing.

• la figure 1 représente schématiquement un mode de réalisation d'un dispositif de séparation selon l'invention ; et
• la figure 2 est un organigramme d'un procédé de séparation selon l'invention.

The invention will be better understood from the study of a particular embodiment, taken by way of non-limiting example and illustrated by the appended drawings, in which:

• the figure 1 schematically represents an embodiment of a separation device according to the invention; and
• the figure 2 is a flowchart of a separation process according to the invention.

où a_i=[a_i ^g , a_i ^d]^T (^T représentant la transposée de la matrice) et a_i ^g et a_i ^d représentent la répartition du signal source i dans chaque canal du signal mixé : (a_i ^g)² + (a_i ^d)² =1.In the remainder of the detailed description, it is considered that the mixed signal s _mix (t) is a stereo signal with a left channel s _mix ^g (t) and a right channel s _mix ^d (t), and comprising p source signals s ₁ (t), ..., s _p (t). The mixed signal s _mix (t) can be written as the product of the p source signals by a mixing matrix A: $$\mathrm{AT}=\left[{{\mathrm{at}}_{\mathrm{1}}}^{\mathrm{boy\; Wut}},...,{{\mathrm{at}}_{\mathrm{p}}}^{\mathrm{boy\; Wut}}\right]=\left[{\mathrm{at}}_{\mathrm{1}},...,{\mathrm{at}}_{\mathrm{p}}\right]\left[{{\mathrm{at}}_{\mathrm{1}}}^{\mathrm{d}},...,{{\mathrm{at}}_{\mathrm{p}}}^{\mathrm{d}}\right]$$

where a _i = [a _i ^g , a _i ^d ] ^T ( ^T representing the transpose of the matrix) and a _i ^g and a _i ^d represent the distribution of the source signal i in each channel of the mixed signal: (a _i ^g ) ² + (a _i ^d ) ² = 1.

More precisely, the coefficients a _i ^g and a _i ^d can be written in the following form: a _i ^g = sin (θ _i ) and a _i ^d = cos (θ _i ), where θ _i represents the balance of the source signal i between the two channels of the mixed signal.

avec : s_mix(t)=[s_mix ^g(t),s_mix ^d(t)]^T et s(t)=[s₁(t),...,s_p(t)]^T (^T représentant la transposée).In other words, we have: $${\mathrm{s}}_{\mathrm{mix}}\left(\mathrm{t}\right)=\mathrm{AT}\mathrm{.}\mathrm{s}\left(\mathrm{t}\right)$$

with: s _mix (t) = [s _mix ^g (t), s _mix ^d (t)] ^T and s (t) = [s ₁ (t), ..., s _p (t)] ^T ( ^T representing the transpose).

Furthermore, it is considered in the remainder of the description, that the signals are audio signals.

où N est une constante et f(n) est une fonction de fenêtre de la transformée de Fourier à court terme.In the case of this description, the short-term Fourier transformation is considered to be a transformation in the time-frequency plane. The transform of the source signal i in the time-frequency plane is thus written in the form: $${\mathrm{S}}_{\mathrm{i}}\left(\mathrm{k},\mathrm{m}\right)=\Sigma {\mathrm{s}}_{\mathrm{i}}\left(\mathrm{k}+\mathrm{not}\right)\mathrm{f}\left(\mathrm{not}\right){\mathrm{e}}^{-\mathrm{2}\mathrm{i\pi mn}/\mathrm{NOT}}$$

where N is a constant and f (n) is a window function of the short-term Fourier transform.

It is considered in the remainder of the description that the linear constraint of the spatial filter is the normalized power. For a given source signal s _i , and a point (k, m) of the given time-frequency plane, one thus obtains as normalized energy or power φ _i (k, m): $${\mathrm{\phi }}_{\mathrm{i}}\left(\mathrm{k},\mathrm{m}\right)={\left|{\mathrm{S}}_{\mathrm{i}}\left(\mathrm{<figure-callout\; id="2"\; label="k\; m"\; filenames="imgf0001.png"\; state="\{\{state\}\}">k\; </figure-callout>},\mathrm{m},\mathrm{}\right)\right|}^{\mathrm{2}}$$

The representative value of the source signal can thus be | S _i (k, m) | (value in module) or else φ _i (k, m) (energy value equal to the value in standardized power). The representative value of the source signal can also be the logarithm of the energy value: $${\mathrm{\Phi }}_{\mathrm{i}}=\mathrm{10}{\log }_{\mathrm{10}}\left({\mathrm{\phi }}_{\mathrm{i}}\left(\mathrm{k},\mathrm{m}\right)\right)$$

The representative value of the source signal can also be determined after processing on the source signal, for example by reducing the frequency resolution of the energy spectrum or by adapting the quantification of the representative values to the sensitivity of the human ear. It is then possible to obtain representative values of the source signals which are less bulky in terms of size, while maintaining a desired sound quality.

It is considered in the remainder of the description that the representative value of the source signals is the quantized normalized power value (or energy) Φ _i (k, m).

The representative values of the source signals Φ _i (k, m) are transmitted to the separation device or decoder. They can be by a dedicated channel (associated with the stereo channels to form the mixed signal), or by incorporation in the mixed signal, for example tattooing or using unused bits of the mixed signal. In the latter case, the separation device may comprise means for extracting the representative values, receiving as input the mixed signal and outputting the representative values of the source signals.

Similarly, the separation device can also receive the distributions of the source signals in each channel (or channel) of the mixed signal: a ₁ ^g , ... a _p ^g , a ₁ ^d , ... a _p ^d . These distributions can be transmitted by a dedicated channel (associated with the stereo channels to form the mixed signal, or independent of the stereo channels), or by incorporation into the mixed signal, for example by tattooing or by using unused bits of the mixed signal. . In the latter case, the separation device may comprise a means for extracting the distributions of the source signals, receiving as input the mixed signal and providing, as output, the distributions of the source signals. The means for extracting the representative values and the means for extracting the distributions can be one and the same means.

Alternatively, the separation device may comprise means for determining the distributions of the source signals: such a determination means may receive as input the mixed signal and the representative values Φ _i (k, m), and output the distribution of said signal source a _i ^g , a _i ^d . This is possible especially when each channel of the mixed signal comprises the values representative of a source signal for said channel of the mixed signal: in other words, the values representative of a given source signal will not be the same for each channel of the mixed signal. the difference between the representative values of the same source signal for the different channels of the mixed signal making it possible to determine the distribution of said source signal between the different channels of the mixed signal.

On the figure 1 schematically an embodiment of a separation device 1 of particular source signals contained in a mixed signal s _mix . The separation device 1 receives as input the stereo channels s _mix ^g and s _mix ^d of the mixed signal s _mix , and delivers separate particular source signals at least partially s' _i , with 1 varying from 1 to p. The purpose of the separation device 1 is to deliver, at least partially, a plurality of particular source signals contained in the mixed signal _mix by using the representative values of said particular source signals Φ _i (k, m).

For the purposes of the present description, it is considered that the separation device 1 receives, as input, the channels of the mixed digital audio signal s _mix ^g (t) and s _mix ^d (t), in which are inserted, for example by tattooing, the values representative of the particular source signals Φ _i (k, m), and possibly the distributions at ₁ ^g , ..., a _p ^g , a ₁ ^d , ..., a _p ^d of particular source signals between the two signal channels Mixed digital audio s _mix ^d (t) and s _mix ^g (t).

The separating device 1 comprises a transformation means 2, an extraction means 3, a processing means 4, a filtering means 5, and an inverse transformation means 6.

The transformation means 2 receives as input the channels of the mixed digital audio signal s _mix ^g (t) and s _mix ^d (t) and delivers, at the output, the transform of the channels of the mixed signal in the time-frequency plane S _mix ^g (k, m) and S _mix ^d (k, m).

The extraction means 3 receives as input the transformation of the channels of the mixed signal in the time-frequency plane S _mix ^d (k, m) and S _mix ^g (k, m), and delivers the representative values Φ _i (k, m) particular source signals contained in the mixed signal. The extraction means 3 can, if necessary, also deliver the distributions a _i ^g , ..., a _p ^g , a ₁ ^d , ..., a _p ^d of particular source signals between the two channels of the mixed signal. digital audio s _mix ^d (t) and s _mix ^g (t), when these are inserted in the mixed signal. The extraction means 3 thus makes it possible to extract from the mixed signal the representative values that have been added thereto a posteriori, for example by tattooing, and to isolate the mixed signal. The representative values Φ _i (k, m) are then transmitted to the processing means 4 and, where appropriate, the distributions at ₁ ^g , ..., a _p ^g , a ₁ ^d , ..., a _p ^d are transmitted by means of filtering 5.

It should be noted that the extraction means 3 may alternatively receive directly at the input, the channels of the mixed signal s _mix ^d (t) and s _mix ^g (t).

The processing means 4 make it possible to process the representative values Φ _i (k, m) received by the extraction means 3, in order to determine an estimate of the normalized power φ ' _i (k, m) of the source signals to be separated, in the time-frequency plane. The estimates of the normalized power φ ' _i (k, m) of the source signals to be separated are then transmitted to the filtering means 5.

The transform of the channels of the mixed signal in the time-frequency plane S _mix ^d (k, m) and S _mix ^g (k, m) provided by the transformation means 2, the estimates of the normalized powers of the particular source signals φ ' _i (k, m), and the distributions at ₁ ^g , ..., a _p ^g , a ₁ ^d , ..., a _p ^d of particular source signals between the two channels of the mixed digital audio signal s _mix ^d (t ) and s _mix ^g (t), are thus provided to the filtering means 5.

The filtering means 5 makes it possible to obtain an estimation S ' _i (k, m) of each particular source signal, by means of spatial filtering. The filtering means 5 makes it possible, in the time-frequency plan, to isolate the particular source signal, by means of spatial filtering with minimum variance under linear stress. More particularly, the filtering means 5 is based on the distribution of said particular source signal between the two channels of the mixed signal to isolate the particular source signal: it is therefore a spatial filtering (in English: "beamforming"). On the other hand, in order to improve the filtering and estimation of the source signal obtained, the spatial filter uses the normalized power of the particular source signal to be separated as a linear constraint, in order to obtain an estimate closer to the original source signal.

avec : $${\mathrm{S}}_{\mathrm{mix}}\left(\mathrm{k},\mathrm{m}\right)={\left[{{\mathrm{S}}_{\mathrm{mix}}}^{\mathrm{g}}\left(\mathrm{k},\mathrm{m}\right),{{\mathrm{S}}_{\mathrm{mix}}}^{\mathrm{d}}\left(\mathrm{k},\mathrm{m}\right)\right]}^{\mathrm{T}}$$

et $$\mathrm{S}\left(\mathrm{k},\mathrm{m}\right)={\left[{\mathrm{S}}_{\mathrm{1}}\left(\mathrm{k},\mathrm{m}\right),\dots ,{\mathrm{S}}_{\mathrm{p}}\left(\mathrm{k},\mathrm{m}\right)\right]}^{\mathrm{T}}$$

More precisely, in the time-frequency plane, we have: $${\mathrm{S}}_{\mathrm{mix}}\left(\mathrm{k},\mathrm{m}\right)=\mathrm{AT}\mathrm{.}\mathrm{S}\left(\mathrm{k},\mathrm{m}\right)$$

with: $${\mathrm{S}}_{\mathrm{mix}}\left(\mathrm{k},\mathrm{m}\right)={\left[{{\mathrm{S}}_{\mathrm{mix}}}^{\mathrm{boy\; Wut}}\left(\mathrm{k},\mathrm{m}\right),{{\mathrm{S}}_{\mathrm{mix}}}^{\mathrm{d}}\left(\mathrm{k},\mathrm{m}\right)\right]}^{\mathrm{T}}$$

and $$\mathrm{S}\left(\mathrm{k},\mathrm{m}\right)={\left[{\mathrm{S}}_{\mathrm{1}}\left(\mathrm{k},\mathrm{m}\right),...,{\mathrm{S}}_{\mathrm{p}}\left(\mathrm{k},\mathrm{m}\right)\right]}^{\mathrm{T}}$$

avec: $${\mathrm{W}}_{\mathrm{ik}}={\left[{{\mathrm{w}}_{\mathrm{ik}}}^{\mathrm{g}},{{\mathrm{w}}_{\mathrm{ik}}}^{\mathrm{d}}\right]}^{\mathrm{T}}\mathrm{et\; S}{\prime }_{\mathrm{i}}\left(\mathrm{k},\mathrm{m}\right)={\left[{\mathrm{S}{\prime }_{\mathrm{i}}}^{\mathrm{g}}\left(\mathrm{k},\mathrm{m}\right),{\mathrm{S}{\prime }_{\mathrm{i}}}^{\mathrm{d}}\left(\mathrm{k},\mathrm{m}\right)\right]}^{\mathrm{T}}$$

Each mixed signal S _mix ^d (k, m) and S _mix ^g (k, m) is then decomposed into estimates of particular source signals S ' ₁ (k, m), ..., S' _p (k, m) using linear spatial filtering: $$\mathrm{S}{\text{'}}_{\mathrm{i}}\left(\mathrm{k},\mathrm{m}\right)={{\mathrm{w}}_{\mathrm{ik}}}^{\mathrm{boy\; Wut}}\mathrm{.}{{\mathrm{S}}_{\mathrm{mix}}}^{\mathrm{boy\; Wut}}\left(\mathrm{k},\mathrm{m}\right)+{{\mathrm{w}}_{\mathrm{ik}}}^{\mathrm{d}}\mathrm{.}{{\mathrm{S}}_{\mathrm{mix}}}^{\mathrm{d}}\left(\mathrm{k},\mathrm{m}\right)={{\mathrm{W}}_{\mathrm{ik}}}^{\mathrm{T}}\mathrm{.}{\mathrm{S}}_{\mathrm{mix}}\left(\mathrm{k},\mathrm{m}\right)$$

with: $${\mathrm{W}}_{\mathrm{ik}}={\left[{{\mathrm{w}}_{\mathrm{ik}}}^{\mathrm{boy\; Wut}},{{\mathrm{w}}_{\mathrm{ik}}}^{\mathrm{d}}\right]}^{\mathrm{T}}\mathrm{and\; S}{\text{'}}_{\mathrm{i}}\left(\mathrm{k},\mathrm{m}\right)={\left[{\mathrm{S}{\text{'}}_{\mathrm{i}}}^{\mathrm{boy\; Wut}}\left(\mathrm{k},\mathrm{m}\right),{\mathrm{S}{\text{'}}_{\mathrm{i}}}^{\mathrm{d}}\left(\mathrm{k},\mathrm{m}\right)\right]}^{\mathrm{T}}$$

W _ik is the spatial filter (or "beamformer") for obtaining the estimate S ' _i (k, m) of the i ^th source signal in the subband k from the mixed signal S _mix (k, m) .

où r(k,m) est la somme des autres signaux sources.In the case of a minimum variance spatial filter under linear stress, the sum of all the interfering source signals, with the exception of the one to be filtered, is considered as noise. Thus, the mixed signal can be rewritten as follows: $${\mathrm{S}}_{\mathrm{mix}}\left(\mathrm{k},\mathrm{m}\right)={\mathrm{at}}_{\mathrm{i}}\mathrm{.}{\mathrm{S}}_{\mathrm{i}}\left(\mathrm{k},\mathrm{m}\right)+\mathrm{r}\left(\mathrm{k},\mathrm{m}\right)$$

where r (k, m) is the sum of the other source signals.

où R_Smix est la matrice de corrélation spatiale des deux canaux S_mix ^d(k,m) et S_mix ^g(k,m) du signal mixé S_mix(k, m).The estimate S ' _i (k, m) is obtained by minimizing the average power of the noise or, equivalently, the mean output power of the spatial filter, according to the direction of the source signal to be separated: $$\mathrm{P}\left({\mathrm{\theta }}_{\mathrm{i}}\right)={{\mathrm{W}}_{\mathrm{ik}}}^{\mathrm{T}}\left(\mathrm{m}\right)\mathrm{.}\mathrm{R}{\text{'}}_{\mathrm{smix}}\left(\mathrm{k},\mathrm{m}\right)\mathrm{.}{\mathrm{W}}_{\mathrm{ik}}\left(\mathrm{m}\right)$$

where R _Smix is the spatial correlation matrix of the two S channels _mix ^d (k, m) and S _mix ^g (k, m) of the mixed signal S _mix (k, m).

The solution is given by: $$W_{\mathit{ik}}\left(m\right)=R{\text{'}}_{\mathit{smix}}^{-1}\left(\mathit{k},\mathit{m}\right)\mathrm{.}{\mathrm{at}}_i\mathrm{.}\sqrt{\frac{\phi {\text{'}}_i\left(\mathit{k},\mathit{m}\right)}{{{\mathrm{at}}_i}^T\mathrm{.}{{R\text{'}}^{-1}}_{\mathit{smix}}\left(\mathit{k},\mathit{m}\right)\mathrm{.}{\mathrm{at}}_i}}$$

avec : R'_Smix ^-1(k, m) = ∑ ϕ'_i(k,m).a_i.a_i ^T.We thus obtain: $$S{\text{'}}_i\left(k,m\right)=\sqrt{\frac{\phi {\text{'}}_i\left(\mathit{k},\mathit{m}\right)}{{{\mathrm{at}}_i}^T\mathrm{.}R{\text{'}}_{\mathit{smix}}^{-1}\left(\mathit{k},\mathit{m}\right)\mathrm{.}{\mathrm{at}}_i}}\mathrm{.}{{\mathrm{at}}_i}^T\mathrm{.}{{R\text{'}}^{-1}}_{\mathit{smix}}\left(k,m\right)\mathrm{.}{S}_{\mathit{mix}}\left(k,m\right)$$

with: R ' _Smix ^-1 (k, m) = Σ φ' _i (k, m) .a _i .a _i ^T.

When applied to the mixed signal S _mix (k, m), the resulting filter makes it possible to reduce the contribution of the power spectrum of the other signals. Moreover, thanks to the linear constraint, the power of the estimated source signal corresponds to the power of the initial source signal for the different points of the time-frequency plane (which can be verified by re-injecting the solution W _ik into the equation defining P (θ _i )). Thus, the filtering means 5 makes it possible to spatially decorrelate the i ^th source signal from the rest of the mixed signal, while adjusting the amplitude of said decorrelated signal to the desired level.

It may also be noted that, when the amount of information tattooed in the mixed signal is too large for the tattoo noise to be neglected, it is also possible to adjust the components of the estimated source signals as follows: $$\mathrm{S}{\text{'}}_{\mathrm{i}}\left(\mathrm{k},\mathrm{m}\right)=\mathrm{S}{\text{'}}_{\mathrm{i}}\left(\mathrm{k},\mathrm{m}\right)\mathrm{.}\left(\mathrm{\surd \phi }{\text{'}}_{\mathrm{i}}\left(\mathrm{k},\mathrm{m}\right)\right)/\left|\mathrm{S}{\text{'}}_{\mathrm{i}}\left(\mathrm{k},\mathrm{m}\right)\right|$$

The transforms of the estimates of the separate particular source signals are then transmitted to the inverse transformation means 6. The means 6 makes it possible to transform the transforms of the estimates of the separate source signals into time signals s' ₁ (t), ..., s'p (t) corresponding, at least partially, to the source signals s ₁ (t), ..., s _p (t).

On the figure 2 there is shown a flow chart showing the different steps of the separation process according to the invention.

The method includes a first step 7 in which the mixed signal is transformed in a time-frequency plane. Then, in a step 8, the information tattooed in the mixed signal is extracted, in particular the representative values and the distributions of the source signals between at least two channels of the mixed signal. During a step 9, the normalized powers of the source signals to be separated are determined, and then, in step 10, a minimum variance spatial filtering under linear stress is performed, the constraint being the normalized power of the source signal to be separated. . Finally, in a step 11, an inverse transformation of the transforms of the separate particular source signals is performed so as to obtain, at least partially, the particular source signals.

In the case of audio signals, it is thus possible to perform at the output of the separation system of the invention a number of major controls in audio listening (volume, tone, effects) so independent on the different elements of the sound stage (instruments and voices obtained by the separation device).

Claims (8)

1. A method of separating, at least in part, one or more particular digital audio source signals (s_i) contained in a mixed multichannel digital audio signal (s_mix), the mixed signal being obtained by mixing a plurality of digital audio source signals (s₁, ..., s_p) and including representative values (Φ_i) of the particular source signal(s), the method comprising:

   · determining (9) the modulus of the amplitude or the normalized power (ϕ'_i) of the particular source signal(s) from the representative values (Φ_i) in the time-frequency plane of said particular source signal(s) contained in the mixed signal;

   the method being characterized by then performing (10) linearly constrained minimum variance spatial filtering in order to obtain, at least in part, each particular source signal (s'_i), said filtering being based on the distribution (a_i) of said particular source signal between at least two channels of the mixed signal, and the modulus of the amplitude or the normalized power (ϕ'_i) of said particular source signal being used as a linear constraint of the filter.
2. A method according to claim 1, wherein the mixed signal includes representative values (Φ_i ^g, Φ_i ^d) of the particular source signal(s) for at least two channels of the mixed signal (s_mix ^g, s_mix ^d), and wherein, prior to performing spatial filtering, the mixed signal and said representative values of the particular signals are used to determine the distribution (a_i ^g, a_i ^d) of each particular source signal (s_i) between said at least two channels of the mixed signal (s_mix ^g, s_mix ^d).
3. A method according to claim 1, wherein the distribution (a_i) of the particular source signal(s) between at least two channels of said mixed signal is received as input, e.g. in the mixed signal.
4. A method according to any preceding claim, wherein determining the modulus of the amplitude or the normalized power (ϕ'_i) of the particular source signal(s) comprises extracting (8) representative values (Φ_i) of the particular source signals that have been inserted into the mixed signal, e.g. by watermarking.
5. A method according to any preceding claim, wherein the modulus of the amplitude or the normalized power (ϕ'_i) of said particular source signal are spectro-temporal values.
6. A device (1) for separating, at least in part, one or more particular digital audio source signals (s_i) contained in a multichannel mixed digital audio signal (s_mix), the mixed signal (s_mix) being obtained by mixing a plurality of digital audio source signals (s₁, ..., s_p) and including representative values (Φ_i) of the particular source signal(s), the device comprising:

   · determination means (4) for determining the modulus of the amplitude or the normalized power (ϕ'_i) of the particular source signal(s) from the representative values (Φ_i) in the time-frequency plane of said particular source signal(s) contained in the mixed signal;

   the device being characterized by a linearly constrained minimum variance spatial filter (5) adapted to isolate, at least in part, each particular source signal (s'_i) from the mixed signal (s_mix), said filter being based on the distribution (a_i) of said particular source signal between at least two channels of the mixed signal (s_mix ^g, s_mix ^d), and the modulus of the amplitude or the normalized power (ϕ'_i) of said particular source signal being used as a linear constraint.
7. A device according to claim 6, wherein the mixed signal includes representative values (Φ_i) of the particular source signal(s) for at least two channels of the mixed signal, the device including determination means for determining the distribution (a_i) of each particular source signal between said at least two channels of the mixed signal from the mixed signal and from said representative values (Φ_i) of the particular source signals.
8. A device according to claim 6 or claim 7, also including extractor means (3) for extracting the representative values (Φ_i) of the particular source signal(s) that have been inserted in the mixed signal, e.g. by watermarking.

Images