GB2368936A

GB2368936A - Digital signal processing method and system

Info

Publication number: GB2368936A
Application number: GB0103839A
Authority: GB
Inventors: James Stone
Original assignee: University of Sheffield
Current assignee: University of Sheffield
Priority date: 2000-11-09
Filing date: 2001-02-16
Publication date: 2002-05-15
Also published as: GB0103839D0; GB0027508D0

Abstract

A method and system for blind source separation which uses a measure of temporal predictability to identify a source signal from a mixture of such source signals. The temporal predictability of any signal mixture is defined to be less than that of its component source signal. The temporal predictability is used to recover source signals from a set of linear mixtures of source signals by finding an unmixing matrix that maximises a measure of temporal predictability. This measure of temporal predictability can also be used for blind deconvolution, by finding a deconvolutionfilter which minimises the degree of predictability of deconvolved signals. Unlike conventional methods and systems for source separation and blind deconvolution, such as independent component analysis, the embodiments of the present invention produce a closed form solution and require minimal assumptions regarding the probability density function of source signals.

Description

DIGITAL SIGNAL PROCESSING METHOD AND SYSTEM

The present invention relates to digital signal processing and, more particularly, to blind source signal separation.

In many signal processing applications, the sample signals provided by suitable detectors are mixtures of many unknown signals from unknown sources. The "separation of sources"problem is to extract at least one of the unknown signals from such a mixture.

Typically, the signal sources as well as the characteristics of the combined mixture signal are unknown. Such signal processing, without knowledge of the signal sources, other than the general statistical assumption of source independence, is known within the art as the"blind source separation problem".

The blind source separation problem can arise and have utility in variety of context such as, for example the separation of radio or radar signals sent by an array of antennas, separation of odours in a mixture by a suitable sense array, the parsing of the environment into separate objects by our biological visual system and the separation of biomagnetic sources by a super conducting quantum interference device array in magnetoencephalography. The blind source separation problem finds particular application in, for example, sonar array signal processing and signal processing within a telecommunications system and in particular within a cellular telecommunication system. An example of a context in which a blind source separation problem may arise is two people speaking simultaneously with

each person being a different distance from two microphones. Each microphone records a linear mixture for the two voices. The separation of those voices from the combined signal without knowing the characteristics of the source of the signals would represent such a blind source separation problem.

Blind signal processing or blind source separation are terms used within the art for the recovery of unknown source signals from a received mixture of composite

signal. US 5, 706, 402 discloses a"Blind signal processing system employing information maximisation to recover unknown signals through unsupervised minimisation of output redundancy". A neural network system and unsupervised learning process is disclosed for separating unknown source signals from their received mixtures by solving a blind source separation problem. The unsupervised learning procedure solves the general blind signal processing problem by maximising joint output entropy through gradient ascent. The neural network system can separate a multiplicity of unknown source signals from measured mixture signals where the mixture characteristics and the original source signals are both unknowns. However, the system disclosed in US 5,706, 402 does not provide an analytical solution and accordingly, is a relatively slow process.

It is an object of the present invention to mitigate at least some of the problems of the prior art.

Accordingly, a first aspect of the present invention provides a digital signal processing method (and system) for deriving a signal from a mixture signal comprising a combination of a plurality of independent digital signals, the method comprising the steps of storing a first data set (x) representing sampled digital data of the mixture signal; defining a measure of the predictability of the first data set, the measure being a function of the first data set (x) and a first operand (Wi) ; processing the first data set representing digital data of a sampled mixture signal comprising a combination of a plurality of a source signals to produce an output signal (yi) which results from a critical point of the variation of the measure of predictability with the first operand such that the measure of predictability of at least one of the source signals is equal to or greater than the measure of predictability of the sampled mixture signal Preferably, an embodiment provides a method wherein the definition of the measure of predictability is

where Yr is the value of the signal y at time T, y, is a short-term moving average of values of y ; y, is a longterm moving average of values of y; U, the denominator, is a measure of the extent to which Yr is predicted by the short-term moving average, ##, of the past values of y; and V, the numerator, is a measure of the variability of y in terms of the extent to which Yr is predicted by the long-term moving average, Yr r of the past values of y.

An embodiment provides a method wherein the shortterm moving average of the values of y is given by =y,) + (l-) )) where 0/Lyl, where Its is arranged to have a first predetermined half-life, hs and in which the long-term moving-average of the values of y is given by y, (l-) +)( -hL) Y (,-I) where is arranged to have a second predetermined half-life, hL. The relationship between a half-life, h, and the parameter À, is given by/1=2-1/ Preferably, the second predetermined half-life is longer than the first predetermined half-life and, more preferably, the second predetermined half-life is at least 100 times longer than the first predetermined halflife.

A second aspect of the present invention provides a method (and system) wherein the definition of the measure

cw of predictability is F= log---, where a scalar signal, gwcwt

Yi, is formed from the application of a lxM matrix, ! Vi, to a vector, x, representing potential or actual linear mixtures of the first data set, C is an MxM long-term covariance matrix of x, and Cis a short-term covariance matrix of x.

Preferably, an embodiment provides a method in which the long-term covariance, C between the ith and jth n mixtures of x are such that Cl) = :c)(-;c) and r cu Y (Xi7--X"'XXtr--xj,). Advantageously, the values (X, r-xr) r

and ("x) only needs to be calculated once for a given set of mixture signals.

The technique of gradient ascent can be used to determine the critical points (e. g. saddle points). Suitably, an embodiment provides a method further

comprising the step of maximising F with respect to Wi,

2) - 2which gives VF=-*-C--'-C.

I V U

Preferably, the determination of critical points is an interative process. Accordingly, an embodiment provides a method further comprising the steps of iteratively optimising F with respect in to W1, using W, = W, +7/VF, where # has a predeterminable value. A preferred value of # is 0.001 Having determined the maxima, an embodiment further comprises the step of calculating y, = W1x from the value of W1 for which F was at a maxima.

Ultimately, an embodiment may output the recovered signal in a physical form. Accordingly, an embodiment provides a method further comprising the step of outputting a signal based on yi.

Preferably, an embodiment provides a method further comprising the step of calculating eigenvectors, Wi, for

the critical points of F. The step of calculating the eigenvectors, Wi, comprises calculating the eigenvalues,

-v ~ / ! =V/U, where VF=0, which gives C=-, C, which, in u terms of the eigenvalues, A, gives C=AL C.

Advantageously, embodiments of the present invention provide a solution to an eigenvalue problem, which if the number of eigenvalues is less then 5, is an analytical solution to the problem of blind source separation by utilising temporal predictability.

The result of voice mixtures described above exemplify three universal properties of linear mixtures of statistically independent source signals which are: (1) temporal predictability (conjecture): the temporal predictability of any signal mixture is less than (or equal to) that of its components or signals, (2) Gaussian probability density function: the central limit theorem ensures that the extent to which the probability density function (pdf) of any mixture approximates a Gaussian distribution is greater than or equal to any of its component source signals, and (3) statistical independence: the degree of statistical independence between any two signal mixtures is less than (or equal to) the degree of independence between any two source signals. Property 2 forms the basis of projection pursuit as disclosed in Freedman, JH. 1987, exploratory projection suit. J amer. Statistical association, 82 (397), 294-266. Properties 1 and 2 are assumptions underlying independent component analysis as disclosed in, for example, Jutten & Herault, 1998, independent component analysis-v-PCA, pages 643-646 of: Proc. Eusipco. and Bell, AJ and Sejnowski, TJ, 1995, and information-maximisation approach to blind separation and blind deconvolution, Neural Computation, 7,1129-1159 the entire contents of which are incorporated herein for all purposes. All three properties are generic characteristics of signal mixtures. However, unlike

properties 2 and 3, property 1, that is, temporal predictability forms the only basis of blind source separation for the embodiments of the present invention.

The embodiments of the present invention are based upon the above temporal predictability conjecture which is that the temporal predictability of any mixture is less than (or equal to) that of any of its component signals. Examples will be given below of the use of the temporal predictability conjecture for separation of physic signals, such as, for example, voices and music.

Problem definition and temporal predictability Before describing an embodiment of the present invention, the theory upon which the embodiments are based will now be described.

Consider a random vector s = (s1#s2#...#sK)t of K statistically independent source signals, where the ith

row in s is a signal si measured at n time points and the superscript t denotes a transpose operator. In the theory presented hereafter it is assumed that the source signals are statistically independent unless otherwise indicated. A random vector x= (x1#x2#...#xM)t of M K linear mixtures of signals in s can be formed with an M x K mixing matrix A such that x = As. If the rows of A are linearly independent then any source signal s, can be recovered from x with a lxM matrix, Wi, such that si=Wix.

The blind source separation problem to be addressed here consists in finding an unmixing matrix W = (Wijwj... jWK) t such that each row vector Wi recovers a different signal Yi, where Yl is a scaled version and/or sign reversed version of a source signal si.

The embodiments of the present invention for recovering source signals are based on the following conjecture: the temporal predictability of a signal mixture X is usually less than that of any of the source signals that contribute to xi. For example, the waveform obtained by adding two sine waves of different frequencies is more"complex"than either of the original sine waves. A measure of temporal predictability, F (Wl, x) is defined and used to estimate the relative predictability of a signal yi recovered by given matrix Wi, where yi=Wix. If source signals are more predictable than any linear mixture, yi of those signals, then the value of Wi which maximises the predictability of an

extracted signal, yi, should yield a source signal, that is, Yi = csi, where c is a constant. Information theoretic analysis of F shows that maximising the temporal predictability of a signal amounts to differentially maximising the power of the Fourier components with low (non-zero) frequencies see, for example, Stone J V, 1996a, a Canonical microfunction for learning perceptual invariances, Perception, 25 (2), 207220 the entire contents of which are incorporated herein for all purposes. In an embodiment, the definition of signal predictability, F, used herein is

where YT is the value of the signal, y at time T.

The term U measures the extent to which YT is predicted by a short-term moving average # of values in y. In contrast, the term V is a measure of the overall variability in y, as measured by the extent to which y vis predicted by a long-term moving average y with straight line above the y of values in y. The predicted values ## and y, are both exponentially weighted sums of signals values measured up to time T, such that recent

values have a larger weighting then those in the distant past. Therefore, it follows that y, (r-))'''') (r-)) where 0 < < 1 Y, = AL Y (,-I) +AL) Y (,-I) where 0 : : : ; 1 The half-life hL of XL is much longer (typically of the order of 100 times longer) than the corresponding half-life hs of ks. It should be noted that maximising only V would result in a high variance signal with no constraints on the temporal structure of the signal. In contrast, minimising only U would result in a DC signal.

It will be appreciated that in both cases, trivial solutions would be obtained for W, because V can be maximised by setting the norm of W, to be large, and U can be minimised by setting #Wl# = 0. In contrast, the ratio V/U can be maximised only if two constraints are both satisfied which are (1) y has a non-zero range, that is, a high variance, and (2) the values in y change relatively slowly over time. It should also be noted that the value of F is independent of the norm of Wi, so that only changes in the direction of Wi effect the value of F.

Extracting Signals by Maximising Signal Predictability Consider a scalar signal, y, formed from the application of a lxM matrix W, to a random vector, x.

Given that y, = Wj. x, equation (1) can be written as :

where C is an MxM long-term covariance matrix of signal mixtures and C is a corresponding matrix of short-term covariances. A long-term covariance CI and a short-term covariance C between an ith and jth mixtures

are defined as :

n =E (-) (-) (4) r C'/= (r-'J-) r

It should be noted that the embodiments advantageously only need to compute C and C once for any given set of signal mixtures and that the terms (x1#-#l#) and (x1#-#l#), can be precomputed using fast convolution operations such as are disclosed in, for example, Eglen, S, Bray, A, and Stone, J V, 1997, Unsupervised discovery of invariances, Network, 8,441-442, the entire contents of which are incorporated herein for all purposes.

A gradient ascent on F with respect to Wi can be used to maximise F, thereby maximising the predictability of yi. The derivative of F with respect to Wi is:

An optimisation procedure is applied which consists of iteratively updating W, until a maximum F is located

such that W, =W, +77VWF, where Il is a small constant that is, preferably, 0.01. It will be appreciated that the function F is a ratio of quadratic forms. Therefore, F has exactly one global maximum and exactly one global minimum, with all other critical points being saddle points. Therefore, the gradient ascent is guaranteed to find the global maximum in F.

Once a single source signal has been extracted, the repeated application of the above procedure to a single set of mixtures extracts the same (most predictable) source signal. Therefore, it will be appreciated that embodiments are required to ensure that different source signals are extracted. Preferably, the embodiments are arranged to achieve this aim using an unmixing matrix W obtained as the solutions to a generalised eigenproblem defined as follows: the gradient of F defined in equation 5 is known to be zero at an eigenvalue. Therefore, at an eigenvalue solution

- w, C=C (7) It can be appreciated that equations 6 and 7 have the form of a generalised eigenproblem, where the eigenvalues, A = V/U, provide the solution. Critical points in F correspond to values of Wi that satisfy equation 7 where Wils are the corresponding eigenvectors. The first such eigenvector defines a maximum in F and each of the remaining eigenvectors define saddle points in F. It should be noted that these eigenvectors are orthogonal in the metrics defined by C and C which implies that different Wi's are linearly independent.

Borga M 1998, Learning multidimensional signal processing, Linkoping University, which is incorporated herein for all purposes, discloses a review of generalised eigenproblems.

It will be appreciated by those skilled in the art that such problems typically have scaling characteristics of O (N3), where N is the number of signal mixtures, that is, the speed of the determination of solutions varies with the cube of the number of independent sources. Once the eigenvectors have been obtained, all K signals can be recovered using y = W x, where each row of y corresponds to exactly one extracted signal yi which, as indicated above, represents a scaled version of a corresponding independent source signal Si.

Preferably, an embodiment provides, in circumstances where the number of mixtures, M, is greater than the number of sources, K, for the reduction of M using principal component analysis as is well known within the

art. Principal component analysis is used to reduce the dimensionality of the signal mixtures by discarding eigenvectors of x which have eigenvalues close to zero.

Embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings in which: figure 1 depicts three signal mixtures to be processed by an embodiment of the present invention; figure 2 shows three signals having different probability density functions of three source signals that were used to synthesise the three signal mixtures shown in figure 1; figure 3 illustrates signals corresponding to two male and two female voices where the original source mixed signal is shown by continuous lines and corresponding recovered signals shown as dotted lines; figure 4 shows a flow chart for implementing a method according to an embodiment of the present invention; figure 5 there is shown two signals comprising an

original signal, Si, denoted by the solid line and a convolved signal, xi, denoted by the broken line, for 100 time steps processed according to the present invention; and figure 6 depicts the correlation between an original source signal, S2, and a deconvolved signal, C2- Referring to figure 1 there is shown three mixed signals 100,102 and 104 each comprising 3000 samples (although only the first 1000 samples of each mixture are shown in figure 1). The three signals were derived from the signals 200 shown in Figure 2. The three source

signals, s = {si,) S2, Sg} shown in figure 2 represent a super-Gaussian signal (such as the sound of train whistle) 202, a sub-Gaussian signal, such as, for example, a sign wave, 204 and a sorted Gaussian noise signal 206. Each of the signals 202 to 206 were used to synthesise the mixture signals 100,102 and 104 shown in figure 1 using a random matrix A applied to the set of three signal mixtures given by a x = As.

During the processing of the mixture signals 100 to

104, the K source signals, where K = 3 in the present embodiment, were used to generate M = K mixture signals.

Again M = 3, using a KxK mixing matrix (A) and the M mixture signals were used as input signals for an embodiment. Preferably, each mixture signal was normalised to have zero-mean and unit variance. Each mixing matrix was obtaining using, for example, a random number generator function. The short-term and long-term half-lives defined in equation 2 above were set to hs = 1 and hL = 9000, respectively. Referring again to figure 2, the signals shown by dotted lines, that is, signals 208, 210 and 212 represent amplitude or dc shifted recovered signals which, as can be seen from figure 2, correlate strongly with the original source signals notwithstanding having been mixed by a random matrix A. Table 1 below shows the correlation between the source signals and the signals recovered from the mixtures of the source signals with different probability density functions. Table 1

Source 1 Source 2 Source 3 Recovered 1 0. 00030. 00111. 0000

Recovered 2 1.0000 0.0000 0.0000 Recovered 3 0. 0417 0. 9991 0. 0016 It can be appreciated that the three recovered signals each had a correlation of r > 0.99 with only one of the original source signals whereas the other correlations were close to 0. Therefore, embodiments of the present invention can be used to separate mixtures of source signals that have different probability density functions.

An embodiment of the present invention was applied to the separation of sounds where 50,000 data points were sampled at a rate of 44100 Hz, using a microphone to record different voices from a VHF radio using a computer. Two sets of 8 sounds were recorded. The two sets represented voice and music. The voices were a mix of male and female voices. The music was classical, with and without accompanying singing. An embodiment was tested on mixtures of normal speech. Referring to figure 3 there is shown a number of signals 300 which show as complete lines the original source signals 302 to 308 and amplitude or dc shifted versions of the recovered signal 314 to 316 as dotted lines. Table 2 below shows the correlation between each of the four source signal (voices) and every signal recovered from the method. It can be appreciated that each source signal has a high correlation with only one recovered signal. Table 2

Source 1 Source 2 Source 3 Source 4

Recovered 1 0.0973 0.9938 0.0281 0.0488 Recovered 2 0. 9963 0. 0809 0. 0124 0. 0191 Recovered 3 0. 0015 0. 0419 0. 9946 0. 0953 Recovered 40. 02950. 07560. 10140. 9916 Table 3 below shows the correlation between each of eight source signals (voices) and every signal recovered using an embodiment. Again, it can be appreciated that each source signal has a high correlation with only one recovered signal. It can be appreciated that with correlations of this magnitude, it is not possible to discern audibly the difference between the original and the recovered speech signals. It can be seen from Table 3 that a correlation of 0.956 was found between source signal 8 and the recovered signal 3. This, in this example, represents a worse case performance of the method for all data processed according to the abovedescribed embodiment of the present invention. Table 3

Source Source Source Source Source Source Source Source 1 2 3 4 5 6 7 8 Recovered 1 0. 0014 0.0076 0.0037 0. 00260. 02780. 0456 0.9881 0.1430 Recovered 2 0. 9936 0. 0128 0. 0029 0. 0013 0. 0011 0. 0172 0. 0155 0. 1093 Recovered 3 0. 1790 0. 0008 0. 1612 0. 01100. 03660. 1020 0. 13380. 9560 Recovered 4 0. 0150 0.0118 0.0065 0. 99900. 02400. 0317 0. 0044 0. 0044 Recovered 5 0. 0035 0. 0204 0. 9934 0. 0002 0. 0207 0. 0082 0. 0074 0. 1088 Recovered 6 0. 0103 0. 0034 0. 0264 0. 0176 0. 0208 0. 9919 0. 0442 0. 112 Recovered 7 0. 0267 0. 9994 0. 0116 0. 0015 0. 0003 0. 0102 0. 0088 0. 0021 Recovered 80. 01510. 00270. 02690. 02150. 99790. 01970. 02080. 0431 The embodiment was also tested using mixtures of eight segments of music. The correlations between the

source signals of the eight segments of music and the recovered signals for the eight segments of music are shown below in Figure 3. It can be appreciated, again, that the correlations are approximately R = 0.99 and it is not possible to discern audibly the difference between the original and recovered music signals.

It has been found that the embodiments are largely insensitive to the values used for the short-term and long-term half-lives defined in equation 2 above providing the latter is much greater than the former.

It will be appreciated from the above that the method used in the embodiments is based on the assumption that different source signals are associated (via Wi) with distinct critical points in F. However, if any two source signals have the same degree of predictability, F, then two eigenvectors Wi and Wj will also have equal eigenvalues (and are associated with the same critical points in F). Therefore, any vector Wk which lies in the plane defined by Wind j also maximises F, but Wk cannot, in general, be used to extract a source signal. This can be demonstrated by creating two mixtures x = A s from two signals si and Sz, where Si is a time reversed version of S2. Even though Si and S2 have different time courses, they share exactly the same degree of predictability F and cannot be extracted from the mixture x using this method. In practice, however, signals from different sources typically can be separated because each source signal has a unique degree of predictability.

Furthermore, every set of signals in which each signal is from a physically distinct source has been successfully separated using embodiments of the present invention.

Although the above embodiments have been described with reference to processing single dimension data, the present invention is not limited thereto. The method can equally well be utilised to realise embodiments which process spatially distributed or N-dimensional data, that is, the method can be applied to realised embodiments that operate in the spatial domain. The definition of predictability can be generalised by replacing the exponentially awaited means y, and Y, with general functions f and g respectively, where the values of f and g at time T are non-linear functions of y computed up to time 1. These functions may implement more accurate predictions of YT for specific signal types.

It will be appreciated that if all three properties listed above apply to any statistically independent source signals and their mixtures, an embodiment can be realised which relies on constraints from all of these properties to deal with a wide-range of signal types. It is widely acknowledged that ICA forces statistical independence on recovered signals, even in the underlying source signals are not independent. Similarly, the embodiments of the present invention may impose temporal predictability on recovered signals even where none exists in the underlying source signal. Therefore, an embodiment which incorporates constraints from all three properties should be relatively insensitive to violations of the assumptions upon which the embodiments are based.

A framework for incorporating experimentally relevant constraints based on physically realistic properties has been formulated in the form of weak models, see, for example, Porrill, J, Stone, J V, Berwick, J, Mayhew, J,

and Coffey, P, Analysis of Optical Imaging Data using Weak Models and ICA, in : Girolami M, (ED) Advances in Independent Component Analysis, Springer-Verlag, which is incorporated herein by reference for all purposes.

It will be appreciated that an application of the present invention may be in the analysis of medical images and EEG data. Alternatively, the present invention could be applied within a hearing aid context in which a selected signal, voice signal, having predeterminable characteristics can be separated from a mixture of voice signals.

Referring to figure 4, in broad terms, there is shown a flow chart 400 for implementing a method according to an embodiment of the present invention. A measure of predictability, F, is defined at step 410 in terms of covariance matrices and at least one unmixing matrix, Wi. Preferably, the definition relates to the definition of F above. Data samples representing a sampled mixture signal which, in turn, is a combination of a plurality of independent source signals is received at step 420 and, optionally, stored in a memory of a digital signal processor. The expression representing the predictability measurement is optimised to determine local critical points in terms of the unmixing matrix, Wi, and the given data samples at step 430. The eigenvalues and, in turn, eigenvectors, of the optimised predictability measurement are determined at step 440 using, preferably, an optimisation procedure. Step 450 calculates using the above equations an independent signal from at least one of the eigenvalues.

Although the embodiments have been described above in relation to a method of processing digital data samples derived from or representing physical signal measurements, the present invention is not limited thereto. Embodiments of the present invention encompass a digital signal processing system or processor that is arranged to implement the above-described methods as well as computer program elements and computer program products for implementing such methods or comprising software for implementing such methods.

Although the above embodiments have been described with reference to recovering source signals are based on the following conjecture: the temporal predictability of a signal mixture xi is usually less than that of any of the source signals that contribute to xi, the present invention is not limited thereto. Embodiments can be realised in which the continuity predictability of a signal is used as the measure of predictability such that the continuity predictability of a signal mixture is less than that of any of the source signals that contribute to the mixture signal. Accordingly, the present invention is not limited to temporal continuity and encompasses spatial continuity and n-dimensional continuity.

Deconvolution The above embodiments have been described with reference to the separation of blind sources. However, the present invention is not limited thereto. Embodiments can be realised in which deconvolution of signals can be performed.

As will be appreciated by one skilled in the art, a signal can be altered by the effects of the environment in which the signal propagates. For example, a room typically causes speech signals to be heard with attenuated high frequencies and with multiple echoes.

The room effectively acts as a filter on the signal.

Such effects can be reversed using appropriate deconvolution filters. Suitably, embodiments of the present invention are arranged to implement deconvolution filters using the above described temporal predictability.

Firstly, the mathematics underlying the present invention will now be described. Consider a filtered version, x, of a signal, s, such that: x=a*s (1) where a is a filter a = [al, a2,... J. The convolution operator * is defined by : xt = α1st-1+α3st-2+... (2) where the subscript t denotes time. If a deconvolution filter, = [ 1, 2,...], exists for a then y = s can be recovered from x using y= * x (3) However, in practice, the correct deconvolution filter cannot be obtained exactly, so that y # s. Therefore, the embodiments of the present invention use the deconvolution strategies discussed below.

A strategy for Deconvolution The strategy used to estimate the deconvolution filter B will be described for a simple example. Consider an IIR filter, a, which is a low-pass filter:

= (4) where T is the half-life of the exponential. The effects of such IIR filters can be reversed using a simple FIR deconvolution filter: = [1,-] (5) A low-pass filter, such as a, effectively smooths s to yield x = a*s, that is the relatively high frequency components are removed from s. This smoothing operation inevitably makes x more predictable than s. In order to emphasise this, consider the limiting case as T o oo. In the limit, x has a constant value, and is therefore very

predictable. It follows that a deconvolution filter which makes x less predictable may be able to approximately recover s.

Therefore, is defined as that filter which at least reduces, and preferably minimises, a measure of predictability of y =a *x. The differences between the mathematics for deconvolution according to embodiments of the present invention and the above-described blindsource separation is as follows. Given a set of signal mixtures, a linear combination of those mixtures usually

exists such that the original signals can be recovered. It was shown above that such a linear combination could be obtained by maximising the predictability of extracted signals. In contrast, it is shown below that minimising predictability is sufficient to deconvolve a signal.

The convolution operations defined above need to be reformulated in terms of mathematically equivalent vector-matrix operations. Consider a deconvolution filter P = [ 1, 2], such that y = *x. This can be rewritten in vector-matrix notation y = Px by defining a vector variable x as: x= {xlxz' (6) where each row of x is a time-shifted version of x, and the superscript t is the transpose operator. The

shift operator Z-k is defined by Xt-k x, xtz-k (7) The convolution y= *x and the vector matrix multiplication y= x are exactly equivalent.

Measuring signal predictability The definition of signal predictability F used here is :

where y-r = Px, is the value of the signal y at time T, and zizis a vector of K signal mixture values of time T.

The term U, reflects the extent to which y, is predicted by a short-term"moving average"y\ of values in y. In contrast, the term Vi is a measure of the overall variability in y, as measured by the extent to which Y. is predicted by a long-term"moving average" of values in y. The predicted values yr and ## of y, are both exponentially weighted sums of signal values measured up to time (#-1,), such that recent values have a larger

weighting than those in the distant past : =-,) + (I-). -01 =,) + (1-) Y (,) : 0 < I (9) Y ?"2 Z Y (r l) + (I i L) Y (r-l) S AL I (9) The half-life hL of is much longer (typically 100 times longer) than the corresponding half-life of #s. The relationship between a half-life h and the parameter A is defined as 2 It should be noted that minimising only Vi would result in a low variance signal with no constraints on its temporal structure. In contrast, maximising only U would result in a signal with arbitrarily high amplitude. In both cases, trivial solutions would be obtained for/3 because Vi can be minimised by setting the norm of to be zero and U can be maximised by setting the norm of to be large. In contrast, the ratio V1/U1 can be minimised only if two constraints are both satisfied: 1) y has a small non-zero range (i. e. low variance), and 2) the values in y change"quickly"over time relative to hs-It

should also be noted that the value of F is independent of the norm of , so that only change in the direction of affect the value of F. Advantageously, conventional iterative gradient descent on F shows that the length of varies only a little throughout the optimisation process. This is in contrast to the embodiment of the present invention.

Deconvolution Using Temporal Predictability Given that y=px, equation (8) can be re-written as :

where C is an M x M matrix of long-term convariances between signal mixtures, and C is a corresponding matrix of short-term convariances. The long-term covariance C and the short-term covariance Cy between the ith and jth mixtures are defined as :

n c, =E'X-) r c, =i (- -) (ii) r n Cli = E (Xlr-5,,) (xjr-yjr) (11) r

It should be noted that that C and C need only be computed once for a given set of signal mixtures, and that the terms and can be pre-computed using fast convolution operations.

Referring to figure 5 there is shown two signals comprising an original signal, si, denoted by the solid

line and a convolved signal, Xi, denoted by the broken line, for 100 time steps.

Figure 6 depicts a further original signal, S2, denoted by the solid line, and a deconvolved, C2, signal y=px, denoted by a broken line.

Gradient descent on F with respect to could be used to minimise F, thereby minimising the predictability of y. The derivative of F with respect to is:

In an embodiment, one optimising procedure would

consist of iteratively updating/3 until a minimum of F is located : = ?-7/V. F, where 17 is a small constant (typically, 77 = 0. 001).

It should be noted that the function F is a ratio of quadratic forms. Therefore, F has exactly one global maximum and exactly one global minimum, with all other critical points being saddle points. This implies that gradient descent is guaranteed to find the global minimum in F.

Deconvolution as an Eigenproblem The gradient of F is zero at a solution where, from Equation (12):

Critical points of F correspond to values of that satisfy Equation (13), which has the form of a generalised eigenproblem (Borga 1998). Solutions for can therefore be obtained as eigenvectors of the matrix (C-'C), which has corresponding eigenvalues γ=V/U. As noted above, the first and last such eigenvector define a maximum and a minimum in F, respectively, and each of the remaining eigenvectors define saddle points.

It should be noted that eigenproblems have scaling characteristics of O (N3), where N is the number of signal mixtures. The vector can be obtained using a generalised eigenvalue routine.

Given an nth order FIR filter, W is an n x n matrix.

The deconvolved signal can then be recovered by using the smallest eigenvector K in W:y= Kx .W : y= KX.

Deconvolution Results The embodiments of the present invention were demonstrated using 5000 samples of a signal, si, which was the sound of a choir singing, sampled at 8192Hz. The

signal, Si, was convolved with a low-pass filter with n = 32 coefficients < x=c*"T, where T = 8 samples. The resultant convolved signal xi=a*s was used as input to for the embodiments.

Figure 5 displays a short segments 500 of Si and Xi, and the correlation between si and Xi is r = 0. 430. After applying the embodiment described above, the correlation between the deconvolved segments 600, that is, signal

y=/ ? *x and C2 is r = 0. 980 as can be appreciated from figure 6. It can be appreciated from figure 6 it can be seen that there is good agreement between s and the recovered signal y.

It will be appreciated from the above that the flowchart shown in figure 4 is equally applicable to deconvolution with the exception that deconvolution, in contrast to the maximisation operations of the bss problem, attempts minimise predictability.

Source Separation for Image Mixtures Although the above embodiments have been described with reference to processing single dimensional signals, embodiments of the present invention are not limited thereto. Embodiments of the present invention can be adapted for a spatial model, although it remains essentially the same. In the spatial model, an embodiment would maximise a function F =iog (VIU) in which the long-term and short-term variances are spatially

defined as

v = Lcy,,}-y,) 2 and U = LCY', J-Y,,}) 2 (14) =Z (-)' =E.-)' < i4)

where i, j are spatial indices spanning, for example, data representing an extracted image y. The quantities yi, and represent long-range and short-range exponentially weighted spatial means of y, where the weighting is centred at position i, j.

As in the temporal case, V and U can be computed as:

and solution vectors W = (W,..., WK) can be obtained using the same eigen decomposition as was used in the temporal case.

However, in the case of 2D images, the long-range covariance Cl, ; and short-range covariance C,, between the ith and jth image mixtures are :

C =W'-'Vx-x ', 7 ZL ('r, c-Xyr-p, c-q) (r, c"r-p, c-q) I j r, c p, q C"j E I (5rt, c-X Ir-p, c-q r, c r-p, c-q) (16) r, c p, q

The quantitiesC,andC are calculated by convolving each image mixture with an exponentially decaying masks 5 and i-D of different half lives :

r, c=oo r e= ! o r, c=- < IO r, c=-ex > r c=-x r c=-x r, c=-o r, c=-oo

It should be noted that the entries in the masks are rotationally symmetric and decay exponentially as a function of distance from the mask centre at r, c. The - rates of decay in < P and C are defined by the spatial short-range and long-range half-lives, hL and hs, respectively. Recommended values for these are hL= 90000 pixels and hs = 1 pixels.

Therefore, the main difference between the spatial case and the temporal case is the source of contributions to y and y. Whereas the temporal version sums outputs of a single unit over time, the spatial version sums the output over a spatial array.

The extracted images y"..., are recovered from the

set of K image mixtures are : =WjcI ++... + (18) Yi 1 2 2 K K where (w,', ,..., ) are elements of the ith row vector in W.

1 2 K It will be appreciated that even though the above embodiment has been described with reference to processing 2-D data representing images, the present invention is not limited thereto. The 2-D data could equally well represent data other than image data.

The above embodiments make reference to the term "critical points". It will be appreciated that the meaning of this term includes maxima, minima and saddle points, see, for example, Penguin Dictionary of Mathematics, J Daintith, R Nelson, 1989.

It will be appreciated from the above that the blind source separation problem requires maximising predictability which implies maximisation of F which, in turn, implies Wl=Wl+##Wi,F. In contrast, deconvolution requires minimisation of predictability which implies

minimisation of F which implies =-7/VF, that is,

where F is a measure of predictability, it is maximised for BSS and minimised for deconvolution.

It will be appreciated by those skilled in the art that the above inventions and embodiments can be implemented using, for example, a conventional computer and appropriate digital signal processing software.

Alternatively or additionally, a specific hardware platform may be realised for implementing the inventions or embodiments.

The reader's attention is directed to all papers and documents which are filed concurrently with or previous to this specification in connection with this application and which are open to public inspection with this specification, and the contents of all such papers and documents are incorporated herein by reference.

All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and/or all of the steps of any method or process so disclosed, may be combined in any combination, except combinations where at least some of such features and/or steps are mutually exclusive.

Each feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise. Thus, unless expressly stated otherwise, each feature disclosed is one example only of a generic series of equivalent or similar features.

The invention is not restricted to the details of any foregoing embodiments. The invention extends to any novel one, or any novel combination, of the features disclosed in this specification (including any accompanying, abstract and drawings), or to any novel one, or any novel combination, of the steps of any method or process so disclosed.

Claims

CLAIMS 1. A digital signal processing method for deriving a signal from a mixture signal comprising a combination of a plurality of independent digital signals, the method comprising the steps of storing or receiving a first data set representing sampled digital data of the mixture signal; defining a measure of the predictability of the first data set, the measure being a function of the first data set (x) and an first operand (Mi) ; processing the first data set representing digital data of a sampled mixture signal comprising a combination of a plurality of a source signals to produce an output signal (yi) which results from a critical point of the variation of the measure of predictability with the first operand such that the measure of predictability of at least one of the source signals is equal to or greater than the measure of predictability of the sampled mixture signal.
2. A method as claimed in claim 1 wherein the definition of the measure of predictability is

where y, ils the value of the signal y at time T,

Yr is a short-term moving average of the values of y; Yr is a long-term moving average of the values of y; U, the denominator, is a measure of the extent to which Yr is predicted by the short-term moving

average, Yr t of the values of y ; and V, the numerator, is a. measure of the variability of y in terms of the extent to which Yr is predicted by the long-term moving average, YT, of the values of y.
3. A method as claimed in claim 2 wherein the short term moving average of the currently predicted

values of y is given by where 0/Ly < l, where As is arranged to have a first Ih, predetermined half-life, hs, where A, =2-1
4. A method as claimed in either of claims 2 and 3 wherein the long-term moving-average of the currently predicted values of y is given by

Y, =. (,)) + (1-ÀJY (T-l) where 0 < 1, where is arranged to have a second predetermined half-life, h h 1 2-lIhL h, where =2''.
5. A method as claimed in claim 4 wherein the second predetermined half-life is longer than the first predetermined half-life.
6. A method as claimed in claim 5 wherein the second predetermined half-life is at least 100 times longer than the first predetermined half-life.
7. A method as claimed in any preceding claim wherein the definition of the measure of predictability is

W, CWII F =log----, where a scalar signal, Yi, is formed wlcw,

from the application of a lxM matrix, Wl, to a vector variable, x, representing potential or actual linear mixtures of the first data set, C is an MxM longterm covariance matrix of x, C is a short-term covariance matrix of x.
8. A method as claimed in claim 7 in which the longterm covariance, Cl, between the ith and jth

n mixtures of x such that C and r r T r c ~, I (XIT ~ x'r XXir x j). r

9. A method as claimed in either of claims 7 and 8, further comprising the step of locating critical points of F with respect to Wu which gives

2W-2W VF=-LC--C. w, V U
9. A method as claimed in claim 9 further comprising the steps of iteratively optimising F with respect to W,, using W, = W, + 77VWI F, where z has a predeterminable value.
10. A method as claimed in claim 9 further comprising the steps of iteratively optimising F with respect to WIT using W,== ,-VF, where I has a predetermined value.
11. A method as claimed in either of claims 9 and 10 in which the predeterminable value ry is 0.001.
12. A method as claimed in any of claims 7 to 11,

further comprising the step of calculating y, =ex from the value of W, for which F was at a critical point.
13. A method as claimed in claim 12, further comprising the step of outputting a signal based on y.
14. A method as claimed in any of claims 7 to 13, further comprising the step of calculating eigenvectors, Wu for the critical point of F.
15. A method as claimed in claim 14 in which the step of

calculating the eigenvectors, Wu comprises calculating the eigenvalues, =V/U, where VF=0,

- vwhich gives C=-C, which, in terms of the U eigenvalues, A, gives i, C=. C.
16. A digital signal processing method substantially as described herein with reference to and/or as illustrated by the accompanying drawings.
17. A digital signal processing system for deriving a signal from a mixture signal comprising a combination of a plurality of independent digital signals, the system comprising memory for storing a first data set representing sampled digital data of the mixture signal; means for defining a measure of the predictability of the first data set, the measure being a function of the first data set (x) and an first operand (Wi) such that the measure of

predictability of at least one of the source signals is equal to or greater than the measure of predictability of the sampled mixture signal; a processor for processing the first data set representing digital data of a sampled mixture signal comprising a combination of a plurality of a source signals to produce an output signal (yi) which results from critical points of the variation of the measure of predictability with the first operand.
18. A system as claimed in claim 17 wherein the

definition of the measure of predictability is

n F log-=log- ---Y-where yr is the value of the = og-= og U I (y,-,)' r=l

signal y at time T, h is a short-term moving average of the past values of y ; YT is a long-term moving average of the past values of y; U, the denominator, is a measure of the extent to which Yr is predicted by the short-term moving average, YT, of the past values of y; and V, the numerator, is a measure of the variability of y in terms of the extent to which y, is predicted by the long-term moving average, ##, of the past values of y.
19. A system as claimed in claim 18 wherein the short term moving average of the values of y is given by

Yf=sY ( !-l) + (1-) Y ( !- !) where 0 < 1, where As is arranged to have a first predetermined half-life, hs, Ih, where =2''"'.
20. A system as claimed in either of claims 18 and 19 wherein the long-term moving-average of the values

of y is given by = (r-)) + (1-AL) y (r-t) where 0 < 1, where AL is arranged to have a second predetermined half-life, hL, =2''-.
21. A system as claimed in claim 20 wherein the second predetermined half-life is longer than the first predetermined half-life.
22. A system as claimed in claim 21 wherein the second predetermined half-life is at least 100 times longer than the first predetermined half-life.
23. A system as claimed in any of claims 17 to 22 wherein the definition of the measure of

I predictability is F=Iog--=-'-, where a scalar , C/

signal, Yu is formed from the application of a lxM matrix, Wl, to a vector variable, x, representing potential or actual linear mixtures of the first data set, C is an MxM long-term covariance matrix of x, Cis a short-term covariance matrix of x.
24. A system as claimed in claim 23 in which the longterm covariance, C., between the ith and jth

n mixtures of x such that Cy =I --Xr), and r - n r r
25. A system as claimed in either of claims 23 and 24, further comprising means for locating critical points of F with respect to Wi, which gives

VF=-LC--LC. w, V U
26. A system as claimed in claim 25 further comprising means for iteratively optimising F with respect in to Wi, using Wl=Wl+##WiF, where # has a predeterminable value.
27. A system as claimed in claim 25 further comprising means for iteratively optimising F with respect to Wi, using W, =W,-77VWF, wherein T) has a predeterminable value.
28. A system as claimed in either of claims 26 and 27 in which the predeterminable value # is 0.001.
29. A system as claimed in any of claims 17 to 28, further comprising means for calculating y, =, xfrom the value of Wi for which F was at a maxima.
30. A system as claimed in claim 29, further comprising means for outputting a signal based on Yi.
31. A system as claimed in any of claims 17 to 30, further comprising means for calculating eigenvectors, Wi, for the critical point of F.
32. A system as claimed in claim 31 in which the means

for calculating the eigenvectors, Wl, comprises calculating the eigenvalues, ll=VIU, where VF=0,

vwhich gives W, C=-C, which, in terms of the U

eigenvalues, A, gives W, C=/HW, C.
33. A digital signal processing method substantially as described herein with reference to and/or as illustrated by the accompanying drawings.
34. A computer program element for implementing a system or method as claimed in any preceding claim.
35. A computer program product comprising a storage medium having stored thereon a computer program element as claimed in claim 34.