GB2488599A

GB2488599A - Adaptive signal processing

Info

Publication number: GB2488599A
Application number: GB1103739.7A
Authority: GB
Inventors: Michael James Knee
Original assignee: Snell Ltd
Current assignee: Snell Advanced Media Ltd
Priority date: 2011-03-04
Filing date: 2011-03-04
Publication date: 2012-09-05
Anticipated expiration: 2031-03-04
Also published as: GB2488599B; US20120226728A1; GB201103739D0; US9002918B2; EP2495874A2

Abstract

In a method for adaptive signal processing (e.g. filtering a sampled input signal), process parameters are adapted to minimize an error between a delayed process output and that of a reference process, such as from a fixed, symmetric filter. A root mean square (RMS) error may be used, with adaptation using the least mean squares (LMS) algorithm. The input signal may describe the position or size of a video frame adaptive window. Also claimed is an apparatus for adaptive signal processing comprising: a reference filter 404 receiving an input signal and providing a reference filter output with a filter delay DR,; an adaptive filter 410 receiving the input signal and providing an adaptive filter output with filter delay DA, less than DR; a unit for delaying the adaptive filter output by a delay corresponding to DR - DA; and an adaption unit 419 measuring the error 418 between the delayed adaptive and reference filter outputs, adapting the adaptive filter in response. Thus, reference and adaptive processes are carried out in parallel, an error between a delayed adaptive output and reference output is measured, and the adaptive process is changed accordingly, minimising the error, e.g. over a time period not less than the delay.

Description

ADAPTIVE SIGNAL PROCESSING

FIELD OF INVENTION

This invention concerns processing of signals in applications where it is important to achieve a low latency or pipeline delay through the process.

BACKGROUND OF THE INVENTION

In real-time sampled-signal processing such as digital filtering, each output sample depends typically on a number of input samples and, in the case of recursive processing such as infinite impulse response (IIR) filtering, additionally on past output samples. In most applications there will be a correspondence in time between input samples and output samples. For example, an event in the input signal that is preserved by the process would be considered to be contemporaneous with the corresponding event in the output signal. In many cases it is desirable that the process has access to input samples both in the past and in the future. However, since a causal system cannot in reality access future samples, the process has to wait until the last required input sample has arrived before it can produce its output. This waiting period is known as the latency or pipeline delay of the process.

The latency of a process is therefore defined as the number of sample periods of delay that elapses between an event at the input to the process and the corresponding event at the output. In general, this delay need not be an integer; for example, the delay of a filter whose output is the average of input signals across a sample-period delay will be 0.5 sample periods. Latency is related to but not in general identical to group delay; however, for a symmetric or linear-phase filter such as the example just given, latency and group delay are identical.

In the description that follows, latency is usually expressed as a whole number and may also be thought of as the number of "future" samples that would have to be made available to the filter in order for there to be no delay between input and output events.

Let us illustrate the property of latency in the case of a linear sampled-signal filter.

Consider a sampled input signal f(t) where t takes integer values. Suppose we wish to perform a filtering operation on this signal where the output depends on surrounding input and output samples: g(t) = G(f(t -n),f(t -n -1),...,f(t + m),g(t -k),g(t -k + 1),...,g(t -i)) where m, n and k are positive integers. In the case of a symmetric finite impulse response (FIR) filter, m = n and k = 0 and there is a direct, unambiguous correspondence between g(t) and f(t). In the more general case, we assume that an input event at a sample labelled I would be manifest at the output at the sample labelled L The output of this filter at any moment thus depends on rn-i-n-i-I input samples, which are considered to extend from n sample periods in the "past" to rn sample periods in the "future", and on k"past" output samples.

This filtering operation may be represented diagrammatically as shown in Figure 1. In this example, n = 4, rn = 3 and k= 5.

Because the output of the filter at time I depends on input samples considered to be up to rn sample periods later than time I, the output cannot actually be calculated until time I-i-rn, as represented in Figure 2, where the samples are aligned with a time axis. We say in this case that the filter has a latency, or pipeline delay, of rn = 3 sample periods.

When designing a filter, there are two conflicting requirements regarding latency.

In many applications, it is desirable to have access to a large number of "future" input samples. For example, if the object is to smooth or low-pass filter the signal, then many input samples are required, and if constant group delay is required, the filter also needs to be symmetric, so about half the samples need to be "future" samples, giving rise to a high latency. However, in real-time processing it is also desirable to minimize the latency so that the filter responds quickly to events at the input.

It is the object of certain aspects of this invention to provide a method or apparatus for real-time signal processing with low latency that retains some or all of the performance advantages available from processes with higher latency.

SUMMARY OF THE INVENTION

The invention consists in one aspect in a method and apparatus for adaptive signal processing in which the parameters of the process are adapted in order to minimize a measure of the error between a delayed version of its output and that of a reference process.

The process of minimizing the error will usefully be smoothed over a time interval which is not less than the delay. Advantages will then for example be secured in applications where variations in the signal which lead to changes in the error occur over a timescale which is shorter than the time interval over which the process of minimizing the error is smoothed The invention consists in another aspect in apparatus for adaptive signal processing comprising: an input terminal for receiving an input signal; a reference filter connected to receive the input signal from the input terminal and configured to provide a reference filter output with a filter delay DR; an adaptive filter connected to receive the input signal from the input terminal and configured to provide an adaptive filter output with a filter delay DA which is less than DR; a delay unit for delaying the adaptive filter output by a delay corresponding to DR -DA; and an adaption unit configured to measure the error between the delayed adaptive filter output and the reference filter output and to adapt the adaptive filter in response to said measured error.

The invention consists in yet another aspect in a method of signal processing to transform an input signal to a processed signal, comprising the steps of conducting a reference process on an input signal; conducting in parallel an adaptive process on said input signal to form the processed signal, said process having controllable parameters; measuring an error between a delayed version of the adaptive process output and the reference process output; and adapting the adaptive process by controlling said parameters to reduce said error.

Suitably, the delay of the adaptive process output serves to bring the adaptive process output into time correspondence with the reference process output.

Preferably, the step of adapting the adaptive process serves to minimize said error over a time period not less than said delay.

BRIEF DESCRIPTION OF THE DRAWINGS

An example of the invention will now be described with reference to the drawings in which: Figure 1 is a diagram of a filter according to the prior ad.

Figure 2 illustrates the latency of a filter according to the prior ad.

Figure 3 illustrates the use of a reference filter according to the invention Figure 4 is a block diagram of a filter according to the invention Figure 5 illustrates the use of a delayed filter output according to the invention Figure 6 is a graph illustrating the performance of the invention on a synthetic data set Figure 7 is a graph illustrating the performance of the invention on a real data set

DETAILED DESCRIPTION OF THE INVENTION

In an adaptive filter according to the invention the coefficients of the filter are adapted in order to minimize a measure of the error between its output and that of a reference filter of higher latency. An illustration of the relative timings of the input signal, the reference filter output and the adaptive filter output is given in Figure 3. In this example, the latency of the reference filter is 5 and the latency of the adaptive filter is 3. A block diagram of a suitable system is shown in Figure 4.

The exemplary process operates on a sampled-signal filter; however the skilled person will appreciate that analogous processing can be carried out on filters operating on non-sampled analogue signals.

Referring to Figure 4, input data (401) is passed through a series of delay elements (402) whose outputs (403) are applied to a reference filter (404). The reference filter (404) consists of a set of coefficients (405) which are multiplied by the respective delayed input signals (403) to form partial sums (406) which are summed in an adder (407) to form the reference filter output (408).

The delayed input signals (403) are also applied to an adaptive filter (410). The adaptive filter (410) has a set of variable coefficients (411) which are multiplied by the respective delayed input signals (403) to form partial sums (412) which are summed in an adder (413) to form the adaptive filter output (414).

The adaptive filter output is also passed through a series of delay elements (415) providing a compensating delay equal to the difference in latency between the reference filter (404) and the adaptive filter (410). The output (416) of the compensating delay is subtracted from the output (408) of the reference filter in subtractor (417) to form an error signal (418) which is applied to an adaptation circuit (419). A further input to the adaptation circuit (419) consists of the outputs (403) of the delay elements (402). The output of the adaptation circuit consists of a set of adjustments (420) which are applied to the adaptive filter coefficients (411).

The detailed operation of an embodiment of the adaptation circuit (419) will now be described. Suitably, the LMS ("least mean squares") algorithm is employed, as described in Adaptive Signal Processing by Bernard Widrow and Samuel D. Stearns, Prentice Hall, 1985, ISBN 0-13-004029-0.

We have an input signal f(t) (401) which passes through an adaptive, instantaneously linear FIR filter G (410) with coefficients w (411) to produce an output g(t) (414), as follows: g(t)= Ewf(t-i) where, in this exemplary embodiment, m = 3 and n = 5, so this filter has a latency of 3 according to the definition given above.

In the known LMS algorithm, we wish to choose coefficients that minimize the long-term error energy E between g(t) and a reference signal h(t): / 2 E = ((g(t)_h()2) = The aim of the LMS algorithm is to adapt the coefficients by steepest descent.

Taking partial derivatives with respect to each of the coefficients, we have = (2ft - -i) -h(t)J) An unbiased estimate of this quantity is 2f(t -i)e(t) where e(t)= Ewf(t-i)-h(t) = g(t)-h(t) i=-m This represents the steepest ascent of the partial derivative, so an iterative adaption by steepest descent leads to the following update formula: w ->w1--2pf(t-i)e(t) This is the classic LMS algorithm, but it is sensitive to the energy of the input f, so in practice a normalized LMS algorithm is used: -pf(t-i)e(t) f(t -i)2 1=-rn for which the optimum value of p is 1.

We may wish to constrain the DC gain of the filter to be a constant value. If we were solving our optimization problem by Lagrange multipliers, we would add a constraint which has the effect of modifying the partial derivatives with a constant value A say. As we wish to maintain the constraint after each iteration, the modification to the original update equation is to add a constant to all the updates such that the gain is maintained: -2pf(t -i)e(t) + 2pe(t) Ef(t -j) in + ii +1 j-rn According to the prior art, the reference signal h(t) could be the output (408) of the reference filter H (404) which has coefficients r1 (405) acting on input signal f(t) as follows: where in this embodiment p = 5 and q = 5, so the reference filter H has a latency of p = 5.

However, a direct attempt to minimize the error between g(t) and h(t) will tend to lead to a filter & with a latency equal to that of H, which in this example is 5 sample periods. The invention seeks to reduce the latency of G by delaying its output before comparison with the output of H. In this example we desire G to have a latency of 3, so we delay its output by 2 sample periods prior to forming the error signal. The LMS algorithm will then be used to minimize the error between h(t) and the output g'(t) of a modified filter G', defined by g'(t)= f(t-i) i=-m' where, in this example = in' = in -2 = 1 = Hi +2 = 7 This filter G' appears to have a latency of 1, but the action of the LMS algorithm will tend to give the filter G' an effective latency of 5, the same as that of the reference filter. The filter G at its output g(t) (414) will then have a latency of 3.

The invention has therefore realised a filter with a latency of 3 whose output has been adapted to be as close as possible to the reference filter of latency 5. The use of the filter G' is illustrated in Figure 5, as a comparison with Figure 3.

Examples of the performance of the invention will now be given.

Figure 6 illustrates the performance of the invention on synthetic input data. The data consists of a low-frequency sinusoid followed by a sharper step function, with added noise. The reference smoothing filter is a 25-tap moving average, which is symmetrical so it has a latency of 12. The goal is to try to match the reference filter using a 15-tap filter with a latency of only 2. The graphs in Figure 6 compare several signals: the unfiltered input (61), the output of the 25-tap smoothing filter (62), the output of an adaptive 15-tap latency-2 filter according to the invention (63), and the output of a non-adaptive 15-tap latency-2 fixed averaging filter (64) for comparison. The normalized version of the LMS algorithm is used.

To begin with, the output of the invention matches the non-adaptive filter to which it is initialized. After about 60 samples it has a much better match to the high-latency filter. It obviously cannot respond as quickly (with respect to output time) as the high-latency filter to the sharper transition after sample 140, but once it has responded it exhibits a much lower time lag than the non-adaptive filter. Overall, there is a 13% improvement in RMS error with respect to the non-adaptive filter.

For the second example, illustrated in Figure 7, the input data comes from the raw output of a "dynamic reframing" process which selects a small window from a video frame for display on a small device such as a mobile phone. The reframing process decides on a frame by frame basis where to centre the window (and what size it should be). In this example we take the horizontal window position for each frame as our input signal (71). This is typically a noisy signal which needs to be smoothed temporally to make the apparent movement of the window acceptable to the viewer. Again, the reference smoothing filter (72) is a 25-frame temporal average with latency 12, but with the additional feature that it does not filter across scene cuts (73). The output of this smoothing filter forms the reference signal which we would like to match using an adaptive filter with a latency of much less than 12 frames. For this example we use the same adaptive filter characteristics as in the first example, namely an overall filter length of 15 taps and a latency of 2. In order to exercise the invention as much as possible, we remove the specific scene cut adaptation from the adaptive filter so that the filter has to try to respond to scene cuts as part of its overall adaptation. The output of the adaptive filter is shown in Figure 7 (74) together with the output of a non-adaptive low-latency filter (75).

The behaviour of the adaptive filter according to the invention can be summarised by saying that it performs comparable smoothing to the high-latency reference filter while being quicker to respond to transitions in the input data.

In the exemplary process first described, the higher latency reference filter is a fixed filter providing a desired output. We have seen in the above example that the reference filter can itself be adaptive -in this case to scene cuts. More generally, the reference filter could be any kind of adaptive filter, for example performing noise reduction on the input signal.

The exemplary process uses the LMS algorithm; however, the skilled person will appreciate that other adaptive algorithms can be used. Likewise, the exemplary process seeks to minimize an RMS error, but the skilled person will appreciate that other error measures can be used, for example mean absolute error. The exemplary process involves linear or instantaneously linear filters, but the skilled person will appreciate that non-linear filters such as rank-order filters or stack filters can be used.

Claims

CLAIMS1. A method for adaptive signal processing in which the parameters of the process are adapted in order to minimize a measure of the error between a delayed version of its output and that of a reference process.
2. A method according to Claim I in which the input signal is a sampled signal.
3. A method according to Claim 2 in which the process is a filtering process.
4. A method according to any of the preceding claims in which the error measure is the root mean square error and in which the adaptation is preferably performed using the LMS algorithm.
5. A method according to any of the preceding Claims in which the reference process is a fixed filter and preferably a symmetric filter.
6. A method according to Claims 2 to 5 in which the input signal describes the position or size of an adaptive window within a video frame.
7. Apparatus for adaptive signal processing comprising: an input terminal for receiving an input signal; a reference filter connected to receive the input signal from the input terminal and configured to provide a reference filter output with a filter delay DR; an adaptive filter connected to receive the input signal from the input terminal and configured to provide an adaptive filter output with a filter delay DA which is less than DR; a delay unit for delaying the adaptive filter output by a delay corresponding to DR -DA; and an adaption unit configured to measure the error between the delayed adaptive filter output and the reference filter output and to adapt the adaptive filter in response to said measured error.
8. Apparatus according to Claims 9 to II in which the error measure is the root mean square error and in which the adaptation is preferably performed using the LMS algorithm.
9. Apparatus according to Claims 9 to 13 in which the reference process is a fixed filter and preferably a symmetric filter.
10. A method of signal processing to transform an input signal to a processed signal, comprising the steps of conducting a reference process on an input signal; conducting in parallel an adaptive process on said input signal to form the processed signal, said process having controllable parameters; measuring an error between a delayed version of the adaptive process output and the reference process output; and adapting the adaptive process by controlling said parameters to reduce said error.
11. A method according to Claim 10, wherein the delay of the adaptive process output serves to bring the adaptive process output into time correspondence with the reference process output.
12. A method according to Claim 10 or Claim 11, wherein said step of adapting the adaptive process serves to minimize said error over a time period not less than said delay.