GB2313011A - Decoding of composite video - Google Patents

Abstract

Composite video is decoded using a motion compensated spatio-temporal separation filter operating on demodulated signals. Interfering luminance derived in one channel, eg. U, is used for cancellation in the other channel eg. V. The filter is divided into forward and backward filters motion compensated using forward and backward motion vectors, respectively.

Description

ENCODING AND DECODING OF COMPOSITE VIDEO This invention relates to the encoding and decoding of composite video and is primarily concerned with decoding.

The decoding of - for example - conventional PAL (ie. PAL that is formed by a simple coder, with no filtering beyond that needed to limit the horizontal chrominance bandwidth) is well known to be very difficult to achieve without luminance-chrominance cross effects. Even the use of picture delays, whilst giving perfect results on stationary pictures, will impair movement, whilst using fixed multi-field filters will result in a compromise between moving and still pictures. Using motion detection, it is possible to switch between, say, filters optimised for spatial resolution with no motion and filters optimised for motion but little spatial detail. However, the result will be sub-optimum in most cases.

It is an object of certain aspects of this invention to provide for improved decoding for pictures which are not stationary.

According to one aspect of the present invention, a method is described of decoding a conventional PAL signal to Y, U and V signals in which the filtering is compensated for motion in the image, according to a signal received from a means of measuring field-to-field displacement of each point in the picture. This means is immaterial for the purposes of the present invention. The method is characterised in that filtering takes place on the demodulated U and V signals, in order that the best possible representation of the original U and V signals entering the PAL coder is obtained. Thereafter, the U and V signals are remodulated and subtracted from the input PAL signal to provide a luminance signal.

In a further aspect the present invention consists in a method of decoding a composite video signal to components, comprising the steps of deriving motion vectors; and separating luminance and chrominance utilising a spatio-temporal separation filter, motion compensated through said motion vectors.

Preferably, said method according to Claim 1, wherein said separation filter operates on a demodulated signal.

Advantageously, said method according to Claim 2, wherein said separation filter derives colour difference signals and wherein luminance is derived by the subtraction of remodulated colour difference signals from the composite video.

Suitably, said a method according to Claim 2 or Claim 3, wherein said separation filter comprises respective quadrature demodulated channels, interfering luminance derived in one channel being utilised for cancellation in the other channel.

In one form of the invention, each method according to Claim 4, wherein each channel in the separation filter comprises a motioncompensated pass filter passing the appropriate chrominance and the interfering luminance and a motion-compensated stop filter passing only the interfering luminance, the output of the stop filter in one channel being used to cancel luminance in the output of the pass filter in the other channel.

Advantageously, the output method according to Claim 5, wherein the output of each stop filter is varied prior to said cancellation in dependence upon the magnitude of the corresponding motion vector.

The invention will now be described by way of example with reference to the accompanying drawings, in which: Figure 1 is a block diagram illustrating a configuration for a PAL decoder; Figure 2 is a general schematic view of the contents of box 1 in Figure 1; Figure 3 illustrates the horizontal temporal spectrum of U and V zero motion; Figure 4 is a graph showing the temporal frequency characteristics of the pass and stop filters of Figure 2; Figure 5 is a block diagram of a configuration of pass and stop filters; Figure 6 illustrates the 3-D frequency characteristics of a pass or stop filter; Figure 7 illustrates, using the same depiction as Figure 3, the horizontal temporal spectrum of U and V for a motion of 0.5 pixels per picture; Figure 8 illustrates, using the same depiction as Figure 6 the 3-D frequency characteristics of pass and stop filters compensated for a motion of 0.5 pixels per picture; Figure 9 is a graph illustrating the change of working point caused by a motion of 0.5 pixels per picture; Figure 10 is a block diagram showing, in one embodiment of the present invention, the contents of box 1 in Figure 1; Figure 11 is a graph showing the cancellation signal waiting employed as WU and WV in Figure 10, as a function of speed; Figure 12 is similar to Figure 7, for a motion of 1 pixel per picture; Figure 13 illustrates at (a) the coefficient pattem and at (b) the vertical temporal frequency characteristic of a field-based stop filter for use in an embodiment of the present invention; Figure 14 is similar to Figure 13 and relates to a field-based pass filter; Figure 15 is a block diagram showing a configuration in one embodiment of this invention of field-based pass and stop filters; Figure 16 illustrates, on three different scales, the vertical frequency characteristic of the U cross-colour for unity weighting, using field-based filters, for a motion of 1 pixel per picture unit; Figure 17 shows effective optimal weighting using the same depiction as Figure 16; Figure 18 is a graph showing the field-based cancellation signal weighting as a function of speed; Figure 19 shows the horizontal-temporal spectrum of U and V for a motion of -3 pixels per picture; Figure 20 is a block diagram of the contents of box 1 in Figure 1 according to a further embodiment of the present invention; Figure 21 illustrates the decomposition of both picture and field filters into forward and backward filters; Figure 22 illustrates the amplitude and phase of Y spectral components in the U channel for a speed of zero for (a) picture filtering and (b) field filtering; and Figure 23 illustrates the inclusion of separate forward and backward weighting to the decomposed forward and backward filters shown in Figure 21.

Figure 1 shows a general block schematic of a base band separation method. This is a complementary arrangement in that everything that is not recognised as chrominance is assumed to be luminance. Operations take place on the demodulated chrominance so as to enable the motion compensation to be free of restrictions that would otherwise be imposed by the presence of the colour subcarrier.

Figure 2 shows a general schematic of the contents box 1. Each of the U and V chrominance channels has a pass filter and a stop filter, all compensated for motion. The pass filter output contains both the appropriate chrominance and the interfering luminance; the stop filter output contains only the interfering luminance, having stopped the appropriate chrominance. The output of the stop filter in one channel is used to cancel the luminance in the output of the pass filter in the other channel. Because of the nature of PAL, the luminance in one channel is in a different spectral place from that of the luminance in the other channel, and because of the quadrature demodulation it is also of a different phase. The phase change must be taken into account by the stop filter, whilst the V axis switch places the luminance in the correct spectral place.

To understand the limitations of this method, we must look at the spectral domain. Figure 3 shows a cross-section of the three-dimensional spectrum, in horizontal-temporal frequency, of both the U and V components after demodulation, for zero motion. The spectra of the Y, U and V are all planar and parallel to the horizontal-vertical frequency plane, the U and V lying at the respective origins. The Y in the U channel has central coordinates (in c/apw, c/aph and Hz) of (23O, 72, 18.75) and +(230, -216, -6.W) whilst the Y in the V channel is centred on #(23O, 216, 625) and #(230, -72, -18.75) The shift of +(0, 144, -12.5) between the two sets of spectra is caused by the V axis switch. Moreover, the U channel Y is in quadrature with the V channel Y. Without loss of generality the Y spectrum can be considered as purely real so that, if the modulated chrominance signal is defined as u sin #t # v cos #t then the spectral Y in the U channel is pure imaginary and the spectral Y in the V channel is real. Because the horizontal component of the subcarrier frequency is less than the horizontal bandwidth of the Y signal, it can be seen that the Y spectra overlap in horizontal frequency by a significant extent, intersecting the temporal frequency axis.

Figure 4 shows the temporal spectral characteristics of pass and stop filters that have been used in a particular form of prior art involving filtering before demodulation. These are realised by taking weights of (1/4, 1/2, 1/4) and (-1/4,0,1/4) across picture delays, and their configuration is shown in Figure 5. It will be noted that the first characteristic is real whereas the second is pure imaginary, thereby effecting the necessary phase shift.

Figure 6 shows the characteristics in horizontal-temporal frequency space. At the frequencies corresponding to stationary luminance, the filter characteristic amplitudes are all -6 dB, so enabling the V stop luminance, after spectral translation by the V axis switch, to cancel the U pass luminance exactly.

Now consider the situation with motion. Figure 7 shows the spectral situation for a horizontal motion of 0.5 (4fsc) pixels per picture. It can be seen that all the spectra tilt, so that the intersections on the pure temporal frequency axis move, in this case by 3.125 Hz.

Figure 8 shows the corresponding frequency characteristics of the filters, compensated by the appropriate amount. As can be seen the effect of the compensation is to skew the characteristics so that the loci of constant value lie parallel to the tilted spectra, intersecting the temporal frequency axis at the same points as before. Whilst compensation of the filters for this motion will still enable the stop filters to reject the appropriate chrominance and keep the characteristics of the passed signals independent of horizontal frequency, this will not be enough, for most applications because the working points on the filters have shifted.

This shift can be deduced by considering what happens along the pure temporal frequency axis, where the characteristics are unchanged but the spectra have moved, and is shown in Figure 9. The points marked U+ correspond to the Y spectra in the U channel centred on positive horizontal frequencies; the points marked U-, to those centred on negative horizontal frequencies, and so also for the V+ and V- points. The spectral component on the pass characteristic at U+ is cancelled by that on the stop characteristic at V+ after V axis switching, and so on. As can be seen, the amplitude of the Y signal from the U pass filter has increased, whereas the amplitude of the Y signal from the V stop filter has decreased so that the two cannot cancel completely. At the same time, although the Y signal from the U stop filter has also decreased by the same amount as that from the V stop filter, the Y signal from the V pass filter has decreased by an even greater amount so that cancellation is again inexact. Now, therefore, the cancellation signals need separate weighting factors, determined by the speed of motion, and the configuration of box 1 becomes as in Figure 10.

It can be seen that, as the speed approaches 1 unit (in 4fsc pixels per picture), the situation becomes untenable as the amplitude of the Y from the V stop filter tends to zero, requiring an ever increasing weighting to cancel the Y in the U channel which tends to merge with the U itself.

Figure 11 shows the weighting needed for the V stop signal as a function of speed. It can be seen that criticality occurs at speeds of 1 and -3 units, the required weighting behaving periodically, with a period of 4 units. The situation for the V chrominance is the mirror image. This means that, at speeds of -3, 1, 5 etc. units, the V signal is still completely recoverable, but the U is not whereas, at speeds of -5, -1, 3 etc. units, the U is recoverable but the V is not.

At these critical speeds, altemative methods of filtering must be used.

Figure 12 shows the spectra for the critical speed of 1 unit. The Y spectra centred on positive and negative horizontal frequencies now coalesce, being perfectly separable from V in the V channel and coincident with U in the U channel. However, the Y spectra in the V channel now fall at the nulls of the stop filter, as previously noted, and, in any case, require conflicting phase changes for the two halves centred on positive and negative horizontal frequency. This conflict cannot be resolved for all vertical frequencies but, by taking advantage of the vertical frequency offset of the positive and negative halves, it can be resolved for low vertical (input) frequencies. This implies a field-based V stop filter, taking contributions from previous and following fields, instead of pictures. Correct operation then requires a field-based U pass filter. At the same time, the Y in the V channel fails at the null of the picture-based V pass filter and so the V is perfectly recoverable with zero contribution from the U channel.

The coefficient pattern of the simplest field-based stop filter that satisfies the requirements for phase at the spectral centres (ie. for zero vertical luminance frequency) is shown in Figure 13, together with its frequency characteristic. Figure 14 shows the coefficient pattern and frequency characteristic of the simplest field-based pass filter and Figure 15 shows the configurations of both pass and stop filters. Using these filters, with a stop signal weighting of unity, the cross-colour characteristic of Figure 16 results. This is plotted with three different horizontal scales to clarify the relationship between input and output frequency. The two mirrored characteristics correspond to the Y spectra centred on positive and negative horizontal frequencies. Using the appropriate scale with the corresponding characteristic, it can be seen that a luminance frequency of, for example, +144 c/aph gives a cross-colour frequency of +216 c/aph whereas a luminance frequency of -144 c/aph gives a cross-colour frequency of +72 c/aph. Complete elimination of cross-colour is obtained only for luminance frequencies of t72 claph whilst full amplitude cross-colour occurs for a luminance frequency of +216 c/aph, which corresponds to the vertical frequency of the subcarrier, giving zero frequency cross-colour. There is a slight overload for luminance frequencies in the region of +288 c/aph whilst there is an attenuation of only 13.7 dB for luminance of zero vertical frequency.

However, by varying the weighting of the stop signal, the characteristic can be tailored to some degree. Whatever weighting is chosen, the characteristic must pass through unity at the origin of crosscolour frequency if the true chrominance has unity gain in uniform areas.

The visibility of the cross-colour depends on its amplitude and frequency. It is no accident that the fine cross-colour is caused by the coarse luminance, which will be statistically of higher amplitude than the fine luminance which causes the coarse cross-colour. It can be argued that since the characteristic must pass though unity at the origin, where the cross-colour is coarsest, it is only profitable to adjust the characteristic at the points where the luminance is coarsest. In other words, it would be better to have complete attenuation of cross-colour caused by luminance of zero vertical frequency, instead of +72 c/aph. This is obtained with a weighting of 22 - 1 for the stop signal and results in the characteristics of Figure 17. As can be seen, the complete attenuation at the luminance frequency of zero and reduction of overload at the frequencies of +288 c/aph is obtained at the expense of decreased attenuation in the region of -72 to -144 c/aph.

As the motion speed alters, the Y spectra fall at different points on the characteristics of Figures 13 and 14 and so, just as with the picture-based filters, the variation in weighting factor that maintains cancellation can be calculated, but now only for zero vertical luminance frequency. This variation is shown in Figure 18. It can be seen that the behaviour now has a periodicity of 8 units, not 4, as with picture-based filtering. However, critical speeds occur at intervals of 4 units, eg. -1, +3, where picture-based filtering is appropriate. As with picture-based filtering, the situation for the V chrominance is the mirror image.

The field mode is appropriate in the U channel for speeds of -7, 1, 9 etc. units, where the vertical frequency offsets of the positive and negative Y spectra, coalescing with the U spectrum are t21 6 c/aph. However, for the picture-based critical speeds of -3,5 etc. units, the situation is not the same because the vertical frequency offsets of the two halves of the Y spectrum is now only +72 c/aph. Figure 19 shows the spectral situation for a speed of -3 units. It turns out that, for these cases, a better cross-colour frequency characteristic is obtained by using line-based pass and stop filters, whose configuration is like that of Figure 5 with line delays replacing picture delays.

The spectral characteristics of these filters, being intra-field, are independent of motion and thus require no variation of stop signal weighting with speed.

Thus the required filtering mode behaviour has a periodicity of 8 units of horizontal motion, the behaviour of the V signal being the mirror of that of the U signal, as shown in Table I.

Speed U Mode V Mode -4 picture picture -3 line picture -2 picture picture -1 picture field o picture picture +1 field picture +2 picture picture +3 picture line +4 . picture picture

Table I - Filtering mode behaviour with speed The existence of alternative filters for critical speeds implies some sort of mode adaptation, dependent on speed. This is best done by cross-fading with a fading region surrounding each critical speed. The extent of this region depends on how much noise can be tolerated in the picture mode, given that the cancellation signals are weighted by higher and higher values as the critical speed is reached. In practice, a region of +0.2 units gives a satisfactory noise performance. The configuration of box 1 therefore becomes as shown in Figure 20.

Thus far, a respectable separation of luminance and chrominance can be demonstrated with a constantly moving section of hyperbolic zone plate, covering the full gamut of vertical frequencies. At the critical speeds, the cross-colour of low vertical frequency is apparent. However, with sinusoidal motion, a problem is caused by inaccuracies in the assumed motion, resulting from acceleration. In such as situation, the cancellation signals are weighted incorrectly.

To overcome this problem, it is necessary to implement a more complicated structure where the two halves of the filters (forward and backward) are compensated by different amounts. Fortunately, it is standard practice to derive such forward and backward motion vectors. The output of the (1/4, 1/2, 1/4) picture-based pass filter can be expressed as the mean of forward and backward (1/2. 1/2) filters whilst the output of the (-1/4, 0, 1/4) picture-based stop filter can be expressed as the mean of forward and backward (-1/2, 1/2) filters, as shown in Figure 21. Similar considerations apply to the field-based filters. Such filters are phasey and this must be taken into account in the analysis. Motion now alters not only the amplitudes of the pass and stop signals, but also their phases.

Figure 22 shows the amplitude and phase of all four pass and stop, advanced and delayed Y spectral components appropriate to the U channel for zero motion, for both picture and field-based filtering. The latter applies to the Y of zero vertical frequency. Taking the picture-based case first, as can be seen, the sum of the components is zero and, in fact, for cancellation, only two signals would be needed. That is, a Y signal from a delayed V stop filter could cancel a Y signal from an advanced U pass filter, or vice versa, but there is a noise advantage in using all four signals. As motion occurs, the complex components follow the circular loci shown, the vectors centred on the circle centres sweeping out angle at the rate of 90 degrees per unit of motion. Thus, the resultant spectral components sweep out angle at 45 degrees per unit, the delayed components remaining mutually perpendicular and the advanced components remaining mutually perpendicular. If the forward and backward motions are equal, then the situation is symmetrical about the imaginary axis and the resultants of the pass and stop signals both lie along it. In this case, the forward and backward weightings are equal, those for the pass signals being 1/2 whilst those for the stop signals follow the relationship of Figure 11, divided by 2.

However, when the forward and backward motions are not equal, neither are the weightings and provision must be made to weight each of the four components separately, as shown in Figure 23. Clearly, such weighting factors are now functions of both forward and backward motion values.

The same remarks apply to the field-based situation of Figure 22 except that the vectors centred on the circle centres sweep out angle at 45 degrees per unit. Thus cancellation at zero motion is again obtained by equal weighting of pass and stop signals but now all four are needed. If forward and backward motions are equal, the pass weighting is again 1/2 and the stop weighting follows the relationship of Figure 9, divided by 2.

In general, the required weightings in both picture and field cases are derived using the following conditions: 1. The sum of the real parts of the weighted spectral components must be zero.

2. The sum of the imaginary parts of the weighed spectral components must be zero.

3. The sum of the two pass weightings must be unity, for unity chrominance gain.

4. The sum of squares of the weightings must be a minimum.

This last condition can be solved using Lagrange's method of undetermined multipliers, subject to the other three conditions which are linear relationships between the weightings. The solution then represents the four weightings as a function of forward and backward speed.

For clarity of description, the example has been taken throughout of PAL decoding. Some advantage is seen in supplementing a PAL encoder with a luminance pre-filter embodying the described features of a decoder so as to remove any luminance components which would lend to artifacts after separation in the decoder. More importantly, it should be understood that the present invention is not restricted to PAL and will apply to other standards. With NTSC, for example, the more straightforward phase relationship of chrominance will simplify the application of the techniques exemplified above.

Claims (10)

1. A method of decoding a composite video signal to components, comprising the steps of deriving motion vectors; and separating luminance and chrominance utilising a spatio-temporal separation filter, motion compensated through said motion vectors.

2. A method according to Claim 1, wherein said separation filter operates on a demodulated signal.

3. A method according to Claim 2, wherein said separation filter derives colour difference signals and wherein luminance is derived by the subtraction of remodulated colour difference signals from the composite video.

4. A method according to Claim 2 or Claim 3, wherein said separation filter comprises respective quadrature demodulated channels, interfering luminance derived in one channel being utilised for cancellation in the other channel.

5. A method according to Claim 4, wherein each channel in the separation filter comprises a motion-compensated pass filter passing the appropriate chrominance and the interfering luminance and a motion-compensated stop filter passing only the interfering luminance, the output of the stop filter in one channel being used to cancel luminance in the output of the pass filter in the other channel.

6. A method according to Claim 5, wherein the output of each stop filter is varied prior to said cancellation in dependence upon the magnitude of the corresponding motion vector.

7. A method according to Claim 6, wherein said variation is a multiplication.

8. A method according to any one of the preceding claims, wherein adaptions are made pixel-by-pixel between picture-based and field based motion-compensated separation, in dependence upon the magnitude of the corresponding motion vector.

9. A method according to Claim 10, wherein adaptions are made pixel by pixel between picture-based motion compensated separation; field-based motion compensated separation and intra-field non motion-compensated separation.

10. A method according to any one of the preceding claims, wherein the motion-compensated spatio-temporal separation filter comprises a forward filter taking information from time present and time future and a backward filter taking information from time present and time past, said filters being motion compensated using forward and backward motion vectors respectively.