WO2001052552A2

WO2001052552A2 - Video signal processing

Info

Publication number: WO2001052552A2
Application number: PCT/GB2001/000107
Authority: WO
Inventors: Martin Weston
Original assignee: Snell & Wilcox Limited
Priority date: 2000-01-11
Filing date: 2001-01-11
Publication date: 2001-07-19
Also published as: WO2001052552A3; GB2358309A; AU2001225339A1; GB0000546D0

Abstract

Motion between adjacent fields of an interlaced video signal is analysed by vertically interpolating the current field to produce a line signal which corresponds in vertical position with lines from the succeeding and preceding fields; and subtracting the signals to provide a field difference. A measure of detail is taken at each pixel and the field difference signal weighted through multiplication by said detail measure. Noise in the field difference signal is therefore reduced through a reduction in the contribution from plain areas.

Description

VIDEO SIGNAL PROCESSING

This invention relates to video signal processing and in an important example to improved methods of processing television signals which have been derived from cinema film, or other camera processes having a temporal sampling rate lower than the field rate of the television system. The invention also relates more generally to the detection of motion in a video signal.

There are many television processes which involve the interpolation or filtering of the signal by combining weighted sums of picture elements; examples include composite decoding, standards conversion, display upconversion, aspect ratio conversion and geometric transformation for special effects. Television system specifications assume that the temporal sampling rate in the camera is equal to the system field rate (typically about 50 Hz or 60 Hz), however film is usually originated with a temporal sampling rate of 24 Hz and, when film is transferred to television, it is common practice to create more than one video field from the same film frame. This can lead to undesirable artefacts when picture elements from different video frames are combined in an interpolation process.

It is often useful to control such processes so that only frames from the same moment in time are combined and many systems incorporate different operating modes for film-originated material.

WO-A-00/27019 describes how motion between adjacent fields of an interlaced television signal can be analysed to produce field difference signals which indicate whether a consecutive pair of video fields correspond to the same moment in time, and how these signals can be processed to generate a signal to select either the video or the film operating mode of an interpolation process.

It has been found that some video pictures can be wrongly identified as film by these prior art methods. It is therefore an object of certain aspects of this invention to provide improved field difference signals and improved techniques for using these to achieve an even more reliable interpolation control signal. Accordingly the present invention consists in one aspect in a method of analysing motion between adjacent fields of a video signal by vertically interpolating one or both of the fields to produce respective field signals for the two fields which correspond in vertical position; subtracting the field signals to provide a field difference signal; and aggregating the field difference signal over a field; wherein a measure is taken of detail at respective field locations and that the field difference signal from each field location is weighted in said aggregation in dependence upon the detail measure at that location. The weighting has the effect of reducing the contribution made to the field difference determination, by plain areas of the picture. Noise is more likely arise in plain areas so the invention has the effect of increasing the signal to noise ratio of the field difference signal.

Preferably, a measure is taken of detail at each pixel. The detail measure may be a measure of horizontal or vertical detail.

Suitably, the field difference signal from each field location is weighted by multiplication with the absolute detail measure at that location.

In another aspect, the present invention consists in a method of automatically distinguishing between film-originated and video-originated video material in which two field difference signals are used, one between the current field and the previous field and one between the current field and the next field, and where each difference signal is compared with two thresholds.

In yet another aspect, the present invention consists in a method of analysing motion in video material in which two field difference signals are used, one between the current field and the previous field, and one between the current field and the next field, where each difference signal is compared with two thresholds and the results of the comparisons are processed in a logic circuit to generate a flag signal indicating the presence of video material and to generate a flag signal indicating the presence of film material. Advantageously, the flag signals are used in a process having hysteresis to create a control signal to change an interpolation process between a film mode of operation and a video mode of operation.

In still a further aspect, the present invention consists in an interpolation process in which an output is created by taking a weighted sum of contributions from one or more selected input fields; comprising the steps of vertically interpolating one or more input fields to produce respective signals for at least two input fields which correspond in vertical position; subtracting the signals to provide a field difference signal at each of a plurality of field locations; taking a measure of detail at each field location; aggregating the field difference signal over a field for at least one pair of adjacent input fields, with the field difference signal from each field location being weighted in said aggregation in dependence upon the detail measure at that location; and utilising the or each field difference signal to select input fields for interpolation.

In still a further aspect, the present invention consists in a video filter process in which an output is created by taking a weighted sum of contributions from one or more selected input fields; comprising the steps of vertically interpolating one or more input fields to produce respective signals for at least two input fields which correspond in vertical position; subtracting the signals to provide a field difference signal at each of a plurality of field locations; taking a measure of detail at each field location; aggregating the field difference signal over a field for at least one pair of adjacent input fields, with the field difference signal from each field location being weighted in said aggregation in dependence upon the detail measure at that location; and utilising the or each field difference signal to select a filter aperture.

An example of the invention will now be described with reference to the accompanying drawings in which:

Figure 1 shows the generation of field difference and detail signals; Figure 2 shows the correction of the field difference signals for the effects of detail;

Figure 3 shows the derivation of a film/video control signal from the corrected field difference signals;

Figure 4 shows an improved version of the system in Figure 3;

Figure 5 shows the generation of an improved interpolation control signal; and

Figure 6 shows a modification to Figure 1 in which a horizontal rather than a vertical detail measure is generated.

The invention is described here for a 625-line television system, the skilled person will appreciate that the techniques to be described are equally applicable to 525-line systems or any other interlaced format.

Referring to Figure 1 an input video signal (1 ) feeds three cascaded delay units (2), (3) and (4) having a total delay of two fields (i.e. one frame) giving a delayed video signal (5). The outer delay units (2) and (4) have delays substantially equal to one field (312 lines) and the inner delay unit (3) has a delay of one line. If the signals bi and b₂ at the input and output respectively of the delay unit (3) are considered to belong to the current field then the input signal (1 ) represents the next field, and the delayed signal (5) represents the previous field. If the input signal (1 ) is designated c and the delayed signal (5) is designated a then the spatio-temporal arrangement of the signals is as shown below (where the vertical axis represents vertical position and the horizontal axis represents temporal position): a c b₂

The signals bi and b₂ are fed to an interpolator (6) which derives a signal b which is vertically aligned with a and c but corresponding in time with the current field thus: bi a b c b₂ It should be noted that although Figure 1 shows only two lines at the interpolator input, more lines could be used without departing from the scope of the invention.

The interpolated signal b is used to form two field difference signals i and f₂ by performing subtractions with the previous field (a) and the next field (c) respectively in the subtractors (7) and (8) and taking the absolute values of the results in rectifiers (9) and (10).

A detail correction signal D is derived by averaging the signals a and c in the averager (11 ), subtracting the result from the interpolated signal b in the subtractor (12) and taking the absolute value of the result in the rectifier (13). A current-field vertical-detail signal d is derived by subtracting the signals bi and b₂ in the subtractor (14) and taking the absolute value of the result in the rectifier (15).

The two field-difference signals are corrected for the effects of detail and noise as shown in Figure 2, to which reference is now directed. The signals f_it f₂ and D are cored in known manner in coring circuits

(21 ), (22) and (23) so as to remove noise. A suitable algorithm is to remove signals which fall below a fixed threshold t thus:

Score_d = max( - t, 0) The cored detail correction signal is subtracted from each of the cored field difference signals in subtractors (24) and (25) and the two resulting signals are summed over all, or a substantial part of the picture area in summers (26) and (27). So as to further reduce the effect of noise in areas which have little vertical detail, the summed, detail-corrected signals are weighted by multiplying them by the current-field vertical-detail signal d in multipliers (28) and (29). The resulting corrected field difference signals F-i and F₂ can be used to control whether the current field should or should not be combined with the previous field and/or the next field respectively in an interpolation process.

Field difference signals such as Fi and F₂ can be used to generate a mode control signal for an interpolation process which switches the operation between a video mode and a film mode of operation. A novel method of doing this is shown in Figure 3, to which reference is now directed. Previous-field and next-field difference signals (31) and (32), which may be the signals Fi and F₂ from Figure 2, are compared with two DC thresholds T_Vi_de_o and T_fji_m in comparators (33), (34), (35), and (36) to determine whether film or video material is being processed.

Moving video originated material will cause both the signals Fi and F₂ to attain high values and Tvide_o is chosen so that both Fi and F₂ exceed it when moving video is present. Moving film originated material will cause one of the signals F^ or F₂ to assume a low value and Tfji_m is chosen so that such low values are below it. T_f,ι_m is lower than T_Vj_deo so as to prevent ambiguous results. Comparators (33) and (35) determine whether their respective input are below T_fii_m and comparators (34) and (36) determine whether their respective inputs are above T_Vi_deo-

Film is likely to be present when either comparator (33) or (35) gives an output, and this condition is detected by the OR-gate (37), which generates a Film-Flag. Video is likely to be present if both the comparators (34) and (36) give an output, and this condition is detected by the AND-gate (38), which generates a Video-Flag. If there is little motion in the scene neither of these flags will be present and so the latch (39) stores the most recent decision and outputs it as a film/video control signal which may be used to change the operating mode of an interpolation process, either directly or after further processing.

The inventor has further appreciated that the system of Figure 3 may be improved by using the comparator outputs from three consecutive fields as is shown in Figure 4, which shows alternative processing of the outputs from comparators (33), (34), (35) and (36).

Referring to Figure 4, the results of comparing the two field-difference signals with T_Video are fed to an AND-gate (44) and the result is fed to two cascaded field delays (45) and (46). The AND-gate (48) gives an output when both field-difference signals have exceeded T_v-,de_o on three consecutive fields; and this output resets the latch (49) to put the film/video signal into its video state.

The set input of the latch (49) is fed from the AND-gate (47) which gives an output if one of the field-difference signals is below T_fj|_m and was also below it two fields ago, as determined by the output from the cascaded field delays (42) and (43), and also that the other field difference signal was below

T_fiim one field ago, as determined by the field delay (41).

This arrangement makes it unlikely that film mode will be selected incorrectly and, in particular, it prevents shot changes in video material from triggering the film mode. It may be noted that the skilled person will be able to implement the arrangement shown in Figure 4 in various ways, including the use of a state- machine, and that such alternative implementations are within the inventive concept here described. It is a yet further aspect of the invention to control a video interpolation process in response to a combination of a film/video mode control signal, a Film-Flag signal and a Video-Flag signal. This is shown in Figure 5.

A Film-Flag signal (51), which is present when there are similarities between adjacent fields, drives the OR-gate (52). A Video-Flag signal (53), which is present when there are differences between adjacent fields, is inverted (54) and drives one input of the AND-gate (55). A second input of the AND-gate is driven from a film/video mode control signal (56) (active in film mode), which is derived either from an automatic system such as that described above, or from prior knowledge of the material being processed. A second input of the OR-gate (52) is driven by the AND-gate (55) and the output of the OR-gate (52) is used to control a video interpolation process such that adjacent fields are combined when the output signal is active and adjacent fields are not combined when the signal is inactive. Whilst the detail signal d has been described as a vertical difference, it is possible and will in some cases be preferable to employ a measure of horizontal detail, to weight the field difference signal and thereby reduce the effect of plain areas. Thus, Figure 6 shows the location at each side of the line delay (3) of Figure 1 , of four pixel delays (60 - 66). Signals are taken from each side of the delays (60,62) to a subtractor (68). Similarly, signals are taken from each side of the delays (64,66) to a subtractor (70). The outputs from these two subtractors are combined in an adder (72) to provide a measure of horizontal detail dj,. This can be used in the arrangement of Figure 2 as an alternative detail signal d. It should be understood that this invention has been described by way of example only and that a wide variety of modifications are possible without departing from the scope of the invention.

Claims

1. A method of analysing motion between adjacent fields of a video signal by vertically interpolating one or both of the fields to produce respective field signals for the two fields which correspond in vertical position; subtracting the field signals to provide a field difference signal; and aggregating the field difference signal over a field; wherein a measure is taken of detail at respective field locations and that the field difference signal from each field location is weighted in said aggregation in dependence upon the detail measure at that location.

2. A method in accordance with Claim 1 , wherein a measure is taken of detail at each pixel.

3. A method in accordance with Claim 1 , wherein the detail measure is a measure of horizontal detail.

4. A method in accordance with Claim 1 , wherein the detail measure is a measure of vertical detail.

A method in accordance with Claim 1 , wherein the field difference signal from each field location is weighted by multiplication with the absolute detail measure at that location.

6. A method in accordance with Claim 1 , in which two field difference signals are used, one between the current field and the previous field and one between the current field and the next field.

7. A method in accordance with Claim 1 , in which one or more of the vertical detail measure and the field difference signal is cored to reduce the effect of noise in a process which reduces the gain when the amplitude of the respective signal is small.

8. A method of automatically distinguishing between film-originated and video-originated video material in which two field difference signals are used, one between the current field and the previous field and one between the current field and the next field, and where each difference signal is compared with two thresholds.

9. A method of analysing motion in video material in which two field difference signals are used, one between the current field and the previous field, and one between the current field and the next field, where each difference signal is compared with two thresholds and the results of the comparisons are processed in a logic circuit to generate a flag signal indicating the presence of video material and to generate a flag signal indicating the presence of film material.

10. A method according to Claim 9, in which the flag signals are used in a process having hysteresis to create a control signal to change an interpolation process between a film mode of operation and a video mode of operation.

11. A method of control of a video interpolation process by the flag signals of Claim 9; characterised in that flag signals from three successive fields are used.

12. A composite video decoder in which the filtering is controlled in accordance with a field difference signal derived in accordance with Claim 1.

13. An aspect ratio converter in which the interpolation is controlled in accordance with a field difference signal derived in accordance with Claim 1.

14. A standards converter in which the filtering is controlled in accordance with a field difference signal derived in accordance with Claim 1.

15. A video display converter in which the interpolation is controlled in accordance with a field difference signal derived in accordance with Claim 1.

16. A digital video effects processor in which the interpolation is controlled in accordance with a field difference signal derived in accordance with Claim 1.

17. An interpolation process in which an output is created by taking a weighted sum of contributions from one or more selected input fields; comprising the steps of vertically interpolating one or more input fields to produce respective signals for at least two input fields which correspond in vertical position; subtracting the signals to provide a field difference signal at each of a plurality of field locations; taking a measure of detail at each field location; aggregating the field difference signal over a field for at least one pair of adjacent input fields, with the field difference signal from each field location being weighted in said aggregation in dependence upon the detail measure at that location; and utilising the or each field difference signal to select input fields for interpolation.

18. A process according to Claim 17, in which field difference signals are provided for the pair of input fields comprising a current input field and a preceding input field and the pair of input fields comprising the current input field and a succeeding input field.

19. A process according to Claim 17, in which a component in the or each field difference signal which arises from vertical detail is removed by taking a measure of vertical detail from one or more input fields and subtracting either all or a proportion of the detail measure from the or each field difference signal.

20. A video filter process in which an output is created by taking a weighted sum of contributions from one or more selected input fields; comprising the steps of vertically interpolating one or more input fields to produce respective signals for at least two input fields which correspond in vertical position; subtracting the signals to provide a field difference signal at each of a plurality of field locations; taking a measure of detail at each field location; aggregating the field difference signal over a field for at least one pair of adjacent input fields, with the field difference signal from each field location being weighted in said aggregation in dependence upon the detail measure at that location; and utilising the or each field difference signal to select a filter aperture.

21. A process according to Claim 20, in which field difference signals are provided for the pair of input fields comprising a current input field and a preceding input field and the pair of input fields comprising the current input field and a succeeding input field.

22. A process according to Claim 20, in which a component in the or each field difference signal which arises from vertical detail is removed by taking a measure of vertical detail from one or more input fields and subtracting either all or a proportion of the detail measure from the or each field difference signal.