GB2358309A

GB2358309A - Analysing motion between adjacent fields using weighted field difference

Info

Publication number: GB2358309A
Application number: GB0000546A
Authority: GB
Inventors: Martin Weston
Original assignee: Snell and Wilcox Ltd
Current assignee: Snell Advanced Media Ltd
Priority date: 2000-01-11
Filing date: 2000-01-11
Publication date: 2001-07-18
Also published as: GB0000546D0; WO2001052552A2; WO2001052552A3; AU2001225339A1

Abstract

Motion between adjacent fields of an interlaced video signal is analysed by vertically interpolating the current field to produce a line signal which corresponds in vertical position with lines from the succeeding and preceding fields; and subtracting the signals to provide a field difference signal. A measure of detail is taken at each pixel and the field difference signal weighted through multiplication by said detail measure. A method of distinguishing between video and film originated material is also disclosed wherein two field difference signals are compared with two thresholds.

Description

2358309 - 1 VIDEO SIGNAL PROCESSING This invention concerns the control of

video interpolation processes so as to prevent artefacts causes by temporal sampling at a rate other than that specified in the relevant television system specification. It is particularly

concerned with the identification of material which has been originated on film at a frame rate lower than the television field rate.

There are many television processes which involve the interpolation or filtering of the signal by combining weighted sums of picture elements; examples include composite decoding, standards conversion, display upconversion, aspect ratio conversion and geometric transformation for special effects. Television system specifications assume that the temporal sampling rate in the camera is equal to the system field rate (typically about 50 Hz or 60 Hz), however film is usually originated with a temporal sampling rate of 24 Hz and, when film is transferred to television, it is common practice to create more than one video field from the same film frame. This can lead to undesirable artefacts when picture elements from different video frames are combined in an interpolation process.

It is often useful to control such processes so that only frames from the same moment in time are combined and many systems incorporate different operating modes for film-originated material.

UK Patent Application No. GB 9824061.7 describes how motion between adjacent fields of an interlaced television signal can be analysed to produce field difference signals which indicate whether a consecutive pair of video fields correspond to the same moment in time, and how these signals can be processed to generate a signal to select either the video or the film operating mode of an interpolation process.

It has been found that some highly-detailed video pictures can be wrongly identified as film by these prior art methods and that improved field difference signals can be obtained and these can be used in a novel way to achieve an even more reliable interpolation control signal.

Accordingly the present invention consists in one aspect of a method of analysing motion between adjacent fields of a video signal by vertically interpolating one or both of the fields to produce respective signals for the two fields which correspond in vertical position and subtracting the signals to provide a field difference signal; characterised in that the field difference signal is weighted in dependence upon a measure of vertical detail.

Preferably two field difference signals are used, one between the current field and the previous field and one between the current field and the next field.

Suitably one or more of the detail measure and the field difference signals is cored in a process which reduces the gain when the amplitude of the respective signal is small so as to reduce the effect of noise.

In another aspect at least one field difference signal between two adjacent fields is compared with more than one threshold so as to derive flag signals indicating the presence of film material when adjacent fields are substantially similar or video material when adjacent fields are substantially different.

Preferably the flag signals are used in a process having hysteresis to create a control signal to change an interpolation process between a film mode of operation and a video mode of operation.

Suitably the derivation of the interpolation control signal uses information from three successive fields.

An example of the invention will now be described with reference to the accompanying drawings in which:

Figure 1 shows the generation of field difference and detail signals; Figure 2 shows the correction of the field difference signals for the effects of detail; Figure 3 shows the derivation of a film/video control signal from the corrected field difference signals;

Figure 4 shows an improved version of the system in Figure 3; and Figure 5 shows the generation of an improved interpolation control signal.

The invention is described here for a 625-line television system, the skilled person will appreciate that the techniques to be described are equally applicable to 525-line systems or any other interlaced format.

Referring to Figure 1 an input video signal (1) feeds three cascaded delay units (2), (3) and (4) having a total delay of two fields (i.e. one frame) giving a delayed video signal (5). The outer delay units (2) and (4) have delays substantially equal to one field (312 lines) and the inner delay unit (3) has a delay of one line. If the signals bl and b2 at the input and output respectively of the delay unit (3) are considered to belong to the current field then the input signal (1) represents the next field, and the delayed signal (5) represents the previous field. If the input signal (1) is designated c and the delayed signal (5) is designated a then the spatio-temporal arrangement of the signals is as shown below (where the vertical axis represents vertical position and the horizontal axis represents temporal position):

bi a c b2 The signals bl and b2 are fed to an interpolator (6) which derives a signal b which is vertically aligned with a and c but corresponding in time with the current field thus:

bi a b c b2 It should be noted that although Figure 1 shows only two lines at the interpolator input, more lines could be used without departing from the scope of the invention.

The interpolated signal b is used to form two field difference signals fl and f2 by performing subtractions with the previous field (a) and the next field (c) respectively in the subtractors (7) and (8) and taking the absolute values of the results in rectifiers (9) and (10).

A detail correction signal D is derived by averaging the signals a and c in the averager (11), subtracting the result from the interpolated signal b in the subtractor (12) and taking the absolute value of the result in the rectifier (13).

A current-field vertical-detail signal d is derived by subtracting the signals bi and b2 in the subtractor (114).and taking the absolute value of the result in the rectifier (15).

The two field-difference signals are corrected for the effects of detail and noise as shown in Figure 2, to which reference is now directed.

The signals fl, f2 and D are cored in known manner in coring circuits (21), (22) and (23) so as to remove noise. A suitable algorithm is to remove signals which fall below a fixed threshold t thus:

Xcored " max(x - lt, 0) The cored detail correction signal is subtracted from each of the cored field difference signals in subtractors (24) and (25) and the two resulting signals are summed over all, or a substantial part of the picture area in summers (26) and (27). So as to further reduce the effect of noise in areas which have little vertical detail, the summed, detail-corrected signals are weighted by multiplying them by the current-field vertical-detail signal d in multipliers (28) and (29). The resulting corrected field difference signals F, and F2 can be used to control whether the current field should or should not be combined with the previous field andlor the next field respectively in an interpolation process.

Field difference signals such as F, and F2 can be used to generate a mode control signal for an interpolation process which switches the operation between a video mode and a film mode of operation. A novel method of doing this is shown in Figure 3, to which reference is now directed.

Previous-field and next-field difference signals (31) and (32), which may be the signals F, and F2 from Figure 2, are compared with two DC thresholds T,ideo and Tfilm in comparators (33), (34), (35), and (36) to determine whether film or video material is being processed.

Moving video originated material will cause both the signals F, and F2 to attain high values and Tvideo is chosen so that both F, and F2 exceed it when moving video is present. Moving tilm originated material will cause one of the signals F, or F2 to assume a low value and TfIm is chosen so that such low values are below it. Tf.,l.. is lower than Tvideo so as to prevent ambiguous results.

Comparators (33) and (35) determine whether their respective input are below Tf.1,, and comparators (34) and (36) determine whether their respective inputs are above Tvideo.

Film is likely to be present when either comparator (33) or (35) gives an output, and this condition is detected by the OR-gate (37), which generates a Film-Flag. Video is likely to be present if both the comparators (34) and (36) give an output, and this condition is detected by the AND-gate (38), which generates a Video-Flag. If there is little motion in the scene neither of these flags will be present and so the latch (39) stores the most recent decision and outputs it as a film/video control signal which may be used to change the operating mode of an interpolation process, either directly or after further processing.

The inventor has further appreciated that the system of Figure 3 may be improved by using the comparator outputs from three consecutive fields as is shown in Figure 4, which shows alternative processing of the outputs from comparators (33), (34), (35) and (36).

Referring to Figure 4, the results of comparing the two field-difference signals with Tvideo are fed to an AND-gate (44) and the result is fed to two cascaded field delays (45) and (46). The AND-gate (48) gives an output when both field-difference signals have exceeded Tvideo on three consecutive fields; and this output resets the latch (49) to put the film/video signal into its video state.

The set input of the latch (49) is fed from the AND-gate (47) which gives an output if one of the field-difference signals is below Tfil,', and was also below it two fields ago, as determined by the output from the cascaded field delays (42) and (43), and also that the other field difference signal was below Tf,1,, one field ago, as determined by the field delay (41).

This arrangement makes it unlikely that film mode will be selected incorrectly and, in particular, it prevents shot changes in video material from triggering the film mode.

It may be noted that the skilled person will be able to implement the arrangement shown in Figure 4 in various ways, including the use of a statemachine, and that such alternative implementations are within the inventive concept here described.

It is a yet further aspect of the invention to control a video interpolation process in response to a combination of a film/video mode control signal, a Film-Flag signal and a Video-Flag signal. This is shown in Figure 5.

A Film-Flag signal (51), which is present when there are similarities between adjacent fields, drives the OR-gate (52). A Video-Flag signal (53), which is present when there are differences between adjacent fields, is inverted (54) and drives one input of the AND-gate (55). A second input of the AND-gate is driven from a film/video mode control signal (56) (active in film mode), which is derived either from an automatic system such as that described above, or from prior knowledge of the material being processed. A second input of the OR-gate (52) is driven by the ANDgate (55) and the output of the OR-gate (52) is used to control a video interpolation process such that adjacent fields are combined when the output signal is active and adjacent fields are not combined when the signal is inactive.

Claims

A method of analysing motion between adjacent fields of a video signal by vertically interpolating one or both of the fields to produce respective signals for the two fields which correspond in vertical position and subtracting the signals to provide a field difference signal; characterised in that the field difference signal is weighted in dependence upon a measure of vertical detail.
2. A method in accordance with Claim 1 in which two field difference signals are used, one between the current field and the previous field and one between the current field and the next field.
3. A method in accordance with Claim 1 or Claim 2 in which one or more of the vertical detail measure and the field difference signal(s) is cored to reduce the effect of noise in a process which reduces the gain when the amplitude of the respective signal is small.
4. A method of automatically distinguishing between film-originated and video-originated video material in which two field difference signals are used, one between the current field and the previous field and one between the current field and the next field, and where each difference signal is compared with two thresholds.
5. A method of analysing motion in video material in which two field difference signals are used, one between the current field and the previous field, and one between the current field and the next field, where each difference signal is compared with two thresholds and the results of the comparisons are processed in a logic circuit to generate a flag signal indicating the presence of video material and to generate a flag signal indicating the presence of film material.
6. A method according to Claim 5 in which the flag signals are used in a process having hysteresis to create a control signal to change an interpolation process between a film mode of operation and a video mode of operation.
7. A method of control of a video interpolation process by the flag signals of Claim 5 characterised in that flag signals from three successive fields are used.
8. A composite video decoder in which the filtering is controlled in accordance with any of the preceding claims 1-6.
9. An aspect ratio converter in which the interpolation is controlled in accordance with any of the preceding claims 1-6.
10. A standards converter in which the filtering is controlled in accordance with any of the preceding claims 1-6.
11. A video display converter in which the interpolation is controlled in accordance with any of the preceding claims 1-6.
12. A digital video effects processor in which the interpolation is controlled in accordance with any of the preceding claims 1-6.