GB2265782A - Films originating video signal standards conversion - Google Patents

Abstract

A digital television standards converter for use with filmoriginating video signals on the NTSC 525160 standard is provided with a sequencer for detecting the repetitive sequence of duplicate fields introduced on conversion of a 24Hz film to the NTSC standard (3:2 sequence). Adjacent video fields are compared with each other so as to detect the position of duplicate fields (high correlation) and scene changes (low correlation). During conversion from NTSC to PAL, the duplicate fields are removed and the remaining fields reordered so as to allow a field rate conversion from 60Hz to 48Hz. This can be increased by recording on a VTR and replaying at a speed 4% higher than the recording speed or by adding 1 field per 12 field sequence. The position of a scene change is required so as to begin the field reordering/removal sequence immediately after the cut. <IMAGE>

Description

DIGITAL TELEVISION STANDARDS CONVERSION This invention relates to digital television standards conversion and is concerned, for example, with apparatus and processes for converting an NTSC 525 line/60 fields per second video signal to the PAL 625 line/50 fields per second standard.

Techniques exist for conducting the necessary field and line interpolation to convert from one standard to the other and these techniques operate well with "true" video signals. A difficulty arises, however, with video signals that derive from film material.

It is common practice, particularly in the United States of America, to transfer film shot at 24 frames per second to video tapes on the NTSC standard. Each film frame is scanned twice to produce two interlaced fields and a system known as "3/2 pull down" is used to produce the necessary 30 video frames from the available 24 film frames. The system operates by duplicating selected film fields to produce the necessary additional frames in the video signal. If material produced in this way is passed through a known standards converter to produce a 625/50 signal, the output is degraded as a result of temporal distortions of the duplicated fields. The degree of perceptible degradation will vary according to the visual subject matter of the film and will in many cases be significant.

It is one object of this invention to provide a television standards conversion process for use with a film-originating video signal which overcomes or considerably reduces the importance of this difficulty.

In one known arrangement, 12 duplicated fields per second are detected and eliminated from a 30 frame per second video signal originating from a 24 frame per second film. Line interpolation is conducted as necessary, in accordance with known line interpolation techniques. It will be understood that a 24 frame video signal produced in this manner and converted, for example, to 625 lines, has fields corresponding one to one with the original film material. A 625 lines/48 fields video tape produced in this way can be reproduced directly on a 625/50 television set using an adapted VTR. In circumstances where the use of an adapted VTR is impractical, the addition of fields has been contemplated to produce a conventional 625/50 signal. This comprises the deliberate inclusion of certain duplicate fields according to a predetermined sequence.

The ability to produce high quality PAL (for example) video from film originating NTSC video is of considerable commercial importance.

Up until the mid eighties, film was the universal medium for television programme makers wishing to reach an international market. Both 1 6mm and 35mm formats were accepted worldwide as readily transmittable on any television standard. The high cost of film stocl < and film processing has more recently led to an increased use of video tape for post-production work and editing. It is still usual for programmes and features to be shot on film, but this is transferred immediately to NTSC video tape. When a television programme produced in this way is eventually broadcast under the NTSC standard, there is no difference in picture quality.If, however, the video tape is required to undergo standards conversion prior to broadcasting, there has until now been the likelihood - as mentioned - of significant picture degradation. This has meant that the flexibility and economy of post production and distribution on video, have not been available to producers with overseas marl < ets unless they and their customers have been prepared to accept a drop in picture quality. The present invention enables full advantage to be tal < en of video tape with quality copies still being available in PAL or other standards for marl < ets outside the USA.

It will be recognised that known techniques for creating video signals from film material, and principally the 3/2 pull down process, generate a pattern of duplicated and reordered fields in the video signal.

This pattern will generally repeat after five frames. Using the process and apparatus of the invention described above, the duplicate fields are identified and the "phase" or starting point of the 3/2 process is determined. With this information, the conversion processor can eliminate the duplicate fields and reorder the remaining fields as appropriate. Wherever there is editing of the video signal, the 3/2 sequence of duplication and reordering is of course disturbed. Until the processor is able once again to identify the phase of the 3/2 sequence, there is a risle of visually perceptible distortion being introduced into the video signal. With increasing reliance on video based post production techniques, what is referred to as "film-originating video" can be expected to include increasing amounts of video or hybrid material.It will be a requirement for a standards conversion process to maintain optimum performance despite these variations in input.

Standards conversion processes can, it is already recognised, be improved by techniques of motion estimation. A particularly appropriate technique of motion estimation is phase correlation as described in BBC Research Department Report BBC RD 1987/11.

The present invention consists in one aspect in a digital television standards converter for use with film-originating video signals on a 525/60 standard, comprising means for comparing fields in successive frames; a multi-field delay line downstream of said means for comparing fields; sequencer means for analysis of successive outputs of said means for comparing fields and providing an output indicative of the sequence of duplicate fields in said video signals and motion estimating standards converter means operating in response to the sequencer output to provide a video signal on a 625/50 standard.

It will be understood that the introduction of a delay, preferably of eight fields, before the conversion process takes place gives sufficient time for the motion detector and analyser to identify the 3/2 sequence even where this is disrupted by an editing cut.

In a practical situation, the field comparison or motion detector will not produce a precise zero result for all duplicate fields added by the 3/2 process and a non-zero result for all other fields. There will always be error in the field comparison and the video signal will often contain successive "true" fields which show no movement. The present invention therefore proposes, when conducting the field comparison or motion detection, to produce a numerical value indicative of the difference between the two fields rather than a simple motion/no motion flag. With these comparison results available over a range of fields, the processor has considerably more information available to it and it therefore able more accurately to identify the 3/2 sequence even where there is occasional ambiguity as to whether a particular field is or is not a duplicate added by the 3/2 process.

The sequencer output can be tal < en to the input of the phase correlation or other motion estimator, in which case no motion estimation will be made as between fields known to correspond with a single film frame.

Alternatively the sequencer output can be taken to the output of the phase correlator, modifying the interpretation of the estimated motion.

The 24 "intermediate" video frames generated through decoding of the 3/2 sequence will typically comprise fields of 262 lines. (Whilst it is helpful to refer to these "intermediate" video frames, the sl < illed man will recognise that there need not necessarily exist an intermediate video signal at any stage of the process.) Line interpolation will be carried out on the intermediate video frames to produce PAL video fields of 312 lines. In the normal situation, with each intermediate video frame consisting of two 262 line fields generated through scanning of the same film frame, it is preferred to treat each intermediate video frame as a single entity, line interpolated to produce two 312 line fields.Thus, the assumption that both fields of the intermediate video frame correspond with the same image, enables the use of a particularly accurate line interpolation procedure. This may be contrasted with standards conversion of true video signals where the correspondence between two fields of a single frame may be no greater than between fields of successive frames. Thus, in line interpolation of a true video signal, there is generally insufficient information in two fields and the line interpolator will typically use information from four or more successive fields. There is accordingly a compromise between spatial and temporal resolution. In the novel method described above, there need be no such compromise. The intermediate video frames may correspond one for one with the original film frames and line interpolation is conducted on essentially a single film frame.

The necessary field interpolation is conducted to convert from 48 fields per second remaining after deletion of duplicate fields to 50 fields per second of PAL. This removes the need for modified VTR's and preserves this length. The field interpolation process can be regarded as a motion estimated conversion from NTSC 60 fields per second to PAL 50 fields per second with the benefit of the additional information of the 3:2.sequence. It will also be possible to tal < e account of the fact that in film originating video, two fields (neglecting duplicates) correspond with a single instant of time. By contrast the two fields of one frame of time video will encompass a discrete time interval.

It has been recognised by the present inventors that the assumption of both fields of an intermediate video frame corresponding with the same film image, may occasionally breal < down if there has been editing of the NTSC video.

it has already been explained that through the appropriate use of field delays, it can be arranged that in the event of a cut in the NTSC video signal, the updated phase information can be available in sufficient time for correct decoding to be re-established quicl < ly after the cut. Preferably, decoding on the new phase information should start immediately after the cut.

Also, steps should be tal < en to ensure that any "artificial" intermediate video frames which consist not of odd and even scans of the same film frame but of the last field in one edited tape sequence and the first field in the succeeding sequence, are identified as such and processed accordingly. It will be understood that since the line interpolation algorithm assumes both fields of the intermediate video frame to represent the same image, the application of this algorithm to a pair of fields which are adjacent only through the coincidence of editing, may produce visually disturbing artefacts.

Accordingly, sequencer means in a converter according to the present invention preferably further comprises cut detector means for detecting cuts in the film originating video signal through a high degree of difference in said compared fields and wherein said converter means is adapted to apply fresh phase information resulting from any such cut at a point in the video signal immediately after said cut.

The invention will now be described by way of examples with reference to the accompanying drawings in which: FIGURE 1 represents schematically the known "3/2 pull down" process for producing a 60 field per second video signal by scanning a film shot at 24 frames per second; FIGURE 2 illustrates schematically a process according to the present invention; FIGURE 3 is a diagrammatic representation of a comparison step according to the present invention; FIGURE 4 illustrates one portion of apparatus according to one embodiment of the present invention; FIGURE 5 is a diagram corresponding to Figure 2 and illustrating a modification to the process of the present invention; FIGURE 6 illustrates one portion of apparatus according to a further embodiment of the present invention;; FIGURE 7 is a plot illustrating the results of a field comparison performed in one embodiment of the present invention; and FIGURE 8 is a flow diagram illustrating a further process according to the present invention.

The conventional "3/2 pull down" system is illustrated in Figure 1.

Each film frame is scanned twice to produce interlaced fields. Thus film frame A is scanned to produce field Al of odd lines and field A2 of even lines. With alternate film frames, a field is duplicated and used with the appropriate field from the next succeeding film frame to produce an additional video frame. Thus, looking at Figure 1, film field B1 is duplicated to produce video fields B1 and B1*. Video field B1 interleaves with corresponding field B2 whilst duplicated video field B1* is matched with field C2. Video field C1 is matched with video field D2 whilst video field D1 is matched with a duplicate field D2*.

It will be seen that four film frames, A, B, C and D, have been utilised to produce five interlaced TV frames, A1:A2, B1:B2, B1*:C2, C1:D2, and Dl:D2*. In this way, a 60 field per second (30 frame per second) video signal is produced from the 24 frame per second film.

It has previously been described how use of conventional NTSC to PAL conversion techniques on a video signal produced in this way will lead to a degraded image. A conversion process can be regarded as a re-sampling at new field and line rates of a notional signal of which the incoming video signal is presumed to be a sample at the existing line and field rates. With an incoming video signal having duplicated frames, it will be seen that the sampling assumption breaks down and so it is not unexpected that degradation of the image is observed.

The present invention operates by removing the duplicated fields, that is to say the asterisked fields, from the NTSC signal. There is shown in Figure 2, a representation on the centre line, of 30 frames of an NTSC signal. There is also shown on the upper line the origin of these frames using the 3/2 pull down system as described with reference to Figure 1. The present invention operates in one form to produce a 24 frame PAL signal as shown on the lower line. In this diagram, the successive frames are referenced numerically.

It will be seen, for example, that duplicate field lodd is eliminated with field 2-even being recombined with field 2-odd. The next duplicate field 3-even is eliminated and the frames 3-odd, 3-even and 4-odd, 4-even remain unchanged. With the continuation of this process it will be recognised that a 24 frame signal is produced having a field on field match with the original film material.

The apparatus used in one embodiment of the present invention will be described with reference to Figures 3 and 4.

As shown in Figure 4, the incoming video signal is passed, through a frame delay 10, to a comparator 12. This comparator 12 also receives the undelayed video signal. The output of the comparator is integrated in difference integrator 14 which passes an output to digital processor 1 6.

This digital processor also receives the incoming video signal.

In operation, each incoming field is compared with the corresponding field of the preceding frame as shown schematically in Figure 3. In the case of a duplicated field, there will in theory be a zero difference signal integrated over the field and the digital processor is accordingly able to detect duplicated fields. In accordance with the strategy illustrated in Figure 2, one of each pair of duplicated fields is eliminated and the remaining fields arranged, where necessary, to repeat the original field sequence of the film material. For this purpose, the apparatus includes a number of field stores, typically four.

It will be understood that since the duplicated fields are detected electronically on the basis of a lack of motion between succeeding frames, no reliance is placed on a unchanging field structure in the incoming video signal. Where there are inevitable discontinuities in the field structure, due for example to video tape editing or a break in the film scanning process to change spools, the process according to the invention will, within a few number of fields, recommence its detection of duplicate frames. If this introduces a perceived irregularity in the image, additional field delays can be introduced enabling apparatus according to the invention effectively to "lool < ahead" beyond a video tape cut. This option will be discussed more fully below.

It has been suggested to produce an adapted VTR to record the 24 fps PAL signal in such a manner as to permit replay of the video tapes on conventional equipment at 25 fps. There will be an inevitable 4% decrease in running time.

An alternative suggested procedure, illustrated in Figure 5, enables direct viewing of a converted PAL signal on conventional equipment.

The centre and upper lines of Figure 5 are identical with that of Figure 2.

The lower line shows a modification in that an additional field is inserted every 1 2 fields. Thus, duplicate frame 5-odd is in this case not eliminated, neither is duplicate frame 19-even. Further, an additional field is inserted every 1000 fields to allow for the fact that the field rate in the NTSC system is not precisely 60 fields per second but in fact 59.94.

A 25 frame PAL signal produced in this way has minor discontinuities which may be perceptible in certain situations such as smooth pans with no subject movement.

The improvement offered by the present situation is to conduct a motion estimated standards conversion direct to PAL making use of the information contained with the sequencer output as to which input fields correspond to the same film frame, that is to say, the same image.

Referring now to Figure 6, there is illustrated apparatus according to a further embodiment of the present invention. The input video signal is passed through a chain of eight video field delays 20.1 to 20.8 before passing to the converter (not shown). A motion detector 22 receives inputs from the video line and the output of video field delays 20.1 and 20.2 and produces a numerical output indicative of the degree of difference between successive, corresponding fields in The interlaced field structure. The output of motion detector 22 is passed through a chain of motion delay elements 24.1 to 24.6 with the value at each point in the delay chain being made available to a motion analysis unit 26. This motion analysis unit 26 provides an output to the converter. This motion analysis unit is adapted to identify from the delayed motion detection values, the phase of the 3/2 sequence.

In theory, there will be a regular sequence of null motion values over a five field interval. (The skilled man will appreciate that the interval over which the correlation is precise in all respects is in fact five frames, but the sequence is apparent from five fields). As has been discussed above, the detection results are unlikely always to be zero and there will also on occasion be null - or very low - motion detection values arising naturally from the film material. The motion analysis unit 26 is therefore adapted to calculate from the information available to it, the most liltely phase of the 3/2 sequence. That is to say the closest match between the measured values and the theoretical sequence of null values. This information is passed to the converter to enable the duplicate fields to be eliminated, as with the previous embodiment.

Since the motion detection is carried out at a point early in the field delay chain, the motion analysis unit is able to operate several fields in advance of the conversion process. This means that where there is a sequence change, the motion analysis unit is likely to have correctly identified the fresh sequence before fields corresponding to that new sequence "arrive" at the converter. The risle of a loss of resolution at a sequence breal < is therefore very much reduced.

With the additional information available in accordance with this embodiment of the present invention, the converter is able to mal < e additional choices to optimise the conversion process. Thus, for example, the converter may select which of two duplicate fields are to be eliminated. To simplify the re-ordering procedure, for example, it might be convenient at one point in the 3/2 sequence to eliminate the first of a duplicate pair of fields, and at another point in the sequence, the second. In certain cases no field will be eliminated as such, the conversion process will simply tal < e into account the fact that the fields contain duplicated information.

It will be possible to arrange that, in the event of a shift in the 3/2 sequence - typically through an editing cut - the fresh phase information begins to be used at the most liltely point of the editing cut. Preferably, the exact position of the cut is determined so that decoding with the new phase information begins immediately after the cut. Also, steps are preferably tal < en to ensure that a frame having fields from either side of a cut, is not treated in the same manner as a normal decoded frame. It has previously been explained that failing to distinguish such "artificial" video frames can lead to undesirable visual artefacts.

The manner in which these preferred features are provided, will now be described with reference to Figure 7. This plots the results of the motion detection analysis or field comparison that is conducted between corresponding fields of successive frames. It should be noted parenthetically that it may in certain cases be desirable to compare successive fields as well as fields of successive frames, to provide for any video sequences among the film originating material.

Figure 7 shows at 50 the generally smooth envelope of the difference between film frames that would be expected in a typical field sequence. At regular intervals, the plot of field difference will drop sharply towards zero, this being shown for example at 52. As has been explained, this arises from the field duplication in a five frame sequence of the 3/2 procedure. At a video cut, in addition to a phase change (most probably) in the 3/2 sequence, there will be expected a larger than average difference between fields. This is shown for example at 54. Detection of this increased difference signal enables the exact point of the cut to be determined, enabling decoding of the new 3/2 sequence to begin immediately after the cut.Additionally, detection of a cut is used to shift the line interpolation procedure temporarily away from the "normal" mode that has been described, to a "cut" mode. In the cut mode of line interpolation a single field at the cut is employed to construct an entire video frame. Thus a 262 line field is used to provide two 312 line fields. There is an inevitable loss of vertical resolution, but as this appears (necessarily) immediately after a significant picture change, the brief loss of resolution is not visually perceptible.

A particularly preferred form of the present invention will now be described with reference to the diagram which is Figure 8.

The illustrated NTSC to PAL standards converter operates with the recognised 601 digital format but can accept video signals in NTSC, YUV or 601 formats. Thus, a 601 switcher 100 is connected to receive inputs from an NTSC decoder 102 and a YUV decoder 104 as well as to receive 601 signals direct. The output of the 601 switcher 100 is passed to a 16 field delay 106.

A film frame analysis unit 108 receives comparison information from the delay, in the manner described previously. Outputs from the analysis unit 108 are taken to the sequencer 110 and cyclic field store 1 1 2. The sequencer 110 functions as has been described above to identify the 3/2 sequence. This information is available continuously and, because of the delay 106, in real time with the passage of the video signal. Any cuts are identifed and fresh sequence information applied immediately after any cut. The output of the sequencer 110 passes to a motion estimated standards converter 114 utilising phase correlation. This construct digital output fields in the PAL standard, and has modes of operation depending upon whether "video" or "film" input is detected.The converter is also able to deal with edits or other discontinuities which arise other than from true movement in the scene.

The output of the converter 114 is available directly as a 601 output and is also passed to digital-to-analogue converter 11 6. The output of converter 11 6 is made available directly as a YUV output and also passes to PAL encoder 118.

It should be understood that this invention has been described by way of examples only and a variety of further modifications are possible without departing from the scope of the invention. Thus, for example, whilst the use of eight video field delay units is believed to be preferable, a lesser multi-field delay may offer some advantages. Increasing the multi-field delay to more than eight fields would improve the accuracy with which the motion analysis unit can identify the 3/2 sequence although a much increased number of field delays might prove economically impracticable.

Claims (1)

1. A digital television standards converter for use with film-originating video signals on a 525/60 standard, comprising means for comparing fields in successive frames; a multi-field delay line downstream of said means for comparing fields; sequencer means for analysis of successive outputs of said means for comparing fields and providing an output indicative of the sequence of duplicate fields in said video signals and motion estimating standards converter means operating in response to the sequence output to provide a video signal on a 625/50 standard.