GB2503833A

GB2503833A - A method of modifying an input sequence of temporal television signals by sine wave judder

Info

Publication number: GB2503833A
Application number: GB1317546.8A
Authority: GB
Inventors: Martin Weston
Original assignee: Snell Ltd
Current assignee: Snell Advanced Media Ltd
Priority date: 2008-12-09
Filing date: 2008-12-09
Publication date: 2014-01-08
Anticipated expiration: 2028-12-09
Also published as: GB201317546D0; US20100141831A1; JP2010141891A; GB2466044A; GB2466044B; US8773586B2; US8432487B2; GB0822413D0; US20130229573A1; GB2503833B

Abstract

A television signal is modified through a sine wave motion-judder component 502 to introduce intentional motion judder which can replicate the appearance of cinematic projection. The sine function may be introduced by sinusoidal judder oscillator 512. The method may present 24 frames per second film via a 60 Hz television system with the "look" of a 72 Hz cinematic projector, a judder oscillator is set to generate a 24 Hz sinewave having a peak amplitude equal to 38.5% of the time difference between 24 Hz frames. The motion-judder component may periodically reverse the apparent direction of motion. An apparatus for carrying out said method is also disclosed. The temporal interpolation may be carried out via a polyphase FIR filter.

Description

Motion Image Rendering System

FIELD OF iNVENTION

This invention concerns the rendering of motion images. It is particularly applicable to the rendering of 24 frame per second film in 60 field per second television systems.

BACKGROUND OF THE INVENTION

Film, television and similar imaging systems portray moving images by presenting a viewer with a sequence of still images that correspond to temporal samples of a scene.

The rate of presentation of new images and the duration of presentation of each image are crucial factors in determining the subjective quality of the viewing experience.

Historically the rate of preseiltation of new images was identical to the rate of image acquisition in a camera; and, the duration of presentation was limited by ayailable display system technology. In modcm image processing systcms it is possible for these display characteristics to bc dctcrmincd indcpcndcntly of thc paramctcrs of the image acquisition, storage and distribution mechanisms.

Whcrc moving imagcs are acquired by photography thc tcmporal sampling frequency (i.e. the frame rate) is limited by the mechanical processes of film transport. This has lead to thc widcspread adoption of a tcmporal sampling frcqucncy of 24 Hz (framcs pcr sccond). In telcvision thcre is no mcchanical limitation, but it is still necessary to allow sufficient "exposure time" for each image to be acquired by the electronic sensor; temporal sampling frequencies (field rates) of 50 Hz aild 60 Hz are common.

In mechanical film projection the duration of presentation of each image is less than the temporal sampling period (because of the time for film transport); however, because of the relatively-low temporal sampling rate, each image is repeated (flashed) one or more times to make the resulting flicker imperceptible. In telcyision, because of the higher temporal sampling rate, each image is usually only displayed once.

Therc arc thus significant subjcctivc differences between the film and television Yiewing experiences.

Further complications arise whell film-originated material is displayed via television.

When (as in Europe) the television field rate is approximately twice the film frame rate, normal practice is to adjust the film frame rate (with possible change to the duration of the material) to equal thc tclcvisions frame ratc, so that cach film framc is displayed twice (as two consecutive television fields). In countries where the television field rate is approximately 60 Hz (such as the USA) film frames are shown in a 3:2' sequence in which alternate film frames arc shown three times and twice, respectively. This is a compromise which gives an inferior viewing experience.

These methods of presenting film via the television medium thus give a viewing experience that may be substantially different from that intended by the cinematographer and represent, in varying degree, a distortion of the intended viewing experience.

It is common to combine film-originated and television-originated material in the same television presentation. Often, the difference in viewing experience due to the different method of image acquisition represents an undesirable distraction for the viewer. It often preferable for the entire presentation to have either, the film-look' or the television look' according to the intention of the director. In other cases it may be desirable to artificially increase the subjective difference between different parts of the same presentation for artistic reasons.

It is thus advantageous to be able to control the subjective viewing experience of televised material in order to achieve a desired type of motion rendition. Known systems include temporal interpolation (sometimes including motion-compensation) that up-converts from the temporal sampling frequency of the film frames to the temporal sampling frequency of the television system. Usually the up-conversion is to the field rate of an interlaced television system; such conversion may be performed optically, for example in a polygon' telecine in which light from more than one film frame is combined in a rotating optical prism.

However, these prior-art systems do not accurately replicate the motion rendition of film projection in a television or other electronic moving image rendition system.

SUMMARY OF THE INVENTION

The inveiltion consists in a method and apparatus for modi'ing an input sequence of temporal image-samples representing a moving image so as to obtain an output sequence of temporal image-samples in which the appearance of the said motion has bccn changed whcrcin the said input tcmporal image-sampics are tcmporally interpolated to create the said output sequence and an intentional, periodic motion- judder component is introduced that is visible when the said output temporal image-samples are displayed, where the said intentional motion-judder component is determined by a sine function.

Advantageously, the magnitude of the motion-judder component applied to a temporal image-sample is proportional to the instantaneous value of a temporal sinewave at the intended time of display of that temporal imagc-sample.

Preferably, the frequency of the said temporal sinewave is equal to the original temporal sampling frequency of the input temporal image-samples.

BRIEF DESCRIPTION OF THE DRAWINGS

An cxample of the invcntion will now bc described with rcfercncc to the drawings in which: Figurc 1 shows a prior-art tcmporal intcrpolation system.

Figure 2 shows the tcmporal sampling pattern of 24 framc pcr second film presented by a double-flash projector.

Figure 3 shows the temporal sampling pattern of 24 frame per second film prcscnted by a triple-flash projcctor.

Figurc 4 shows thc tcmporal sampling pattcrn of 24 framc pcr second film presented by a 60 field per second, 3:2 pulldown television system.

Figure 5 shows a temporal interpolation system.

Figure 6 shows a temporal sampling pattern of 24 frame per second film presented by a 60 field per second television system.

Figure 7 shows an alternative temporal sampling pattern of 24 frame per second film presented by a 60 field per second television system.

Figure 8 shows a tcmporal sampling pattern of 24 frame pcr second film prcscntcd by a 50 field pcr second telcvision system.

Figures 9, 10 and 11, show enlarged portions of the temporal sampling pattem of Figure 8.

Figure 12 shows a temporal sampling pattern of 25 frarne per second film presented by a 50 field per second television system.

Figure 13 shows a temporal sampling pattern of 50 field per second television modified to resemble 25 frame per second film projection Figure 14 shows a temporal sampling pattern of 60 field per second television modified to resemble 24 frame per second film projection.

Figure 15 shows a bilinear interpolation aperture.

Figure 16 shows a higher-order interpolation aperture

DETAILED DESCRIPTION OF THE INVENTION

An object of embodiments of the invention is to conduct a temporal resampling operation on an input sequence of temporal image-samples, such as film frames or video fields, so as to obtain a modified output sequence of temporal image-samples having a temporal sampling pattern that gives a viewer a particular intended viewing experience. The input temporal pattern will initially be assumed to be regular; i.e. having a fixed temporal sampling frequency. (Irregularly sampled inputs will be considered later in this specification.) The presentation rate of output temporal image-samples is regular, and will usually have some relationship to the input temporal sampling frequency; i.e. they will usually be sub-multiples of a higher common factor.

The output temporal sampling pattern is often different from that represented by the regular presentation frequency of the input temporal image-samples. In other words, the relationship between the output presentation time and the corresponding input acquisition time may vary.

An example of a prior-art system to achieve temporal resampling is shown in Figure 1. The figure assumes a real-time process, but the skilled person will appreciate that other techniques, including software techniques can be used to achieve the same result.

A current temporal image-sample (1) is input to a temporal intcrpolator (2), together with a number of earlier temporal image-samples 3) and a number of later temporal image-samples (4). In some implementations all these samples are derived by applying different delays to a stream of input image-samples. The current input temporal image-sample (1) has associated timing reference information (5) such as a field-or frame-rate clock, or time code. A desired output temporal sampling pattern is generated by an output timing reference generator (6), which produces output sample- request data (7). This data defines the time at which each output temporal image-sample is required for display. A temporal phase comparator (8) compares the required output sample time of the sample-request data (7) with the timing reference information (5) corresponding to the current input temporal image-sample (1). The result of this comparison is a temporal phase signal (9) that defines the timing difference between the required output samplc and the current input sample (1). This timing difference is expressed in units of the input temporal sample period. Usually the temporal phase signal (9) is constrained to lie in the range _1/2 to +1/2.

The temporal interpolator (2) is controlled by the temporal phasc signal (9). Every time an output temporal image-sample is requested, the temporal phase value is used to interpolate between the current input temporal image-sample (1) and a temporally adjacent input temporal image-sample. For example, if the temporal phase value is +,4, an output image will be generated that corresponds to a time half-way between the current input temporal image-sample (1) and the succeeding temporal image-sample. The sequence of interpolated temporal image-samples is output via a terminal (10).

The temporal interpolator (2) can use any known method, including motion compensated temporal interpolation (in which the positions of objects are calculated from motion vectors and pixels are shifted by distances calculated from those vectors), and non-motion-compensated interpolation in which the required temporal phase offset is achieved by adjusting the phase of a polyphase FIR filter. Typically this process will make use of one or more preceding temporal image-samples (3) and/or succeeding temporal image-sample (4).

Typical prior-art systems use the arrangement of Figure 1 to convert film-originated material to television field rates, or to convert between television field rates. Another system for altering the motion rendition of television images is described in US patent 4,935,816, in which television fields are processed so as to replicate the motion pattem of the well-known 3:2 pull-down system. However these prior systems do not produce a viewing experience which replicates cinematic projection. Typical film projectors, employ double-or triple-flash shutters in which each film frame is displayed two or three times, respectively. This gives motion rendition quite different from television.

Figure 2 shows the timing of presentation of 24 Hz-rate film frames by a double flash projector. The horizontal axis represents time, and the vertical axis represents the position of an object moving at a constant velocity. The projector flashes at twice the film frame rate, i.e. at 48Hz; the time of each flash is indicated by the vertical grid in the Figure. The change in position of the moving object having constant velocity is represented by the straight line (21). The positions of the object as captured on successive film frames by a film camera (having a 24 Hz frame rate) are indicated by circles, such as the position (22). As both the movement and the frame rate are constant, the difference in position of the moving object between succeeding frames is constant; this distancc is indicatcd by thc dimcnsion (23). Each film framc is shown twice by the projector; the times of presentation of frames by the projector, and the corresponding object positions are indicated in the figure by crosses. The frame corresponding to the point (22) is projected at the times indicated by the points (24) and (25). The moving object is thus shown twice at the same position. In the next film frame the object has the position (28), and that frame is projected twice, at the positions (26) and (27).

It can be seen that the motion of the object, as seen by a viewer, is no longer uniform.

There is a motion judder error which is represented by the respective vertical distances between each of the points (24) (25) (26) (27) and the line (21). The error varies with time, and the fhndamental frequency of this variation is 24 Hz. A sincwavc at this frcqucncy can be addcd to the positions indicated by the line (21) and its phase and amplitude chosen so that the sum passes through the points corresponding to all the projected images. This sum is indicated in Figure 2 by the curve (29). The phase of the sinewave is 900 retarded with respect to the second presentation of each film frame, and the peak amplitude is 25% of the positional difference (23) between succeeding film frames (as captured by the camera).

The motion judder of a triple-flash projector displaying 24 frame per second film is shown in a similar way in Figure 3. In this case each film frame is shown three times, once early, once at the correct time, and once late. The addition of a 24 Hz sinusoidal judder component to the uniform motion is shown by the curve (39). This sinewave, which represents the motion judder, is phase retarded by 60° with respect to the third presentation of each film frame, and has a peak amplitude of 38.5% of the positional difference (33) between (camera) film frames.

This analysis can be applied to other motion rendition systems. Figure 4 shows the motion of the well-known 3:2 pulldown system in which 24 frame per second film is shown at a television field rate of 60 Hz. In this case the judder cannot be described by a single 24 Hz sincwavc. However thc sum of a 2 Hz sincwavc (peak amplitudc 20% of thc positional difference (43) between film framcs) and a 24 Hz sinewave (peak amplitude 34%) accurately describes thc judder. The sum of these two judder frequency components and the constant motion is the curve (49). The additional 12 Hz judder component (half the frequency of the camera frame rate) makes the motion rendition of thc 3:2 system subjectivcly much worse than for convcntional film projection.

The inventor has appreciated that the subjective appearance of motion in an image rendering system can bc controllcd by controlling thc amplitudc of thc motion juddcr component, and that the appearance of a particular system can be replicated in another system by making the respective judder components equal in amplitude.

For example, the system of Figure 1 can be modified as shown in Figure 5. In this system the temporal phase input (509) to the temporal interpolator (502) has a motion-judder component added to it in an adder (511). The judder to be added is created by a sinusoidal judder oscillator (512). At the time of each output sample request (507) from the output timing refcrcncc gcnerator (506), the instantaneous value of the sinewave from the judder oscillator (512) is added to the temporal phase value from the subtractor (508) If it is intended to present 24 frames per second film via a 60 Hz television system with the "look" of a 72Hz projector, the judder oscillator (512) is set to generate a 24 Hz sinewave having a peak amplitudc equal to 38.5% of thc time diffcrcncc between 24 Hz frames. This gives the temporal sampling pattern shown in Figure 6.

In Figurc 6 the times and positions corrcsponding to the input film framcs arc indicatcd by circlcs, such as thc position (62). The phasc of thc addcd juddcr component has been adjusted so that a regular sub-set of the input frames have no temporal shift; for example the frame at the position (67). A sequence of five consecutive output television fields is derived from two consecutive input film frames by applying tcmporal shifts as shown in the following tablc: Output tclevision Applicd Temporal Shift to obtain Input film franc

field output television field (as indicated in

(as indicated in (as percentage of time between input Figure 6) Figure 6) film frames) (64) +16.6% (62) (65) -16.6% (62) (66) -17.4% (67) (67) 0 (67) (68) +17.4% (67) Note that the addition of the 24 Hz motion-judder component has momentarily reversed thc dircction of thc motion between the output fields (64) and (65). This reversal occurs periodically every five fields. Note also that the magnitudes of the temporal shifts are similar to each other. If the amplitude of the judder component werc changcd from 38.5% to 39.0% of the temporal distancc betwccn film framcs, all the required temporal shifts become equal at 17.1%, which may simplify the implementation of the invention in hardware.

A doublc-flash projector prcscntation could be rcplicated by reducing thc pcak value of the judder to 25% of the time between input film frames. This is shown in Figure 7.

The required temporal shifts are shown in the following table: Output television Applied Temporal Shift to obtain Input film frame

field output television field (as indicated in

(as indicated in (as percentage of time between input Figure 7) Figure 7) film frames) (74) +3.8% (72) (75) -3.8% (72) (76) -25.3% (77) (77) 0 (77) (78) +25.3% (77) Once again, the direction of motion is reversed every five fields, as at the fields (74) and (75).

The invention can also be applied to the presentation of 24 frame per second film via a 50 field per second television system. An example is shown in Figure 8, which shows 24 Hz sinusoidal judder added to smooth motion. The times and positions of the film frames are marked by circles, and the times of output television fields are indicated by vertical grid lines in the same way in previous Figures. The positive zero crossings of the judder are synehronised to the times of the film frames. It can be seen that the phase relationship of the judder (and the film frame times) to the output field times varies, and so the required temporal shift that must be applied to each output

field (so as to realise the judder) also varies.

Three exemplary regions (809), (810) and (811) within Figure 8 are shown at an enlarged scale in Figures 9, 10 and 11 respectively. In all these Figures the judder has a peak amplitude of 38.5% of the positional difference between film frames; i.e. equivalent to a 72 Hz, triple flash projector. Note that only the fir st frame (801) of the ten film frames shown in Figure 8, coincides with the time of an output television field; this coincidence occurs once every 25 fields, or 12 film frames.

It can be seen from Figures 8 to 11 that a large number of different temporal shift values are required, and that the film frame which is closest in time to a particular output television field may not be closest to the required position. For example, in Figure 10, the output field (101) is closest in time to the input film frame (102), but the succeeding film frame (103) is closest to the required position. If the frame (102) were to be shifted to the required output (101), a (forward) shift of more than 50% of the positional difference between film frames would be required; however, the field (103) need only be shifted (backwards) 32.4%. (The skilled person will appreciate that shifted fields may be derived from information from more than one film frame.) The varying magnitude of the shift values, as the relative phase of the judder relative to the output field times varies, may be visible as a low frequency variation in the character of the jitter. This may be avoided by setting the judder frequency to a sub-multiple of the television field rate. For example 25 Hz judder could be applied to 24 frame per second film portrayed via a 50 field per second television system. However, the change in the operation of the temporal interpolator as the required shift reaches 50% of the time between film frames may still be visible.

A more satisfactory approach is to run the film at a frame rate equal to a sub multiple of the television field rate and apply judder at that rate; i.e. 25 Hz judder applied to frames per second film in a 50 field per second television system. Figure 12 shows this situation with 38.5% judder added. The temporal shift values all have a magnitude of 13.5%.

The examples described so far have had inputs with film-like temporal sampling rates.

Another application of the invention is to process the higher temporal sampling rates of television inputs so that they look as though they were presented cinematically.

This is achieved by adding motion judder, at the frequency of a typical film frame-rate, to the input temporal samples. Figure 13 shows how 50 field per second video material can be modified to replicate the appearance of 25 frame per second film. As in other figures the times and positions corresponding to the input video fields are indicated by circles, and the positionally shifted output fields are indicated by crosses.

If a 25 Hz sinusoidal judder component is phased so that the peaks of the sinewave correspond with the times of the video fields alternate fields are shifted forwards and backwards respectively by the peak judder magnitude. Figure 13 shows 3 8.5% judder that would replicate triple flash projection.

Figure 14 shows how 60 field per second video fields can be modified to appear similar to 24 frame per second projection. In the figure a sinusoidal judder component at 24 Hz has been phased so that a regular sub-set of fields have no shift; for example the fields (1401) and (1402). The pattern of shifts repeats over a period of five frame intervals. The amplitude of the judder has been chosen to minimise the required temporal processing by making the shifted positions of two of the fields in the five-field pattern identical with the original positions of adjacent input fields. The input field (1403) should be shifted to the position (1404), however this corresponds exactly with the input field (1405) and so, rather than temporally interpolating the field (1403), the input field (1405) can be output directly in place of input field (1403).

Similarly input field (1403) is output in place of input field (1405) as it corresponds to the required temporal position (1406).

The judder amplitude that achieves this simplification is 42.1% of the temporal distance between (the notional) 24 frame per second film frames. Only two fields in the five field sequence require interpolation: the field (1407) is moved backwards by 30.9%; and the field (1408) is moved forwards by the same distance. Clearly other values ofjudder may involve temporally interpolating more fields.

The system of Figure 5 need not be implemented with a motion-compensated temporal interpolator. Bilinear interpolation may be used in which a weighted sum of two adjacent input temporal samples is formed such that the centre of gravity' of the weights corresponds to a required output temporal position. The term centre of gravity of a weighted sum of contributions is used here to mean the time at which: w.x1 = 0 Where: w1 is the weight applied to contribution i, x1 is the time difference between the centre of gravity and the time of the input sample weighted by v1; and, the summation is made over all the contributions to the weighted sum.

For example, if the temporal phase value (509) is 1⁄4, the weighted sum will comprise 75% of the current input temporal sample added to 25% of the following input temporal sample. And, the centre of gravity is situated 25% of the input sample pitch after the first input sample.

The skilled person will appreciate that a temporal interpolation process that forms weighted sums of input samples to derive an interpolated output sample is a polyphase filter process that can be described by an aperture function', which defines the sample weights in terms of the respective time differences between the respective input samples and the required output sample time. Figure 15 shows the aperture function for a bilinear temporal interpolator. The horizontal axis represents time, relative to the time of the required interpolated output sample, in units of the period of the input sampling frequency. As no more than two input samples contribute to an output sample, the width (of the non-zero portion) of the aperture is two input sample periods.

Figure 16 shows an example of a higher-order interpolation aperture that is three sample periods wide. As is well-known, higher order interpolators can be designed to have an improved frequency response with less loss of wanted frequency components and better control of alias components. The choice of aperture width (i.e. the number of input samples that are used to make each output sample) is a compromise between complexity and propagation delay, as opposed to the degree of control of the frequency response.

Particular required temporal shift values correspond to discrete phases' of the interpolation aperture, each phase comprising a different weighted sum. In an alternative embodiment of the invention, the design of the filter phases of the interpolator (502) takes into account the required judder component that corresponds to each phase, so that the temporal phase values (507) can be passed directly to the interpolator (502), thus avoiding the need for the judder oscillator (512) and the adder (511). Examples of suitable interpolation phases according to this principle will now be described.

If the timing of the input temporal samples is related to the req uired judder frequency, as is usually the ease, the number of required filter phases can be quite small. In the examples shown in Figures 6 and 7, two pairs of equal and opposite temporal shift values are required. In the example shown in Figure 7 only five interpolation phases are needed and the corresponding sample weights are shown in the table below. The table shows bilinear interpolation with the aperture shown in Figure 15. Each row in the table corresponds to a filter phase; the phases are symmetrical about a central phase and so the corresponding rows in the table are numbered from -2 through to zero and up to +2. The second column shows the output sample temporal phase value (507) that selects the respective phase in the interpolator (502). The position of each interpolated output sample corresponds to the centre of the interpolation aperture, which corresponds to the position of the centre of gravity of the input sample weights.

The right-most column of the table shows this centre of gravity position relative to the position of current input sample, as a percentage of the time between adjacent input samples.

_____________ __________________ ________________________ __________________________________________ _____________

Filter Temporal Exemplary Bilinear Interpolation Weights Centre Phase Phase Value Sample Shifts of Number (percentage (as in Figure 7) Gravity of input Current Output Weight Weight of Weight of Filter sample nd rd Phase Input samplc of 1 2 of3 Sample (earlier) (current) (later) Input Input Input Sample Samplc Sample -2 -40% 77 76 25.3% 74.7% -25.3% -1 -20% 72 74 96.2% 3.8% +3.8% 0 0 77 77 100% 0 -Fl +20% 72 75 3.8% 96.2% -3.8% +2 +40% 77 78 74.7% 25.3% +25.3% An equivalent table for the higher-order, three-tap interpolation aperture shown in Figure 16 is shown below. The position of each interpolated output sample corresponds to the centre of the aperture, and, for some phases, there is a smaH difference between this position and the centre of gravity of the input sample weights.

This difference has a maximum value a lift le less than 9% of the time between adjacent input samples, and is rdated to the frequency response of the interpolation process. For this reason the table shows the aperture centre position rather than the position of the centre of gravity. (As in the preceding table, the position is shown relative to the current input sample position.) Filter Temporal Exemplary Tnterpolation Weights Centre of Phase Phase Value Sample Shifts Interpolation Number (percentage (as in Figure 7) Aperture of input Current Output Weight Weight of Weight sample Input sample of 1 of pic) Sample (earlier) (current) (later) Input Input Input Sample Sample Sample -2 -40% 77 76 15.68% 85.53% -1.21% -25.3% -1 -20% 72 74 -0.70% 99.45% 1.25% +3.8% 0 0 77 77 100% 0 -Fl +20% 72 75 1.25% 99.45% -0.70% -3.8% +2 +40% 77 78 -1.21% 85.53% 15.68% +25.3% In the case shown in Figure 12 there are only two filter phases and the corresponding table for the Figure 15 bilinear interpolation aperture is: Filter Temporal Exemplary Sample Bilinear Interpolation Centre Phase Phase Shifts Weights of Number Value (as in Figure 12) Gravity (percentage of of input Filter sample Phase pitch) Currcnt Output Wcight Weight Weight Input sample of P 0f211d of 31 Sample (earlier) (current) (later) Input Input Input Sample Sample Sample -1 -25% 1202 1201 86.5% 13.5% +13.5% +1 +25% 1202 1203 13.5% 86.5% -13.5% If the higher-order interpolation aperture of Figure 16 is used the sample weights are: Filter Temporal Exemplary Interpolation Weights Centre of Phase Phase Value Sample Shifts Interpolation Number (percentage (as in Figure 12) Aperture of input Current Output Weight Weight of Weight sample pitch) Input sample of I 2nd 0f3rd Sample (earlier) (current) (later) Input Input Input Sample Sample Sample -1 -25% 1202 1201 -1.59% 95.69% 5.90% +13.5% +1 +25% 1202 1203 5.90% 95.69% -1.59% -13.5% Note that, apart from the central phase of the Figure 7 case, the centres of gravity of the bilinear filter phases and the centres of the higher-order interpolation apertures do not correspond to the temporal phases of the required output samples. This is in marked contrast to prior-art temporal interpolators where the centre of each interpolation filter phase corresponds directly with the temporal phase of the corresponding output temporal sample.

Up till now the input temporal samples have been assumed to be evenly spaced. This may not always be so and the invention can be applied to irregular input temporal samples, provided that the timing of the samples is known. Any deviation from uniform timing (i.e. an input judder component) can be subtracted from the added judder to obtain the required output motion characteristic. An important case of unevenly timed input samples is 3:2 pulldown television. If the repeated input fields arc discarded, and thc remaining input ficlds are cvcnly rc-timcd, an input motion profile as at (41) to (44) in Figure 4 results. This pattern can be modified as described above to achieve a desired output motion-judder pattern.

In the above description the temporal samples have been considered as instantaneous, i.e. as temporal delta functions. However, as explained in the introduction to this specification, this is not the case. Display devices hold' each displayed image for some period; this may determined by the projector shutter, the persistence of a cathode ray tube phosphor, or the light-valve of some other electronic display device.

It is well known that the frequency response of a sample-and-hold device is a (sin x)/x function which falls monotonically to a first null where x is equal to it, and then has a set of band-pass responses separated by nulls at multiples of it. The relationship bctwcen x and frequcncy dcpcnds on the ratio of the hold timc to the sample period. If the hold time is equal to the sample period the fir st null occurs at the sampling frequency; for shorter hold times the frequencies corresponding to the nulls are increased.

An exact analysis of the equivalence of the judder from different display devices would include consideration of display hold time. However, as the objective is to achieve a particular subjective level ofjudder, it will often be preferable to choose the level of addedjudder by subjective assessment.

The filter aperture of an FIR temporal interpolator will have temporal frequency response that is not flat. Ifjudder is present in the input signal, then the filter aperture can be chosen to attenuate or ampli' components at the temporal frequency of that input judder, or to eliminate a particular judder frequency. In this case there is no need to add a judder frequency to the temporal phase value that controls the temporal interpolator; the input judder amplitude is changed to a desired judder amplitude by virtue of the temporal frequency response of the temporal interpolator. For example the 12 Hz judder component that is present in conventional 3:2 pulldown television film presentation can be attenuated by a null in the interpolator's temporal frequency response at 12 Hz.

The inyention can be applied at any point in a production, transmission or distribution chain; and, the process can be carried out on analogue signals, streaming data or stored data. Provided the timing of the acquisition of the input temporal samples (in the original camera) is known (or can be inferred from other data), and the intended display timing of output temporal samples is also known (or can be inferred from other data), the value of the motion judder component to be applied to each output temporal sample can be calculated. The inputs to this calculation are: * The required judder function (typically a sinewave) * The required judder frequency (typically the original temporal sampling rate or a lower, harmonically-related frequency).

* The required peak judder amplitude.

* The phase relationship between the required judder function and the intended presentation times of the output temporal samples.

Once the instantaneous value of the judder function at the intended presentation time of each output temporal sample is calculated, the required temporal position of that output temporal sample is known; the temporal difference from the nearest input sample can then be determined and used to control the temporal interpolator.

The judder function and its amplitude may be chosen on artistic grounds to obtain a particular subjective effect, or on technical grounds so as to replicate the characteristics of a known display technology.

The phase relationship between the judder function and the intended presentation times of the output temporal samples may be chosen to simplify the implementation of the temporal interpolation process -for example to reduce the number of required phases of an FIR temporal interpolation ifiter.

The application of the invention may, or may not, change the number of temporal samples. All that is required is that the motion portrayed by the output samples is modified by the intended motion judder.

The spatial sampling of television has not been previously mentioned in this specification as it is not relevant to the understanding of the invention by the skilled person. It is well known that, in interlaced television systems, separate temporal presentations of the same film frame may have different spatial sampling patterns.

The known art of dc-interlacing' may be used convert between these different spatial sampling patterns.

In this specification nominal field and frame rates have been quoted as examples. The above described principles are applicable to 59.94 Hz field rate television systems as well as systems with integer-frequency rates.

The present inventions are also defined by way of the following clauses: 1. A method of modifying an input sequence of temporal image-samples representing a moving image so as to obtain an output sequence of temporal image-samples in which the appearance of the said motion has been changed wherein the said input temporal image-samples are temporally interpolated to create the said output sequence and an intentional, periodic motion-judder component is introduced that is visible when the said output temporal image-samples arc displayed.

2. A method according to Clause I in which the said intentional, periodic motion-judder component is chosen so that the said appearance of motion replicates cinematic projection.

3. A method according to Clause I or Clause 2 where the said intentional, periodic motion-judder component periodically reverses the apparent direction of motion.

4. A method according to any previous Clause where the said intentional motion-judder component is determined by a sine thnction.

5. A method according to Clause 4 where the magnitude of the motion-judder component applied to a temporal image-sample is proportional to the instantaneous value of a temporal sinewave at the intended time of display of that temporal image-sample.

6. A method according to Clause 4 or Clause S where the frequency of the said sine frmnction is equal to the original temporal sampling frequency of the input temporal image-samples.

7. A method according to any previous Clause in which the said temporal interpolation is motion-compensated.

8. A method according to any previous Clause in which the said temporal interpolation is performed by a polyphase FIR filter.

9. A method according to Clause 8 in which at least one phase of the said polyphase FIR filter is derived from an interpolation aperture position that does not correspond to the intended display time of the respective interpolated output temporal sample created by that filter phase.

10. A method of representing nominal 24 frame per second film frames in a nominal 60 field per second television system wherein respective television fields arc created in a five field sequence in which: the first television field corresponds to a film frame; and the second, third, fourth and fifth fields are created by temporally interpolating film frames either forwards or backwards by a time equal to 17.1% of the reciprocal of the film frame rate.

11. A method of modifying a sequence of television images corresponding to a nominal temporal sampling rate of 60 Flz in which the said images arc modified according to a five-image sequence where: the first image is unmodified; the second image is temporally shifted backwards by 30.9% of the time between adjacent images in the sequence;

the third and fourth fields are interchanged; and

the fifth image is temporally shifted forwards by 30.9% of the time between adjacent images in the sequence.

12. Apparatus for modifying an input sequence of tcmporal imagc-samplcs representing a moving image so as to obtain an output sequence of temporal image-samples in which the appearance of the said motion has been changed wherein the said input temporal image-samples are temporally interpolated to create the said output sequence and an intentional, periodic motion-judder component is introduced that is visible when the said output temporal image-samples arc displayed.

13. Apparatus according to Clause 12 in which the said intentional, periodic motion-judder component is chosen so that the said appearance of motion replicates cinematic projection.

14. Apparatus according to Clause 12 or Clause 13 in which the said intentional, periodic motion-judder component periodically reverses the apparent direction of motion.

15. Apparatus according to any of Clauses 12 to 14 in which the said intentional motion-judder component is determined by a sine function.

16. Apparatus according to Clause 15 in which the magnitude of the motion-judder component applied to a temporal image-sample is proportional to the instantaneous value of a temporal sinewave at the intended time of display of that temporal image-sample.

17. Apparatus according to Clause 15 or Clause 16 in which the frequency of the said sine function is equal to the original temporal sampling frequency of the input temporal image-samples.

18. Apparatus according to any of Clauses 12 to 17 in which the said temporal interpolation is motion-compensated.

19. Apparatus according to any of Clauses 12 to 18 in which the said temporal intcrpolation is performed by a polyphasc FIR filter.

20. Apparatus according to Clause 19 in which at least onc phase of the said polyphase FIR filter is derived from an interpolation aperture position that does not correspond to the intended display time of the respective interpolated output temporal sample created by that filter phase.

21. Apparatus for representing nominal 24 frame per second film frames in a nominal 60 field per second television system wherein respective television fields are created in a five field sequence in which: the first television field corresponds to a film frame; and the second, third, fourth and fifth fields arc crcatcd by temporally interpolating film frames either forwards or backwards by a time equal to 17.1% of the reciprocal of the film frame rate.

22. Apparatus for modi'ing a sequence of television images corresponding to a nominal temporal sampling rate of 60 Hz in which the said images are modified according to a five-image sequence where: the first image is unmodified; the second image is temporally shifted backwards by 30.9% of the time between adjacent images in the sequence;

the third and fourth fields are interchanged; and

23. A computer program product comprising instructions adapted to implement a method according to any one of Clauses 1 to 11.

Claims

CLAIMSI. A method of modifying an input sequence of temporal image-samples representing a moving image so as to obtain an output sequence of temporal image-samples in which the appearance of the said motion has been changed wherein the said input temporal image-samples are temporally interpolated to create the said output sequence and an intentional, periodic motion-judder component is introduced that is visible when the said output temporal image-samples are displayed where the said intentional motion-judder component is determined by a sine function.
2. A method according to Claim 1 where the magnitude of the motion-judder component applied to a temporal image-sample is proportional to the instantaneous value of a temporal sinewave at the intended time of display of that temporal image-sample.
3. A method according to Claim I or Claim 2 where the frequency of the said sine function is equal to the original temporal sampling frequency of the input temporal image-samples.
4. A method according to any previous Claim in which the said temporal interpolation is motion-compensated.
5. A method according to any previous Claim in which the said temporal interpolation is performed by a polyphase FIR filter.
6. A method according to Claim 5 in which at least one phase of the said polyphase FIR filter is derived from an interpolation aperture position that does not correspond to the intended display time of the respective interpolated output temporal sample created by that filter phase.
7. Apparatus for modif'ing an input sequence of temporal image-samples representing a moving image so as to obtain an output sequence of temporal image-samples in which the appearance of the said motion has been changed wherein the said input temporal image-samples are temporally interpolated to crcatc thc said output scquencc and an intentional, pcriodic motion-juddcr component is introduced that is visible when the said output temporal image-samples are displayed in which the said intentional motion-judder component is dctermined by a sinc ifinction.
8. Apparatus according to Claim 7 in which the magnitude of the motion-judder component applied to a temporal image-sample is proportional to the instantancous value of a tcmporal sinewayc at thc intcndcd timc of display of that temporal image-sample.
9. Apparatus according to Claim 7 or Claim 8 in which the frequency of the said sine function is equal to the original temporal sampling frequency of the input temporal image-samples.
10. Apparatus according to any of Claims 7 to 9 in which the said temporal interpolation is motion-compensated.
11. Apparatus according to any of Claims 7 to 10 in which the said temporal interpolation is performed by a polyphase FIR filter.
12. Apparatus according to Claim 11 in which at least one phase of the said polyphase FIR filter is derived from an interpolation aperture position that does not correspond to the intended display time of the respective interpolated output temporal sample created by that filter phase.
13. A computer program product compnsrng instructions adapted to implement a method according to any one of Claims I to 6.