GB2459684A - Television Signal having high frame rate - Google Patents

Abstract

Television/film capture, production and/or transmission systems are disclosed operating at a frame rate which is substantially higher than existing conventional television systems such as PAL, NTSC or standard film frame rates. The frame rate may be between 80 frames per second (fps) and 100,000 fps. Video processing and editing, and in particular effects processing, may be performed at the substantially higher frame rate. The video format used by the system may use a bit depth of one bit. The high frame rate video may be down-converted to standard frame rates for transmission, or may be transmitted at the high frame rate to suitably modified or specially designed end-user equipment. Motion information determined for a high frame-rate signal can be used to improve compression of a corresponding lower frame rate signal or to enable reproduction by a receiver of a higher framer rate signal from a lower frame rate signal. Processing at the high frame rate may include adding a shuttering or lighting effect.

Description

Television system The present invention relates in particular examples to methods and systems for providing and processing video and television signals.

Recently, significant efforts have been made to provide video and television products and services with improved visual quality, for example by way of high-definition televisions, set-top-boxes and broadcasts, and high-definition video media such as HD-DVD and Blu-ray. However, these new products and services are often still based to a large extent on old video formats that give rise to certain quality limitations, for example in relation to the reproduction of fast motion. Motion blur and motion aliasing artefacts are examples of problems that can arise in these systems.

The present invention seeks to alleviate certain problems and limitations of prior art television and video processing systems.

Accordingly, in a first aspect, there is provided a television system adapted to provide, process or use a television signal having a frame rate substantially higher than conventional television and/or film frame rates.

By using frame rates substantially higher than conventional television frame or field rates, video image quality can be improved, in particular in relation to the portrayal of fast motion and motion artefacts.

Specifically, the frame rate is preferably substantially higher than one or more (preferably each) of: PAL frame or field rate, NTSC frame or field rate, and standard film frame rate. Such frame rates (ranging from 24fps for film to 3Ofps for NTSC) have been in use widely for decades (indeed since the very beginning of film and television) and have generally been felt to be adequate since they allow for the reproduction of apparently smooth moving images.

This, together with the high data rates required for encoded video, has led to a prejudice in the art for choosing such relatively low frame rates. However, underlying the present invention is the realisation that significant quality gains can be obtained by using much higher frame rates, without necessarily requiring a proportional increase in data rates.

Advantageously, to achieve some of these improvements, the frame rate is at least 8Ofps, preferably at least 1 OOfps, more preferably at least l5Ofps. Even higher frame rates may lead to correspondingly greater improvements in video quality. Accordingly, the frame rate is advantageously at least 300fps, preferably at least 600fps, more preferably at least l200fps.

Still higher frame rates may be used, for example at least 2500fps or at least 5000fps. Some embodiments use frame rates of at least 1O,000fps or even at least 100,000fps. Examples of such embodiments are described below in connection with a one-bit colour component encoding.

Preferably, the frame rate is a multiple of one or more selected conventional television frame or field rates or film rates. The frame rate may be a multiple of each of a plurality of selected conventional television frame or field rates or film rates. This enables easier conversion between the relevant conventional rates and the high frame rate.

Specifically, the frame rate may be a multiple of one or more, preferably each, of: 25fps PAL frame rate, 3Ofps approximate NTSC rate and 24fps standard film rate. Preferably, the frame rate is a multiple of one or more, preferably each of: 50 fields-per-second PAL field rate, 60 fields-per-second approximate NTSC field rate, and 24fps standard film rate.

Preferably, to allow easier conversion to and from all of the most widely used conventional formats, the frame rate is a multiple of each of the above (24fps, 25fps, 3Ofps), and thus is preferably 600fps or a multiple thereof.

The term television signal preferably refers to a video signal which can be broadcast on a television network, but may also include such a signal as recorded by a camera and/or processed in a production/editing system prior to broadcast. The signal preferably defines a sequence of images displayable at the frame rate. In other words, a sequence of images (or frames) for display at the frame rate and at a given display resolution are encoded in the signal (though the encoding may use compression including, for example, frame prediction, so that each frame need not necessarily be encoded pixel-by-pixel).

The signal is preferably a digital television signal, more preferably a digital television broadcast signal for broadcast on a digital television network.

Different colour components of the signal may be sampled at different temporal rates. For example, to reduce the data rate, the method may comprise sampling one or more chroma components at a lower temporal rate than a luma (or luminance) component, preferably at half the luma sampling rate or less, more preferably at a quarter the luma sampling rate or less.

Spatial chroma sub-sampling may be used in addition to temporal chroma sub-sampling.

The signal may be encoded with a bit depth of at most two bits per colour component, preferably with one bit per colour component or with one-bit plus a sign bit (depending on the colour space used). A frame rate of at least 1O,000fps is preferably used in this case. Given a sufficiently high frame rate, this can enable efficient encoding and processing of the signal.

The system preferably comprises means for providing source video at the high frame rate, the providing means preferably including one or more cameras operable to capture source video at the high frame rate. Other high frame rate sources may be used, for example a high frame rate animation system. The source video may form the television signal or may be incorporated into the television signal (for example by editing to combine with other sources).

The system may also comprise means for performing video editing or video processing at the high frame rate (on the high frame rate signal or to produce the high frame rate signal). Preferably, the system comprises means for performing video effects processing on the high frame rate signal to add a video effect to the high frame rate signal, preferably a shuttering effect or a lighting effect. By using a high frame rate, certain new video processing effects can be implemented.

The system preferably comprises means for transmitting the high frame rate signal to end user equipment, the end user equipment preferably adapted to output the signal at the high frame rate. The system may include the end user equipment, which may comprise one or more televisions, set top boxes, cinema display systems, personal computers, mobile devices, or the like. The transmitting means preferably includes a broadcast system for broadcasting the television signal on a broadcast network, such as a digital terrestrial, cable or satellite network.

The television system may thus include one or more video cameras or other sources, video production/editing system(s), transmission system(s) and end user equipment all operating at the high frame rate, thus providing a complete high frame rate television production, distribution and display system.

The system may comprise means for compressing the signal, preferably using three-dimensional block-based coding, preferably using temporal and/or spatial prediction of three-dimensional blocks of video data.

The system may comprise means for converting the television signal at the high frame rate to a low frame rate signal, preferably at a standard television frame rate or film rate, in which case the system may further provide means for transmitting the low frame rate signal to end user equipment. This can enable interoperability with conventional end user equipment which is not compatible with the higher frame rate.

The system may comprise means for deriving information for use in compression from the high frame rate signal, and means for compressing the low frame rate signal using the derived information. Specifically, the system may comprise means for deriving motion information from the high frame rate signal and means for compressing the low frame rate signal using the derived motion information. This can improve compression efficiency and quality, allowing the low frame rate signal to benefit from the additional information available in the high frame rate source signal.

The system may alternatively or additionally comprise means for deriving motion information from the high frame rate signal, and means for transmitting the motion information with the low frame rate signal to a receiver.

The receiver (e.g. end user equipment) may be adapted, using the received motion information, to output a signal at a frame rate higher than the low frame rate at which the signal was transmitted, preferably at the original high frame rate. This is another way in which reproduction quality of the low frame rate signal may be improved using information derived from the high quality signal.

In further aspects corresponding to the above, the invention provides apparatus having means for processing a television signal having a frame rate substantially higher than conventional television or film frame rates, and a method of providing a television signal having a frame rate substantially higher than conventional television or film frame rates, both optionally with corresponding preferred features. The apparatus may be a digital television receiver or set-top box, or a display, for displaying the signal at the high frame rate, a camera for capturing the signal at the high frame rate, or a video processing/editing apparatus for processing/editing video at the high frame rate. The invention also provides a corresponding video editing system comprising means for editing video at a frame rate substantially higher than conventional television or film frame rates, again optionally with corresponding preferred features.

In a further aspect of the invention, there is provided a method of providing a digital video signal, comprising encoding the signal with a bit depth of at most two bits per colour component. The encoding may use one bit plus a sign bit for one, some or all colour components and/or may use a single bit for one, some or all colour components. In case of monochrome video, there may only be a single colour component for each pixel, whilst a colour image will have multiple (typically three) colour components per pixel.

Preferred embodiments use a single bit to encode each colour component. The video signal is preferably colour video.

In this way, a simple encoding scheme can be provided, which can enable more efficient processing of the signal. The signal preferably has a frame rate substantially higher than conventional television or film frame rates.

Advantageously, the signal has a frame rate which is sufficiently high so that display of the signal at the high frame rate produces the effect of a full colour image on a viewer (due to the integration performed by the eye the one-bit nature of each frame becomes substantially imperceptible, i.e. the viewer sees colours that are not actually present at any given instant in time because the pixels are switched too rapidly to be perceived separately). The term "full colour image" preferably means an image having a colour range corresponding at least to conventional standard or high definition television or video formats.

Preferably, the signal has a frame rate of at least 1O,000fps, more preferably at least 100,000fps. In some cases, frame rates of at least 250,000fps, at least 500,000fps or even at least 1,000,000fps may be used.

By using an extremely high frame rate of for example at least 1O,000fps the effect of a colour image can be created despite the 1-bit sampling used.

The method may further comprise converting between the one-bit (or one-bit plus sign) video format and a video format using multiple (more) bits per colour component, preferably a standard television or film format.

The invention also correspondingly provides a camera adapted to record a digital video signal with a bit depth of at most two bits, preferably one bit per colour component (or one-bit plus sign as described above), and a display adapted to display a digital video signal having a bit depth of at most two bits, preferably one bit (or one-bit plus sign) per colour component. These aspects may include corresponding preferred features.

Similarly, the invention provides a video editing system comprising means for editing video encoded with a bit depth of at most two bits, preferably one bit per colour component (or one-bit plus sign), preferably at a frame rate substantially higher than conventional television or film frame rates, preferably at least 1O,000fps, more preferably at least 100,000fps.

The invention also provides a video conversion system comprising means for converting between a first video format having a high frame rate and a bit depth of at most two bits, preferably one bit per colour component (or one bit plus sign) and a second video format having a lower frame rate and a bit depth of multiple (or more) bits per colour component. The second video format is preferably a standard television or film format, preferably a PAL or NTSC television format.

In a further aspect, the invention provides a method of providing a video signal, comprising sampling at least one colour component of the signal at a different temporal rate to one or more other colour components.

Preferably, one or more chroma components are sampled at a lower temporal rate than a luma (or luminance) component, preferably at half the luma sampling rate or less, more preferably at a quarter the luma sampling rate or less. Spatial chroma sub-sampling may be used in addition to temporal chroma sub-sampling. The signal preferably has a frame rate substantially higher than conventional rates as described above and/or may be encoded with two, preferably one, bits per colour component as described above.

In a further aspect, the invention provides a method of providing a compressed video signal, comprising: receiving a video signal at a first frame rate; deriving information for use in compression from the video signal at the first rate; converting the video signal at the first frame rate to a video signal at a second frame rate different from the first frame rate; and compressing the video signal at the second frame rate using the derived information. The second frame rate is preferably lower than the first frame rate. In this way, compression quality and efficiency for the lower frame rate signal can be improved using information from the higher frame rate signal. The information is preferably motion prediction or motion vector information (e.g. one or more motion vectors) for use in interirame compression of the video signal. The video signal may be encoded using a bit depth of one bit or another encoding described herein.

In a further aspect, the invention provides a method of transmitting a video signal, comprising: receiving a video signal at a high frame rate; deriving motion information from the high frame rate video signal; converting the high frame rate video signal to a low frame rate video signal; transmitting the low frame rate video signal and the motion information to a receiver; and at the receiver, deriving from the received low frame rate signal a signal at an output frame rate higher than the low frame rate at which the signal was transmitted using the received motion information, and outputting the derived signal at the output frame rate. In this way, display quality may be improved for a lower frame rate signal. The output frame rate may be equal to the high frame rate.

In a further aspect, the invention provides a method of processing video, comprising: receiving a video signal at a high frame rate, the high frame rate being substantially higher than conventional television frame rates; and performing video processing at the high frame rate to produce a processed video signal at the high frame rate. Preferably, the method comprises converting the processed video signal to a low frame rate lower than the high frame rate, preferably a standard television or film frame rate (e.g. 24fps, 25fps or 3Ofps); and outputting the low frame rate processed video signal.

This can enable a variety of processing techniques and effects that might not be possible if starting from a low frame rate signal. Also, by performing video processing in the high frame rate domain prior to down-converting to a lower frame rate, image quality can in some cases be improved.

The step of performing video processing may comprise performing video editing. Video editing at high frame rate can enable greater editing accuracy. Alternatively or additionally, video processing may comprise performing effects processing, preferably to apply a shuttering or lighting effect.

For example, the step of performing video processing may comprise applying a synthetic camera shuttering effect to the video signal. The shuttering effect may be applied differentially across the video image.

As a further example, the step of performing video processing may comprise detecting the effect on a scene (as represented in one or more frames) caused by a variable light source based on the characteristic rate and/or phase of variation of the light source, and may optionally comprise modifying the scene accordingly.

Specifically, the step of performing video processing may comprise identifying regions of or objects in one or more video images where elements of the scene are illuminated by a given light source by detecting the characteristic rate and/or phase of flicker of the light source. The method may further comprise modifying one or more video images or identified objects or regions thereof to modify, reduce or remove illumination caused by the light source. The method may comprise distinguishing between multiple light sources based on respective flicker rates or phases of the light sources.

The method may also comprise processing the video image to remove a short instantaneous flash from a particular light source (e.g. due to flash photography or lightning). The high frame rate can thus enable detection of image contributions which might not be detectable at conventional frame rates.

The step of performing video processing may also comprise filtering a video sequence. Filtering may be applied differentially across an image.

Temporal filtering may be performed. In one example, filtering may be performed by applying a three dimensional convolution function to the video sequence, wherein the convolution function is shaped to filter along the trajectory of motion present in the video sequence.

In a further aspect, the invention provides a method of processing video to detect or modify the contribution to a scene from a regularly varying light source, the method comprising: capturing the video at a frame rate which is greater than the rate of variation of the light source; and detecting a contribution to one or more frames of video due to the light source based on the rate and/or phase of variation of the light source; and optionally further comprising modifying one or more of the frames of video based on the detected contribution. The frame rate is preferably substantially greater than the rate of variation or flicker rate of the light source, preferably at least twice the flicker rate, more preferably at least four times or at least ten times the flicker rate. The frame rate may be as described above. The method may comprise modifying the frame or frames to increase, reduce or remove the detected contribution due to the light source.

The invention also provides a method of compressing a video signal, preferably (though not necessarily) having a frame rate substantially higher than conventional television or film frame rates, the video signal having two spatial dimensions and a temporal dimension, the method comprising: dividing the signal into a plurality of three-dimensional blocks of video data; and encoding the three-dimensional blocks. Encoding a block preferably comprises calculating a frequency domain transform of the block; and quantising the resulting transform coefficients. The method preferably comprises calculating prediction information for a block, and encoding the block using the prediction information. The method preferably comprises performing spatial and/or temporal prediction. Using such a three-dimensional compression process, in particular for a high frame rate signal, can improve compression efficiency by exploiting the high degree of similarity between frames. The invention also provides a method for decompressing a signal compressed using such a compression process.

The invention also provides a video processing system or apparatus adapted to perform any method as described herein, and a computer program or computer program product comprising software code adapted, when executed on a data processing apparatus, to perform any method as described herein.

Where reference is made above (and in the appended claims) to "means for" performing some act, such means may, for example, include a processor and associated memory, suitably programmed to perform the act.

More generally, the invention also provides a computer program and a computer program product for carrying out any of the methods described -10 -herein and/or for embodying any of the apparatus features described herein, and a computer readable medium having stored thereon a program for carrying out any of the methods described herein and/or for embodying any of the apparatus features described herein.

The invention also provides a signal embodying a computer program for carrying out any of the methods described herein and/or for embodying any of the apparatus features described herein, a method of transmitting such a signal, and a computer product having an operating system which supports a computer program for carrying out any of the methods described herein and/or for embodying any of the apparatus features described herein.

The invention extends to methods and/or apparatus substantially as herein described with reference to the accompanying drawings.

Any feature in one aspect of the invention may be applied to other aspects of the invention, in any appropriate combination. In particular, method aspects may be applied to apparatus aspects, and vice versa.

Furthermore, features implemented in hardware may generally be implemented in software, and vice versa. Any reference to software and hardware features herein should be construed accordingly.

Preferred features of the present invention will now be described, purely by way of example, with reference to the accompanying drawings, in which:-Figure 1 illustrates a high frame rate television production and broadcast system in overview; Figure 2 illustrates a process of capturing, processing and displaying a high frame rate video signal; Figure 3 illustrates low frame rate and high frame rate signals; Figure 4 illustrates a process for high frame rate video processing and downsampling; Figure 5 illustrates a video compression process using a high frame rate video source; Figure 6 illustrates a method of video compression; and Figure 7 illustrates three-dimensional (spatial/temporal) block coding and prediction. -11-

Hicih frame rate video system A high frame rate television/film production system is shown in overview in Figure 1.

The system comprises a high frame rate camera 12 for providing source video and a high frame rate editing system 16 for editing the captured video. Storage 14 is provided for storing unedited and edited video content, and may include disc storage, tape storage, and/or any other suitable storage media.

Editing system 16 may make use of a variety of editing/post-production components, such as an animation generator 18, effects unit 20, graphics generator 22 and video mixer 24.

Edited or raw video content is transmitted using a broadcast subsystem 26 to a broadcast network 30, such as a digital or analogue terrestrial, satellite or cable network. Video content may also be made accessible via a media server 28 to be accessed over a data network, for example the Internet 32.

Transmitted content is received from the broadcast network or Internet by end-user devices such as a set-top box 34, personal computer 36 or cinema display system 38.

All the elements of the system are adapted to operate at a substantially higher frame rate than is used in current conventional television/film systems, as is described in more detail below.

Figure 1 shows representative elements of such a television system. A given system may contain only some of the elements shown and may contain other elements not shown. Only one of each element is shown, though the system will typically include multiple such elements, e.g. multiple cameras, set-top boxes, effects processors etc. The system will typically also include compression/decompression functionality to enable compression of digital video data for storage and transmission.

Embodiments of the invention provide a high frame/field rate television and film system, in which the quality of the reproduced image is improved by the use of increased frame and/or field rates throughout the entire system.

Thus, video or film pictures are captured at a rate in excess of the conventional 24/50/60 frames/fields per second and are stored, transmitted

and reproduced at this higher frame/field rate.

-12 -For example, the system could run at a rate of 300 frames per second.

In that case all the cameras 12, editing system components (e.g. animation generators 18, graphics generators 22, video mixers 24 and effects units 20) would run at 300 frames per second to create an output which is stored or distributed as a 300 frames per second video signal. The transmission system then broadcasts this high frame rate signal to the home or to cinernas or other places for decoding and display, again at the higher frame rate. The high frame rate signal can also be recorded onto fixed media such as disc/tape or downloaded file for sale to the home or onto other storage media for distribution to cinemas or other venues.

The resulting TV pictures can give far better rendition of motion and of moving detail than current TV and film systems, in particular by reducing motion blur. The reduction in motion blur can greatly improve the sharpness of anything in the picture which is moving relative to the camera/display. The removal of motion blur can also improve the sense of reality of the image, removing the tendency for moving objects to become transparent. The sharpening of moving edges can lead to an improved occlusion of backgrounds improving the sense of the image being real and three-dimensional.

The resulting TV images can also have lower levels of motion aliasing.

This can apply at the level of object structure, such as the effect of wagon wheels appearing to be going backwards, and at the level of image structure, where an edge moves across the sampling structure. The lower levels of temporal aliasing can also lead to improved compression efficiencies so that the increase in frame rate does not necessarily lead to proportionately higher storage requirements, as both lossy and lossless compression rates can be improved as the frame rate is increased.

Increased frame rates at the camera and display can also reduce other flicker and motion related effects such as photosensitive and pattern-sensitive epilepsy.

Video effects processing (e.g. by animation generator 18, effects unit 20, graphics generator 22 and video mixer 24) can be performed at the higher frame rate to achieve higher-quality effects. Video editing can be performed -13 - (e.g. by editing system 16) more accurately due to the increased temporal resolution.

Instead of camera 12, other video sources may be used. For example, a 3D animation/rendering system may provide the source video at the high frame rate.

The frame rate chosen is preferably substantially higher (preferably many times higher) than the conventional film / television frame or field rates.

Such conventional rates include 24fps film rate; 25fps television (e.g. PAL), 3Ofps/29.97fps television (e.g. NTSC); and corresponding field rates of 50 / 60(59.94) fields-per-second. For many practical purposes, the NTSC rate of 29.97fps can be approximated to 3Ofps; for example, standards conversion can be achieved by accelerating a 29.97fps signal to 3Ofps prior to performing further processing. Thus, references herein to a 3Ofps rate are intended to also include the standard NTSC rate of 29.97fps (and references to 60 fields-per-second field rates similarly include the standard NTSC field rate of 59.94

fields-per-second).

Some of the benefits of the high frame rate system can be realised at frame rates of, for example, 65 or 7Ofps (for example, the improvements in the portrayal of motion can in some respects be proportional to the frame rate).

However, other advantages, such as the lower delay in video processing systems and improved lip synch would not be significant until much higher frame rates were used (for example in excess of 1 OOfps).

For a practical system, a frame rate of at least 8Ofps is therefore preferably used. Preferably, the frame rate is at least lOOfps, more preferably at least 150 fps, more preferably at least 300fps. In some embodiments, an even higher frame rate may be selected, for example at least 600fps, at least 900fps or even at least 1 200fps.

Preferably, the frame rate chosen has a simple mathematical relationship with one or more selected conventional frame rates. Preferably, the chosen frame rate is a multiple of at least one selected conventional frame rate or field rate. This simplifies conversion between a conventional signal and a high frame rate signal. More preferably, the frame rate is a multiple of several conventional frame/field rates, to improve interoperability with selected existing systems. The rate can be chosen based on requirements, i.e. based -14 -on the conventional systems with which interoperability is desired. For example, a frame rate may be selected which is a multiple of both conventional television frame rates of 25 and 30, e.g. l5Ofps, or which is a multiple of both conventional television field rates of 50 and 60, e.g. 300fps or 600fps to allow conversion and interoperability with existing PAL/NTSC systems.

A frame rate with a simple mathematical relationship with (e.g. being an exact multiple of) each of the television frame/field rates of 25fps/3Ofps and 50 and 60 fields-per-second and also the conventional film rate of 24fps is particularly preferred, such as 600fps or a multiple thereof, since this simplifies conversion between film sources as well as NTSC and PAL systems.

The frame information for the high frame rate signal may be encoded in any suitable way, for example using conventional 8-bit or 10-bit per colour component encoding in RGB, YUV, YCrCb or any other suitable colour space.

Conventional compression techniques may also be used to reduce the data rate of the signal, including intraframe and interframe compression (e.g. using motion prediction).

In summary, embodiments of the invention thus provide an improved television, video and/or film system utilising higher frame/field rates than current systems. The system comprises a camera and display, connected together via a link capable of capturing, conveying and displaying pictures at rates higher than current television, video and film rates. This can be attached to storage for the recording of these pictures or connected to a link for live broadcast of these pictures.

Further embodiments provide: * A production system capable of editing, mixing and otherwise manipulating these pictures at the higher frame/field rates.

* A link or broadcast system capable of sending these high frame/field rate pictures to other locations and a receiver.

* A storage system capable of recoding the signals from the camera and production system and playing them back for further manipulation or broadcasting.

-15 -The use of a very high frame rate source video signal enables a range of improved video processing, editing and compression techniques. For example, embodiments of the invention may provide: A system that produces motion information from the high frame rate captured pictures and then transmits those pictures at a lower frame rate along with the motion information such that pictures a higher frame rate than transmitted can be reproduced at the receiver or display device (this is discussed in more detail below) * High frame/field rates which have a simple mathematical relationship with both 50 and 60 fps (e.g. 300 fps) can allow simple standards conversion to both field rates. (A frame/field rate of 600fps would also provide simple conversion to 24fps.) * A video/film editing system which gives the opportunity to make finer grained edits than conventional video/film editing systems, i.e. giving an editing accuracy down to a single frame at the higher frame rate.

* A system of volumetric coding for video compression which takes advantage of the improved temporal sampling, eliminating conventional motion compensation and using volumetric and directional transformations.

* A system of noise reduction which takes advantage of the much greater similarity of adjacent frames, by means of thresholding or quantisation of coefficients in three-dimensional transform domains.

* A system of noise reduction which takes advantage of the higher frame/field rate to improve the ability to differentially reduce noise in different parts of the picture * A system of noise reduction which takes advantage of the higher frame/field rate to improve the ability to reduce noise in moving objects by filtering along the direction of motion Capturing video at very high frame rates in conventional artificial lighting conditions may be problematic in that light intensity may vary in -16-accordance with the phase of 50Hz or 60Hz mains power sources. This could introduce marked illumination differences between frames and "beat notes" in the resulting video which could have a temporal frequency on only a few hertz due to timing differences between the video and the light sources.

Embodiments may therefore also include: * A set of temporal-spatial filters to remove colour and level differences between frames in conventional artificial lighting * A system of illumination using a direct current source or an alternating current source at a much higher rate than the camera frame rate or an alternating current source synchronised with the camera frame rate or some other means, such that the variation in scene illumination between frames is minimised.

Hicih frame-rate video with one-bit samplinq A further embodiment will now be described which provides a television system whereby the quality of the reproduced image is improved by the use of one bit sampling of the image at very high frame rates.

In this embodiment, video or film pictures are captured at a bit-depth of one bit per colour component at a rate greatly in excess of the current 24/50/60 frames/fields per second, and are stored, transmitted and reproduced using the one bit representation. The representation may use RGB, YUV, YCrCb or any other suitable colour space, with each component sampled with a one-bit bit depth. The frame rate is selected to be sufficiently high so that the integration performed by the human eye renders the one-bit nature of each frame imperceptible. To achieve this, the 1-bit system preferably uses a frame rate of at least around 10,000fps. Preferred embodiments use a frame rate between 100,000fps and 1,000,000fps, although still higher frame rates could also be used.

The system is illustrated in overview in Figure 2. A high frame rate camera 40 captures a high frame rate 1-bit signal 42. Processing 44 may optionally be performed. The high frame rate 1-bit signal is then provided to a suitably adapted display 46. Alternatively, conversion stage 48 may be provided to convert the signal to a conventional low frame rate signal suitable -17 -for a standard display 50. The system may include the various other elements discussed previously in relation to Figure 1.

This embodiment shares many advantages with the previously described embodiments, for example in relation to the realism of the images and the rendition of motion, in particular the reduction of motion blur and motion aliasing artefacts. The use of one bit sampling and reproduction can also reduce and possibly eliminate other flicker and motion related effects such as photosensitive and pattern-sensitive epilepsy.

The system can be a good match to certain modern display technologies such as Plasma Display Panels or back-and front-projected systems based on Digital Mirror Devices, as only the driving electronics would need to be changed to make them one-bit video compatible.

According to one embodiment, an improved television, video and/or film system utilising bit-depths of one bit and much higher frame/field rates than current systems is provided.

The system includes a display capable of displaying video frames with a bit-depth of one bit (per colour component) at a frame rate much higher than that of conventional television and cinema: high enough that the integration performed by the human eye renders the one-bit nature of each frame imperceptible. Plasma Display Panels and Digital Mirror Devices are examples of technologies that could be used to implement such a display.

Specifically, in such a display, instead of displaying a pixel with different brightness values for each of a number of colour components (typically RGB for digital displays), each colour component simply has two states: on or off. However, due to the very high frame rate of the signal, the pixel component may be switched on and off rapidly to produce the effect of intermediate brightness values. This effect occurs since the human eye is unable to resolve the transitions at the very high frame rates used. Different switching frequencies of a colour component then correspond to different brightness values for the colour component.

The viewer in effect sees a full colour image (i.e. an image providing a greater colour range than a static single-bit-per-component image could actually provide), preferably substantially corresponding in colour range at least to a conventional television image.

-18 -The system may further include a television camera containing a sensor capable of capturing, digitising and outputting video frames with a bit-depth of one bit, at a rate equal to or greater than the display rate described above. A conventional digital video camera that supports image capture at a sufficiently high frame rate could be used to acquire one-bit video merely by applying a threshold to each pixel value and making appropriate use of dithering or error feedback techniques. Alternatively, an image sensor specifically adapted for 1-bit sampling may be used.

Figure 3 illustrates an example of a high-frame-rate signal 64 including a sequence of frames at a high rate. A given pixel 66 of a frame's array of pixels comprises RGB colour components encoded using a single bit each. A multi-bit sampled low frame rate signal 60 is also shown by way of comparison.

The system also includes a digital link or broadcast system with a bandwidth sufficient to carry the data being output by the camera and/or fed into the display in a compressed or uncompressed form.

The system may also incorporate the following: * A production system capable of editing, mixing and otherwise manipulating video in the one-bit format.

* A storage system capable of recording the signals from the camera and production system and playing them back for further manipulation or broadcasting.

* A conversion system such as a system based on the principles of delta-sigma modulation to enable conventional film and video pictures to be converted to the one-bit video format, and video in the one-bit format to be down converted to current film and TV frame/field rates. This is discussed in more detail below.

* The use of a data-rate reduction system to reduce the bandwidth required by the system. For example, run-length coding could be employed. Other, more complex compression techniques may also be used.

* A video/film editing system which gives the opportunity to make finer grained edits in time than conventional video/film editing -19 -systems, i.e. giving an editing accuracy down to a single frame at the higher frame rate of the one-bit video format.

A higher spatial resolution than conventional television formats may also be used for recording and/or display of the one-bit format.

Conversion between one-bit and conventional video formats could be accomplished in a number of ways. Existing conversion techniques for converting between PCM (pulse code modulation) and PWM (pulse width modulation) signals can be extended into the video domain.

For example, to go from conventional video to a one-bit representation, the video signal can be put through a video standards-converter to increase its frame rate to that of the one-bit representation, and then the pixel signals (the signals corresponding to the values of each pixel as a function of time in the upconverted video stream) put through a delta-sigma modulator. To go from a one-bit representation to a conventional video representation, successive pixel values can be summed over a period corresponding to the length of a frame in the conventional representation, with the resulting pixel signals put through independent low-pass filters.

In colour spaces such as YCrCb where signals can take negative values, those signals may be represented with either one or two bits per sample; in the former case, a signal with a time-average of 0.5 represents the zero point; in the latter case one of the two bits encodes the signal's sign.

Such a two-bit implementation may be substituted wherever reference is made herein to a one-bit encoding.

One-bit video is particularly suited to a novel method of chroma sub-sampling: in colour-space representations that separate chrominance and luminance (e.g. YCrCb), chroma sub-sampling can be implemented by sampling the chroma signals at a lower temporal rate, for example half the sampling rate used for the luminance (or luma) signal. This can be combined with conventional spatial chroma sub-sampling techniques to obtain a further advantage. However, the above described temporal chroma sub-sampling technique may also be used with multi-bit sampled high frame rate signals (e.g. using conventional bit depths).

Conversion between colour spaces ("matrix conversions") in the one-bit video domain that involve multiplication by non-integer scaling factors (e.g. -20 -RGB->YCrCb) may be implemented on a sample-by-sample basis using a probabilistic multiplication method, in which the weighted sum of the R, G and B samples is used as the probability that the output luminance will be 1, by comparing it to the output of a random number generator.

Compression and delivery of high frame rate video Conventional HDTV and Digital Cinema systems require 1.5Gb/s or more for uncompressed data transfer. This is already substantial, and compression is widely used in television and even film production. HFR systems will require correspondingly more data rate, equal to or greater than the 50Gb/s required by the emerging UHDTV standards. Compression is required in production and post-production, for transfer and storage of video, and for final delivery. Current production/post-production compression systems are intra-frame only, with compression ratios at most 10:1 for high quality video. This is insufficient for HFR video. Conventional video compression systems also provide inter-frame compression by means of motion compensation but such systems would also be limited by the very large number of motion vectors required: a set of vectors for every frame of video.

Embodiments may therefore include HFR compression systems using alternate techniques based on 3-dimensional transforms, exploiting the very great similarity between frames (in general, the higher the frame rate, the more similar adjacent frames are likely to be). An example is illustrated in Figure 6. Here an HFR source 99 provides frames of video, which are buffered in a video frame buffer 100. Data is extracted from the frame buffer in three dimensional blocks of samples for each video component (Y, U or V, or R, G or B) by the sample blocker 101. Each block may then be transformed into the frequency domain and quantised by means of a 3-D transform and quantiser 103 and then passed to an entropy coder 104. The transform used in 103 may be a wavelet transform, or a Discrete Cosine Transform or a Lapped Orthogonal transform or some other transform. The blocks of samples may overlap, spatially or temporally, with other blocks and a window function (for example a raised cosine function) may be applied to the blocks prior to quantisation and coding.

-21 -The system may be enhanced by using prediction from previously coded data, for example from spatially neighbouring samples or samples from other frame-groups, either earlier or (if frames are re-ordered prior to coding) later in time. Here prediction data is subtracted from 3D data to be coded by a subtractor 102. After transform and quantisation an inverse 3D transform and quantisation block 105 reconstructs the coded coefficients, and the prediction initially subtracted from the block is added back in by an adder 106. The reconstructed samples are then fed into a sample deblocker 107 and thence into a buffer of reconstructed video frames 108, from which further predictions may be made. A 3D prediction estimator 109 produces a prediction vector for a block from data currently in the input buffer 100 and reconstructed data in the prediction buffer 108, and the 3D prediction generator 110 produces a prediction block from the previously coded samples and the prediction vector, which will be subtracted from the latest set of sample blocks.

A possible prediction method is illustrated in Figure 7, whereby reconstructed neighbouring samples in space and time are all available for use as a prediction. A vector may extrapolate these values in any combination of vertical, horizontal and temporal directions to obtain a prediction block for the rectangular block of uncoded coefficients.

The above-described compression method and system may be provided as an independent aspect of the invention.

Effects processinq and downsarnpling of high frame rate video In the above examples, the video signal is captured, processed and transmitted to the end user at the high frame rate.

In alternative embodiments, the video signal may be converted to a lower frame rate signal prior to transmission, with the higher frame rate signal used for video processing/editing. For example, a lower frame rate can be achieved by combining captured frames. Additional motion information generated from the high frame rate source signal can be used to improve the compression of the signal.

Thus, one embodiment provides a television system that captures video at a substantially over-sampled frame rate but then down-samples to -22 -produce video at a conventional television frame/field rate (the target frame rate), such as 50Hz or 60Hz.

Such a system affords improved video quality and also the advantage of allowing a range of effects to be achieved in post-production, including effects which would conventionally be achieved using the shutter of the camera apparatus. The process is illustrated in Figure 4.

A high frame rate (H FR) signal is obtained from an HFR source 80 (at a source frame rate), such as from a camera or from storage. A video effect (for example a shuttering effect or lighting effect as described in more detail below) is then applied to the signal by effects processor 82. A downsampler 84 generates a video output signal 86 at a target frame rate from the effects-processed HFR signal.

By processing effects at the high frame rate, visual quality can be improved (even if the signal is subsequently down-sampled).

Instead of downsampling the signal, the processed signal could also be output to the end-user in HFR format as described in relation to the system of Figure 1.

Some examples of effects will now be described.

Shutterinci effects As one example, the effect of camera shuttering can be applied as a post processing step rather than being fixed at the time of capture. This process can be considered as the application of a multi-tap filter in the temporal domain, equivalent to the temporal shape of the desired shutter.

These artistic decisions can be made at leisure during the post production process, rather than having to be made at the time of video acquisition.

The applied shuttering effect could be equivalent to a camera shutter of any duration, less than, equal to, or greater than the duration of a frame at the target frame rate, enabling effects to be achieved such as motion blur, smooth motion, or crisp jagged (highly shuttered) motion. As a further example, temporal aliasing effects, for example such as "wagon wheels", can be selectively eliminated or intentionally introduced.

The applied shuttering effect can be temporally shaped by using a non-rectangularly shaped temporal filter, making possible shuttering effects not currently realisable with existing camera technology.

-23 -The applied shuttering effect can be applied differentially across a scene, allowing, for example, an object of interest to appear crisp whilst the remainder of the scene appears blurred even though both may contain similar rates of motion.

Noise reduction Temporal filtering can be applied differentially across a scene to vary the amount of noise reduction applied to different parts of the scene.

Such a temporal filter can also be generalised to a three dimensional convolution function to be applied to the video sequence, where the third dimension is temporal. The convolution can be varied across the scene. The convolution to be applied at any given point on any given moment could be shaped to integrate along the path of motion present at that location. The effect of such a convolution is to apply filtering, such as noise reduction, along the path of motion.

Certain lighting-related effects can also be achieved using high frame rate video processing as will now be described.

Removal of Licihtinq Flicker A frequent problem with television systems where the frame rate differs from that of the mains electricity is that the brightness of the lighting varies between one frame and the next. If the frame rate is close to, but different from, the mains frequency, this may cause a beat between the two, causing a noticeable brightness flicker on the picture.

In a simple high frame rate television system, this flicker may be present but at too high a frequency to be noticeable to the human eye, but it may reduce the efficiency of a bit-rate reduction system. The removal of this effect is therefore desirable, and the following describes how it may be achieved.

Additionally, this technique can be used to ensure that there is no flicker when video material shot under lighting at one mains frequency (for example, 50Hz) is converted for transmission or display at another rate (for

example, 60Hz).

To remove lighting variation, if the video has been captured at a frame rate significantly higher than the lighting flicker rate, the lighting level may be determined by calculating the mean luma level over certain parts or all of the -24 -image, and by means of a suitable band-pass filter on a signal representing the varying mean level, a correction signal may be derived which when subtracted from all the luma values in that certain part or whole image will remove the periodic variation in luma caused by the fluctuations in illumination. The low-pass element of the band-pass filter will remove changes in the real scene. The high-pass element will remove noise or other unwanted components from the signal.

Identifyinci liqht sources The high speed continuous flicker of some sources of illumination, such as fluorescent lighting, is not normally perceivable by the human eye and brain or by conventional television systems. However with a sufficiently high over-sampling frame rate (such as a frame rate that is twice as fast as the flicker rate, or faster) this can be detected by a television system and used to distinguish elements of the image illuminated by such a light source. Several sources can potentially be distinguished if their characteristic flicker varies with respect to the other sources in phase and or frequency.

Filtering can also be applied to separate out the components of a scene's brightness due to the flicker of a particular illumination source, and then subtract that from the scene, or add it back in whilst also multiplying it by an arbitrary scalar. In the final down-sampled video, this will create the effect of selectively removing a particular light source from the scene or varying its apparent brightness.

Instantaneous high speed flicker, such as the flash from flash photography, can be detected and filtered out by applying the same techniques whilst introducing few visual artefacts.

Improved compression of conventional video By using an HFR source signal, more accurate motion vectors can also be computed for a video sequence to improve the efficiency of motion based inter-frame video compression of the down-sampled (conventional frame rate) video. The process is illustrated in Figure 5.

An HFR signal is obtained from an HFR source 90. Motion vector information is obtained by motion analysis 94. The HFR signal is also temporally downsampled by down-sampler 92 to produce a sequence of -25 -images at the target frame rate. The sequence of images at the target frame rate is compressed by compressor 96, using the motion vector information derived from the HFR signal to perform interirame compression, to produce compressed output 98. Any suitable known compression algorithms (e.g. an MPEG-based algorithm) may be used and modified appropriately to make use of the motion vector information derived from the HFR signal.

As an alternative or in addition to using motion information to improve compression of a temporally downsampled signal, motion information derived from the HFR signal may also be transmitted together with the low frame rate signal to a receiver (e.g. set-top box). The receiver can then generate an output signal at a frame rate which is higher than the transmission frame rate using the motion information. Specifically, the receiver can generate additional frames using the motion information. The output signal is then displayed e.g. on a television. A receiver not adapted or configured to provide a higher frame rate signal may simply ignore the motion information and just output the low frame rate signal as received, thus maintaining compatibility with standard equipment.

Embodiments of the invention thus provide an improved conventional television system incorporating a camera that captures at a frame rate substantially higher than that used by a conventional television system, together with various production apparatuses that manipulate or utilise this high frame rate video, but which eventually down-sample temporally back to a conventional television frame rate.

Such production apparatuses may, for example, include: * An apparatus that temporally down-samples back to the frame rate of conventional television and applies a video compression scheme that includes inter-frame motion based compression techniques, where the required motion vectors needed for such a compression scheme are computed from the temporally over-sampled video.

* the application of multi-tap filtering in the temporal domain, differentially across the image, when down-sampling to a conventional television frame rate, to synthesise a variety of -26 -camera shuttering effects, including many not achievable using an ordinary physical camera's shutter.

* the application of temporal filtering to distinguish objects or regions of video images where elements of the scene are illuminated by different light sources by detecting the characteristic flicker rate of the light source, whilst also removing that flicker when temporally down-sampling.

* the application of temporal filtering to separate out the aspects of an image's brightness due to a particular light source with a characteristic rate and/or phase of flicker, and then scale that component arbitrarily.

* the application of temporal filtering to separate out the aspects of an image's brightness due to a short instantaneous flash from a particular light source, such as flash photography, allowing that component to be scaled arbitrarily or completely removed.

* An apparatus that applies a three dimensional convolution function to a video sequence where the particular function is shaped to filter along the trajectory of motion present at any particular point Though the effects described above (for example shutter effects or lighting effects) are described in the context of a system which performs downsampling after the effects have been applied, the described effects can also be implemented in a system in which video is provided to the end user at the high frame rate at which it is captured and processed, as previously described.

In summary, embodiments of the invention provide television/film capture, production and/or transmission systems operating at a frame rate which is substantially higher than existing conventional television systems.

Video processing and editing, and in particular effects processing, may be performed at the substantially higher frame rate. The video format used by the system may use a bit depth of one bit. The high frame rate video may be down-converted to standard frame rates for transmission, or may be -27 -transmitted at the high frame rate to suitably modified or specially designed end-user equipment. Motion information determined for a high frame-rate signal can be used to improve compression of a corresponding lower frame rate signal or to enable reproduction by a receiver of a higher framer rate signal from a lower frame rate signal.

Embodiments of the invention can provide the following advantages:- * Better representation of motion and sharper moving detail (possibly to the point where there is little or no difference between the rendition of static and moving detail) and hence a more realistic look due to the much higher frame rate compared to conventional television and cinema systems.

* Improved sense of three-dimensionality through sharper moving edges and more solid occlusion.

* A reduction of temporal aliasing due to the much higher frame rate compared to conventional television systems, reducing the "backwards-rotating wagon-wheel" and similar effects.

* Reduction of the perceptible flicker, judder and jerkiness associated with current TV and film frame/field rates and hence an expected reduction in related physiological and medical conditions such as photosensitive and pattern-sensitive epilepsy, eye-strain, headaches and nausea.

* Full backwards compatibility with conventional television cameras and displays can be achieved, albeit possibly by sacrificing to some extent the aforementioned better representation of motion.

* Simpler conversion to both 5Ofps and 6Ofps conventional television formats, making complicated standards-converters unnecessary.

* Increased tolerance of noise within individual frames, permitting an optional reduction in the number of bits used to represent each pixel without reducing the perceived image quality of the video.

-28 - * Lower delay in video processing systems such as standards-converters, graphics and effects units.

* Improved lip-sync * Increased artistic freedom for directors and camera operators, allowing the use of shots made impossible at present by the current frame/field rates of television and cinema, such as fast pans and scenes containing fast-moving objects moving across the image.

It will be understood that the present invention has been described above purely by way of example, and modification of detail can be made within the scope of the invention.

Claims (63)

1. -29 -CLAIMS1. A television system adapted to process a television signal having a frame rate substantially higher than conventional television or film frame rates.
2. 2. A system according to claim 1, wherein the frame rate is substantially higher than one or more (preferably each) of: PAL frame or field rate, NTSCframe or field rate, and standard film frame rate.
3. 3. A system according to any of the preceding claims, wherein the frame rate is at least 8Ofps, preferably at least lOOfps more preferably at least 1 5Ofps.
4. 4. A system according to any of the preceding claims, wherein the frame rate is at least 300fps, preferably at least 600fps, more preferably at least 1 200fps.
5. 5. A system according to any of the preceding claims, wherein the frame rate is a multiple of one or more selected conventional television frame or field rates or film rates, preferably wherein the frame rate is a multiple of each of a plurality of selected conventional television frame or field rates or film rates.
6. 6. A system according to claim 5, wherein the frame rate is a multiple of one or more, preferably each, of: 25fps PAL frame rate, 3Ofps approximate NTSC rate and 24fps standard film rate; more preferably wherein the frame rate is a multiple of one or more, preferably each of: 50 fields-per-second PAL field rate, 60 fields-per-second approximate NTSC field rate, and 24fps standard film rate.
7. 7. A system according to any of the preceding claims, wherein the frame rate is 600fps or a multiple thereof.
8. 8. A system according to any of the preceding claims, wherein the signal defines a sequence of images displayable at the frame rate.

   -30 -
9. 9. A system according to any of the preceding claims, wherein the signal is a digital television signal.
10. 10. A system according to claim 9, wherein the signal is encoded with a bit depth of at most two bits per colour component, preferably one bit per colour component.
11. 11. A system according to any of the preceding claims, comprising means for providing source video at the high frame rate, the providing means preferably including one or more cameras operable to capture source video at the high frame rate.
12. 12. A system according to any of the preceding claims, comprising means for performing video editing or video processing at the high frame rate.
13. 13. A system according to any of the preceding claims, comprising means for performing video effects processing on the high frame rate signal to add a video effect to the high frame rate signal, preferably a shuttering effect or a lighting effect.
14. 14. A system according to any of the preceding claims, comprising means for transmitting the high frame rate signal to end user equipment, the end user equipment preferably adapted to output the signal at the high frame rate.
15. 15. A system according to any of the preceding claims, comprising means for compressing the signal using three-dimensional block-based coding, preferably using temporal and/or spatial prediction of three-dimensional blocks of video data.
16. 16. A system according to any of the preceding claims, comprising means for converting the television signal at the high frame rate to a low frame rate signal, preferably at a standard television frame rate or film rate.

    -31 -
17. 17. A system according to claim 16, comprising means for transmitting the low frame rate signal to end user equipment.
18. 18. A system according to claim 16 or 17, comprising means for deriving information for use in compression from the high frame rate signal, and means for compressing the low frame rate signal using the derived information.
19. 19. A system according to claim 18, comprising means for deriving motion information from the high frame rate signal and means for compressing the low frame rate signal using the derived motion information.
20. 20. A system according to any of claims 16 to 19, comprising means for deriving motion information from the high frame rate signal, and means for transmitting the motion information with the low frame rate signal to a receiver.
21. 21. A system according to clam 20, wherein the receiver is adapted, using the received motion information, to output a signal at a frame rate higher than the low frame rate at which the signal was transmitted, preferably at the original high frame rate.
22. 22. Apparatus for processing a television signal having a frame rate substantially higher than conventional television or film frame rates.
23. 23. A method of providing a television signal having a frame rate substantially higher than conventional television or film frame rates.
24. 24. A video editing system comprising means for editing video at a frame rate substantially higher than conventional television or film frame rates.
25. 25. A method of providing a digital video signal, comprising encoding the signal with a bit depth of at most two bits per colour component, preferably using one bit per colour component or one bit plus a sign bit.

    -32 -
26. 26. A method of providing a digital video signal, comprising encoding the signal with a bit depth of one bit per colour component.
27. 27. A method according to claim 25 or 26, wherein the signal has a frame rate substantially higher than conventional television or film frame rates.
28. 28. A method according to any of claims 25 to 27, wherein the signal has a frame rate which is sufficiently high so that display of the signal at the high frame rate produces the effect of a full colour image on a viewer.
29. 29. A method according to any of claims 26 to 28, wherein the signal has a frame rate of at least 1 0,000fps, more preferably at least 1 00,000fps.
30. 30. A method according to any of claims 25 to 29, further comprising converting between the one-bit video format and a video format using multiple bits per colour component, preferably a standard television or film format.
31. 31. A camera adapted to record a digital video signal with a bit depth of one bit per colour component.
32. 32. A display adapted to display a digital video signal having a bit depth of one bit per colour component.
33. 33. A video editing system comprising means for editing video encoded with a bit depth of one bit per colour component, preferably at a frame rate substantially higher than conventional television or film frame rates, preferably at least 10,000fps, more preferably at least 100,000fps.
34. 34. A video conversion system comprising means for converting between a first video format having a high frame rate and a bit depth of one bit per colour component and a second video format having a lower frame rate and a bit depth of multiple bits per colour component.

    -33 -
35. 35. A video conversion system according to claim 34, wherein the second video format is a standard television or film format, preferably a PAL or NTSC television format.
36. 36. A method of providing a compressed video signal, comprising: receiving a video signal at a first frame rate; deriving information for use in compression from the video signal at the first rate; converting the video signal at the first frame rate to a video signal at a second frame rate different from the first frame rate; and compressing the video signal at the second frame rate using the derived information.
37. 37. A method according to claim 36, wherein the second frame rate is lower than the first frame rate.
38. 38. A method according to claim 36 or 37, wherein the information is motion vector information for use in interirame compression of the video signal.
39. 39. A method according to any of claims 36 to 38, wherein the video signal is encoded using a bit depth of one bit per colour component.
40. 40. A method of transmitting a video signal, comprising: receiving a video signal at a high frame rate; deriving motion information from the high frame rate video signal; converting the high frame rate video signal to a low frame rate video signal; transmitting the low frame rate video signal and the motion information to a receiver; and at the receiver, deriving from the received low frame rate signal a signal at an output frame rate higher than the low frame rate at which the signal was transmitted using the received motion information, and outputting the derived signal at the output frame rate.

    -34 -
41. 41. A method according to claim 40, wherein the output frame rate is equal to the high frame rate.
42. 42. A method of processing video, comprising: receiving a video signal at a high frame rate, the high frame rate being substantially higher than conventional television or film frame rates; and performing video processing at the high frame rate to produce a processed video signal at the high frame rate.
43. 43. A method according to claim 42, further comprising: converting the processed video signal to a low frame rate lower than the high frame rate, preferably a standard television or film frame rate; and outputting the low frame rate processed video signal.
44. 44. A method according to claim 42 or 43, wherein the step of performing video processing comprises performing video editing.
45. 45. A method according to any of claims 42 to 44, wherein the step of performing video processing comprises performing effects processing, preferably to apply a shuttering or lighting effect.
46. 46. A method according to any of claims 42 to 45, wherein the step of performing video processing comprises applying a synthetic camera shuttering effect to the video signal.
47. 47. A method according to claim 46, wherein the shuttering effect is applied differentially across the video image.
48. 48. A method according to any of claims 42 to 47, wherein the step of performing video processing comprises detecting the effect on a scene caused by a variable light source based on the characteristic rate and/or phase of variation of the light source.

    -35 -
49. 49. A method according to claim 48, wherein the step of performing video processing comprises identifying regions or objects of one or more video images where elements of the scene are illuminated by a given light source by detecting the characteristic rate and/or phase of flicker of the light source.
50. 50. A method according to claim 48 or 49, comprising modifying one or more video images or identified objects or regions thereof to modify, reduce or remove illumination caused by the light source.
51. 51. A method according to any of claims 48 to 50, comprising distinguishing between multiple light sources based on respective flicker rates or phases of the light sources.
52. 52. A method according to any of claims 48 to 51, comprising processing the video image to remove a short instantaneous flash from a particular light source.
53. 53. A method according to any of claims 42 to 52, wherein the step of performing video processing comprises applying a three dimensional convolution function to the video sequence, wherein the convolution function is shaped to filter along the trajectory of motion present in the video sequence.
54. 54. A method of processing video to modify the contribution to a scene from a regularly varying light source, the method comprising: capturing the video at a frame rate which is greater than the rate of variation of the light source; detecting a contribution to one or more frames of video due to the light source based on the rate and/or phase of variation of the light source; and modifying one or more of the frames of video based on the detected contribution.
55. 55. A method according to claim 54, comprising modifying the frame or frames to increase, reduce or remove the detected contribution due to the light source.

    -36 -
56. 56. A method of compressing a video signal having a frame rate substantially higher than conventional television or film frame rates, the video signal having two spatial dimensions and a temporal dimension, the method comprising: dividing the signal into a plurality of three-dimensional blocks of video data; and encoding the three-dimensional blocks.
57. 57. A method according to claim 56, wherein encoding a block comprises calculating a frequency domain transform of the block; and quantising the resulting transform coefficients.
58. 58. A method according to claim 56 or 57, comprising calculating prediction information for a block, and encoding the block using the prediction information.
59. 59. A method according to claim 58, comprising performing spatial and/or temporal prediction.
60. 60. Video processing apparatus adapted to perform a method as claimed in any of claims 25 to 30 and 36 to 59.
61. 61. A computer program or computer program product comprising software code adapted, when executed on a data processing apparatus, to perform a method as claimed in any of claims 25 to 30 and 36 to 59.
62. 62. A television system or video processing apparatus substantially as herein described with reference to and/or as illustrated in any of the accompanying drawings.
63. 63. A method of providing or processing a video signal substantially as herein described with reference to and/or as illustrated in any of the accompanying drawings.