GB2541713A

GB2541713A - Processing of high frame rate video data

Info

Publication number: GB2541713A
Application number: GB1515254.9A
Authority: GB
Inventors: Rowan Graham
Original assignee: Individual
Current assignee: Individual
Priority date: 2015-08-27
Filing date: 2015-08-27
Publication date: 2017-03-01
Also published as: GB201515254D0

Abstract

A method and apparatus for processing video data comprising: capturing data, using an image sensor (12, Fig.1), of a scene with region of high interest 32 and of low interest 34; transmitting and/or storing the video data where the data representing the region of high interest has a higher frame rate (e.g. 180fps) and the region of low interest has a lower frame rate (e.g. 60fps); compiling (combining) the data to give video data representing areas of both high and low interest. Each frame of the transmitted and/or stored data may contain both data representing the region of high interest and a portion 34a 34b 34c of the region of low interest or frames may alternate between the two. The compiled data has a frame rate higher than the data of the region of low interest; this may be achieved by duplicating data at the low frame rate and then merging with the high frame rate data.

Description

PROCESSING OF HIGH FRAME RATE VIDEO DATA

The present invention relates to the processing of data produced during high frame rate (HFR) video photography.

Most video systems capture images at frame rates of between 24 and 60 frames per second (fps). HFR cameras capture images at rates of several hundred to many thousands of fps, and typical frame rates may be around 1,000 fps.

The effect of slow motion is produced when a video captured at a high frame rate is played back at standard frame rates. Hence, such cameras are also sometimes known as slow motion cameras. Slow motion video is used in a diverse range of applications, and examples include scientific research, where a high speed event needs to be slowed down for analysis, analysis of the movements of athletes, and for special effects in movies and promotional videos.

Most HFR cameras today use CMOS or CCD image sensors, which capture the image as an array of picture elements (pixels) that are then transferred to a digital memory system for storage. This memory system could be a high speed local memory or external storage media, such as hard disk drives or solid state storage media.

Compared to normal frame rate video photography, HFR video photography requires correspondingly higher data transfer rates from the image sensor to the storage system, and higher write speeds at the storage system. Higher data transfer rates and write speeds require faster data paths and memory systems, and hence HFR video cameras often cost significantly more than standard cameras.

Furthermore, in recent years, the demand for higher resolution video cameras has increased. For example, the resolution of video cameras has increased from PAL (576 lines) to HD (1280 x 720) and then to FHD (1920 x 1080). Furthermore, 4K (3840 x 2160) systems are starting to appear, and 8K (7680x4320) systems are soon expected to be introduced.

The performance of an image capture system can be described in terms of bandwidth (bits of data per second flowing from the sensor to the memory). The bandwidth increases in proportion to frame rate and resolution.

Required Bandwidth (bits per second) oc frame rate (fps) x resolution (pixels)

Thus, compared to a standard HD camera capturing at 60fps, a HFR camera capturing at 4K resolution and 1,000 fps requires a bandwidth almost 100 times higher. There are very few cameras on the market today that are capable of supplying this bandwidth, and those that can are very expensive.

One way to reduce the bandwidth requirement, and hence cost, for a HFR camera is to capture the slow motion elements of a video at a lower resolution, and then digitally edit these parts of the video into the full video post-production. To do this, the low resolution, HFR clip needs to be upscaled to full resolution, which produces blurring or other artefacts.

Other ways of reducing bandwidth requirements involve compressing the image, which reduces the image quality - compressed image data loses some of the image information as decisions are made about the final colour balance and dynamic range before saving the image. Raw images are easy to edit and adjust in post-production, whereas compressed images are less so.

It is thus desirable to reduce the bandwidth requirements of a HFR camera without reducing the resolution of the captured image or compressing the frames in a manner causing loss of image data.

The present invention provides a method of processing video data, comprising: capturing video data using an image sensor, the video data representing a scene having a region of high interest and a region of low interest; transmitting and/or storing the video data, wherein the transmitted or stored data representing the region of high interest has a first, higher frame rate and the transmitted or stored data representing the region of low interest has a second, lower frame rate; and compiling the transmitted or stored data to give compiled video data representing both the region of high interest and the region of low interest, the compiled video data having a higher frame rate than the second, lower frame rate.

For the avoidance of doubt, reference to the frame rate of video data should be understood to refer to the “natural” frame rate of the video data, i.e. the playback frame rate required for the video data to appear at natural/actual speed. The natural frame rate of video data will initially correspond to the capture frame rate, but can be changed. For example, it can be decreased by dropping frames, or increased by adding frames, such as by duplication or interpolation. Other techniques include periodically sampling the video data at a higher or lower frame rate to produce new video data having a different natural frame rate.

This method takes advantage of the fact that, usually, the subject of a video will not fill the whole frame - there is usually some context for the subject. With the conventional method of decreasing resolution, either the resolution of the whole frame is reduced causing blurring, or the frame size may be reduced to make the subject fill the full frame to avoid loss in resolution. Thus either the context must be lost or resolution must suffer.

With this method, a high frame rate video can be captured without decreasing the resolution because the data rate required for storing or transmitting the video data is instead decreased by reducing the frame rate of the region of low interest, which would usually be expected to have a lower rate of change. The loss of data here Is thus less noticeable.

This technique is particularly applicable to high frame rate photography, due to the correspondingly higher data rates required for storage and transmission of the video data. Thus, preferably, both the first (higher) frame rate and second (lower) frame rate are above 60 fps. Preferably, the first, higher frame rate is at least 180fps.

In various embodiments, the first, higher frame rate is an integer multiple of the second, lower frame rate. Preferably, each frame of transmitted or stored video data contains approximately the same number of pixels (e.g. +/-10%).

In one embodiment, each frame of transmitted and/or stored video data may contain both data representing the region of high interest and data representing only a portion of the region of low interest.

In another embodiment, the frames of transmitted and/or stored video data alternatingly contain data representing the region of high interest and then data representing only a portion of the region of low interest.

In both cases, the portion of the region of low interest may be one of a predetermined number of tiles of the region of low interest. Preferably, each of the tiles are sequentially transmitted.

The method need not necessarily require the compiling step. Thus, in another aspect, the present invention also provides a method of capturing video data, comprising: capturing video data using an image sensor, the video data representing a scene having a region of high interest and a region of low interest; and transmitting or storing the captured video data from the image sensor, wherein the transmitted or stored frames of the video data each contain both data representing the region of high interest and data representing only a portion of the region of low interest, or alternatingly contain data representing the region of high interest and then data representing only a portion of the region of low interest.

The methods may further comprise dividing the region of low interest into a plurality of tiles, wherein each frame of the transmitted or stored video data containing data representing a portion of the region of low interest contains data representing one of the tiles of the region of low interest.

Preferably, the tiles of the background are sequentially transmitted or stored, such that each tile of the region of low interest is transmitted before restarting the sequence. Thus, the entire region of low interest is transmitted before the sequence restarts.

Where the method does include the compiling step, the compiling may comprise increasing the frame rate of the video data representing the region of low interest, for example by duplication of the data in a subsequent frame, and merging the video data representing the region of low with the video representing the region of high interest.

Thus, the natural frame rate of the data representing the region of low interest may be increased to match the natural frame rate of the data representing the region of high interest (or an intermediate frame rate, if desired). This could be done using simple techniques, such as duplication of frames, or by more complex techniques such as interpolation between subsequent frames.

For example, the compiling may comprise: receiving the video data at a buffer memory; and, for each frame received, overwriting corresponding portions of data stored in the buffer memory representing the scene with the data contained in the transmitted frame representing the region of high interest and/or the portion of the region of low region, such that the buffer memory always contains data representing a complete frame, wherein the region of high interest is updated at a first, higher frame rate and portions of the region of low region are updated at a second, lower frame rate.

As can be seen, the buffer memory will always contain a complete frame. The method may thus further comprise sampling the buffer memory at a frame rate higher than the second (lower) frame rate to give compiled video data, i.e. the compiled video data having a natural frame rate higher than the second frame rate. This technique effectively duplicates frames of data representing the region of low interest to create compiled video data having a high frame rate.

In various embodiments, the region of interest may comprise multiple, discrete sub-regions within a frame. That is to say, the region of interest is not limited to any particular shape or to a single, connected group of pixels within the frame, but may include any selection of pixels within the frame. For example, in the case of an athlete throwing a ball, both the athlete and the ball may each have their own, separate, sub-region of the region of interest. The entire selection of pixels forming the region of interest is then captured and stored or transmitted at the higher frame rate.

In other embodiments of the method, the region of high interest may be one of a plurality of regions of high interest. For example, the frame may include (at least) the region of low interest, the first region of high interest, a second region of high interest. The transmitted or stored data representing the second region of high interest may have a third frame rate, different from the first frame rate and higher than the second, lower frame rate. Thus, different regions of interest may have different degrees of interest. For example, in the case of the athlete throwing a ball, the ball may move faster than the athlete and thus a region of interest including the ball may be captured at a higher frame rate than a region of interest including the athlete.

Viewed from another aspect, the present invention can further be seen to provide a method of compiling video data representing a scene having a region of high interest and a region of low interest, comprising: receiving video data at a buffer memory, wherein the frames of the video data each contain both data representing the region of high interest and data representing only a portion of the region of low interest, or alternatingly contain data representing the region of high interest and then data representing only a portion of the region of low interest; and compiling the transmitted video data by, for each frame, overwriting corresponding portions of data stored in the buffer memory representing scene with the data contained in the transmitted frame representing the region of high interest and/or the portion of the region of low interest, such that the buffer memory always contains data representing a complete frame, wherein the region of high interest is updated at a first, higher frame rate and portions of the region of low region are updated at a second, lower frame rate.

The methods may, for example, further comprise displaying the compiled video data at a frame rate at or below 60fps to cause a slow motion effect. Alternatively, or in addition, the method may comprise storing the compiled video data for later processing, wherein each frame of the compiled video data is a complete frame. Thus, this method can be used either for displaying video data produced by the methods above, or for converting said video data into a more conventional format where each frame of video data is a complete frame representing the entire scene.

The present invention can also be seen to provide a high frame rate video camera (for example, capable of capturing video data at over 60fps, and preferably over 180fps) comprising an image sensor and a processing device, the image sensor being configured to capture video data representing a scene having a region of high interest and a region of low interest, and the processing device being configured to produce video data for transmission or storage, wherein the frames of the video data for transmission or storage each contain both data representing the region of high interest and data representing only a portion of the region of low interest, or alternatingly contain data representing the region of high interest and then data representing only a portion of the region of low interest.

The present invention also provides an apparatus comprising a processor and a memory containing computer-readable instructions, or a computer program product containing computer-readable instructions for controlling an apparatus comprising a processor, the computer-readable instructions being for causing the processor to perform a method of compiling video data representing a scene having a region of high interest and a region of low interest, the method comprising: receiving video data at a buffer memory, wherein the frames of the video data each contain both data representing the region of high interest and data representing only a portion of the region of low interest, or alternatingly contain data representing the region of high interest and then data representing only a portion of the region of low interest: and compiling the transmitted video data by, for each frame, overwriting corresponding portions of data stored in the buffer memory representing scene with the data contained in the transmitted frame representing the region of high interest and/or the portion of the region of low interest, such that the buffer memory always contains data representing a complete frame, wherein the region of high interest is updated at a first, higher frame rate and portions of the region of low interest are updated at a second, lower frame rate.

The present invention yet further provides, in one aspect, video data comprising a plurality of sequential frames, the video data representing a scene having a region of high interest and a region of low interest, wherein each frame contains both data representing the region of high interest and data representing only a portion of the region of low interest.

Another aspect of the invention provides video data comprising a plurality of sequential frames, the video data representing a scene having a region of high interest and a region of low interest, wherein the frames alternatingly contain data representing the region of high interest and then data representing only a portion of the region of low interest.

Preferably, the video data in these aspects is high frame rate video data, i.e. having a (natural) frame rate of over 60fps, and preferably over 180fps.

Video data in these formats is more compact than video data stored in more conventional formats where each frame contains data representing the entire scene. Thus, these formats are more suitable for storage or transmission of video data, and particularly high frame rate video data, which can otherwise encounter difficulties due to the high bandwidth requirements.

Certain preferred embodiments of the present invention will now be described in greater detail, by way of example only and with reference to the accompanying drawings, in which:

Figure 1 is a schematic drawing of a high frame rate video camera;

Figure 2 is a schematic drawing of an apparatus for replaying high frame rate video in slow motion;

Figure 3 illustrates regions within a frame of video data;

Figures 4A to 4C illustrate a series of sequential frames of the captured video data;

Figures 5A to 5F illustrate an alternative series of sequential frames of the captured video data; and

Figures 6 and 7 illustrate respective regions of interest within exemplary frames of different videos.

Figure 1 is a schematic diagram of a high frame rate (HFR) video camera 10. The camera 10 includes an image sensor 12 for capturing video data comprising sequential frames of a scene, a processing portion 14 for processing the video data, and a memory portion 16 for storing the video data.

The image sensor 12 may be any suitable image sensor 12, such as a CMOS or CCD image sensor 12. The memory portion 16 in this embodiment is a removable memory device 16, such as a solid state device.

Figure 2 is a schematic diagram of an apparatus 20 for replaying video data captured by the HFR video camera 10. The apparatus 20 comprises a memory system 22 storing video data, which may be the same memory system 16 as in Figure 1, a processing portion 24 for re-compiling the video data, and a display device 26 for displaying the video.

Figure 3 illustrates an exemplary frame 30 captured by the camera 10. The frame 30 has a 4K full-frame resolution (3840 x 2160), which is approximately 8 million pixels.

Within the frame 28, a region of interest (ROI) 32 is identified having a size equivalent to FHD (1920 x 1080), which is about 2 million pixels. The background region 34 is divided into three equal-area tiles 34a, 34b, 34c of about 2 million pixels each. Thus, the background 6 million pixels plus the 2 million ROI pixels make up the full 8 million pixels of the frame 30.

In this example, the camera 10 can render FHD video data (about 2 million pixels per frame) at 1,000 fps and hence 4K video data (about 8 million pixels per frame) at about 250 fps. These figures are typical of a mid-range HFR camera 10.

For each captured frame 28, the camera 10 captures the ROI 32 plus one of the background tiles 34a-c, for a total of about 4 million pixels per frame 30 for storage. Thus, the camera 10 can render this reduced area of the frame at 500 fps (half the area of the frame, at twice the original frame rate). For example, for each captured frame 28, the image sensor 12 may capture an entire scene, and the processor 14 may then sample only the desired regions to create the frame 30 for transmission and/or storage.

On each sequential frame 30, a different background tile 34a, 34b, 34c is captured, but the ROI 32 is captured on every frame 30. For example, on a first frame 30a, the ROI 32 and the first background tile 34a are captured (Figure 4A), on a second frame 30b, the ROI 32 and the second background tile 34b are captured (Figure 4B), and on a third frame 30c, the ROI 32 and the third background tile 34c are captured (Figure 4A). After three frames 30a-c, the entire background 34 has been captured once, and the ROI 32 has been captured three times.

Thus the effective frame rate of ROI 32 capture is 500fps and the effective frame rate of background 34 capture is 166fps (500fps / 3). This is still a high frame rate for the background 34, which typically does not include any fast moving features, but as can be seen, the frame rate for the ROI 32 has been doubled compared to conventional techniques, without loss of resolution.

For a higher ROI 32 frame rate, the background 34 could be divided into a greater number of smaller tiles. For example, dividing the background 34 into ten tiles, would capture 2.6 million pixels per frame (2 million ROI pixels and 0.6 million background pixels per frame) and thus achieve an effective rate of 770 fps for the ROI 32 and 77 fps for the background 34. This approaches the performance of the current, top tier HFR video cameras 10 for at least the region of interest 32.

Alternatively, a smaller ROI 32 could be used to achieve even higher frame rates. For example, using a ROI 32 of about 1 million pixels (equivalent to HD video - 1280 x 720), the camera 10 could achieve an effective frame rate of 1,000 fps for the ROI 32 and 140 fps for the background 32, when using seven background tiles.

In another embodiment illustrated in Figures 5A to 5F, for each frame 28, the camera 10 captures, the frame 30 for storage alternatingly contains either the ROI 32 or one of the background tiles 34a-c, for a total of about 2 million pixels per frame 30. Thus, the camera 10 can render this reduced area 30 of the frame 28 at 1000 fps (a quarter the area of the full frame size, at four times the original frame rate). As above, for each frame 28, the image sensor 12 may capture an entire scene, and the processor 14 may then sample only the desired region to create the frame 30 for storage.

On each sequential frame 30 containing a background tile 34a, 34b, 34c, a different background tile 34a, 34b, 34c is captured. For example, on a first frame 30a’ the ROI 32 is captured (Figure 5A), on a second frame 30b’ the first background tile 34a is captured (Figure 5B), on a third frame 30c’, the ROI 32 is again captured (Figure 5C), on a fourth frame 30d’ the second background tile 34b is captured (Figure 5D), on a fifth frame 30e’ the ROI 32 yet again captured (Figure 5E), and on the sixth frame 30f, the third background tile 34c is captured (Figure 5F). In this embodiment, after six frames 30a’-30f, the entire background 34 has been captured once, and the ROI 32 has been captured three times.

Again, this technique achieves an effective frame rate of the ROI 32 of 500fps and of the background 34 of 166fps.

Figure 6 and 7 illustrate exemplary frames 28 of video data. Within each frame 28, a region of interest 32 has been identified. Figure 6 illustrates an exemplary frame 28 having a relatively small region of interest 32, including for approximately 20% of the area of the frame. Figure 7 illustrates an exemplary frame 28 having a larger region of interest 32, including approximately 40% of the area of the frame 28.

As can be appreciated, the area of the video within the region of interest 32 will have a relatively high rate of change compared with the background area 34 outside of the region of interest 32. Thus, the slower update rate of the background area 34 will be less noticeable.

The frames 30 of video data produced by the camera 10 are stored, via a memory interface 18, on a memory storage device 16. In order to reassemble complete frames of video data, a memory storage device 22 is connected to the apparatus 20, which processes the stored video data as follows.

The video data is processed in a frame-by-frame manner. The pixels contained in each frame 30 are rendered into a frame buffer of the processing device 24, which has at least sufficient memory to store a complete frame of the video at full resolution, i.e. 4K in this example.

Each frame 30 is written onto the frame buffer such that, as each frame 30 is rendered, the background tiles 34a-c from previous frames 30 remain in the frame buffer. Thus, a complete image is always maintained in the frame buffer. In the same way, the ROI 32 of each frame is rendered more frequently and overwrites previous ROI 32 pixels in the frame buffer and combines with the rest of the background 34 in a seamless way.

The tiling effect is not very visible as even the background region 34 has still captured at least 60 times per second, which is the frame rate normally selected to provide the illusion of continuous updates from a visual point of view.

The frame buffer is periodically sampled, for example once for every time the ROI 32 is updated, to produce the frames of complied video data. Although the maximum (useful) frame rate is achieved by sampling at the same frame rate as the ROI 32 is updated, lower frame rates could also be used if desired. For example, the frame buffer could be sampled at a frame rate to produce compiled video data having a natural frame rate of, for example, 60fps for display at actual speed on a conventional display device 26.

The complete image contained in the frame buffer may be displayed at any desired frame rate using the display device 26. For example, compiled video data at a high natural frame rate could be displayed at a frame rate of 60 fps to provide a slow motion effect. Alternatively, the sampling frequency could be reduced, or frames could be dropped from the high frame rate compiled video data, to reduce the natural frame rate of the video such that display by the display device 26 at 60fps (or other frame rate) produces a video at natural/actual speeds.

In another embodiment, for example for manipulation of the video data in post-production, the above re-compiling technique may be applied in order to reproduce complete frames of video data that are then stored in a memory so that the recompiled video data may later be processed or displayed using normal video editing or display techniques.

In other embodiments, the data may never be stored as incomplete frames 30, but this format might instead be used simply for transmission. For example, the video data could be captured, the partial frames 30 produced, those frames 30 transmitted, complete frames re-assembled, and the complete frames stored. This may be particularly applicable for transmission of video data via a network having limited bandwidth, but where high speed processing is readily available at the receiving end.

In yet further embodiments, the simple overwriting technique may be replaced by more advanced image processing techniques, such as interpolation of the missing frames. Thus, for example, panning movements of the camera can be determined and the omitted frames of the background portion (which is essentially a static image) can be re-created, further reducing the visibility of the tiling effect.

In yet further embodiments, the ROI 32 can be moved on a frame-by-frame basis to track the high speed motion. For example, automatic movement detection algorithms could automatically identify the areas of the image that are changing at the fastest rate and set the ROI 32 automatically to track them. Alternatively, this could be done under operator control, for example by moving a finger over a touch screen while previewing the image.

The ROI 32 can also be changed dynamically in shape to track the rotation of a subject as illustrated in Figures 6 and 7. This could be again be done by automatic movement detection or manually, such as with gestures on a touch screen, such as using two fingers to show the area of the ROI 32. Indeed, the ROI 32 need not necessarily be a quadrilateral as shown in the drawings, but could be any collection of pixels within the frame 28.

As long as the ROI 32 contains a fixed number of pixels, then the image frame rate will remain constant. Where the size of the ROI 32 can be varied, the processing device 14 may be configured to automatically determine the number of tiles 34a-c into which to divide the background 34 in order to achieve a target frame rate for the ROI 32.

In further embodiments, multiple ROIs 32 could be used to track the movement of multiple subjects within the frame 28.

With most systems of this nature there is a trade-off between resolution and frame rate. For high frame rates, the resolution is often reduced. Thus for 1,000 fps, existing systems might reduce the resolution from 4K (3840 x 2160) to FHD (1920 X 1080). However, when this portion of the video is integrated into a completed production, the resolution of this portion of video must be upscaled to 4K, which causes the pixels to be expanded and will be at a lower resolution compared to the rest of the movie. This also requires much post-processing work and will never produce a seamless result.

The technique described above takes advantage of the fact that, usually, the subject of a video will not fill the whole frame - there is usually some context for the subject - and so this less frequently changing data can be sampled at a lower frame rate, without causing significant impact on the viewer. Also, with the above technique, there is no visual discontinuity between the high frame rate portions of the video and the low frame rate portions of the video, because all of the video data can be captured at the same resolution.

Other ways of reducing bandwidth requirements that are currently employed involve compressing the image, which reduces the image quality. However, heavily compressed image data loses some of this information by making decisions about the final colour balance and dynamic range before saving the image. The above technique allows the use of full quality raw image data (although the technique is not restricted to this format and could make use of lossless compression, for example). Raw image data contains all of the original image information - such as resolution and colour - and raw images are easy to edit and adjust in post-production than compressed images. A considerable saving in cost and image bandwidth is thus achieved by only capturing a relatively small portion of the frame with a high frame rate. This enables a relatively low cost HFR camera to be used instead of more expensive equipment. The cost savings can be orders of magnitude. The cost savings are achieved by: • Reducing the bandwidth performance of the entire system. This reduces component cost, and heat dissipation and memory capacity requirements. • Requiring less memory to save each image. This either requires less memory in the system or allows longer videos to be stored in the same memory space. • Allowing the use of a relatively low cost camera rather than more expensive equipment sourced from a very select number of specialist manufacturers.

This technique further allows filming of the high frame rate sections of a movie to be enabled and disabled seamlessly during filming on the same equipment. It is no longer necessary to switch to a specialist camera to perform the high frame rate filming. High frame rate filming can be switched on instantly, or even automatically, should it be desired to capture a part of the filming in high frame rate without prior warning or planning.

In most cases the image can be framed so that a high frame rate ROI can be selected. Thus this technique can replace the need to buy or hire expensive camera equipment.

Claims

1. A method of processing video data, comprising: capturing video data using an image sensor, the video data representing a scene having a region of high interest and a region of low interest; transmitting and/or storing the video data, wherein data representing the region of high interest has a first, higher frame rate and data representing the region of low interest has a second, lower frame rate; and compiling the transmitted or stored data to give compiled video data representing both the region of high interest and the region of low interest, the compiled video data having a higher frame rate than the second, lower frame rate.

2. A method according to claim 1, wherein the first, higher frame rate is at least 180fps.

3. A method according to claim 1 or 2, wherein the second, lower frame rate is at least 60fps

4. A method according to claim 1,2 or 3, wherein the first, higher frame rate is an integer multiple of the second, lower frame rate.

5. A method according to any preceding claim, wherein the compiling comprises increasing the frame rate of the video data representing the region of low interest, for example by duplication of the data in a subsequent frame, and merging the video data representing the region of low with the video representing the region of high interest.

6. A method according any preceding claim, wherein the compiling comprises: receiving the video data at a buffer memory; and for each frame received, overwriting corresponding portions of data stored in the buffer memory representing the scene with the data contained in the transmitted frame representing the region of high interest and/or the portion of the region of low interest, such that the buffer memory always contains data representing a complete frame, wherein the region of high interest is updated at a first, higher frame rate and portions of the region of low interest are updated at a second, lower frame rate.

7. A method according to any of claims 1 to 6, wherein each frame of transmitted and/or stored video data contains both data representing the region of high interest and data representing only a portion of the region of low interest.

8. A method according to any of claims 1 to 6, wherein the frames of transmitted and/or stored video data alternatingly contain data representing the region of high interest and then data representing only a portion of the region of low interest.

9. A method of capturing video data, comprising: capturing video data using an image sensor, the video data representing a scene having a region of high interest and a region of low interest; and transmitting or storing the captured video data from the image sensor, wherein the transmitted or stored frames of the video data each contain both data representing the region of high interest and data representing only a portion of the region of low interest, or alternatingly contain data representing the region of high interest and then data representing only a portion of the region of low interest.

10. A method according to claim 7, 8 or 9, further comprising: dividing the region of low interest into a plurality of tiles, wherein each frame of the transmitted or stored video data containing data representing a portion of the region of low interest contains data representing one of the tiles of the region of low interest.

11. A method according to claim 10, wherein the tiles of the background are sequentially transmitted or stored, such that each tile of the region of low interest is transmitted before restarting the sequence,

12. A method of compiling video data representing a scene having a region of high interest and a region of low interest, comprising: receiving video data at a buffer memory, wherein the frames of the video data each contain both data representing the region of high interest and data representing only a portion of the region of low interest, or alternatingly contain data representing the region of high interest and then data representing only a portion of the region of low interest; and compiling the transmitted video data by, for each frame, overwriting corresponding portions of data stored in the buffer memory representing the scene with the data contained in the transmitted frame representing the region of high interest and/or the portion of the region of low interest, such that the buffer memory always contains data representing a complete frame, wherein the region of high interest is updated at a first, higher frame rate and portions of the region of low interest are updated at a second, lower frame rate.

13. A method according to any of claims 1 to 8 or 12, further comprising displaying the compiled video data at a frame rate at or below 60fps to cause a slow motion effect.

14. A method according to any of claims 1 to 8 or 10, further comprising storing the compiled video data for later processing, wherein each frame of the compiled video data is a complete frame.

15. A high frame rate video camera comprising an image sensor and a processing device, the image sensor being configured to capture video data representing a scene having a region of high interest and a region of low interest, and the processing device being configured to produce video data for transmission or storage, wherein the frames of the video data for transmission or storage each contain both data representing the region of high interest and data representing only a portion of the region of low interest, or alternatingly contain data representing the region of high interest and then data representing only a portion of the region of low interest.

16. An apparatus comprising a processor and a memory containing computer-readable instructions, or a computer program product containing computer-readable instructions for controlling an apparatus comprising a processor, the computer-readable instructions being for causing the processor to perform a method of compiling video data representing a scene having a region of high interest and a region of low interest, the method comprising: receiving video data at a buffer memory, wherein the frames of the video data each contain both data representing the region of high interest and data representing only a portion of the region of low interest, or alternatingly contain data representing the region of high interest and then data representing only a portion of the region of low interest; and compiling the transmitted video data by, for each frame, overwriting corresponding portions of data stored in the buffer memory representing scene with the data contained in the transmitted frame representing the region of high interest and/or the portion of the region of low interest, such that the buffer memory always contains data representing a complete frame, wherein the region of high interest is updated at a first, higher frame rate and portions of the region of low interest are updated at a second, lower frame rate.

17. Video data comprising a plurality of sequential frames, the video data representing a scene having a region of high interest and a region of low interest, wherein each frame contains both data representing the region of high interest and data representing only a portion of the region of low interest.

18. Video data comprising a plurality of sequential frames, the video data representing a scene having a region of high interest and a region of low interest, wherein the frames alternatingly contain data representing the region of high interest and then data representing only a portion of the region of low interest.

19. A method of processing video data substantially as hereinbefore described with reference to the drawings.

20. A high frame rate video camera substantially as hereinbefore described with reference to the drawings.