GB2489020A

GB2489020A - A method of communicating a traffic stream over a communications network comprising a variable traffic bandwidth

Info

Publication number: GB2489020A
Application number: GB1104403.9A
Authority: GB
Inventors: David White
Original assignee: Individual
Current assignee: Individual
Priority date: 2011-03-16
Filing date: 2011-03-16
Publication date: 2012-09-19
Also published as: GB201104403D0

Abstract

A method of communicating a traffic stream 200a over a variable traffic bandwidth comprises: processing the traffic stream to produce a modified traffic stream 200b comprising a plurality of traffic clips 210, each clip comprising a plurality of traffic frames 230; processing the traffic frames of each clip to produce a set of traffic structures corresponding to each clip, the structures of the set comprising a component of the traffic frames, such that an assembly of corresponding components of the structures of each set collectively comprise the respective traffic frame; communicating the structures of each set sequentially wherein the number of structures communicated is arranged to vary according to the variable traffic bandwidth. Preferably each traffic structure corresponds with an iterative processing step of the traffic frames comprising an iterative extraction of components from the frames, each iteration including further components in the representation of each frame.

Description

A Method of Communicating Traffic over a Communications Network comprising a Variable Traffic Bandwidth The present invention relates to a method of communicating traffic over a communications network comprising a variable traffic bandwidth.

The internet is often used to communicate audio and video traffic for example, from a server or host, to a client requesting the traffic, and this is typically achieved using a Transfer Connection Protocol/Internet Protocol (TCP/IP) standard. However, the communication of traffic is dependant on the bandwidth which is available to the client and in situations where there is insufficient video bandwidth for example, the client may suffer loss of video picture or the picture may become frozen. This is a particular problem when the video being communicated comprises a live video stream.

In view of the foregoing, various mechanisms have been proposed for reducing the size of the video content to avoid suffering any loss of picture, and these mechanisms include compressing the video content using standard compression techniques, such as MPEG-2, MPEG-4 and H.264 standards. However, these compression techniques are not receptive to variations in available bandwidth and as such, less than optimum video content may be communicated in situations where further content may be supported.

W02009/020552 discloses a method for multi-bitrate content streaming in which video streams are provided at different quality levels, namely different bitrates. The streams allow a client to switch between quality levels so that a client can initially receive a low quality stream and then switch to the best quality stream which fits the available bitrate. However, this method requires complex equipment to ensure a synchronisation of video when switching between streams, and thus an apparent seamless video stream.

In accordance with the present invention as seen from a first aspect, there is provided a method of communicating a traffic stream over a communications network comprising a variable traffic bandwidth, between a traffic host and a traffic client, the method comprising: -processing the traffic stream to produce a modified traffic stream comprising a plurality of traffic clips, each clip comprising a plurality of traffic frames; -processing the traffic frames of each clip to produce a set of traffic structures corresponding to each clip, the structures of the set comprising a component of the traffic frames, such that an assembly of corresponding components of the structures of each set collectively comprise the respective traffic frame; -communicating the structures of each set sequentially over the network from the host to the client in accordance with the variable traffic bandwidth wherein the number of structures communicated over the network is arranged to vary in accordance with the variable traffic bandwidth.

Advantageously, the traffic structures comprise the traffic clip, for example an encoded video clip, in varying degrees of video quality, such that only a portion of the clip, namely a video structure, needs to be acquired to enable the recovery of the clip.

Accordingly, as successive structures become acquired, the quality of the recovered video clip will improve.

Preferably, successively communicated traffic structures comprise further components of the respective traffic clip, such that the more traffic structures that are communicated, the closer the recovered traffic clip will be to the original traffic clip. The number of traffic structures corresponding to each clip which are communicated over the network preferably increases as the available bandwidth increases and preferably decreases as the available bandwidth decreases. Preferably, components of the traffic structures, which are communicated over the network are assembled to recover the traffic clip.

The traffic structures are preferably separately communicated over the network via hyper text transfer protocol (HTTP) as a sequence of objects, each object comprising a unique uniform resource locator (URL). The communication of traffic as objects permits the traffic to be treated like any other data delivered by HTTP, and thus facilitates the use of caching support. The traffic may thus be cached by an intermediate network device and where available, the traffic may be delivered from cache, rather that the originating host server, for example.

Preferably, each traffic structure corresponds with an iterative processing step of the traffic frames. The iterative processing of the traffic frames preferably comprises an iterative application of the discrete cosine transform to each traffic frame, each iteration including further components in the representation of each frame. The traffic structures of each set are preferably arranged such that a first structure comprises a set of components corresponding to a first representation of the traffic clip, and each successive structure of the set comprises those components of a successive iteration which were not present in the previous iteration, to enable further successive representations of the traffic clip to be recovered. In this manner, the process comprises recursively extracting components from the traffic frames of each clip and moving the extracted components at each step into respective traffic structures.

Preferably, the traffic structures are received by the client and corresponding components of the traffic structures of each set are assembled to recover the corresponding traffic frame. Preferably, the assembled traffic structures comprise a compressed traffic format. The time taken to receive a set of traffic structures is preferably used to determine the number of structures of the next set which are to be communicated to the receiver. Accordingly, if the time taken to download a video clip is less that the duration of the clip, then the client may request more of the next video clip in the sequence, to thereby increase the quality of the video being displayed.

The number of structures communicated over the network for a particular traffic clip is preferably adaptive to a variation in traffic bandwidth. Accordingly, the method is arranged to dynamically adapt to changes in available network bandwidth, to vary the traffic quality without suffering a complete loss of traffic.

Preferably, the traffic structures corresponding to each traffic clip are communicated in a single traffic stream. As a result, since the traffic structures corresponding to each clip are communicated in a single traffic stream, the method enables many clients to receive the traffic in accordance with the bandwidth available to the respective client, thereby making the method suitable for communicating live traffic streaming to multiple viewers.

The traffic preferably comprises video traffic and/or audio traffic and the traffic structures communicated over the network for a particular traffic clip are preferably adaptive to client resources, such as resources for creating a 3-D display effect and/or an audio resource for creating a surround sound effect.

In accordance with the present invention as seen from a second aspect, there is provided a communications network traffic encoder which is arranged to encode a traffic stream for communication over a communications network comprising a variable traffic bandwidth, between a traffic host and a traffic client, the encoder comprising a processor which is arranged to: -separate the traffic stream into a plurality of traffic clips, each clip comprising a plurality of traffic frames; -process the traffic frames of each clip to produce a set of traffic structures corresponding to each clip, the structures of the set comprising a component of the traffic frames, such that an assembly of corresponding components of the structures of each set collectively comprise the respective traffic frame; and -communicate the structures of each set sequentially over the network from the host to the client in accordance with the variable traffic bandwidth.

The traffic clips are preferably encoded into a format which is convertible to an encoding standard, such as H.264.

Preferably, the encoder is arranged to communicate the traffic structures in a single traffic stream. The structures with the stream are preferably separated by a flag.

The encoder is preferably further arranged to initially communicate a first traffic structure comprising a set of components corresponding to a first representation of the traffic clip, and subsequently communicate successive structures of the set, to enable further successive representations of the traffic clip to be recovered. Successive data structures are arranged to provide further detail to the initial representation to refine the initial representation.

In accordance with the present invention as seen from a third aspect, there is provided a communications network traffic decoder which is arranged to decode a traffic stream received over a communications network comprising a variable traffic bandwidth, from a communications network encoder according to the second aspect, the decoder being arranged to: -receive traffic structures of each traffic clip and assemble corresponding components of each structure to recover the traffic clip.

In accordance with the present invention as seen from a fourth aspect, there is provided a communications network for communicating a traffic stream, the network comprising a communications network encoder according to the second aspect and a communications network decoder according to the third aspect.

An embodiment of the present invention will now be described by way of example only and with reference to the accompanying drawings, in which: Figure Ia is a schematic illustration of a method of communicating a video stream over a communications network comprising a variable traffic bandwidth, according to an embodiment of the present invention; Figure 1 b is a schematic illustration of a video stream, a video clip, a video frame and macroblocks associated with a video frame; Figure Ic is a schematic illustration of a HTTP file of video structures separated by flags; and, Figure 2 is a schematic illustration of a communications network according to an embodiment of the present invention comprising a communications network transmitter device according to an embodiment of the present invention and a communications network receiver device according to an embodiment of the present invention.

Referring to figures I and 2 of the drawings, there is illustrated a method 100 of communicating a traffic stream 200a over a communications network 500, such as the internet, comprising a variable traffic bandwidth, according to an embodiment of the present invention. For exemplary purposes, the present embodiment will be described with reference to a video stream, however it is to be appreciated that the embodiment applies equally to data and audio streams. The network 500 comprises a host 300 for encoding the video to be communicated over the network 500 and a client 400 for decoding the video which is requested by the client 400 from the host 300.

The host 300 is arranged to receive the video stream (at step 110) to be communicated over the network 500 and comprises a processor 310 which processes the traffic stream 200a to provide a modified stream 200b comprising a contiguous stream of video clips 210 (at step 120). The clips 210 associated with the modified stream 20Db are separated by a flag 220, and separately comprise a plurality of video frames 230 to correspond with a specified duration of video, for example 5 seconds. The video clips 210 associated with the modified stream 200b are then separately encoded at step 130 according to a video compression standard based on MPEG-2, MFEG-4 or H.264 for example, but the format of the encoded clips 210 is designed to be more efficient for communication over the network 500. The format is arranged to act as a meta format which is convertible to a standard H.264 format to drive a standard H.264 decoder 420, for example.

The encoded clips 210 are separately arranged in a plurality of hierarchical video structures (not shown) with the video structures in each hierarchy comprising components of each video frame 230 associated with the video clip 210. The components of the frames are extracted using a discrete cosine transform or similar to convert portions 240 of the video frames 230 into a spatial frequency description of that portion 240. The transform is iteratively applied to the portions 240 of the frames 230 and the components extracted at each iteration are separately arranged in successive structures (not shown) of the hierarchy, such that successive structures comprise additional components of the framed 230 associated with the video clip 210. In this manner, the components representing the lowest quality video clip 210 are arranged in the first structure and successive structures comprise additional components to provide successively higher quality video.

Following the encoding of a video clip 210, the corresponding hierarchical video structures (not shown) are cached in a network server 320 at step 140, for delivery to the client device 400. The hierarchical video structures (not shown) are communicated over the network 500 as a sequence of objects, each object comprising a unique uniform resource locator (URL). The structures are communicated in order of increasing video quality, so that the client can request the number of structures to be communicated in accordance with the bandwidth available at that instance. The use of hyper text transfer protocol (HTTP) provides an optimal way of transmitting the video because of the support which HTTP offers in requesting data ranges of objects and caching. For live streaming of the video, these structures are stored are separate files for a short period of time, as they are only required for real-time buffering. Fre-recorded video however, may be stored as a single concatenated file of hierarchical video structures, or as separate files.

The delivery of the files over the network 500 is based on a "pull" model, whereby the video delivery is controlled by a request from the client 400. The files 250 associated with the video structures (not shown) comprise a header 260, which may be delivered separately to the video clip to which it pertains, and comprises information providing data pointers to the separate video structures of the hierarchy, within the file 250. HTTP allows so-called GET requests to specify values relating to the range of the data file required by the client. In this system, the values are used to specify which divisions of the file 250 which are required, namely those video structures which can be downloaded over the network 500. Upon referring to figure Ic of the drawings, for example, which illustrates a file 250 comprising a header 260 and three video structures 270a-c separated by a flag 280, the separate video structures 270a-c associated with the file 250 may be downloaded by specifying a value of a, b or c in the GET request, which correspond with the end positions of the first, second and third video structures 270a-c within the file 250.

The client 400 is responsible for deciding how much of each file 250 to acquire and is arranged to determine the available bandwidth before downloading the next file.

The available bandwidth may be determined by simply timing how long it takes to download the previous file 250 (or pad thereof). Accordingly, if the available bandwidth reduces suddenly, for example due to a sudden increase of traffic on the network 500, then the client 400 will not lose the video, provided the available bandwidth does not fall below that required to download the lowest quality representation of the video clip 210, namely the first data structure 270a in the hierarchy. The video quality may therefore appear to the client 400 to reduce suddenly, but the viewing will not be interrupted.

Upon receiving the video structures 270a-c at step 150, the client 400 passes the structures 270a-c to an assembler 410, which combines the corresponding components of successively received structures to create a H.264 video stream according to the number of structures received at step 160. The assembler 410 further decodes the resulting stream at step 170 using a standard H.264 video decoder 420, which subsequently passes the decoded stream to a display device 430 at step 180.

The creation of the H.264 stream requires some resources of the client 400 however, the video assembler 410 only has to combine corresponding components of the video structures 270a-c and does not have to perform the computationally demanding inverse discrete cosine transform.

It is envisaged that a hierarchical arrangement of audio tracks may be included in the hierarchical video structures and that these may be included at the beginning of the HTTF file 250. By adding separate audio tracks as separate divisions at the beginning of the file 250, they can be selected by setting a suitable first value in the GET request. The separate structures of video quality can still be selected by setting a suitable end value in the GET request. In this way additional audio tracks can be selected independently of the video structures to provide additional audio to the video, irrespective of video quality, and thus a surround sound effect.

Similar to surround sound, additional video information can be included in the hierarchical video structures to present a depth of field, namely a 3-D visual effect. It is envisaged that this may be achieved by including an additional hierarchical arrangement of video tracks for stereoscopic vision. The additional video tracks may be included at the beginning of the HTTP file and accessed by the client by setting a suitable first value in the GET request. In this respect, the additional video tracks may be selectively accessed according to the display capability of the display device 430.

From the foregoing therefore, it is evident that the method provides for an improved communication of traffic over a communications network.

Claims

Claims 1. A method of communicating a traffic stream over a communications network comprising a variable traffic bandwidth, between a traffic host and a traffic client, the method comprising: -processing the traffic stream to produce a modified traffic stream comprising a plurality of traffic clips, each clip comprising a plurality of traffic frames; -processing the traffic frames of each clip to produce a set of traffic structures corresponding to each clip, the structures of the set comprising a component of the traffic frames, such that an assembly of corresponding components of the structures of each set collectively comprise the respective traffic frame; -communicating the structures of each set sequentially over the network is from the host to the client, wherein the number of structures communicated over the network is arranged to vary in accordance with the variable traffic bandwidth.
2. A method according to claim 1, wherein successively communicated traffic structures comprise further components of the respective traffic clip.
3. A method according to claim I or 2, wherein the number of traffic structures corresponding to each clip which are communicated over the network is arranged to increase as the available bandwidth increases and decrease as the available bandwidth decreases.
4. A method according to any preceding claim, wherein the components of the traffic structures, which are communicated over the network are assembled to recover the traffic clip.
5. A method according to any preceding claim, wherein the traffic structures are received by the client and corresponding components of the traffic structures of each set are assembled to recover the corresponding traffic frame.
6. A method according to claim 4 or 5, wherein the assembled traffic structures comprise a compressed traffic format.
7. A method according to any preceding claim, wherein the traffic structures are separately communicated over the network via hyper text transfer protocol as a sequence of objects, each object comprising a unique uniform resource locator.
8. A method according to any preceding claim, wherein each traffic structure corresponds with an iterative processing step of the traffic frames.
9. A method according to claim 8, wherein the iterative processing of the traffic frames comprises an iterative extraction of components from the frames, each iteration including further components in the representation of each frame.
10. A method according to claim 8 or 9, wherein the traffic structures of each set are arranged such that a first structure comprises a set of components corresponding to a first representation of the traffic clip, and each successive structure of the set comprises those components of a successive iteration which were not present in the previous iteration, to enable further successive representations of the traffic clip to be recovered.
11. A method according to any preceding claim, wherein the time taken to receive a set of traffic structures is used to determine the number of structures of the next set which are to be communicated to the receiver.
12. A method according to any preceding claim, wherein the number of structures communicated over the network for a particular traffic clip is adaptive to a variation in traffic bandwidth.
13. A method according to any preceding claim, wherein the traffic structures corresponding to each traffic clip are communicated in a single traffic stream.
14. A method according to any preceding claim, wherein the traffic structures communicated over the network for a particular traffic clip are adaptive to client resources.
15. A method according to any preceding claim, wherein the traffic comprises video traffic.
16. A method according to claim 15 as appended to claim 14, wherein the client resources comprise at least a stereoscopic video resource for creating a 3-0 display effect.
17. A method according to any preceding claim, wherein the traffic comprises or further comprises audio traffic.
18. A method according to claim 17 as appended to claim 14, wherein the client resources comprise or further comprise an audio resource for creating a surround sound effect.
19. A communications network traffic encoder which is arranged to encode a traffic stream for communication over a communications network comprising a variable traffic bandwidth, between a traffic host and a traffic client, the encoder comprising a processor which is arranged to: -process the traffic stream to produce a modified traffic stream comprising a plurality of traffic clips, each clip comprising a plurality of traffic frames; -process the traffic frames of each clip to produce a set of encoded traffic structures corresponding to each clip, the structures of the set separately comprising a component of the traffic frames, such that a summation of corresponding components of each set collectively comprise the respective traffic frame; and -communicate the structures of each set sequentially over the network from the host to the client in accordance with the variable traffic bandwidth.
20. An encoder according to claim 19, wherein the traffic clips are encoded into a format which is convertible to an encoding standard.
21. An encoder according to claim 19 or 20, which is arranged to communicate the traffic structures in a single traffic stream.
22. An encoder according to claim 21, wherein the structures with the stream are separated by a flag.
23. An encoder according to any of claims 19 to 22, which is arranged to initially communicate a first traffic structure comprising a set of components corresponding to a first representation of the traffic clip, and subsequently communicate successive structures of the set, to enable further successive representations of the traffic clip to be recovered.
24. A communications network traffic decoder which is arranged to decode a traffic stream received over a communications network comprising a variable traffic bandwidth, from a communications network encoder according to the second aspect, the decoder being arranged to: -receive traffic structures of each traffic clip and assemble corresponding components of each structure to recover the traffic clip.
25. A communications network for communicating a traffic stream, the network comprising a communications network encoder according to any of claims 19 to 23 and a communications network decoder according to claim 24.