AU2004303129B2 - Method for transmitting data over lossy packet-based communication networks - Google Patents

Description

WO 2005/027430 PCT/AU2004/001246 METHOD FOR TRANSMITTING DATA OVER LOSSY PACKET-BASED COMMUNICATION NETWORKS FIELD OF THE INVENTION 5 The present invention relates to a method for transmitting data, more particularly to a method for transmitting streaming scalable data over lossy packet based communication networks. The invention will primarily be described hereinafter with reference to the transmission of streaming scalably compressed media data, but it should be appreciated that the invention is not limited to this particular field of use. 10 BACKGROUND TO THE INVENTION Traditionally, the possibility of transmission errors has been addressed either through the use of forward error correction (FEC) or by automatic repeat request (ARQ) retransmission schemes. FEC approaches are often advocated for the transmission of 15 real-time compressed data, based on the assumption that retransmission might cause unacceptable delays. Much recent research into the protection of scalable compressed imagery over packet-based networks [1], [2], [3], [4], [5], [6], [7], [8], has adopted this perspective, exploiting the "Priority Encoding Transmission" (PET) scheme of Albanese et al. [9] as a framework for unequal error protection. 20 In some cases, retransmission of lost data may indeed not be possible, due to stringent constraints on the delivery time, or because the transmission medium provides no means for the transmitter to discover whether or not a packet has been correctly received. (It is worth noting, though, that existing protection schemes universally assume the existence of some feedback mechanism, whereby the transmitter is able to 25 discover the prevailing network conditions and hence estimate loss probabilities.) In many applications, however, delivery time constraints merely serve to limit the number of round trips, and hence the number of retransmission opportunities which exist. This perspective is adopted by a considerable body of literature [10], [11], [12], [13], [14], [15], which deals with the optimized delivery of streaming media with limited 30 retransmission. These works include retransmission decisions within the optimization framework. The combination of both limited retransmission and FEC has also been considered in a variety of settings [16], [17], [18], [19], [20], [21], [22]. We shall WO 2005/027430 PCT/AU2004/001246 -2 mention some of these again shortly. For the moment, however, we note that the advantages of the PET framework have not previously been considered in the context of limited retransmission. The one exception to this is the work of Gan and Ma [22], [23]. As we shall see, our proposed strategy differs in a number of important respects from 5 that advanced by Gan and Ma. Moreover, our experimental results confirm that these differences are responsible for virtually all the performance advantages of the proposed scheme, relative to other schemes which do not allow for retransmission. SUMMARY OF THE INVENTION 10 Accordingly, the present invention provides a method for transmitting streaming scalably compressed media data over lossy packet-based communication networks, comprising the steps of: a) obtaining source data comprising independently encoded groups of elements; 15 b) subjecting each element in a current group to primary channel coding with one from a family of channel codes, the channel code family being augmented to include a symbol corresponding to no transmission of the element (redundancy factor of 0); c) subjecting each element from a previously transmitted group to secondary 20 channel coding with one from a family of channel codes, the channel code family being augmented to include a symbol corresponding to no transmission of the element (redundancy factor of 0); and d) transmitting the source data over a sequence of transmission slots, each transmission slot comprising a time interval wherein, in each transmission slot, primary 25 channel coded symbols from elements in the current group are transmitted together with secondary channel coded symbols from elements in the previously transmitted group; e) wherein the primary and secondary channel coded symbols transmitted within each transmission slot are arranged into network packets such that each transmission slot has its own set of network packets; 30 f) wherein the primary and secondary channel coded symbols sent in each transmission slot are arranged within that slot's network packets following the Priority Encoding Transmission (PET) procedure; WO 2005/027430 PCT/AU2004/001246 -3 g) wherein information concerning lost packets which are not received by a receiver is either received or inferred by the transmitter; h) wherein the information concerning lost packets which contain primary coded symbols from said previously transmitted group is used by the transmitter to 5 select secondary channel codes for the elements of the previously transmitted group; and i) wherein the secondary channel coding of each element of the previously transmitted group is performed in a manner which takes advantage of the availability at the receiver of primary coded symbols from the previously transmitted group in packets 10 which were successfully transmitted, so as to avoid or reduce redundant retransmission. The present invention also provides a transmitter for transmitting streaming scalably compressed media data over lossy packet-based communication networks, comprising: 15 a) means for obtaining source data comprising independently encoded groups of elements; b) means for subjecting each element in a current group to primary channel coding with one from a family of channel codes, the channel code family being augmented to include a symbol corresponding to no transmission of the element 20 (redundancy factor of 0); c) means for subjecting each element from a previously transmitted group to secondary channel coding with one from a family of channel codes, the channel code family being augmented to include a symbol corresponding to no transmission of the element (redundancy factor of 0); and 25 d) means for transmitting the source data over a sequence of transmission slots, each transmission slot comprising a time interval wherein, in each transmission slot, primary channel coded symbols from elements in the current group are transmitted together with secondary channel coded symbols from elements in the previously transmitted group; 30 e) wherein the primary and secondary channel coded symbols transmitted within each transmission slot are arranged into network packets such that each transmission slot has its own set of network packets; WO 2005/027430 PCT/AU2004/001246 -4 f) wherein the primary and secondary channel coded symbols sent in each transmission slot are arranged within that slot's network packets following the Priority Encoding Transmission (PET) procedure; g) wherein information concerning lost packets which are not received by a 5 receiver is either received or inferred by the transmitter; h) wherein the information concerning lost packets which contain primary coded symbols from said previously transmitted group is used by the transmitter to select secondary channel codes for the elements of the previously transmitted group; and 10 i) wherein the secondary channel coding of each element of the previously transmitted group is performed in a manner which takes advantage of the availability at the receiver of primary coded symbols from the previously transmitted group in packets which were successfully transmitted, so as to avoid or reduce redundant retransmission. 15 The present invention also provides a method for receiving streamed scalably compressed media data over a lossy packet-based communication network, wherein the data is received and sorted into independently encoded groups of elements, comprising the steps of: a) receiving data over a sequence of reception slots, each reception slot 20 comprising a time interval wherein primary channel coded symbols from elements in a current group are received together with secondary channel coded symbols from elements in a previous group; b) wherein the primary and secondary channel coded symbols within each reception slot are arranged into network packets such that each reception slot has its 25 own set of network packets; c) wherein the primary and secondary channel coded symbols received in each reception slot are arranged within that slot's network packets following the Priority Encoding Transmission (PET) procedure; d) wherein the method further comprises the step of unpacking the PET 30 packets and sorting the received data into primary channel coded symbols from elements in the current group and secondary channel coded symbols from elements in the previous group; WO 2005/027430 PCT/AU2004/001246 -5 e) assembling the secondary channel coded symbols from elements in the previous group with primary channel coded symbols from elements in the previous group which have already been received in a previous reception slot; f) decoding said assembled information for the previous group; and 5 g) determining and sending information to the transmitter, concerning lost packets and/or packets which have been received, h) wherein the information concerning lost and/or available packets which contain primary coded symbols from said current group is capable of being used by the transmitter to select secondary channel codes for the elements of the current group in a 10 manner which takes advantage of the availability at the receiver of primary coded symbols from the current group in packets which were successfully received, so as to avoid or reduce redundant retransmission. The present invention also provides a receiver for receiving streamed scalably 15 compressed media data over a lossy packet-based communication network, wherein the data is received and sorted into independently encoded groups of elements, comprising: a) means for receiving data over a sequence of reception slots, each reception slot comprising a time interval wherein primary channel coded symbols from elements in a current group are received together with secondary channel coded symbols from 20 elements in a previous group; b) wherein the primary and secondary channel coded symbols within each reception slot are arranged into network packets such that each reception slot has its own set of network packets; c) wherein the primary and secondary channel coded symbols received in 25 each reception slot are arranged within that slot's network packets following the Priority Encoding Transmission (PET) procedure; d) wherein the receiver further comprises means for unpacking the PET packets and sorting the received data into primary channel coded symbols from elements in the current group and secondary channel coded symbols from elements in 30 the previous group; e) means for assembling the secondary channel coded symbols from elements in the previous group with primary channel coded symbols from elements in the previous group which have already been received in a previous reception slot; WO 2005/027430 PCT/AU2004/001246 -6 f) means for decoding said assembled information for the previous group; and g) means for determining and sending information to the transmitter, concerning lost packets and/or packets which have been received, h) wherein the information concerning lost packets and/or available packets 5 which contain primary coded symbols from said current group is capable of being used by the transmitter to select secondary channel codes for the elements of the current group in a manner which takes advantage of the availability at the receiver of primary coded symbols from the current group in packets which were successfully received, so as to avoid or reduce redundant retransmission. 10 Preferably, one or both of said families of channel codes, include a symbol with rate 1 (redundancy factor 1). In one embodiment, the families of channel codes used for primary and secondary channel coding have the properties of maximum distance separable (MDS) codes. Preferably also, one or both of said families of channel codes 15 is augmented to include the option for sending no symbols whatsoever (neither data nor parity symbols). The transmitter may determine primary and secondary channel codes to use during a particular transmission slot, using estimates of the anticipated packet loss rate.Preferably, primary and secondary channel codes are selected within each 20 transmission slot in a manner which is consistent with maximizing a measure of total expected quality of the reconstructed media stream, taken over all groups of elements, where the expectation is conditioned upon knowledge of the channel codes selected in previous transmission slots and the packets lost from previous transmission slots, subject to a constraint on the total transmitted data rate. The maximization procedure 25 may be constrained by a limit on the number of symbols which can be transmitted within the current transmission slot. Preferably, the transmitter formulates and optimizes an optimization objective which involves hypotheses regarding the secondary channel codes which will be assigned to elements from the current frame in a future transmission slot, in the event 30 that one packet from the current slot is lost, two packets from the current slot are lost, three packets from the current slot are lost, and so forth. More particularly, preferably the optimization objective involves a lambda-balance equation, in which lambda is assumed to take the same value in both the current transmission slot and the future WO 2005/027430 PCT/AU2004/001246 -7.

transmission slot in which the hypothetical secondary channel codes shall be selected. In one embodiment the scalably compressed media data represents a compressed video signal wherein each group of elements comprises a video frame. However, the present invention may be employed in other embodiments, involving video, audio or other 5 media signals, wherein each group of elements represents compressed data (e.g., media sample values or transformed media sample values) which can meaningfully be decompressed without reference to the elements in other groups. Preferably, in the receiving method, the steps of assembling and decoding said secondary and primary channel coded information for elements in the previous group 10 comprise the steps of: decoding said secondary channel coded symbols; augmenting the primary channel coded information for those same elements; and decoding said augmented channel coded information to recover elements from the previous group. The present invention also provides computer program including instructions for controlling a computing system for implementing a method as defined above, and a 15 computer readable medium having such a computer program stored thereon. The present invention shows how the PET (Priority Encoding Transmission) framework may be leveraged to exploit both unequal error protection and limited retransmission, for RD optimized delivery of streaming media. Previous work on scalable media protection with PET has largely ignored the possibility of 20 retransmission. Conversely, the PET framework has not been harnessed by the substantial body of previous work on RD optimized hybrid FEC/ARQ schemes. The present invention is applicable to sources which can be modelled as independently compressed groups (e.g., video frames). Each element in the scalable representation of each frame can be transmitted in one or both of two transmission slots, depending on 25 the availability of the data received from the first of the two transmission slots. An optimization algorithm determines the level of protection which should be assigned to each element in each slot, subject to transmission bandwidth constraints. To balance the protection assigned to elements which are being transmitted for the first time with those which are being retransmitted, the proposed algorithm in preferred embodiments 30 formulates a collection of hypotheses concerning its own behaviour in future transmission slots. We show how the PET framework allows for a decoupled optimization algorithm with only modest complexity. Experimental results demonstrate that substantial performance benefits can be obtained using the proposed framework.

WO 2005/027430 PCT/AU2004/001246 BRIEF DESCRIPTION OF THE DRAWINGS Preferred embodiments of the present invention will now be described with reference to the accompanying drawings, in which: Figure 1 illustrates an example of a PET frame comprising N=5 packets, into 5 which Q=4 elements are coded; Figure 2 illustrates the relationship between source frames and transmission slots; Figure 3 illustrates an example of the relationship between PNk and RNk; Figure 4 illustrates an example of the convex utility-length characteristic for a 10 source consisting of 4 elements; Figure 5 illustrates the upper convex hull of a non-convex P(r) vs. R(r) characteristic; and Figure 6 illustrates the decoding of an element in a source frame from its primary and secondary transmission slots. 15 DETAILED DESCRIPTION The invention is concerned with the robust transmission of streaming scalable data through lossy communication channels, for applications in which limited retransmission is possible. For explanatory purposes, the discussion which follows is limited to scalably compressed video frames. However, the invention described here 20 can clearly be applied to the protection of other real-time media, such as audio. Scalable Source Frames To study the delivery of streaming scalable media in the context of limited retransmission, we model the scalable data source as a sequence of "frames", each of 25 which is compressed independently. Highly scalable image compression schemes have been widely investigated over the past 15 years. Some of the most notable algorithms to emerge from these efforts are the EZW [24] and SPIHT [25] algorithms, and the JPEG2000 [26] image compression standard. The latter is based on a variety of important concepts, including context-adaptive bit-plane coding [27], context-adaptive 30 bit-plane reordering [28], [29], and embedded block coding with optimized truncation [30]. These algorithms all produce compressed bit-streams, with the property that WO 2005/027430 PCT/AU2004/001246 low quality representations are embedded within higher quality representations. Equivalently, the compressed bit-stream may be viewed as a succession of elements Eq , 1 ! q Q , having coded lengths Lq , which progressively augment the received image quality. For the embedded representation to be efficient, the image quality 5 obtained when the image is reconstructed from an initial Q' Q of these elements should be comparable to that which could be obtained if the image were compressed using any other efficient algorithm, be it scalable or otherwise, to the same total length L = XY Lq . Modem scalable image compression algorithms such as SPIHT and JPEG2000 are efficient in this sense. 10 Scalable video compression algorithms have also been the subject of intense research [31], [32], [33], [34], [35], [36], [37], with recent results demonstrating performance approaching that of the most advanced non-scalable techniques, while offering a large number of embedded elements (see, for example, [38]). The key difference between scalable image compression and scalable video compression is that 15 a motion-compensated temporal transform is required, to efficiently exploit inter-frame redundancies. For the purpose of this description, it is convenient to model any scalably compressed video stream as a sequence of independently compressed source frames F[n] , each of which has its own collection of embedded elements. In the simplest case, 20 each original frame of the video sequence is independently compressed using a scalable image coder. More generally, when the original video frames are compressed jointly, so as to exploit motion redundancy, the compressed representation produced by any of the highly scalable video compressors of which we are aware may still be modeled as a sequence of element groups, where the elements in each group are coded independently 25 from those in other groups. These element groups are identified variously in the literature as groups of pictures, frame slots, and so forth. The theory and algorithms described in this document may be applied directly to such data streams by treating each independent element group as a separate source frame, F[n] . For the discussion which follows, however, it is simplest to think of the F[n] as actual compressed video 30 frames.

WO 2005/027430 PCT/AU2004/001246 - 10 The PET Framework The lossy communication channels considered in this description are packet based, where the packet transport model is that of an erasure channel. An erasure channel is one in which each packet either arrives intact or is entirely lost. A key 5 property of erasure channels is that the receiver knows exactly which packets have been lost (the erasures). This is a good model for most packet-based communication applications, although we note that packet loss in IP networks is usually assessed on the basis of excessive arrival delay, rather than certain knowledge that it has been lost. In the context of erasure channels, Albanese et al. [9] introduced PET. The 10 PET scheme works with a family of (N, k) channel codes, all of which have the same codeword length N , but different source lengths, 1 k N . We consider only "maximum distance separable" (MDS) codes, which have the key property that receipt of any k out of the N symbols in a codeword is sufficient to recover the k source symbols. The amount of redundancy, RN,k = N/k , determines the strength of the 15 code, where smaller values of k correspond to stronger codes. It is convenient to augment this set of channel codes with the special value k = oo , for which RN, = 0 meaning that the element is not transmitted at all. Note that redundancy, RN,k , is the reciprocal of the more commonly used "code rate," k / N . We find it more convenient to refer to redundancy, rather than code rate, in large part due to the importance of the 20 case RN,= 0 . We measure the length Lq of each scalable source element Eq in symbols. Each source symbol might, for example, correspond to an 8-bit byte, although other symbol sizes may also be used. Given a collection of source elements E,E 2 , .. , EQ having uncoded lengths L 1

,L

2 ,..., LQ , and channel code redundancies 25 RNk 1 ! RN,k2 !... RN,kQ , the PET scheme packages the encoded elements into N network packets, which we call a PET frame. Lq / kq source symbols are placed in each of the first kq packets, while each of the remaining N - kq packets contains Lq / kq parity symbols. This arrangement guarantees that receipt of any k' kq packets is sufficient to recover element Eq . The total encoded transmission length is WO 2005/027430 PCT/AU2004/001246 - 11 XqLqRNk , which must be arranged into N packets, each having S symbols. Figure 1 shows an example of arranging Q = 4 elements into a PET frame consisting of N = 5 packets. Consider element E 2 , which is assigned a (5,3) code. Since k2 =3 , three out of the five packets together contain the source element's L 2 5 symbols, while the remaining 2 packets contain parity symbols. Hence, receiving any 3 packets guarantees recovery of element E 2 and also El , but not E 3 or E4 . The PET frame construction described here ignores the possibility that Lq might not be exactly divisible by kq . To deal with this eventuality in practice, some symbols from Eq+, 1 may be protected with the (N, kq) code used for Eq , rather than the 10 (potentially weaker) code which is otherwise used for the elements of E q+. Using the PET framework, several strategies [1], [2], [3], [4], [5], [6], [7], [8] have been proposed for finding the optimal channel code assignment for each source element, under the condition that the total encoded transmission length should not exceed a specified limit, L., = NS . Generally, the optimization objective is an 15 expected utility U , which must be an additive function of the source elements that are correctly recovered. That is, Q U =Uo + LUqPN,k( q=1 where UO is the amount of utility at the receiver when no source element is received, U, is the utility associated with receiving Eq , and PN,k is the probability of 20 recovering the source element Eq , which is assigned an (N, kq ) channel code. Specifically, PN,k is equal to the probability of receiving at least kq out of N packets. As already mentioned, the special value kq = oo is reserved for source elements which are not transmitted, having probability PNp = 0 . Commonly, - U represents the Mean Squared Error (MSE) of the reconstructed image or video frame, 25 and Uq corresponds to a reduction in MSE associated with recovery of element Eq The term U 0 is included only for completeness; it plays no role in the intuitive or computational aspects of the optimization problems with which we are concerned.

WO 2005/027430 PCT/AU2004/001246 - 12 Limited Retransmission As mentioned already, we are interested in applications where limited retransmission of lost data is possible. To simplify matters, this discussion is concerned with the case in which the scalable source elements which constitute each source frame 5 have two opportunities for transmission. Each source frame F[n] is assigned a primary transmission slot T[n] , and a secondary transmission slot T[n + K] , during which information may be retransmitted. Each transmission slot has a fixed duration of T seconds, so that source frames are transmitted at a constant rate of 1 / T frames per second. The primary and secondary transmission slots are separated by at least 10 (K - 1)T seconds, and the value of K iS chosen so that this separation is long enough for the transmitter to discover whether or not packets sent during the primary transmission slot arrived successfully. Figure 2 illustrates the relationship between source frames and transmission slots. In the context of Figure 2, it is possible to appreciate the key challenges of the 15 protection assignment problem with which we are concerned. During any given transmission slot T[n] , the transmitter must distribute the available bandwidth between the primary transmission of elements from source frame F[n] and the secondary transmission (retransmission) of elements from source frame F[n - K] . We write L. for the maximum number of symbols which may be transmitted within slot T[n] , and 20 adopt the PET framework to encode elements from both source frames within a single PET frame, having N packets, each with at most S symbols, where NS = L. Since the source data must be transmitted in real-time, and the channel conditions may be subject to change over time, the transmitter must determine how best to distribute the total of L. symbols between two different source frames, without knowing how 25 many packets will be lost or what protection might be applied to the retransmission of any lost data from frame F[n] in its secondary transmission slot T[n + K] . A key element of the invention described here is a general framework for solving the protection assignment problem mentioned above. To do this, we formulate a collection of hypotheses concerning the current frame's secondary transmission, which 30 will take place in the future. We coin the term LR-PET (Limited Retransmission PET) to describe this novel framework, and we show how an optimal solution to the LR-PET WO 2005/027430 PCT/AU2004/001246 - 13 assignment problem may be found with remarkably low computational complexity. To appreciate the potential advantage of LR-PET over frame-by-frame PET, consider again the example of Figure 1, and suppose that only packets 1 and 3 arrive at the receiver. Without the possibility of retransmission, only element El may be 5 successfully recovered. If retransmission is permitted, however, only the L 2 / 3 source symbols of E2 which belong to packet 2 need be retransmitted in order for the receiver to recover the whole of element E 2 . The relative value of these symbols is thus much higher during retransmission than it was during the original transmission, since the full utility of E 2 can be realized at one third of its full transmission cost. A similar 10 argument may be applied to the retransmission of symbols from the other elements, E 3 and E 4 . Thus, LR-PET essentially augments the set of source symbols which may be transmitted (and protected) during a transmission slot with retransmitted source symbols, where the latter generally have elevated utility-length ratios. Higher utility length ratios allow a larger expected utility to be achieved with LR-PET than with PET, 15 subject to the transmission slot's length budget, L. In any given transmission slot, T[n] , the temptation to favour retransmitted elements from frame F[n - K] , with their higher effective utility-length ratios, over elements from frame F[n] , must be balanced by the fact that the utility-length advantage of future retransmissions (in slot T[n + K] ) hinges on assigning the elements 20 of F[n] some bandwidth during slot T[n] . This emphasizes the importance of including hypotheses concerning the future retransmission of elements from F[n] , when optimizing the distribution of bandwidth between elements from frames F[n] and Fn-] . 25 Related Work At this point it is possible to compare and contrast our proposed approach with other relevant protection optimization frameworks. Perhaps the most relevant previous work is that of Chou and his co-authors, which is well represented by [14], [17], [18], amongst other papers. Chou et al. advance a powerful framework for rate-distortion 30 optimal delivery of streaming scalable media, allowing for both FEC and retransmission WO 2005/027430 PCT/AU2004/001246 - 14 with deadlines. At each transmission opportunity (or transmission slot), the server explores the impact of various policies for both the current transmission and hypothetical future retransmissions, searching for the policy which optimizes the expectation of a Lagrangian cost function. In this regard, their approach is essentially 5 identical to our own, as developed in Section IV. Related ideas may be found to varying degrees in [10], [11], [12]. In each case, the idea is to model the effect of different transmission policies on the properties of a Markov decision process, finding the policy which maximizes the expectation of a utility objective over all paths through the state transition trellis. 10 A key challenge associated with such schemes is to manage the explosion in complexity (size of the state space) which arises from the interaction between transmission policies for each source element, especially in the context of a rich set of error correction options and/or acknowledgment failure possibilities. To the best of our knowledge, the optimization problem has not previously been cast within the PET 15 framework. We show here that the PET framework allows the optimization problem to be decomposed into a set of independent optimization problems, associated with each successive source element in the primary frame. This allows us to avoid sub-optimal iterative policy optimization schemes, such as the SA algorithm which underpins the work of Chou et al. Besides simplifying the optimization algorithm, the PET framework 20 is also fundamentally more efficient than one in which each source element is separately packetized and channel encoded, as envisaged by previous works which consider retransmission. This is because whenever a source element arrives at the decoder, all higher priority source elements on which it depends are also guaranteed to arrive. It is this property which yields the linear utility expression in equation (1), as 25 opposed to the necessarily smaller polynomial utility expressions manipulated by Chou and others. Very recently, Gan and Ma [22], [23] proposed a transmission optimization framework which does combine PET with the possibility of limited retransmission. Their approach, however, differs in many important respects from that proposed here. 30 Gan and Ma do not recognize that it is generally only necessary to retransmit a subset of the symbols associated with a lost source element. As explained in the example above, this has the important effect of increasing the effective utility-length ratio for retransmitted elements, accounting for much of the advantage of LR-PET over frame- WO 2005/027430 PCT/AU2004/001246 - 15 by-frame PET. Quite to the contrary, results reported in [22] indicate a loss in overall expected utility, while those reported in [23] indicate only small gains and occasional overall losses, as retransmission is added to frame-by-frame PET. Further, the optimization objective in [22], [23], does not account for the potential retransmission of 5 elements which are currently being transmitted/protected for the first time. The formulation of hypotheses to account for future retransmission is a central theme in our framework. In fact, our experimental investigations indicate that the formulation of these hypotheses is responsible for most of the transmission efficiency gains from the invention described herein. It should be noted that Gan and Ma's work is motivated 10 principally by the desire to minimize quality fluctuations from frame to frame, rather than maximizing an overall expected utility objective, as formulated in our present work. The remainder of this document is organized as follows. Section II explores the erasure channel model more thoroughly, introducing the notation which we shall use 15 throughout. Section III then reviews the problem of optimal protection assignment for a single scalable data source (e.g., a compressed image), with a linearly dependent sequence of elements Eq , without the possibility of retransmission. Section IV describes our proposed LR-PET framework and the associated protection assignment strategy. 20 II. CHANNEL MODEL AND CODING The channel model we use is that of an erasure channel, where the receiver knows exactly which packets have been lost. Packet loss may result from either corruption or congestion. The simplest example of such a channel model is an IID 25 packet loss model, with packet loss probability p . However, the theoretical results and analytical methods of the invention described here are not restricted to IID models. The Gilbert-Elliott~[39] model, for example, is commonly used to model the bursty loss patterns observed in networks which are subject to congestion. The invention's performance benefits have been experimentally validated in the case of both the HID and 30 Gilbert-Elliott models. With the IID loss model, the probability of receiving at least k out of N packets with no error is given simply by WO 2005/027430 PCT/AU2004/001246 - 16 N ( N . PN, k ~j P P-ip (2) i=k Z Figure 3 shows an example of the relationship between PN,k and RNk , for the cases p = 0.1 and p =0.4 , with codeword length N =100 . Evidently, PN, k S monotonically increasing with RNk , but note that the curve is not convex. Throughout 5 this paper, we use the term 'convex" to mean "convex n ." Convex r functions are also sometimes called "concave," although we do not use that term here. As described in the next section, optimal solutions to the protection assignment problem for a single scalable data source must inevitably be drawn from the upper convex hull of this curve. However, this need not necessarily be the case for the limited retransmission scenario 10 with which we are ultimately concerned. It is convenient to parametrize PN,k and RNk by a single parameter r defined by r=N+l-k , k#oo 0 ,k=o r may be interpreted as a redundancy index. Evidently, r lies in the range 0 to N 15 , with r = 0 corresponding to the case where no information is transmitted, and r = N corresponding to the case where only 1 of the N transmitted packets need be received for correct decoding. The redundancy index is more intuitive than k , as a measure of the strength of the channel code, particularly in view of the importance of the case r = 0 . 20 Since each of our PET frames will always involve N packets, we find it convenient to drop the explicit dependence on N . This leads to the simplified notation P(r)= PN,kg.(r), R(r)= RN,kj&() (3) where krr)= (4) o0, r=0 25 is the minimum number of packets which must be received if an element which has been assigned redundancy index r is to be recovered. III. PET PROTECTION ASSIGNMENT FOR A SINGLE SOURCE WO 2005/027430 PCT/AU2004/001246 - 17 This section reviews the problem of assigning an optimal set of channel codes to the elements El through E. of a single source frame F , subject to a constraint Lm on the length of the PET frame. For applications where retransmission is not possible, we use the methods described here to protect each source frame F[n] within its own 5 transmission slot T[n] .The material presented here also serves to facilitate our later discussion of the more complex LR-PET assignment problem, where primary and secondary transmissions of two different frames must be jointly protected. Several schemes have been reported for finding optimal channel codes within the PET framework [1], [2], [3], [4], [5], [6], [7]. For the purpose of this review, we follow the 10 development in [8]. In that work, a general set of results is provided for sources whose utility-length characteristic need not be convex and channel codes whose probability redundancy characteristic need also not be convex. For our present purposes, however, we shall assume that the source utility-length characteristic is convex n-) , as illustrated in Figure 4. That is, 15 U ... y (5) L1 L2 LQ We assume that the source elements exhibit a linear dependency structure, El .< E2 - -< Eg , meaning that elements E, through Eq must all be available before Egg can be correctly decoded. To each element Eq , we assign a channel code with codeword length N and redundancy index r, , placing all encoded elements 20 within a PET frame with N packets. Our goal is to find the set of redundancy indices r, , which maximize the expected utility, subject to the length constraint L = Q LR(rq) ; L. q=1 (6) Without any loss of generality, we may restrict our attention to solutions which 25 satisfy r, r 2 --r . To see this, suppose the expected utility is maximized by a set of redundancy indices rJ ,where rq < rq for some q . An alternate set of redundancy indices rq may be formed from ,by setting rk = r and leaving all other indices unchanged. This alternate set has a smaller encoded length, since WO 2005/027430 PCT/AU2004/001246 - 18 R(r 1 )= R(r )< R(r,) . However, it has exactly the same expected utility, since E q+1 cannot contribute to the utility unless Eq is successfully recovered, so there is no point in protecting Eqg 1 more strongly than Eq Iterating the construction, if necessary, we see that any optimal set of redundancy indices (r} may be progressively converted 5 into a (possibly different) set (rq' I , having exactly the same expected utility, at most the same encoded length, and satisfying r," r 2 - . r" . It follows that the indices {r" also maximize the expected utility, subject to the same length constraint. Since r, r 2 -.. -- r , the PET framework guarantees that elements El through Eq will all be correctly received, so long as Eq is correctly received. The 10 expected utility U , available at the receiver, can then be expressed as U =U + L UP(rq) q=1 and our task is to maximize U , subject to r, r 2 - rQ and the length constraint of equation (6). The length-constrained optimization problem may be converted to a family of 15 unconstrained optimization problems, parametrized by a quantity A >0 . Specifically, let U(") and L) denote the expected utility and transmission length associated with the set of redundancy indices {(1)} , which maximize the Lagrangian-style functional J(A) -U(A) -,&A) = l(UqP(r}))-2qQqt) 7 q=1 subject to r' > r' 2 --- > r . . Evidently, it is impossible to increase U beyond 20 U(") , without also increasing L beyond L(") . Thus, if we can find A such that L() = L , the set (r will form an optimal solution to our problem. In practice, the discrete nature of the problem may prevent us from finding a value A such that L(") is exactly equal to L. , but if the elements are small enough, we should be justified in ignoring this small source of sub-optimality and selecting the smallest value of A such 25 that L() L. Now suppose we temporarily ignore the constraint that the redundancy indices WO 2005/027430 PCT/AU2004/001246 - 19 r, must be non-increasing. We shall find that the resulting solutions will then be automatically guaranteed to satisfy this constraint. The unconstrained optimization problem decomposes into a collection of Q separate maximization problems. In particular, we seek r(2) which maximizes 5 J() =UP(r)-iLR(r for each q = 1,2,...,Q . Equivalently, r2) = argmax. [P(r)- zqR(r)j where Zq = ALq / Uq This optimization problem arises in other contexts, such as the optimal 10 truncation of embedded compressed bit-streams [26, chapter 8]. It is known that the solution r must belong to the set Hc which describes the upper convex hull of the P(r) vs. R(r) characteristic, as shown in Figure 5 . If 0 = jo <ij ... is an enumeration of the elements in HC , and Sc(i)= {R(j)-R ,i > 0 (8) o0 ,i=0 15 are the ''slope" values on the convex hull, then Sc (0) Sc(1) --- . The solution to our optimization problem can then be found from = maxj eHc I Sc (i) Aq (9) = max ji e Hc Sc(i)Uq U Moreover, since the source is convex (equation (5)), we must have Aq q +, from which we deduce that r - r( . That is, the redundancy indices which 20 maximize the unconstrained objective of equation (7) are guaranteed to satisfy the constraint rl,' > .A .. >r A n2 --- r , as promised. In summary, the optimal set of channel code redundancy indices is found by searching for the smallest value of A such that the solutions r t ) yielded by equation (9) produce a PET frame length L") which is no larger than L. . The monotonic 25 dependence of r , and hence U(") and L(") , on A , renders this search problem WO 2005/027430 PCT/AU2004/001246 -20 particularly simple. A trivial bisection search, for example, limits the number of distinct values for A which must be tried to at most the numerical precision (number of bits) used to represent slope values. 5 IV. THE LR-PET FRAMEWORK In the previous section, we assumed that each frame F[n] has its own transmission slot T[n] , without any possibility for retransmission of lost data in future transmission slots. This led to a relatively simple strategy for assigning optimal redundancy indices r [n] to its elements E, [n] . In this section, we consider the more 10 general context, in which each source frame F[n] has both a primary transmission slot T[n] , and a secondary transmission slot T[n + K] . Figure 2 illustrates the relationship between source frames and transmission slots. During transmission slot T[n] , the transmitter selects redundancy indices ri[n] for each element of the primary frame F n] , as well as retransmission 15 redundancy indices sq[n - K] for each element of frame F [n - K] , subject to a limit L. on the total number of symbols which can be transmitted within any slot. We shall consistently use the notation rq and Sq to refer to the redundancy indices associated with primary and secondary transmission, respectively. We also remind the reader that the case in which an element is not transmitted at all is captured by the assignment of a 20 redundancy index of 0 . The objective is to jointly optimize the parameters r[n] and sq[n -r], subject to the length constraint L. on slot T[n] . To appreciate the nature of this problem, note that the utility of frame F[n] at the receiver depends on both r[n] and Sq [] , yet we must optimize the r[n] and sq [n - K] parameters together, without 25 precise knowledge of which packets from frame F[n] will be lost, or what protection might be assigned to the retransmission of lost data. The invention adopts the PET framework, jointly protecting all source elements transmitted in slot T[n] with a single PET frame of N packets. To address the dependence of rq[n] on the sq[n] values, which will not be assigned WO 2005/027430 PCT/AU2004/001246 -21 until the future transmission slot T[n + K] , a collection of hypotheses is formulated regarding what might happen in that slot. Write Pk for the probability that exactly k of the N packets transmitted in T[n] is correctly received. Each value of k generates a separate hypothesis. The expected utility of frame F[n] may then be 5 expressed as E[U[n]]= U 0 n]+ p k U, [n]P(k, r, [n, S (n]) (10) k=0 q where P(k, rq [n), s [n]) is the probability that element Eq [n] will be correctly received if it is assigned a redundancy index of rq [n] in slot T[n] , if only k of the N packets transmitted in that slot are received correctly, and if the remaining data is 10 retransmitted with a redundancy index of s [n] in slot T[n + K] . It is important to note that the additive formulation in equation (10) is valid only so long as the real and hypothetical redundancy indices satisfy rq [n] rqg E[n) Vq and s [n] s k n] Vq, k (11) A similar argument to that presented for the case of simple source protection in Section 15 III shows that no loss of generality is incurred by imposing these constraints. Actually, the value of s[n] is immaterial if k kj. (rq [n]) , since then no retransmission of E is required in slot T[n + K] .As a result, the requirement that s [n] s,*En] strictly applies only for those k <kn (r, [n]) .Nevertheless, it remains true that the constraints of equation (11) can be imposed as-is, without adversely affecting the expected utility. 20 The expected total transmission length associated with frame F n] may be expressed as E[L[n] = Pk I Lq [n]R(k, r, [n],s [n]) (12) k=o q where R (k,qrn],s4 [n]) is the ratio between the total coded length (in its primary and secondary slots) and the uncoded length of Eq [n] , if redundancy index rq En] is 25 assigned in the primary slot T[n] , if only k of the N packets transmitted in that slot are received correctly, and if the remaining data is retransmitted with a redundancy index of s [n] in slot T[n +r] .

WO 2005/027430 PCT/AU2004/001246 - 22 In order to account for the complex interaction between current and future redundancy indices, we adopt a global perspective. Consider for a moment the global objective of maximizing the total expected utility E[E, U[n] , subject to some constraint on the total expected length E[X, L[n] . Following the same reasoning 5 presented in Section III, solutions which maximize such a global objective form a family, parametrized by A . The redundancy indices associated with each member of this family (i.e., with each A ) are those which maximize E [U[n]] -AZE[L[n]] (13) The basic idea behind our formulation is as follows. In each transmission slot 10 T[n] , we select redundancy indices which maximize the Lagrange-style objective of equation (13), where the expectations are conditioned on whatever information is already known at the start of T[n] , adjusting A until the redundancy indices selected in this way satisfy the encoded length constraint for slot T[n] as tightly as possible, i.e., 15 EL, [n] R(r, [n])+ EL, [n - K]. [R(k,_,r, [n - K] s, [n - K])- R(r, [n - K]) ] L. (14) q q Note that R(k,., ,rq [n - 4 q,[n - ])- R(r[n - K]) represents the ratio between the encoded length contribution to T[n] and the uncoded length of element Eq [n - K] Before moving to the optimization algorithm, it is worth mentioning that the actual value of A which satisfies the length constraint in equation (14) may vary from slot to 20 slot. This may happen for any number of reasons which cannot be foreseen ahead of time -- e.g., variations in the source statistics, variations in the channel error rate, and so forth. As a result, the redundancy indices found in each successive transmission slot may be solutions to different global optimization problems, jeopardizing the global optimality of the solution found in any given transmission slot. One might attempt to 25 account for this by generating statistical models for the evolution of A in future transmission slots. Details of such modeling, however, are beyond the scope of the present invention. Instead, when formulating the expectations found in equations (10) and (12), the preferred embodiment of the invention presumes that the hypothetical redundancy indices s [n] will be selected in accordance with the same global WO 2005/027430 PCT/AU2004/001246 - 23 optimization objective, having the same value of A as that selected for the present transmission slot. A. Rewriting The Optimization Objective 5 Eliminating all terms from equation (13) which do not depend upon rq[n] and sq [n - K] , and writing k,_, for the actual number of packets from slot T[n - K] which were correctly received, we see that the objective in transmission slot T[n] is to maximize JA (Irq[n]q s[]Iq'*,,SqN[nIqSq[n - ]}q) = N , U [n]P(k, rq [n], st [n])+ Z Uq [n - ]P(k,, r [n- Sq [n - ]) k=O q q -A PA ELq [n]R(k, rq [n],s [n])- Lq [n - K]R(k,_,,rq[n - ],sq(n - (16) k=0 q 10 subject to the constraints rq [n] rq+, [n, Vq, sk [n] > s [n], Vk,q and sq [n -'K] Sq+i [n -], Vq finding the smallest value of A for which the actual transmission cost in slot T[n] satisfies equation (14). For the sake of clarification we note that the values of rq [n - K] are known 15 when we come to solve this optimization problem. For this reason, we omit the superscript from the redundancy indices sq [n - K] which are not hypothetical. It is particularly worth noting that the optimization must be performed jointly over both the actual redundancy indices and the hypothetical redundancy indices, s q[n] , even though the latter do not contribute to the length constraint in equation (14). 20 At this point, it is convenient to simplify the notation in equation (15), using primes to distinguish quantities which belong to frame F[n - K] from those belonging to Fn] . Thus, for example, U = Uq[n - K] , s Sq [n - K] , s s [n] and k'= k,_-, . With this notation, the optimization objective becomes WO 2005/027430 PCT/AU2004/001246 -24 J2({Jr1q ,{S ,..,{S N I"IS'}) PA [UqP(k,rq ,Sq ALqR(k,rq,s+)] UP(k',r,s k= q~} q and the parameter constraints become r r 1 , s s q, and s s+ 1 , Vk, q (1 5 Note also that the task of maximizing J, subject to equation (18) is equivalent to the two independent tasks of maximizing J, subject to s s, and T,, subject to rq : r, and s sq+1 , where J, and T, are identified parenthetically in equation (17). The remainder of this section is organized as follows. We first describe the 10 mechanics of retransmission, providing expressions for the terms P(krqSq) and R(k,rq,sq) .Next, we consider the solution to the above optimization objective, in the simple case where each source frame has only one element. Finally, we show that the solution to the general problem is essentially no more complex than that developed for the single element case since, under reasonable conditions, J, may be optimized 15 separately for each q , and the resulting solutions are guaranteed to satisfy the constraints of equation (18). B. The Mechanics ofRetransmission Following the definition in equation (4), the number of packets which must be 20 received if element Eq is to be recovered from its primary transmission slot is k (rq) If k > km (r,) , there is no need for any retransmission. Suppose that k <ks (rq) primary transmission packets are received. Since the transmitter knows exactly which packets were received, it has only to retransmit the information from Eq which is contained within any k. (rq)- k of the N - k packets which were lost. Thus, for the 25 purpose of retransmission, the length of element Eq is effectively reduced to 0,rL, where WO 2005/027430 PCT/AU2004/001246 -25 k.j r J-k k k, = r" )-k =1- ,0k ! k < k (r) Note that this expression is also valid for the special case rq = 0 , for which kj(0)=oo and k, 0 = 1 During retransmission, the k,,, Lq source symbols from element Eq are 5 assigned the secondary redundancy index sq , leading to a transmission cost of Ok,,,LqR(Sq) , and an expected utility of UqP(Sq) . The receiver is able to recover Eq so long as k, (sq) of the N packets sent during the element's secondary transmission slot are received. To do so, the receiver must first decode the secondary channel code, recovering the OrL missing source symbols which are required to 10 decode the primary channel code. These procedures are illustrated in Figure 6. The following expressions are readily obtained for P(k, rq ,Sq) and R(k, rq ,Sq) ( ) P(sk) ,<k n(r) (19) P(k,r,sq)={ , k ! k. (rq) ( 0,,,R(s, ) ,k < k. (r) (20) R(k,rq,s)= R(rq)+{~ 0 ( , k > kg (rq) C. Single Element Frames 15 To provide a more accessible introduction to the channel code assignment problem, we first consider the case in which each frame F[n] has only one source element. Let Eq [n] denote this source element; the subscript q is superfluous in this case, but we preserve it for the sake of later extension to source frames with multiple elements. Substituting the expressions from equations (19) and (20) into equation (17), 20 we obtain N)Pk N - (rq )Lq A,,q(rq,,so,,...,IsN Ukr p +~ j (r 1pk(UqP(sq-)O,,qR (S k)Lq) k=O =UqP(rq)-AR(r)Lq + I p( P)s-, R )L, (21) k=O WO 2005/027430 PCT/AU2004/001246 -26 and ,'P(s)-AR(rq, -10,, (s')L , k'P< kU1 (r AU( )Lq k' k r) Recall that k' and r are known indices. Noting that sq is relevant only when k' < kg, (ri) , the task of maximizing J( is equivalent to that of 5 maximizing U P(s )-A'R(s jq, where A '/= As explained in Section II, the maximizing s must be drawn from Hc , the upper convex hull of the channel coding probability-redundancy characteristic. In fact, the maximizing sq is found using equation (9) to be 10 sq = max j, e I Sc ()#>Ak',r; Similarly, for any given value of rq = r , the hypothetical retransmission indices s"' which maximize T2, (r,s',.. ) are simply those which independently maximize the terms UqP(sk ')- 0,R(s )Lq , the solution to which is 1 maxji E Hc E c Lq (22) Here, we use the notation sq'' to express the dependence of the optimal hypothetical redundancy indices on the primary redundancy index, rq = r . We are then left with the problem of determining r. . Due to the complex interaction between rq and the s, it is not generally sufficient to consider only those indices which belong to the convex 20 hull, Hc c R . In the case of Reed-Solomon codes, the set R consists of all r in the range 0 through N and for the purpose of assessing complexity, we shall henceforth assume this maximally sized set of possible redundancy indices. At first glance, the apparent need for an exhaustive search may appear to render the invention computationally unattractive. For each of the N + 1 possible 25 indices r E R , the kn (r) -1< N hypothetical optimization problems represented by WO 2005/027430 PCT/AU2004/001246 -27 equation (22) must each be solved, and each of these involves up to IIHcl N +1 comparisons. This would suggest a complexity of at most N(N+1)2 ~ N 3 comparisons. We can readily simplify this by exploiting the monotonicity of Sc (i) in a binary search for the solution to equation (22), leaving us with a complexity of 5 N(N+1)log 2 Hc1|~ N 2 log 2 N comparisons. Fortunately, the exhaustive search may be even further simplified by observing that the sk'' values recovered using equation (22) are monotonically non-decreasing with r . To see this, observe that kj (r) is a strictly decreasing function of r , hence 6k, =1-k /kn (r) is also strictly decreasing with r , but s 'r is a non-increasing 10 function of ZOk,, . In the preferred embodiment of the invention, this property is exploited by the following strategy for finding the sk'r for each k and each r E R For each k , s,' is evaluated at r=0,1,...,N in sequence. Thus, when we come to evaluate sk,r we already know the value of S r~1 ; we also know that s,r > s k,r-1 . The inequality in equation (22) is evaluated starting with the first j, e Hc 15 which is greater than s k, r-1 , and continuing until some jg E Hc yields Sc (i)Uq /Lq <Z 0 kr . In this way, the inequality must be evaluated exactly 1+ (s,r-1, s k,r] r) Hc times. It follows that the total number of times the inequality must be evaluated, in order to compute sk'' for all r e R , is bounded by (N+1)+1(0,sN)r Hc (2N+1) (23) 20 Actually, the bound on the right hand side of equation () can always be reduced by 1 . To see this, note that the complexity expression on the left assumes that we always test the inequality of equation (22) until it fails. However, (sk,N ]-) HC can only equal N if sk,N = N , in which case we will never perform a test for skN in which the inequality fails. The complexity bound can thus be reduced to 2N tests. 25 This observation will prove more significant in Section IV-E. To conclude, we note that the number of values for k which must be considered is bounded by N . Thus, the total number of comparisons required to maximize 'P,, is at most 2N 2 . A WO 2005/027430 PCT/AU2004/001246 -28 tighter bound, of the same order, could be developed by exploiting the fact that Skr need not be evaluated for k k, (r) D. Extension to Multiple Source Elements 5 We now tackle the general problem of maximizing the objective function J, in equation (17), subject to the constraints in equation (18). As in the single element case, maximization of the second term J,(s ,s 2 ,...) , subject to s' s, , is an independent problem whose solution has already been described in Section III. For the first term, T,({r, , s , coupling between the primary and hypothetical redundancy 10 indices, r and sk , would appear to present some difficulties. While exhaustive search through all possible r values is tractable for a single element, the number of possible combinations for the indices r, through rQ grows exponentially with Q . Fortunately, however, this is not necessary. Evidently, q qr E,, q ,..,s, I NO.. q 15 where the terms ,,q are given by equation (21). Thus, it is sufficient to maximize each T,, independently, following the method outlined in Section IV-C, so long as we can be guaranteed that the independent solutions will satisfy the constraints in equation (18). We now show that this is the case, subject to some reasonable assumptions. 20 As in Section III, we assume that the source utility-length characteristic is convex, alternatively, we first find the convex hull and then enforce the constraint that any element not on the convex hull must receive the same protection assignment as the next element which does lie on the convex hull, satisfying equation (5). The following lemma shows that the hypothetical retransmission indices, s , will satisfy the 25 necessary constraints so long as the primary redundancy indices rq do. This simplifies our task to that of showing that the solutions which maximize each T, independently satisfy rq rq, Vq Lemma 1: Suppose the redundancy indices {rq and {s which WO 2005/027430 PCT/AU2004/001246 -29 independently maximize each T,,,q in equation (21) satisfy rq rqH, Vq . Suppose also that the source utility-length characteristic is convex, following equation (5). Then sq s, Vq,k Proof: We showed at the end of Section IV-C that 0 k,r is a strictly decreasing 5 function of r . Using equation (22) then, we must have Sk=max jieHc U > Lq Sc 0) max{j e HC > r LJ-SC (i) q+ The second inequality above follows from the fact that the set of ji for which 'U k se Aohich Uq~ Ok, q+1 q+ ' <___ necessarily contains the set of j, for which - since Uqsi < . fT - -SCTiTo Lq+ -sc~i) ' q+1 - Lq To show that rq rq+ , we begin by writing T2,q (r) = T 2 , q q) 10 where the s,' are given by equation~0. That is, T,,(r) is the maximum value which the objective function can attain, given that rq is set to r . The following lemma embodies the key result which we must show. Lemma 2: Let ra and rB be any two redundancy indices satisfying rA > B and suppose that T,,q (rA)> Tq (rB) for some q > 1 . Then we must also have 15 TI,q1 (rA)> TA,q_ 1 (rB) , so long as the source utility-length characteristic is convex, satisfying equation (5), and the set of channel code redundancies which belong to Hc is sufficiently dense. We defer the proof of this key result to the appendix. The last condition, however, deserves some comment here. Our proof relies upon passing to a continuous 20 model for the convex hull of the channel coding probability-redundancy characteristic. This indicates that the result might hold only when a very dense collection of channel codes is available. In practice, extensive numerical studies suggest that the result always holds, even in the discrete case, but we do not have a formal proof that this must be true. Our idealized proof in the appendix at least shows that enforcing the constraint WO 2005/027430 PCT/AU2004/001246 -30 rq >! rq while passing from the optimization of T,,q to that of T,, is unlikely to produce sub-optimal results, and that any such likelihood diminishes as the available set of channel code redundancies increases. Corollary 3: If r =rA maximizes T,,(r) then it is always possible to find 5 r = rB which maximizes T,,q (r) such that rB r. Proof: Suppose, to the contrary, that any r =rB which maximizes T 2 ,,,q (r) has rB> . This means, in particular, that TZ,q+1 (rB) >T2,q+ (rA ) . According to Lemma 2 then, we must also have P;,q (rB) WT,q (rA) so that rA cannot be the value of r which maximizes TAq W(r) 10 E: Complexity With Multiple Source Elements We saw in Section IV-C that the inequality in equation (22) must be evaluated at most 2N2 times in order to maximize T,, . Since each source frame has Q elements, it is clear that all Q optimization problems can be solved using at most 15 2N 2 Q comparisons. For large Q , however, the preferred embodiment of the invention exploits the monotonicity relationships of Lemma 1 and Corollary 3 to substantially reduce this complexity. This is achieved using the following branch and bound strategy. We start by solving the single element optimization problem associated with 20 element Eq, , where q, ~ Q /2 , at a cost of 2N 2 comparisons (we are ignoring the much lower cost of maximizing the J,,q terms here). The solution to this problem divides the set of redundancy indices which may be selected for r and sk into two halves, depending on whether q < q 1 or q > qi . We next solve the two single element optimization problems associated with q 2 ~ Q /4 and q 3 ~ 3Q /4 . In solving these 25 problems, we need only consider r, e [rq,N] , e rJ , s e , N] and Se [o,s . A trivial modification of the argument provided at the end of Section IV-C shows that Sk;r may be evaluated for all r E rq,N] using at most (N - r + 1) + s qsN I) Hc comparisons, while sq' may be evaluated for all WO 2005/027430 PCT/AU2004/001246 -31 r E [O, rq using at most (rq + 1) + 1(0, sq Hc comparisons. The total number of comparisons required to evaluate both s';' and s for all relevant r is thus bounded by (N+2)+(|Hc-1) ! 2N+2 . Following the same argument as in Section IV-C, we note that the bound of 5 2N +2 can always be reduced by 2 . This is because the above complexity expressions are based on the assumption that we always test the inequality in equation (22) until it fails, but there is no need to test s and s' values which exceed the known bounds of N and s' , respectively. As before, we note that the number of values for k which must be considered is bounded by N (although tighter bounds 10 can be found), so that the total number of comparisons required to maximize both V2, and TF 2 q, is at most 2N 2 We proceed by solving the four single element optimization problems associated with q 4 ~ Q /8 , q, ~ 3Q /8 , q 6 : 5Q /8 and q 7 z 7Q /8 . The reader may verify that the total number of comparisons required to evaluate skr , sI', s q 4 q 5 I q 6 15 and sk, , for all relevant r , is again given by 2N 2 . Continuing in this way, after a total of V subdivision steps, we solve a total of 2' -1 single element optimization problems using at most 2N 2 V comparisons. Associating 2 v -1 with Q , we see that the total complexity is of order 2N 2 log 9 2(Q + 1) , which is substantially less than 2N 2 Q , for large Q . 20 The above analysis is concerned only with maximization of the T, terms. As already noted, maximization of Jq is significantly simpler, having complexity no larger than that of regular frame-by-frame PET. For fixed length transmission slots, the optimization of ' 2 q and Jq must be conducted within an outer loop, which adjusts A until equation (14) is satisfied sufficiently tightly. While more sophisticated A 25 search strategies exist, a simple bisection search requires only one iteration for each bit in the numerical representation with which we choose for A . For most practical applications, a logarithmic fixed point representation of A suffices, with between 8 and 16 bits of precision. Depending on the application, an alternate rate control WO 2005/027430 PCT/AU2004/001246 -32 strategy may be adopted, in which A is adapted slowly from transmission slot to transmission slot, subject to a conventional leaky bucket model which constrains the average transmission rate. In this case, ,,q and JI,q must be optimized for only one value of /I within any given transmission slot. 5 Appendix Proof of Lemma 2: We are given rA >r, and F 2 ,q(r,)>7TPq(rB) ,from which we wish to show that ,,q 1 (rA) > T ,,q(rB) . Writing Sq = Uq /Lq and noting 10 that Sq_ > Sq , it is sufficient to show that WA-B(Sq) TA, ,q(rA )-T2,q(rB) Lq is an increasing function of Sq .Using equation (21), we expand T.A-B (Sq) as N )k (r )-1 YA-B(S),q P -AR(r )+ I pk(Sqp(s,rAI)_2,,,,R(S,:r)) k=k n(r )k= ( kmJ(rB)-1 S, p -MB) B Pk 3,P(sk,''-, ,,Rs'') q km.(rA)P? ) A k=0 q q Now recall that P(r) = Efki.(r)Pk , so we can write R(r) as 15 E-kmin(r)PR(r)/P(r). This, together with the fact that kr (rA)<k (rB) allows us to express WA-B (Sq) as a sum of terms, weighted by p . Specifically, ki -(rA - )-1,(Skr T A-B (Sq Pk (Sq [P(sq"''^ - P(sk'B ,r R(s [ ,, ''^ ,-'B R (srB k=0 + pmjn) S E1 - P(sk,'') R kr" k=kmi (rA) IP(rA) + N Pk(_-[R(rA) R(rB) k=knln(rB) P(rB) __ Note that the terms s,' depend only on k , r , A and Sq Specifically, from equation (22), we have 20 s,'' = maxj, e Hc I SqSc 0) Ak,r (24) To complete the proof, we show that each term in the above summations is an WO 2005/027430 PCT/AU2004/001246 -33 increasing function of Sq . The terms in the third summation do not depend on Sq either directly or through s'',A and Sk''B , so we need only concern ourselves with the first two summations. Specifically, we show that Wr,, (Sq )= Sq [P(lr'^ p(kr )-Z [t9,,,R('r' )^-0 ,rR(s,')] 5 and Zkr (Sq Sq [1 - P(s,, )]+ 0,,r, R(S r,) are both increasing functions of Sq . As noted in connection with the statement of Lemma 2, we find it necessary to pass to a continuous model of the channel coding probability-redundancy characteristic 10 on its convex hull. This allows us to express equation (24) as SqGk,r(Sq)=LOk,r, where Gk,(Sq)= s' ( Here, we are using the fact that Sc (i) is the slope of the channel coding probability-redundancy characteristic on its convex hull, as expressed by equation (8). In the limit, as the density of available channel codes grows without bound, we may 15 replace Sc (iQ.k,, with the derivative, '(s '') . The notation G,,,(Sq) makes explicit the fact that sk,r , and hence this derivative, depends only on k , r and S In the same way, it is convenient to write Pk,r(Sq) for P(sk'r) and Rk,,(Sq) for Rkr (s,'r) , from which we obtain WA','B () S[Pkr, (S)- Ik,,B (S)|-S[G),, (S)Rkr, (S)- Gkr, (S)R,,, (S)j = S [(Pr,, (S)- Gkr, (S)Rk,r, (S))- (Pr, (S)- Gk,,, (S)RkrB (S))] 20 and Zkr, (S) = S-[1 - Pkr, (S)+ Gk',B (S)Rkr, (S) Taking the derivative of WkrArB (S) with respect to S , and using the fact that apk,r - ak,r aRk,,, G R S aRk, r as G, as 25 we get -34 as k,,,a ,, - G4,,,R -kG 4 ) (pFc- k,,,R as (____r~ aGk ___ a aGk~, a +S R'^- kG G, 8 R k,'' - ,s 8G''' sG,,,,R aS as as) as as as = -Gk,r R r - Gkr, SR, + SR aGkrE Differentiating equation (25) yields Gk,. = -S , which we substitute into the above to obtain W ''^'' - =1 - rG Rk) r - G R ,,, - k, r as *,ir, k,r'A ,r k,rBk,ra )+Rk,AGk,rA R = kr rA _ p SkrE =L. P -P,,,, = P s -Pq >0 s Here we have used the fact that s ' is a non-decreasing function of r , as shown in Section and that P(s) is an increasing function of s . Similarly, taking the derivative of Z,,, (S) with respect to S , we also find that =[1 - ,

-

+ G ,,, * + R G kr 1 as ,r ~ 8 R~BiI-. a+ as+ k, rB as =1- P,,,+ Gk,rBRk,r, + SRk,,r a =1-P,,, + Gk,,,Rk,, -G,, R,,

,

, = 1 -P,r, >0 10 Any references to prior art contained herein is not to be taken as an admission that the information is common general knowledge, unless otherwise indicated.

WO 2005/027430 PCT/AU2004/001246 - 35 REFERENCES [1] R. Puri and K. Ramchandran, "Multiple description source coding using forward 5 error correction codes," Signals, Systems and Computers, 1999. Conference Record of the Thirdy-Third Asilomar Conference, vol. 1, pp. 342-346, Oct 1999. [2] A. Mohr, E. Riskin, and R. Ladner, "Unequal loss protection: graceful degradation 10 of image quality over packet erasure channels through forward error correction," IEEE Journal on Selected Areas in Comm., vol. 18, pp. 819-828, Jun 2000. [3] A. Mohr, R. Ladner, and E. Riskin, "Approximately optimal assignment for unequal 15 loss protection," Proc. IEEE Int. Conf. Image Proc., vol. 1, pp. 367-370, Sep 2000. [4] T. Stockhammer and C. Buchner, "Progressive texture video streaming for lossy packet networks," Proc. 11 'th International 20 Packet Video Workshop, May 2001. [5] S. Dumitrescu, X. Wu, and Z. Wang, "Globally optimal uneven error-protected packetization of scalable code streams," Proc. IEEE Data Compression Conf. (Snowbird), pp. 73-82, Apr 2002. 25 [6] V. Stankovic, R. Hamzaoui, and Z. Xiong, "Packet loss protection of embedded data with fast local search," Proc. IEEE Int. Conf. Image Proc., vol. 2, pp. 165-168, Sep 2002. 30 [7] J. Thie and D. Taubman, "Optimal protection assignment for scalable compressed images," Proc. IEEE Int. Conf. Image Proc., vol. 3, pp. 713-716, Sept 2002.

WO 2005/027430 PCT/AU2004/001246 -36 [8] J. Thie and D. Taubman, "Optimal erasure protection assignment for scalable compressed data with small channel packets and short channel codewords," to appear in EURASIP JASP: Multimedia over IP and Wireless Networks, first quarter 2004. 5 [9] A. Albanese, J. Blomer, J. Edmonds, M. Luby, and M. Sudan, "Priority encoding transmission," IEEE Trans. Inf. Theory, vol. 42, pp. 1737-1744, Nov 1996. 10 [10] M. Podolsky, M. Vetterli, and S. McCanne, "Limited retransmission of real-time layered multimedia," IEEE 2'nd Workshop on Multimedia Signal Processing, pp. 591-596, Dec 1998. [11] M. Podolsky, S. McCanne, and M. Vetterli, "Soft arq for layered streaming 15 media," Journal of VLSI Signal Processing for Signal, Image and Video Technology, vol. 27, pp. 81-97, Feb 2001. [12] V. Chande, N. Farvardin, and H. Jafarkhani, "Image communication over noisy channels with feedback," Proc. IEEE Int. 20 Conf. Image Proc., vol. 2, pp. 540-544, Oct 1999. [13] C.-Y. Hsu and A. Ortega, "Rate control for robust video transmission over burst error wireless channels," IEEE Journal on Selected Areas in Comm., vol. 17, pp. 756-773, May 1999. 25 [14] P. Chou and Z. Miao, "Rate-distortion optimized streaming of packetized media," Microsoft Tech. Report MSR-TR-2001-35 (submitted to IEEE Trans. Multimedia), Feb 2001. 30 [15] G. Cheung, W.-t. Tan, and T. Yoshimura, "Rate-distortion optimized application level retransmission using streaming agent for video streaming over 3g wireless network," Proc. IEEE Int. Conf. Image Proc., vol. 1, pp. 529-532, Sep 2002.

WO 2005/027430 PCT/AU2004/001246 -37 [16] R. Puri, K. Ramchandran, and A. Ortega, "Joint source channel coding with hybrid FEC/ARQ for buffer constrained video transmission," IEEE Second Workshop on Multimedia Signal Processing, pp. 567-572, 5 Dec 1998. [17] P. Chou, A. Mohr, A. Wang, and S. Mehrotra, "Fec and pseudo-arq for receiver driven layered multicast of audio and video," Proc. IEEE Data Compression Conf. (Snowbird), pp. 440-449, Mar 2000. 10 [18] J. Chakareski, P. Chou, and B. Aazhang, "Computing rate-distortion optimized policies for streaming media to wireless clients," Proc. IEEE Data Compression Conf. (Snowbird), pp. 53-62, Apr 2002. 15 [19] G. Wang, Q. Zhang, W. Zhu, and Y.-Q. Zhang, "Channel-adaptive error control for scalable video over wireless channel," 7'th Int. Workshop on Mobile Multimedia Communications (MoMuC), Oct 2000. [20] T. Stockhammer, H. Jenkac, and C. Weiss, "Feedback and error protection 20 strategies for wireless progressive video transmission," IEEE Trans. Circ. Syst. for Video Tech., vol. 12, pp. 465-482, Jun 2002. [21] X. Zheng, S.-H. Chan, Q. Zhang, W.-W. Zhu, and Y.-Q. Zhang, "Feedback-free packet loss recovery for video multicast," 25 IEEE Int. Conf. Communications, vol. 2, pp. 870-874, Apr 2003. [22] T. Gan and K.-K. Ma, "Sliding-window packetization for forward error correction based multiple description transcoding," Proc. IEEE Int. Conf. Acoust. Speech and Sig. Proc., vol. 5, pp. 756-759, Apr 2003. 30 [23] T. Gan and K.-K. Ma, "Sliding-window packetization for unequal loss protection based multiple description coding," Proc. IEEE Int. Conf. Image Proc., vol. 3, pp. 641-644, Sep 2003.

WO 2005/027430 PCT/AU2004/001246 -38 [24] J. Shapiro, "An embedded hierarchical image coder using zerotrees of wavelet coefficients," Proc. IEEE Data Compression Conf. (Snowbird), pp. 214-223, 1993. 5 [25] A. Said and W. Pearlman, "A new, fast and efficient image codec based on set partitioning in hierarchical trees," IEEE Trans. Circ. Syst. for Video Tech., pp. 243-250, June 1996. [26] D. Taubman and M. Marcellin, JPEG2000: Image Compression Fundamentals, 10 Standards and Practice. Boston: Kluwer Academic Publishers, 2002. [27] D. Taubman and A. Zakhor, "Multi-rate 3-D subband coding of video," IEEE Trans. Image Proc., vol. 3, pp. 572-588, is September 1994. [28] J. Li and S. Lei, "Rate-distortion optimized embedding," Proc. Picture Coding Symposium, Berlin, pp. 201-206, September 1997. 20 [29] E. Ordentlich, M. Weinberger, and G. Seroussi, "A low-complexity modeling approach for embedded coding of wavelet coefficients," Proc. IEEE Data Compression Conf. (Snowbird), pp. 408-417, March 1998. 25 [30] D. Taubman, "High performance scalable image compression with EBCOT," IEEE Trans. Image Proc., vol. 9, pp. 1158 1170, July 2000. 30 [31] J. Ohm, "Three dimensional subband coding with motion compensation," IEEE Trans. Image Proc., vol. 3, pp. 559-571, Sep 1994.

WO 2005/027430 PCT/AU2004/001246 -39 [32] D. Taubman and A. Zakhor, "Highly scalable low-delay video compression," Proc. IEEE Int. Conf. Image Proc., vol. 1, pp. 740-744, Nov 1994. 5 [33] S. Choi and J. Woods, "Motion compensated 3-D subband coding of video," IEEE Trans. Image Proc., vol. 8, pp. 155-167, Feb 1999. [34] A. Secker and D. Taubman, "Motion-compensated highly scalable video 10 compression using an adaptive 3D wavelet transform based on lifting," Proc. IEEE Int. Conf. Image Proc., pp. 1029-1032, Oct 2001. [35] B. Pesquet-Popescu and V. Bottreau, "Three-dimensional lifting schemes for 15 motion compensated video compression," Proc. Int. Conf. Acoust. Speech and Sig. Proc., pp. 1793-1796, 2001. [36] A. Secker and D. Taubman, "Lifting-based invertible motion adaptive transform (LIMAT) framework for highly scalable 20 video compression," to appear in IEEE Trans. Image Proc., Dec 2003. [37] G. Van der Auwera, P. Munteanu, P. Schelkens, and J. Cornelis, "Bottom-up motion compensated prediction in wavelet domain for spatially scalable video coding," Electronic Letters, vol. 38, pp. 1251-1253, 25 October 2002. [38] D. Taubman, "Successive refinement of video: fundamental issues, past efforts and new directions," Int. Symp. Visual Comm. and Image Proc., vol. 5150, pp. 791-805, July 2003. 30 [391 E. Elliott, "Estimates of error rates for codes on burst-noise channels," Bell Sys. Tech. Journal, vol. 42, pp. 1977-1997, Sep 1963.

Claims (18)

1. A method for transmitting streaming scalably compressed media data over lossy packet-based communication networks, comprising the steps of: a) obtaining source data comprising independently encoded groups of 5 elements; b) subjecting each element in a current group to primary channel coding with one from a family of channel codes, the channel code family being augmented to include a symbol corresponding to no transmission of the element (redundancy factor of 0); 10 c) subjecting each element from a previously transmitted group to secondary channel coding with one from a family of channel codes, the channel code family being augmented to include a symbol corresponding to no transmission of the element (redundancy factor of 0); and d) transmitting the source data over a sequence of transmission slots, each 15 transmission slot comprising a time interval wherein, in each transmission slot, primary channel coded symbols from elements in the current group are transmitted together with secondary channel coded symbols from elements in the previously transmitted group; e) wherein the primary and secondary channel coded symbols transmitted within each transmission slot are arranged into network packets such that each 20 transmission slot has its own set of network packets; f) wherein the primary and secondary channel coded symbols sent in each transmission slot are arranged within that slot's network packets following the Priority Encoding Transmission (PET) procedure; g) wherein information concerning lost packets which are not received by a 25 receiver is either received or inferred by the transmitter; h) wherein the information concerning lost packets which contain primary coded symbols from said previously transmitted group is used by the transmitter to select secondary channel codes for the elements of the previously transmitted group; and 30 i) wherein the secondary channel coding of each element of the previously transmitted group is performed in a manner which takes advantage of the availability at the receiver of primary coded symbols from the previously transmitted group in packets which were successfully transmitted, so as to avoid or reduce redundant retransmission. 35

2. A method according to claim 1, wherein one or both of said families of channel codes, include a symbol with rate 1 (redundancy factor 1). WO 2005/027430 PCT/AU2004/001246 - 41

3. The method of Claim 1 or 2, in which the families of channel codes used for primary and secondary channel coding have the properties of maximum distance separable (MDS) codes. 5

4. The method of Claim 1 or 2, in which the transmitter determines primary and secondary channel codes to use during a particular transmission slot, using estimates of the anticipated packet loss rate.

5. The methods of any of the preceding claims, in which primary and secondary 10 channel codes are selected within each transmission slot in a manner which is consistent with maximizing a measure of total expected quality of the reconstructed media stream, taken over all groups of elements, where the expectation is conditioned upon knowledge of the channel codes selected in previous transmission slots and the packets lost from previous transmission slots, subject to a constraint on the total transmitted data rate. 15

6. The method of Claim 5, in which the maximization procedure is constrained by a limit on the number of symbols which can be transmitted within the current transmission slot. 20

7. The method of claim 5, in which the transmitter formulates and optimizes an optimization objective which involves hypotheses regarding the secondary channel codes which will be assigned to elements from the current frame in a future transmission slot, in the event that one packet from the current slot is lost, two packets from the current slot are lost, three packets from the current slot are lost, and so forth. 25

8. The method of Claim 7, in which the optimization objective involves a lambda-balance equation, in which lambda is assumed to take the same value in both the current transmission slot and the future transmission slot in which the hypothetical secondary channel codes shall be selected. 30

9. The methods of any of the preceding claims, in which the scalably compressed media data represents a compressed video signal.

10. The methods of any of the preceding claims, in which the scalably compressed 35 media data represents a compressed audio signal.

11. A method for receiving streamed scalably compressed media data over a lossy WO 2005/027430 PCT/AU2004/001246 - 42 packet-based communication network, wherein the data is received and sorted into independently encoded groups of elements, comprising the steps of: a) receiving data over a sequence of reception slots, each reception slot comprising a time interval wherein primary channel coded symbols from elements in a 5 current group are received together with secondary channel coded symbols from elements in a previous group; b) wherein the primary and secondary channel coded symbols within each reception slot are arranged into network packets such that each reception slot has its own set of network packets; 10 c) wherein the primary and secondary channel coded symbols received in each reception slot are arranged within that slot's network packets following the Priority Encoding Transmission (PET) procedure; d) wherein the method further comprises the step of unpacking the PET packets and sorting the received data into primary channel coded symbols from 15 elements in the current group and secondary channel coded symbols from elements in the previous group; e) assembling the secondary channel coded symbols from elements in the previous group with primary channel coded symbols from elements in the previous group which have already been received in a previous reception slot; 20 f) decoding said assembled information for the previous group; and g) determining and sending information to the transmitter, concerning lost packets and/or packets which have been received, h) wherein the information concerning lost and/or available packets which contain primary coded symbols from said current group is capable of being used by the 25 transmitter to select secondary channel codes for the elements of the current group in a manner which takes advantage of the availability at the receiver of primary coded symbols from the current group in packets which were successfully received, so as to avoid or reduce redundant retransmission. 30

12. A method according to claim 11, wherein the steps of assembling and decoding said secondary and primary channel coded information for elements in the previous group comprise the steps of: a) decoding said secondary channel coded symbols; b) augmenting the primary channel coded information for those same 35 elements; and c) decoding said augmented channel coded information to recover elements from the previous group. WO 2005/027430 PCT/AU2004/001246 - 43

13. A computer program including instructions for controlling a computing system for implementing the method of any one of the preceding claims. 5

14. A computer readable medium having stored thereon the computer program of claim 13.

15. A transmitter for transmitting streaming scalably compressed media data over lossy packet-based communication networks, comprising: 10 a) means for obtaining source data comprising independently encoded groups of elements; b) means for subjecting each element in a current group to primary channel coding with one from a family of channel codes, the channel code family being augmented to include a symbol corresponding to no transmission of the element 15 (redundancy factor of 0); c) means for subjecting each element from a previously transmitted group to secondary channel coding with one from a family of channel codes, the channel code family being augmented to include a symbol corresponding to no transmission of the element (redundancy factor of 0); and 20 d) means for transmitting the source data over a sequence of transmission slots, each transmission slot comprising a time interval wherein, in each transmission slot, primary channel coded symbols from elements in the current group are transmitted together with secondary channel coded symbols from elements in the previously transmitted group; 25 e) wherein the primary and secondary channel coded symbols transmitted within each transmission slot are arranged into network packets such that each transmission slot has its own set of network packets; f) wherein the primary and secondary channel coded symbols sent in each transmission slot are arranged within that slot's network packets following the Priority 30 Encoding Transmission (PET) procedure; g) wherein information concerning lost packets which are not received by a receiver is either received or inferred by the transmitter; h) wherein the information concerning lost packets which contain primary coded symbols from said previously transmitted group is used by the transmitter to 35 select secondary channel codes for the elements of the previously transmitted group; and i) wherein the secondary channel coding of each element of the previously WO 2005/027430 PCT/AU2004/001246 - 44 transmitted group is performed in a manner which takes advantage of the availability at the receiver of primary coded symbols from the previously transmitted group in packets which were successfully transmitted, so as to avoid or reduce redundant retransmission. 5

16. A receiver for receiving streamed scalably compressed media data over a lossy packet-based communication network, wherein the data is received and sorted into independently encoded groups of elements, comprising: a) means for receiving data over a sequence of reception slots, each reception slot comprising a time interval wherein primary channel coded symbols from elements 10 in a current group are received together with secondary channel coded symbols from elements in a previous group; b) wherein the primary and secondary channel coded symbols within each reception slot are arranged into network packets such that each reception slot has its own set of network packets; 15 c) wherein the primary and secondary channel coded symbols received in each reception slot are arranged within that slot's network packets following the Priority Encoding Transmission (PET) procedure; d) wherein the receiver further comprises means for unpacking the PET packets and sorting the received data into primary channel coded symbols from 20 elements in the current group and secondary channel coded symbols from elements in the previous group; e) means for assembling the secondary channel coded symbols from elements in the previous group with primary channel coded symbols from elements in the previous group which have already been received in a previous reception slot; 25 f) means for decoding said assembled information for the previous group; and g) means for determining and sending information to the transmitter, concerning lost packets and/or packets which have been received, h) wherein the information concerning lost and/or available packets which contain primary coded symbols from said current group is capable of being used by the 30 transmitter to select secondary channel codes for the elements of the current group in a manner which takes advantage of the availability at the receiver of primary coded symbols from the current group in packets which were successfully received, so as to avoid or reduce redundant retransmission. -45

17. A method for transmitting streaming scalable compressed media data over lossy packet based communication networks substantially as herein described with reference to the accompanying drawings and/or references. 5

18. A receiver for receiving streamed scalabilities known scalably compressed media data over a lossy packet based communication network wherein the data is received and stored into independently coded groups of elements substantially as herein described with reference to the accompanying drawings and/or references. 2160601_1 (GHMatters) 8/01/10