US20040179598A1 - Multi-path transmission of fine-granular scalability video streams - Google Patents
Multi-path transmission of fine-granular scalability video streams Download PDFInfo
- Publication number
- US20040179598A1 US20040179598A1 US10/371,086 US37108603A US2004179598A1 US 20040179598 A1 US20040179598 A1 US 20040179598A1 US 37108603 A US37108603 A US 37108603A US 2004179598 A1 US2004179598 A1 US 2004179598A1
- Authority
- US
- United States
- Prior art keywords
- bit
- rate
- layers
- low
- rate enhancement
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000005540 biological transmission Effects 0.000 title description 5
- 238000000034 method Methods 0.000 claims abstract description 24
- 238000000638 solvent extraction Methods 0.000 claims description 10
- 230000000007 visual effect Effects 0.000 claims description 5
- 238000013139 quantization Methods 0.000 claims description 3
- 230000032258 transport Effects 0.000 abstract description 5
- 238000004891 communication Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000005192 partition Methods 0.000 description 2
- 238000012552 review Methods 0.000 description 2
- 238000007619 statistical method Methods 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000010420 art technique Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
- H04N21/2662—Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/146—Data rate or code amount at the encoder output
- H04N19/147—Data rate or code amount at the encoder output according to rate distortion criteria
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/187—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a scalable video layer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
- H04N19/19—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding using optimisation based on Lagrange multipliers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
- H04N19/33—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the spatial domain
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
- H04N19/34—Scalability techniques involving progressive bit-plane based encoding of the enhancement layer, e.g. fine granular scalability [FGS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
- H04N19/37—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability with arrangements for assigning different transmission priorities to video input data or to video coded data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
- H04N19/39—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability involving multiple description coding [MDC], i.e. with separate layers being structured as independently decodable descriptions of input picture data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234327—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by decomposing into layers, e.g. base layer and one or more enhancement layers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/631—Multimode Transmission, e.g. transmitting basic layers and enhancement layers of the content over different transmission paths or transmitting with different error corrections, different keys or with different transmission protocols
Definitions
- FGS provides continuous rate-control of the streaming video because the enhancement layer bitstream can be truncated at any point to achieve a target bit-rate.
- Another advantage of the FGS compared to traditional scalable coding schemes, is its error resiliency. Losses or corruptions in one or more frames in the decoded enhancement layers do not propagate to following frames. Following frames are always first decoded from the base layer before the enhancement layers are applied.
- the base layer and the low bit-rate layers are duplicated into multiple base layers and multiple low bit-rate enhancement layers.
- FIG. 2 shows a method 200 according to our invention.
- An input video 201 is encoded 210 using fine-granularity scalability into a base layer 211 , and one or more low bit-rate enhancement layers 212 , e.g., less than about 30 to 40 Kbps.
- Each of the bitstreams 211 - 212 is duplicated 220 into multiple streams 221 - 222 for transmission 240 over the multiple paths 105 of the network 110 to the receiver 102 .
- the partitioning of the enhancement layer can be performed as follows for two possible paths.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
Description
- This invention relates generally to streaming compressed videos, and more particularly to transmitting fine-granular scalability streaming of videos over multiple network paths.
- For applications that stream a compressed video over a network, such as the Internet, one important concern is to deliver the video stream to a receiver with different resources, access paths, and processors. Therefore, content of the video is dynamically adapted to heterogeneous environments found in such networks.
- Fine-granular scalability (FGS) has been developed for the MPEG-4 standard to adapt videos to such dynamically varying network environments, see “ISO/IEC 14496-2:1999/FDAM4, “Information technology—coding of audio/visual objects, Part 2: Visual.” An overview of this amendment to the MPEG-4 standard is described by Li, “Overview of Fine Granularity Scalability in MPEG-4 Video Standard,” IEEE Trans. on Circuits and Systems for Video Technology, Vol.11, No.3, pp. 301-317, March 2001.
- An MPEG-4 FGS encoder generates two bitstreams: one is a base layer stream, and the other is an enhancement layer stream. The purpose and importance of the two bitstreams are different. The base layer stream provides a basic decoded video. The base layer stream must be correctly decoded before the enhancement layer stream can be used. Therefore, the base layer must be strongly protected. The enhancement layer stream can be used to improve the quality of the basic video.
- FGS coding is a radical departure from traditional scalable encoding. With traditional scalable encoding, the content is encoded into a base layer bitstream and possibly several enhancement layers, where the granularity is only as fine as the number of enhancement layers that are formed. The resulting rate-distortion curve resembles a step-like function.
- In contrast, FGS encoding provides an enhancement layer bitstream that is continually scalable. Reconstructed frames of the base layer bitstream are subtracted from the input video generates the enhancement layer bitstream. This yields an FGS residual signal in the spatial domain. Discrete cosine transform (DCT) encoding is applied to the residual signal. Then the DCT coefficients are coded by a bit-plane coding scheme. Bit-plane coding can generate multiple sub-layers for the enhancement layer bitstream. Hereinafter, the sub-layers are also referred to as enhancement layers. Bit-plane encoding allows the enhancement layer bitstream to be truncated at any point. Thus, the quality of the reconstructed video is proportional to the number of bits that are decoded from the enhancement layer bitstream.
- FGS provides continuous rate-control of the streaming video because the enhancement layer bitstream can be truncated at any point to achieve a target bit-rate. Another advantage of the FGS, compared to traditional scalable coding schemes, is its error resiliency. Losses or corruptions in one or more frames in the decoded enhancement layers do not propagate to following frames. Following frames are always first decoded from the base layer before the enhancement layers are applied.
- FGS effort has focused on the following areas: improving coding efficiency, see Kalluri, “Single-Loop Motion-Compensated based Fine-Granular Scalability (MC-FGS),” MPEG2001/M6831, July 2001, and Wu et al., “A Framework for Efficient Fine Granularity Scalable Video Coding,” IEEE Trans. on Circuits and System for Video Technology, Vol. 11, No. 3, pp. 332-344, March 2001; truncating the enhancement layers to minimize quality variation between adjacent frames, see Zhang et al., “Constant Quality Constrained Rate Allocation for FGS Video Coded Bitstreams,” Visual Communications and Image Processing 2002, Proceedings of SPIE, Vol. 4671, pp. 817-827, 2000, Cheong et al., “FGS coding scheme with arbitrary water ring scan order,” ISO/IEC JTC1/SC29/WG11, MPEG 2001/M7442, July 2001, and Lim et al., “Macroblock reordering for FGS,” ISO/IEC JTC1/SC29/WG11, MPEG 2000/M5759, March 2000; and modifying the FGS coding structure to add time scalability, see Van der Schaar et al., “A Hybrid Temporal-SNR Fine Granular Scalability for Internet Video,” IEEE Trans. on Circuits and System for Video Technology, Vol. 11, No. 3, pp. 318-331, March 2001, and Yan et al., “Macroblock-based Progressive Fine Granularity Spatial Scalability (mb-PFGSS),” ISO/IEC JTC1/SC29/WG11, MPEG2001/M7112, March 2001.
- The problem of protecting the FGS stream has also been addressed. Various techniques are available for encoding and transmitting the base layer, see Wang et al., “Error Control and Concealment for Video Communication: A Review,” Proceedings of the IEEE, vol. 86, No. 5, pp. 974-997, May 1998, and Wang et al., “Error Resilient Video Coding Techniques,” IEEE Signal Processing Magazine, pp. 61-82, July 2000.
- An adaptive forward error correcting code (FEC) can be used to protect the FGS enhancement layers, see Wang et al., “Channel-Adaptive Error Control for Scalable Video over Wireless Channel,” The 7th International Workshop on Mobile Multimedia Communications (MoMuC) 2000, October 2000. Unequal error protection can also be used to transport the FGS bitstreams, see Van der Schaar et al. “Unequal packet Loss Resilience for Fine Granularity Scalability Video,” IEEE Trans. on Multimedia, Vol. 3, No. 4, pp. 381-394, December 2001.
- A big challenge for transmitting a streaming video is instantaneous congestion in the network. Theoretically, FGS can adapt dynamically to the available bandwidth by measuring characteristics of the network end-to-end, and estimating its components. However, in many cases, the duration of the congestion is shorter than the time it takes to measure and estimate to effectively adapt the FGS video stream.
- FEC and re-transmission can sometimes recover packets lost due to the instantaneous congestion. However, both of these solutions can also aggravate congestion by introducing additional traffic.
- Numerous multi-path technologies are known for both wired and wireless networks, see Cidon et al., “Analysis of Multi-path Routing,” IEEE/ACM transaction on Networking,” Vol. 7, No. 6, pp. 885-896, December 1999, Su et al., “Dynamic multi-path routing: asymptotic approximation and simulations,” ACM SIGMETRICS Performance Evaluation Review, Volume: 29, Issue: 1, pp. 25-36, June 2001, and Pham et al., “Multi-path routing protocol with load balancing policy in mobile ad hoc network,” 4th International Workshop on Mobile and Wireless Communications Network, 2002.
- In those cases, each path may have a lower bandwidth, but the total available bandwidth is higher than a single path.
- Multiple description coding (MDC) has been developed to transport a standard (non-FGS) compressed video stream over multiple independent paths. In MDC, different descriptions of the content of the video are transmitted over each path. Because errors occur independently in the different paths, the probability that all paths incur the same error is low, see Wang et al., “Error Resilient Video Coding Techniques,” IEEE Signal Processing Magazine, pp. 61-82, July 2000. MDC techniques for transporting video streams over multiple wireless channels are also known, see Reibman et al., “Transmission of Multiple Description and Layered Video over an EGPRS Wireless Network,” ICIP 2000, pp. 136-139, 2000, Zheng et al., “Wireless Video Performance Through BLAST Testbed,” VTC 2001, pp. 141-146, 2001, and Kamaci et al., “Multiple description coding with multiple transmit and receive antennas for wireless channels: the case of digital modulation,” Global Telecommunications Conference, 2001, GLOBECOM '01. IEEE, Volume: 6 pp. 3272-3276, 2001.
- However, none of the known prior art techniques describe how to transmit an MPEG-4 FGS compressed video stream over multiple path of a network.
- Therefore, it is desired to provide a system and method for transmitting an FGS video stream over multiple paths so that errors and instantaneous congestion are reduced.
- A method transports a video over a network that includes multiple paths. First, a video is encoded into a base layer, one or multiple low bit-rate enhancement layers, and one or multiple-high bit-rate enhancement layers using fine-granularity scalability.
- The base layer and the low bit-rate layers are duplicated into multiple base layers and multiple low bit-rate enhancement layers.
- The one or more high bit-rate layers are partitioned into multiple partial low-bit rate layers, and then each layer is transmitted over a different path of the network to a receiver.
- FIG. 1 is a block diagram of a communications system that uses the invention;
- FIG. 2 is a flow diagram of a method for transmitting an FGS video stream over multiple paths of a network according to the invention; and
- FIG. 3 is a diagram of a trellis used by the invention.
- System Structure
- FIG. 1 show a
communications system 100, such as the Internet, that can use our invention. Thesystem 100 includes a sender 101, areceiver 102, anetwork 110 includingmultiple nodes 103, e.g., routers and bridges, all connected bymultiple paths 105. Our invention provides a method for transmitting an MPEG-4 fine-granular scalability (FGS) video stream over themultiple paths 105 of thenetwork 110. - System Operation
- FIG. 2 shows a method200 according to our invention. An input video 201 is encoded 210 using fine-granularity scalability into a
base layer 211, and one or more low bit-rate enhancement layers 212, e.g., less than about 30 to 40 Kbps. Each of the bitstreams 211-212 is duplicated 220 into multiple streams 221-222 for transmission 240 over themultiple paths 105 of thenetwork 110 to thereceiver 102. - One or more high bit-rate enhancement layers213 are encoded 210, using multiple descriptions 205, and then partitioned 230 into partial high bit-rate layers 231 for transmission 240 over the
multiple paths 105 of thenetwork 110 for delivery to thereceiver 102. - Statistical Analysis of Bit-Rates and Errors
- The idea behind our invention results from a careful statistical analysis of characteristics of encoded FGS bit-streams. We begin by encoding standard video sequences such as “Coast Guard” and “Akiyo.” Coast Guard shows a boat moving through highly textured water, and Akiyo shows a fairly static news announcer against simple background. The sequences are encoded in a common-intermediate-format (CIF) by setting a quantization parameters Q of the base layer to 31 for both I frames and P frames. There are no B frames in these two standard video sequences.
- For the base layer (BL), and the low bit-rate enhancement layers such as EL1 and EL2, which carry the information of large residual errors, the bit-rate is relatively low, e.g., about 31.3 Kbps and 6.8 Kbps, respectively. However, for enhancement layers EL3 and up, the bit-rate can be hundreds of Kbps. Consequently, the peak average signal-to-noise ratio (PSNR) of the entire sequence increases significantly. For instance, the EL3 bit-rate is about six times of the bit-rate of the base layer, and the EL7 bit-rate is over eighteen times of the base layer bit-rate. In other words, there is a significant difference between the rates of the low and high bit-rate streams.
- Because of the small bandwidth requirement, it is reasonable to use forward error correction (FEC) or other known error protection schemes to protect the
base layer 211 and the low bit-rate enhancement layers 212. More overhead-bits are required to protect the high bit-rate enhancement layers 213. However, as stated above, the network often encounters instantaneous congestion. This causes the high bit-rate enhancement layer to be truncated when the bandwidth is limited, and the quality of the decoded video is substantially decreased. In such cases, the prior art schemes are not effective to protect the high bit-rate enhancement layers 213. - Multiple Paths
- In a congested network, it may be easier to find several lower bandwidth paths than to find a single large bandwidth path. Therefore, we duplicate the streams, and transmit240 the multiple streams 221-222, 231 over
multiple paths 105. Using the multiple path solution according to our invention also decreases the likelihood that packets are lost. - One could simply duplicate the original video content for each path. This would improve the robustness of the video transmission, because a packet lost in one path can be recovered from other paths. However, if the bandwidth of each path is insufficient, then only a low quality video can be reconstructed. Another simple solution could partition video stream into non-overlapping stream, and use a different path for each stream. This would improve the transmit bit-rate. However, if a packet is lost in any path, then the corresponding lost information cannot be reconstructed.
- Therefore, we provide the method200 for transmitting an MPEG-4 FGS video stream over the
multiple path 105 of the network. - Duplication of Base Layer and Low Bit-Rate Enhancement Layers
- The
base layer 211 and low bit-rate enhancement layers 212 of the input video 201 are encoded 210 using FGS and traditional forward error correction (FEC) schemes. These bitstreams of the video are duplicated 220 into multiple streams 221-222 for every available path. In this way, the base layer and the low bit-rate enhancement layers are protected. - Partitioning of High Bit-Rate Enhancement Layers
- The high bit-rate enhancement layers213 are partitioned 230 into multiple partial low bit-rate bitstreams 231. Each one of the multiple partial low-bit-rate bitstreams is converted using a different one of the multiple descriptions 205 to take advantage of the multiple paths. Each path carries one of the multiple bitstreams description of the original high bit-rate enhancement layer 213. In this way, each partitioned low bit-rate enhancement layer 231 has a lower bit-rate when compared with the input high bit-rate enhancement layers 213. Thus, it possible to transmit 240 each partitioned bitstream 231 over a lower bit-rate path.
- With the partitioning of the high bit-rate FGS enhancement layers213, errors are reduced. If the information in one packet is lost, then similar information can be recovered from another path to improve the quality of the decoded video.
- Bit Distribution
- To partition the high bit-rate enhancement layers213, we distribute “1” bits of the high bit-rate layer using different descriptions within the available bandwidth. A simple method distributes the “1” bits evenly among the multiple partial enhancement layers 231.
- For example, if two paths are available, and an input block includes 64 bits
- 000011000101 . . . 110
- then the “1” bits are evenly distributed into two descriptions as:
- description1: 000010000100 . . . 100
- description2: 000001000001 . . . 010.
- Alternatively, the “1” bits are distributed according to their positions in the input block and their corresponding importance to the perceived visual quality of the video. For example, coefficients from the upper-left corner in the block significantly reflect luminance changes. Therefore, the corresponded “1” bits are duplicated in all of the partial layers, and the remaining “1” bits are assigned evenly among the partial layers.
- Generally, if two paths are available, then a “1” bit in the input enhancement layer can be assigned to the 1st partial layer, or to the 2nd partial layer, or to both partial layers, or to neither partial layers, i.e., there are four possible choices. This yields different rates and distortions for each description. Therefore, rate-distortion optimization can determine how the “1” bits should be distributed. The manner in which the bits are distributed over the four choices is described in greater detail below.
- Because different paths can have different available bandwidths and different packet loss characteristics, the partitioning of the enhancement layer can be performed as follows for two possible paths.
- Trellis Search
- First, distribute the appropriate number of bits to each image block301 according to the available bandwidth. Then, for each
block 301, apply atrellis search method 300 with four stages 311-314 to determine the distribution of the “1” bits for two paths. - When a “1” bit is encountered in the
block 301, a new stage, i.e., “10” 311, “01” 312, “11” 313, and “00” 314 is used to determine whether the “1” bit is assigned to a 1st partial layer, to a 2nd partial layer, to both layers, or to none. For each stage, there are four possible incoming routes from a previous stage. - Cost Function
- Each route has a corresponding cost function
- J(λ)=D(R 1 , R 2 , P loss)+λ(R 1 +R 2),
- where R1 and R2 are the numbers of bits produced up to the current stage in each partial layer, Ploss is a parameters related to a status of the
network 100, D(.) is an overall distortion of the two descriptions associated with the bit rate and the network status, and λ is an empirical parameter specified according to the quantization parameter of the base layer block. The status can include an average packet loss ratio and a loss burst rate. - Then, the “1” bit is assigned according to the cost function that produces a minimum cost J(λ). This process is repeated until a bit-budget for each description is satisfied.
- With the multi-path transport of FGS videos according to the invention, the PSNR gain can be in the range of 2.0 to 2.6 dB, depending on the complexity of the input video.
- Although the invention has been described by way of examples of preferred embodiments, it is to be understood that various other adaptations and modifications may be made within the spirit and scope of the invention. Therefore, it is the object of the appended claims to cover all such variations and modifications as come within the true spirit and scope of the invention
Claims (11)
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/371,086 US6973128B2 (en) | 2003-02-21 | 2003-02-21 | Multi-path transmission of fine-granular scalability video streams |
CNA2004800000326A CN1698384A (en) | 2003-02-21 | 2004-02-19 | Method and system for transporting a video over a network including multiple |
PCT/JP2004/001930 WO2004075561A1 (en) | 2003-02-21 | 2004-02-19 | Method and system for transporting a video over a network including multiple |
JP2006502675A JP4664276B2 (en) | 2003-02-21 | 2004-02-19 | Method and system for transferring video over a network including multiple paths |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/371,086 US6973128B2 (en) | 2003-02-21 | 2003-02-21 | Multi-path transmission of fine-granular scalability video streams |
Publications (2)
Publication Number | Publication Date |
---|---|
US20040179598A1 true US20040179598A1 (en) | 2004-09-16 |
US6973128B2 US6973128B2 (en) | 2005-12-06 |
Family
ID=32907676
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/371,086 Expired - Fee Related US6973128B2 (en) | 2003-02-21 | 2003-02-21 | Multi-path transmission of fine-granular scalability video streams |
Country Status (4)
Country | Link |
---|---|
US (1) | US6973128B2 (en) |
JP (1) | JP4664276B2 (en) |
CN (1) | CN1698384A (en) |
WO (1) | WO2004075561A1 (en) |
Cited By (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030009577A1 (en) * | 2001-07-03 | 2003-01-09 | Apostolopoulos John G. | Method for handing off multiple description streaming media sessions between servers in fixed and mobile streaming media systems |
US20080013620A1 (en) * | 2006-07-11 | 2008-01-17 | Nokia Corporation | Scalable video coding and decoding |
US20090201990A1 (en) * | 2008-02-04 | 2009-08-13 | Alcatel-Lucent | Method and device for reordering and multiplexing multimedia packets from multimedia streams pertaining to interrelated sessions |
US20090296821A1 (en) * | 2008-06-03 | 2009-12-03 | Canon Kabushiki Kaisha | Method and device for video data transmission |
US20100091839A1 (en) * | 2006-09-28 | 2010-04-15 | Zhenyu Wu | Flexible redundancy coding |
US20110050843A1 (en) * | 2009-09-01 | 2011-03-03 | Avermedia Information, Inc. | Video Conference Apparatus and Method Thereof |
US20110075592A1 (en) * | 2008-05-15 | 2011-03-31 | Gnodal Limited | Method of Data Delivery Across a Network |
US20110274180A1 (en) * | 2010-05-10 | 2011-11-10 | Samsung Electronics Co., Ltd. | Method and apparatus for transmitting and receiving layered coded video |
US20130057767A1 (en) * | 2011-09-01 | 2013-03-07 | Canon Kabushiki Kaisha | Communication of video information |
US20160227229A1 (en) * | 2015-02-04 | 2016-08-04 | Harris Corporation | Mobile ad hoc network media aware networking element |
WO2021056575A1 (en) * | 2019-09-29 | 2021-04-01 | 华为技术有限公司 | Low-delay joint source-channel coding method, and related device |
Families Citing this family (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR100353851B1 (en) * | 2000-07-07 | 2002-09-28 | 한국전자통신연구원 | Water ring scan apparatus and method, video coding/decoding apparatus and method using that |
FR2821509B1 (en) * | 2001-02-27 | 2003-05-30 | Thomson Csf | METHOD AND DEVICE FOR TRANSFERRING DATA IN A COMMUNICATION NETWORK HAVING LOW-RATE ARTERY |
WO2007026604A1 (en) * | 2005-08-29 | 2007-03-08 | Nec Corporation | Multicast node apparatus, multicast transfer method and program |
KR101340937B1 (en) | 2005-09-29 | 2013-12-13 | 톰슨 리서치 펀딩 코포레이션 | Method and apparatus for constrained variable bit rate(vbr) video encoding |
KR20070077059A (en) * | 2006-01-19 | 2007-07-25 | 삼성전자주식회사 | Method and apparatus for entropy encoding/decoding |
EP2041976A4 (en) * | 2006-07-12 | 2012-06-20 | Nokia Corp | Signaling of region-of-interest scalability information in media files |
US8576858B2 (en) * | 2006-12-13 | 2013-11-05 | Viasat, Inc. | Multiple transmission paths for hierarchical layers |
US8411572B2 (en) * | 2006-12-13 | 2013-04-02 | Viasat, Inc. | ACM and fixed coding and modulation of hierarchical layers |
US8358690B2 (en) * | 2006-12-13 | 2013-01-22 | Viasat, Inc. | Predictive adaptive coding and modulation |
US8456986B2 (en) * | 2006-12-13 | 2013-06-04 | Viasat, Inc. | Video and data network load balancing |
US20090041100A1 (en) | 2006-12-13 | 2009-02-12 | Viasat, Inc. | Link aware mobile data network |
US8395993B2 (en) * | 2006-12-13 | 2013-03-12 | Viasat, Inc. | Video and data network load balancing with video placeholder |
US8411571B2 (en) * | 2006-12-13 | 2013-04-02 | Viasat, Inc. | Video and data network load balancing with video drop |
KR100937590B1 (en) * | 2007-10-23 | 2010-01-20 | 한국전자통신연구원 | Multiple quality image contents service system and upgrade method thereof |
JP5253571B2 (en) * | 2008-06-20 | 2013-07-31 | ドルビー ラボラトリーズ ライセンシング コーポレイション | Video compression under multiple distortion suppression |
JP6103003B2 (en) * | 2010-02-24 | 2017-03-29 | 株式会社リコー | Transmission terminal, transmission method, and program |
US20120076204A1 (en) * | 2010-09-23 | 2012-03-29 | Qualcomm Incorporated | Method and apparatus for scalable multimedia broadcast using a multi-carrier communication system |
US9363131B2 (en) * | 2013-03-15 | 2016-06-07 | Imagine Communications Corp. | Generating a plurality of streams |
US9306597B1 (en) * | 2015-03-30 | 2016-04-05 | Microsoft Technology Licensing, Llc | Data compression |
CN105872610A (en) * | 2015-12-10 | 2016-08-17 | 乐视体育文化产业发展(北京)有限公司 | Method, equipment and system for playing multiple paths of video data |
CN109587531A (en) * | 2017-09-28 | 2019-04-05 | 华为技术有限公司 | Video transmission method, apparatus and system |
CN112866750B (en) * | 2020-12-31 | 2022-01-11 | 抖动科技(深圳)有限公司 | Video data processing method, video playing method and related equipment |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6700933B1 (en) * | 2000-02-15 | 2004-03-02 | Microsoft Corporation | System and method with advance predicted bit-plane coding for progressive fine-granularity scalable (PFGS) video coding |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030009578A1 (en) * | 2001-07-03 | 2003-01-09 | Apostolopoulos John G. | Method and system for delivering streaming media to fixed clients or mobile clients with multiple description bitstreams |
US20030007515A1 (en) * | 2001-07-03 | 2003-01-09 | Apostolopoulos John G. | System and method for receiving mutiple description media streams in fixed and mobile streaming media systems |
US6757735B2 (en) * | 2001-07-03 | 2004-06-29 | Hewlett-Packard Development Company, L.P. | Method for distributing multiple description streams on servers in fixed and mobile streaming media systems |
US7200402B2 (en) * | 2001-07-03 | 2007-04-03 | Hewlett-Packard Development Company, L.P. | Method for handing off streaming media sessions between wireless base stations in a mobile streaming media system |
-
2003
- 2003-02-21 US US10/371,086 patent/US6973128B2/en not_active Expired - Fee Related
-
2004
- 2004-02-19 JP JP2006502675A patent/JP4664276B2/en not_active Expired - Fee Related
- 2004-02-19 WO PCT/JP2004/001930 patent/WO2004075561A1/en active Application Filing
- 2004-02-19 CN CNA2004800000326A patent/CN1698384A/en active Pending
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6700933B1 (en) * | 2000-02-15 | 2004-03-02 | Microsoft Corporation | System and method with advance predicted bit-plane coding for progressive fine-granularity scalable (PFGS) video coding |
Cited By (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6996618B2 (en) * | 2001-07-03 | 2006-02-07 | Hewlett-Packard Development Company, L.P. | Method for handling off multiple description streaming media sessions between servers in fixed and mobile streaming media systems |
US20030009577A1 (en) * | 2001-07-03 | 2003-01-09 | Apostolopoulos John G. | Method for handing off multiple description streaming media sessions between servers in fixed and mobile streaming media systems |
TWI423679B (en) * | 2006-07-11 | 2014-01-11 | Nokia Corp | Scalable video coding and decoding |
US20080013620A1 (en) * | 2006-07-11 | 2008-01-17 | Nokia Corporation | Scalable video coding and decoding |
US8699583B2 (en) * | 2006-07-11 | 2014-04-15 | Nokia Corporation | Scalable video coding and decoding |
US20100091839A1 (en) * | 2006-09-28 | 2010-04-15 | Zhenyu Wu | Flexible redundancy coding |
US20090201990A1 (en) * | 2008-02-04 | 2009-08-13 | Alcatel-Lucent | Method and device for reordering and multiplexing multimedia packets from multimedia streams pertaining to interrelated sessions |
US8681854B2 (en) * | 2008-02-04 | 2014-03-25 | Alcatel Lucent | Method and device for reordering and multiplexing multimedia packets from multimedia streams pertaining to interrelated sessions |
US8774063B2 (en) * | 2008-05-15 | 2014-07-08 | Cray Uk Limited | Method of data delivery across a network |
US20110075592A1 (en) * | 2008-05-15 | 2011-03-31 | Gnodal Limited | Method of Data Delivery Across a Network |
FR2932050A1 (en) * | 2008-06-03 | 2009-12-04 | Canon Kk | METHOD AND DEVICE FOR TRANSMITTING VIDEO DATA |
US8605785B2 (en) | 2008-06-03 | 2013-12-10 | Canon Kabushiki Kaisha | Method and device for video data transmission |
US20090296821A1 (en) * | 2008-06-03 | 2009-12-03 | Canon Kabushiki Kaisha | Method and device for video data transmission |
US8233027B2 (en) * | 2009-09-01 | 2012-07-31 | Avermedia Information, Inc. | Video conference apparatus and method thereof |
US20110050843A1 (en) * | 2009-09-01 | 2011-03-03 | Avermedia Information, Inc. | Video Conference Apparatus and Method Thereof |
US20110274180A1 (en) * | 2010-05-10 | 2011-11-10 | Samsung Electronics Co., Ltd. | Method and apparatus for transmitting and receiving layered coded video |
US20130057767A1 (en) * | 2011-09-01 | 2013-03-07 | Canon Kabushiki Kaisha | Communication of video information |
US9420225B2 (en) * | 2011-09-01 | 2016-08-16 | Canon Kabushiki Kaisha | Communication of video information |
US20160227229A1 (en) * | 2015-02-04 | 2016-08-04 | Harris Corporation | Mobile ad hoc network media aware networking element |
WO2021056575A1 (en) * | 2019-09-29 | 2021-04-01 | 华为技术有限公司 | Low-delay joint source-channel coding method, and related device |
Also Published As
Publication number | Publication date |
---|---|
JP2006520554A (en) | 2006-09-07 |
US6973128B2 (en) | 2005-12-06 |
JP4664276B2 (en) | 2011-04-06 |
WO2004075561A1 (en) | 2004-09-02 |
CN1698384A (en) | 2005-11-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6973128B2 (en) | Multi-path transmission of fine-granular scalability video streams | |
Ha et al. | Layer-weighted unequal error protection for scalable video coding extension of H. 264/AVC | |
Van Der Schaar et al. | Adaptive motion-compensation fine-granular-scalability (AMC-FGS) for wireless video | |
KR100952185B1 (en) | System and method for drift-free fractional multiple description channel coding of video using forward error correction codes | |
Stockhammer et al. | Progressive texture video streaming for lossy packet networks | |
Zhang et al. | Robust scalable video streaming over Internet with network-adaptive congestion control and unequal loss protection | |
Zhou et al. | Multi-path transport of FGS video | |
Chan et al. | A multiple description coding and delivery scheme for motion-compensated fine granularity scalable video | |
Le Leannec et al. | Error-resilient video transmission over the Internet | |
Zhu et al. | Network-adaptive rate control with unequal loss protection for scalable video over Internet | |
Yang et al. | Degressive error protection algorithm for MPEG-4 FGS video streaming | |
Mrak et al. | Scalable video coding in network applications | |
Bajic et al. | EZBC video streaming with channel coding and error concealment | |
Le et al. | Multiple description video coding for real-time applications using HEVC | |
Zhao et al. | RD-Based Adaptive UEP for H. 264 Video Transmission in Wireless Networks | |
Zhao et al. | Video streaming using standard-compatible scalable multiple description coding based on SVC | |
Khan et al. | Unequal error protection (UEP) of wavelet coded video using multi-layered hierarchical QAM | |
Gang et al. | Error resilient multiple reference selection for wireless video transmission | |
Kazemi | End-to-end distortion modeling and channel adaptive optimization of mixed layer multiple description coding scheme | |
Atta et al. | An efficient layered video codec based on DCT pyramid | |
Radakovic et al. | Low complexity adaptation of H. 264/MPEG-4 SVC for multiple description video coding | |
Zhang et al. | Estimation of the utilities of the NAL units in H. 264/AVC scalable video bitstreams | |
Jarnikov et al. | Wireless Streaming based on a Scalability Scheme using Legacy MPEG-2 Decoders. | |
Naccari et al. | Unequal error protection based on flexible macroblock ordering for robust H. 264/AVC video transcoding | |
Hua et al. | H. 264 scalable video over finite-state Markov chain wireless channels |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: MITSUBISHI ELECTRIC RESEARCH LABORATORIES, INC., M Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ZHOU, JIAN;SHAO, HUAI-RONG;SHEN, CHIA;REEL/FRAME:013809/0882 Effective date: 20030220 |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
FPAY | Fee payment |
Year of fee payment: 8 |
|
REMI | Maintenance fee reminder mailed | ||
LAPS | Lapse for failure to pay maintenance fees |
Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.) |
|
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20171206 |