WO2007043770A1 - Method and apparatus for scalable video adaptation using adaptation operators for scalable video - Google Patents

Method and apparatus for scalable video adaptation using adaptation operators for scalable video Download PDF

Info

Publication number
WO2007043770A1
WO2007043770A1 PCT/KR2006/003989 KR2006003989W WO2007043770A1 WO 2007043770 A1 WO2007043770 A1 WO 2007043770A1 KR 2006003989 W KR2006003989 W KR 2006003989W WO 2007043770 A1 WO2007043770 A1 WO 2007043770A1
Authority
WO
WIPO (PCT)
Prior art keywords
adaptation
svc
bitstream
scalability
operators
Prior art date
Application number
PCT/KR2006/003989
Other languages
French (fr)
Inventor
Jung-Won Kang
Jae-Gon Kim
Jin-Woo Hong
Yong-Man Ro
Young-Suk Kim
Cong-Thang Truong
Original Assignee
Electronics And Telecommunications Research Institute
Research And Industrial Cooperation Group
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Electronics And Telecommunications Research Institute, Research And Industrial Cooperation Group filed Critical Electronics And Telecommunications Research Institute
Priority to JP2008534439A priority Critical patent/JP2009510966A/en
Priority to EP06799070A priority patent/EP1932354A4/en
Priority to CN2006800370962A priority patent/CN101283596B/en
Priority to US12/088,480 priority patent/US20080247460A1/en
Priority claimed from KR1020060097262A external-priority patent/KR100848310B1/en
Publication of WO2007043770A1 publication Critical patent/WO2007043770A1/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234327—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by decomposing into layers, e.g. base layer and one or more enhancement layers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/235—Processing of additional data, e.g. scrambling of additional data or processing content descriptors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462—Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4621—Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H04N21/8543—Content authoring using a description language, e.g. Multimedia and Hypermedia information coding Expert Group [MHEG], eXtensible Markup Language [XML]

Definitions

  • the present invention relates to an apparatus and method of adapting a bitstream to which scalable video coding (SVC) technology is applied, and more particularly, to an apparatus and method in which a bitstream is adapted using SVC adaptation operators, and the SVC adaptation operators for the adapted bitstream is additionally described, thereby allowing the SVC adaptation operators to be used later for new adaptation.
  • SVC scalable video coding
  • DMB digital multimedia broadcasting
  • Mobile communication networks support a variety of terminals, including personal digital assistants (PDAs), mobile phones, and notebook computers, , and wired networks, such as ADSL, support personal computers (PCs).
  • PDAs personal digital assistants
  • ADSL personal computers
  • PCs personal computers
  • IPTV Internet protocol TV
  • MPEG-21 framework to provide more varieties of multimedia content efficiently supports many functions, such as digital rights management (DRM), digital item adaptation (DIA), and digital item declaration (DID).
  • DRM digital rights management
  • DIA digital item adaptation
  • DID digital item declaration
  • the present invention provides an apparatus and method of supporting adaptation of multimedia content to which scalable video coding (SVC) technology is applied.
  • SVC scalable video coding
  • the present invention also provides an apparatus and method in which SVC adaptation operators for appropriately performing adaptation of scalable video at a bitstream level are defined, and effective meanings and description examples for describing the descriptors are suggested, thereby performing effective adaptation suitable for a variety of networks and user environments by using described the Adaptation QoS information.
  • an apparatus for adapting a bitstream to which scalable video coding (SVC) technology is applied including: an Adaptation QoS information extraction unit extracting SVC adaptation operators, and relationships between the SVC adaptation operators and the usage environment information of a terminal from the Adaptation QoS information on the bitstream to which SVC technology is applied; an Adaptation Decision Taking Engine(ADTE) unit determining the SVC adaptation operators corresponding to the usage environment of the terminal receiving the transmitted bitstream among the SVC adaptation operators; and a SVC bitstream extraction unit extracting the bitstream based on the determined SVC adaptation operators.
  • SVC scalable video coding
  • the Adaptation QoS information comprises information on SVC adaptation operators for spatial scalability, temporal scalability and SNR scalability among the standardized SVC adaptation operators.
  • the Adaptation QoS information describes relationships among usage environment information of terminal, SVC adaptation operators for spatial scalability, temporal scalability and SNR scalability, and measurements indicating the overall quality of the bitstream such as a peak SNR(PSNR) and utility rank.
  • the Adaptation QoS information includes descriptions paired with the bandwidth of the terminal, SVC adaptation operators for the spatial scalability, the temporal scalability and the SNR scalability, and the PSNR vector having identical degrees in the bandwidth of the terminal, SVC adaptation operators for the spatial scalability, the temporal scalability and the SNR scalability, and the PSNR vectors formed with an arbitrary degree.
  • the Adaptation QoS information includes descriptions paired with the bandwidth of the terminal, SVC adaptation operators for the spatial scalability and the temporal scalability having identical degrees in the bandwidth of the terminal and SVC adaptation operators for the spatial scalability and the temporal scalability formed with an arbitrary degree and expressed SVC adaptation operators for the SNR scalability in the form of a matrix.
  • the usage environment information comprises network environment information and user environment information, the network environment information includes a bandwidth, and the user environment information includes the the terminal characteristics or the user preferences for video quality including spatial, temporal, and SNR resolution.
  • the SVC adaptation operators determined by the ADTE unit comprise information on the SVC adaptation operators for the spatial scalability, the temporal scalability, and the SNR scalability among the standardized SVC adaptation operators.
  • the bitstream extracted by the SVC bitstream extraction unit satisfies the
  • the Adaptation QoS information extraction unit extracts information on SVC adaptation operators for the spatial scalability, the temporal scalability, and the SNR scalability among the standardized SVC adaptation operators.
  • the ADTE unit determines optimal SVC adaptation operators for the spatial scalability, the temporal scalability, and the SNR scalability satisfying the usage environment, among the standardized SVC adaptation operators.
  • the ADTE unit determines an SVC adaptation operator for SNR scalability by finding the appropriate value of the SVC adaptation operator for SNR scalability that satisfies an available bandwidth of terminal in the range of the highest quality point and the base quality point for the specific value of the SVC adaptation operator for spatial scalability and the specific value of the SVC adaptation operator for temporal scalability.
  • the SVC bitstream extraction unit extracts the bitstream to satisfy the determined SVC adaptation operators for the spatial scalability, the temporal scalability, and the SNR scalability among the standardized SVC adaptation operators.
  • the SVC bitstream extraction unit When the bitstream is adapted to satisfy the SVC adaptation operator for the spatial scalability among the standardized SVC adaptation operators, the SVC bitstream extraction unit numerically expresses an SVC adaptation operator for the spatial scalability corresponding to the number of the spatial enhancement layers to be truncated, and, according to the value of the SVC adaptation operator for spatial scalability, the SVC bitstream extraction unit does not perform adaptation for spatial scalability or truncates the same number of the highest spatial enhancement layers of the bitstream as the value of the SVC adaptation operator for the spatial scalability, thereby performing adaptation.
  • the SVC bitstream extraction unit When the bitstream is adapted to satisfy the SVC adaptation operator for the temporal scalability among the standardized SVC adaptation operators, the SVC bitstream extraction unit numerically expresses an SVC adaptation operator for the temporal scalability corresponding to the number of the temporal enhancement layers to be truncated, and according to the value of the SVC adaptation operator for temporal scalability, the SVC bitstream extraction unit dose not perform adaptation for temporal scalability or truncates the same number of the highest temporal layers of the bitstream as the value of the SVC adaptation operator for the temporal scalability, thereby performing adaptation.
  • the SVC bitstream extraction unit does not perform adaptation for the SNR scalability or truncates the FGS layers starting from the highest FGS layer.
  • the SVC bitstream extraction unit truncates the CGS quality layers according the ratio of the sum of the bitrates of the highest CGS layers to be truncated to the sum of the bitrates of the entire CGS layers of the bitstream, thereby performing adaptation.
  • CGS coarse grain scalability
  • the SVC bitstream extraction unit truncates the CGS quality layers according the ratio of the sum of the bitrates of the highest CGS layers to be truncated to the sum of the bitrates of the entire CGS layers of the bitstream, thereby performing adaptation.
  • the SVC bitstream extraction unit truncates an appropriate number of the highest CGS layers or highest FGS layers to satisfy the ratio, thereby performing adaptation.
  • the Adaptation QoS information on the bitstream to which SVC technology is applied is recorded in XML format.
  • the apparatus may further include an Adaptation QoS information description unit describing the Adaptation QoS information of the bitstream, to which SVC technology is applied and that is adapted through the SVC bitstream extraction unit, with SVC adaptation operators.
  • an apparatus for adapting a bitstream to which an SVC technology is applied including: a digital item input unit inputting the bitstream to which SVC technology is applied, and Adaptation QoS information including SVC adaptation operators for the bitstream; an usage environment information input unit in which user environment information and network environment information of a terminal to which the bitstream is transmitted is inputted; an adaptation processing unit determining the SVC adaptation operators for the bitstream based on the network environment information and the user environment information, and extracting the bitstream to satisfy the determined SVC adaptation operators; and a digital item output unit transmitting the bitstream extracted by the adaptation processing unit, to the terminal, and generating an Adaptation QoS information including the SVC adaptation operators with respect to the adapted bitstream extracted by the adaptation processing unit.
  • the digital item input unit may include: an Adaptation QoS information input unit in which the Adaptation QoS information described in an XML format, of the bitstream to which SVC technology is applied is inputted; and an SVC video input unit in which the bitstream to which SVC technology is applied is inputted.
  • the usage environment information input unit may include: a network environment information input unit obtaining network environment information including a bandwidth; and an user environment information input unit obtaining user environment information including the terminal characteristics or the user preferences for video quality including spatial, temporal, and SNR resolution.
  • the adaptation processing unit may include: an Adaptation QoS information extraction unit parsing Adaptation QoS information recorded in XML format and extracting SVC adaptation operators for adaptation of the bitstream to which SVC technology is applied; an ADTE unit determining optimal SVC adaptation operators based on the network environment information and the user environment information among the extracted SVC adaptation operators; and an SVC bitstream extraction unit extracting the bitstream to satisfy the determined SVC adaptation operators.
  • the digital item output unit may include: an adaptation SVC bitstream output unit transmitting the extracted bitstream to which SVC technology is applied, to the user terminal; and an Adaptation QoS information description unit describing the Adaptation QoS information to be used for future adaptation of the bitstream to which SVC technology is applied, in an XML format including SVC adaptation operators.
  • it is provided method of adapting a bitstream to which a SVC technology is applied including: extracting SVC adaptation operators, and relationships between the SVC adaptation operators and the usage environment information of a terminal from the Adaptation QoS information of the bitstream to which SVC technology is applied; determining the SVC adaptation operators corresponding to the usage environment of the terminal receiving the transmitted bitstream among the SVC adapatation operators; and extracting the bitstream based on the determined SVC adaptation operators.
  • a method of adapting a bitstream to which an SVC technology is applied including: receiving an input of the bitstream to which SVC technology is applied, and Adaptation QoS information including SVC adaptation operators for the bitstream; receiving inputs of user environment information and network environment information of a terminal to which the bitstream is transmitted is inputted; determining the SVC adaptation operators for the bitstream based on the network environment information and the user environment information, and extracting the bitstream to satisfy the determined SVC adaptation operators; and transmitting the extracted bitstream to the terminal, and generating an Adaptation QoS information including the SVC adaptation operators with respect to the adapted bitstream.
  • FIG. 1 shows the structure of an apparatus for adapting a bitstream according to an embodiment of the present invention
  • FIG. 2 illustrates scalable video coding (SVC) adaptation operators according to an embodiment of the present invention
  • FIG. 3 shows the structure of a network for explaining compound adaptation (re-adaptation) according to an embodiment of the present invention
  • FIG. 4 illustrates a method of describing the Adaptation QoS information by using highest quality points and base quality points for adaptation of a SVC bitstream according to an embodiment of the present invention
  • FIG. 5 illustrates SVC adaptation operators for adapting a SVC bitstream in the form of AQoSCIassification sheme according to an embodiment of the present invention
  • FIG. 6 illustrates SVC adaptation operators for adapting a SVC bitstream in the form of a Utilityfunction type according to an embodiment of the present invention
  • FIG. 7 illustrates SVC adaptation operators for adapting an SVC bitstream in the form of a LookupTable type according to an embodiment of the present invention
  • FIG. 8 is a flowchart illustrating a method of adapting a bitstream according to an embodiment of the present invention.
  • FIG. 9 is a flowchart illustrating an operation for inputting a digital item in a method of adapting a bitstream according to an embodiment of the present invention.
  • FIG. 10 is a flowchart illustrating an operation for inputting usage environment information in a method of adapting a bitstream according to an embodiment of the present invention
  • FIG. 11 is a flowchart illustrating an operation for processing adaptation in a method of adapting a bitstream according to an embodiment of the present invention.
  • FIG. 12 is a flowchart illustrating an operation for o ⁇ tputting a digital item in a method of adapting a bitstream.
  • FIG. 1 shows the structure of an apparatus for adapting a bitstream according to an embodiment of the present invention.
  • the apparatus for adapting a bitstream is composed of a digital item input unit 100, an usage environment information input unit 110, an adaptation processing unit 120, and a digital item output unit 130.
  • the digital item input unit 100 includes an Adaptation QoS information input unit 101 and a scalable video coding (SVC) video input unit 102.
  • Information in which a Adaptation QoS information of an SVC video stream is described in an extensible markup language (XML) format is input to the Adaptation QoS information input unit 101 , and a video bitstream to which SVC technology is applied is input to the SVC video input unit 102.
  • the digital item input unit 100 includes all functions for receiving individual digital items.
  • the Adaptation QoS information is extracted for adaptation of SVC video obtained in the Adaptation QoS information extracting unit 121.
  • the usage environment information input unit 110 includes a function for obtaining information on the usage environment of an individual digital item input through the digital item input unit 100.
  • the usage environment information input unit 110 includes a network environment information input unit 111 and a user environment information input unit 112.
  • the network environment information input unit 111 includes a function for obtaining network environment information for transmission of an SVC video stream.
  • the user environment information input unit 112 includes a function for obtaining the environment information of a user (the terminal characteristics or the user preferences for video quality including spatial, temporal, and SNR resolution) for using an SVC video stream.
  • the digital item input unit 100 obtains media resources (including Adaptation QoS information) to be adapted, and the usage environment information input unit obtains environment information items for transmission and usage in a terminal.
  • the adaptation processing unit 120 performs an SVC video adaptation process.
  • the network information obtained by the network environment information input unit 11 1 , the user environment information obtained by the user environment information input unit 112, and the Adaptation QoS information of an SVC bitstream extracted by the Adaptation QoS information extraction unit 121 are input to an
  • the ATDE unit 123 determines information suitable for the obtained information (network and user environment) among the Adaptation QoS information extracted in the Adaptation QoS information extraction unit 121.
  • the information determined by the ATDE unit 123 is the form of the SVC adaptation operators, and is input to an SVC bitstream extraction unit 122.
  • SVC bitstream extraction unit 122 performs the actual process of extracting an SVC bitstream, and the SVC bitstream is extracted according to the SVC adaptation operators determined by the ATDE unit 123.
  • the SVC bitstream adapted (extracted) according to the SVC adaptation operators in the adaptation processing unit 120 is transmitted to an SVC bitstream output unit 132.
  • the Adaptation QoS information of the adapted SVC bitstream is redescribed by an Adaptation QoS information description unit 131 describing Adaptation QoS information for re-adaptation.
  • the SVC bitstream is transmitted to a terminal through the digital item output unit 130.
  • FIG. 2 illustrates scalable video coding SVC adaptation operators according to an embodiment of the present invention.
  • SVC adaptation operators 200 supporting SVC adaptation include an SVC adaptation operator for spatial scalability - Spatial Layers 210, an SVC adaptation operator for temporal
  • Scalability - Temporal Levels 220 Scalability - Temporal Levels 220, and an SVC adaptation operator for signal to noise ratio (SNR) Scalability - Quality Reduction 230.
  • SVC defines video quality with three elements: spatial resolution, temporal resolution(frame rate), and SNR quality, and performs adaptation based on these.
  • the SVC adaptation operators 200 indicates an adaptation quality corresponding to the three elements.
  • an SVC bitstream is formed of a base layer and enhancement layers.
  • the enhancement layer is a bitstream used for improving the spatial resolution, temporal resolution (the frame rate), and the SNR quality of a bitstream in the base layer.
  • the SVC adaptation operator for the spatial scalability - Spatial Layers 210 is used to increase or decrease the spatial resolution whose resolution is low or high, respectively.
  • the SVC adaptation operator for the temporal scalability- Temporal Levels 220 makes 30 frame/sec images into 60 frame/sec images by adding enhancement layers, as a method of increasing or decreasing temporal resolution.
  • the SVC adaptation operator for the SNR scalability - Quality Reduction 230 is used to increase or decrease the SNR quality of a decoded image by adding or removing an enhancement layers (or partially truncating an enhancement layer), as a method of increasing or decreasing an SNR (picture quality).
  • FIG. 3 shows the structure of a network for explaining compound adaptation (re-adaptation) according to an embodiment of the present invention.
  • the network is composed of an SVC streaming server 300, a first SVC adaptation server 310, and a second SVC adaptation server 320.
  • FIG. 4 illustrates a method of describing the Adaptation QoS information by using highest quality points and base quality points for adaptation of a SVC bitstream according to an embodiment of the present invention.
  • each spatio-temporal quality interval illustrates description representing entire Adaptation QoS information by using the SNR quality highest point (0) of each spatio-temporal quality interval and SNR quality base points (P1 , P2, P3, P4, P5) of each spatio-temporal quality interval.
  • the highest quality point expresses an original video quality for which adaptation is not performed, and each base quality point expresses the base point of an SNR quality in each quality interval having identical spatio-temporal quality.
  • This Adaptation QoS information description method indicates quality by minimum number of representative values, in relation to Adaptation QoS information with respect to a decrease in available network bandwidth, thereby enabling efficient calculation of Adaptation QoS information. Determination of Adaptation QoS information in an arbitrary interval using representative values can be explained through the following example. Spatio-temporal quality information of an interval between the first base quality point (P1) and the second base quality point (P2) is same as the spatio- temporal quality information at the second base quality point (P2), and the SNR quality information (QualityReduction) is determined by reducing the SNR quality information of the second base quality point (P2) by the same amount as increased to a current available bandwidth. Determination of quality will be described later in more detail referring to equation 6.
  • FIG. 5 illustrates SVC adaptation operators for adapting an SVC bitstream in the form of AQoSCIassification sheme according to an embodiment of the present invention.
  • SVC adaptation operators In order to use SVC adaptation operators efficiently and generally, it is required to define the SVC daptation operators in AQoSCIassification.
  • Spatial Layers indicate the number of spatial enhancement layers for spatial resolution to be truncated from the full bitstream, and for the adaptation, the highest spatial enhancement layer in the bitstream is truncated first.
  • a bitstream coded at layer 2 has integer values 0 or 1 as the value of Spatial Layers. If the value is 0, spatial quality adaptation is not performed, and if the value is 1 , only the base layer is extracted and an enhancement layer (the highest layer between the base layer and the enhancement layer) is truncated.
  • Temporal Levels indicate the number of temporal enhancement layers for temporal resolution to be truncated, from the full bitstream and for the adaptation, the highest temporal enhancement layer in the bitstream is truncated first.
  • a bitstream coded at 30 frames/sec has integer values 0, 1 , 2, 3 or 4 as the value of Temporal Levels. If the value is 0, adaptation of temporal quality is not performed (maintaining 30 frames/sec), and if the value is 1 , the highest temporal enhancement layer is truncated, thereby adapting the temporal quality from 30 frames/sec to 15 frames/sec.
  • the highest temporal enhancement layer and the second highest temporal enhancement layer are truncated, thereby adapting the temporal quality to 7.5 frames/sec; if the value is 3, the highest temporal enhancement layer, the second highest temporal enhancement layer and the third highest temporal enhancement layer are truncated, thereby adapting the temporal quality to 3.75 frames/sec; and if the value is 4, the highest temporal enhancement layer, the second highest temporal enhancement layer, the third highest temporal enhancement layer and the fourth highest temporal enhancement layer are truncated, thereby adapting the temporal quality to 1.875 frames/sec.
  • Quality Reduction indicates the SNR enhancement fraction to be truncated for adaptation of SNR quality (SNR resolution). For example, if fine grain scalability (FGS) is used, the coded bitstream has a floating-point decimal number in 0-1 range as a value of Quality Reduction. If the value is 0.00, adaptation of the SNR quality is not performed. If the value is 1.00, all FGS enhancement layers are truncated and only the base layer is extracted, thereby performing SNR quality adaptation. If the value is 0.50, the highest FGS enhancement layers corresponding to 50% of all FGS enhancement layers are truncated, thereby performing SNR quality adaptation.
  • FGS fine grain scalability
  • the coded bitstream has a floatingpoint decimal number in 0-1 range as a value of Quality Fraction. If the value is 0.00, adaptation of the SNR quality adaptation is not performed. If the value is 1.00, all CGS enhancement layers are truncated and SNR quality adaptation is performed. For example, if two CGS layers exist, when the first CGS layer includes 60% of all the SNR quality layers, and the second CGS layer includes 40% of all the SNR quality layers, three Quality Reduction 1.00, 0.40, and 0.00 can be described in the Adaptation QoS information.
  • CGS coarse grain scalability
  • the FGS and CGS are used at the same time, for example, if 2 CGS layers exist and the FGS is applied, adaptation of more precise SNR quality is enabled compared to the case where only the CGS is used.
  • the first CGS layer includes 40% of all the SNR quality
  • the FGS layer of the first CGS layer includes 20% of all the SNR quality
  • the second CGS layer includes 30% of all the SNR quality
  • the FGS layer of the second CGS layer includes 10% of all the SNR quality
  • a more precise SNR quality control such as 0.45, is enabled while when only the CGS is used, three types of Quality Reduction, 1.00, 0.40, and 0.00, can be provided.
  • Adaptation QoS information of an SVC video stream is described using UtilityFunction type as illustrated in FIG. 6, it can be described by using SVC adaptation operators (Spatial Layers, Temporal Levels, Quality Reduction).
  • Adaptation QoS information of an SVC video stream is described using LookupTable type as illustrated in FIG. 7, it can be described by using SVC adaptation operators (Spatial Layers, Temporal Levels, Quality Reduction).
  • Spatial Layers that is an SVC adaptation operator for spatial scalability (Qfs) are expressed as equation 1 above. If the value is 0, adaptation of the spatial quality is not performed, and if the value is 1 , the highest spatial enhancement layer is truncated. If the value is 2, the highest spatial enhancement layer and the second highest spatial enhancement layer are truncated.
  • TQs NR ⁇ Bf GS + ⁇ n (3)
  • TQ SNR is the SNR bitrate of the video quality to be truncated for adaptation of the SNR quality satisfying the constraints
  • OQ sm is the SNR bitrate of the input original video
  • Bf GS is the bitrate of i-th highest FGS layer
  • n* is the number of FGS layers to be truncated
  • ⁇ n is an FGS fraction to be truncated
  • n is the number of FGS layers of the original video.
  • Quality Reduction that is an SVC adaptation operator for SNR scalability (QF S NR) is expressed as equation 3 above. If the value is 0.00, adaptation of the SNR quality is not performed, and if the value is 1.00, the highest SNR enhancement layer is truncated.
  • OQ SNR is the bitrate of the SNR quality of the input original video
  • TQ sm is the SNR bitrate of the SNR quality to be truncated
  • BTM s is the bitrate of a k-th highest CGS layer
  • m* is the number of highest CGS layers to be truncated.
  • SNR quality can be provided in units suitable for the bitrate included in each CGS layer.
  • the first CGS layer(the second highest CGS layer in this case) includes 70% of all the SNR quality layers
  • the second CGS layer (the first highest CGS layer in this case) includes 30% of all the SNR quality layers
  • three SNR adaptation qualities, 1.00, 0.30, and 0.00 can be described in the Adaptation QoS information (AQoS). If the value is 1.00, all CGS quality layers are truncated, if the value is 0.30, the second CGS layer (the first highest CGS layer), corresponding to 30% of all the SNR quality layers, is truncated, and if the value is 0.00, all CGS layers are extracted, thereby performing adaptation of the SNR quality.
  • TQ sm is the SNR bitrate to be truncated
  • OQ sm is the SNR bitrate of the input original video
  • Bf GS is the bitrate of the i-th highest CGS layer
  • Bf GS is the bitrate of the j-th highest FGS layer of i-th highest CGS layer
  • /? mV is the bitrate of an FGS fraction of the n * -th highest FGS layer of the m*-th highest CGS layer to be truncated
  • ni is the number of FGS layers of the i-th highest CGS layer
  • m is the number of the CGS layers of the original video
  • m* is the number of highest CGS layers to be truncated.
  • the FGS and CGS are used at the same time, for example, if 2 CGS layers exist and the FGS is applied, adaptation of more precise SNR quality is enabled compared to the case when only the CGS is used.
  • the first CGS layer and its FGS layer respectively include 40% and 20% of all the SNR quality
  • the second CGS layer and its FGS layer respectively include 30% and 10% of all the SNR quality
  • all the second CGS layer and the FGS layer of the second CGS layer are truncated, and 5% of the FGS layer of the first CGS layer is fraction-truncated, thereby performing more precise adaptation of the SNR quality than when only the CGS is used.
  • Qfs NR , ⁇ f/ , and Qf ⁇ are values of Quality Reduction, Spatial Layers, and Temporal Levels, respectively, at an arbitrary point x existing in a quality interval ⁇ O,P ⁇
  • Qff m , Qf ⁇ , and Qf ⁇ p are values of Quality Reduction, Spatial Layers, and Temporal Levels, respectively, at a base quality point (P) in the quality interval ⁇ O,P ⁇ .
  • B x and B P are available transmission bitrates at the arbitrary point x and the base quality point (P), respectively
  • OQ SNR is the SNR bitrate of the input original video. For example, when the bitrate of the SNR quality of the original input video is
  • a currently available transmission bitrate is 500kbps
  • an transmission bitrate at the base quality point (P) is 400kbps
  • Quality Reduction is 0.7
  • Spatial Layers are 1
  • Temporal Levels are 1
  • Spatial Layers are determined to be 1
  • Temporal Levels are determined to be 1.
  • FIG. 8 is a flowchart illustrating a method of adapting a bitstream according to an embodiment of the present invention.
  • the method includes an operation S800 for digital item inputting in which a bitstream to which SVC technology is applied, and Adaptation QoS information including SVC adaptation operators for the bitstream are input, and an operation S810 for user environment information and network environment information of a terminal to which the bitstream is transmitted is inputted. Then, in operation S820 for adaptation processing, the SVC adaptation operators for the bitstream based on the network environment information and the user environment information is determined, and the bitstream to satisfy the determined SVC adaptation operators is extracted. In operation S830 for digital item outputting, the extracted bitstream is transmitted to the terminal, and an Adaptation QoS information including the SVC adaptation operators with respect to the adapted bitstream is generated.
  • FIG. 9 is a flowchart illustrating an operation for inputting a digital item in a method of adapting a bitstream according to an embodiment of the present invention.
  • the Adaptation QoS information described in an XML format, of the bitstream to which SVC technology is applied is input in operation S901 , and the bitstream to which SVC technology is applied is input in operation S902.
  • FIG. 10 is a flowchart illustrating an operation for inputting usage environment information in a method of adapting a bitstream according to an embodiment of the present invention.
  • the network environment information including a bandwidth is obtained in operation S1001
  • the user environment information including the terminal characteristics or the user preferences for video quality including spatial, temporal, and SNR resolution is obtained in operation S1002. Then, the network and user environment information is used as basic information for determining SVC adaptation operators.
  • FIG. 11 is a flowchart illustrating an operation for processing adaptation in a method of adapting a bitstream according to an embodiment of the present invention.
  • Optimal SVC adaptation operators based on the network environment information and the user environment information among the extracted SVC adaptation operators is determined in operation S1102.
  • bitstream to satisfy the determined SVC adaptation operators is extracted in operation S1103.
  • FIG. 12 is a flowchart illustrating an operation for outputting a digital item in a method of adapting a bitstream.
  • Adaptation QoS information to be used for future adaptation of the bitstream to which SVC technology is applied, in an XML format including SVC adaptation operators is described in operation S1202.
  • the Adaptation QoS information for adapting an SVC video stream can be described generally, and by using the described Adaptation QoS information, SVC adaptation can be performed. Since SVC adaptation operators capable of supporting the SVC adaptation have not been supported so far, Adaptation QoS information (AQoS description) for adaptation of an SVC video stream can be described generally based on the present invention. Based on the description, the method and system of the present invention capable of supporting adaptation can effectively support SVC adaptation.
  • the present invention can also be embodied as computer readable code on a computer readable recording medium.
  • the computer readable recording medium is any data storage device that can store data which can be thereafter read by a computer system. Examples of the computer readable recording medium include read-only memory (ROM), random-access memory (RAM), CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, and carrier waves (such as data transmission through the Internet).
  • ROM read-only memory
  • RAM random-access memory
  • CD-ROMs compact discs
  • magnetic tapes magnetic tapes
  • floppy disks optical data storage devices
  • carrier waves such as data transmission through the Internet

Abstract

An apparatus for and method of adapting a bitstream to which scalable video coding (SVC) technology is applied are provided. The apparatus for adapting a bitstream includes: an Adaptation QoS information extraction unit extracting SVC adaptation operators, and relationships between the SVC adaptation operators and the usage environment information of a terminal from the Adaptation QoS information on the bitstream to which SVC technology is applied; an Adaptation Decision Taking Engine(ADTE) unit determining the SVC adaptation operators corresponding to the usage environment of the terminal receiving the transmitted bitstream among the SVC adaptation operators; and a SVC bitstream extraction unit extracting the bitstream based on the determined SVC adaptation operator. According to the apparatus and method, scalable video can be efficiently provided for changing network environments and multimedia usage environments, through adaptation of scalable video streams using an adaptation operator suggested in Classification Scheme (AQoSJDS).

Description

METHOD AND APPARATUS FOR SCALABLE VIDEO ADAPTATION USING ADAPTATION OPERATORS FOR SCALABLE VIDEO
TECHNICAL FIELD
The present invention relates to an apparatus and method of adapting a bitstream to which scalable video coding (SVC) technology is applied, and more particularly, to an apparatus and method in which a bitstream is adapted using SVC adaptation operators, and the SVC adaptation operators for the adapted bitstream is additionally described, thereby allowing the SVC adaptation operators to be used later for new adaptation.
BACKGROUND ART With the development of communication technology, network environments have become increasingly complicated, and a variety of multimedia content has come to be consumed through different networks and terminals. Users can now enjoy high definition (HD) video products at home, while moving, or in a car, through digital multimedia broadcasting (DMB) or mobile communication networks. Mobile communication networks support a variety of terminals, including personal digital assistants (PDAs), mobile phones, and notebook computers, , and wired networks, such as ADSL, support personal computers (PCs). In the near future, it will be supported by a network integrating more varieties of terminal types such as Internet protocol TV (IPTV). The moving picture experts group (MPEG)-21 framework to provide more varieties of multimedia content efficiently supports many functions, such as digital rights management (DRM), digital item adaptation (DIA), and digital item declaration (DID).
In order to provide a variety of terminals with video streaming service in this different network environment, a consideration of quality suitable for the usage environment is essential, and content of a quality suitable for the network bandwidth, the type of terminal, and user preference must be provided. In order to more efficiently adapt multimedia content to a variety of usage environments, standardization of a scalable video coding technology is currently proceeding, and in order to adapt video content to a usage environment, direct adaptation in a bitstream is supported without the need to perform reproduction in order to adapt video content to usage environments. In this way, video content can be more efficiently and quickly adapted to network and user environments compared with the pre-method of reproducing video content to fit the usage environment.
In order to support adaptation of scalable video in the MPEG-21 framework,
SVC adaptation operators of scalable video needs to be described, but so far no SVC adaptation operators for scalable video exist. Accordingly, it is difficult to efficiently describe adaptation at a bitstream level for scalable video in the MPEG-21 framework.
DETAILED DESCRIPTION OF THE INVENTION TECHNICAL PROBLEM
The present invention provides an apparatus and method of supporting adaptation of multimedia content to which scalable video coding (SVC) technology is applied.
The present invention also provides an apparatus and method in which SVC adaptation operators for appropriately performing adaptation of scalable video at a bitstream level are defined, and effective meanings and description examples for describing the descriptors are suggested, thereby performing effective adaptation suitable for a variety of networks and user environments by using described the Adaptation QoS information.
TECHNICAL SOLUTION
According to an aspect of the present invention, it is provided an apparatus for adapting a bitstream to which scalable video coding (SVC) technology is applied, including: an Adaptation QoS information extraction unit extracting SVC adaptation operators, and relationships between the SVC adaptation operators and the usage environment information of a terminal from the Adaptation QoS information on the bitstream to which SVC technology is applied; an Adaptation Decision Taking Engine(ADTE) unit determining the SVC adaptation operators corresponding to the usage environment of the terminal receiving the transmitted bitstream among the SVC adaptation operators; and a SVC bitstream extraction unit extracting the bitstream based on the determined SVC adaptation operators.
The Adaptation QoS information comprises information on SVC adaptation operators for spatial scalability, temporal scalability and SNR scalability among the standardized SVC adaptation operators. The Adaptation QoS information describes relationships among usage environment information of terminal, SVC adaptation operators for spatial scalability, temporal scalability and SNR scalability, and measurements indicating the overall quality of the bitstream such as a peak SNR(PSNR) and utility rank.
The Adaptation QoS information includes descriptions paired with the bandwidth of the terminal, SVC adaptation operators for the spatial scalability, the temporal scalability and the SNR scalability, and the PSNR vector having identical degrees in the bandwidth of the terminal, SVC adaptation operators for the spatial scalability, the temporal scalability and the SNR scalability, and the PSNR vectors formed with an arbitrary degree. The Adaptation QoS information includes descriptions paired with the bandwidth of the terminal, SVC adaptation operators for the spatial scalability and the temporal scalability having identical degrees in the bandwidth of the terminal and SVC adaptation operators for the spatial scalability and the temporal scalability formed with an arbitrary degree and expressed SVC adaptation operators for the SNR scalability in the form of a matrix.
The usage environment information comprises network environment information and user environment information, the network environment information includes a bandwidth, and the user environment information includes the the terminal characteristics or the user preferences for video quality including spatial, temporal, and SNR resolution.
The SVC adaptation operators determined by the ADTE unit comprise information on the SVC adaptation operators for the spatial scalability, the temporal scalability, and the SNR scalability among the standardized SVC adaptation operators. The bitstream extracted by the SVC bitstream extraction unit satisfies the
SVC adaptation operators for the spatial scalability, the temporal scalability, and the SNR scalability among the standardized SVC adaptation operators.
The Adaptation QoS information extraction unit extracts information on SVC adaptation operators for the spatial scalability, the temporal scalability, and the SNR scalability among the standardized SVC adaptation operators.
The ADTE unit determines optimal SVC adaptation operators for the spatial scalability, the temporal scalability, and the SNR scalability satisfying the usage environment, among the standardized SVC adaptation operators.
The ADTE unit determines an SVC adaptation operator for SNR scalability by finding the appropriate value of the SVC adaptation operator for SNR scalability that satisfies an available bandwidth of terminal in the range of the highest quality point and the base quality point for the specific value of the SVC adaptation operator for spatial scalability and the specific value of the SVC adaptation operator for temporal scalability. The SVC bitstream extraction unit extracts the bitstream to satisfy the determined SVC adaptation operators for the spatial scalability, the temporal scalability, and the SNR scalability among the standardized SVC adaptation operators.
When the bitstream is adapted to satisfy the SVC adaptation operator for the spatial scalability among the standardized SVC adaptation operators, the SVC bitstream extraction unit numerically expresses an SVC adaptation operator for the spatial scalability corresponding to the number of the spatial enhancement layers to be truncated, and, according to the value of the SVC adaptation operator for spatial scalability, the SVC bitstream extraction unit does not perform adaptation for spatial scalability or truncates the same number of the highest spatial enhancement layers of the bitstream as the value of the SVC adaptation operator for the spatial scalability, thereby performing adaptation.
When the bitstream is adapted to satisfy the SVC adaptation operator for the temporal scalability among the standardized SVC adaptation operators, the SVC bitstream extraction unit numerically expresses an SVC adaptation operator for the temporal scalability corresponding to the number of the temporal enhancement layers to be truncated, and according to the value of the SVC adaptation operator for temporal scalability, the SVC bitstream extraction unit dose not perform adaptation for temporal scalability or truncates the same number of the highest temporal layers of the bitstream as the value of the SVC adaptation operator for the temporal scalability, thereby performing adaptation.
When the bitstream is adapted to satisfy the SVC adaptation operator for a fine grain scalability (FGS) of an SNR scalability among the standardized SVC adaptation operators, according to the SVC adaptation operator for the FGS of SNR scalability that is the ratio of the sum of bitrates of the FGS layers and part of an FGS layer to be truncated to the sum of bitrates of the entire FGS layers of the bitstream, the SVC bitstream extraction unit does not perform adaptation for the SNR scalability or truncates the FGS layers starting from the highest FGS layer.
When the bitstream is adapted to satisfy the SVC adaptation operator for a coarse grain scalability (CGS) of an SNR scalability among the standardized SVC adaptation operators, the SVC bitstream extraction unit truncates the CGS quality layers according the ratio of the sum of the bitrates of the highest CGS layers to be truncated to the sum of the bitrates of the entire CGS layers of the bitstream, thereby performing adaptation. When the bitstream is adapted to satisfy the SVC adaptation operator for the
FGS and CGS of an SNR scalability among the standardized SVC adaptation operators, according to the SVC adaptation operator for the FGS and CGS of an SNR scalability that is the ratio of the sum of the bitrates of the CGS layers to be truncated, the bitrates of the FGS layers associated to the CGS layers to be truncated, and the bitrates of the FGS layers and the part of FGS layers to be truncated to the sum of the bitrates of the entire CGS layers and the entire FGS layers of the bitstream, the SVC bitstream extraction unit truncates an appropriate number of the highest CGS layers or highest FGS layers to satisfy the ratio, thereby performing adaptation. The Adaptation QoS information on the bitstream to which SVC technology is applied is recorded in XML format.
The apparatus may further include an Adaptation QoS information description unit describing the Adaptation QoS information of the bitstream, to which SVC technology is applied and that is adapted through the SVC bitstream extraction unit, with SVC adaptation operators.
According to another aspect of the present invention, it is provided an apparatus for adapting a bitstream to which an SVC technology is applied, including: a digital item input unit inputting the bitstream to which SVC technology is applied, and Adaptation QoS information including SVC adaptation operators for the bitstream; an usage environment information input unit in which user environment information and network environment information of a terminal to which the bitstream is transmitted is inputted; an adaptation processing unit determining the SVC adaptation operators for the bitstream based on the network environment information and the user environment information, and extracting the bitstream to satisfy the determined SVC adaptation operators; and a digital item output unit transmitting the bitstream extracted by the adaptation processing unit, to the terminal, and generating an Adaptation QoS information including the SVC adaptation operators with respect to the adapted bitstream extracted by the adaptation processing unit.
The digital item input unit may include: an Adaptation QoS information input unit in which the Adaptation QoS information described in an XML format, of the bitstream to which SVC technology is applied is inputted; and an SVC video input unit in which the bitstream to which SVC technology is applied is inputted.
The usage environment information input unit may include: a network environment information input unit obtaining network environment information including a bandwidth; and an user environment information input unit obtaining user environment information including the terminal characteristics or the user preferences for video quality including spatial, temporal, and SNR resolution.
The adaptation processing unit may include: an Adaptation QoS information extraction unit parsing Adaptation QoS information recorded in XML format and extracting SVC adaptation operators for adaptation of the bitstream to which SVC technology is applied; an ADTE unit determining optimal SVC adaptation operators based on the network environment information and the user environment information among the extracted SVC adaptation operators; and an SVC bitstream extraction unit extracting the bitstream to satisfy the determined SVC adaptation operators. The digital item output unit may include: an adaptation SVC bitstream output unit transmitting the extracted bitstream to which SVC technology is applied, to the user terminal; and an Adaptation QoS information description unit describing the Adaptation QoS information to be used for future adaptation of the bitstream to which SVC technology is applied, in an XML format including SVC adaptation operators. According to another aspect of the present invention, it is provided method of adapting a bitstream to which a SVC technology is applied, including: extracting SVC adaptation operators, and relationships between the SVC adaptation operators and the usage environment information of a terminal from the Adaptation QoS information of the bitstream to which SVC technology is applied; determining the SVC adaptation operators corresponding to the usage environment of the terminal receiving the transmitted bitstream among the SVC adapatation operators; and extracting the bitstream based on the determined SVC adaptation operators.
According to another aspect of the present invention, it is provided a method of adapting a bitstream to which an SVC technology is applied, including: receiving an input of the bitstream to which SVC technology is applied, and Adaptation QoS information including SVC adaptation operators for the bitstream; receiving inputs of user environment information and network environment information of a terminal to which the bitstream is transmitted is inputted; determining the SVC adaptation operators for the bitstream based on the network environment information and the user environment information, and extracting the bitstream to satisfy the determined SVC adaptation operators; and transmitting the extracted bitstream to the terminal, and generating an Adaptation QoS information including the SVC adaptation operators with respect to the adapted bitstream.
ADVANTAGEOUS EFFECTS DESCRIPTION OF THE DRAWINGS
FIG. 1 shows the structure of an apparatus for adapting a bitstream according to an embodiment of the present invention;
FIG. 2 illustrates scalable video coding (SVC) adaptation operators according to an embodiment of the present invention;
FIG. 3 shows the structure of a network for explaining compound adaptation (re-adaptation) according to an embodiment of the present invention; FIG. 4 illustrates a method of describing the Adaptation QoS information by using highest quality points and base quality points for adaptation of a SVC bitstream according to an embodiment of the present invention;
FIG. 5 illustrates SVC adaptation operators for adapting a SVC bitstream in the form of AQoSCIassification sheme according to an embodiment of the present invention;
FIG. 6 illustrates SVC adaptation operators for adapting a SVC bitstream in the form of a Utilityfunction type according to an embodiment of the present invention;
FIG. 7 illustrates SVC adaptation operators for adapting an SVC bitstream in the form of a LookupTable type according to an embodiment of the present invention;
FIG. 8 is a flowchart illustrating a method of adapting a bitstream according to an embodiment of the present invention;
FIG. 9 is a flowchart illustrating an operation for inputting a digital item in a method of adapting a bitstream according to an embodiment of the present invention;
FIG. 10 is a flowchart illustrating an operation for inputting usage environment information in a method of adapting a bitstream according to an embodiment of the present invention; FIG. 11 is a flowchart illustrating an operation for processing adaptation in a method of adapting a bitstream according to an embodiment of the present invention; and
FIG. 12 is a flowchart illustrating an operation for oυtputting a digital item in a method of adapting a bitstream.
BEST MODE
MODE OF THE INVENTION
The present invention will now be described more fully with reference to the accompanying drawings, in which exemplary embodiments of the invention are shown.
FIG. 1 shows the structure of an apparatus for adapting a bitstream according to an embodiment of the present invention. Referring to FIG. 1 , the apparatus for adapting a bitstream is composed of a digital item input unit 100, an usage environment information input unit 110, an adaptation processing unit 120, and a digital item output unit 130.
The digital item input unit 100 includes an Adaptation QoS information input unit 101 and a scalable video coding (SVC) video input unit 102. Information in which a Adaptation QoS information of an SVC video stream is described in an extensible markup language (XML) format is input to the Adaptation QoS information input unit 101 , and a video bitstream to which SVC technology is applied is input to the SVC video input unit 102. The digital item input unit 100 includes all functions for receiving individual digital items.
By parsing the XML formatted Adaptation QoS information description through the Adaptation QoS information input unit 101 , the Adaptation QoS information is extracted for adaptation of SVC video obtained in the Adaptation QoS information extracting unit 121.
The usage environment information input unit 110 includes a function for obtaining information on the usage environment of an individual digital item input through the digital item input unit 100. The usage environment information input unit 110 includes a network environment information input unit 111 and a user environment information input unit 112.
The network environment information input unit 111 includes a function for obtaining network environment information for transmission of an SVC video stream. The user environment information input unit 112 includes a function for obtaining the environment information of a user (the terminal characteristics or the user preferences for video quality including spatial, temporal, and SNR resolution) for using an SVC video stream.
In order to perform SVC adaptation, the digital item input unit 100 obtains media resources (including Adaptation QoS information) to be adapted, and the usage environment information input unit obtains environment information items for transmission and usage in a terminal. With the Information obtained in the digital item input unit 100 and the usage environment information input unit 110, the adaptation processing unit 120 performs an SVC video adaptation process.
The network information obtained by the network environment information input unit 11 1 , the user environment information obtained by the user environment information input unit 112, and the Adaptation QoS information of an SVC bitstream extracted by the Adaptation QoS information extraction unit 121 are input to an
Adaptation Decision Taking Engine unit(ATDE) 123.
The ATDE unit 123 determines information suitable for the obtained information (network and user environment) among the Adaptation QoS information extracted in the Adaptation QoS information extraction unit 121.
The information determined by the ATDE unit 123 is the form of the SVC adaptation operators, and is input to an SVC bitstream extraction unit 122. The
SVC bitstream extraction unit 122 performs the actual process of extracting an SVC bitstream, and the SVC bitstream is extracted according to the SVC adaptation operators determined by the ATDE unit 123.
The SVC bitstream adapted (extracted) according to the SVC adaptation operators in the adaptation processing unit 120 is transmitted to an SVC bitstream output unit 132. The Adaptation QoS information of the adapted SVC bitstream is redescribed by an Adaptation QoS information description unit 131 describing Adaptation QoS information for re-adaptation. The SVC bitstream is transmitted to a terminal through the digital item output unit 130.
FIG. 2 illustrates scalable video coding SVC adaptation operators according to an embodiment of the present invention. Referring to FIG.2, SVC adaptation operators 200 supporting SVC adaptation include an SVC adaptation operator for spatial scalability - Spatial Layers 210, an SVC adaptation operator for temporal
Scalability - Temporal Levels 220, and an SVC adaptation operator for signal to noise ratio (SNR) Scalability - Quality Reduction 230. SVC defines video quality with three elements: spatial resolution, temporal resolution(frame rate), and SNR quality, and performs adaptation based on these.
The SVC adaptation operators 200 indicates an adaptation quality corresponding to the three elements.
In order to allow adaptation to a variety of qualities, an SVC bitstream is formed of a base layer and enhancement layers. The enhancement layer is a bitstream used for improving the spatial resolution, temporal resolution (the frame rate), and the SNR quality of a bitstream in the base layer.
The SVC adaptation operator for the spatial scalability - Spatial Layers 210 is used to increase or decrease the spatial resolution whose resolution is low or high, respectively.
The SVC adaptation operator for the temporal scalability- Temporal Levels 220 makes 30 frame/sec images into 60 frame/sec images by adding enhancement layers, as a method of increasing or decreasing temporal resolution.
The SVC adaptation operator for the SNR scalability - Quality Reduction 230 is used to increase or decrease the SNR quality of a decoded image by adding or removing an enhancement layers (or partially truncating an enhancement layer), as a method of increasing or decreasing an SNR (picture quality).
FIG. 3 shows the structure of a network for explaining compound adaptation (re-adaptation) according to an embodiment of the present invention. Referring to FIG. 3, the network is composed of an SVC streaming server 300, a first SVC adaptation server 310, and a second SVC adaptation server 320.
The necessity for describing Adaptation QoS information for re-adaptation of an adapted SVC bitstream in an environment of networks mixing a variety of network characteristics will now be explained. An SVC bitstream provided by the SVC streaming server 300 and an SVC bitstream adapted by the first SVC adaptation server 310 according to the Adaptation QoS information are adapted by the second SVC adaptation server 320 for a mobile client. At this time, the adaptation is performed by using the Adaptation QoS information (AQoS) generated by the first SVC adaptation server 310. FIG. 4 illustrates a method of describing the Adaptation QoS information by using highest quality points and base quality points for adaptation of a SVC bitstream according to an embodiment of the present invention. FIG. 4 illustrates description representing entire Adaptation QoS information by using the SNR quality highest point (0) of each spatio-temporal quality interval and SNR quality base points (P1 , P2, P3, P4, P5) of each spatio-temporal quality interval. The highest quality point expresses an original video quality for which adaptation is not performed, and each base quality point expresses the base point of an SNR quality in each quality interval having identical spatio-temporal quality.
This Adaptation QoS information description method indicates quality by minimum number of representative values, in relation to Adaptation QoS information with respect to a decrease in available network bandwidth, thereby enabling efficient calculation of Adaptation QoS information. Determination of Adaptation QoS information in an arbitrary interval using representative values can be explained through the following example. Spatio-temporal quality information of an interval between the first base quality point (P1) and the second base quality point (P2) is same as the spatio- temporal quality information at the second base quality point (P2), and the SNR quality information (QualityReduction) is determined by reducing the SNR quality information of the second base quality point (P2) by the same amount as increased to a current available bandwidth. Determination of quality will be described later in more detail referring to equation 6.
FIG. 5 illustrates SVC adaptation operators for adapting an SVC bitstream in the form of AQoSCIassification sheme according to an embodiment of the present invention. In order to use SVC adaptation operators efficiently and generally, it is required to define the SVC daptation operators in AQoSCIassification.
In FIG. 5, Spatial Layers indicate the number of spatial enhancement layers for spatial resolution to be truncated from the full bitstream, and for the adaptation, the highest spatial enhancement layer in the bitstream is truncated first. For example, a bitstream coded at layer 2 has integer values 0 or 1 as the value of Spatial Layers. If the value is 0, spatial quality adaptation is not performed, and if the value is 1 , only the base layer is extracted and an enhancement layer (the highest layer between the base layer and the enhancement layer) is truncated.
Temporal Levels indicate the number of temporal enhancement layers for temporal resolution to be truncated, from the full bitstream and for the adaptation, the highest temporal enhancement layer in the bitstream is truncated first. For example, a bitstream coded at 30 frames/sec has integer values 0, 1 , 2, 3 or 4 as the value of Temporal Levels. If the value is 0, adaptation of temporal quality is not performed (maintaining 30 frames/sec), and if the value is 1 , the highest temporal enhancement layer is truncated, thereby adapting the temporal quality from 30 frames/sec to 15 frames/sec. If the value is 2, the highest temporal enhancement layer and the second highest temporal enhancement layer are truncated, thereby adapting the temporal quality to 7.5 frames/sec; if the value is 3, the highest temporal enhancement layer, the second highest temporal enhancement layer and the third highest temporal enhancement layer are truncated, thereby adapting the temporal quality to 3.75 frames/sec; and if the value is 4, the highest temporal enhancement layer, the second highest temporal enhancement layer, the third highest temporal enhancement layer and the fourth highest temporal enhancement layer are truncated, thereby adapting the temporal quality to 1.875 frames/sec.
Quality Reduction indicates the SNR enhancement fraction to be truncated for adaptation of SNR quality (SNR resolution). For example, if fine grain scalability (FGS) is used, the coded bitstream has a floating-point decimal number in 0-1 range as a value of Quality Reduction. If the value is 0.00, adaptation of the SNR quality is not performed. If the value is 1.00, all FGS enhancement layers are truncated and only the base layer is extracted, thereby performing SNR quality adaptation. If the value is 0.50, the highest FGS enhancement layers corresponding to 50% of all FGS enhancement layers are truncated, thereby performing SNR quality adaptation. If coarse grain scalability (CGS) is used, the coded bitstream has a floatingpoint decimal number in 0-1 range as a value of Quality Fraction. If the value is 0.00, adaptation of the SNR quality adaptation is not performed. If the value is 1.00, all CGS enhancement layers are truncated and SNR quality adaptation is performed. For example, if two CGS layers exist, when the first CGS layer includes 60% of all the SNR quality layers, and the second CGS layer includes 40% of all the SNR quality layers, three Quality Reduction 1.00, 0.40, and 0.00 can be described in the Adaptation QoS information. If the Quality Reduction is 1.00, all CGS enhancement layers are truncated, and if the Quality Reduction is 0.40, the second CGS layer that is corresponding to 40% of all the SNR quality layers is truncated. If the Quality Reduction is 0.00, SNR quality adaptation is not performed.
If the FGS and CGS are used at the same time, for example, if 2 CGS layers exist and the FGS is applied, adaptation of more precise SNR quality is enabled compared to the case where only the CGS is used. If the first CGS layer includes 40% of all the SNR quality, the FGS layer of the first CGS layer includes 20% of all the SNR quality, the second CGS layer includes 30% of all the SNR quality, and the FGS layer of the second CGS layer includes 10% of all the SNR quality, a more precise SNR quality control, such as 0.45, is enabled while when only the CGS is used, three types of Quality Reduction, 1.00, 0.40, and 0.00, can be provided. In order to apply a Quality Reduction of 0.45, all the second CGS layers (including the associated FGS layer) are truncated, and 5% of the FGS layer of the first CGS layer is truncated, thereby adapting the SNR quality.
When the Adaptation QoS information of an SVC video stream is described using UtilityFunction type as illustrated in FIG. 6, it can be described by using SVC adaptation operators (Spatial Layers, Temporal Levels, Quality Reduction).
Also, when the Adaptation QoS information of an SVC video stream is described using LookupTable type as illustrated in FIG. 7, it can be described by using SVC adaptation operators (Spatial Layers, Temporal Levels, Quality Reduction).
Qf s e {0 ,1,..., n - 1}, the number of spatial layers = n (1 )
Spatial Layers that is an SVC adaptation operator for spatial scalability (Qfs) are expressed as equation 1 above. If the value is 0, adaptation of the spatial quality is not performed, and if the value is 1 , the highest spatial enhancement layer is truncated. If the value is 2, the highest spatial enhancement layer and the second highest spatial enhancement layer are truncated.
Qf τ e {0,1,..., k - 1}, the number of Decomposit ion Statges = k (2)
Temporal Levels that is an SVC adaptation operator for temporal scalability
(QFT) are expressed as equation 2 above. If the value is 0, adaptation of the temporal quality is not performed, and if the value is 1 , the highest temporal enhancement layer is truncated. If the value is 2, the highest temporal enhancement layer and the second highest temporal quality layer are truncated.
TQ SNR
Qf SNR = , (0.00 ≤ QFsm ≤ 1.00)
OQ SNR
TQsNR = ∑ BfGS + βn (3)
1=1
Figure imgf000016_0001
Here, TQSNR is the SNR bitrate of the video quality to be truncated for adaptation of the SNR quality satisfying the constraints, OQsm is the SNR bitrate of the input original video, BfGS is the bitrate of i-th highest FGS layer, n* is the number of FGS layers to be truncated, βn is an FGS fraction to be truncated, and n is the number of FGS layers of the original video. Quality Reduction that is an SVC adaptation operator for SNR scalability (QFSNR) is expressed as equation 3 above. If the value is 0.00, adaptation of the SNR quality is not performed, and if the value is 1.00, the highest SNR enhancement layer is truncated.
In the case of the FGS, if the value is 0.30, 30% of all the FGS enhancement layers is truncated, and only 70% of all the FGS enhancement layers is extracted.
Figure imgf000017_0001
CGS
TQsMR - ∑B 'k^ (4) 1=1
Figure imgf000017_0002
Here, OQSNR is the bitrate of the SNR quality of the input original video, TQsm is the SNR bitrate of the SNR quality to be truncated, B™s is the bitrate of a k-th highest CGS layer, and m* is the number of highest CGS layers to be truncated. In the case of the CGS, SNR quality can be provided in units suitable for the bitrate included in each CGS layer. For example, if 2 CGS layers exist, and the first CGS layer(the second highest CGS layer in this case) includes 70% of all the SNR quality layers, and the second CGS layer (the first highest CGS layer in this case) includes 30% of all the SNR quality layers, three SNR adaptation qualities, 1.00, 0.30, and 0.00, can be described in the Adaptation QoS information (AQoS). If the value is 1.00, all CGS quality layers are truncated, if the value is 0.30, the second CGS layer (the first highest CGS layer), corresponding to 30% of all the SNR quality layers, is truncated, and if the value is 0.00, all CGS layers are extracted, thereby performing adaptation of the SNR quality.
Qf SHR = 7~S (°-00 ≤ QF Sm ≤ 1 -00)
U*£sNR
IB -I
TQsHR = Z (B?" + ∑ B™ ) + βm.. (5) i=i y=i
OQSNR = ∑ (B?S + ∑ B™ ) i=i y=i
Here, TQsm is the SNR bitrate to be truncated, OQsm is the SNR bitrate of the input original video, BfGS is the bitrate of the i-th highest CGS layer, BfGS is the bitrate of the j-th highest FGS layer of i-th highest CGS layer, /?mV is the bitrate of an FGS fraction of the n*-th highest FGS layer of the m*-th highest CGS layer to be truncated, ni is the number of FGS layers of the i-th highest CGS layer, m is the number of the CGS layers of the original video, and m* is the number of highest CGS layers to be truncated.
If the FGS and CGS are used at the same time, for example, if 2 CGS layers exist and the FGS is applied, adaptation of more precise SNR quality is enabled compared to the case when only the CGS is used. If the first CGS layer and its FGS layer respectively include 40% and 20% of all the SNR quality, and the second CGS layer and its FGS layer respectively include 30% and 10% of all the SNR quality, in order to apply a Quality Reduction of 0.45, all the second CGS layer and the FGS layer of the second CGS layer are truncated, and 5% of the FGS layer of the first CGS layer is fraction-truncated, thereby performing more precise adaptation of the SNR quality than when only the CGS is used.
Qf SMR = Qf SNK - (Bx - B , ) / OQ SNR
Qf s = Qf s (6) Qf T = Qf T
Here, QfsNR , βf/ , and Qf^ are values of Quality Reduction, Spatial Layers, and Temporal Levels, respectively, at an arbitrary point x existing in a quality interval {O,P}, and Qffm , Qfζ , and Qfτ p are values of Quality Reduction, Spatial Layers, and Temporal Levels, respectively, at a base quality point (P) in the quality interval {O,P}. Bx and BP are available transmission bitrates at the arbitrary point x and the base quality point (P), respectively, and OQSNR is the SNR bitrate of the input original video. For example, when the bitrate of the SNR quality of the original input video is
1 Mbps, a currently available transmission bitrate is 500kbps, an transmission bitrate at the base quality point (P) is 400kbps, and it is described that Quality Reduction is 0.7, Spatial Layers are 1 , Temporal Levels are 1 , Quality Reduction at the currently available transmission bitrate is determined to be 0.6 (= 0.7-(500-400)/1000), Spatial Layers are determined to be 1 , and Temporal Levels are determined to be 1.
FIG. 8 is a flowchart illustrating a method of adapting a bitstream according to an embodiment of the present invention.
The method includes an operation S800 for digital item inputting in which a bitstream to which SVC technology is applied, and Adaptation QoS information including SVC adaptation operators for the bitstream are input, and an operation S810 for user environment information and network environment information of a terminal to which the bitstream is transmitted is inputted. Then, in operation S820 for adaptation processing, the SVC adaptation operators for the bitstream based on the network environment information and the user environment information is determined, and the bitstream to satisfy the determined SVC adaptation operators is extracted. In operation S830 for digital item outputting, the extracted bitstream is transmitted to the terminal, and an Adaptation QoS information including the SVC adaptation operators with respect to the adapted bitstream is generated.
FIG. 9 is a flowchart illustrating an operation for inputting a digital item in a method of adapting a bitstream according to an embodiment of the present invention.
In the digital item inputting operation, the Adaptation QoS information described in an XML format, of the bitstream to which SVC technology is applied is input in operation S901 , and the bitstream to which SVC technology is applied is input in operation S902.
In this way, the Adaptation QoS information and the digital item of the bitstream are input. FIG. 10 is a flowchart illustrating an operation for inputting usage environment information in a method of adapting a bitstream according to an embodiment of the present invention.
The network environment information including a bandwidth is obtained in operation S1001 , and the user environment information including the terminal characteristics or the user preferences for video quality including spatial, temporal, and SNR resolution is obtained in operation S1002. Then, the network and user environment information is used as basic information for determining SVC adaptation operators.
FIG. 11 is a flowchart illustrating an operation for processing adaptation in a method of adapting a bitstream according to an embodiment of the present invention.
In the adaptation processing, parsing Adaptation QoS information and extracting SVC adaptation operators for adaptation of the bitstream to which SVC technology is applied in operation S1101.
Optimal SVC adaptation operators based on the network environment information and the user environment information among the extracted SVC adaptation operators is determined in operation S1102.
The bitstream to satisfy the determined SVC adaptation operators is extracted in operation S1103.
FIG. 12 is a flowchart illustrating an operation for outputting a digital item in a method of adapting a bitstream.
The extracted bitstream to which SVC technology is applied, to the user terminal is transmitted in operation S1201.
The Adaptation QoS information to be used for future adaptation of the bitstream to which SVC technology is applied, in an XML format including SVC adaptation operators is described in operation S1202.
INDUSTRIAL APPLICABILITY
According to the present invention as described above, the Adaptation QoS information for adapting an SVC video stream can be described generally, and by using the described Adaptation QoS information, SVC adaptation can be performed. Since SVC adaptation operators capable of supporting the SVC adaptation have not been supported so far, Adaptation QoS information (AQoS description) for adaptation of an SVC video stream can be described generally based on the present invention. Based on the description, the method and system of the present invention capable of supporting adaptation can effectively support SVC adaptation.
The present invention can also be embodied as computer readable code on a computer readable recording medium. The computer readable recording medium is any data storage device that can store data which can be thereafter read by a computer system. Examples of the computer readable recording medium include read-only memory (ROM), random-access memory (RAM), CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, and carrier waves (such as data transmission through the Internet). The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the present invention as defined by the following claims. The preferred embodiments should be considered in a descriptive sense only and not for purposes of limitation. Therefore, the scope of the invention is defined not by the detailed description of the invention but by the appended claims, and all differences within the scope will be construed as being included in the present invention.

Claims

1. An apparatus for adapting a bitstream to which scalable video coding (SVC) technology is applied, comprising: an Adaptation QoS information extraction unit extracting SVC adaptation operators, and relationships between the SVC adaptation operators and the usage environment information of a terminal from the Adaptation QoS information on the bitstream to which SVC technology is applied; an Adaptation Decision Taking Engine(ADTE) unit determining the SVC adaptation operators corresponding to the usage environment of the terminal receiving the transmitted bitstream among the SVC adaptation operators; and a SVC bitstream extraction unit extracting the bitstream based on the determined SVC adaptation operator.
2. The apparatus of claim 1 , wherein the Adaptation QoS information comprises information on SVC adaptation operators for spatial scalability, temporal scalability and SNR scalability among the standardized SVC adaptation operators.
3. The apparatus of claim 1 , wherein the Adaptation QoS information describes relationships among usage environment information of terminal, SVC adaptation operators for spatial scalability, temporal scalability and SNR scalability, and measurements indicating the overall quality of the bitstream such as a peak
SNR(PSNR) and utility rank.
4. The apparatus of claim 1 , wherein the Adaptation QoS information includes descriptions paired with the bandwidth of the terminal, SVC adaptation operators for the spatial scalability, the temporal scalability and the SNR scalability, and the PSNR vector having identical degrees in the bandwidth of the terminal, SVC adaptation operators for the spatial scalability, the temporal scalability and the SNR scalability, and the PSNR vectors formed with an arbitrary degree.
5. The apparatus of claim 1 , wherein the Adaptation QoS information includes descriptions paired with the bandwidth of the terminal, SVC adaptation operators for the spatial scalability and the temporal scalability having identical degrees in the bandwidth of the terminal, and SVC adaptation operators for the spatial scalability and the temporal scalability formed with an arbitrary degree and expressed SVC adaptation operators for the SNR scalability in the form of a matrix.
6. The apparatus of claim 1 , wherein the usage environment information comprises network environment information and user environment information, the network environment information includes a bandwidth, and the user environment information includes the terminal characteristics or the user preferences for video quality including spatial, temporal, and SNR resolution.
7. The apparatus of claim 1 , wherein the SVC adaptation operators determined by the ADTE unit comprises information on the SVC adaptation operators for the spatial scalability, the temporal scalability, and the SNR scalability among the standardized SVC adaptation operators.
8. The apparatus of claim 1 , wherein the bitstream extracted by the SVC bitstream extraction unit satisfies the SVC adaptation operators for the spatial scalability, the temporal scalability, and the SNR scalability among the standardized SVC adaptation operators.
9. The apparatus of claim 1 , wherein the Adaptation QoS information extraction unit extracts information on SVC adaptation operators for the spatial scalability, the temporal scalability, and the SNR scalability among the standardized SVC adaptation operators.
10. The apparatus of claim 1 , wherein the ADTE unit determines optimal
SVC adaptation operators for the spatial scalability, the temporal scalability, and the SNR scalability satisfying the usage environment, among the standardized SVC adaptation operators.
11. The apparatus of claim 1 , wherein the ADTE unit determines an SVC adaptation operator for SNR scalability by finding the appropriate value of the SVC adaptation operator for SNR scalability that satisfies an available bandwidth of terminal in the range of the highest quality point and the base quality point for the specific value of the SVC adaptation operator for spatial scalability and the specific value of the SVC adaptation operator for temporal scalability.
12. The apparatus of claim 1 , wherein the SVC bitstream extraction unit extracts the bitstream to satisfy the determined SVC adaptation operators for the spatial scalability, the temporal scalability, and the SNR scalability among the standardized SVC adaptation operators.
13. The apparatus of claim 1 , wherein when the bitstream is adapted to satisfy the SVC adaptation operator for the spatial scalability among the standardized SVC adaptation operators, the SVC bitstream extraction unit numerically expresses an SVC adaptation operator for the spatial scalability corresponding to the number of the spatial enhancement layers to be truncated, and, according to the value of the SVC adaptation operator for spatial scalability, the SVC bitstream extraction unit does not perform adaptation for spatial scalability or truncates the same number of the highest spatial enhancement layers of the bitstream as the value of the SVC adaptation operator for the spatial scalability, thereby performing adaptation.
14. The apparatus of claim 1 , wherein when the bitstream is adapted to satisfy the SVC adaptation operator for the temporal scalability among the standardized SVC adaptation operators, the SVC bitstream extraction unit numerically expresses an SVC adaptation operator for the temporal scalability corresponding to the number of the temporal enhancement layers to be truncated, and according to the value of the SVC adaptation operator for temporal scalability, the SVC bitstream extraction unit dose not perform adaptation for temporal scalability or truncates the same number of the highest temporal layers of the bitstream as the value of the SVC adaptation operator for the temporal scalability, thereby performing adaptation.
15. The apparatus of claim 1 , wherein when the bitstream is adapted to satisfy the SVC adaptation operator for a fine grain scalability (FGS) of an SNR scalability among the standardized SVC adaptation operators, according to the SVC adaptation operator for the FGS of SNR scalability that is the ratio of the sum of bitrates of the FGS layers and part of an FGS layer to be truncated to the sum of bitrates of the entire FGS layers of the bitstream, the SVC bitstream extraction unit does not perform adaptation for the SNR scalability or truncates the FGS layers starting from the highest FGS layer.
16. The apparatus of claim 1 , wherein when the bitstream is adapted to satisfy the SVC adaptation operator for a coarse grain scalability (CGS) of an SNR scalability among the standardized SVC adaptation operators, the SVC bitstream extraction unit truncates the CGS quality layers according the ratio of the sum of the bit rates of the highest CGS layers to be truncated to the sum of the bitrates of the entire CGS layers of the bitstream, thereby performing adaptation.
17. The apparatus of claim 1 , wherein when the bitstream is adapted to satisfy the SVC adaptation operator for the FGS and CGS of an SNR scalability among the standardized SVC adaptation operators, according to the SVC adaptation operator for the FGS and CGS of an SNR scalability that is the ratio of the sum of the bit rates of the CGS layers to be truncated, the bitrates of the FGS layers associated to the CGS layers to be truncated, and the bitrates of the FGS layers and the part of FGS layers to be truncated to the sum of the bitrates of the entire CGS layers and the entire FGS layers of the bitstream, the SVC bitstream extraction unit truncates an appropriate number of the highest CGS layers or highest FGS layers to satisfy the ratio, thereby performing adaptation.
18. The apparatus of claim 1 , wherein the Adaptation QoS information on the bitstream to which SVC technology is applied is recorded in XML format.
19. The apparatus of claim 1 , further comprising an Adaptation QoS information description unit describing the Adaptation QoS information of the bitstream, to which SVC technology is applied and that is adapted through the SVC bitstream extraction unit, with SVC adaptation operators.
20. An apparatus for adapting a bitstream to which SVC technology is applied, comprising: a digital item input unit inputting the bitstream to which SVC technology is applied, and Adaptation QoS information including SVC adaptation operators for the bitstream; an usage environment information input unit in which user environment information and network environment information of a terminal to which the bitstream is transmitted is inputted; an adaptation processing unit determining the SVC adaptation operators for the bitstream based on the network environment information and the user environment information, and extracting the bitstream to satisfy the determined SVC adaptation operators; and a digital item output unit transmitting the bitstream extracted by the adaptation processing unit, to the terminal, and generating an Adaptation QoS information including the SVC adaptation operators with respect to the adapted bitstream extracted by the adaptation processing unit.
21. The apparatus of claim 20, wherein the digital item input unit comprises: an Adaptation QoS information input unit in which the Adaptation QoS information described in an XML format, of the bitstream to which SVC technology is applied is inputted; and an SVC video input unit in which the bitstream to which SVC technology is applied is inputted.
22. The apparatus of claim 20, wherein the usage environment information input unit comprises: a network environment information input unit obtaining network environment information including a bandwidth; and a user environment information input unit obtaining user environment information including the terminal characteristics or the user preferences for video quality including spatial, temporal, and SNR resolution.
23. The apparatus of claim 20, wherein the adaptation processing unit comprises: an Adaptation QoS information extraction unit parsing Adaptation QoS information recorded in XML format and extracting SVC adaptation operators for adaptation of the bitstream to which SVC technology is applied; an ADTE unit determining optimal SVC adaptation operators based on the network environment information and the user environment information among the extracted SVC adaptation operators; and an SVC bitstream extraction unit extracting the bitstream to satisfy the determined SVC adaptation operators.
24. The apparatus of claim 20, wherein the digital item output unit comprises: an adaptation SVC bitstream output unit transmitting the extracted bitstream to which SVC technology is applied, to the user terminal; and an Adaptation QoS information description unit describing the Adaptation QoS information to be used for future adaptation of the bitstream to which SVC technology is applied, in an XML format including SVC adaptation operators.
25. A method of adapting a bitstream to which SVC technology is applied, the method comprising: extracting SVC adaptation operators, and relationships between the SVC adaptation operators and the usage environment information of a terminal from the Adaptation QoS information of the bitstream to which SVC technology is applied; determining the SVC adaptation operators corresponding to the usage environment of the terminal receiving the transmitted bitstream among the SVC adapatation operators; and extracting the bitstream based on the determined SVC adaptation operators.
26. The method of claim 25, wherein the Adaptation QoS information comprises information on SVC adaptation operators for spatial scalability, temporal scalability and SNR scalability among the standardized SVC adaptation operators.
27. The method of claim 25, wherein the Adaptation QoS information describes relationships among usage environment information of terminal, SVC adaptation operators for spatial scalability, temporal scalability and SNR scalability, and measurements indicating the overall quality of the bitstream such as a peak SNR(PSNR) and utility rank.
28. The method of claim 25, wherein the Adaptation QoS information includes descriptions paired with the bandwidth of the terminal, SVC adaptation operators the spatial scalability, the temporal scalability and the SNR scalability, and the PSNR vector having identical degrees in the bandwidth of the terminal, SVC adaptation operators for the spatial scalability, the temporal scalability and the SNR scalability, and the PSNR vectors formed with an arbitrary degree.
29. The method of claim 25, wherein the Adaptation QoS information includes descriptions paired with the bandwidth of the terminal, SVC adaptation operators for the spatial scalability and the temporal scalability having identical degrees in the bandwidth of the terminal, and SVC adaptation operators for the spatial scalability and the temporal scalability formed with an arbitrary degree and expressed SVC adaptation operators for the SNR scalability in the form of a matrix.
30. The method of claim 25, wherein the usage environment information comprises network environment information and user environment information, the network environment information includes a bandwidth, and the user environment information includes the terminal characteristics or the user preferences for video quality including spatial, temporal, and SNR resolution.
31. The method of claim 25, wherein the determined SVC adaptation operators comprises information on the SVC adaptation operators for the spatial scalability, the temporal scalability, and the SNR scalability among the standardized SVC adaptation operators.
32. The method of claim 25, wherein in the extracting the bitstream, the extracted bitstream satisfies the SVC adaptation operators for the spatial scalability, the temporal scalability, and the SNR scalability among the standardized SVC adaptation operators.
33. The method of claim 25, wherein in the extracting the Adaptation QoS information including the SVC adaptation operators for the spatial scalability, the temporal scalability, and the SNR scalability among the standardized SVC adaptation operators.
34. The method of claim 25, wherein in the determining the Adaptation QoS, optimal SVC adaptation operators for the spatial scalability, the temporal scalability, and the SNR scalability satisfying the usage environment, among the standardized SVC adaptation operators is determined.
35. The method of claim 25, wherein in the determining the Adaptation QoS, an SVC adaptation operator for SNR scalability by finding the appropriate value of the SVC adaptation operator for SNR scalability that satisfies an available bandwidth of terminal in the range of the highest quality point and the base qulity point for the specific value of the SVC adaptation operator for spatial scalability and the specific value of the SVC adaptation operator for temporal scalability is determined.
36. The method of claim 25, wherein in the extracting the bitstream, the bitstream is extracted to satisfy the determined SVC adaptation operators for the spatial scalability, the temporal scalability, and the SNR scalability among the standardized SVC adaptation operators.
37. The method of claim 25, wherein in the extracting the bitstream, when the bitstream is adapted to satisfy the spatial scalability among the standardized
SVC adaptation operators, the extracting the bitstream numerically expresses an SVC adaptation operator for the spatial scalability corresponding to the number of the spatial enhancement layers to be truncated, and, according to the value of the SVC adaptation operator for spatial scalability, the extracting the bitstream does not perform adaptation for spatial scalability or truncates the same number of the highest spatial enhancement layers of the bitstream as the value of the SVC adaptation operator for the spatial scalability, thereby performing adaptation.
38. The method of claim 25, wherein in the extracting the bitstream, when the bitstream is adapted to satisfy the temporal scalability among the standardized
SVC adaptation operators, the extracting the bitstream numerically expresses an SVC adaptation operator for the temporal scalability corresponding to the number of the temporal enhancement layers to be truncated, and according to the value of the SVC adaptation operator for temporal scalability, the extracting the bitstream dose not perform adaptation for temporal scalability or truncates the same number of the highest temporal layers of the bitstream as the value of the SVC adaptation operator for the temporal scalability, thereby performing adaptation.
39. The method of claim 25, wherein in the extracting the bitstream, when the bitstream is adapted to satisfy the SVC adaptation operator for a fine grain scalability (FGS) of an SNR scalability among the standardized SVC adaptation operators, according to the SVC adaptation operator for the FGS of SNR scalability that is the ratio of the sum of bitrates of the FGS layers and part of an FGS layers to be truncated to the sum of bit rates of the entire FGS layers of the bitstream, the extracting the bitstream does not perform adaptation for the SNR scalability or truncates the the FGS layers starting from the highest FGS layer.
40. The method of claim 25, wherein in the extracting the bitstream, when the bitstream is adapted to satisfy the SVC adaptation operator for a coarse grain scalability (CGS) of an SNR scalability among the standardized SVC adaptation operators, the extracting the bitstream truncates the CGS quality layers according the ratio of the sum of the bitrates of the highest CGS layers to be truncated to the sum of the bitrates of the entire CGS layers of the bitstream, thereby performing adaptation.
41. The method of claim 25, wherein in the extracting the bitstream, when the bitstream is adapted to satisfy a SVC adaptation operator for the FGS and CGS of an SNR scalability among the standardized SVC adaptation operators, according to the SVC adaptation operator for the FGS and CGS of an SNR scalability that is the ratio of the sum of the bitrates of the CGS layers to be truncated, the bitrates of the FGS layers associated to the CGS layers to be truncated, and the bitrates of the FGS layers and the part of FGS layers to be truncated to the sum of the bitrates of the entire CGS layers and the entire FGS layers of the bitstream, the extracting the bitstream truncates an appropriate number of the highest CGS layers or highest FGS layers to satisfy the ratio, thereby performing adaptation.
42. The method of claim 25, wherein the Adaptation QoS information on the bitstream to which SVC technology is applied is recorded in XML format.
43. The method of claim 25, further comprising describing the Adaptation
QoS information on the bitstream, to which SVC technology is applied and that is adapted through the extracting the bitstream, with SVC adaptation operators.
44. A method of adapting a bitstream to which SVC technology is applied, the method comprising: receiving an input of the bitstream to which SVC technology is applied, and Adaptation QoS information including SVC adaptation operators for the bitstream; receiving inputs of user environment information and network environment information of a terminal to which the bitstream is transmitted is inputted; determining the SVC adaptation operators for the bitstream based on the network environment information and the user environment information, and extracting the bitstream to satisfy the determined SVC adaptation operators; and transmitting the extracted bitstream to the terminal, and generating an Adaptation QoS information including the SVC adaptation operators with respect to the adapted bitstream.
45. The method of claim 44, wherein the receiving of the input of the bitstream comprises: receiving an input of the Adaptation QoS information described in an XML format, of the bitstream to which SVC technology is applied; and receiving an input of the bitstream to which SVC technology is applied.
46. The method of claim 44, wherein the receiving of the usage environment information comprises: obtaining network environment information including a bandwidth; and obtaining user environment information including the terminal characteristics or the user preferences for video quality including spatial, temporal, and SNR resolution.
47. The method of claim 44, wherein the determining of the SVC adaptation operator, and the extracting and adapting of the bitstream comprises: parsing Adaptation QoS information recoded in XML format and extracting SVC adaptation operators for adaptation of the bitstream to which SVC technology is applied; determining optimal SVC adaptation operators based on the network environment information and the user environment information among the extracted SVC adaptation operators; and extracting the bitstream to satisfy the determined SVC adaptation operators.
48. The method of claim 44, wherein the transmitting of the bitstream and the generating of Adaptation QoS information comprises: transmitting the extracted bitstream to which SVC technology is applied, to the user terminal; and describing the Adaptation QoS information to be used for future adaptation of the bitstream to which SVC technology is applied, in an XML format including SVC adaptation operators.
49. A computer readable recording medium having embodied thereon a computer program for executing the method of any one of claim 25.
PCT/KR2006/003989 2005-10-07 2006-10-02 Method and apparatus for scalable video adaptation using adaptation operators for scalable video WO2007043770A1 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
JP2008534439A JP2009510966A (en) 2005-10-07 2006-10-02 Bitstream adaptive conversion apparatus and method to which scalable video coding technology is applied
EP06799070A EP1932354A4 (en) 2005-10-07 2006-10-02 Method and apparatus for scalable video adaptation using adaptation operators for scalable video
CN2006800370962A CN101283596B (en) 2005-10-07 2006-10-02 Method and apparatus for scalable video adaptation using adaptation operators for scalable video
US12/088,480 US20080247460A1 (en) 2005-10-07 2006-10-02 Method and Apparatus For Scalable Video Adaption Using Adaption Operators For Scalable Video

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
KR10-2005-0094386 2005-10-07
KR20050094386 2005-10-07
KR20050127710 2005-12-22
KR10-2005-0127710 2005-12-22
KR1020060097262A KR100848310B1 (en) 2005-10-07 2006-10-02 Method and apparatus for scalable video adaptation using adaptation operators for scalable video
KR10-2006-0097262 2006-10-02

Publications (1)

Publication Number Publication Date
WO2007043770A1 true WO2007043770A1 (en) 2007-04-19

Family

ID=37942971

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2006/003989 WO2007043770A1 (en) 2005-10-07 2006-10-02 Method and apparatus for scalable video adaptation using adaptation operators for scalable video

Country Status (2)

Country Link
EP (1) EP1932354A4 (en)
WO (1) WO2007043770A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009148270A2 (en) * 2008-06-05 2009-12-10 Electronics And Telecommunications Research Institute Apparatus and method for adapting scalable video coding bitstream

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5768537A (en) * 1996-02-22 1998-06-16 International Business Machines Corporation Scalable MPEG2 compliant video encoder
KR20020087810A (en) * 2001-05-16 2002-11-23 주식회사 넷앤티비 Apparatus and method for applying adaptive selective enhancement in the fine granular scalable coding
US6580759B1 (en) * 2000-11-16 2003-06-17 Koninklijke Philips Electronics N.V. Scalable MPEG-2 video system
US20050166245A1 (en) * 2004-01-28 2005-07-28 Samsung Electronics Co., Ltd. Method and device for transmitting scalable video bitstream

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6650783B2 (en) * 1998-01-14 2003-11-18 Canon Kabushiki Kaisha Image processing apparatus and method for processing images with different scalabilites
CN1636394A (en) * 2000-10-11 2005-07-06 皇家菲利浦电子有限公司 Spatial scalability for fine granular video encoding
JP4150951B2 (en) * 2002-02-19 2008-09-17 ソニー株式会社 Video distribution system, video distribution apparatus and method, and program

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5768537A (en) * 1996-02-22 1998-06-16 International Business Machines Corporation Scalable MPEG2 compliant video encoder
US6580759B1 (en) * 2000-11-16 2003-06-17 Koninklijke Philips Electronics N.V. Scalable MPEG-2 video system
KR20020087810A (en) * 2001-05-16 2002-11-23 주식회사 넷앤티비 Apparatus and method for applying adaptive selective enhancement in the fine granular scalable coding
US20050166245A1 (en) * 2004-01-28 2005-07-28 Samsung Electronics Co., Ltd. Method and device for transmitting scalable video bitstream

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP1932354A4 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009148270A2 (en) * 2008-06-05 2009-12-10 Electronics And Telecommunications Research Institute Apparatus and method for adapting scalable video coding bitstream
WO2009148270A3 (en) * 2008-06-05 2012-06-14 Electronics And Telecommunications Research Institute Apparatus and method for adapting scalable video coding bitstream

Also Published As

Publication number Publication date
EP1932354A1 (en) 2008-06-18
EP1932354A4 (en) 2011-01-05

Similar Documents

Publication Publication Date Title
US20080247460A1 (en) Method and Apparatus For Scalable Video Adaption Using Adaption Operators For Scalable Video
CN108605160B (en) Information processing apparatus, information processing method, and computer program
KR101281845B1 (en) Method and apparatus for visual program guide of scalable video transmission device
US10063812B2 (en) Systems and methods for media format transcoding
KR20050007348A (en) Method and system for optimal video transcoding based on utility function descriptors
US10187648B2 (en) Information processing device and method
KR20020064891A (en) System and method for dynamic adaptive decoding of scalable video to balance CPU load
CN102450014A (en) A framework for quality-aware video optimization
US20170332142A1 (en) Method and system for video stream personalization
CN101669369A (en) The signal transmission of a plurality of decode times in the media file
WO2009002109A2 (en) Method and apparatus for composing scene using laser contents
CN102263942A (en) Scalable video transcoding device and method
CA2843718C (en) Methods and systems for processing content
Vetro et al. Media conversions to support mobile users
Kim et al. An optimal framework of video adaptation and its application to rate adaptation transcoding
EP1932354A1 (en) Method and apparatus for scalable video adaptation using adaptation operators for scalable video
KR20120012089A (en) System and method for proving video using scalable video coding
KR100898769B1 (en) Svc video extraction apparatus for real-time video stream and the method thereof
Arnaiz et al. Efficient personalized scalable video adaptation decision-taking engine based on MPEG-21
WO2023130893A1 (en) Streaming media based transmission method and apparatus, electronic device and computer-readable storage medium
Cha et al. Adaptive scheme for streaming MPEG-4 contents to various devices
Sangeetha et al. A Survey on Performance Comparison of Video Coding Algorithms
Thomas-Kerr et al. Semantic-aware delivery of multimedia
Gollapudi et al. A Novel and Optimal approach for Multimedia Cloud Storage and Delivery to reduce Total Cost of Ownership
Almaoui Metadata driven multimedia transcoding

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 200680037096.2

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application
WWE Wipo information: entry into national phase

Ref document number: 2006799070

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 12088480

Country of ref document: US

ENP Entry into the national phase

Ref document number: 2008534439

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE