WO2013122768A1 - Multi-layer rate control - Google Patents

Multi-layer rate control Download PDF

Info

Publication number
WO2013122768A1
WO2013122768A1 PCT/US2013/024686 US2013024686W WO2013122768A1 WO 2013122768 A1 WO2013122768 A1 WO 2013122768A1 US 2013024686 W US2013024686 W US 2013024686W WO 2013122768 A1 WO2013122768 A1 WO 2013122768A1
Authority
WO
WIPO (PCT)
Prior art keywords
video stream
encoded video
buffer
layer
bit usage
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2013/024686
Other languages
English (en)
French (fr)
Inventor
Mei-Hsuan Lu
Ming-Chieh Lee
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Microsoft Corp
Original Assignee
Microsoft Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Microsoft Corp filed Critical Microsoft Corp
Priority to JP2014556597A priority Critical patent/JP2015510355A/ja
Priority to KR1020147022689A priority patent/KR20140124415A/ko
Priority to EP13749904.2A priority patent/EP2798848A4/en
Priority to CN201380009421.4A priority patent/CN104106265A/zh
Publication of WO2013122768A1 publication Critical patent/WO2013122768A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/187Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a scalable video layer
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/115Selection of the code volume for a coding unit prior to coding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146Data rate or code amount at the encoder output
    • H04N19/15Data rate or code amount at the encoder output by monitoring actual compressed data size at the memory before deciding storage at the transmission buffer
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146Data rate or code amount at the encoder output
    • H04N19/152Data rate or code amount at the encoder output by measuring the fullness of the transmission buffer
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability

Definitions

  • bitrates of encoded video data can pose problems for various video delivery and storage mechanisms. For example, videos can be interrupted if bitrates exceed available bandwidth. Similarly, if bitrates are reduced during streaming or transmission of the video, some details of the video may be lost or removed from the video to accommodate the lower bitrates, which may be noticeable to viewers.
  • some systems employ bitrate control mechanisms to regulate bitrates of the encoded video data and/or to manage the bitrates during transmission.
  • One such approach includes analyzing the bitstream and determining a maintainable bitrate for the entire bitstream. This approach may be practical for some non-scalable bitstreams.
  • a base layer includes the bulk of information in the layered video data
  • the base layer of the video may be most significantly impacted by applying a bitrate control on the entire layered video stream.
  • this approach may fail to maximize quality of the various layers. Such a reduction may negatively impact the user experience more than the variations eliminated or reduced by applying the bitrate control mechanism.
  • scalable video is sometimes used during video conferencing and/or other applications for multiple classes of devices that have varied downlink and/or uplink bandwidth capabilities.
  • using traditional rate control mechanisms may result in reduced quality of the various bitstreams to accommodate a receiver's bandwidth constraints, thus resulting in reduced quality for all of the users.
  • the video server may encode the video as a scalable video stream having a base layer encoded at 300Kbps and an enhancement layer encoded at 200Kbps.
  • the base layer may be encoded at less than the targeted 300kbps, resulting in a reduced quality of the overall stream that fails to maximize the bandwidth available to the second receiver.
  • the maximum stream receivable by the second device may be 480Kbps, which fails to utilize the available bandwidth and provides a less-than-ideal user experience.
  • bit usage feedback information is obtained by a rate controller executing at the video server.
  • the bit usage feedback information includes feedback indicators associated with each of multiple layers of encoded video content of an encoded video stream.
  • the bit usage feedback information is obtained from an encoder encoding raw video data and outputting encoded video, from monitoring the encoded video stream, and/or from real or virtual buffers into which the multiple layers of encoded video content are output by the encoder prior to or during transmission or streaming of the encoded video.
  • the functionality of the buffers is provided by leaky bucket virtual buffers.
  • the rate controller obtains the feedback indicators and associates each of the feedback indicators with a respective layer of the video content.
  • the rate controller also can consider feedback associated with a particular layer when considering any layer above that particular layer. For example, if the rate controller obtains bit usage feedback associated with a base layer, this bit usage feedback can be considered when considering bit usage of enhancement layers as well. As such, dependencies between the various layers of the encoded video can be considered during application of the multi-layer rate control mechanisms described herein.
  • the rate controller also can be configured to generate quantization parameters indicating how bitrates of the layers are to be controlled.
  • the rate controller can determine multiple quantization parameters, each of the quantization parameters being generated for a respective layer of the video stream and taken into account when considering higher layers of the video stream.
  • the quantization parameters can also be determined while considering and accounting for dependencies between the multiple layers of the video stream.
  • the rate controller can be configured to output the quantization parameters to the encoder, and the encoder can adjust bitrates of the various layers based upon the quantization parameters.
  • the video server can provide multi-layer rate control by controlling bitrates of each layer of the video stream while taking into account dependencies of the layers of the video stream, instead of, or in addition to, controlling a bitrate associated with a layer independently or controlling a bitrate associated with the entire videos stream.
  • a video server obtains video data from a local or remote data storage device.
  • the video server executes an encoder configured to encode the video data into a multi-layer video stream.
  • the encoder outputs the video stream to buffers, and the buffers track or output bit usage feedback corresponding to amounts or numbers of bits that are not transmitted to a client device receiving the encoded video stream.
  • the bit usage feedback can correspond to an amount or number of bits that exceeds available network resources.
  • a rate controller executing at the video server can monitor the buffers and/or obtain the bit usage feedback for each layer of the encoded video stream.
  • the rate controller can determine, based upon the bit usage feedback, a quantization parameter associated with each layer of the encoded video stream. In determining the quantization parameters, the rate controller can consider not only bitrates of the entire encoded video stream, but also bitrates and bit usage feedback associated with each layer of the encoded video stream. Furthermore, the rate controller can be configured to consider bit usage feedback associated with a particular layer of the encoded video stream when analyzing each layer above that layer, thereby taking into account dependencies of enhancement layers upon lower layers of the video. Thus, embodiments of the concepts and technologies disclosed herein can be used to maximize bitrates for the base layer and the lowest enhancement layers before bandwidth is used for higher enhancement layers. Additionally, some embodiments of the concepts and technologies disclosed herein can be used to maximize bitrates of a particular layer by moving residual bit budget associated with the particular layer (due to imperfect rate control) to a next higher layer for each layer considered.
  • FIGURE 1 is a system diagram illustrating an illustrative operating environment for the various embodiments disclosed herein.
  • FIGURE 2 is a flow diagram showing aspects of a method for providing multi-layer rate control, according to an illustrative embodiment.
  • FIGURE 3 is a flow diagram showing aspects of a method for determining quantization parameters, according to an illustrative embodiment.
  • FIGURE 4 is a computer architecture diagram illustrating an illustrative computer hardware and software architecture for a computing system capable of implementing aspects of the embodiments presented herein.
  • a video server obtains video data from a data storage device.
  • the video server can host or execute an encoder.
  • the encoder can be configured to encode the video data into a multilayer video stream.
  • the encoder can output the video stream to multiple buffers.
  • each layer of the video stream can be passed into a buffer and monitored during streaming or transmission to determine bit usage.
  • the buffers or other mechanisms can track or output bit usage feedback corresponding to amounts or numbers of bits that are not transmitted with the encoded video stream.
  • the bit usage feedback corresponds to a degree to which the encoded video stream transfer rates exceed available network resources.
  • a rate controller can monitor the buffers and/or obtain the bit usage feedback for each layer of the encoded video stream and determine, based upon the bit usage feedback, a quantization parameter associated with each layer of the encoded video stream. In determining the quantization parameters, the rate controller can consider not only bitrates of the entire encoded video stream, but also bitrates and bit usage feedback associated with each layer of the encoded video stream. Furthermore, the rate controller can be configured to consider bit usage feedback associated with a particular layer of the encoded video stream when analyzing each layer above that layer, thereby taking into account dependencies of enhancement layers upon lower layers of the video. Thus, embodiments of the concepts and technologies disclosed herein can be used to maximize bitrates for the base layer and the lowest enhancement layers before bandwidth is used for higher enhancement layers. Additionally, some embodiments of the concepts and technologies disclosed herein can be used to maximize bitrates of a particular layer by moving residual bit budget associated with a particular layer (due to imperfect rate control) to a next higher layer for each layer considered.
  • program modules include routines, programs, components, data structures, and other types of structures that perform particular tasks or implement particular abstract data types.
  • program modules include routines, programs, components, data structures, and other types of structures that perform particular tasks or implement particular abstract data types.
  • program modules include routines, programs, components, data structures, and other types of structures that perform particular tasks or implement particular abstract data types.
  • the subject matter described herein may be practiced with other computer system configurations, including hand-held devices, multiprocessor systems, microprocessor- based or programmable consumer electronics, minicomputers, mainframe computers, and the like.
  • the operating environment 100 shown in FIGURE 1 includes a video server 102.
  • the video server 102 operates as part of, or in communication with, a communications network ("network") 104, though this is not necessarily the case.
  • the functionality of the video server 102 is provided by a server computer; a personal computer (“PC") such as a desktop, tablet, or laptop computer system; a handheld computer; an embedded computer system; or another computing device.
  • PC personal computer
  • video server 102 functionality of the video server 102 is described herein as being provided by server computer, it should be understood that these embodiments are illustrative, and should not be construed as being limiting in any way.
  • One illustrative computing architecture of the video server 102 is illustrated and described in additional detail below with reference to FIGURE 4.
  • the video server 102 can be configured to execute an operating system 106 and one or more software modules such as, for example, a rate controller 108, an encoder 110, and/or other software modules. While the rate controller 108 and the encoder 110 are illustrated in FIGURE 1 as residing at the video server 102, it should be understood that this is not necessarily the case. In particular, the rate controller 108 and/or the encoder 110 can be embodied as separate devices or modules in communication with the video server 102, if desired. As such, the illustrated embodiment should be understood as being illustrative and should not be construed as being limiting in any way.
  • the operating system 106 is a computer program for controlling the operation of the video server 102.
  • the software modules are executable programs configured to execute on top of the operating system to provide various functions described herein for providing multi-layer rate control. Because additional and/or alternative software, application programs, modules, and/or other components can be executed by the video server 102, the illustrated embodiment should be understood as being illustrative and should not be construed as being limiting in any way.
  • the encoder 110 is configured to receive video data 112 such as video frames, raw video data, or other video information.
  • video data 112 is received or retrieved from a data storage 114 operating in communication with the network 104 and/or the video server 102.
  • the functionality of the data storage 114 can be provided by one or more data storage devices including, but not limited to, databases, network data storage devices, hard drives, memory devices, or other real or virtual data storage devices.
  • the data storage 114 includes a memory device or other data storage associated with the video server 102.
  • FIGURE 1 illustrates the data storage 114 as residing remote from the video server 102, it should be understood that this embodiment is illustrative, and should not be construed as being limiting in any way.
  • the encoder 110 is configured to encode the video data 112 to obtain two or more layers of video information.
  • the layers of video information can be output by the encoder 110 and transmitted or streamed by the video server 102 as an encoded video data stream ("encoded video stream") 116.
  • the encoded video stream 116 can include multiple video layers Lj...L N (hereinafter collectively and/or generically referred to as "layers L").
  • the first layer Lj can correspond to a base layer of the encoded video stream 116 and each of the subsequent layers L can correspond to enhancement layers of the encoded video stream 116.
  • the encoded video stream 116 can be received or accessed by a client device 118 and at least the base layer Lj of the encoded video stream 116 can be viewed.
  • the base layer Lj of the encoded video stream 116 can be viewed at the client device 118 with detail provided by the one or more enhancement layers L 2 (not shown in FIGURE 1) through L N ).
  • the client device 118 can receive and view the encoded video stream 116 with various layers of detail, according to the ability of the client device 118 to establish and/or sustain network bandwidth for receiving the multiple layers L of the encoded video stream 116. It should be understood that this embodiment is illustrative, and should not be construed as being limiting in any way.
  • the rate controller 108 is configured to obtain bit usage feedback data ("bit usage feedback") 120 from the encoder 110 (as shown in FIGURE 1), or by monitoring the encoded video stream 116 output by the encoder 110.
  • bit usage feedback data indicating the bit usage feedback 120 from, other reporting mechanisms associated with or incorporated into, the video server 102 such as the buffers 122 described below.
  • the rate controller 108 also can be configured to access, receive, or determine a downlink bandwidth BWD from each subscribed client such as the client device 118, as well as an uplink bandwidth BWu, both of which can be inputs to the rate controller 108.
  • the uplink bandwidth BWu and the downlink bandwidth BWD can be used to determine a target bitrate of each "sub-stream" of the encoded video stream 116 and therefore can be considered an input to the rate controller 108.
  • the term "sub-stream” can include and/or contain a base layer Lj and several successive enhancement layers L of an encoded video stream 116.
  • the video server 102 determines a target bitrate associated with the encoded video stream 116 based, at least partially, upon the downlink bandwidth B WD- According to some embodiments, the video server 102 imposes a limitation that the maximum target bitrate of any sub-stream of the encoded video stream 116 cannot exceed the uplink bandwidth BWu associated with the video server 102, and a limitation that the target bitrate of any sub-stream of the encoded video stream 116 cannot exceed the downlink bandwidth B WD associated with the client that consumes the sub-stream, for example, the client device 118. It should be understood that these embodiments are illustrative and should not be construed as being limiting in any way.
  • the bit usage feedback 120 can indicate an amount or number of bits that are not transmitted to a recipient of the encoded video stream 116. Thus, the bit usage feedback 120 can be analyzed to ascertain how much of the video data 112 encoded as the encoded video stream 116 is prevented, dropped, or lost during transmission or streaming at any particular time. Thus, the bit usage feedback 120 can be understood by the video server 102 to be an indicator of bandwidth or other aspects of the transmission medium used to stream the encoded video stream 116.
  • the bit usage feedback 120 can include data indicating a number of feedback indicators FBI . . .FBN (hereinafter generically referred to as the “feedback indicators FB" and/or collectively referred to as the "bit usage feedback 120").
  • the multiple feedback indicators FB can correspond, respectively, to the multiple layers L discussed above.
  • the feedback parameter FBj can correspond to a bit usage feedback indicator associated with the base layer Lj of the encoded video stream 116. It should be understood that this embodiment is illustrative, and should not be construed as being limiting in any way.
  • the rate controller 108 can be configured to load the bit usage feedback
  • bit usage feedback 120 into multiple leaky bucket buffers Bj ... BN (hereinafter generically referred to as a "buffer 2?” and/or collectively referred to as the "buffers 122").
  • the rate controller 108 loads the bit usage feedback 120 into buffers Bj...B N , which can correspond, respectively, to the multiple layers L and/or the multiple feedback parameters FB.
  • the rate controller 108 can be configured to obtain the bit usage feedback 120, load feedback parameters FB associated with the multiple layers L into the buffers 122, and determine, for each of the layers L, corresponding quantization parameters QPj...
  • the rate controller 108 can be configured to output the quantization parameters 124 to the encoder 110, and the encoder 110 can use the quantization parameters 124 during encoding of the video data 112.
  • the video server 102 can execute the rate controller 108 and the encoder 110 to control bitrates of each layer L of the encoded video stream 116, while taking dependencies between the layers L into account.
  • bit usage rate information associated with a base layer Lj of the encoded video stream 116 can be identified in the bit usage feedback 120, and added to a corresponding buffer Bj.
  • a video includes three layers L.
  • the bit usage rate information such as the feedback indicator FBj associated with the base layer Lj can be added to the buffers Bj, B 2 , and B 3 .
  • the video server 102 can consider the bitrate feedback indicator FBj associated with the base layer Lj and the first enhancement layer L 2 .
  • bitrates for enhancement layers L can be dependent upon bitrates of the base layer Lj and lower enhancement layers L. It should be understood that this embodiment is illustrative, and should not be construed as being limiting in any way.
  • the video server 102 receives the video data
  • the buffers 122 self-report or are monitored by the rate controller 108 or other modules, devices, or software to determine or obtain the bit usage feedback 120, as explained above.
  • the bit usage feedback 120 can include multiple feedback indicators FB, which can correspond to the multiple buffers B included within the buffers 122.
  • the rate controller 108 can be configured to analyze the bit usage feedback 120 and to generate quantization parameters 124 based upon the bit usage feedback 120.
  • the quantization parameters 124 include respective quantization parameters QP I . . . QP N , which can be determined for each of the multiple layers L of the encoded video stream 116, and can take into account dependencies between the layers L as explained above with regard to the buffers B.
  • embodiments of the concepts and technologies disclosed herein can take bitrate usage information for each layer L of an encoded video stream 116 while taking into account dependencies between the layers L the encoded video stream 116 when applying rate control mechanisms, instead of, or in addition to, controlling a bitrate associated with a layer independently or controlling a bitrate associated with the entire output encoded video stream 116.
  • FIGURE 1 illustrates one video server 102, one network 104, and one client device 118. It should be understood, however, that some implementations of the operating environment 100 include multiple video servers 102, multiple networks 104, and no or multiple client devices 118. Thus, the illustrated embodiments should be understood as being illustrative, and should not be construed as being limiting in any way.
  • the logical operations described herein are implemented (1) as a sequence of computer implemented acts or program modules running on a computing system and/or (2) as interconnected machine logic circuits or circuit modules within the computing system.
  • the implementation is a matter of choice dependent on the performance and other requirements of the computing system.
  • the logical operations described herein are referred to variously as states, operations, structural devices, acts, or modules. These operations, structural devices, acts, and modules may be implemented in software, in firmware, in special purpose digital logic, and any combination thereof.
  • the method 200 begins at operation 202, wherein the video server 102 receives video data such as the video data 112 described above with reference to FIGURE 1.
  • the video data 112 can be stored at the video server 102 or can be stored at a remote data storage device such as the data storage 114.
  • operation 202 can include retrieving the video data 112 from a local or remote data storage device.
  • the method 200 can be repeated a number of times by the video server 102 during streaming of the encoded video stream 116.
  • a first iteration of the method 200 may use default quantization parameters 124 to encode the video data 112, as the operations described herein with respect to FIGURE 3 may not yet have been performed by the video server 102.
  • Subsequent iterations of the method 200 can rely upon the quantization parameters 124 determined in operation 204 and illustrated in more detail below with reference to FIGURE 3.
  • the embodiment of the method 200 illustrated in FIGURE 2 may or may not correspond to a particular iteration of the method 200 during streaming of video content from the video server 102.
  • the illustrated embodiment should not be construed as being limiting in any way.
  • embodiments of the concepts and technologies disclosed herein can control a bitrate associated with each layer L of an encoded video stream 116. While providing the multi-layer rate control described herein, embodiments of the video server 102 can consider not only overall bitrates, but also bitrates of layers L, dependencies between the layers L. Thus, an enhancement layer L of an encoded video stream 116 may not be analyzed until a base layer L of the encoded video stream 116 is considered, thus enforcing dependencies between the enhancement layer L and the base layer L. It should be understood that this embodiment is illustrative, and should not be construed as being limiting in any way.
  • the method begins at operation 302, wherein the video server 102 selects a base layer Lj of the encoded video stream 116.
  • the encoded video stream 116 can include multiple layers L, and the base layer Lj can correspond to a first layer of the encoded video stream 116.
  • the base layer Lj can, but does not necessarily, include a majority of the video data 112 associated with the encoded video stream 116 and/or a disproportionate amount of the video data 112 that is greater than portions of the video data 112 included in other layers L of the encoded video stream 116.
  • the video server 102 selects the base layer Lj as a starting point to determine the quantization parameters 124, though this is not necessarily the case.
  • the illustrated embodiment should be understood as being illustrative of one contemplated embodiment and should not be construed as being limiting in any way.
  • the method 300 proceeds to operation 304, wherein the video server 102 obtains the bit usage feedback data 120 or other data indicating bit usage information associated with the selected layer L.
  • the bit usage information can include a feedback parameter FBj associated with the base layer Lj.
  • the bit usage information can include a feedback parameter FB N associated with a selected layer Ljy. It should be understood that these embodiments are illustrative, and should not be construed as being limiting in any way.
  • the bit usage information can be included in the bit usage feedback 120.
  • the bit usage feedback 120 is received by the rate controller 108 from the encoder 110.
  • the rate controller 108 monitors the encoded video stream 116 or the buffers 122 to determine the bit usage feedback 120.
  • the rate controller 108 receives the bit usage feedback 120 or other data for indicating bit usage from other devices or modules that are configured to monitor the encoded video stream 116 or the buffers 122.
  • operation 304 can include obtaining the bit usage feedback 120, receiving the bit usage feedback 120, and/or receiving other information, as well as identifying bit usage information in the bit usage feedback 120 associated with a particular level L being analyzed by the video server 102.
  • the video server 102 can add bit usage information from the layer L being analyzed to an associated buffer B and any higher buffers 122.
  • the feedback information associated with the layer L2 can be added to a buffer B2 associated with the layer L2 and buffers B3. . .BN associated with any higher enhancement layers L3. . .LN.
  • the video server 102 can omit the feedback information associated with the layer L2 from the buffer Bj associated with the base layer Lj.
  • some embodiments of the video server 102 consider the dependency of layers upon lower layers and not upon higher layers. It should be understood that this embodiment is illustrative, and should not be construed as being limiting in any way.
  • the method 300 proceeds to operation 308, wherein the video server 102 determines a quantization parameter 124 associated with the selected layer L.
  • the video server 102 can determine the quantization parameter QPj for the base layer Lj in operation 308.
  • the video server 102 can determine quantization parameters 124 for each analyzed layer L of the encoded video stream 116. It should be understood that the quantization parameters 124 can indicate how the encoder 110 is to encode each layer L of the encoded video stream 116 and can be based upon the various buffers B filled in operation 306 and bitrate usage feedback of the buffers B filled in operation 306.
  • operation 308 can include examining the bit usage feedback of the buffers B associated with the analyzed layer L as well as any higher layers L.
  • the quantization parameters 124 determined by the video server 102 can be based upon the dependencies discussed above with regard to the layers L.
  • the encoder 110 can modify encoding of the video data 112 according to the quantization parameters 124.
  • the methods 200 and 300 can be executed by the video server 102 to provide multi-layer rate control. From operation 314, the method 300 proceeds to operation 316. The method 300 ends at operation 316.
  • the computer architecture 400 illustrated in FIGURE 4 includes a central processing unit 402 ("CPU"), a system memory 404, including a random access memory 406 (“RAM”) and a read-only memory (“ROM”) 408, and a system bus 410 that couples the memory 404 to the CPU 402.
  • the computer architecture 400 further includes a mass storage device 412 for storing the operating system 106, the rate controller 108, the encoder 110, and the buffers 122.
  • the mass storage device 412 also can be configured to store the video data 112, data corresponding to the encoded video stream 116, the optimization parameters 124, and/or other data, if desired.
  • the computer architecture 400 may operate in a networked environment using logical connections to remote computers through a network such as the network 104.
  • the computer architecture 400 may connect to the network 104 through a network interface unit 414 connected to the bus 410. It should be appreciated that the network interface unit 414 also may be utilized to connect to other types of networks and remote computer systems, for example, the client device 118.
  • the computer architecture 400 also may include an input/output controller 416 for receiving and processing input from a number of other devices, including a keyboard, mouse, or electronic stylus (not shown in FIGURE 4). Similarly, the input/output controller 416 may provide output to a display screen, a printer, or other type of output device (also not shown in FIGURE 4).
  • Encoding the software modules presented herein also may transform the physical structure of the computer-readable media presented herein.
  • the specific transformation of physical structure may depend on various factors, in different implementations of this description. Examples of such factors may include, but are not limited to, the technology used to implement the computer-readable media, whether the computer-readable media is characterized as primary or secondary storage, and the like.
  • the computer-readable media is implemented as semiconductor- based memory
  • the software disclosed herein may be encoded on the computer-readable media by transforming the physical state of the semiconductor memory.
  • the software may transform the state of transistors, capacitors, or other discrete circuit elements constituting the semiconductor memory.
  • the software also may transform the physical state of such components in order to store data thereupon.
  • the computer-readable media disclosed herein may be implemented using magnetic or optical technology.
  • the software presented herein may transform the physical state of magnetic or optical media, when the software is encoded therein. These transformations may include altering the magnetic characteristics of particular locations within given magnetic media. These transformations also may include altering the physical features or characteristics of particular locations within given optical media, to change the optical characteristics of those locations. Other transformations of physical media are possible without departing from the scope and spirit of the present description, with the foregoing examples provided only to facilitate this discussion.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
PCT/US2013/024686 2012-02-14 2013-02-05 Multi-layer rate control Ceased WO2013122768A1 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
JP2014556597A JP2015510355A (ja) 2012-02-14 2013-02-05 コンピュータにより実行される方法及び記憶媒体
KR1020147022689A KR20140124415A (ko) 2012-02-14 2013-02-05 다층 레이트 제어 기법
EP13749904.2A EP2798848A4 (en) 2012-02-14 2013-02-05 MULTILAYER RATES CONTROL
CN201380009421.4A CN104106265A (zh) 2012-02-14 2013-02-05 多层速率控制

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US13/372,512 2012-02-14
US13/372,512 US20130208809A1 (en) 2012-02-14 2012-02-14 Multi-layer rate control

Publications (1)

Publication Number Publication Date
WO2013122768A1 true WO2013122768A1 (en) 2013-08-22

Family

ID=48945528

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2013/024686 Ceased WO2013122768A1 (en) 2012-02-14 2013-02-05 Multi-layer rate control

Country Status (6)

Country Link
US (1) US20130208809A1 (enExample)
EP (1) EP2798848A4 (enExample)
JP (1) JP2015510355A (enExample)
KR (1) KR20140124415A (enExample)
CN (1) CN104106265A (enExample)
WO (1) WO2013122768A1 (enExample)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130318251A1 (en) * 2012-05-22 2013-11-28 Alimuddin Mohammad Adaptive multipath content streaming
US9363131B2 (en) 2013-03-15 2016-06-07 Imagine Communications Corp. Generating a plurality of streams
US10893266B2 (en) * 2014-10-07 2021-01-12 Disney Enterprises, Inc. Method and system for optimizing bitrate selection
EP3151562B1 (en) * 2015-09-29 2020-06-17 Dolby Laboratories Licensing Corporation Feature based bitrate allocation in non-backward compatible multi-layer codec via machine learning
US11830225B2 (en) 2018-05-30 2023-11-28 Ati Technologies Ulc Graphics rendering with encoder feedback
KR20230028250A (ko) * 2020-06-30 2023-02-28 마이크로소프트 테크놀로지 라이센싱, 엘엘씨 강화 학습 기반 속도 제어

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6263022B1 (en) * 1999-07-06 2001-07-17 Philips Electronics North America Corp. System and method for fine granular scalable video with selective quality enhancement
US20070206674A1 (en) * 2006-03-01 2007-09-06 Streaming Networks (Pvt.) Ltd. Method and system for providing low cost robust operational control of video encoders
US20080056352A1 (en) * 2006-08-31 2008-03-06 Samsung Electronics Co., Ltd. Video encoding apparatus and method and video decoding apparatus and method
US20080159404A1 (en) * 2006-12-28 2008-07-03 Danny Hong System and method for in-loop deblocking in scalable video coding
US20090148054A1 (en) * 2007-12-06 2009-06-11 Samsung Electronics Co., Ltd. Method, medium and apparatus encoding/decoding image hierarchically

Family Cites Families (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
SE9103381L (sv) * 1991-11-15 1993-03-08 Televerket Foerfarande och apparat foer bildkodning med val av steghoejd i kvantiseraren
JP2979900B2 (ja) * 1993-05-26 1999-11-15 日本ビクター株式会社 記録媒体
US5515377A (en) * 1993-09-02 1996-05-07 At&T Corp. Adaptive video encoder for two-layer encoding of video signals on ATM (asynchronous transfer mode) networks
US5652616A (en) * 1996-08-06 1997-07-29 General Instrument Corporation Of Delaware Optimal disparity estimation for stereoscopic video coding
JP3263901B2 (ja) * 1997-02-06 2002-03-11 ソニー株式会社 画像信号符号化方法及び装置、画像信号復号化方法及び装置
JPH11112998A (ja) * 1997-10-01 1999-04-23 Matsushita Electric Ind Co Ltd 映像信号階層化符号化装置
JP2000333163A (ja) * 1999-05-24 2000-11-30 Sony Corp 復号装置及び方法、符号化装置及び方法、画像処理システム、画像処理方法
US6351491B1 (en) * 1999-06-23 2002-02-26 Sarnoff Corporation Apparatus and method for optimizing the rate control for multiscale entropy encoding
US6788740B1 (en) * 1999-10-01 2004-09-07 Koninklijke Philips Electronics N.V. System and method for encoding and decoding enhancement layer data using base layer quantization data
US6687779B1 (en) * 2000-07-14 2004-02-03 Texas Instruments Incorporated Method and apparatus for transmitting control information across a serialized bus interface
US7646816B2 (en) * 2001-09-19 2010-01-12 Microsoft Corporation Generalized reference decoder for image or video processing
KR100540655B1 (ko) * 2003-01-23 2006-01-10 삼성전자주식회사 비디오 코딩시의 비트율 제어 방법 및 장치
US7403660B2 (en) * 2003-04-30 2008-07-22 Nokia Corporation Encoding picture arrangement parameter in picture bitstream
US20050201629A1 (en) * 2004-03-09 2005-09-15 Nokia Corporation Method and system for scalable binarization of video data
KR101117840B1 (ko) * 2004-04-02 2012-03-15 톰슨 라이센싱 복잡도 가변 비디오 디코더를 위한 방법 및 장치
BRPI0515551A (pt) * 2004-09-17 2008-07-29 Matsushita Electric Industrial Co Ltd aparelho de codificação de áudio, aparelho de decodificação de áudio, aparelho de comunicação e método de codificação de áudio
US9124907B2 (en) * 2004-10-04 2015-09-01 Nokia Technologies Oy Picture buffering method
US7974341B2 (en) * 2005-05-03 2011-07-05 Qualcomm, Incorporated Rate control for multi-layer video design
US8229983B2 (en) * 2005-09-27 2012-07-24 Qualcomm Incorporated Channel switch frame
US8879857B2 (en) * 2005-09-27 2014-11-04 Qualcomm Incorporated Redundant data encoding methods and device
US8107537B2 (en) * 2006-02-02 2012-01-31 Sharp Laboratories Of America, Inc. Picture layer rate control for video encoding
CN101043619A (zh) * 2006-03-24 2007-09-26 华为技术有限公司 视频编码的误差控制系统和方法
EP1855271A1 (en) * 2006-05-12 2007-11-14 Deutsche Thomson-Brandt Gmbh Method and apparatus for re-encoding signals
US20080212673A1 (en) * 2007-03-01 2008-09-04 Peisong Chen Systems and Methods for Adaptively Determining I Frames for Acquisition and Base and Enhancement Layer Balancing
JP2009182442A (ja) * 2008-01-29 2009-08-13 Univ Of Fukui 動画像符号化・復号システム、並びにそれに用いる動画像符号化装置および動画像復号装置
JP4844595B2 (ja) * 2008-06-26 2011-12-28 日本ビクター株式会社 階層符号化装置、非階層符号化変換装置、階層符号化プログラム、および非階層符号化変換プログラム
EP2663076B1 (en) * 2009-04-20 2016-09-14 Dolby Laboratories Licensing Corporation Filter selection for video pre-processing in video applications
CN102714725B (zh) * 2010-01-06 2015-04-29 杜比实验室特许公司 用于多层视频编码应用的高性能码率控制
EP2529551B1 (en) * 2010-01-27 2022-03-16 Dolby Laboratories Licensing Corporation Methods and systems for reference processing in image and video codecs
US8644383B2 (en) * 2011-03-10 2014-02-04 Microsoft Corporation Mean absolute difference prediction for video encoding rate control

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6263022B1 (en) * 1999-07-06 2001-07-17 Philips Electronics North America Corp. System and method for fine granular scalable video with selective quality enhancement
US20070206674A1 (en) * 2006-03-01 2007-09-06 Streaming Networks (Pvt.) Ltd. Method and system for providing low cost robust operational control of video encoders
US20080056352A1 (en) * 2006-08-31 2008-03-06 Samsung Electronics Co., Ltd. Video encoding apparatus and method and video decoding apparatus and method
US20080159404A1 (en) * 2006-12-28 2008-07-03 Danny Hong System and method for in-loop deblocking in scalable video coding
US20090148054A1 (en) * 2007-12-06 2009-06-11 Samsung Electronics Co., Ltd. Method, medium and apparatus encoding/decoding image hierarchically

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP2798848A4 *

Also Published As

Publication number Publication date
US20130208809A1 (en) 2013-08-15
CN104106265A (zh) 2014-10-15
EP2798848A4 (en) 2016-01-06
KR20140124415A (ko) 2014-10-24
JP2015510355A (ja) 2015-04-02
EP2798848A1 (en) 2014-11-05

Similar Documents

Publication Publication Date Title
US11310546B2 (en) Distributed multi-datacenter video packaging system
US10271112B2 (en) System and method for dynamic adaptive video streaming using model predictive control
US10911796B2 (en) Dynamic quality adjustments for media transport
US20220030244A1 (en) Content adaptation for streaming
US8588297B2 (en) Quantization parameter prediction
US20130208809A1 (en) Multi-layer rate control
US8379670B2 (en) Method and device for transmitting video data
US20150207841A1 (en) Methods and systems of storage level video fragment management
US12192478B2 (en) Adaptively encoding video frames using content and network analysis
CN111787322B (zh) 视频编码的方法、装置、电子设备及计算机可读存储介质
US10003626B2 (en) Adaptive real-time transcoding method and streaming server therefor
US11595716B2 (en) Segment quality-guided adaptive stream creation
US10250657B2 (en) Streaming media optimization
CN110049336A (zh) 视频编码方法和视频解码方法
EP2798845B1 (en) Dynamic insertion of synchronization predicted video frames
WO2020018465A1 (en) Techniques for determining an upper bound on visual quality over a completed streaming session
EP3322189B1 (en) Method and system for controlling video transcoding
US9313243B2 (en) Video streaming over data networks
US9525641B1 (en) Facilitating buffer wait time determination based on device- or entity-related conditions
US9979765B2 (en) Adaptive connection switching
KR20230140276A (ko) 슬라이딩 윈도우 단위 예측 기반의 동영상 인코딩 최적화 방법 및 시스템
CN114025190A (zh) 多码率调度方法和多码率调度装置
US9253484B2 (en) Key frame aligned transcoding using statistics file
US9854260B2 (en) Key frame aligned transcoding using key frame list file

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13749904

Country of ref document: EP

Kind code of ref document: A1

REEP Request for entry into the european phase

Ref document number: 2013749904

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2013749904

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2014556597

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 20147022689

Country of ref document: KR

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE