WO2010034020A2 - Video streaming apparatus with quantization and method thereof - Google Patents

Video streaming apparatus with quantization and method thereof Download PDF

Info

Publication number
WO2010034020A2
WO2010034020A2 PCT/US2009/057917 US2009057917W WO2010034020A2 WO 2010034020 A2 WO2010034020 A2 WO 2010034020A2 US 2009057917 W US2009057917 W US 2009057917W WO 2010034020 A2 WO2010034020 A2 WO 2010034020A2
Authority
WO
WIPO (PCT)
Prior art keywords
video
quantizer
coefficients
bit stream
streaming
Prior art date
Application number
PCT/US2009/057917
Other languages
French (fr)
Other versions
WO2010034020A9 (en
WO2010034020A3 (en
Inventor
Brainerd Sathianathan
Original Assignee
Avot Media, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US12/235,107 external-priority patent/US8279925B2/en
Priority claimed from US12/235,129 external-priority patent/US8345746B2/en
Priority claimed from US12/235,118 external-priority patent/US8295345B2/en
Application filed by Avot Media, Inc. filed Critical Avot Media, Inc.
Priority to CA2737913A priority Critical patent/CA2737913C/en
Priority to BRPI0917087A priority patent/BRPI0917087A2/en
Priority to CN200980137200.9A priority patent/CN102160377A/en
Priority to EP09815405A priority patent/EP2335417A4/en
Publication of WO2010034020A2 publication Critical patent/WO2010034020A2/en
Publication of WO2010034020A9 publication Critical patent/WO2010034020A9/en
Publication of WO2010034020A3 publication Critical patent/WO2010034020A3/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44209Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/115Selection of the code volume for a coding unit prior to coding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124Quantisation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/156Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/40Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234354Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering signal-to-noise ratio parameters, e.g. requantization
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/266Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
    • H04N21/2662Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/61Network physical structure; Signal processing
    • H04N21/6106Network physical structure; Signal processing specially adapted to the downstream path of the transmission network
    • H04N21/6131Network physical structure; Signal processing specially adapted to the downstream path of the transmission network involving transmission via a mobile phone network
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/643Communication protocols
    • H04N21/6437Real-time Transport Protocol [RTP]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/65Transmission of management data between client and server
    • H04N21/658Transmission by the client directed to the server
    • H04N21/6582Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number

Definitions

  • the appliance and method relates generally to video streaming, changing the formats of video and a unit and method for quantization of digital data.
  • the coding may involve transcoding that converts a compressed video from one format into another compressed format.
  • the traditional approach is transcoding in the pixel domain referred to as the cascaded pixel domain approach.
  • the cascaded pixel domain approach involves decoding the original video signal (decompressing it), performing the intermediate processing and then fully re-encoding the processed signal.
  • the problem is that this type of transcoding is computationally expensive and very slow to perform which means that it is not possible to perform real-time transcoding.
  • the major areas of research in transcoding today focus on increasing the efficiency, i.e., decreasing the computational time. However, any gain in efficiency must have minimal impact on the quality of the transcoded video so that the viewer of the transcoded video does not perceive a degradation in video quality.
  • the streaming is done using constant bit rate video streams.
  • the bitrate, quantization, group of pixels and the regions of interest are constant.
  • the problem is that a constant bit rate video requires a large amount of bandwidth to stream the video. It is desirable to provide a variable bit rate video streaming technique so that the bit rate of the stream can be adjusted depending on the available bandwidth of the link between the originator of the streaming video and recipient of the streaming video.
  • One current technique to provide variable bit rate streaming videos is to store multiple version of each piece of the same video wherein each version of each piece of the video has different characteristics such as bit rate, quantization, group of pixels or the regions of interest. Then, when a video is going to be streamed, a picker selects the appropriate pieces of the video with the appropriate characteristics to stream to the user.
  • the problem with this approach is the massive amount of storage required to store all of the versions of each video. For example, a video that is 5 MB in size may require 200 + MB of storage since all of the different versions of the different pieces of the video must be stored. This massive amount of storage makes the current technique unworkable for sites/companies that have a large number of videos to stream to users.
  • the process of transcoding converts a compressed video from one format into another compressed video with a different format.
  • the traditional approach is to perform transcoding in the pixel domain referred to as the cascaded pixel domain approach.
  • This traditional approach involves decoding the original signal, perform the intermediate processing, and fully re-encoding the processed signal subject to any new constraints.
  • This traditional transcoding is very costly (from a computing resources stand-point and a length of time standpoint) to do.
  • the major area of research in transcoding now focuses on increasing the efficiency, i.e. decreasing the computational time.
  • any gains in efficiency should have a minimal impact on the quality of the transcoded video.
  • Figure 1 illustrates an embodiment of a video steaming apparatus with quantization
  • FIG. 2 illustrates more details of the embodiment of the video steaming apparatus with quantization
  • Figure 3 illustrates a method for video steaming with quantization
  • Figure 4 illustrates an example of a method for calculating quantization adjustments
  • Figure 5 illustrates a transcoding process
  • FIG. 6 illustrates an open loop transcoding process
  • Figure 7 illustrates a closed loop transcoding process
  • Figure 8 illustrates a closed loop transcoding process without motion estimation.
  • the apparatus and method are particularly applicable to the streaming of video signals over the web from a server to one or more mobile devices and it is in this context that the apparatus and method will be described. It will be appreciated, however, that the apparatus and method has greater utility as it may be used to stream video data from any origination source to any recipient and the video data may be streamed over any type of link with sufficient bandwidth to communicate the video data.
  • FIG. 1 illustrates an embodiment of a video steaming apparatus 10 with quantization that may be used to stream, in one implementation, video data from an originator, such as a server computer, over a link 11 , such as the Internet, to one or more target devices 13 that each have a media player to play the streaming videos.
  • Each target device may be a cellular phone, a mobile device with wireless telephone capabilities, a smart phone (such as the Blackberry products or the Apple iPhone) and the like which have a wireless capability and the capability to display/play a streaming video.
  • the apparatus 10 in one implementation, is one or more server computers executing a plurality of lines of computer code to implement the operations described below for the apparatus 10,
  • the apparatus 10 may also be implemented in hardware which may include one or more programmable logic devices, a processing unit executing a program and the like.
  • the appliance 10 may be two Intel E5420 Quad Core processors, 4 2Gb DDR2 DIMMs, two 1 TB RAID drives with 32MB cache, an Ubuntu 6.1 operating system and Dual Gigabyte Ethernet NICs.
  • the apparatus 10 transcodes and stream video files over a mobile link whose bandwidth and quality changes often using adjustable video quantization to provide a variable bit rate streaming apparatus and method without the enormous storage requirements of the typical systems.
  • the apparatus 10 may receive video links (for example, uniform resource locators) that are links to web-sites with source video data that is going to be transcoded and streamed using the apparatus.
  • the source video data may be in different formats such as in an MPEG format, an flash video format (.flv), a 3gp format, an mp4 format, an h.263 format or an h.264 format, etc. which need to be transcoded so that it can be streamed by the apparatus.
  • a crawler 12 may resolve the video links and then download the source video data to the apparatus 10.
  • the source video data is then passed onto a transcoder 14 that performs transcoding of the source video data to generate a transcoded video wherein all of the transcoded videos have the same form.
  • the source video data in the different formats may also require form conversion, resolution conversion, bit rate adjustment and/or frame rate adjustment.
  • the video Once the video had been transcoded into a transcoded video, it is passed onto a quantizer 16 that performs quantization on the transcoded video to generate a quanitized video (having a particular bit rate and quantization appropriate for the link over which the video will be streamed) which can then be streamed over the mobile link to one or more mobile devices.
  • a quantizer 16 that performs quantization on the transcoded video to generate a quanitized video (having a particular bit rate and quantization appropriate for the link over which the video will be streamed) which can then be streamed over the mobile link to one or more mobile devices.
  • Figure 2 illustrates more details of the embodiment of the video steaming apparatus 10 with quantization and Figure 3 illustrates a method 40 for video steaming with quantization.
  • the units and elements shown in Figure 2 may be implemented, in one embodiment, as a plurality of lines of computer code which are executed by an appliance with one or more processing units and memory wherein the plurality of lines of computer code are stored in the memory and executed by the one or more processing units.
  • the units and elements shown in Figure 2 may be implemented using ASICs or in software.
  • a piece of source video data is received by the transcoder 14 (process 42 as shown in Figure 3).
  • the source video data may include a flash video ( flv) or a live video on demand wherein the source video includes one or more elementary video pieces of data (in an h.263 format, for example) and one or more elementary audio pieces of data (in an mp3 format for example) which are wrapped in container structure (such as the flv format).
  • the elementary video and audio elements are extracted from the container (process 44 as shown in Figure 3).
  • the elementary video and audio elements then may be fed into a streamer unit 20 that includes the quantizer 16 that performs quantization of the elementary video and audio data (process 46 in Figure 3).
  • the quantizer may use a typical, known codec quantization function.
  • the level of quantization for each piece of elementary video and audio data depends on a feedback unit 26 that receives feedback data from the stream units 22, 24 (process 48 in Figure 3) and then feeds the feedback data to a controller 28 that adjusts the quantization of the quantizer based on the feedback data (process 50 in Figure 3) as described below in more detail.
  • the streamer unit 20 also may include an real time streaming protocol (RTSP) streaming unit 22 and an HTTP streaming unit 24 that, respectively, stream RTSP data and HTTP data.
  • RTSP real time streaming protocol
  • HTTP HTTP streaming unit
  • FIG. 4 illustrates an example of a method 60 for calculating quantization adjustments when using RTSP protocols.
  • the RTSP protocol standard defines a sender report (SR) and a receiver report (RR) that are exchanged between the streaming unit 61 , such as a streaming server, and a target device 62, such as a mobile phone over a link.
  • SR sender report
  • RR receiver report
  • video and audio elements of the streaming data 64 are sent from the streaming unit 61 to the target device 62 along with a well known sender report.
  • the target device according to the RTSP protocol, returns a receiver report.
  • the HTTP streaming will take place on top up TCP. Due to the connection mode of TCP , we can simple count the outgoing packets and the error rate of the network from the sockets.
  • the receiver report identifies and provides data about current link conditions.
  • the RR may include data about round trip time (RT) of the sender report being sent out until the receiver report is received, jitter, fractional packet loss and complete packet loss all of which can be used to assess the condition of the link between the streaming unit 61 and the target device.
  • feedback unit 26 gathers this information from each streamer unit 22, 24 and the controller performs an analysis of the feedback data to adjust the quantization of the quantizer 16 based on the condition of the link between the appliance 10 and each target device 13 as shown in Figure 1.
  • the method calculates a value called link throughput called AVTH.
  • the receiver report (RR) is obtained every two second from the target device(s) and the AVTH is calculated every second .
  • an average of two or three consecutive values of AVTH is determined and the average lambda is calculated.
  • d'jubK shape 116] [16]
  • the method may: 1. Obtain the Receiver Report
  • the resulting bit rate is calculated by the following: resulting_bit_rate: Quant( State_index * 2).
  • the unit and method are particularly applicable to transcoding compressed digital video data and it is in this context that the unit and method will be described. It will be appreciated, however, that the unit and method has greater utility since it can also be used to transcode other types of digital data or other compressed bitstreams and may be implemented differently than the example shown in the diagrams and described below, and these different uses of the transcoding and the different implementations are within the scope of the disclosure.
  • FIG. 5 illustrates a transcoding process 10 in which a transcoder 112, such as the video transcoder shown that may be implemented in software, hardware or a combination of hardware and software, may be used to convert a first compressed bit stream 114 into a second compressed bit stream 116.
  • the first compressed bit stream may have a bit rate BRl , a frame rate FRl , a spatial resolution RES 1 and be encoded using a standard Codec 1.
  • the compressed bit stream passes through the transcoder 112, it is converted into the second compressed bit stream that has bit rate BR2, a frame rate FR2, a spatial resolution RES2 and be encoded using a standard Codec2 wherein each of these parameters may be different from the parameters of the first compressed bit stream.
  • Figure 6 illustrates an open loop transcoding process 120 that is low-complexity and faster than typical transcoders.
  • the primary goal of the open-loop transcoder is to modify the output bit-rate through quantization adjustments directly on the transform domain.
  • the first compressed bit stream 114 is variable length decoded using a variable length decoder unit/module 122 which results in quantized coefficients
  • the current transcoder technique is applicable for most block-based and transform based video compression standards. This transcoder architecture can be extended to many other transforms including wavelets.
  • the quantized coefficients are inverse quantized using an inverse quantizer module/unit 124 which results in a set of coefficients which are then re-quantized with a quantizer unit/module 126 that has a different quantization level (so that it has a different bit rate) to generate the second compressed bit stream with the new output bit rate. Then, the re-quantized coefficients are variable length coded using a variable length coder unit/module 128 which results in the second compressed bit stream 116.
  • open loop transcoder In this open loop transcoding, the original motion vectors and video frame headers are preserved and re-used without any modification. However, the macroblock headers are re-evaluated. This is done due to the change of quantization, open loop transcoder does not require a frame memory (since no motion estimation is performed) and, since no motion estimation is performed, there is no need for a IDCT or DCT process unit. The open-loop transcoder may be subject to drift effect caused by the change in quantization.
  • Figure 7 illustrates a closed loop transcoding process 130 that is low-complexity and faster than typical transcoders.
  • the closed-loop system aims to eliminate the mismatch between predictive and residual components by approximating the cascading decoder-encoder architecture.
  • the closed loop transcoder may use a data re-use scheme or a fast pixel domain transcoder.
  • the closed loop transcoder 130 which may be a DCT domain transcoder with hybrid loop in one embodiment, may use the same VLD unit 122, inverse quantizer unit 124, quantizer 126 that quantizes at a second different quantization level and VLC unit 128 that may be implemented in the same manner as the open loop transcoder and may perform the same functions and operations as described above.
  • the closed loop transcoder 130 may decode the variable length coding using the VLD unit 122, inverse quantize the output of the VLD using the inverse quantizer 124 which results in a stream of coefficients. The stream of coefficients may then be combined with frame motion compensation changes by a combiner unit 136 and the resultant signal is quantized by the quantizer 126, combined with any motion vectors and variable length coded by the VLC unit 128 to generate the second compressed bit stream 116.
  • the closed loop transcoder 130 may implement, using a number of components, a first transcoder loop 132, a second transcoder loop 134 and a third transcoder loop 136 as shown by the dotted lines in Figure 7.
  • the first transcoder loop 132 may include the VLD and VLC units 122, 128 and operate to communicate the motion vector information (for a well known MPEG or h.263 P frame) in the first compressed bit stream 114 Rl from the VLD unit 122 to the VLC unit 128 so that, without recalculating the motion vector during the transcoding, so that the motion vector can be inserted back into the second compressed bit stream 116 (and avoid motion estimation) before it is variable length coded.
  • the motion vector information for a well known MPEG or h.263 P frame
  • the second transcoder loop 134 may perform motion compensation between on each frame of data wherein the quantized coefficients output from quantizer 126 are inverse quantized using an inverse quantizer unit 140 to generate coefficients, summed together with unquantized coefficients output from the summer 136 to generate a coefficient stream.
  • the coefficient stream may have an inverse DCT process performed using an inverse DCT unit 144 to generate uncompressed video data that is compared to a prior frame (F n- ]) using a unit 146 and a motion compensation unit 148 to generate new motion compensated frames.
  • the new motion compensated frame have discrete cosine transforms performed by a DCT unit 150 and a stream of DCT coefficients for the new motion compensated frame are fed into the summer unit 136 to perform motion compensation between frames.
  • the third transcoder loop 136 may remove drift caused by any bit rate change (such as the bit rate change caused by quantizer 126) by summing the original coefficients to the second quanitzer coefficients during the motion compensation.
  • this closed loop transcoder performs transcoding with minimal DCT/IDCT conversions.
  • FIG 8 illustrates a closed loop transcoding process 160 without motion estimation that shares many of the same components of the closed loop transcoder shown in Figure 7.
  • this transcoder 160 has the first and second transcoder loops 132, 134 that perform the same operations as described above.
  • the transcoder 160 may decode the variable length coding using the VLD unit 122, inverse quantize the output of the VLD using the inverse quantizer 124 which results in a stream of coefficients which are converted back into video data values using an inverse DCT unit 161. Then, the video data values are summed by the summer unit 136 with a prior frame of video data stored in a first frame buffer 162 as shown.
  • the sum of the two frames of video data is summed using a summer unit 163 with the output of the second transcoder loop 134 that performs motion compensation for each frame (and removes drift caused by varying the bit rate by varying the quantization level of Q2).
  • quantizes are simplified cascaded transcoders with a decoding and then an encoding unit.
  • Traditional quantizers' will have 4 inverse IDCT functions and 4 motion estimation/compensation functions.
  • the proposed quantizer will only have 2 inverse DCT function and completely eliminate the motion estimation fiinctions.
  • the quantizer 160 may avoid motion estimation during the quantization which increases the speed of the quantizer.
  • the quantizer 160 also avoids the discrete cosine transform (DCT) domain (or any other transform domain if another type of transform is being used in the compression) as much as possible since the process of working in the DCT domain is more computationally expensive.
  • DCT discrete cosine transform
  • the quantizer may receive an input video 114 that has been compressed at a first quantization level Ql (and thus have a first bit rate) and may output an output signal 116 that has been compressed at a second quantization level Q2 (and thus have a second bit rate).
  • the quantizer may include a variable length decode (VLD) unit 122 that decodes the variable length decoding of the incoming compressed video bitstream to generate a stream of quantized DCT coefficients, an inverse quantizer unit 124 that removes the quantization and results in a stream of unquantized DCT coefficients.
  • VLD variable length decode
  • the current quantizer's technique is applicable for most block-based and transform based video compression standards. This quantizer architecture can be extended to many other transforms including wavelets.
  • the quantizer may also include an inverse discrete cosine transform unit 161 that performs reverse DCT on the stream of coefficients and generates a stream of raw video data which means that the video data is in the pixel domain.
  • the resultant stream of raw video data (which represents a frame of video pixel data) is then summed/compared by a summer unit 136 to a prior frame of pixel data (to perform motion compensation in the pixel domain) that is stored in a frame buffer 162 and the resultant video pixel data is fed into a second summer unit 163 which combines the resultant video pixel data with a motion compensated frame of pixel data (at the second quantization level) that is stored in a second frame buffer 164 (to again perform motion compensation in the pixel domain) to generate video pixel data that has been motion compensated for the previous frame at the old quantization level as well as the frame at the new quantization level.
  • the resultant pixel data is then fed into a discrete cosine transform unit 150 that generate a new set of DCT coefficients based on the motion compensated video data frames.
  • the new set of DCT coefficients are then quantized by a quantizer unit 36 at a second, different quantization level to generate quantized DCT coefficients that may be variable length coded using a variable length coder unit 128) and output as a compressed output signal at the second quantization level (and thus at the second bit rate.)
  • the quantizer 160 may implement, using a number of components, a first quantizer loop 132, a second quantizer loop 134 and a third quantizer loop 156 as shown by the dotted lines in Figure 8.
  • the first quantizer loop 132 may include the VLD and VLC units 122, 128 and operate to communicate the motion vector information in the first compressed bit stream 14 Rl from the VLD unit 22 to the VLC unit 138 so that, without recalculating the motion vector (used for inter-frame motion compensation) during the transcoding, the motion vector can be inserted back into the second compressed bit stream 116 before it is variable length coded.
  • the second quantizer loop 134 may perform motion compensation between each frame of data on the encode side wherein the quantized coefficients output from quantizer 126 may be inverse quantized using an inverse quantizer unit 40 to generate coefficients, inverse discrete cosine transformed using an inverse discrete cosine transform unit 42 to generate frame pixel data at the new quantization level that is stored in the second frame buffer 164.
  • the frame pixel data at the new quantization level is then compared to/summed with the pixel data from the inverse DCT unit 161.
  • the third quantizer loop 156 may enhance motion compensation of the decoder side by summing/comparing the video pixel data from the current frame to the video pixel data from the prior frame that is stored in the first frame buffer 162.
  • the motion compensation occurs in the pixel domain which is computationally more efficient that performing the motion compensation in the DCT domain.
  • the three quantizer loops described above avoids motion estimation.
  • an I frame (which does not contain any motion compensation) is simply decompressed and then recompressed using the new quantization level.
  • a P frame (which has a motion vector associated with it) has its motion vector sent through the outer loop of the quantizer to avoid motion estimation.
  • the above quantizer is also capable of adjusting region of interest on any given video frame, the essential idea behind this is that in a given video frame there will be two different quantization values will be used.
  • the region that will have a lower quantization(higher quality) is determined by the motion vector information on the frame header .
  • the region of interest is configurable, by default a 4x4 macro-block region is considered as the region of interest.
  • the two different quantization levels are determined by configuration options provided to the qunatizer (for example : region of interest quantization factor is 6 and the other region is kept at 8).

Abstract

A streaming appliance having an internal quantizer is disclosed so that the bit rate of a streamed video can be adjusted based on the current conditions of a link between the streaming appliance and one or more target devices. In addition, a transcoder is provided that achieves lower complexity without picture quality loss wherein the transcoder may be either an open loop transcoder or a closed loop transcoder. Furtermore, a quantizer and method are disclosed.

Description

VIDEO SHtEAMING APPARATUS WITH QUANTIZATION AND METHOD THEREOF
Brianerd Sathianathan Priority Claims/Related Applications
This application claims priority under 35 USC 120 and 119(e) to U.S. patent application number 12/235,107, filed on 9/22/2008 and entitled "Video Streaming Apparatus with Quantization and Method Thereof, to U.S. patent application number 12/235,118, filed on 9/22/2008 and entitled "Transcoder Unit and Method" and to U.S. patent application number 12/235,129, filed on 9/22/2008 and entitled "Video Quantizer Unit and Method Thereof, all of which are herein incorporated by reference.
Field
The appliance and method relates generally to video streaming, changing the formats of video and a unit and method for quantization of digital data.
Background
Systems and methods that code video are well known. The coding may involve transcoding that converts a compressed video from one format into another compressed format. The traditional approach is transcoding in the pixel domain referred to as the cascaded pixel domain approach. The cascaded pixel domain approach involves decoding the original video signal (decompressing it), performing the intermediate processing and then fully re-encoding the processed signal. The problem is that this type of transcoding is computationally expensive and very slow to perform which means that it is not possible to perform real-time transcoding. The major areas of research in transcoding today focus on increasing the efficiency, i.e., decreasing the computational time. However, any gain in efficiency must have minimal impact on the quality of the transcoded video so that the viewer of the transcoded video does not perceive a degradation in video quality.
Currently, when a video is being streamed to a media player, the streaming is done using constant bit rate video streams. In the constant bit rate video streams, the bitrate, quantization, group of pixels and the regions of interest are constant. The problem is that a constant bit rate video requires a large amount of bandwidth to stream the video. It is desirable to provide a variable bit rate video streaming technique so that the bit rate of the stream can be adjusted depending on the available bandwidth of the link between the originator of the streaming video and recipient of the streaming video.
One current technique to provide variable bit rate streaming videos is to store multiple version of each piece of the same video wherein each version of each piece of the video has different characteristics such as bit rate, quantization, group of pixels or the regions of interest. Then, when a video is going to be streamed, a picker selects the appropriate pieces of the video with the appropriate characteristics to stream to the user. The problem with this approach is the massive amount of storage required to store all of the versions of each video. For example, a video that is 5 MB in size may require 200 + MB of storage since all of the different versions of the different pieces of the video must be stored. This massive amount of storage makes the current technique unworkable for sites/companies that have a large number of videos to stream to users. Thus, it is desirable to provide an apparatus and method that overcomes the problems with the current techniques and it is to this end that the apparatus and method are directed. hi addition, the process of transcoding converts a compressed video from one format into another compressed video with a different format. The traditional approach is to perform transcoding in the pixel domain referred to as the cascaded pixel domain approach. This traditional approach involves decoding the original signal, perform the intermediate processing, and fully re-encoding the processed signal subject to any new constraints. This traditional transcoding is very costly (from a computing resources stand-point and a length of time standpoint) to do. Thus, the major area of research in transcoding now focuses on increasing the efficiency, i.e. decreasing the computational time. However, any gains in efficiency should have a minimal impact on the quality of the transcoded video. Thus, it is desirable to achieve bit rate reduction using low-complexity, faster transcoding methods and it is to this end that the unit and method are directed.
Furthermore, systems and methods that quantize digital data, such as video data, are well known. For example, most video, still image and audio compression schemes use some form of quantization to reduce the bandwidth of the compressed data. Most of these known quantization systems are too slow for some applications and/or are too computationally expensive. Thus, it is desirable to provide a quantizer and method that overcomes the problems with the current techniques and it is to this end that the apparatus and method are directed. Brief Description of the Drawings
Figure 1 illustrates an embodiment of a video steaming apparatus with quantization;
Figure 2 illustrates more details of the embodiment of the video steaming apparatus with quantization;
Figure 3 illustrates a method for video steaming with quantization;
Figure 4 illustrates an example of a method for calculating quantization adjustments;
Figure 5 illustrates a transcoding process;
Figure 6 illustrates an open loop transcoding process;
Figure 7 illustrates a closed loop transcoding process; and
Figure 8 illustrates a closed loop transcoding process without motion estimation.
Detailed Description of One or More Embodiments
The apparatus and method are particularly applicable to the streaming of video signals over the web from a server to one or more mobile devices and it is in this context that the apparatus and method will be described. It will be appreciated, however, that the apparatus and method has greater utility as it may be used to stream video data from any origination source to any recipient and the video data may be streamed over any type of link with sufficient bandwidth to communicate the video data.
Figure 1 illustrates an embodiment of a video steaming apparatus 10 with quantization that may be used to stream, in one implementation, video data from an originator, such as a server computer, over a link 11 , such as the Internet, to one or more target devices 13 that each have a media player to play the streaming videos. Each target device may be a cellular phone, a mobile device with wireless telephone capabilities, a smart phone (such as the Blackberry products or the Apple iPhone) and the like which have a wireless capability and the capability to display/play a streaming video. The apparatus 10, in one implementation, is one or more server computers executing a plurality of lines of computer code to implement the operations described below for the apparatus 10, The apparatus 10 may also be implemented in hardware which may include one or more programmable logic devices, a processing unit executing a program and the like. In one implementation, the appliance 10 may be two Intel E5420 Quad Core processors, 4 2Gb DDR2 DIMMs, two 1 TB RAID drives with 32MB cache, an Ubuntu 6.1 operating system and Dual Gigabyte Ethernet NICs.
The apparatus 10 transcodes and stream video files over a mobile link whose bandwidth and quality changes often using adjustable video quantization to provide a variable bit rate streaming apparatus and method without the enormous storage requirements of the typical systems. As shown in Figure 1, the apparatus 10 may receive video links (for example, uniform resource locators) that are links to web-sites with source video data that is going to be transcoded and streamed using the apparatus. The source video data may be in different formats such as in an MPEG format, an flash video format (.flv), a 3gp format, an mp4 format, an h.263 format or an h.264 format, etc. which need to be transcoded so that it can be streamed by the apparatus. In the apparatus 10, a crawler 12 may resolve the video links and then download the source video data to the apparatus 10. The source video data is then passed onto a transcoder 14 that performs transcoding of the source video data to generate a transcoded video wherein all of the transcoded videos have the same form. During the transcoding, the source video data in the different formats may also require form conversion, resolution conversion, bit rate adjustment and/or frame rate adjustment.
Once the video had been transcoded into a transcoded video, it is passed onto a quantizer 16 that performs quantization on the transcoded video to generate a quanitized video (having a particular bit rate and quantization appropriate for the link over which the video will be streamed) which can then be streamed over the mobile link to one or more mobile devices.
Figure 2 illustrates more details of the embodiment of the video steaming apparatus 10 with quantization and Figure 3 illustrates a method 40 for video steaming with quantization. The units and elements shown in Figure 2 may be implemented, in one embodiment, as a plurality of lines of computer code which are executed by an appliance with one or more processing units and memory wherein the plurality of lines of computer code are stored in the memory and executed by the one or more processing units. Alternatively, the units and elements shown in Figure 2 may be implemented using ASICs or in software. As shown in Figure 2, a piece of source video data is received by the transcoder 14 (process 42 as shown in Figure 3). The source video data may include a flash video ( flv) or a live video on demand wherein the source video includes one or more elementary video pieces of data (in an h.263 format, for example) and one or more elementary audio pieces of data (in an mp3 format for example) which are wrapped in container structure (such as the flv format). Thus, as part of the transcoding process, the elementary video and audio elements are extracted from the container (process 44 as shown in Figure 3). There are three primary outputs that are expected from the streamer, these outputs are based on the currently supported mobile phone standards. They are shown below as pairs of video and audio.
1. H263/AMR
2. H264/AAC
3. MPEG4/AAC
Based on this output expectation the streamer will expect either on of the above elemetry stream formats from the transcoder. However this architecture is not limited to the above formats and can easily be extended to support future formats as well with minimal changes to the transcoder and quantizer.
The elementary video and audio elements then may be fed into a streamer unit 20 that includes the quantizer 16 that performs quantization of the elementary video and audio data (process 46 in Figure 3). The quantizer may use a typical, known codec quantization function. The level of quantization for each piece of elementary video and audio data depends on a feedback unit 26 that receives feedback data from the stream units 22, 24 (process 48 in Figure 3) and then feeds the feedback data to a controller 28 that adjusts the quantization of the quantizer based on the feedback data (process 50 in Figure 3) as described below in more detail. Once the elementary video and audio have been quantized, the quantized video and audio data are combined and output for streaming. As shown in Figure 2, the streamer unit 20 also may include an real time streaming protocol (RTSP) streaming unit 22 and an HTTP streaming unit 24 that, respectively, stream RTSP data and HTTP data. The two streaming units shown in Figure 2 allow the streamer 20 (and therefore the appliance 10) to stream data in the two well known streaming output protocols.
Figure 4 illustrates an example of a method 60 for calculating quantization adjustments when using RTSP protocols. In particular, the RTSP protocol standard defines a sender report (SR) and a receiver report (RR) that are exchanged between the streaming unit 61 , such as a streaming server, and a target device 62, such as a mobile phone over a link. As shown, video and audio elements of the streaming data 64 are sent from the streaming unit 61 to the target device 62 along with a well known sender report. In response, the target device, according to the RTSP protocol, returns a receiver report. The HTTP streaming will take place on top up TCP. Due to the connection mode of TCP , we can simple count the outgoing packets and the error rate of the network from the sockets. This data will be used to calculate the average throughput(AVTH). The receiver report identifies and provides data about current link conditions. For example, the RR may include data about round trip time (RT) of the sender report being sent out until the receiver report is received, jitter, fractional packet loss and complete packet loss all of which can be used to assess the condition of the link between the streaming unit 61 and the target device. As shown in Figure 2, feedback unit 26 gathers this information from each streamer unit 22, 24 and the controller performs an analysis of the feedback data to adjust the quantization of the quantizer 16 based on the condition of the link between the appliance 10 and each target device 13 as shown in Figure 1.
In one implementation, the controller 28 may be implemented in software executing on a processing unit and may operate as set forth in the pseudocode below: double getAVTH(double RTT, double PL , double FPL , double Jitter) double rtt_multi= 0.0; double pl_multi = 0.0; double jitter_multi = 0.0; double fpljmulti = 0.0; rtt_multi = (RTT==0) ? 0: RTT; Plmulti = (PL<=1) ? PL: (PL/10000); jitterjnulti = (Jitter<1)? Jitter : (Jitter/100000); fpLmulti = (FPL<=1) ? FPL: (FPL/1000); return (rtt_multi + pl_multi + jitter_multi +fpl_multi); } wherein the inputs are: Round trip delay (RTT), Packet Loss (PL), Fractional Packet
Loss (FPL) and jitter (Jitter) which are derived from the RR. The method calculates a value called link throughput called AVTH. The receiver report (RR) is obtained every two second from the target device(s) and the AVTH is calculated every second . Depending on the configuration, an average of two or three consecutive values of AVTH is determined and the average lambda is calculated. There are 16 states of calibrated values for lambda: d'jubK shape 116] [16] ={
{2.18,-1,-1, "1, -1, "1, -1, -1,-1, -1,-1,-1, -1,-1, -1, -1), {2.18,?.19, -1,-1, -1,-1,-1, -1,-1, -1,-1, -1,-1, -1, -1, -l) , {2.18, 5.17, 5. la, -1,-1, -1, -1, -1, -1, -1, -1, -1,-1,-1,-1,-1), {?..18, 5.17, 10.3, 10.31, -1,-1, -1,-1, -1,-1, -1,-1,-1, -1,-1, -l}, {2.18,3.74,8.96,20.32,20.33,-1,-1,-1,-1,-1, -1,-1, -1,-1,-1,-1 }, {2.18, 3.74, 8.96, 18.36, 33.3b, 33.37, -1,-1, -1,-1, -1,-1, -1, -1, -1,-1), {2. IS, 3.74, 6.2, 13.26, 25.16, 45.69, 45.70, -1,-1, -1,-1, -1,-1,-1,-1,-1}, {2.18, 3.74, 6.2, 13.26, 25.16, 40.28, 58.76, 58.77, -1, -1, -1, -1, -1,-1,-1,-1},
{2.18, 3.74, 6.2, 13.26, 20.32, 40.28, 79.55, 142.94, 142.95, -1,-1, -1, -1, -1,-1, -1}, {2. IS, 3.18, 5.17, 11.22, 13.36, 33.35, 58.76, 142.9, 197.05, 197.06, -1,-1, -1,-1,-1,-1}, {1.76, 2.18, 3.18, 5.17, 11.22, 18.36, 33.35, 47.71, 58.76, 142.9, 143, -1,-1,-1,-1,-1}, {] .76, 2.18, 3.18, 5.17, 11.22,18.36, 33.35,47,71, 58.76,142.9,197.1,197.12,-1,-1,-1, -l}, {1.76, 2.18,3.54, 6.2, 12.24, 18.36, 29.58, 40.28, 47.71, 79.55 ,142.9, 197.1,197.12, -1,-1,-1},
{1.76,2.18,3.54,6.2,12.24,18.36,29.58,40.28,47.71,79.55, 142.9,197.1, 663.85,663.86,-1,-1),
{1.76,2.18,3.54, 6.2, 12.24, 18.36,29.58, 40.26,47.71, 79.55, 142.9,197.1, 663.85,1945.06, 1945.06, -l),
{1.76,2.18,3.18,4.29,5.17,12.24,25.16,33.35,45.69,58.76,79.55,90.28,142.94,663.85,1945.06,1945.0
},-
In most cases we assume that there are 16 states and every state has two 2 value differences in quantization . The higher the state, the higher the quantization which means a lower bit rate. The lower the state, the lower the value in quantization and a higher bit rate.
In more detail, the method may: 1. Obtain the Receiver Report
2. Calculate AVTH
3. Calculate Average lambda
4. Select on a state , assume there is 16 states , compare where the average labmda value with the state . For example say lambda is 45.7 in that case that value is between the 9 th and the 10 index on the matrix .
The resulting bit rate is calculated by the following: resulting_bit_rate: Quant( State_index * 2).
Trancoder Unit
The unit and method are particularly applicable to transcoding compressed digital video data and it is in this context that the unit and method will be described. It will be appreciated, however, that the unit and method has greater utility since it can also be used to transcode other types of digital data or other compressed bitstreams and may be implemented differently than the example shown in the diagrams and described below, and these different uses of the transcoding and the different implementations are within the scope of the disclosure.
Figure 5 illustrates a transcoding process 10 in which a transcoder 112, such as the video transcoder shown that may be implemented in software, hardware or a combination of hardware and software, may be used to convert a first compressed bit stream 114 into a second compressed bit stream 116. The first compressed bit stream may have a bit rate BRl , a frame rate FRl , a spatial resolution RES 1 and be encoded using a standard Codec 1. Once the compressed bit stream passes through the transcoder 112, it is converted into the second compressed bit stream that has bit rate BR2, a frame rate FR2, a spatial resolution RES2 and be encoded using a standard Codec2 wherein each of these parameters may be different from the parameters of the first compressed bit stream. Now, a two implementations of a transcoder that achieves bit rate reduction using low-complexity, faster transcoding methods that result in the highest possible quality of video are described. In particular, an open loop low-complexity, faster transcoder implementation and a closed loop low-complexity, faster transcoder implementation are described.
Figure 6 illustrates an open loop transcoding process 120 that is low-complexity and faster than typical transcoders. The primary goal of the open-loop transcoder is to modify the output bit-rate through quantization adjustments directly on the transform domain. In the open loop process shown, the first compressed bit stream 114 is variable length decoded using a variable length decoder unit/module 122 which results in quantized coefficients
(whether a DCT process or other transform process is used). The current transcoder technique is applicable for most block-based and transform based video compression standards. This transcoder architecture can be extended to many other transforms including wavelets. The quantized coefficients are inverse quantized using an inverse quantizer module/unit 124 which results in a set of coefficients which are then re-quantized with a quantizer unit/module 126 that has a different quantization level (so that it has a different bit rate) to generate the second compressed bit stream with the new output bit rate. Then, the re-quantized coefficients are variable length coded using a variable length coder unit/module 128 which results in the second compressed bit stream 116. In this open loop transcoding, the original motion vectors and video frame headers are preserved and re-used without any modification. However, the macroblock headers are re-evaluated. This is done due to the change of quantization, open loop transcoder does not require a frame memory (since no motion estimation is performed) and, since no motion estimation is performed, there is no need for a IDCT or DCT process unit. The open-loop transcoder may be subject to drift effect caused by the change in quantization.
Figure 7 illustrates a closed loop transcoding process 130 that is low-complexity and faster than typical transcoders. The closed-loop system aims to eliminate the mismatch between predictive and residual components by approximating the cascading decoder-encoder architecture. The closed loop transcoder may use a data re-use scheme or a fast pixel domain transcoder. The closed loop transcoder 130, which may be a DCT domain transcoder with hybrid loop in one embodiment, may use the same VLD unit 122, inverse quantizer unit 124, quantizer 126 that quantizes at a second different quantization level and VLC unit 128 that may be implemented in the same manner as the open loop transcoder and may perform the same functions and operations as described above.
The closed loop transcoder 130 may decode the variable length coding using the VLD unit 122, inverse quantize the output of the VLD using the inverse quantizer 124 which results in a stream of coefficients. The stream of coefficients may then be combined with frame motion compensation changes by a combiner unit 136 and the resultant signal is quantized by the quantizer 126, combined with any motion vectors and variable length coded by the VLC unit 128 to generate the second compressed bit stream 116. The closed loop transcoder 130 may implement, using a number of components, a first transcoder loop 132, a second transcoder loop 134 and a third transcoder loop 136 as shown by the dotted lines in Figure 7. The first transcoder loop 132 may include the VLD and VLC units 122, 128 and operate to communicate the motion vector information (for a well known MPEG or h.263 P frame) in the first compressed bit stream 114 Rl from the VLD unit 122 to the VLC unit 128 so that, without recalculating the motion vector during the transcoding, so that the motion vector can be inserted back into the second compressed bit stream 116 (and avoid motion estimation) before it is variable length coded.
The second transcoder loop 134 may perform motion compensation between on each frame of data wherein the quantized coefficients output from quantizer 126 are inverse quantized using an inverse quantizer unit 140 to generate coefficients, summed together with unquantized coefficients output from the summer 136 to generate a coefficient stream. The coefficient stream may have an inverse DCT process performed using an inverse DCT unit 144 to generate uncompressed video data that is compared to a prior frame (Fn-]) using a unit 146 and a motion compensation unit 148 to generate new motion compensated frames. The new motion compensated frame have discrete cosine transforms performed by a DCT unit 150 and a stream of DCT coefficients for the new motion compensated frame are fed into the summer unit 136 to perform motion compensation between frames.
The third transcoder loop 136 may remove drift caused by any bit rate change (such as the bit rate change caused by quantizer 126) by summing the original coefficients to the second quanitzer coefficients during the motion compensation. Thus, this closed loop transcoder performs transcoding with minimal DCT/IDCT conversions.
Figure 8 illustrates a closed loop transcoding process 160 without motion estimation that shares many of the same components of the closed loop transcoder shown in Figure 7. As with the transcoder shown in Figure 7, this transcoder 160 has the first and second transcoder loops 132, 134 that perform the same operations as described above. The transcoder 160 may decode the variable length coding using the VLD unit 122, inverse quantize the output of the VLD using the inverse quantizer 124 which results in a stream of coefficients which are converted back into video data values using an inverse DCT unit 161. Then, the video data values are summed by the summer unit 136 with a prior frame of video data stored in a first frame buffer 162 as shown. Then, the sum of the two frames of video data is summed using a summer unit 163 with the output of the second transcoder loop 134 that performs motion compensation for each frame (and removes drift caused by varying the bit rate by varying the quantization level of Q2).
Quantizer Unit
Referring back to Figure 8, an example of an embodiment of a quantizer 160. In traditional implementations quantizes are simplified cascaded transcoders with a decoding and then an encoding unit. Traditional quantizers' will have 4 inverse IDCT functions and 4 motion estimation/compensation functions. The proposed quantizer will only have 2 inverse DCT function and completely eliminate the motion estimation fiinctions. The quantizer 160 may avoid motion estimation during the quantization which increases the speed of the quantizer. The quantizer 160 also avoids the discrete cosine transform (DCT) domain (or any other transform domain if another type of transform is being used in the compression) as much as possible since the process of working in the DCT domain is more computationally expensive.
The quantizer may receive an input video 114 that has been compressed at a first quantization level Ql (and thus have a first bit rate) and may output an output signal 116 that has been compressed at a second quantization level Q2 (and thus have a second bit rate). The quantizer may include a variable length decode (VLD) unit 122 that decodes the variable length decoding of the incoming compressed video bitstream to generate a stream of quantized DCT coefficients, an inverse quantizer unit 124 that removes the quantization and results in a stream of unquantized DCT coefficients. The current quantizer's technique is applicable for most block-based and transform based video compression standards. This quantizer architecture can be extended to many other transforms including wavelets. The quantizer may also include an inverse discrete cosine transform unit 161 that performs reverse DCT on the stream of coefficients and generates a stream of raw video data which means that the video data is in the pixel domain. The resultant stream of raw video data (which represents a frame of video pixel data) is then summed/compared by a summer unit 136 to a prior frame of pixel data (to perform motion compensation in the pixel domain) that is stored in a frame buffer 162 and the resultant video pixel data is fed into a second summer unit 163 which combines the resultant video pixel data with a motion compensated frame of pixel data (at the second quantization level) that is stored in a second frame buffer 164 (to again perform motion compensation in the pixel domain) to generate video pixel data that has been motion compensated for the previous frame at the old quantization level as well as the frame at the new quantization level. The resultant pixel data is then fed into a discrete cosine transform unit 150 that generate a new set of DCT coefficients based on the motion compensated video data frames. The new set of DCT coefficients are then quantized by a quantizer unit 36 at a second, different quantization level to generate quantized DCT coefficients that may be variable length coded using a variable length coder unit 128) and output as a compressed output signal at the second quantization level (and thus at the second bit rate.)
The quantizer 160 may implement, using a number of components, a first quantizer loop 132, a second quantizer loop 134 and a third quantizer loop 156 as shown by the dotted lines in Figure 8. The first quantizer loop 132 may include the VLD and VLC units 122, 128 and operate to communicate the motion vector information in the first compressed bit stream 14 Rl from the VLD unit 22 to the VLC unit 138 so that, without recalculating the motion vector (used for inter-frame motion compensation) during the transcoding, the motion vector can be inserted back into the second compressed bit stream 116 before it is variable length coded.
The second quantizer loop 134 may perform motion compensation between each frame of data on the encode side wherein the quantized coefficients output from quantizer 126 may be inverse quantized using an inverse quantizer unit 40 to generate coefficients, inverse discrete cosine transformed using an inverse discrete cosine transform unit 42 to generate frame pixel data at the new quantization level that is stored in the second frame buffer 164. The frame pixel data at the new quantization level is then compared to/summed with the pixel data from the inverse DCT unit 161.
The third quantizer loop 156 may enhance motion compensation of the decoder side by summing/comparing the video pixel data from the current frame to the video pixel data from the prior frame that is stored in the first frame buffer 162.
In the quantizer shown in Figure 8, the motion compensation occurs in the pixel domain which is computationally more efficient that performing the motion compensation in the DCT domain. In addition, the three quantizer loops described above avoids motion estimation. Using the quantizer, an I frame (which does not contain any motion compensation) is simply decompressed and then recompressed using the new quantization level. A P frame (which has a motion vector associated with it) has its motion vector sent through the outer loop of the quantizer to avoid motion estimation.
The above quantizer is also capable of adjusting region of interest on any given video frame, the essential idea behind this is that in a given video frame there will be two different quantization values will be used. The region that will have a lower quantization(higher quality) is determined by the motion vector information on the frame header . As a result elements that have higher movements in the picture will be lot clearer than the static elements. The region of interest is configurable, by default a 4x4 macro-block region is considered as the region of interest. The two different quantization levels are determined by configuration options provided to the qunatizer (for example : region of interest quantization factor is 6 and the other region is kept at 8). While the foregoing has been with reference to a particular embodiment of the invention, it will be appreciated by those skilled in the art that changes in this embodiment may be made without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims.

Claims

Claims:
1. A streaming appliance that receives a video input and outputs streaming content, the streaming appliance comprising: a streamer unit that receives a video input; a quantizer that quantizes the video input at a quantization level and generates a quantized video; and one or more streaming units that stream the quantized video to one or more target devices.
2. The streaming appliance of claim 1 further comprising a transcoder that separates the video input into one or more audio elements and one or more video elements wherein the one or more audio elements and one or more video elements are fed into the quantizer to be quantized.
3. The streaming appliance of claim 1, wherein the one or more streaming units further comprises an real time streaming protocol streaming unit and a hypertext transfer protocol streaming unit.
4. The streaming appliance of claim 1 further comprising a feedback unit that receives feedback data from the one or more streaming units and a controller that uses the feedback data to determine whether to change the quantization level of the quantizer.
5. The streaming appliance of claim 4, wherein the feedback data further comprises a receiver report.
6. The streaming appliance of claim 5, wherein controller receives the receiver report that contains a round trip time, a fractional packet loss value, a packet loss value and a jitter value and calculates, based on the round trip time, the fractional packet loss value, the packet loss value and the jitter value, a new quantization level for the quantizer to adjust the bit rate of the streaming content to each target device.
7, The streaming appliance of claim 2 further comprising a crawler that resolves a link to video input and downloads the video input to the transcoder.
8. A method for streaming video using a streaming appliance, the method comprising: receiving a video input at a streamer unit; quantizing, using a quantizer in the streamer unit, the video input at a quantization level to generate a quantized video; streaming the quantized video to one or more target devices at the quantization level; and adjusting the quanitization level of the quantized video based on the condition of a link between the streamer unit and the one or more target devices,
9. The method of claim 8 further comprising separating, using a transcoder, the video input into one or more audio elements and one or more video elements wherein the one or more audio elements and one or more video elements are fed into the quantizer to be quantized.
10. The method 8 further comprising receiving feedback data from one or more streaming units and wherein adjusting the quanitization level of the quantized video further comprising adjusting the quanitization level of the quantized video based on the feedback data.
11. The method of claim 10, wherein the feedback data further comprises a receiver report.
12. The method of claim 11 , wherein controller receives the receiver report that contains a round trip time, a fractional packet loss value, a packet loss value and a jitter value and wherein adjusting the quanitization level of the quantized video further comprising adjusting the quanitization level of the quantized video based on the round trip time, the fractional packet loss value, the packet loss value and the jitter value.
13. An open loop transcoder that transcodes an first compressed bit stream having a first bit rate to a second compressed bit stream having a second bit rate, the transcoder comprising: an input element receives a first compressed bit stream having a first bit rate; an inverse quantizer unit that receives the first compressed bit stream and generates, based on the first bit rate corresponding to a first quantization level, a stream of coefficients; a quantizer that quantizes the stream of coefficients at a second quantization level to generate a stream of coefficients at the second quantization level; and an output element that, based on the stream of coefficients at the second quantization level, generate a second bit stream having a second bit rate,
14. The transcoder of claim 13, wherein the input element further comprises a variable length decoder that generates a first variable length decoded bit stream that is fed into the inverse quantizer unit.
15. The transcoder of claim 14, wherein the output element further comprises a variable length coder that, based on the stream of coefficients at the second quantization level, generate a second bit stream having a second bit rate.
16. A closed loop transcoder, comprising: a first transcoder loop that passes a motion vector for a frame in the first compressed bit stream from an input element to an output element of the transcoder to insert the motion vectors into an output compressed bit stream; an inverse quantizer unit that generates a stream of coefficients from the first compressed bit stream based on a first quantization level; a second transcoder loop that performs inter-frame motion compensation based on the stream of coefficients from the first compressed bit stream and a stream of coefficients from a previous frame of the first compressed bit stream; and a quantizer that outputs a stream of coefficients at a second quantization level based on the inter-frame motion compensation.
17. The closed loop transcoder of claim 16, wherein each coefficient further comprises a discrete transform coefficient.
18. A quantizer, comprising: an input that receives a first compressed bit stream having a first quantization level; an inverse quantizer unit that performs inverse quantization of the first compressed bit stream at the first quantization level to generate a set of coefficients; an inverse transform unit that performs an inverse transform to generate a frame of pixel data from the set of coefficients; a motion compensation unit that performs motion compensation between the frame of pixel data and a prior frame of pixel data to generate motion compensated pixel data; a transform unit that generates a set of coefficients based on the motion compensated pixel data; and a quantizer that performs quantization at a second quantization level based on the set of coefficients from the motion compensated pixel data to generate a second compressed bit stream having the second quantization level.
19. The quantizer of claim 18 further comprising a quantizer loop that extracts a motion vector from the first compressed bit stream and provides the extracted motion vector to the motion compensation unit.
20. The quantizer of claim 19 further comprising a variable length decoder that decodes a variable length code first compressed bit stream to generate a set of quantized coefficients that are input to the inverse quantizer unit.
21. The quantizer of claim 20 further comprising a variable length coder that performs variable length coding of the second compressed bit stream.
22. The quantizer of claim 21, wherein the coefficients further comprises discrete cosine coefficients.
23. A method for quantizing, comprising: receiving a first compressed bit stream having a first quantization level; generating a set of coefficients based on the first compressed bit stream using an inverse quantizer operating at the first quantization level; generate a frame of pixel data from the set of coefficients; performing motion compensation between the frame of pixel data and a prior frame of pixel data to generate motion compensated pixel data; generating a set of coefficients at a second quantization level based on the motion compensated pixel data; and generating a second compressed bit stream having the second quantization level.
24. The method of claim 23 further comprising extracting a motion vector from the first compressed bit stream and performing the motion compensating using the extracted motion vector.
25. The method of claim 24 further comprising decoding a variable length code to generate a set of quantized coefficients that are input to the inverse quantizer.
26. The method of claim 25, further comprising performing variable length coding of the set of coefficients at the second quantization level to generate the second compressed bit stream.
27. The method of claim 26, wherein the coefficients further comprises discrete cosine coefficients.
PCT/US2009/057917 2008-09-22 2009-09-22 Video streaming apparatus with quantization and method thereof WO2010034020A2 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
CA2737913A CA2737913C (en) 2008-09-22 2009-09-22 Video streaming apparatus with quantization and method thereof
BRPI0917087A BRPI0917087A2 (en) 2008-09-22 2009-09-22 video transmission device with quantization and quantization method
CN200980137200.9A CN102160377A (en) 2008-09-22 2009-09-22 Video streaming apparatus with quantization and method thereof
EP09815405A EP2335417A4 (en) 2008-09-22 2009-09-22 Video streaming apparatus with quantization and method thereof

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US12/235,107 2008-09-22
US12/235,129 2008-09-22
US12/235,107 US8279925B2 (en) 2008-09-22 2008-09-22 Video streaming apparatus with quantization and method thereof
US12/235,129 US8345746B2 (en) 2008-09-22 2008-09-22 Video quantizer unit and method thereof
US12/235,118 US8295345B2 (en) 2008-09-22 2008-09-22 Transcoder unit and method
US12/235,118 2008-09-22

Publications (3)

Publication Number Publication Date
WO2010034020A2 true WO2010034020A2 (en) 2010-03-25
WO2010034020A9 WO2010034020A9 (en) 2010-05-14
WO2010034020A3 WO2010034020A3 (en) 2010-07-08

Family

ID=42040209

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2009/057917 WO2010034020A2 (en) 2008-09-22 2009-09-22 Video streaming apparatus with quantization and method thereof

Country Status (5)

Country Link
EP (1) EP2335417A4 (en)
CN (1) CN102160377A (en)
BR (1) BRPI0917087A2 (en)
CA (1) CA2737913C (en)
WO (1) WO2010034020A2 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130308699A1 (en) * 2012-05-18 2013-11-21 Home Box Office, Inc. Audio-visual content delivery

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105306950B (en) * 2015-12-07 2018-06-15 河南工程学院 A kind of video compress distance transmission system for feeding back coarse quantization reconstructed frame

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6434197B1 (en) * 1999-01-07 2002-08-13 General Instrument Corporation Multi-functional transcoder for compressed bit streams
US7400588B2 (en) * 2003-08-01 2008-07-15 Thomson Licensing Dynamic rate adaptation using neural networks for transmitting video data
EP1580914A1 (en) * 2004-03-26 2005-09-28 STMicroelectronics S.r.l. Method and system for controlling operation of a network
EP1615447B1 (en) * 2004-07-09 2016-03-09 STMicroelectronics Srl Method and system for delivery of coded information streams, related network and computer program product therefor
US8447121B2 (en) * 2005-09-14 2013-05-21 Microsoft Corporation Efficient integrated digital video transcoding
KR20070081949A (en) * 2006-02-14 2007-08-20 엘지전자 주식회사 Transcoding apparatus and method thereof
US8428142B2 (en) * 2007-03-05 2013-04-23 Stmicroelectronics International N.V. System and method for transcoding data from one video standard to another video standard

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
None
See also references of EP2335417A4

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130308699A1 (en) * 2012-05-18 2013-11-21 Home Box Office, Inc. Audio-visual content delivery
WO2013173721A1 (en) * 2012-05-18 2013-11-21 Home Box Office, Inc. Audio-visual content delivery
US9538183B2 (en) 2012-05-18 2017-01-03 Home Box Office, Inc. Audio-visual content delivery with partial encoding of content chunks

Also Published As

Publication number Publication date
EP2335417A2 (en) 2011-06-22
CA2737913C (en) 2014-07-22
EP2335417A4 (en) 2012-04-04
WO2010034020A9 (en) 2010-05-14
CA2737913A1 (en) 2010-03-25
CN102160377A (en) 2011-08-17
WO2010034020A3 (en) 2010-07-08
BRPI0917087A2 (en) 2016-06-14

Similar Documents

Publication Publication Date Title
US8311095B2 (en) Method and apparatus for transcoding between hybrid video codec bitstreams
EP2088782B1 (en) A method and a device for transcoding video
US20060018378A1 (en) Method and system for delivery of coded information streams, related network and computer program product therefor
US20090147856A1 (en) Variable color format based video encoding and decoding methods and apparatuses
KR20030073254A (en) Method for adaptive encoding motion image based on the temperal and spatial complexity and apparatus thereof
US20110246673A1 (en) Method and System for Optimizing the Content and Transfer of Media Files
WO2008091687A2 (en) Reduced resolution video transcoding with greatly reduced complexity
JP5021167B2 (en) Apparatus and method for multiple description coding
US8199825B2 (en) Reducing the resolution of media data
US8565302B2 (en) Video streaming apparatus with quantization and method thereof
US20130121404A1 (en) Transcoder Unit and Method
Björk et al. Video transcoding for universal multimedia access
CA2737913C (en) Video streaming apparatus with quantization and method thereof
KR20070081949A (en) Transcoding apparatus and method thereof
Assuncao et al. Rate-reduction techniques for MPEG-2 video bit streams
US8345746B2 (en) Video quantizer unit and method thereof
US8175151B2 (en) Encoders and image encoding methods
Yuan et al. The fast close-loop video transcoder with limited drifting error
Lee et al. Computation reduction in cascaded DCT-domain video downscaling transcoding
KR100918499B1 (en) Apparatus and method for multi-pass encoding
US20230269386A1 (en) Optimized fast multipass video transcoding
KR100932727B1 (en) Video stream switching device and method
Wegner et al. Homogenous HEVC video transcoding by transform coefficient removal
KR20180113868A (en) Image Reencoding Method based on Decoding Data of Image of Camera and System thereof
Khan et al. Extreme rate transcoding for dynamic video rate adaptation

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 200980137200.9

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 09815405

Country of ref document: EP

Kind code of ref document: A2

WWE Wipo information: entry into national phase

Ref document number: 1981/DELNP/2011

Country of ref document: IN

WWE Wipo information: entry into national phase

Ref document number: 2737913

Country of ref document: CA

NENP Non-entry into the national phase

Ref country code: DE

REEP Request for entry into the european phase

Ref document number: 2009815405

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2009815405

Country of ref document: EP

REG Reference to national code

Ref country code: BR

Ref legal event code: B01E

Ref document number: PI0917087

Country of ref document: BR

Free format text: SOLICITA-SE A REGULARIZACAO DA PROCURACAO, TENDO EM VISTA QUE A APRESENTADA ESTA DATADA DE 20/04/2011, SENDO QUE A PETICAO DE ENTRADA NA FASE NACIONAL OCORREU EM 21/03/2011, E O TEXTO DA MESMA NAO POSSUI CLAUSULA QUE RATIFICA OS ATOS PRATICADOS ANTERIORMENTE.

ENP Entry into the national phase

Ref document number: PI0917087

Country of ref document: BR

Kind code of ref document: A2

Effective date: 20110321