Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
  • H04N19/436—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation using parallelised computational arrangements
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
  • H04N19/115—Selection of the code volume for a coding unit prior to coding
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
  • H04N19/124—Quantisation
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
  • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
  • H04N19/136—Incoming video signal characteristics or properties
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
  • H04N19/142—Detection of scene cut or scene change
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
  • H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
  • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
  • H04N19/174—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a slice, e.g. a line of blocks or a group of blocks
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
  • H04N19/177—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a group of pictures [GOP]
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
  • H04N19/18—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a set of transform coefficients
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
  • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding

Definitions

• the present invention relates to video processing, and more particularly to a method and apparatus for controlling the rate of data production by multiple encoding engines provided to compress video in a television encoder, such as a digital high definition television (HDTV) or standard definition television (SDTV) encoder.
• a television encoder such as a digital high definition television (HDTV) or standard definition television (SDTV) encoder.
• Rate control is an essential part of a video encoder.
• the picture is processed using multiple encoding engines for data compression. These multiple encoding engines operate on the picture simultaneously, but share a common data buffer. Thus, the rate at which data is produced by the multiple engines must be carefully regulated in order to prevent buffer overflow, buffer underflow, and other problematic conditions.
• rate control algorithms have been proposed for use in conjunction with a single video compressor. See, e.g., "MPEG2 Test Model 5" referred to above.
• the parallel encoding engines are synchronized at the video slice instead of frame level; thus, the rate control can be treated as if only a single compressor were provided. None of the prior art schemes provide a robust solution to the problem of regulating the rate of data production by a plurality of compressors, in order to control the quantization of a digital video encoder that uses parallel compression engines .
• rate control is provided in a video encoder having multiple compression engines that process a picture simultaneously and share a common buffer.
• Nonlinear feedback is used to force a quantizer scale at the last slice of an image panel to be approximately the same as the quantizer scale at the first slice of the image panel. This avoids visible artifacts caused by abrupt changes in the quantizer scale at the panel boundaries.
• a target bit rate for a group of pictures (GOP) is adjusted based on the number of film pictures and non- film pictures currently in the processing pipeline of the compression engines. Since non-film processing in general requires a higher target bit rate, this adjustment allows the rate control algorithm to adapt when the video switches from film to non-film or vice versa .
• the buffer level is used to control the start of a new GOP. If there is not sufficient space in the buffer to accommodate the intra-coded (I) frame, the start of a new GOP is delayed.
• a target quantization level is determined for a video frame.
• the target quantization level is applied to the first slice of every video image panel of a plurality of panels representing the video frame and being processed in parallel by the
• the quantization level is driven toward the target quantization level at the last slice of each of the image panels.
• the driving step can use, for example, piecewise linear feedback to drive the quantization level of the last slice of each of the image panels toward the target quantization level.
• the feedback can compensate for variations in the quantization level between the first and last slice of each image panel.
• a group of pictures (GOP) target bit rate can be adjusted based on a number of film pictures and non- film pictures currently in a processing pipeline of at least one of the compression engines. A higher target bit rate can be provided for non-film pictures.
• a buffer level of the video encoder can be used to control the start of a new group of pictures (GOP) . The start of a new GOP is delayed if there is not sufficient space in the buffer to accommodate an intra- coded (I) frame for the new GOP.
• Every input picture can be partitioned into a fixed number of horizontal panels, wherein each of the panels is compressed by a separate compression engine.
• the panels can be simultaneously compressed during a frame time.
• the compressed panel data are stored locally at the compression engines for subsequent transfer to a video buffer of the video encoder within a next frame time.
• data are retrieved from the buffer at an average rate equal to a specified video bit rate whenever the buffer has at least one transport packet payload's worth of data. Null packets are substituted for video packets to maintain a constant transport bit rate whenever the buffer level falls below one transport packet payload's worth of data.
• a reference quantizer scale is calculated for each of the compression engines.
• the reference quantizer scale is calculated based on an accumulation of quantizer scale values for the compression engine (PanelSumQuant ) , an accumulation of the number of bits generated on that compression engine (PanelBitCount ) ; an accumulation of the number of macroblocks processed on that compression engine (PanelMbCount ) ; and a fullness level of a video buffer of the video encoder (VideoBufferLevel) .
• the compression engine modifies its reference quantizer scale based on a local buffer fullness to generate a final quantizer scale value for use in quantization.
• a panic mode can be initiated by the compression engine if the final quantizer scale value is higher than a predetermined maximum value. The panic mode forces all or part of the DCT coefficient to take on a zero value.
• a method is also provided for controlling the rate at which data is processed by a digital video encoder using a plurality of parallel compression engines to compress successive macroblocks of video data.
• Frame level rate control is provided for successive video frames in accordance with a picture start interrupt service routine once per coded picture.
• Macroblock level rate control is provided in accordance with a rate control interrupt service routine a plurality of times per coded picture.
• the picture start interrupt routine updates frame level statistical variables by processing data collected from a prior coded frame.
• a target bit rate is calculated for a new group of pictures (GOP) if a new frame to be processed comprises an intra-coded (I) frame.
• I intra-coded
• upper and lower limits are calculated for the number of bits that a new frame is allowed to generate.
• a target number of bits is computed for each frame and for each of a plurality of panels into which each frame is divided. The target number of bits can be computed by distributing the bits in proportion to relative complexity values of the video frames being processed.
• a frame target quantizer scale is calculated from the target number of bits computed for a frame.
• the frame target quantizer scale is provided to each of the plurality of compression engines as an initial reference quantizer scale for use in commencing coding of the video frame.
• the compression engines can force some of the Discrete Cosine Transform (DCT) coefficients of the video frame macroblocks to zero, if the reference quantizer scale is too high to maintain the number of bits that the new frame is allowed to generate within the applicable upper limit.
• the rate control interrupt service routine provides interrupts which initiate a modulation of a frame target quantizer scale. The modulation can be based on feedback to attempt to bring the actual number of bits for each of the video frame panels to the corresponding target number of bits for the panel.
• FIG. 1 is a block diagram illustrating an HDTV encoder with which the present invention may be utilized.
• the present invention provides a method and apparatus for rate control in a digital video encoder that uses multiple compression engines running in parallel.
• every input picture is partitioned into eight horizontal panels by an HDTV panel splitter 10.
• Each panel is then compressed by an individual video panel compressor 12, for example, the DvXpert chip available from C-Cube Microsystems of 1778 McCarthy Blvd., Milpitas, CA 95035, USA.
• an individual video panel compressor 12 for example, the DvXpert chip available from C-Cube Microsystems of 1778 McCarthy Boulevard., Milpitas, CA 95035, USA. It should be appreciated that a different number of panels can be utilized, and that the invention can be implemented on video encoders other than the DvXpert encoder described herein.
• the panels into which the input pictures are partitioned are simultaneously compressed during a frame time.
• the compressed panel data are stored locally on the panel compressor, and then transferred to the primary video FIFO 16 ("video buffer") within the next frame time.
• the bits are retrieved from the FIFO by a packet processor 18 at an average rate equal to the user specified video bit rate (bit_rate) .
• the retrieved bits are used to form a transport packet in accordance with the well known MPEG (Motion Picture Experts Group) standard.
• MPEG Motion Picture Experts Group
• the rate control function is mainly performed by a master compression controller (MCC) 14.
• MCC master compression controller
• the MCC collects the statistics (PanelSumQuant, PanelBitCount, and PanelMbCount) from the panel compressors and the buffer level from the video FIFO, and then calculates an 8 bit reference quantizer scale (PanelQuant) for each panel compressor.
• the panel compressor modifies the reference quantizer scale based on local activities and local buffer fullness to generate the final quantizer scale value to use for quantization. If the desired quantizer scale is higher than the maximum supported by the MPEG quantization table, the panel compressor may apply a panic mode and force some of the DCT (Discrete Cosine Transform) coefficients to zero.
• DCT Discrete Cosine Transform
• a sync interrupt generator 20 which receives the input HDTV video, generates a field sync interrupt to the MCC at the beginning of every input field, and a Rate-Control (RC) Sync interrupt periodically (e.g., every 1.67 ms for 10801 mode) within a field time.
• RC Rate-Control
• a field sync interrupt is qualified as a picture start if the panel compressors are ready to encode a new picture at the current field time.
• Rate control by the MCC is performed in two parts: (1) Frame level rate control is carried out at the Picture Start interrupt service routine once per coded picture; (2) Macroblock level rate control is carried out at the RC interrupt service routine up to 20 times per coded picture for 1080 line interlaced (10801) mode (up to 10 times per coded picture for 720 line progressive scan (720P) mode). On average, only about eight RC interrupts are used to regulate the quantization because the panel compressor only uses a portion of the field time to carry out the encoding process.
• the goal of rate control is to maintain a consistent video quality over time and over the picture.
• a basic assumption of the rate control algorithm is that the coding complexity of a particular picture type is approximately constant from frame to frame of a steady scene, and is measured by the product of the average quantizer scale and the number of bits generated over the frame (or a panel) .
• the complexity of the current frame is normally estimated from that of the previous frame, except a when a scene change or fade-in from black is detected, in which case the complexity estimates are reset to some provisional values .
• the MCC At picture start interrupt, the MCC first updates all frame level statistical variables by processing the data collected from the last coded frame.
• a target bit rate for the new GOP is calculated if the upcoming frame is an intra-coded frame (I frame) .
• the GOP target bit rate calculation aims to bring the encoder buffer to a consistently low level at the beginning of every GOP so that there is enough room in the buffer to accommodate the surge of bits produced by the new I frame .
• the MCC calculates the upper and lower limits on the number of bits that the new frame is allowed to generate. These limits are based on the constraint that neither the encoder buffer nor the decoder buffer is allowed to overflow nor underflow.
• the MCC also computes the target number of bits to be generated for the frame and for each panel by distributing the bits in proportion to the relative complexity values. Then, a frame target quantizer scale is calculated from the target frame bit rate. The frame target quantizer scale is delivered to every one of the multiple panel compressors as the initial reference quantizer scale to start coding the frame.
• the MCC modulates the frame target quantizer scale by a feedback mechanism which tries to bring the panel bit count to a value that equals the targets. If the accumulated bit count is approaching the upper limit that is calculated at the picture start, the modulated quantizer scale is further increased to slow down the production of bits. The resulting reference quantizer scale value is then clipped to fit into an eight bit integer value for delivery to every panel compressor.
• the panel compressor Upon receiving the reference quantizer scale, the panel compressor checks its local buffer level and increases the reference quantizer scale if needed to protect the buffer. The panel compressor may also force some DCT coefficients to be zero if the reference quantizer scale is too high. Then, adaptive quantization is performed in a manner similar to the MPEG Test Model 5 and the actual quantization scale is
• MPEG Moving Picture Experts Group
• the Packet Processor sends the data packet from the video FIFO as soon as the first 184 bytes of the compressed data packet are written into the FIFO.
• the rate control task receives the following configuration parameters: 1. Elementary video stream bit rate (BitRate) .
• Video Format (1920x10801 or 1440x10801 or 1280x720P) .
• Video frame rate (29.97 Hz or 30 Hz or 59.94 Hz or 60 Hz) . 1302
• the frame bit count and average quantizer scales are calculated from the data read from the panel compressors.
• the bit count used in the complexity calculation is only for the slice layer bits.
• the bits generated for the headers, which are generated by the MCC, are not included.
• Avg uant sum of PanelSumQuant over all 8 panels / number of macroblocks per frame
• SumPanelBitCount sum of PanelBitCount over all 8 panels
• the rate control algorithm maintains a queue to store the complexity values of the four most recently coded P frames, and a queue to store the complexity values of the four most recently coded B frames.
• the complexity estimate for P (or B) frame, X[P] (or X[B]), is then calculated as the average of the four complexity values stored in the P (or B) complexity queue.
• the complexity estimates for I, P and B frames are used to calculate the target quantizer scale value and the bit budget for upcoming frames to be encoded. Details are described in the following sections.
• the I complexity estimate and the P and B complexity queues are initialized to a set of default startup complexity values (StartupX [I/P/B] ) . If the current frame to be encoded is the first I or P frame after fade-in from black or scene change, the I complexity estimate and the values stored in the P and B complexity queues are conditionally replaced by the default startup value. Every stored value in the P and B complexity queue and the I complexity, are compared with the startup complexity. The stored complexity is replaced by the startup complexity if the stored complexity is smaller.
• the complexity estimates are reset only once at the first scene change. This is achieved by skipping the reset if the complexity adjustment occurred in the previous P frame.
• the default startup complexities are moderate complexity values calculated based on the assumption that an I frame consumes approximately 4 times as many bits as a P frame, and B frames consume approximately 20% fewer bits than P frames. This ratio of 4.0:1.0:0.8 is obtained from empirical observations.
• I:P:B ratios are possible.
• the startup complexities assume a provisional average quantizer scale value (StartupQuant ) which varies linearly between 20 to 28 for bit rate values between 10 Mbps to 26 Mbps . This provisional average quantizer value is clamped at 20 for bit rate values lower than 10 Mbps, and at 28 for bit rate values higher than 26 Mbps.
• the startup complexity values for I, P, and B frames are calculated as follows:
• StartupX [I] 4.0 * StartupQuant * N * (BitRate / FrameRate) / (4.0 + 1.0*Np + 0.8*Nb)
• StartupX [P] 1.0 * StartupQuant * N * (BitRate / FrameRate) / (4.0 + 1.0*Np + 0.8*Nb)
• StartupX [B] 0.8 * StartupQuant * N * (BitRate / FrameRate) / (4.0 + 1.0*Np + 0.8*Nb)
• a state machine is implemented to handle the complexity update for the P and B frames following the complexity reset.
• the complexity estimates are set to the reset value before the frame is encoded. After encoding the initial frame, the complexity estimates are updated using the new encoding complexity (AvgQuant * SumPanelBitCount). If the initial frame is an I frame, the I complexity estimate is updated. If the initial frame is a P frame, the entire P complexity queue is replaced by the new encoding complexity. The P encoding complexity is used to estimate the B encoding complexity, so the entire B complexity queue is replaced by the encoding complexity of the P frame multiplied by the value 0.8, which value is the estimated ratio of B complexity to P complexity. Other reasonable values can, of course, be used.
• the B complexity queue is updated normally except for the last B frame in the group. After encoding the last B frame in the group, the B complexity queue is set to the reset value so that statistics possibly from the old scene will not contaminate the complexity estimates of the new scene.
• the complexity queue is updated normally after encoding the first P frame following the initial frame.
• the entire B complexity queue is replaced by the new encoding complexity (AvgQuant * SumPanelBitCount) of the B frame.
• the complexity queues are updated normally (by replacing the oldest entry by a new value) for every frame thereafter. For example, consider a scene change that occurs at frame 10 in the following sequence of frames (in encoding order) : 10, Bl, B2, P3, B4, B5,
• complexity queue is set to the new encoding complexity after encoding Bl. This new complexity estimate is used for encoding B2.
• the B complexity queue is set to the reset value.
• the entire P complexity queue is replaced by the encoding complexity of P3
• the entire B complexity queue is replaced by the same value multiplied by, e.g., 0.8.
• the entire B complexity queue is replaced by the encoding complexity of B . From B5 and P6 onward, the complexity queues are updated normally.
• a fade-out to black increases the complexity for P and B frames but decreases the complexity of I frames. 1302
• a constant factor larger than one is multiplied with every entry of the B and P complexity queues to increase the complexity estimate.
• the multiplier 1.25 can be used for P complexity and the multiplier 1.5 can be used for B complexity.
• the I complexity estimate is reduced by multiplying a constant factor less than one for every frame where fade-out is detected.
• the multiplier 0.975 can, for example, be applied on I complexity for every frame during a fade-out to black.
• the data transfer rate of the compressed data from the panel compressors' local buffers to the video FIFO via the PCI bus is much faster than the data output rate from the video FIFO, we can consider the combined video FIFO and the panel compressor's local buffers together as a single entity for the encoder's buffer level considerations.
• the level of the video FIFO is latched at Field Sync and is read by the MCC via the PCI bus interface.
• EncoderBufferLevel VideoBufferLevel + FrameBitCount ⁇ participateate ⁇ - 00/21302
• a new GOP is set up if the current frame is an I frame.
• the MCC rate control algorithm calculates the bit budget for the new GOP, assuming a nominal number of frames for the GOP (N) .
• the MCC rate control algorithm tries to bring the video buffer to a constant low level (TargetBufferLevel) at the start of every GOP.
• the TargetBufferLevel is set to be a fraction (one-eighth) of the usable buffer space in the encoder and decoder combined.
• TargetBufferLevel (BufferDelay * BitRate)/ 8
• BufferDelay is the total buffer delay which is detailed in section D below.
• the MCC estimates the time duration of the upcoming GOP (GopDuration) based on the film mode status.
• the GOP is divided into two parts, namely, (i) the frames already preprocessed and waiting in the encoder's pipeline, and (ii) the future frames that have not yet entered the encoder's pipeline.
• the exact duration of the frames can be calculated from the number of frames in the pipeline (PipelineSize) and the number of frames that have a repeat field (PipelineRepeatFields) .
• the future frames that have not yet entered the pipeline it is assumed that those frames have the same film mode classification as the most recently preprocessed frame, and the three-two pulldown pattern is continued if they are classified as film.
• the current I frame is not included in the calculation of the PipelineSize and
• GopinputFrames PipelineSize + ( 0.5 * PipelineRepeatFields ) + 1.25 * ( N - 1 - PipelineSize )
• GopinputFrames 1.0 + PipelineSize + ( 0.5 * PipelineRepeatFields ) + 1.25 * (N - 2 - PipelineSize)
• GopinputFrames 1.5 + PipelineSize + ( 0.5 * PipelineRepeatFields ) + 1.25 * (N - 2 - PipelineSize)
• GopDuration PictureDuration + (GopinputFrames / InputFrameRate )
• the target bit rate of the new GOP is the average number of bits transmitted during the GOP plus the surplus or deficit carried over from the previous GOP. This carry over is reflected by the deviation of the current buffer level from the target buffer level.
• GOP_target ( GopDuration * BitRate ) + ⁇ * (TargetBufferLevel - EncoderBufferLevel )
• This arrangement allows the rate control algorithm to respond quickly to make use of the unused bit (if any) left over from the previous GOP, yet avoids starving the new GOP if the previous GOP consumes more bits than the budget.
• any negative feedback is absorbed by the buffer and spread over future GOPs to avoid sudden degradation in the video quality.
• the GopTarget may be adjusted later during the GOP if the GOP size is extended to align the next I frame with a scene change, or if the film mode status changes .
• the GOP target is adjusted every frame for the following: i) If a scene change is detected, the encoder may lengthen the GOP to align the next I frame with the scene change. In this case, the GopTarget is increased accordingly to provide the bits needs for coding the extra bits. This adjustment is only done on P frames, since extra B frames must be accompanied by P frames.
• the increment is (1.25 * M * BitRate/FrameRate ) , otherwise the increment is (M * BitRate / FrameRate) .
• M is the distance (in frames) between two consecutive P frames.
• the GopTarget is increased accordingly to account for the expected increase in the expected duration of the GOP.
• the increase in GopTarget is calculated as:
• the GopTarget is decreased by the same amount to account for the expected decrease in the duration of the GOP.
• the film to video (or video to film) adjustment is skipped if the number of frames
• the end-to-end buffer delay is set by the packet processor and is defined as the time elapsed from the time the first bit of the picture enters the encoder FIFO to the time the picture is pulled out from the decoder FIFO (Decoder Time Stamp (DTS) of the picture) .
• the buffer delay is calculated from the following equation:
• BufferDelay Min (7995392 bits / BitRate , 0.5 second )
• the delay is fixed at, e.g., 0.5 second for bit rates below 16 Mbps, and decreases linearly for bit rates above 16 Mbps, e.g. 0.2 second at 40Mbps.
• the encoder's current buffer level is higher than the nominal number of bits (BitRate * PictureDuration) to be transmitted from the encoder buffer to the decoder buffer during the current frame time, a constant output bit rate (BitRate) will be maintained. Otherwise, the packet processor will generate null packets to fill up the gap in the transport bandwidth, which effectively reduces the video bit rate. Therefore, the projected video FIFO level one frame time later when the current picture enters the video FIFO is: max ( EncoderBufferLevel - (BitRate * PictureDuration ) , 0 )
• FrameMaxDecoder (BufferDelay -MaxJitter) *BitRate - Max (EncoderBufferLevel - (BitRate * PictureDuration) , 0)
• the size of the current frame plus the level of the FIFO when the current frame is transferred to the video FIFO should not exceed the size of the video FIFO (VideoFifoSize) , i.e.:
• Frame axEncoder VideoFifoSize - Max EncoderBufferLevel - (BitRate * PictureDuration) , 0)
• the size of the video FIFO is 12582912 bits, although any other suitable FIFO size can be used as will be appreciated by those skilled in the art.
• the maximum allowable bitstream size of the current frame is then calculated as the smaller of FrameMaxEncoder and FrameMaxDecoder.
• FrameMaxBits MIN (FrameMaxEncoder, FrameMaxDecoder) - 2944
• the packet processor inserts null packets to effectively reduce the video bandwidth when there is not enough data in the video FIFO, it is not necessary to set a lower bound on the frame size to protect the encoder or decoder buffer.
• the MCC rate control algorithm calculates a target for the number of bits to be generated for the frame (FrameTargetBits) .
• the basic idea is to distribute the available bits to the I, P, and B frames in proportion to the complexity estimates adjusted by a set of constant weighting factors (K[I], K[P], K[B]).
• the weighting factors (K[]) account for the different perceptual and statistical characteristics of I, P and B frames.
• the weighting factors (0.75, 1.0, 0.75) can be used, although other weighting factors are equally applicable.
• a portion of the deviation of the actual bits used from the target is fed back to adjust the target for the future frames.
• FrameTargetBits GopTarget * K[I] * X[I] / (K[I]*X[I] + Np*K[P]*X[P] + Nb*K[B]*X[B] )
• the smaller of FrameTargetBits value of the I frame and the actual number of bits generated by the I frame is then deducted from the GopTarget before allocating the remaining bits to the P and B frames. Note that if the actual number of bits generated exceeds the target, the target is deducted from the GopTarget instead of the actual bits because the I frame consumes bits at a much higher rate than P and B frames. Deviation of the I frame's bit consumption from the target may deplete the bit budget of the remaining P and B frames. By decoupling the actual I frame bits from the B and P bit allocation, the feedback is deferred to the next GOP.
• AllocationFactortP] (N-l) / (Np + Nb* Max (1/2 , Min (3, (K[B]*X[B] / K[P]*X[P]) ) ) ) ) )
• AllocationFactor[B] (N-l) / (Nb + Np* Max (1/3 , Min (2, (K[P]*X[P] / K[B]*X[B]) ) ) ) ) ) )
• the ratio of the weighting complexity (K*X) of P to B frames are bounded by and 3 to avoid extreme allocation caused by an inaccurate complexity estimate.
• An initial frame target is then computed by applying the allocation factor to the average number of bits available to the P and B frames, as follows:
• InitialFrameTargetBits AllocationFactor * GopTarget / (GopLength-1)
• GopLength is the current length of the GOP, which may be larger than NominalGopLength because the GOP is extended to align the next I frame with the scene change. If the current frame count with respect to the start of the GOP is less than the nominal GOP length (N) , GopLength is set to the same value as N. If the current frame count exceeds N, GopLength is increased by M frames for every time a P frame is encountered. Feedback is then added to the InitialFrameTargetBits to compensate the deviation of the actual bit consumption from the target. The feedback is computed as:
• GopBitCount AccumulatedFrameTargetBits is the accumulation of the FrameTargetBits since the beginning of the GOP. The PictureFeedback portion of the FrameTargetBits is excluded in the accumulation in every P and B frame. GopBitCount is the number of bits generated by the current GOP up to and including the last coded frame.
• the GopBitCount is increased by the effective number of bits generated by the frame. Since the null packet is inserted to sustain the transport bit rate when the video FIFO level is lower than that required to produce one transport packet, the effective number of bits produced for the last coded frame is lower bounded by the number of bits that are required to sustain a constant output bit rate during the upcoming frame time minus the number of bits that are already in the video FIFO.
• the effective number of bits generated is:
• ⁇ controls the response of the system. If ⁇ is too big, the system becomes unstable; if ⁇ is too small, the response is slow.
• ⁇ can be set to 1/N, although other values can be substituted.
• the magnitude of PictureFeedback is bounded to not exceed one-third of the InitialFrameTargetBits if the feedback is negative.
• FrameTargetBits InitialFrameTargetBits + PictureFeedback
• the FrameTargetBits is bounded both from above and below. It is necessary that the FrameTargetBits does not lie in the proximity of the maximum allowable number of bits that applies for buffer protection. Otherwise, panic mode may be invoked too early and the picture quality will be excessively degraded. Although it is not necessary to set a minimum picture size to protect the buffer, it is desirable that the encoded frame is big enough such that bandwidth would not be wasted by sending null packets. Therefore, a minimum value for the FrameTargetBits is calculated as the worst case number of bits required to prevent the encoder video FIFO from emptying.
• FrameMinTarget (1.5 * BitRate / FrameRate) - Max (EncoderBufferLevel - (BitRate * PictureDuration) , 0 )
• FrameTargetBits Min ( FrameTargetBits, 0.8 * FrameMaxBits, MaxTarget)
• FrameTargetBits Max (FrameTargetBits, FrameMinTarget, MinTarget )
• MaxTarget and MinTarget are defined as: 1302
• MaxTarget [I frame] GopTarget - (N-l) * (Number of Macroblocks per frame) * 8
• MaxTarget [P or B frame] GopTarget / N
• MinTarget [I frame] GopTarget / N
• MinTarget [P or B frame] (Number of Macroblocks per frame) * 8
• the target quantizer scale is calculated from the target frame. bit rate and the complexity estimate, lower bounded to prevent incorrect complexity estimates from driving the quantizer scale too low.
• the quantizer scale is also upper bounded by the maximum MPEG nonlinear quantizer scale value.
• FrameQuant Max (MinFrameQuant, X [picture_type] / FrameTargetBits )
• MinFrameQuant is set to 2.
• FrameQuant Min (StartupQuant, (FrameQuant + Max (LastFrameQuant, LastAvgQuant) ) / 2 where LastFrameQuant is the target quantizer (FrameQuant) value of the most recently coded frame of the same picture type, and LastAvgQuant is the actual average quantizer (AvgQuant) value of the most recently coded frame of the same picture type.
• the target quantizer scale (FrameQuant) value calculated on the MCC is then clamped by a maximum value and delivered to the panel compressors to be used initially to code the panels. Details for setting the upper bound and the quantization strategy on the panel compressors are provided hereinafter.
• the quantizer scale values are adjusted periodically at RC sync interrupts by a feedback process to drive the actual number of bits generated in the picture towards the target .
• the algorithm makes sure that there is enough room in the buffer to accommodate the surge of bits to be generated by the upcoming I frame. Since the rate control task is carried out after the decision of picture coding type, the buffer is inspected to determine the feasibility of starting a new GOP at the next frame. If there is not enough space in the buffer, the rate control task sends a signal to prevent the next frame from starting a new GOP. The next frame is allowed to start a new GOP if: 302
• EncoderBufferLevel + FrameTargetBits - (PictureDuration * BitRate) is below a threshold which is precomputed as:
• the frame target quantizer scale (FrameQuant) should be used throughout the picture to produce a uniform quality over the entire picture.
• the rate control model based on the frame complexity estimate is not always accurate.
• adjustments to FrameQuant are necessary to control the bit rate.
• a feedback mechanism is used such that the accumulated bit count of the frame is compared to the target bit rate of the frame, scaled by the portion of macroblocks coded.
• the MCC reads two parameters from every panel compressor. These parameters are (i) the number of macroblocks currently encoded, and (ii) the number of bits generated so far. The MCC sums up these statistics for all eight panels (SumPanelBitCount and SumPanelMbCount ) . The MCC then computes a partial target and a partial maximum for the number of bits generated.
• PartialFrameTargetBits FrameTargetBits * SumPanelMbCount / (Number of macroblocks per frame) 02
• PartialFrameMaxBits FrameMaxBits * SumPanelMbCount / (Number of macroblocks per frame)
• the adjustment of the quantizer scale is based on the ratios of the SumPanelBitCount to PartialFrameTargetBits and to PartialFrameMaxBits. If the bit count (SumPanelBitCount) exceeds 0.9 of PartialFrameMaxBits, the quantizer scale is adjusted as :
• the final quantizer scale value of the panel would be close to the initial value which is FrameQuant. Such arrangement avoids visible seams at panel boundaries that can be caused by abrupt changes in the quantizer scale.
• the panel compressors perform quantization by multiplying the DCT coefficients with precomputed sets of reciprocal values of the quantizers. There are 32 302
• the first 31 tables correspond to the MPEG quantizer scale index of 1 to 31, which are mapped to the quantizer scale value of 1 to 112 according to the MPEG nonlinear quantizer mapping table.
• the 32 nd table is reserved for a panic mode in which all multipliers are set to zero, except for the intra DC, so that the DCT coefficients may be discarded to minimize the number of bits generated. This panic mode should be invoked only as an emergency measure to protect the buffers.
• the DC value of the non- intra panic multiplier table may be written by the MCC to determine whether to normally quantize the non-intra DC by the maximum MPEG quantizer scale (112), or set it to a fraction of the normally quantized value. If the panel compressor receives from the MCC a quantizer scale value (PanelQuant) between 1 to 112, the normal tables are selected. For any PanelQuant value above 112, the panic table will be selected.
• the quantization scale value calculated by the MCC (FrameQuant or CurrentQuant) spans a range of 1 to 512. This value is converted to an 8-bit value from 1 to 255 (PanelQuant) to be sent to the panel compressor to select the multiplier table, and is also used to determine the non-intra DC multiplier of the panel compressor' s panic table:
• PanicNonlntraDCMul 0.3 * MaxQuantNonlntraDCMul
• MaxQuantNonlntraDCMul is the non-intra DC entry of the multiplier table for the maximum MPEG quantizer scale (112).
• MaxRcQuant is a constant value current set to 144.
• the FrameQuant value should not be allowed to exceed MaxRcQuant above which the panel compressor may invoke panic mode that severely degrade the picture quality.
• the panel compressors should be allowed to go into panic mode at the onset of encoding the frame. Therefore, the following adaptive algorithm is used to set the upper bound of FrameQuant base on the value of FrameMaxBits. If FrameMaxBits is big enough to support the worst case number of bits generated by normal MPEG quantization (FrameMaxBits exceeds a thresholdPanicThresholdl) , FrameQuant is capped by MaxRcQuant to avoid going into panic mode.
• FrameMaxBits goes below PanicThresholdl, the upper bound is changed, e.g., to 176, which enables panic mode on the panel compressors but preserves the non-intra DC coefficient at panic. If the FrameMaxBits further drops below a second threshold PanicThreshold2, the upper bound is set to 512 so that the quantized value of the non-intra DC coefficient can be reduced beyond that of normal quantization.
• a hysteresis mechanism is provided so that
• the thresholds ⁇ PanicThresholdl, PanicThreshold2, PanicThreshold3 ⁇ can be set, for example, to the values ⁇ 750000, 500000, 1500000 ⁇ for P and B frames, and ⁇ 1500000, 1000000, 3000000 ⁇ for I frames.
• Each panel compressor receives the PanelQuant value from the MCC at every RC Sync and at picture start.
• the panel compressors then perform the following routine on a macroblock by macroblock basis:
• the packet processor retrieves the data from the primary video FIFO at an average data rate equal to the video bit rate
• bit__rate (bit__rate) as specified by the user. Whenever the FIFO level falls below 184 bytes, the packet processor will stop retrieving data from the FIFO to protect the FIFO from underflow. Null packets will then be substituted for video packets to maintain a constant transport bit rate.
• the present invention provides a rate control algorithm for optimizing the performance of a video encoder that utilizes multiple parallel compression engines.
• the invention has application to both HDTV and SDTV encoders .

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Computing Systems (AREA)
• Theoretical Computer Science (AREA)
• Compression Or Coding Systems Of Tv Signals (AREA)

Applications Claiming Priority (3)

Publications (1)

Family

ID=22291739

Family Applications (1)

Country Status (6)

Families Citing this family (22)

Citations (3)

Family Cites Families (5)

• 1999
  • 1999-09-29 EP EP99950029A patent/EP1118225A1/de not_active Withdrawn
  • 1999-09-29 CA CA002345538A patent/CA2345538A1/en not_active Abandoned
  • 1999-09-29 CN CN99813843A patent/CN1328748A/zh active Pending
  • 1999-09-29 WO PCT/US1999/022663 patent/WO2000021302A1/en not_active Application Discontinuation
  • 1999-09-29 KR KR1020017004165A patent/KR20010075530A/ko not_active IP Right Cessation
  • 1999-09-29 AU AU62771/99A patent/AU6277199A/en not_active Abandoned

Patent Citations (3)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events