JP2006121657A - Efficient rate control technique for video encoding - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a rate control technique that can improve video encoding. <P>SOLUTION: The rate control technique exploits a relationship between the number of bits encoded per frame and the number of non-zero coefficients of the video blocks after quantization. The number of non-zero coefficients of the video blocks after quantization is referred to as rho (ρ). The value of ρ is generally proportional to the number of bits used in the video encoding. This disclosure utilizes a relationship between ρ and a quantization parameter (QP) in order to achieve rate controlled video encoding. More specifically, this disclosure provides a technique for generating a lookup table (LUT) that maps values of ρ to different QPs. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

Related Patents This application claims the benefit of US Provisional Application No. 60 / 586,963, filed July 8, 2004.
The present invention relates to digital video processing, and more particularly to rate control coding of video sequences.

  Video functions including digital television, digital satellite broadcasting systems, wireless communication devices, personal digital assistants (PDAs), laptop computers, desktop computers, digital cameras, digital recording devices, mobile or satellite radiotelephones, etc. Is built into a wide range of devices. Digital video devices can make significant improvements over conventional analog video systems in the creation, modification, transmission, storage, recording and display of full motion video sequences.

A number of different video encoding standards have been developed for encoding digital video sequences. For example, the Motion Picture Expert Committee (MPEG) has created several standards including MPEG-1, MPEG-2 and MPEG-4. Other standards include the International Telecommunication Union (ITU) H.264. 263 standard, QuickTime ^TM technology that was created by the California Kupachino City of Apple Computer, Video for Windows (registered ^trademark) that was created by Microsoft Corporation of Redmond, Washington ^TM, Indeo ^TM, Washington State that was created by Intel Corporation There is RealVideo ^{™ from} RealNetworks in Seattle, and Cinepak ^™ created by Supermac. Furthermore, ITU H. New standards have emerged and continue to evolve, including H.264 and some proprietary standards.

  Many video coding standards allow for improved transmission rates of video sequences by encoding data in a compressed manner. Compression can reduce the overall amount of data required for efficient transmission of video frames. For example, most coding standards utilize graphics and video compression techniques designed to facilitate video and image transmission with a narrower bandwidth than can be achieved without compression.

  MPEG standards and ITU H.264 263 and ITU H.264. The H.264 standard, for example, corresponds to a video coding technique that uses similarity between successive video frames, called temporal or inter-frame correlation, to perform inter-frame compression. Interframe compression techniques take advantage of data redundancy across frames by converting a pixel representation of a video frame into a motion representation. In addition, some video coding techniques make use of intraframe similarity called spatial or intra-frame correlation to further compress video frames.

  To perform compression, digital video devices typically include an encoder that compresses the digital video sequence and a decoder that decompresses the digital video sequence. In many cases, the encoder and decoder form an integrated encoder / decoder (codec) that operates on blocks of pixels within a frame that defines a video sequence. In MPEG, for example, an encoder divides a transmitted video frame into video blocks called “macroblocks”. ITU H. The H.264 standard is 16x16 video block, 16x8 video block, 8x16 video block, 8x8 video block, 8x4 video block, 4x8 video block, 4x4 video block Corresponds to the block. Other standards support different sized video blocks.

  For each video block in the video frame, the encoder calls one of the one or more previous video frames (or subsequent frames) to identify the most similar video block, called “best prediction”. Search for video blocks of similar size. The process of comparing the current video block with the video blocks of other frames is commonly referred to as motion estimation. Once the “best prediction” is identified for the video block, the encoder encodes the difference between the current video block and the best prediction. This process of encoding the difference between the current video block and the best prediction includes a process called motion compensation. Motion compensation involves creating a difference block that represents the difference between the current video block to be encoded and the best prediction. In particular, motion compensation usually refers to the process of taking the best prediction block using a motion vector and subtracting the best prediction from the input block to generate a difference block.

  After motion compensation has created the difference block, a series of additional encoding procedures are typically performed to encode the difference block. These additional encoding procedures depend on the encoding standard used. In an MPEG-4 compliant encoder, for example, the additional encoding procedure includes an 8 × 8 discrete cosine transform followed by scalar quantization, raster zigzag reordering, run length encoding, and Huffman encoding. The encoded difference block is transmitted with a motion vector indicating which video block of the previous frame was used for encoding. The decoder receives the motion vector and the encoded difference block and decodes the received information to recover the video sequence.

  Several rate control techniques have been developed for video coding. Rate control techniques are particularly important for facilitating real-time transmission of video sequences, but are also used in non-real-time coding settings. For rate control, encoding techniques dynamically adjust the number of bits encoded per frame. In particular, rate control limits the number of bits encoded per frame to ensure that the video sequence is efficiently encoded and transmitted at a rate with the allocated bandwidth. If the coding technique does not support scene changes in the video sequence, the bit rate for real-time transmission of the video sequence will vary significantly as the scene changes. Thus, the number of bits per frame is dynamically adjusted during encoding to define a substantially constant bit rate.

  One way to achieve rate control coding is to allow adjustment of the quantization parameter (QP) during the video coding process. QP directly affects the number of bits encoded per second. As QP increases, less data is maintained and the quality of video coding degrades. As QP decreases, more data is maintained and the quality of video encoding is improved. However, if the QP is too small, the number of coded bits per second will exceed the allocated bandwidth, reducing the ability to transfer frames within a limited amount of bandwidth. By selecting QP in a dynamic manner, the bit rate of transmission of video frames can be made substantially constant.

  The present invention describes a rate control technique that can improve video coding. In particular, the rate control technique described takes advantage of the relationship between the number of bits encoded per frame and the number of non-zero coefficients of the video block after quantization. The number of non-zero coefficients of the video block of the frame after quantization is called low (ρ). The value of ρ is generally proportional to the number of bits used in the video encoding process. The present invention utilizes the relationship between ρ and the quantization parameter (QP) to achieve rate controlled video coding. In particular, the present invention provides a technique for generating a lookup table (LUT) that maps the value of ρ to different QPs. QP can then be selected to achieve the desired coding rate, which is primarily related to ρ. The described technique simplifies the video encoder and can significantly reduce the number of computations required to generate the LUT used for rate-controlled video encoding.

  The present invention also describes a video encoding device that implements a rate control technique as shown herein. In one embodiment, the video encoding device calculates a threshold for unquantized coefficients of the video block (where the quantization coefficient is non-zero for different quantization parameters) and uses the threshold An encoder is provided that maps the number of non-zero coefficients (ρ) after quantization to a ρ-QP lookup table (LUT). The video encoding device includes a memory that stores the ρ-QP LUT. In a more specific embodiment, the encoder generates a threshold-QP LUT that maps the threshold to QP and uses the threshold-QP LUT to generate a ρ-QP LUT. In any case, the described technique can significantly simplify the generation of a ρ-QP LUT by utilizing a threshold that identifies when the quantization factor is non-zero for different quantization parameters.

  These and other techniques described herein are implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the software is executed in a digital signal processor. In that case, software that implements the technique is initially stored on a computer-readable medium, and is installed and executed on a DSP for efficient rate control coding in a digital video device. Further details of various embodiments are set forth in the accompanying drawings and description below. Other features, objects, and advantages will be apparent from the description and drawings, and from the claims.

  The present invention describes a rate control technique that can be used to improve video coding. The technology is MPEG-4 standard, ITU H.264. H.263 standard, ITU H.264 It is used with any of a wide variety of video coding standards, such as the H.264 standard or other standards. In particular, the technique takes advantage of the relationship between the number of bits encoded per frame (coding rate) and the number of non-zero coefficients of the video block of the quantized frame. The number of non-zero coefficients of the video block of the frame after quantization is called low (ρ). The value of ρ is generally proportional to the quantization parameter (QP) used for video coding. Thus, the present invention utilizes the relationship between ρ and QP to achieve rate controlled video coding. More specifically, the present invention provides a technique for generating a lookup table (LUT) that maps the value of ρ to different QPs. The described technique simplifies the video encoder and can significantly reduce the number of calculations required to generate the LUT used for rate-controlled video coding in the ρ-domain. The video encoder uses the LUT to select a QP that ensures that the desired coding rate is achieved.

  FIG. 1 is a block diagram illustrating an example system 10 in which a calling device 12 transmits a video data encoding sequence to a receiving device 14 via a communication line 15. Both the sending device 12 and the receiving device 14 are digital video devices. In particular, the transmitting device 12 is based on the MPEG-4 standard, ITU H.264 H.263 standard, ITU H.264 Encode video data that conforms to the video standard, such as the H.264 standard, or any of a wide variety of standards available for rate-controlled video encoding. One or both devices 12, 14 of the system 10 implement rate control techniques as described in more detail below to improve the video encoding process. Such rate control techniques are particularly useful for real-time transmission of video sequences over limited bandwidth communication lines 15, such as wireless communication lines.

  The communication line 15 is a wireless line, a physical transmission line, fiber optics, a packet network such as a local area network (LAN), a wide area network (WAN), a global network such as the Internet, or the public. Includes the switched telephone network (PSTN) or any other communication line that can transfer data. Thus, communication line 15 represents any suitable communication medium for sending video data from originating device 12 to receiving device 14, or perhaps a collection of different networks. As described above, the communication line 15 has performed very important rate control on the line 15 for real-time transmission of video sequences and has limited bandwidth.

  The originating device 12 includes any digital video device capable of encoding and transmitting video data. The originating device 12 includes a video memory 16 that stores the digital video sequence, a video encoder 18 that encodes the sequence, and a transmitter 20 that transmits the encoded sequence to the receiving device 14 over the communication line 15. Including. As described herein, video encoder 18 may include, for example, one or more digital signal processors (DSPs) that execute various hardware, software or firmware, or programmable software modules that control video encoding techniques. )including. Associated memory and logic circuitry is provided to assist the DSP in controlling the video encoding technique.

  The originating device 12 also includes a video capture device 23, such as a video camera, for acquiring the video sequence and storing the acquired sequence in the memory 16. In particular, the video acquisition device 23 can acquire charge coupled devices (CCD), charge injection devices, photodiode arrays, complementary metal oxide semiconductor (CMOS) devices, or video images or digital video sequences. Includes photosensitive devices.

  As a further example, video acquisition device 23 may convert video from analog video data to digital video data, eg, from a television, video cassette recorder, video camera (camcorder), or another video device. A video converter. In some embodiments, originating device 12 is configured to transmit a real-time video sequence over communication line 15. In such cases, receiving device 14 receives the real-time video sequence and displays the video sequence to the user. Instead, the originating device 12 obtains and encodes a video sequence that is sent to the receiving device 14 as a video data file, ie, not in real time. Thus, originating device 12 and receiving device 14 correspond to applications such as video telecommunications, video clip playback, video mail, or video conferencing, for example, in a mobile wireless network. Devices 12 and 14 include various other elements not specifically illustrated in FIG.

  The receiving device 14 takes the form of any digital video device capable of receiving and decoding video data. For example, the receiving device 14 includes a receiver 22 that receives an encoded digital video sequence from the transmitter 20 via, for example, intermediate lines, routers, other network equipment, and the like. The receiving device 14 also includes a video decoder 24 that decodes the sequence and a display device 26 that displays the sequence to the user. In some embodiments, however, the receiving device 14 does not include the integrated display device 26. In such cases, receiving device 14 serves as a receiver that decodes the received video data to drive a separate display device, eg, a television or monitor.

  Examples of devices for sending device 12 and receiving device 14 include a computer network, a workstation or other desktop computing device, and a server located in a portable computing device such as a laptop computer or personal digital assistant (PDA). Other examples include digital television broadcast satellite and receiving devices such as digital television, digital cameras, digital video cameras or other digital recording devices, digital video phones such as mobile phones with video capabilities, There are direct dual communication devices with video capabilities, other wireless video devices, and so on.

  In some cases, originating device 12 and receiving device 14 each include an encoder / decoder (codec) (not shown) for encoding and decoding digital video data. In particular, both the sending device 12 and the receiving device 14 include a transmitter and a receiver as well as a memory and a display. Many of the encoding techniques outlined below are described in the context of a digital video device that includes an encoder. However, it is natural that the encoder forms part of the codec. In that case, the codec may be hardware, software, firmware, DSP, microprocessor, application specific integrated circuit (ASIC), field programmable gate array (FPGA), individual hardware components, or various combinations thereof. Implemented in

  Video encoder 18 within originating device 12 operates on blocks of pixels in a series of video frames to encode video data. For example, video encoder 18 performs motion estimation and motion compensation techniques, in which a video frame transmitted therein is divided into blocks of pixels (referred to as video blocks). Video blocks include blocks of any size and vary within a video sequence for illustrative purposes. As an example, ITU H. The H.264 standard is 16x16 video block, 16x8 video block, 8x16 video block, 8x8 video block, 8x4 video block, 4x8 video block, 4x4 video block Corresponds to the block. The use of smaller video blocks in video encoding can achieve better resolution in encoding, especially for video frame locations that contain high levels of detail. In addition, video encoder 18 is designed to operate on 4 × 4 video blocks and reconstruct larger video blocks from 4 × 4 video blocks as needed.

  Each pixel in the video block is represented by various n-bit values, eg, 8 bits, which define the color and visual characteristics of the pixel such as the intensity of chromaticity and lightness values. Each pixel has one or more 8-bit values for both chromaticity and lightness. The principles of the present invention, however, are not limited to pixel formats, but extend to use with simpler minority bit pixel formats or more complex multibit pixel formats. Pixels are also defined according to other color coordinate systems.

  For each video block in the video frame, the video encoder 18 of the originating device 12 sends a previous video frame (or subsequent video frame) that has already been transmitted to identify a similar video block called the predictive video block. Motion estimation is performed by searching for video blocks stored in memory 16 for (frame). In some cases, the predicted video block includes a “best prediction” from the previous or subsequent video frame, but the invention is not limited in that respect. Video encoder 18 performs motion compensation to create a difference block that represents the difference between the current video block to be encoded and the best prediction. Motion compensation usually refers to the process of taking the best predictive video block that uses motion vectors and then subtracting the best prediction from the input block to generate a difference block.

  After the motion compensation process creates the difference block, a series of additional encoding procedures are typically performed to encode the difference block. These additional encoding procedures depend on the encoding standard used. In an MPEG-4 compliant encoder, for example, the additional encoding procedure includes an 8x8 discrete cosine transform, followed by scalar encoding, followed by raster zigzag reordering, followed by run length encoding, and Huffman encoding. Then follow.

  Once encoded, the encoded difference block is transmitted with a motion vector that identifies the video block from the previous frame (or subsequent frame) used for encoding. In this method, instead of encoding each frame as an independent image, video encoder 18 encodes the difference between adjacent frames. Such techniques can significantly reduce the amount of data that needed to accurately represent each frame of the video sequence.

  The motion vector defines the pixel location with respect to the upper left hand corner of the encoded video block, but other formats can be used for the motion vector. In any case, by encoding video blocks using motion vectors, the bandwidth required for transmission of a stream of video data can be significantly reduced.

  In some cases, video encoder 18 may support intraframe coding in addition to interframe coding. Intraframe coding utilizes intraframe similarity, called spatial or intraframe correlation, to further compress video frames. Intraframe coding is based on texture coding for compressing still images, such as discrete cosine transform (DCT) coding. Intraframe coding is often used with interframe compression, but is also used as an alternative in some implementations.

  The receiver 22 of the receiving device 14 receives encoded video data in the form of motion vectors and an encoded difference block that indicates the encoded difference between the encoded video block and the best prediction used in motion estimation. Receive. In some cases, however, the difference between the motion vector and the previously calculated motion vector predictor (MVP) is transmitted instead of the motion vector. In any case, the decoder 24 can perform video decoding to generate a video sequence to the user via the display device 26 for display. The decoder 24 of the receiving device 14 is also implemented as an encoder / decoder (codec). In that case, both the sending device 12 and the receiving device 14 can encode, transmit, receive and decode the digital video sequence.

  In accordance with the present invention, video encoder 18 selects a quantization parameter (QP) in a dynamic manner to achieve efficient rate control coding. The video encoder generates a look-up table (LUT) based on the video block or video frame to map the value of rho (ρ) to the quantization parameter (QP) used for video encoding. The quantity ρ represents the number of non-zero coefficients of the video block after quantization and is generally proportional to the number of bits used for video coding. Thus, video encoder 18 utilizes the relationship between ρ and QP to achieve rate controlled video encoding. More specifically, video encoder 18 implements a technique for generating a look-up table (LUT) that maps the values of ρ to different QPs. Since ρ is roughly linearly related to the rate, the ρ-QP LUT can be used to select a QP that yields efficient rate-controlled video coding. The ρ-QP LUT is also labeled here as ρ (QP).

  FIG. 2 is an exemplary block diagram of device 30, which corresponds to originating device 12. In general, device 30 includes a digital video device capable of performing rate control coding techniques as shown herein. The device 30 is MPEG-4, ITU H.264, or the like. 263, ITU H.264. H.264, conforms to video coding standards such as another video coding standard corresponding to motion estimation and motion compensation techniques for inter-frame video coding.

  As shown in FIG. 2, device 30 includes an apparatus 32 for encoding a video sequence and a video memory 34 for storing the video sequence before and after encoding. A device 30 transmits a coded sequence to another device, and a video acquisition device 38, such as a video camera, that acquires a video sequence and possibly stores the acquired sequence in a memory 34. including. Various elements of the device 30 are communicatively connected by a communication bus 35. Various other elements, such as intra-frame encoder elements, various filters, or other elements, are also included in device 30, but are not specifically illustrated for simplicity.

  Video memory 34 typically includes a relatively large memory space. Video memory 34 includes, for example, dynamic random access memory (DRAM) or flash (FLASH) memory. In other examples, video memory 34 includes non-volatile memory or other data storage devices.

  Video encoding device 32 includes a chip set for a portable radiotelephone, including a combination of hardware, software, firmware, and / or a processor or digital signal processor (DSP). Video encoder 32 generally includes an encoder 28 connected to a local memory 37. Local memory 37 is smaller and contains faster memory space compared to video memory 34. As an example, the local memory 37 includes a synchronous random access memory (SRAM). Local memory 37 includes "on-chip" memory integrated with the other elements of video encoder 32 for very fast access to data during the processor intensive encoding process. During the encoding of a given video frame, the current video block to be encoded is loaded from video memory 34 into local memory 37. The search space used in setting the best prediction is also loaded from the video memory 34 to the local memory 37.

  The search space includes a subset of pixels of one or more previous video frames (or subsequent frames). The selected subset is pre-specified as the maximum likelihood location for identification of the best prediction that closely matches the current video block to be encoded. In addition, if different search stages are used, the search space changes on coarse motion estimation. In that case, the search space becomes progressively smaller with respect to the size of the search space with respect to these subsequent searches performed at a higher resolution than the previous search.

  The local memory 37 carries a search space containing the current video block to be encoded and some or all of one or more video frames used in interframe encoding. Motion estimator 40 compares the current video block with various video blocks in the search space to identify the best prediction. In some instances, a proper match for coding is identified earlier without specifically examining all possible candidates, in which case a proper match may be appropriate for efficient video coding. May not actually be the "best" prediction. In general, the phrase “predictive video block” refers to an appropriate match, which is the best prediction.

  Motion estimator 40 performs a comparison between the current video block to be encoded and the candidate video blocks in the search space of memory 37. In some cases, the candidate video block includes non-integer pixel values generated for fractional interpolation. As an example, motion estimator 40 may use a sum of absolute difference (SAD) technique, a sum of squared difference (SSD) technique, or a comparison technique to define difference values for candidate video blocks. Execute. A lower difference value generally indicates that the candidate video block is a better match, and thus better candidates than other candidate video blocks used in motion estimation coding yield a higher difference value.

  Eventually, the motion estimator identifies the “best prediction”, which is the candidate video block that most closely matches the video block being encoded. However, in many cases, it may be natural that an appropriate match is located before the best prediction, in which case an appropriate match is used for encoding. Furthermore, the predictive video block references an appropriate match, which is the best prediction.

  Once the best prediction is identified for the video block by motion estimator 40, motion compensator 42 creates a difference block that indicates the difference between the current video block and the best prediction. The difference block encoder 44 further encodes the difference block to compress the difference block, and to identify which candidate video block was used for encoding from the search space, the encoded difference block is It is sent to another device along with the motion vector (or the difference between the motion vector and the motion vector estimator). For simplicity, the specific component components will vary depending on the corresponding specific standard, so the additional component components used to perform the encoding after motion compensation are the difference block encoder 44. Generalized as: In other words, the difference block encoder 44 performs one or more conventional encoding techniques on the difference block, which is generated as shown here.

  The encoding process, including motion estimation and motion compensation, is very computationally intensive. However, the number of calculations required to perform rate control can be limited. To perform such rate control, video encoder 28 includes a rate control unit 46. Rate control unit 46 utilizes the relationship between the number of bits encoded per frame and the number of non-zero coefficients of the quantized video block. The value of ρ representing the number of non-zero coefficients of the video block after quantization is the number of bits encoded per frame (and thus the number of bits encoded per second at a constant frame rate). Generally proportional. Thus, rate control unit 46 utilizes the relationship between ρ and QP to achieve rate controlled video coding. More specifically, rate control unit 46 generates one or more LUTs to map the value of ρ to different QPs. The QP can then select for the desired value of ρ corresponding to the desired rate for video coding.

  As described in further detail below, the rate control unit 46 of the video encoder 28 calculates a threshold for the unquantized coefficients of the video block, where the threshold is for quantization parameters (QP) with different quantization coefficients. Identify when non-zero. The rate control unit 46 then uses the threshold to generate a ρ-QP LUT that maps the number of non-zero coefficients after quantization to QP. The local memory 37 stores the ρ-QP LUT. In a more specific embodiment, rate control unit 46 of video encoder 28 generates a threshold-QP LUT that maps the threshold to QP and uses the threshold-QP LUT to generate a ρ-QP LUT. LUT 48 generally represents one or more LUTs, such as a ρ-QP LUT and possibly a threshold-QP LUT used to generate a ρ-QP LUT, as outlined herein.

  In generating the ρ-QP LUT, the video encoder 28 selects a QP from the ρ-QP LUT for rate-controlled video encoding of the video block, and the video encoder 28 selects a video signal based on the selected QP. The block coefficients are quantized and the video block is encoded, for example, according to an inter-frame video encoding technique as outlined above.

  The ρ-QP LUT can be created on a video block basis or a video frame basis. In the former case, video encoder 28 generates different ρ-QP LUTs for different video blocks of the video frame and uses the corresponding ρ-QP LUT for a given video block. QP for rate controlled video coding is selected. In the latter case, video encoder 28 generates different ρ-QP LUTs for different video frames of the video sequence and uses the corresponding ρ-QP LUT for a given video frame. Select a QP for rate-controlled video coding of the video blocks.

  For video coding, it is generally desirable to have an accurate rate-distortion model. An accurate rate-distortion model can result in a rate control approach that not only achieves the target bit rate but also reduces the variation in the number of bits per frame. Rate control techniques that use the number of non-zero coefficients (ρ) to predict bit rate use many other rate control algorithms, particularly a fixed quadratic rate-distortion model Achieves significantly better performance than the algorithm. A conventional algorithm that uses a fixed second-order rate-distortion model is referred to herein as a conventional on-line baseline approach or a baseline approach.

Basically, ρ-region rate control is based on two simple principles:
1. ρ is linearly related to the number of bits used, and 2. The QP value that yields a particular ρ value can be estimated from the unquantized transform coefficients.
If R is the number of bits required for use on the frame, the number of non-zero coefficients that occur after quantization is given by:
R = Aρ + B Formula 1
In Equation 1 above, parameter B can be considered non-texture information, and parameter A can be considered the average number of bits required to encode the non-zero coefficients. Parameters A and B can be estimated from the values of R and ρ from the previous frame. From Equation 1 above, given R, it is fairly simple to estimate ρ. FIG. 3 is a plot of ρ for the number of bits used per frame and the different frames. As can be seen from FIG. 3, the relationship between R and ρ is approximately linear.

Thus, the rate control technique can select a QP value that yields the desired value of ρ. In order to select such a QP value, however, it is necessary to know the number of non-zero coefficients that will be derived from each acceptable QP corresponding to the standard used. If the transform coefficient is given by T _i and the quantization procedure corresponding to _QP is given by S _QP , then the ρ-QP table is:

によって与えられ、ここで
ｆ(Ｔ_ｉ，Ｓ_ＱＰ)＝１ （｜Ｔ_ｉ＋ｒ_ｑｐ｜≧Ｓ_ＱＰであれば）
＝０ （それ以外）
上記の式２では、ステップ・サイズＳ_ＱＰ及び丸め因数（rounding factor）ｒ_ｑｐは次の要件に依存する：
１．符号化規格、例えば、ＩＴＵ Ｈ．２６４、ＭＰＥＧ‐４またはＩＴＵ Ｈ．２６３、
２．フレーム内符号化またはフレーム間符号化のいずれかが実行されている、及び
３．係数指標（coefficient index）。

Where f (T _i , S _QP ) = 1 (if | T _i + r _qp | ≧ S _QP )
= 0 (other than that)
In Equation 2 above, the step size S _QP and the rounding factor r _qp depend on the following requirements:
1. Coding standards such as ITU H.264. H.264, MPEG-4 or ITU H.264. 263,
2. 2. either intra-frame coding or inter-frame coding is being performed, and Coefficient index.

For example, in MPEG-4, the mediating macroblock for all coefficients S _QP is given by 2QP, and r _QP is given by-(QP / 2). Equation 2 used to generate the ρ-QP table can be implemented by adders and comparators. The table entry ρ (QP) needs to be calculated for all values of QP. QP ranges from 1 to 31 for MPEG-4; H.264 ranges from 0 to 51. The calculation generally needs to be repeated for all possible values of QP. If a particular quantization parameter ρ is zero, ρ is guaranteed to remain zero for all larger quantization parameter values and therefore need not be calculated. In other words, if ρ (QP _i ) = 0, ρ (QP) = 0 for all QP> QP _i .

  The ρ-QP table obtained from the macroblock can be accumulated over the entire frame to obtain a ρ-QP table for the frame. A “macroblock” generally refers to a 16 × 16 pixel video block. Assuming that the video sequence image is fairly stationary, the ρ-QP table estimated from the previous frame can be used to predict the QP for the current frame. Further, the initial value of QP can be changed on a macroblock basis using macroblock level rate control.

The following procedure is used to implement a basic ρ-region rate control algorithm.
Procedure 1: Estimate the bit amount R _f of the current frame.
Procedure 2: Estimate the corresponding ρ _f using Equation 1 above. For the first frame, the initial values for A and B are used.
Procedures 3: Use ρ _f a (QP), to select the QP = QP _f that gives a ρ closest to ρ _f.
Procedure 4 (a): Macroblock level rate control initialization: ρ _a = 0; i = 0, where N is the number of macroblocks per frame and i is the macroblock index.
Procedure 4 (b): ρ _m = (ρ _f −ρ _a ) / (N−i). Use [rho _m of (QP), choose a QP = QP _m giving [rho closest to [rho _m. ρ _m (QP) is a scaled version of ρ _f (QP) or can be estimated from previous macroblocks. QP _m is clamped so that the fluctuation is within [−2 +2] from the previous macroblock (in the case of MPEG4).
Procedure 4 (c): For each QP, calculate ρ _i (QP) using Equation 2 above.
Procedure 4 (d): For each QP, replace ρ (QP) with ρ (QP) + ρ _i (QP).
Procedure 4 (e): Replace ρ _a (QP) with ρ _a (QP) + ρ _i (QP). Replace i with i + 1.
Procedure 4 (f): Returns to procedure 4 (b) until (i <N) (loop).
Procedure 5: Replace ρ _f (QP) with ρ (QP). Re-estimate A and B using ρ _a and the texture bits (R _t ) and non-texture bits (R _n ) spent in the current frame. B = R _n and A = (R _t / ρ _a ).
Procedure 6: Return to procedure (1) by the end of all frames.

  MPEG-4 and ITU H.264 For 263 compliant encoders, the calculation of the ρ-QP table is ITU H.264. This can be done in a simpler way than for an H.264 compliant encoder. The following pseudo code is MPEG-4 and ITU H.264. One such calculation for a video block in accordance with the H.263 standard is illustrated.

For QP = 1to 31
{
p (QP) = 0;
If (INTRA)
{
if (| (DCT (0) + (dc_scalar >> 1) |> = dc_scalar)
p (QP) ++;
for i = 1 to 63
if (| DCT (i) |> = (QP << 1)}
p (QP} ++;
end
}
else
{
for i = 0 to 63
if (| (| DCT (i) |-(QP >> 1)) |> = (QP << 1))
p (QP) ++;
end
}
if (p (QP)) = 0)
break;
}
4 shows MPEG-4 or ITU H.264. 3 illustrates an exemplary hardware circuit that can be used to calculate a ρ (QP) table for an encoder that conforms to the H.263 coding standard. The accumulation shown in FIG. 4 will need to be repeated for all luminosity (luma) and chromaticity (chroma) blocks, eg, 4 brightness and 2 chromaticity blocks.

  As illustrated in FIG. 4, circuit 400 receives DCT (i) as an input. DCT (i) is a discrete cosine transform coefficient having an index (i). Block 401 generates the absolute value of DCT (i), which is sent to adder 402. The absolute value of DCT (i) is then added to the output of multiplexer 403. Multiplexer 403 selects one of three values. The inputs to the multiplexer 403 are the dc_scaler value shifted to the right by 1 and the negative quantization parameter QP shifted to the right by 1. Multiplexer 403 selects the dc_scaler value shifted to the right by 1 when both I and DC are 1, selects 0 when I is 1 and DC is 0, and 1 when I is 0 regardless of the value of DC. Only the negative quantization parameter QP shifted to the right is selected. The value of I is 1 when the input block is an intra block and 0 when the input block is an inter block. The value of DC is 1 when the input parameter is DC, and 0 when the input parameter is AC. The DC coefficient represents the average value of the video block, while the AC coefficient is the remaining coefficient of the video block.

  The output of multiplexer 403 is added to the output of block 401 by adder 402. Block 404 then generates the absolute value of the output of adder 402 and provides the negative value of this value as an input to adder 405, which is added to the output of multiplexer 406. Multiplexer 406 receives the dc_scaler value and quantization parameter QP shifted to the left by one. The multiplexer 406 selects the dc_scaler value as an output when both I and DC are 1, and in other cases selects the quantization parameter QP shifted to the left by 1 as an output.

  Adder 405 subtracts the output of block 404 from the output of multiplexer 406. Block 407 examines the twelfth bit (positive / negative sign bit) of the output of adder 405 and provides this bit to adder 408. Block 409 accumulates the ρ (QP) table by adding positive and negative sign bits to each previous input. Thus, the adder 408 and the ρ (QP) table can be viewed collectively as an accumulator that generates the ρ (QP) table. Circuit 400 repeats these calculations from I = 1 to 63 and QP = 1 to 31.

ITU H. In an H.264 compliant encoder, the quantization step is combined with a coefficient dependent scaling of the transform. This further complicates the calculation of the ρ (QP) table. The quantization can be performed using the following equation:
F _ij = (MW _ij + A) >> S Equation 3
Here, F _ij is a quantization coefficient, and W _ij is a transform coefficient that is not quantized. The multiplication factor M, the addition factor A, and the transition factor S depend on the QP, the coefficient indices i and j, the intra prediction mode, and on whether the block is lightness or chromaticity. From Equation 3, the quantized value F _ij is:
W _ij > = (2 ^S −A) / M Equation 4
> = C
Will only be non-zero.
Where S = 16 + (QP / 6) (Intra 16 × 16 DC and chromaticity)
S = 15 + (QP / 6) (Others)
A = 2 ^S-1 (Intra)
A = 2 ^S-2 (Inter)
Quantization can be designed to double the quantization step for an increase in QP of 6. Thus, M has a set of 6 unique values, and M has a range of values as a function of (QP% 6) and the indices i, j. The symbol% represents the MOD function, which provides the remainder. Thus, QP% 6 is the same as QP MOD 6, which divides QP by 6 and gives the remainder of the division. The dependence of M on the exponent is due to the fact that the integer transformation absorbs the necessary scaling by the integer transformation. The following Table 1 shows the value of M. In particular, Table 1 lists multiplier values (also called multiplication factors) M for various indices and QP% 6.

式４から、量子化されない値 Ｗ_ij がＣより大きいか等しければ、結果は非ゼロ係数であることに気付くであろう。式４の右辺は分数であるから、Ｗ_ij が比較される整数値は：

From Equation 4, if the unquantized value W _ij is greater than or equal to C, you will notice that the result is a non-zero coefficient. Since the right side of Equation 4 is a fraction, the integer value to which W _ij is compared is:

によって与えられる。次の表２はイントラ及びインター・マクロブロックについて様々な指数及びＱＰ％６に関する比較器値Ｃの表である。

Given by. The following Table 2 is a table of comparator values C for various indices and QP% 6 for intra and inter macroblocks.

特に、表２は様々なＱＰ％６についてＳ＝２４の最大移行値に関する比較器値Ｃを与える。これらの値は３６×１３ＲＯＭ中に縦列ごとに左から右へ記憶することができる。図５Ａ及び５ＢはＩＴＵ Ｈ．２６４規格に従ってρ（ＱＰ）表を計算するために使用できる典型的な回路構成を例示する回路図である。

In particular, Table 2 gives the comparator value C for the maximum transition value of S = 24 for various QP% 6. These values can be stored from left to right for each column in a 36 × 13 ROM. 5A and 5B show ITU H.264. 2 is a circuit diagram illustrating an exemplary circuit configuration that can be used to calculate a ρ (QP) table according to the H.264 standard. FIG.

  As illustrated in FIG. 5A, the value 15 is added to the value QP / 6 by the adder 501. The output of adder 502 is provided to multiplexer 505 in the same manner as the output of adder 501, which adds 1 to the output of adder 501. The output of the multiplexer 505 is selected based on control signals from the AND gate 503 and the OR gate 504. Inputs i and j to the AND gate 503 are exponent values of input coefficients and are inverted. The input to OR gate 504 indicates whether the current block is an intra block and whether the current block is a chromaticity block. When both inputs to the multiplexer 505 are 1, the output of the adder 502 is selected by the multiplexer, otherwise the output of the adder 501 is selected. The output S of circuit 500 is used by circuit 600 of FIG. 5B.

  As shown in FIG. 5B, the multiplexer 603 selects one of the values 0, 1 and 2 based on the input signals from the AND gate 601 and the NOR gate 602. The inputs to gates 601 and 602 are the zeroth bit of the index coefficient in i and j dimensions, respectively. If the output of gate 601 is zero and the output of gate 602 is 1, the value 0 is selected by multiplexer 603. If the output of gate 601 is 1 and the output of gate 602 is zero, the value 1 is selected by multiplexer 603. If the output of gate 601 is zero and the output of gate 602 is zero, the value 2 is selected by multiplexer 603.

  The multiplexer 604 multiplies the output of the multiplexer 603 by 6 and inputs this value to the memories 605 and 606. The IROM 605 corresponds to an intra block, and the PROM 606 corresponds to an inter block. When the inter block is being processed, the input value P to multiplexer 607 is one.

The multiplexer 607 selects one of the memories 605 and 606 based on the input P. The output of multiplexer 607 is provided to adder 608. The value S from the circuit of FIG. The value 511 is shifted right by the result of (15-S) by block 609 and the output of block 609 is provided to adder 608. Adder 608 sums the outputs of block 609 and multiplexer 607. The output of adder 608 is then shifted to the right by 24-S, which is represented by variable C, representing the comparator value, and value C is provided to adder 611. The value W _ij represents the input coefficient. Block 612 generates the absolute value of value W _ij , which is subtracted from value C by adder 611.

  Block 613 generates the sign bit (15th bit) of the output of adder 611. The sign bit is then provided to adder 614. Block 615 accumulates in the ρ (QP) table by adding a sign bit to each previous input. Thus, adder 614 and ρ (QP) table block 615 can be viewed collectively as an accumulator that generates a ρ (QP) table. Circuits 500 and 600 repeat these calculations from i, j = 1 to 3, and QP = 1 to 51.

  As more generally shown in FIGS. 5A and 5B, the rounding factor is added to the stored comparator value before it is transferred. This factor can be chosen so that the fractional value resulting from the division is always rounded up. Doing so ensures that the resulting C for all values of S between 15 and 24 is rounded up in Equation 5. In addition, many of the circuit configurations illustrated in FIGS. Already placed in a hardware element that performs forward quantization according to the H.264 standard. The accumulation shown in FIGS. 5A and 5B is repeated for all lightness and chromaticity blocks (16 lightness and 8 chromaticity blocks).

  The direct calculation of the ρ-QP table described above and illustrated in FIGS. 4 and 5A and 5B is computationally complex. In the worst case, the direct calculation involves one comparison and one addition per coefficient per QP. If QP has M possible values and there are N coefficients, the number of operations required for direct calculation of the ρ-QP table is M * N. More simplified approaches, including various “threshold” approaches, are described in further detail below.

In the “threshold” approach, the ρ-QP table includes the calculation of the QP threshold τ _i for all unquantized coefficients C _i . The threshold τ _i is a quantization parameter such as:
Q (C _i , QP) = 0 for all QP> τ _i
For all QP ≦ τ _i , Q (C _i , QP) ≠ 0 Equation 6
The function Q (C _i , QP) represents the quantization operation of the coefficient C _i using the quantization parameter QP. The threshold-QP table T (QP) can be calculated using the following pseudo code:
for QP = 1 to 31
T (QP) = 0
End
for 1 = 1 to N
T (τ _i ) ++;
End
The ρ-QP table can then be easily calculated from T (QP) using the following pseudo code:
for QP = QP _MAX −1 to QP _MIN
T (QP) = T (QP + 1) + T (QP)
End
ρ (QP) = T (QP)
Using the threshold method (also called the threshold method), the number of operations required to calculate the ρ-QP table can be reduced to 2N + M compared to the N * M calculation process required for direct calculation. This generally involves N operations for searching the table to calculate for each coefficient, and N operations for accumulating the threshold of N coefficients, followed by a T (QP) table to obtain a ρ-QP table. Suppose we need M operations to accumulate.

  If the calculation of the ρ-QP table is performed for all macroblocks, the threshold method reduces the calculation process by 90% or more. Furthermore, if the table is only calculated once per frame, the reduction in calculation processing will still be significant. The threshold method of calculating the ρ-QP table is also very advantageous for hardware implementations because it avoids the need to have M accumulators operating in parallel. If implemented in hardware, the hardware can compute a T (QP) table for each macroblock and pass it to the DSP. The DSP will use this threshold table to calculate the ρ-QP table.

MPEG-4 and H.264 For H.263, the calculation of τ _i can be performed as follows:
τ _i = | C _i | >> 1 (intra AC and DC coefficients)
τ _i = | 2C _i | / 5
= (| C _i | × 0x666 + 0x4CD) (inter AC and DC coefficients)
In the case of intra, | C _i | is limited to [062], and in the case of inter, | C _i | is limited to [077]. 0x666 is (2/5) and 0x4CD is (3/10) in Q12. From these equations, MPEG-4 and H.264 are shown. It will be noted that for 263, only a few calculations are needed to estimate τ _i . Since it is unlikely that these coefficients will be zero after quantization, it is not necessary to use another formula (or table) for the intra AC coefficients. This can be observed from the histogram of the intra DC coefficient shown in FIG.
Table 3 below is a typical look-up table (LUT) for determining threshold values for MPEG-4 inter coefficients.

下記の表４はＭＰＥＧ‐４のイントラ係数に関する閾値を求めるための典型的な一つの参照表（ＬＵＴ）である。

Table 4 below is a typical look-up table (LUT) for determining a threshold value for an MPEG-4 intra coefficient.

特に、ハードウェアを考慮し表を使用してＭＰＥＧ‐４の閾値計算を実施することが必要であれば、インター及びイントラ係数に関する典型的入力は表３及び表４においてそれぞれ与えられる。
ＩＴＵ Ｈ．２６４規格について、ＩＴＵ Ｈ．２６４規格では、除数はＱＰに依存しているだけでなく、他の因数にも依存しているので、量子化されない係数からの閾値の直接計算はさらに難しい。ＩＴＵ Ｈ．２６４規格に従って閾値を計算するために使用できる一つの一般的なハードウェア構造は図７に示される。

In particular, if it is necessary to implement the MPEG-4 threshold calculation using tables taking into account hardware, typical inputs for inter and intra coefficients are given in Tables 3 and 4, respectively.
ITU H. For the H.264 standard, ITU H.264 In the H.264 standard, the divisor is not only dependent on the QP, but also on other factors, so the direct calculation of the threshold from the unquantized coefficients is even more difficult. ITU H. One common hardware structure that can be used to calculate thresholds according to the H.264 standard is shown in FIG.

As shown in FIG. 7, the input coefficient W _ij is an input to block 701, which generates the absolute value of W _ij . Block 702 limits the absolute value of W _ij to 2047 and sends its output to adder 707. The output of block 702 is also shifted right by 5 by block 703. The output of block 703 is an input signal to multiplexer 704. The most significant bit of the input to multiplexer 704 is used to select the output from the values 0, 1, 2, 3, 4, 5 and 6.

  Adder 705 subtracts 1 from the output of multiplexer 704. The output of the adder 705 is shifted to the left by 1. The output of block 706 is provided to adder 707 and added to the output of block 702. Then, the output of the adder 707 is shifted to the right by the value output by the multiplexer 704. Block 709 limits the value of block 708 to 31. The output of block 709 is then provided to address generator 710.

  Address generator 710 generates the address used by LUT 711. In particular, address generator 710 receives the output of block 709 as well as i, j, and intra to generate an address that is provided to LUT 711. Examples of LUT 711 are provided in Tables 6 and 7 below. Table 6 shows LUTs related to intra values, and Table 7 shows LUT examples related to inter values. Each input value to the address generator 710 can be mapped to one specific value in the LUT. Given an address (addr), the appropriate data is selected from the LUT 711.

  Each output of LUT 711 is provided to adder 717. The output of multiplexer 704 is also provided to multiplexer 715. Adder 712 subtracts 1 from the output of multiplexer 704 and provides this value to multiplexer 715 as the other output. The output of the multiplexer 715 is selected based on control signals from the AND gate 713 and the OR gate 714. Inputs i and j to the AND gate 713 are exponent values of input coefficients and are inverted. The input to OR gate 714 indicates whether the current block is an intra block and whether the current block is a chromaticity block. The output of adder 712 is selected by the multiplexer when both inputs to multiplexer 715 are 1, otherwise the output of multiplexer 704 is selected by multiplexer 715.

The output of the multiplexer 715 is multiplied by 6 by the multiplier 716. The output of the multiplier 716 is added by one output of the LUT 711 selected by the adder 717. The output of adder 717 is limited by block 718 to a lower limit of 0 and an upper limit of 52. The output of circuit 700 is a quantization parameter threshold QP _T. The value QP _T is used as an input to the circuit 900 of FIG. 9, as discussed below.

In the hardware circuit diagram of FIG. 7, the limiting factor L is used to limit the absolute value of the unquantized coefficient below the maximum divisor used for quantization. For W _ij greater than L, it is generally guaranteed that τ _ij (QP _T ) is equal to QP _max . Therefore, there is no need to store the LUT input corresponding to W _ij greater than L. The value of L depends on the coefficient index and the macroblock mode (intra or inter). Various values of L are shown in Table 5 below. In particular, Table 5 lists absolute values above which the QP threshold will be clamped to QP _max .

いくつかの実施例では、２の累乗である最大制限値（２０４７）が共通のクリップ因数として使用される。ＬＵＴは僅か３２個の入力値［０〜３１］について記憶することができる。ＩＴＵ Ｈ．２６４では、量子化ステップ・サイズ（除数）はＱＰが６増えるごとに二倍になる。従って、入力値を写像するために、クリップされた入力を０と３１の間の数に変換するであろう（２の累乗である）最小除数 Ｄ＝２^Ｓが計算されることになる。これは入力を３２で除算し、且つ出力を表すために使用される数を計数する（シフト・レジスタ及びマルチプレクサで実施される）ことによって行われる。そして、クリップされた値はＳだけ下に移行される。この除算が丸めによって行われれば、参照における誤差はあまり偏らないであろう。図８Ａ及び８Ｂは閾値ＬＵＴのアドレス計算に使用される偏り（bias）を丸める効果を例示するグラフである。

In some embodiments, a maximum limit value (2047) that is a power of 2 is used as a common clip factor. The LUT can store only 32 input values [0-31]. ITU H. In H.264, the quantization step size (divisor) doubles for every QP increased by 6. Thus, to map the input value, the minimum divisor D = 2 ^S (which is a power of 2) will be calculated which will convert the clipped input to a number between 0 and 31. This is done by dividing the input by 32 and counting the number used to represent the output (implemented with shift registers and multiplexers). The clipped value is shifted down by S. If this division is done by rounding, the error in the reference will not be very biased. 8A and 8B are graphs illustrating the effect of rounding the bias used in the address calculation of the threshold LUT.

  The transition factor S is multiplied by 6 and then added to the reference result to obtain the final threshold QP. The LUT depends on whether the macroblock is an intra block or an inter block and the coefficient index. Typical inputs for intra and inter coefficients are given in Tables 6 and 7, respectively. In particular, Table 6 shows ITU H.264. A typical look-up table for determining thresholds for H.264 intra coefficients, while Table 7 shows ITU H.264. 6 is a typical look-up table for obtaining threshold values for H.264 inter coefficients.

さらに閾値‐ＱＰ表の作成を簡単化するために、Ｔ（ＱＰ）表を形成するため全ての係数上で閾値ＱＰを累積するハードウェアが設計される。Ｔ（ＱＰ）表を形成するため閾値ＱＰを累積するために使用できる典型的な回路構成は図９で示される。

In addition, to simplify the creation of the threshold-QP table, hardware is designed that accumulates the threshold QP on all coefficients to form a T (QP) table. A typical circuit configuration that can be used to accumulate the threshold QP to form a T (QP) table is shown in FIG.

As shown in FIG. 9, circuit 900 receives a quantization parameter threshold QP _T input. Adder 901 subtracts 30 from QP _T and provides this value to chromaticity QP LUT 902, examples of which are provided in Table 8. Multiplexer 904 receives the input signal and lightness corresponding to the sign of the output of adder 901 determined by block 903 and H.264. Based on the H.264 input signal, a selection is made between the QP _T value and the value stored in the chromaticity QP LUT 902. The sign value is zero, the lightness value indicating that the block is not a lightness block is zero, and the encoding standard is ITU H.264. H.264 indicating H.264. If the H.264 value is 1, the value stored in the chromaticity QP LUT is selected. Otherwise, multiplexer 904 selects QP _T as an output. The output of multiplexer 904 includes the input address (addr) stored in T (QP) table 905. The original address value output from multiplexer 904 is output to adder 906, incremented by 1 and stored on the original address value in T (QP) table 905. A specific QP _T is generated for each coefficient and each of these QP _Ts .

  ITU H. The LUT used to convert chromaticity QP values to QP values according to H.264 is given in Table 8. In particular, Table 8 illustrates a table that maps the chromaticity-QP threshold to the QP threshold.

表９は異なるビデオ・クリップのビット・レート変動の実験的結果を表にする。比較の目的のために、表は従来の基線手法を用いた結果、及びここに概説されたρ‐領域手法を用いた結果を表にする。表９は実験に使用された五つの異なるビデオ・クリップを表にする。異なるクリップは四つの異なるレート制御方法で符号化された：即ち、６４キロビット／秒（Kbps）及び四分の一共通インタフェース・フォーマット（QCIF）‐１５フレーム／秒（FPS）；２８Kbps 及び QCIF‐１５FPS；５５Kbps 及び QCIF‐１０ FPS；そして３５Kbps 及び QCIF ‐１０FPS。基線手法は前に述べた固定二乗手法を一般的に参照する。

Table 9 tabulates the experimental results of bit rate variations for different video clips. For comparison purposes, the table tabulates the results using the conventional baseline approach and the results using the ρ-region approach outlined here. Table 9 tabulates five different video clips used in the experiment. Different clips were encoded in four different rate control methods: 64 Kbit / s (Kbps) and 1/4 common interface format (QCIF) -15 frames / second (FPS); 28 Kbps and QCIF-15FPS 55 Kbps and QCIF-10 FPS; and 35 Kbps and QCIF-10 FPS. The baseline method generally refers to the previously described fixed square method.

表９から、従来の基線手法及び提案されたρ‐領域手法の双方は目標ビット・レートを達成するのに十分であることに気付くであろう。ρ‐領域手法は様々なクリップにわたってビット・レートの変動を低減する。
表１０は異なる目標ビット・レートで符号化された様々なビデオ・クリップについてフレームごとに使用されるビットの標準偏差を定量化する実験結果の表である。表１０は従来の基線手法を用いた結果及びここに概説されたρ‐領域手法を用いた結果の表である。表１０は異なる目標ビット・レートで符号化された、五つの異なるビデオ・クリップに関するデータを含む。異なるクリップは四つの異なるレート制御方法で符号化された：即ち、２８キロビット／秒（Kbps）及び四分の一共通インタフェース・フォーマット（QCIF）‐１５フレーム／秒（FPS）；３５Kbps 及び QCIF‐１０FPS；５５Kbps 及び QCIF‐１０ FPS；そして６４Kbps 及び QCIF ‐１５FPS。

From Table 9, it will be noted that both the conventional baseline approach and the proposed ρ-domain approach are sufficient to achieve the target bit rate. The ρ-domain approach reduces bit rate variation across various clips.
Table 10 is a table of experimental results that quantify the standard deviation of bits used per frame for various video clips encoded at different target bit rates. Table 10 is a table of results using the conventional baseline method and results using the ρ-region method outlined herein. Table 10 contains data for five different video clips encoded at different target bit rates. Different clips were encoded in four different rate control methods: 28 Kbit / s (Kbps) and 1/4 common interface format (QCIF)-15 frames per second (FPS); 35 Kbps and QCIF-10 FPS 55 Kbps and QCIF-10 FPS; and 64 Kbps and QCIF-15 FPS.

表１０から理解されるように、ρ‐領域技術は従来の基線システムより著しく優れている。平均して、ρ‐領域技術は標準偏差を５０％以上低減する。いくつかのクリップ（mother_daughter）について、この低減は劇的な８５％であった。

As can be seen from Table 10, the ρ-region technique is significantly superior to conventional baseline systems. On average, the ρ-region technique reduces the standard deviation by more than 50%. For some clips (mother_daughter), this reduction was dramatic 85%.

  FIG. 10 includes four different plots showing the bits used per frame and the peak signal-to-noise ratio (PSNR) per frame provided by the conventional baseline technique and the ρ-region rate control technique described herein. The plot in FIG. 10 corresponds to the encoding of a mother-daughter clip at 64 Kbps.

  From the plot of FIG. 10, it will be appreciated that ρ-region rate control does not negatively affect PSNR when the reduction in variation of bits spent on a frame is dramatic. In this example, ρ-region rate control results in a smoother behavior in the PSNR curve over time, which is a desirable result.

  Further improvements to the encoding performance can be achieved by adapting the parameters A and B in Equation 1 listed above. In particular, adapting parameters A and B of Equation 1 can further improve rate control by reducing the frame-level bit variation shown in FIG. Adaptation further reduces the variation by 37% with another average. In this example, the parameters are updated once for every frame in the low (ρ) -adaptive technique. The adaptation of parameters A and B associated with the threshold approach described here is called the adaptive threshold approach.

  Reducing QP variation within a frame helps reduce the bits required to encode delta-QP, and reducing QP variation reduces coding quality by making the quality relatively stable across frames. It may also be important to reduce QP variation within a frame, as it can be improved. This goal of reducing QP variations within a frame can be achieved by using a ρ-QP table adjusted from the previous frame to provide macroblock level rate control. This helps to reduce fluctuations in the ρ-QP table estimate and thus reduce fluctuations in fluctuations in the QP value within the frame. If the ρ-QP table from the previous macroblock is used for rate control, only about 25% of the macroblocks have invariant QP values. It will be noted that for almost 80% of the macroblocks, there is no change in the value of QP by using the ρ-QP table adjusted from the previous frame.

  As shown in Table 11, the significant reduction in frame-level bit rate variation caused by ρ-region rate control does not result in a degradation in the quality of the encoded video sequence. Table 11 provides the PSNR obtained from various coding sequences.

図１２はρ‐領域レート制御によってもたらされるフレーム・レベルのビット・レート変動の著しい低減が符号化されたビデオ・シーケンスの品質の低下を生じないことをさらに例示する。ρ‐領域方法の使用によってさらに緊密なレート制御が達成されることが既に示されてきたが、図１２からρ‐領域レート制御はまたさらに低いＱＰ値を維持することができることに気付かれるであろう。平均して、これはＰＳＮＲを増加しない。さらに、品質の変動は提案されたレート制御手法によって低減される。

FIG. 12 further illustrates that the significant reduction in frame-level bit rate variation caused by ρ-region rate control does not result in a degradation in the quality of the encoded video sequence. Although it has already been shown that tighter rate control is achieved by using the ρ-domain method, it can be noted from FIG. 12 that ρ-domain rate control can also maintain lower QP values. Let's go. On average, this does not increase the PSNR. Furthermore, quality variations are reduced by the proposed rate control technique.

  Various embodiments have been described. In particular, ρ-region rate control has been demonstrated to produce superior results that reduce bit-to-frame bit variations by 50-80% compared to conventional baseline methods. This reduction in bit rate variation is also shown to be without any degradation in PSNR and image quality recognition. A number of techniques have also been described that can simplify the generation of a ρ-QP LUT, including threshold methods used to create Threshold-QP LUTs and create ρ-QP LUTs.

  The techniques described herein may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the techniques are applied to a computer-readable medium containing program code that performs one or more of the methods described above when implemented in a device that encodes a video sequence. In that case, the computer readable medium may be a random access memory (RAM) such as a synchronous dynamic random access memory (SDRAM), a read only memory (ROM), a non-volatile random access memory (NVRAM), an electrical Erasable programmable read only memory (EEPROM), flash (FLASH) memory, and so on.

  The program code is stored in memory in the form of computer readable instructions. In that case, a processor such as a DSP executes instructions stored in memory to perform one or more of the techniques described herein. In some cases, the technique is performed by a DSP that exercises various hardware elements to facilitate the encoding process. In other cases, the video encoder is a microprocessor, one or more application specific integrated circuits (ASICs), one or more field programmable gate arrays (FPGAs), or some other hardware-software combination. As implemented. These and other embodiments are within the scope of the claims.

1 is a block diagram illustrating an exemplary system in which a source digital video device transmits an encoded sequence of video data to a receiving digital video device. FIG. 1 is an exemplary block diagram of a digital video device according to an embodiment of the present invention. FIG. 6 is a graph illustrating a substantially linear relationship between the number of bits per frame and row (ρ). FIG. MPEG-4 standard or ITU H.264 FIG. 2 is an exemplary circuit diagram illustrating a circuit that can directly calculate a ρ (QP) lookup table (LUT) according to the H.263 standard. ITU H. 2 is an exemplary circuit diagram illustrating a circuit that can be used to directly calculate a ρ (QP) lookup table (LUT) according to the H.264 standard. ITU H. 2 is an exemplary circuit diagram illustrating a circuit that can be used to directly calculate a ρ (QP) lookup table (LUT) according to the H.264 standard. FIG. 4 is a histogram of intra DC coefficients illustrating that it is not necessary to use another equation (or look-up table) for intra AC coefficients when these coefficients are likely to be zero after quantization. ITU H. FIG. 2 is an exemplary circuit diagram used to calculate a threshold used to create a threshold-QP LUT according to the H.264 standard. 6 is a graph illustrating the effect of rounding a bias used for address calculation of a threshold LUT. 6 is a graph illustrating the effect of rounding a bias used for address calculation of a threshold LUT. FIG. 3 is an exemplary circuit diagram of a circuit that can be used to accumulate threshold QPs to form a threshold-QP table. 4 includes four different graphs showing the bits used per frame and the peak signal-to-noise ratio (PSNR) per frame obtained from the conventional baseline method and the rho-domain rate control method described herein. FIG. 6 is a graph of the standard deviation of bits used for different video clips using the conventional baseline method, the ρ-region method described herein, and the adaptive ρ-region method described herein. FIG. 5 is a graph and corresponding video frame illustrating that the significant reduction in frame-level bit rate variation provided by the ρ-region rate control method does not cause any degradation in the quality of the encoded video sequence.

Claims (31)

Compute a threshold for the unquantized coefficients of the video block, where the threshold identifies when the quantized coefficient is non-zero for different quantization parameters (QP), and uses the threshold to A video encoding device comprising: an encoder that generates a ρ-QP lookup table (LUT) that maps a number of zero coefficients (ρ) to a QP; and a memory that stores the ρ-QP LUT.   The video encoding device of claim 1, wherein a QP is selected for rate-controlled video encoding of a video block using a ρ-QP LUT.   The video encoding device of claim 2, wherein the encoder quantizes the coefficients of the video block based on the selected QP and encodes the video block according to an inter-frame video encoding technique. The encoder is
Generate a threshold-QP LUT that maps the threshold to QP, and use the threshold-QP LUT to generate a ρ-QP LUT;
The video encoding device of claim 1, wherein the memory stores threshold-QP LUT and ρ-QP LUT.   The video encoding device of claim 4, wherein the encoder generates a threshold-QP LUT by accumulating thresholds for QPs from a maximum QP to a minimum QP. The predetermined threshold (T _i ) is
Q (C _i , QP) = 0 for all QP> T _i
Q (C _i , QP) ≠ 0 for all QP ≦ T _i
5. The video encoding device of claim 4, wherein the function Q (C _i , QP) represents a quantization operation of the predetermined coefficient C _i using the quantization parameter QP.   The encoder generates different ρ-QP LUTs for different video blocks of a video frame and uses the corresponding ρ-QP LUT for a given video block to perform rate-controlled video coding of the given video block. The video encoding device of claim 1, wherein the video encoding device selects a QP for.   The encoder generates different ρ-QP LUTs for different video frames of the video sequence and uses the corresponding ρ-QP LUT for a given video frame to rate control the video blocks of the given video frame The video encoding device of claim 1, wherein the video encoding device selects a QP for video encoding.   The total number of computations required by the encoder to generate a ρ-QP LUT is about 2N + M, where N represents the number of coefficients and M represents the number of possible values of QP. The described video encoding device. Calculating a threshold for the unquantized coefficients of the video block, where the threshold specifies when the quantized coefficients are non-zero for different quantization parameters (QP), and using the threshold to quantize A method for providing rate controlled video coding, including generating a ρ-QP lookup table (LUT) that maps a number of non-zero coefficients (ρ) to a QP.   12. The method of claim 10, further comprising selecting a QP for rate control coding of a video block using a ρ-QP LUT.   The method of claim 11, further comprising quantizing the coefficients of the video block based on the selected QP and encoding the video block according to an inter-frame video encoding technique. The method of claim 10, further comprising: generating a threshold-QP LUT that maps the threshold to a quantization parameter (QP); and generating the ρ-QP LUT using the threshold-QP LUT.   The method of claim 13, wherein generating the threshold-QP LUT includes accumulating thresholds for quantization parameters QP from a maximum QP to a minimum QP.   In addition, for generating different ρ-QP LUTs for different video blocks of a video frame, and for rate-controlled video coding of a given video block using the corresponding ρ-QP LUT for a given video block 11. The method of claim 10, comprising selecting a QP.   Furthermore, generating different ρ-QP LUTs for different video frames of the video sequence and rate controlling video of the video block of a given video frame using the corresponding ρ-QP LUT for a given video frame The method of claim 10, comprising selecting a QP for encoding. The predetermined threshold (T _i ) is
Q (C _i , QP) = 0 for all QP> T _i
Q (C _i , QP) ≠ 0 for all QP ≦ T _i
11. The method of claim 10, wherein the function Q (C _i , QP) represents a quantization operation of a predetermined coefficient C _i using a quantization parameter QP.   11. The total number of computations required by the encoder to generate a ρ-QP LUT is about 2N + M, where N represents the number of coefficients and M represents the number of possible values for QP. The method described. Means for calculating a threshold for unquantized coefficients of a video block, wherein the threshold identifies when the quantization coefficient is non-zero for different quantization parameters (QP), and uses the threshold to quantize Means for generating a ρ-QP lookup table (LUT) that maps the number of non-zero coefficients (ρ) after conversion into QP.   20. The apparatus of claim 19, further comprising means for selecting a QP for rate control coding of a video block using a ρ-QP LUT.   21. The apparatus of claim 20, further comprising: means for quantizing video block coefficients based on the selected QP; and means for encoding the video block according to an inter-frame video encoding technique. 20. The apparatus of claim 19, further comprising: means for generating a threshold-QP LUT that maps the threshold to a quantization parameter (QP); and means for generating a ρ-QP LUT using the threshold-QP LUT.   23. The apparatus of claim 22, wherein the means for generating a threshold-QP LUT comprises means for accumulating thresholds for quantization parameters QP from a maximum QP to a minimum QP. The predetermined threshold (T _i ) is:
Q (C _i , QP) = 0 for all QP> T _i
Q (C _i , QP) ≠ 0 for all QP ≦ T _i
23. The apparatus of claim 22, wherein the function Q (C _i , QP) represents a quantization operation of a predetermined coefficient C _i using a quantization parameter QP.   Furthermore, means for generating different ρ-QP LUTs for different video blocks of a video frame, and for rate-controlled video coding of a given video block using the corresponding ρ-QP LUT for a given video block 20. The apparatus of claim 19, comprising means for selecting a QP.   Furthermore, means for generating different ρ-QP LUTs for different video frames of the video sequence, and rate control video of the video block of a given video frame using the corresponding ρ-QP LUT for a given video frame The apparatus of claim 19, comprising means for selecting a QP for encoding.   20. The total number of computations required to generate a ρ-QP LUT is about 2N + M, where N represents the number of coefficients and M represents the number of possible values for QP. apparatus.   20. The apparatus of claim 19, wherein the means for calculating a threshold for unquantized coefficients of the video block includes software executing on the digital signal processor.   20. The apparatus of claim 19, wherein the means for calculating a threshold for unquantized coefficients of the video block includes firmware.   20. The apparatus of claim 19, wherein the means for calculating a threshold for unquantized coefficients of a video block includes hardware.   32. The apparatus of claim 30, wherein the means for generating a [rho] -QP LUT includes software executing on a digital signal processor.

Images