JP5684342B2 - Method and apparatus for processing digital video data - Google Patents

Description

  This disclosure relates to video coding. This disclosure relates more specifically to estimating coding costs for coding video sequences.

  Digital video capability includes digital TV, digital direct broadcasting system, wireless communication device, personal digital assistant (PDA), laptop computer, desktop computer, video game console, digital camera, digital recording device, mobile phone, satellite wireless telephone, etc. Can be incorporated into a wide range of devices. Digital video devices can provide significant improvements over conventional analog video systems in processing and transmitting video sequences.

  Different video coding standards have been established for coding digital video sequences. For example, the Moving Picture Experts Group (MPEG) has developed several standards including MPEG-1, MPEG-2 and MPEG-4. Other examples are the International Telecommunication Union (ITU) -TH. 263 standard and ITU-T H.264. H.264 standard and its equivalent standard, ISO / IEC MPEG-4, Part-10, that is, Advanced Video Coding (AVC). These video coding standards support improved transmission efficiency of video sequences by coding data in a compressed form.

Many current techniques utilize block-based coding. In block-based coding, a frame of a multimedia sequence is divided into individual pixel blocks that are coded based on differences from other blocks that can be located in the same frame or in different frames. Is done. Some pixel blocks, often referred to as “macroblocks”, comprise a group of sub-blocks of pixels. As an example, a 16 × 16 macroblock can comprise four 8 × 8 sub-blocks. These sub-blocks can be coded separately. For example, H.M. The H.264 standard allows coding of blocks with a variety of different block sizes, eg, 16 × 16, 16 × 8, 8 × 16, 8 × 8, 4 × 4, 8 × 4, and 4 × 8. Furthermore, as an extension, sub-blocks of any size, eg 2 × 16, 16 × 2, 2 × 2, 4 × 16, and 8 × 2, can be included in the macroblock.

  In certain aspects of this disclosure, a method for processing digital video data identifies one or more transform coefficients for pixel block residual data that remains non-zero when quantized. Estimating the number of bits associated with coding of the residual data based on at least the identified transform coefficient; and at least the estimated bits associated with coding the residual data Estimating a coding cost for coding the pixel block based on a number.

  In certain aspects, an apparatus for processing digital video data includes: a transform module that generates transform coefficients for residual data of a pixel block; and the transform coefficients that remain non-zero when quantized. A bit estimation module that identifies one or more and estimates a number of bits associated with coding of the residual data based at least on the identified transform coefficients; and at least associated with coding the residual data A control module for estimating a coding cost for coding the pixel block based on the estimated number of bits.

In certain aspects, an apparatus for processing digital video data comprises: means for identifying one or more transform coefficients for pixel block residual data that remains non-zero when quantized; Means for estimating the number of bits associated with the coding of the residual data based on the identified transform coefficient; and based on at least the estimated number of bits associated with coding the residual data. Means for estimating a coding cost for coding the pixel block.

In certain aspects, a computer program product for processing digital video data comprises a computer readable medium having instructions stored thereon. The instructions include a code for identifying one or more transform coefficients for pixel block residual data that remains non-zero when quantized, and the residual data based at least on the identified transform coefficients Estimating a coding cost for coding the pixel block based on a code for estimating the number of bits associated with the coding of and at least the estimated number of bits associated with coding the residual data The code | symbol for doing.

  The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description and drawings, and from the claims.

1 is a block diagram illustrating a video coding system that employs a coding cost estimation technique described herein. FIG. FIG. 3 is a block diagram illustrating an exemplary encoding module in further detail. FIG. 3 is a block diagram illustrating another exemplary encoding module in further detail. 5 is a flow diagram illustrating an exemplary operation of an encoding module that selects an encoding mode based on an estimated coding cost. FIG. 5 is a flow diagram illustrating an exemplary operation of an encoding module that estimates the number of bits associated with coding residual data without quantizing or encoding the residual data of the block. FIG. 5 is a flow diagram illustrating an exemplary operation of an encoding module that estimates the number of bits associated with coding residual data without encoding the residual data of the block.

  This disclosure describes video coding mode selection techniques using estimated coding costs. For example, to provide high compression efficiency, the encoding device may attempt to select a pixel block coding mode that codes pixel block data with high efficiency. For this purpose, the encoding device can make a coding mode selection based on a coding cost estimate for at least some of the possible modes. With the techniques described herein, an encoding device estimates coding costs for different modes without actually coding the block. Indeed, in some aspects, the encoding module device can estimate the coding cost for a mode without quantizing the block of data for each mode. By this method, the disclosed coding cost estimation technique reduces the computationally intensive amount of computation required to make an effective mode selection.

  FIG. 1 is a block diagram illustrating a multimedia coding system 10 that employs the coding cost estimation techniques described herein. The coding system 10 includes an encoding device 12 and a decoding device 14 connected by a transmission channel 16. Encoding device 12 encodes one or more digital multimedia data sequences and decodes device 14 in transmission channel 16 for decoding and possibly presenting the encoded sequence to a user of device 14. Send to. Transmission channel 16 may comprise any wired or wireless medium, or combination thereof.

  Encoding device 12 may form part of a broadcast network component that is used to broadcast one or more multimedia data channels. As one example, encoding device 12 forms part of a wireless base station, server, or any infrastructure node used to broadcast one or more channels of encoded multimedia data to the wireless device. can do. In this case, the encoding device 12 can transmit the encoded data to a plurality of wireless devices, eg, the decoding device 14. However, for simplicity, a single decoding device 14 is shown in FIG. Alternatively, encoding device 12 may comprise a handset that transmits locally captured video for video telephony or other similar applications.

  The decoding device 14 may comprise a user device that receives the encoded multimedia data transmitted by the encoding device 12 and decodes the multimedia data for presentation to the user. As an example, the decoding device 14 is sold under the name “iPod”, a digital television, a wireless communication device, a game play device, a portable digital assistant (PDA), a laptop computer, a desktop computer, a digital music and video device, for example. It can be implemented as part of a device, a radiotelephone such as a cellular, satellite or ground based radiotelephone, or other wireless mobile terminal equipped for video and / or audio streaming, video telephony, or both. The decoding device 14 can be associated with a mobile device or a stationary device. In broadcast applications, encoding device 12 may transmit encoded video and / or audio to a plurality of decoding devices 14 associated with a plurality of users.

  In some aspects, for bi-directional communication applications, the multimedia coding system 10 is based on Session Initiation Protocol (SIP), International Telecommunication Union Standardized Sector (ITU-T) H.264, 323 standard, ITU-T H.264. Video telephony or video streaming may be supported according to the 324 standard or other criteria. For one-way or two-way communication, the encoding device 12 may use video compression standards such as Moving Picture Experts Group (MPEG) -2, MPEG-4, ITU-T H.264. H.263 or ITU-H.MP that supports MPEG-4, Part 10, and advanced video coding (AVC). H.264, multimedia data encoded according to H.264 can be generated. Although not shown in FIG. 1, encoding device 12 and decoding device 14 may be integrated with a speech encoder and decoder, respectively, for audio and video in a common data sequence or separate data sequences. Appropriate multiplexer-demultiplexer (MUX-DEMUX) modules to handle both encodings, or other hardware, firmware, or software may be included. If applicable, the MUX-DEMUX module is ITU-H. It can be compliant with other protocols such as the H.223 multiplexer protocol or User Datagram Protocol (UDP).

  In certain aspects, this disclosure is disclosed in technical standard TIA-1099, Aug. Forward Link Only Air Interface Specification for Terrestrial Mobile Multimedia Multicast issued as 2006 (“FLO Specification”) Enhanced H.264 for delivering real-time multimedia services in terrestrial mobile multimedia multicast (TM3) systems Application to H.264 video coding is contemplated. However, the coding cost estimation techniques described in this disclosure are not limited to a particular type of broadcast, multicast, unicast, or point-to-point system.

  As shown in FIG. 1, the encoding device 12 includes an encoding module 18 and a transmitter 20. The encoding module 18 receives one or more input multimedia sequences that can include one or more data frames in the case of video encoding and selectively encodes the frames of the received multimedia sequences. . Encoding module 18 receives input multimedia sequences from one or more sources (not shown in FIG. 1). In some aspects, the encoding module 18 may receive input multimedia sequences from one or more video content providers, eg, via satellite. As another example, encoding module 18 may receive a multimedia sequence from an image capture device (not shown in FIG. 1) that is incorporated into or coupled to encoding device 12. Alternatively, encoding module 18 may receive a multimedia sequence from a memory or archive (not shown in FIG. 1) within or coupled to encoding device 12. The multimedia sequence can comprise live real-time or near real-time video, audio, or video and audio sequences that are coded and transmitted as broadcast or on demand, or can be coded as broadcast or on demand. Video, audio, or video and audio pre-recorded and stored for transmission. In some aspects, at least a portion of the multimedia sequence can be generated by a computer, such as in game play.

  In any case, the encoding module 18 encodes the plurality of frames and transmits the plurality of coded frames to the decoding device 14 via the transmitter 20. Encoding module 18 may encode the frame of the input multimedia sequence as an intra-coded frame, an inter-coded frame, or a combination thereof. Frames encoded using intra-frame coding techniques are coded without reference to other frames and are often referred to as intra ("I") frames. Frames that are encoded using interframe coding techniques are coded with respect to one or more other frames. An inter-frame coded frame may include one or more predicted “P” frames, bi-directional (“B”) frames, or combinations thereof. The P frame is encoded with reference to at least one temporally previous frame, and the B frame is encoded with reference to at least one temporally subsequent frame. In some cases, a B frame may be encoded with reference to at least one temporally subsequent frame and at least one temporally previous frame.

  Encoding module 18 may be further configured to divide the frame into a plurality of blocks and encode each of these blocks separately. As an example, encoding module 18 may divide a frame into a plurality of 16 × 16 blocks. Some blocks are often referred to as “macroblocks” and comprise groups of subdivision blocks (often referred to herein as “subblocks”). As an example, a 16 × 16 macroblock can comprise four 8 × 8 sub-blocks, or other subdivision blocks. For example, H.M. The H.264 standard allows for the encoding of blocks having a variety of different sizes, eg 16 × 16, 16 × 8, 8 × 16, 8 × 8, 4 × 4, 8 × 4, 4 × 8. Furthermore, as an extension, sub-blocks of any size, eg 2 × 16, 16 × 2, 2 × 2, 4 × 16, and 8 × 2, can be included in the macroblock. Accordingly, encoding module 18 may be configured to divide a frame into several blocks and encode each of the pixel blocks as an intra-coded block or an inter-coded block. Each can be generally referred to as a block.

  Encoding module 18 may support multiple coding modes. Each of these modes can correspond to a different combination of block size and coding technique. For example, H.C. In the case of the H.264 standard, there are 7 inter modes and 13 intra modes. The seven variable block size inter modes are SKIP mode, 16 × 16 mode, 16 × 8 mode, 8 × 16 mode, 8 × 8 mode, 8 × 4 mode, 4 × 8 mode, 4 X4 mode. The 13 intra modes include an INTRA 4 × 4 mode with 9 possible interpolation directions and an INTRA 16 × 16 mode with 4 possible interpolation directions.

  In order to provide high compression efficiency, according to various aspects of this disclosure, encoding module 18 attempts to select a mode for coding block data with high efficiency. For this purpose, the encoding module 18 estimates the coding cost for at least part of all modes for each block. Encoding module 18 estimates the coding cost as a function of rate and distortion. With the techniques described herein, encoding module 18 estimates the coding cost for a mode without actually coding the block to determine the rate and distortion metrics. In this way, the encoding module 18 can select one of the modes based at least on the coding cost without performing complex coding of the block data for each mode. Conventional mode selection requires actual coding of the data with each mode to determine which mode to select. Thus, these techniques save time and computational resources by selecting modes based on coding costs without actually coding the data for each mode. Indeed, in some aspects, the encoding module 18 can estimate the coding cost for a mode without quantizing the block's data for each mode. By this method, the disclosed coding cost estimation technique reduces the computationally intensive amount of computation required to make an effective mode selection.

  Encoding device 12 applies the selected mode to code a block of frames and transmits the coded data frame via transmitter 20. The transmitter 20 may include appropriate modem and driver circuit software and / or firmware for transmitting the encoded multimedia on the transmission channel 16. For wireless applications, the transmitter 26 includes RF circuitry for transmitting wireless data that carries encoded multimedia data.

  The decoding device 14 includes a receiver 22 and a decoding module 24. The decoding device 14 receives the encoded data from the encoding device 12 via the receiver 22. Similar to transmitter 20, receiver 22 may include appropriate modem and driver circuit software and / or firmware for receiving encoded multimedia on transmission channel 16, and encoding in wireless applications. And RF circuitry for receiving wireless data carrying multimedia data. The decoding module 24 decodes the coded data frame received via the receiver 22. The decoding device 14 is decoded via a display (not shown) that can be incorporated into the decoding device 14 or provided as a separate device coupled to the decoding device 14 via a wired or wireless connection. A data frame can be further presented to the user.

  In some examples, encoding device 12 and decoding device 14 may each act as both a transmitting device and a receiving device for encoded multimedia and other information transmitted in transmission channel 16. As described above, each of them may include a reciprocal transmission / reception circuit. In this case, both the encoding device 12 and the decoding device 14 can transmit and receive multimedia sequences and thus participate in two-way communication. In other words, the illustrated components of coding system 10 may be integrated as part of an encoder / decoder (CODEC).

  The components in encoding device 12 and decoding device 14 are example components that can be used to implement the techniques described herein. However, encoding device 12 and decoding device 14 may include a number of other components if desired. For example, encoding device 12 may include multiple encoding modules that each receive one or more multimedia data sequences and encode each multimedia data sequence from the techniques described herein. In this case, encoding device 12 may further include at least one multiplexer that combines the data segments for transmission. Furthermore, the encoding device 12 and the decoding device 14 include appropriate modulation components, demodulation components, frequency conversion components, filtering components, and amplifier components for transmission and reception of encoded video. And may include radio frequency (RF) radio components and antennas as appropriate. However, for ease of illustration, the components are not shown in FIG.

  FIG. 2 is a block diagram illustrating an exemplary encoding module 30 in further detail. The encoding module 30 can represent, for example, the encoding module 18 of the encoding device 12 of FIG. As shown in FIG. 2, the encoding module 30 receives input multimedia data frames of one or more multimedia sequences from one or more sources and processes the frames of the received multimedia sequences. A control module 32 is included. In particular, the control module 32 analyzes incoming frames of the multimedia sequence and determines whether these incoming frames should be encoded or skipped based on the analysis of the frames. In some aspects, the encoding device 12 may encode information contained in the multimedia sequence at a reduced frame rate by using frame skip to conserve bandwidth in the transmission channel 16. it can.

  Further, for incoming frames that are to be encoded, the control module 32 may be configured to determine whether these frames should be encoded as I-frames, P-frames, or B-frames. The control module 32 decides to encode the incoming frame as an I-frame at the start of the multimedia sequence or at a scene change in the sequence for use as a channel switch frame or as an intra-refresh frame. be able to. In other cases, the control module 32 encodes the frame as an inter-frame coded frame (ie, a P frame or a B frame) to reduce the amount of bandwidth associated with coding the frame.

  The control module 32 divides the frame into a plurality of blocks and a coding mode for each of these blocks, e.g. It can be further configured to select one of the H.264 coding modes. As described in detail below, encoding module 30 reduces the coding cost for at least some of these modes to help select the most efficient coding mode among these coding modes. Can be estimated. After selecting a coding mode for use in coding one of the blocks, encoding module 30 generates residual data for the block. For blocks selected for intraframe coding, the spatial prediction module 34 generates residual data for the blocks. Spatial prediction module 34 may generate a predicted version of the block via interpolation using, for example, one or more neighboring blocks and an interpolation direction corresponding to the selected intra-frame coding mode. The spatial prediction module 34 can now calculate the difference between the input frame block and the predicted block. This difference is called residual data or residual coefficient.

  For the block selected for interframe coding, the motion estimation module 36 and motion compensation module 38 generate residual data for the block. In particular, motion estimation module 36 identifies at least one reference frame and looks for a block in the reference frame that best matches the block in the input frame. The motion estimation module 36 calculates a motion vector to represent the offset between the position of the block in the input frame and the position of the identified block in the reference frame. Motion compensation module 38 calculates the difference between the block of the input frame and the identified block in the reference frame pointed to by the motion vector. This difference is called residual data for that block.

  The encoding module 30 also includes a transform module 40, a quantization module 46, and an entropy encoder 48. The conversion module 40 converts the residual data of the block according to the conversion function. In some aspects, transform module 40 applies an integer transform, such as a 4 × 4 or 8 × 8 integer transform or a discrete cosine transform (DCT), to the residual data to generate transform coefficients for the residual data. To do. The quantization module 46 quantizes the transform coefficients and provides the quantized transform coefficients to the entropy encoder 48. The entropy encoder 48 encodes the quantized transform coefficients using a context adaptive coding technique, such as context adaptive variable length coding (CAVLC) or context adaptive binary arithmetic coding (CABAC). As described in detail below, entropy encoder 48 applies the selected mode to code the data for the block.

  Entropy encoder 48 may also encode additional data associated with the block. For example, in addition to residual data, the entropy encoder 48 may include one or more motion vectors of the block, an identifier indicating the coding mode of the block, one or more reference frame indexes, quantization parameter (QP) information, Slice information, etc. can be encoded. Entropy encoder 48 may receive this additional block data from other modules within encoding module 30. For example, motion vector information can be received from the motion estimation module 36 and block mode information can be received from the control module 32. In some aspects, the entropy encoder 48 uses fixed length coding (FLC) techniques or universal variable length coding (VLC) techniques, such as exponential-Golomb coding (“Exp-Golomb”). At least a portion can be coded. Alternatively, entropy encoder 48 may encode a portion of the additional block data using the context adaptive coding technique described above, ie, CABAC or CAVLC.

  To assist the control module 32 in selecting a mode for the block, the control module 32 estimates the coding cost for at least some of the possible modes. In certain aspects, the control module 32 can estimate the cost of coding a block in each of the possible coding modes. Cost can be estimated, for example, with respect to the number of bits associated with coding a block in a given mode versus the amount of distortion that occurs in that mode. For example, H.C. For the H.264 standard, the control module 32 determines that there are 22 different coding modes (interframe and intraframe coding modes) for blocks selected for interframe coding and 13 for blocks selected for intraframe coding. Coding costs for different coding modes can be estimated. In other aspects, the control module 32 first reduces the set of possible modes using other mode selection techniques and then uses the techniques of this disclosure to estimate the coding cost for the remaining modes of the set. can do. In other words, in some aspects, the control module 32 may narrow the number of mode possibilities before applying the cost estimation technique. Advantageously, the encoding module 30 estimates the coding cost for a mode without actually coding the data for blocks for different modes, thereby reducing the computational overhead associated with coding decisions. In fact, in the example shown in FIG. 2, the encoding module 30 can estimate the coding cost without quantizing the block data for different modes. By this method, the disclosed coding cost estimation technique reduces the computationally intensive amount of computation required to calculate the coding cost. In particular, it is not necessary to encode the block using various coding modes to select one of the modes.

As described in further detail herein, control module 32 estimates the coding cost of each analyzed mode according to the following equation:

Where J is the estimated coding cost, D is the block distortion metric, λmode is the Lagrange multiplier for each mode, and R is the block rate metric. The distortion metric (D) includes, for example, the sum of absolute values of differences (SAD), the sum of squares of differences (SSD), the sum of absolute values of conversion differences (SATD), and the sum of squares of conversion differences (SSTD). ), And the like. The rate metric (R) can be, for example, the number of bits associated with coding data in a given block. As described above, different types of block data can be coded using different coding techniques. Therefore, equation (1) can be rewritten as:

Here, R _context represents a rate metric for block data coded using a context adaptive coding technique, and R _{non_context} represents a rate metric for block data coded using a non-context adaptive coding technique. For example, H.M. In the H.264 standard, residual data can be coded using context adaptive coding such as CAVLC or CABAC. Other block data such as motion vectors, block modes, etc. can be coded using FLC or universal VLC techniques such as Exp-Golomb. In this case, equation (2) can be rewritten as:

  Where Rresidual represents a rate metric for coding residual data using context adaptive coding techniques, eg, the number of bits associated with coding the residual data, and Rother is FLC or Universal VLC. Represents a rate metric for coding other block data using techniques, eg, the number of bits associated with coding the other block data.

In calculating the estimated coding cost (J), the encoding module 30 can relatively easily calculate the number of bits associated with coding block data using FLC or universal VLC, ie, R _other. Can be determined. Encoding module 30 may use a code table to identify the number of bits associated with coding block data using, for example, FLC or universal VLC. The code table can include, for example, a plurality of codewords and the number of bits associated with coding the codeword. However, determining the number of bits associated with coding the _residual data (R _residual ) is much more due to the fact that context adaptive coding as a function of the context of the data has an adaptive nature. It becomes a difficult task. Coding the residual data To determine the exact number of associated bits, or what data is in context adaptive coding, the encoding module 30 transforms the residual data and converts The quantized residual data must be quantized and the transformed-quantized residual data must be encoded. However, according to the techniques of this disclosure, bit estimation module 42 may estimate the number of bits associated with coding residual data using context adaptive coding techniques without actually coding the residual data. it can.

  In the example shown in FIG. 2, bit estimation module 42 estimates the number of bits associated with coding the residual data using transform coefficients for the residual data. Thus, for each mode to be analyzed, encoding module 30 need only calculate a transform coefficient for the residual data to estimate the number of bits associated with coding the residual mode. Thus, the encoding module 30 is associated with reducing resource complexity and coding residual data by not quantizing the transform coefficients for each mode and not encoding the quantized transform coefficients. Reduce the time required to determine the number of bits.

  Bit estimation module 42 analyzes the transform coefficients output by transform module 40 and identifies one or more transform coefficients that remain non-zero after quantization. In particular, bit estimation module 42 compares each of the transform coefficients with a corresponding threshold value. In some aspects, the corresponding threshold can be calculated as a function of the QP of the encoding module 30. Bit estimation module 42 identifies transform coefficients that are greater than or equal to the corresponding threshold as transform coefficients that remain non-zero after quantization.

Bit estimation module 42 estimates the number of bits associated with coding the residual data based on at least the transform coefficients identified as remaining non-zero after quantization. In particular, the bit estimation module 42 determines the number of non-zero transform coefficients that are not affected by quantization. Bit estimation module 42 sums at least some of the absolute values of the transform coefficients identified as unaffected by quantization. The bit estimation module 42 then estimates the rate metric for the residual data, i.e., the number of bits associated with coding the residual data, using the following equation:

Here, SATD is the sum of at least part of absolute values of non-zero transform coefficients that are predicted to be unaffected by quantization, and NZ _est is predicted to be unaffected by quantization. It is an estimated number of non-zero transform coefficients, and a ₁ , a ₂ , and a ₃ are coefficients. The coefficients a ₁ , a ₂ , and a ₃ can be calculated using, for example, least squares estimation. The sum of the conversion coefficients is the sum SATD of the absolute values of the conversion differences in the equation example (4), but other difference coefficients such as SSTD can be used.

An example of calculating _{Rresidual for} a 4x4 block is shown below. Similar calculations can be performed for blocks of different sizes. The encoding module 30 calculates a matrix of transform coefficients for the residual data. A typical matrix of transform coefficients is shown below.

  The number of rows of the transform coefficient matrix (A) is equal to the number of rows of pixels in the block, and the number of columns of the transform coefficient matrix is equal to the number of columns of pixels in the block. Therefore, in the above example, the dimension of the transform coefficient matrix is 4 × 4 to correspond to 4 × 4 blocks. Each entry A (i, j) in the transform coefficient matrix is a transform of the respective residual coefficient.

During quantization, transform coefficients with smaller values in matrix A tend to be zero after quantization. Accordingly, the encoding module 30 compares the residual transform coefficient matrix A with a threshold matrix and predicts which transform coefficients of the matrix A remain non-zero after quantization. A typical threshold matrix is shown below.

The matrix C can be calculated as a function of the QP value. The dimension of the matrix C is the same as the dimension of the matrix A. For example, H.C. For the H.264 standard, the entries in matrix C can be calculated based on the following equation:

  Where QBITS {QP} is a parameter that determines scaling as a function of QP, Level_Offset (i, j) {QP} is a dead zone parameter for entries in row i and column j of the matrix, and QP Is also a function, Level_Scale (i, j) {QP} is a multiplication factor for entries in row i and column j of the matrix, is also a function of QP, i corresponds to a row of the matrix, and j is a column of the matrix QP corresponds to the quantization parameter of the encoding module 30. In example equation (5), the variable is H.264. In the H.264 coding standard, it can be defined as a function of the operation QP.

  Other equations can be used to determine which of these variables exist after quantization, and other coding criteria are based on the quantization method employed by that particular criterion. Can be defined. In some aspects, the encoding module 30 can be configured to operate within a QP value range. In this case, the encoding module 30 can previously calculate a plurality of comparison matrices corresponding to each QP value within the QP value range. The encoding module 30 selects a comparison matrix corresponding to the QP of the encoding module 30 for comparison with the transform coefficient matrix.

The comparison result between the transform coefficient matrix A and the threshold matrix C is a matrix of 1s and zeros. In the above example, this comparison is a matrix of ones and zeros as shown below.

  Where 1 represents the position of the transform coefficient identified as likely to be unaffected by quantization, i.e., likely to remain non-zero, and zero is likely to be affected by quantization. That is, it represents the position of the transform coefficient that is expected to be zero. As described above, a transform coefficient is identified as likely to remain non-zero when the absolute value of the transform coefficient of matrix A is greater than or equal to the corresponding threshold value of matrix C.

Using the resulting 1 and zero matrix, the bit estimation module 42 determines the number of transform coefficients that are not affected by quantization. In other words, the bit estimation module 42 determines the number of transform coefficients identified as remaining non-zero after quantization. Bit estimation module 42 determines the number of transform coefficients identified as remaining non-zero after quantization according to the following equation:

Here, NZ _est is the estimated number of non-zero transform coefficients, and M (i, j) is the value of the matrix M in row i and column j. In the above example, NZ _est is equal to 8.

The bit estimation module 42 also calculates the sum of at least some of the absolute values of the transform coefficients estimated to be unaffected by quantization. In certain aspects, the bit estimation module 42 may calculate a sum of at least some of the absolute values of the transform coefficients according to the following equation:

  Where SATD is the sum of the transform coefficients identified as remaining non-zero after quantization, M (i, j) is the value of matrix M in row i and column j, and A (i, j j) is the value of matrix A in row i and column j, and abs (x) is an absolute value function that calculates the absolute value of x. In the example described above, SATD is equal to 2361. Other difference metrics, such as SSTD, can also be used for transform coefficients.

Using these values, bit estimation module 42 approximates the number of bits associated with coding the residual coefficient using equation (3) above. The control module 32 can calculate an estimate of the total coding cost of the mode using the estimate of R _residual . Encoding module 30 may estimate the total coding cost for one or more other possible modes in the same way and select the mode with the lowest coding cost. Encoding module 30 then applies the selected coding mode to code the block or blocks of the frame.

  The techniques described above can be implemented individually within encoding device 12, or two or more or all techniques can be implemented together. The components in encoding module 30 are typical examples of components that can be applied to implement the techniques described herein. However, the encoding module 30 includes many other components if desired, and includes a smaller number of components that combine the functions of one or more of the modules described above. Can do. The components within the encoding module 30 may include one or more processors, digital signal processors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any of these. It can be implemented as a combination. Drawing different features as modules is intended to emphasize different functional aspects of the encoding module 30 and that the modules must be realized by separate hardware or software components. Not necessarily. Rather, functionality associated with one or more modules may be incorporated within common or separate hardware or software components.

FIG. 3 is a block diagram illustrating another exemplary encoding module 50. The encoding module 50 of FIG. 3 substantially conforms to the encoding module 30 of FIG. 2, but the bit estimation module 52 of the encoding module 50 codes the residual data after quantizing the transform coefficients for the residual data. Estimate the number of bits associated with In particular, after quantizing the transform coefficients, the bit estimation module 52 estimates the number of bits associated with coding the residual coefficients using the following equation:

Where SATQD is the sum of absolute values of non-zero quantized transform coefficients, NZ _TQ is the number of non-zero quantized transform coefficients, and a ₁ , a ₂ , and a ₃ are It is a coefficient. The coefficients a ₁ , a ₂ , and a ₃ can be calculated using, for example, least squares estimation. Encoding module 50 quantizes the transform coefficients before estimating the number of bits associated with coding the residual data, but encoding module 50 still does not actually code the block data. Estimate the coding cost for the mode. Therefore, the calculation amount of the calculation intensive type is still reduced.

  FIG. 4 is a flow diagram illustrating exemplary operation of an encoding module that selects an encoding mode based at least on the estimated coding cost, eg, encoding module 30 of FIG. 2 and / or encoding module 50 of FIG. is there. However, for purposes of illustrating an example, FIG. 4 will be described with respect to encoding module 30. The encoding module 30 selects a mode for which a coding cost is to be estimated (60). Encoding module 30 generates a distortion metric for the current block (62). Encoding module 30 may calculate a distortion metric based on, for example, a comparison between the block and at least one reference block. In the case of a block selected for intra-frame coding, the reference block can be an adjacent block in the same frame. On the other hand, for a block selected for interframe coding, the reference block may be a block from an adjacent frame. The distortion metric can be, for example, SAD, SSD, SATD, SSTD, or other similar distortion metric.

  In the example of FIG. 4, encoding module 30 determines a number of bits associated with coding a portion of data that is coded using a non-context adaptive coding technique (64). As described above, this data includes one or more motion vectors of the block, an identifier indicating the coding mode of the block, one or more reference frame indexes, QP information, slice information of the block, etc. Can be included. Encoding module 30 may use a code table to identify the number of bits associated with coding data using, for example, FLC, universal VLC, or other non-context adaptive coding techniques.

  Encoding module 30 estimates and / or calculates the number of bits associated with coding the portion of data to be coded using context adaptive coding techniques (66). For example, H.C. For the H.264 standard, encoding module 30 may estimate the number of bits associated with coding the residual data using context adaptive coding. Encoding module 30 may estimate the number of bits associated with coding the residual data without actually coding the residual data. In certain aspects, the encoding module 30 may estimate the number of bits associated with coding the residual data without quantizing the residual data. For example, encoding module 30 may calculate transform coefficients for residual data and identify transform coefficients that are likely to remain non-zero after quantization. Using these identified transform coefficients, encoding module 30 estimates the number of bits associated with coding the residual data. In other aspects, the encoding module 30 may estimate the number of bits associated with quantizing the transform coefficients and coding the residual data based at least on the quantized transform coefficients. In either case, encoding module 30 saves time and processing resources by estimating the number of bits required. If there are sufficient computational resources, the encoding module 30 can calculate the actual number of bits required instead of estimating.

  Encoding module 30 estimates and / or calculates a total coding cost for coding the block in the selected mode (68). Encoding module 30 is associated with coding a data portion that is coded using distortion metrics, bits associated with coding a data portion that is coded using non-context adaptive coding, and context adaptive coding. A total coding cost for coding the block based on the given bits can be estimated. For example, encoding module 30 may estimate the total coding cost for coding a block in the mode selected using equations (2) or (3) above.

  Encoding module 30 determines whether there are other coding modes for which coding costs are to be estimated (70). As described above, the encoding module 30 estimates the coding cost for at least some of the possible modes. In certain aspects, encoding module 30 may estimate the cost of coding a block in each possible coding mode. For example, H.C. In the H.264 standard, the encoding module 30 determines that there are 22 different coding modes (interframe and intraframe coding modes) for blocks selected for interframe coding and 13 for blocks selected for intraframe coding. Coding costs for different coding modes can be estimated. In other aspects, encoding module 30 may use other mode selection techniques to reduce the first possible mode set and estimate the coding cost for the reduced set of coding modes. The techniques of this disclosure can be used.

  When there are additional coding modes for which coding costs are to be estimated, encoding module 30 selects the next coding mode and estimates the cost of coding data in the selected coding mode. When there are no additional coding modes for which coding costs are to be estimated, encoding module 30 selects one of the modes used to code the block based at least on the estimated coding costs (72). . In one example, coding module 30 may select the coding mode that has the lowest estimated coding cost. Once the mode is selected, coding module 30 may apply the selected mode to code a particular block (74). The process can continue for additional blocks within a given frame. As an example, the process can continue until all blocks in the frame have been coded using the coding mode selected by the techniques described herein. Furthermore, the process can continue until blocks of multiple frames are coded using the high efficiency mode.

  FIG. 5 is a flow diagram illustrating exemplary operation of an encoding module that estimates the number of bits associated with coding the residual coefficients of the block, eg, encoding module 30 of FIG. After selecting one of the coding modes for which the coding cost is to be estimated, the encoding module 30 generates block residual data for the selected mode (80). For example, for a block selected for intra-frame coding, spatial prediction module 34 generates residual data for that block based on comparing the block to a predicted version of the block. Alternatively, for a block selected for interframe coding, motion estimation module 36 and motion compensation module 38 calculate residual data for that block based on a comparison of the block with the corresponding block in the reference frame. To do. In some aspects, the residual data may have been calculated to generate a block distortion metric. In this case, the encoding module 30 can retrieve the residual data from the memory.

  The transform module 40 transforms the block residual coefficients according to the transform function to generate transform coefficients for the residual data (82). The transform module 40 applies, for example, 4 × 4 or 8 × 8 integer transform or DCT transform to the residual data to generate transform coefficients for the residual data. Bit estimation module 42 compares one of the transform coefficients with a corresponding threshold value to determine if the transform coefficient is greater than or equal to the threshold value (84). The threshold corresponding to the transform coefficient can be calculated as a function of the QP of the encoding module 30. If the transform coefficient is greater than or equal to the corresponding threshold, bit estimation module 42 identifies the transform coefficient as a coefficient that remains non-zero after quantization (86). If the transform coefficient is less than the corresponding threshold, bit estimation module 42 identifies the transform coefficient as a coefficient that becomes zero after quantization (88).

  Bit estimation module 42 determines whether there are additional transform coefficients for the residual data of the block (90). If there are additional transform coefficients for the block, bit estimation module 42 selects the other one of the coefficients and compares it to the corresponding threshold. If there are no additional transform coefficients to analyze, bit estimation module 42 determines the number of coefficients identified as remaining non-zero after quantization (92). Bit estimation module 42 also sums (94) the absolute values of at least some of the absolute values of the transform coefficients identified as remaining non-zero after quantization. Bit estimation module 42 estimates the number of bits associated with coding the residual data using the determined number of non-zero coefficients and the sum of the portions of the non-zero coefficients (96). Bit estimation module 42 may estimate the number of bits associated with coding the residual data using, for example, equation (4) above. In this manner, encoding module 30 estimates the number of bits associated with coding the residual data in the selected mode without quantizing or encoding the residual data of the block.

  FIG. 6 is a flow diagram illustrating exemplary operation of an encoding module that estimates the number of bits associated with coding the residual coefficients of the block, eg, encoding module 50 of FIG. After selecting one of the coding modes for which coding cost is to be estimated, the encoding module 50 generates a residual coefficient for the block (100). For example, for a block selected for intra-frame coding, the spatial prediction module 34 calculates residual data for the block based on comparing the block to a predicted version of the block. Alternatively, for a block selected for interframe coding, motion estimation module 36 and motion compensation module 38 calculate residual data for that block based on a comparison of the block with the corresponding block in the reference frame. To do. In some aspects, the residual coefficients may have been calculated to generate a block distortion metric.

  The transform module 40 transforms the block residual coefficients according to the transform function to generate transform coefficients for the residual data (102). The transform module 40 may generate a transformed residual coefficient by applying a 4 × 4 or 8 × 8 integer transform or DCT transform to the residual data, for example. The quantization module 46 quantizes the transform coefficient according to the QP of the encoding module 50 (104).

  Bit estimation module 52 determines the number of non-zero quantized transform coefficients (106). Bit estimation module 42 also sums the absolute values of the non-zero level or quantized transform coefficients (108). Bit estimation module 52 estimates the number of bits associated with coding the residual data using the calculated number of non-zero quantized transform coefficients and the sum of the non-zero quantized transform coefficients ( 110). Bit estimation module 52 may estimate the number of bits associated with coding the residual coefficient using, for example, equation (4) above. In this manner, the encoding module estimates the number of bits associated with coding the block residual data in the selected mode without encoding the residual data.

  Based on the teachings described herein, it should be apparent that the aspects disclosed herein can be implemented independently of other aspects and that two or more of these aspects can be combined in various ways. is there. The techniques described herein may be implemented in hardware, software, firmware, or any combination thereof. If implemented in hardware, these techniques can be implemented using digital hardware, analog hardware, or a combination thereof. If implemented in software, these techniques may be implemented at least in part by a computer program product that includes a computer-readable medium having instructions or codes stored thereon. The instructions or symbols associated with a computer readable medium of a computer program product may be transmitted by a computer, for example, one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, ASICs, FPGAs, It can be implemented by other equivalent integrated circuits or individual logic circuits.

  By way of example and not limitation, the computer readable medium may be RAM, such as synchronous dynamic random access memory (SDRAM), read only memory (ROM), non-volatile random access memory (NVRAM), ROM, Instructions or instructions for electrically erasable programmable read-only memory (EEPROM), EEPROM, FLASH memory, CD-ROM, or other optical disk storage, magnetic disk storage or other magnetic storage, or desired program code Any other tangible medium that can be used for transporting or storing in the form of a structure and accessible by a computer can be provided.

Several aspects and examples have been described. However, various modifications of these examples are possible, and the principles presented herein can be applied to other aspects as well. These and other aspects are within the scope of the following claims.
Hereinafter, the invention described in the scope of claims of the present application will be appended.
[C1]
A method for processing digital video data,
Identifying one or more transform coefficients for pixel block residual data that remains non-zero when quantized;
Estimating the number of bits associated with the coding of the residual data based at least on the identified transform coefficients;
Estimating a coding cost for coding the pixel block based on at least the estimated number of bits associated with coding the residual data.
[C2]
Identifying the transform coefficients comprises identifying the transform coefficients that remain non-zero when quantized by comparing each of the transform coefficients with a corresponding one of a plurality of threshold values. The method of C1, wherein each of the plurality of thresholds is calculated as a function of a quantization parameter (QP).
[C3]
Identifying each transform coefficient that remains non-zero when quantized by comparing each of the transform coefficients with a corresponding one of a plurality of threshold values is less than the corresponding threshold value The method of C2, comprising identifying the transform coefficients as transform coefficients that remain non-zero when quantized.
[C4]
Pre-calculating a plurality of sets of thresholds, each of the sets of thresholds corresponding to a different value of the QP;
The method of C2, further comprising: selecting one of the plurality of threshold sets based on the value of the QP used to encode the pixel block.
[C5]
Estimating the number of bits associated with coding the residual data is
Determining the number of said transform coefficients identified as remaining non-zero when quantized;
Summing the absolute value of at least one of the transform coefficients identified as remaining non-zero when quantized;
Estimating the number of bits associated with the coding of the residual data based on at least the determined number of non-zero transform coefficients and a sum of the absolute values of the at least one non-zero transform coefficient. The method according to C1.
[C6]
Estimating the number of bits associated with coding of the residual data comprises estimating the number of bits required to code the residual data in each of at least two block modes. Estimating the cost comprises, in each of the at least two block modes, estimating the coding cost based on at least the estimated number of bits in each one of the block modes, The method of C1, further comprising selecting one of the block modes based on at least the estimated coding cost for each of the modes.
[C7]
For each of the modes, estimating a total coding cost for coding the pixel block using at least the estimated number of bits associated with coding of the residual data;
Selecting a mode having the lowest estimated total coding cost among the plurality of modes;
Applying the selected mode to coding the pixel block; and the method of C6.
[C8]
Estimating the total coding cost is:
Calculating a distortion metric for the pixel block;
Calculating the number of bits associated with the coding of non-residual data for the pixel block;
The total coding cost for coding the pixel block based on at least the distortion metric, the number of bits associated with the coding of the non-residual data, and the number of bits associated with the coding of the residual data; Estimating the method according to C7.
[C9]
Selecting a coding coding based on at least the estimated number of bits associated with the coding of the residual data;
Quantizing the transform coefficients for the residual data after selecting the coding mode;
Encoding the quantized transform coefficients for the residual data;
Transmitting the encoded coefficients for the residual data; and the method of C1.
[C10]
Generating a matrix of transform coefficients, wherein the number of rows of the transform coefficient matrix is equal to the number of rows of pixels in the block, and the number of columns of the transform coefficient matrix is the number of columns of pixels in the block Is equal to
Comparing the transform coefficient matrix with a threshold matrix, the threshold matrix having the same dimensions as the transform coefficient matrix, and the comparison yields a matrix of 1 and zero; The zero represents a position in the transform coefficient matrix that becomes zero after quantization, and the 1 represents a position in the transform coefficient matrix that remains non-zero after quantization;
Summing the number of ones in the one and zero matrix to calculate the number of transform coefficients identified as remaining non-zero upon quantization;
Summing the absolute value of at least one of the transform coefficients in the transform coefficient matrix corresponding to the 1 position in the 1 and zero matrix;
The method of C1, further comprising: estimating the number of bits associated with coding of the residual data based on at least the number of non-zero transform coefficients and the sum of the at least one non-zero transform coefficient.
[C11]
An apparatus for processing digital video data,
A transform module that generates transform coefficients for the residual data of the pixel block;
Bits identifying one or more of the transform coefficients that remain non-zero when quantized and estimating a number of bits associated with coding of the residual data based at least on the identified transform coefficients An estimation module;
A control module for estimating a coding cost for coding the pixel block based on at least the estimated number of bits associated with coding the residual data.
[C12]
The bit estimation module identifies a transform coefficient that remains non-zero when quantized by comparing each of the transform coefficients with a corresponding one of a plurality of threshold values; Each of which is calculated as a function of a quantization parameter (QP).
[C13]
The apparatus of C12, wherein the bit estimation module identifies the transform coefficients that are less than a corresponding threshold as transform coefficients that remain non-zero when quantized.
[C14]
The bit estimation module pre-calculates a plurality of sets of thresholds, each of the threshold sets corresponding to a different value of the QP, and each of the threshold sets is of the QP The apparatus of C12, corresponding to different values and selecting one of the plurality of threshold sets based on the value of the QP used to encode the pixel block.
[C15]
The bit estimation module determines the number of transform coefficients that are identified as being non-zero when quantized and of the transform coefficients that are identified as remaining non-zero when quantized. The at least one absolute value of and summing the residual data based on at least the determined number of non-zero transform coefficients and the sum of the absolute values of the at least one non-zero transform coefficient The apparatus according to C11, which estimates the number of bits.
[C16]
The bit estimation module estimates the number of bits associated with coding of the residual data in each of at least two block modes;
The control module estimates a coding cost for each of the blocks based on at least the estimated number of bits in each one of the at least two block modes, and at least the estimated for each of the modes. The apparatus of C11, wherein one of the block modes is selected based on a coding cost.
[C17]
The control module estimates, for each of the modes, a total coding cost for coding the pixel block using at least the estimated number of bits associated with coding of the residual data, and the plurality of modes The apparatus of C16, wherein the mode with the lowest estimated total coding cost is selected and the selected mode is applied to code the pixel block.
[C18]
The control module calculates a distortion metric for the pixel block, calculates a number of bits associated with coding of the non-residual data of the pixel block, and is associated with at least the distortion metric, coding of the non-residual data. The apparatus of C17, wherein the total coding cost for coding the pixel block is estimated based on the number of bits and the number of bits associated with the coding of the residual data.
[C19]
A control module that selects a coding mode based on at least the estimated number of bits associated with coding the residual data;
A quantization module for quantizing the transform coefficient for the residual data after selection of the coding mode;
An entropy encoding module that encodes the quantized transform coefficients for the residual data;
The apparatus of C11, further comprising: a transmitter that transmits the encoded coefficients for the residual data.
[C20]
The transform module generates a matrix of transform coefficients, the number of rows of the transform coefficient matrix is equal to the number of rows of pixels in the block, and the number of columns of the transform coefficient matrix is the number of columns of pixels in the block. Equal to the number,
The bit estimation module compares the transform coefficient matrix with a threshold matrix, the threshold matrix having the same dimensions as the transform coefficient matrix, and the comparison yields a matrix of 1 and zero. The zero represents a position in the transform coefficient matrix that becomes zero after quantization, and the one represents a position in the transform coefficient matrix that remains non-zero after quantization;
The bit estimation module calculates the number of transform coefficients identified as remaining non-zero when quantized by summing the number of ones in the one-and-zero matrix; Sum the absolute values of at least one of the transform coefficients in the transform coefficient matrix corresponding to the one position in the matrix, and at least the number of non-zero transform coefficients and the at least one non-zero transform coefficient The apparatus of C11, wherein the number of bits associated with coding of the residual data is estimated based on a sum.
[C21]
An apparatus for processing digital video data,
Means for identifying one or more transform coefficients for pixel block residual data that remains non-zero when quantized;
Means for estimating a number of bits associated with coding of the residual data based at least on the identified transform coefficients;
Means for estimating a coding cost for coding the pixel block based on at least the estimated number of bits associated with coding the residual data.
[C22]
Said means for identifying said transform coefficients that remain non-zero when quantized by comparing each of said transform coefficients with a corresponding one of a plurality of threshold values; Each of C21 is computed as a function of a quantization parameter (QP).
[C23]
The apparatus of C22, wherein the means for identifying identifies the transform coefficients that are less than a corresponding threshold as transform coefficients that remain non-zero when quantized.
[C24]
Means for pre-calculating a plurality of sets of thresholds, wherein each of the threshold sets corresponds to a different value of the QP;
The apparatus of C22, further comprising: means for selecting one of the plurality of threshold sets based on the value of the QP used to encode the pixel block.
[C25]
The means for estimating determines a number of the transform coefficients that are identified as being non-zero when quantized and of the transform coefficients that are identified as being non-zero when quantized. At least one absolute value of and summed with the coding of the residual data based on at least the determined number of non-zero transform coefficients and the sum of the absolute values of the at least one non-zero transform coefficient The apparatus of C21, wherein the number of bits is estimated.
[C26]
The bit estimation means estimates the number of bits associated with the coding of the residual data in each of at least two block modes, and the coding cost estimation means includes one of each of the at least two block modes. Estimating a coding cost for each of the block modes based on at least the estimated number of bits in one, and determining one of the block modes based on at least the estimated number of bits for each of the block modes The apparatus of C21, further comprising means for selecting.
[C27]
And further comprising means for estimating a total coding cost for coding the pixel block using at least the estimated number of bits associated with coding of the residual data for each of the modes. The apparatus according to C26, wherein the means selects a mode having the lowest estimated total coding cost among the plurality of modes.
[C28]
The coding cost estimation means calculates a distortion metric for the pixel block, calculates a number of bits associated with coding of the non-residual data of the pixel block, and at least the distortion metric, coding of the non-residual data The apparatus of C27, wherein the total coding cost for coding the pixel block is estimated based on the number of bits associated with and the number of bits associated with coding of the residual data.
[C29]
Means for selecting a coding mode based on at least the estimated number of bits associated with coding of the residual data;
Means for quantizing the transform coefficients for the residual data after selecting the coding mode;
Means for encoding the quantized transform coefficients for the residual data;
The apparatus of C21, further comprising means for transmitting the encoded coefficients for the residual data.
[C30]
Means for generating a matrix of transform coefficients, wherein the number of rows of the transform coefficient matrix is equal to the number of rows of pixels in the block, and the number of columns of the transform coefficient matrix is the number of pixels in the block; Equal to the number of columns,
The identifying means compares the transform coefficient matrix with a threshold matrix, the threshold matrix having the same dimensions as the transform coefficient matrix, and the comparison is obtained as a matrix of 1 and zero. The zero represents a position in the transform coefficient matrix that becomes zero after quantization, and the one represents a position in the transform coefficient matrix that remains non-zero after quantization;
The means for estimating calculates the number of transform coefficients identified as remaining non-zero when quantized by summing the number of ones in the one-and-zero matrix; Sum the absolute values of at least one of the transform coefficients in the transform coefficient matrix corresponding to the one position in the matrix, and at least the number of non-zero transform coefficients and the at least one non-zero transform coefficient The apparatus of C21, wherein the number of bits associated with coding of the residual data is estimated based on a sum.
[C31]
A computer program product for processing digital video data comprising a computer readable medium having instructions stored thereon, the instructions comprising:
A code for identifying one or more transform coefficients for the residual data of the pixel block that remains non-zero when quantized;
A code for estimating a number of bits associated with coding of the residual data based at least on the identified transform coefficients;
A computer program product comprising: a code for estimating a coding cost for coding the pixel block based on at least the estimated number of bits associated with coding the residual data.
[C32]
The code for identifying the transform coefficient identifies a transform coefficient that remains non-zero when quantized by comparing each of the transform coefficients with a corresponding one of a plurality of threshold values, The computer program product according to C31, wherein each of the plurality of thresholds is calculated as a function of a quantization parameter (QP).
[C33]
A code for identifying a transform coefficient that remains non-zero when quantized by comparing each of the transform coefficients with a corresponding one of a plurality of threshold values is less than the corresponding threshold value The computer program product of C32 comprising a code for identifying the transform coefficient as a transform coefficient that remains non-zero when quantized.
[C34]
A code for pre-calculating a plurality of sets of thresholds, each of the threshold sets being a code corresponding to a different value of the QP;
A computer program product according to C32, further comprising: a code for selecting one of the plurality of threshold sets based on the value of the QP used to encode the pixel block. .
[C35]
The code for estimating the number of bits associated with the coding of the residual data is
A code for determining the number of transform coefficients identified as remaining non-zero when quantized;
A sign for summing the absolute values of at least one of the transform coefficients identified as remaining non-zero when quantized;
A code for estimating the number of bits associated with coding of the residual data based on at least the determined number of non-zero transform coefficients and the absolute value of the at least one non-zero transform coefficient; A computer program product according to C31, comprising:
[C36]
The code for estimating the number of bits associated with the coding of the residual data is a code for estimating the number of bits associated with the coding of the residual data in each of at least two block modes. And a code for estimating the coding cost estimates the coding cost for each of the at least two block nodes based on at least the estimated number of bits in each one of the block modes. A computer program product according to C31, further comprising a code for selecting one of the block modes based on at least the estimated number of bits for each of the block modes.
[C37]
For each of the modes, a code for estimating a total coding cost for coding the pixel block using at least the estimated number of bits associated with coding of the residual data;
A code for selecting a mode having the lowest estimated total coding cost among the plurality of modes;
A computer program product according to C36, further comprising: a code for applying the selected mode to code the pixel block.
[C38]
The code for estimating the total coding cost is:
A code for calculating a distortion metric for the pixel block;
A code for calculating the number of bits associated with the coding of the non-residual data of the pixel block;
Estimating the total coding cost for coding the pixel block based at least on the distortion metric, the number of bits associated with the coding of the non-residual data and the number of bits associated with the coding of the residual data And a computer program product according to C37.
[C39]
A code for selecting a coding mode based on at least the estimated number of bits associated with coding of the residual data;
A code for quantizing the transform coefficients for the residual data after selecting the coding mode;
A code for encoding the quantized transform coefficients for the residual data;
A computer program product according to C31, further comprising: a code for transmitting the encoded coefficients for the residual data.
[C40]
A code for generating a matrix of transform coefficients, wherein the number of rows of the transform coefficient matrix is equal to the number of rows of pixels in the block, and the number of columns of the transform coefficient matrix is the number of pixels in the block. A sign equal to the number of columns,
A code for comparing the transform coefficient matrix with a threshold matrix, the threshold matrix having the same dimension as the transform coefficient matrix, and the comparison is obtained as a matrix of 1 and zero. The zero represents a position in the transform coefficient matrix that becomes zero after quantization, and the 1 represents a position in the transform coefficient matrix that remains non-zero after quantization;
A code for calculating the number of transform coefficients identified as remaining non-zero when quantized by summing the number of ones in the one and zero matrix;
A sign for summing at least one absolute value of the transform coefficients in the transform coefficient matrix corresponding to the position of 1 in the 1 and zero matrices;
The code for estimating the number of bits associated with coding of the residual data based on at least the number of non-zero transform coefficients and the sum of the at least one non-zero transform coefficient, according to C31. Computer program product.

Claims (32)

A method for processing digital video data,
Identifying by the processor one or more transform coefficients for residual data of pixel blocks that are non-zero and remain non-zero when quantized, wherein said identifying is said one or more transform coefficients Executed without quantizing
Generating a matrix of transform coefficients, wherein the number of rows of the matrix of transform coefficients is equal to the number of rows of pixels in the block, and the number of columns of the matrix of transform coefficients is the number of pixels in the block. Equal to column,
Comparing the matrix of transform coefficients with a threshold matrix, wherein the threshold matrix has the same dimensions as the matrix of the transform coefficients matrix, and the comparison yields a matrix of 1 and 0; 0 represents a location in the matrix of transform coefficients that becomes zero after quantization, 1 represents a location in the matrix of transform coefficients that remains non-zero after quantization,
Adding the number of 1s in the matrix of 1s and 0s to calculate the number of transform coefficients identified as remaining non-zero when quantized;
Adding at least one absolute value of the transform coefficient in the matrix of transform coefficients corresponding to a location of 1 in the 1 and 0 matrix;
Estimating the number of bits associated with the coding of the residual data based on at least the number of non-zero transform coefficients and the addition of the at least one non-zero transform coefficient;
Estimating a coding cost for coding the pixel block based on the following equation:
J = D + λmode (Rresidual + Rother)
Where J is the estimated coding cost, D is the distortion metric of the block, λmode is the Lagrange multiplier for each of the multiple coding modes, and Residual is the coding for residual data using context adaptive coding techniques. Wherein Rother represents the number of bits for coding other block data using a fixed length coding (FLC) technique or a universal variable length coding (VLC) technique.   The method of claim 1, wherein each of the threshold values of the threshold matrix is calculated as a function of a quantization parameter (QP).   The method of claim 2, wherein locations in the matrix of transform coefficients that remain non-zero after quantization correspond to transform coefficients that satisfy respective corresponding thresholds. Calculating a plurality of threshold matrices in advance, each of the threshold matrices corresponding to a different value of the QP;
The method of claim 2, further comprising: selecting the threshold matrix from the plurality of threshold matrices based on the value of the QP used to encode the pixel block. In a device that processes digital video data,
A transform module for generating transform coefficients for residual data of a pixel block and generating a matrix of transform coefficients, wherein the number of rows of the matrix of transform coefficients is equal to the number of rows of pixels in the block; The number of columns of the matrix of transform coefficients is equal to the number of columns of pixels in the block;
Identifying one or more of the transform coefficients that are not quantized and non-zero and remain non-zero when quantized;
Comparing the matrix of transform coefficients with a threshold matrix, wherein the threshold matrix has the same dimensions as the matrix of the transform coefficients matrix, and the comparison yields a matrix of 1 and 0; 0 represents a location in the matrix of transform coefficients that becomes 0 after quantization, 1 represents a location in the matrix of transform coefficients that remains non-zero after quantization,
Adding the number of 1's in the 1's and 0's matrix and calculating the number of said transform coefficients identified to remain non-zero when quantized;
Adding at least one absolute value of the transform coefficients in the matrix of transform coefficients corresponding to a location of 1 in the 1 and 0 matrices;
Estimating the number of bits associated with coding of the residual data based on at least the number of non-zero transform coefficients and the addition of the at least one non-zero transform coefficient;
A bit estimation module;
A control module for estimating a coding cost for coding the pixel block based on the following equation:
J = D + λmode (Rresidual + Rother)
Where J is the estimated coding cost, D is the distortion metric of the block, λmode is the Lagrange multiplier for each of the multiple coding modes, and Residual is the coding for residual data using context adaptive coding techniques. Wherein Rother represents the number of bits for coding other block data using fixed length coding (FLC) or universal variable length coding (VLC) techniques.   6. The apparatus according to claim 5, wherein each threshold of the threshold matrix is calculated as a function of a quantization parameter (QP). The locations of the matrix of transform coefficients that remain non-zero after quantization correspond to transform coefficients that satisfy respective corresponding thresholds;
The apparatus according to claim 6.   The bit estimation module pre-calculates a plurality of sets of thresholds, each of the threshold sets corresponding to a different value of the QP and used to encode the pixel block. 7. The apparatus of claim 6, wherein one of the plurality of threshold sets to be used as the threshold matrix is selected based on the value of. In an apparatus for processing digital video data,
Means for identifying one or more transform coefficients for the residual data of a pixel block that is non-zero and not quantized and remains non-zero when quantized;
Means for generating a matrix of transform coefficients, wherein the number of rows of the matrix of transform coefficients is equal to the number of rows of pixels in the block, and the number of columns of the matrix of transform coefficients is the number of pixels in the block; Equal to the number of columns, the identification means compares the matrix of transform coefficients with a threshold matrix, the threshold matrix having the same dimensions as the matrix of the transform coefficients, and the comparison is between 1 and 0 Yielding a matrix, where 0 represents the location in the matrix of transform coefficients that is zero after quantization, and 1 represents the location in the matrix of transform coefficients that remains non-zero after quantization,
Calculating the number of transform coefficients identified as remaining non-zero when quantized by adding the number of 1s in the 1 and 0 matrices;
Adding at least one absolute value of the transform coefficients in the matrix of transform coefficients corresponding to a location of 1 in the 1 and 0 matrices;
Associated with the coding of the residual data by estimating the number of bits associated with the coding of the residual data based on the addition of at least the non-zero transform coefficients and the at least one non-zero transform coefficient. Means for estimating the number of bits taken;
Means for estimating a coding cost for coding the pixel block based on:
J = D + λmode (Rresidual + Rother)
Where J is the estimated coding cost, D is the distortion metric of the block, λmode is the Lagrange multiplier for each of the multiple coding modes, and Residual is the coding for residual data using context adaptive coding techniques. Wherein Rother represents the number of bits for coding other block data using fixed length coding (FLC) or universal variable length coding (VLC) techniques.   10. The apparatus of claim 9, wherein each of the threshold values of the threshold matrix is calculated as a function of a quantization parameter (QP).   10. The apparatus of claim 9, wherein a location of the matrix of transform coefficients that remains non-zero after quantization corresponds to a transform coefficient that satisfies a respective corresponding threshold. Means for pre-calculating a plurality of sets of thresholds, wherein each of the threshold sets corresponds to a different value of the QP;
Means for selecting one of the plurality of threshold sets for use in the threshold matrix based on the value of the QP used to encode the pixel block; The apparatus of claim 10, further comprising: A computer-readable recording medium having instructions stored thereon for processing digital video data that can be executed by a computer, the instructions comprising:
A code for identifying one or more transform coefficients for the residual data of the pixel block that is not quantized and non-zero and remains non-zero when quantized;
A code for generating the matrix of transform coefficients, wherein the number of rows of the matrix of transform coefficients is equal to the number of rows of pixels in the block, and the number of columns of the matrix of transform coefficients is Equal to the number of columns of
A code for comparing the matrix of transform coefficients with a threshold matrix, wherein the threshold matrix has the same dimensions as the matrix of the transform coefficients, and the comparison is a matrix of 1 and 0 Occurs, 0 represents a location in the matrix of transform coefficients that becomes zero after quantization, 1 represents a location in the matrix of transform coefficients that remains non-zero after quantization,
A code for adding the number of 1s in the 1 and 0 matrix to calculate the number of transform coefficients identified to remain non-zero when quantized;
A code for adding at least one absolute value of the transform coefficient in the matrix of transform coefficients corresponding to a location of 1 in the 1 and 0 matrix;
Code for estimating at least a few of the non-zero transform coefficients, and the number of bits associated with coding the residual data based on the addition of the transform coefficients the at least one nonzero,
A code for estimating a coding cost for coding the pixel block based on the following equation:
J = D + λmode (Rresidual + Rother)
Where J is the estimated coding cost, D is the distortion metric of the block, λmode is the Lagrange multiplier for each of the multiple coding modes, and Residual is the coding for residual data using context adaptive coding techniques. of the number of bits, Rother comprises, it represents the number of bits for coding the other block data by using a fixed length coding (FLC) technique or a universal variable length coding (VLC) technique, a computer-readable recording medium. Each of the threshold of the threshold matrix, a computer-readable recording medium of claim 13, which is calculated as a function of quantization parameter (QP). Remain non-zero the location of the matrix of transform coefficients corresponding to the transform coefficients that satisfy the respective corresponding threshold after quantization, a computer-readable recording medium according to claim 14. A code for pre-calculating a plurality of sets of thresholds, each of the sets of thresholds corresponding to a different value of the QP;
To select one of the plurality of threshold sets to be used to add the threshold matrix based on the value of the QP used to encode the pixel block the computer-readable medium of claim 14, further comprising a code, the.   Estimating the number of bits associated with the coding of the residual data comprises estimating the number of bits that the residual data needs to be encoded in each of at least two block modes, the coding Estimating the cost comprises estimating the coding cost in each of the at least two block modes based on at least the number of bits estimated in each one of the block modes, and at least the mode 2. The method of claim 1, comprising selecting one of the block modes based on the estimated coding cost for each of. For each of the modes, estimating a total coding cost for coding the pixel block using at least the estimated number of bits associated with coding of the residual data;
Selecting one of the plurality of modes having the least estimated total coding cost;
18. The method of claim 17, further comprising applying the selected mode to encode the pixel block. Estimating the total coding cost is
Calculating a distortion metric for the pixel block;
Calculating the number of bits associated with encoding the non-residual data of the pixel block;
The total coding cost for encoding the pixel block based at least on the distortion metric, the number of bits associated with encoding the non-residual data, and the number of bits associated with encoding the residual data Estimating
The method of claim 18 comprising: Selecting an encoding mode based on at least the estimated number of bits associated with encoding of the residual data;
After selecting the encoding mode, quantizing the transform coefficients for the residual data;
Encoding the quantized transform coefficients for the residual data;
Transmitting the encoded coefficients for the residual data;
The method of claim 1, further comprising: The bit estimation module estimates a number of bits associated with encoding of the residual data in each of at least two block modes;
The control module estimates a coding cost for each of the at least two block modes based at least on the estimated number of bits in each one of the block modes, and the estimated coding cost for each of the modes 6. The apparatus of claim 5, wherein one of the block modes is selected based at least on.   The control module estimates, for each of the modes, a total coding cost for encoding the pixel block using at least the estimated number of bits associated with the encoding of the residual data, with a minimum The apparatus of claim 21, wherein one of the plurality of modes having an estimated total coding cost is selected and the selected mode is applied to encode the pixel block.   The control module calculates a distortion metric for the pixel block, calculates a number of bits associated with the encoding of the non-residual data of the pixel block, and associates the distortion metric with the encoding of the non-residual data. 23. The apparatus of claim 22, wherein a total coding cost for encoding the pixel block is estimated based at least on the number of bits assigned and the number of bits associated with encoding the residual data. A control module that selects an encoding mode based at least on the estimated number of bits associated with encoding the residual data;
A quantization module that quantizes the transform coefficients for the residual data after selection of the encoding mode;
An entropy encoding module that encodes the quantized transform coefficients for the residual data;
A transmitter for transmitting the encoded coefficients for the residual data;
6. The apparatus of claim 5, further comprising: The means for estimating the number of bits associated with the coding of the residual data estimates the number of bits associated with the coding of the residual data in each of at least two block modes, and the coding cost estimation The means estimates a coding cost for each of at least two block modes based on at least the estimated number of bits in each one of the block modes, and based on at least the estimated number of bits for each of the modes 10. The apparatus of claim 9, further comprising means for selecting one of the block modes. For each of said modes, further comprising means for estimating the total coding cost for coding the pixel block using the number of bits the estimated associated with the encoding of at least said residual data, said mode It means for selecting one of, for selecting one of the plurality of modes having the smallest estimated total coding cost, apparatus according to claim 25. The means for estimating the total coding cost calculates a distortion metric for the pixel block, calculates a number of bits associated with encoding of the non-residual data of the pixel block, and at least the distortion metric, the non-residual 27. The apparatus of claim 26, wherein a total coding cost for encoding the pixel block is estimated based on a number of bits associated with encoding data and a number of bits associated with encoding residual data. Means for selecting a coding mode based on at least the estimated number of bits associated with coding of the residual data;
Means for quantizing the transform coefficients for the residual data after selecting the encoding mode;
Means for encoding the quantized transform coefficients for the residual data; and means for transmitting the encoded coefficients for the residual data;
10. The apparatus of claim 9, further comprising: The code for estimating the number of bits associated with the encoding of the residual data is a code for estimating the number of bits associated with the encoding of the residual data in each of at least two block modes. And the code for estimating the coding cost comprises a code for estimating the coding cost for each of the at least two block modes based on at least the estimated number of bits in each of the block modes. further comprises code for selecting one of the block mode on the basis of at least each number of bits the estimated regarding the mode, the computer-readable recording medium according to claim 13. For each of the modes, a code for estimating a total coding cost for encoding the pixel block using at least the estimated number of bits associated with encoding of the residual data;
A code for selecting one of the plurality of modes having a minimum estimated total coding cost, and a code for applying the selected mode to encode the pixel block;
Further comprising a computer readable medium of claim 29. The code for estimating the total coding cost is:
Code for calculating a distortion metric for the pixel block;
A code for calculating the number of bits associated with encoding the non-residual data of the pixel block;
Estimating the total coding cost for encoding the pixel block based on at least the distortion metric, the number of bits associated with the encoding of the non-residual data, and the number of bits associated with the encoding of the residual data And code to do
Comprising a computer readable medium of claim 30. A code for selecting an encoding mode based on at least the estimated number of bits associated with encoding of the residual data;
A code for quantizing the transform coefficient for the residual data after selecting the encoding mode;
The code of claim 13, further comprising: a code for encoding the quantized transform coefficient for the residual data; and a code for transmitting the encoded coefficient for the residual data. computer readable recording medium.

Images