JP5259828B2 - Video coding using transforms larger than 4x4 and 8x8 - Google Patents

Description

PRIORITY CLAIM The entire contents of each application are incorporated herein by reference. This application is filed on US Provisional Application No. 61/102783 filed Oct. 3, 2008 and May 18, 2009. Claims the benefit of US Provisional Patent Application No. 61 / 179,228.

  The present invention relates to encoding and decoding video data using a transform size larger than 8 × 8.

  Digital video functions include digital television, digital direct broadcast system, wireless communication device such as wireless telephone handset, wireless broadcast system, personal digital assistance (PDA), laptop or desktop computer, digital camera, digital recording device, video game device, It can be incorporated into a wide range of devices including video game consources. Digital video devices include MPEG-2, MPEG-4, H.264, etc. H.264 / MPEG-4, Part 10, Advanced Video Coding (AVC) and other video compression techniques are implemented to transmit and receive digital video more efficiently. Video compression techniques perform spatial and temporal prediction to reduce or eliminate redundancy inherent in video sequences.

  Video compression generally includes spatial prediction and / or temporal prediction. In particular, intra-screen coding reduces or eliminates spatial redundancy between video blocks within a given post-coding unit that may include video frames, slices of video frames, etc., by spatial prediction. In contrast, inter-frame coding reduces or eliminates temporal redundancy between video blocks in consecutive post-coding units of a video sequence by temporal prediction. For intra-picture encoding, the video encoder performs spatial prediction and compresses data based on other data in the same post-encoding unit. For inter-screen coding, the video encoder performs motion estimation and motion compensation to track the movement of two or more adjacent post-coding units of matching video blocks.

  After spatial or temporal prediction, a residual block is generated by subtracting the predicted video block generated during the prediction process from the original video block being encoded. Thus, the residual block indicates the difference between the prediction block and the current block being encoded. The video encoder may apply a transform process, a quantization process, and an entropy coding process to further reduce the bit rate associated with the transmission of the residual block. These conversion techniques can convert a set of pixel values into conversion coefficients that represent the energy of the pixel values in the frequency domain. Quantization is applied to the transform coefficients and generally involves a process that limits the number of bits associated with any given coefficient. Prior to entropy coding, the video encoder scans the quantized coefficient block and converts it to a one-dimensional vector of coefficients. Video encoder entropy encodes a quantized vector of transform coefficients to further compress the residual data.

  The video decoder can perform entropy decoding operations to capture the coefficients. A reverse scan can also be performed at the decoder to form a two-dimensional block from the received one-dimensional vector of coefficients. The video decoder then dequantizes and inverse transforms each coefficient to obtain a reconstructed residual block. The video decoder then decodes the predicted video block based on the prediction information including motion information. The video decoder then adds the predicted video block to the corresponding reconstructed residual block to generate a reconstructed video block and a decoded sequence of video information.

  Each of the systems, methods, and apparatus of the present application has several aspects, and none of the aspects relate only to its desired attributes. In the following, the salient features of the present application will be briefly discussed here without limiting the scope of the present application as represented by the claims. After reviewing this discussion and reading the section entitled “Detailed Description” in particular, the exemplary features of the present application show some improvements, including, for example, improved video coding efficiency, among others. It is understood whether it can be realized.

  In one embodiment, in a method for encoding video data, applying spatial prediction or motion compensation to an original video block in a video frame to generate a predicted video block based on a prediction mode; Indicating the selected transform, subtracting the predictive video block from the original video block in the video frame to form, selecting a transform having a first transform size to apply to the remaining blocks Generating header data, the header data comprising a first syntax element having a first value indicating at least one transform size and a second syntax element indicating a predicted block size of the predicted video block; So that the combined first syntax element and second syntax element indicate a first transform size. Generating a data to generate residual transform coefficients, a method comprising the applying the selected transform to the residual block, and generating a video signal based on the header data and the residual transform coefficients.

  In another embodiment, in a method for decoding video data, receiving a video signal indicative of at least one block in a frame of video comprising header data for at least one block and residual transform coefficients for at least one block. The header data comprises a first syntax element having a first value indicating at least one transform size and a second syntax element indicating a predicted block size of at least one block, together Receiving the video signal, wherein the first syntax element and the second syntax element are indicative of a transform having a first transform size used to encode at least one block. And less to generate a predictive video block with a predictive block size of at least one block Applying a spatial prediction or motion compensation to one block and a first transform size used to encode at least one block based on the first syntax element and the second syntax element. Determining, applying an inverse transform of the first transform size determined to obtain a decoded residual block to the residual transform coefficients, and decoding the residual block decoded to obtain a decoded video block There is a method comprising adding to a predicted video block.

  In another embodiment, in an apparatus for encoding video data, means for applying spatial prediction or motion compensation to an original video block in a video frame to generate a predicted video block based on a prediction mode And means for subtracting the predicted video block from the original video block in the video frame to form a residual block, and for selecting a transform having a first transform size to apply to the residual block Means for generating header data indicative of the selected transform, wherein the header data has a first syntax element having a first value indicative of at least one transform size and a prediction block of the prediction video block A second syntax element indicative of a size, wherein the first syntax element and the second syntax element together define a first transform size. Means for generating the header data, means for applying the selected transform to the residual block to generate a residual transform coefficient, and a video signal based on the header data and the residual transform coefficient. There are devices comprising means for generating.

  In another embodiment, in an apparatus for decoding video data, a video signal indicative of at least one block in a frame of video comprising header data for at least one block and a residual transform coefficient for at least one block. Means for receiving, wherein the header data comprises a first syntax element having a first value indicative of at least one transform size and a second syntax element indicative of a predicted block size of at least one block. The video signal, wherein the combined first and second syntax elements are indicative of a transform having a first transform size used to encode at least one block. Means for receiving and generating a predicted video block of predicted block size of at least one block And means for applying spatial prediction or motion compensation to at least one block and encoding at least one block based on the first syntax element and the second syntax element. Means for determining a first transform size, means for applying an inverse transform of the determined first transform size to a residual transform coefficient to obtain a decoded residual block, and a decoded residual block And a means for adding to the predicted video block and obtaining a decoded video block.

  In another embodiment, a system for encoding video data is configured to apply spatial prediction or motion compensation to an original video block in a video frame to generate a predicted video block based on a prediction mode. Selected residual unit, an adder configured to subtract the predicted video block from the original video block in the video frame to form a residual block, and a transform having a first transform size to select the residual A processor configured to apply to a block and generate header data indicative of a selected transformation, wherein the header data has a first value indicative of at least one transformation size and a first syntax element and prediction A second syntax element indicating a predicted block size of the video block, the first syntax element and the second syntax element; A processor that generates the header, together with a block transform unit configured to apply a selected transform to the residual block to generate a residual transform coefficient, such that There is a system comprising an entropy coding unit configured to generate a video signal based on the header data and residual transform coefficients.

  In another embodiment, in a system for decoding video data, a video signal indicative of at least one block in a frame of video comprising header data for at least one block and residual transform coefficients for at least one block. A receiver configured to receive, wherein the header data includes a first syntax element having a first value indicative of at least one transform size and a second indicative of a predicted block size of at least one block; A receiver comprising a syntax element, the combined first syntax element and the second syntax element indicating a transform having a first transform size used to encode at least one block; At least to generate a predicted video block of predicted block size of at least one block A prediction unit configured to apply spatial prediction or motion compensation to one block and a first used to encode at least one block based on the first syntax element and the second syntax element; A processor configured to determine a transform size of the first transform size, and an inverse transform unit configured to apply an inverse transform of the first transform size determined to obtain a decoded residual block to the residual transform coefficients There are systems that comprise an adder configured to add the decoded residual block to the predicted video block to obtain a decoded video block.

The block diagram which shows the transmission origin apparatus and transmission destination apparatus for encoding and decoding a video signal. FIG. 2 is a block diagram of an embodiment of the video encoder of FIG. 2 is a flowchart of one embodiment of a process for setting flag values to indicate the type of transform used by the encoder of FIG. 1 to the decoder of FIG. FIG. 6 is a flowchart of another embodiment of a process for setting flag values to indicate the type of transform used by the encoder of FIG. 1 to the decoder of FIG. FIG. 4 is a flowchart of one embodiment of a process for selecting the correct inverse transform for decoding video data encoded by the process of FIG. FIG. 5 is a flowchart of another embodiment of a process for selecting a correct inverse transform for decoding video data encoded by the process of FIG. FIG. 2 is a block diagram of an embodiment of the video decoder of FIG.

  The following detailed description is directed to certain specific embodiments. However, the teachings herein can be applied in many different ways. In this description, reference is made to the drawings wherein like parts are designated with like reference numerals.

  One embodiment is directed to transform size syntax elements for video encoding and decoding. By implementing a simplified set of transform selection rules and guidelines in the image and video signal encoding and decoding processes, it was possible to create a low bit rate syntax. As described above, the transform size syntax is a means for indicating a specific transform size in the encoder and a means for interpreting the transform size in the decoder. The transform size syntax element may be used to indicate the size of the transform to be used and may include a flag value comprising a number of bits. Note that in the following detailed description, the terms “video”, “image”, and “picture” may be used interchangeably. Accordingly, the scope of the various forms of the present invention should not be limited by the concept of differences between these terms.

  FIG. 1 is a block diagram illustrating a video encoding / decoding system 10 that implements the encoding techniques described in this disclosure. As shown in FIG. 1, the system 10 includes a source device 12 that transmits encoded video data to a destination device 14 via a communication channel 16. The source device 12 may include a video transmission device 18, a video encoder 20, and a transmitter 22. The video transmission device 18 of the source device 12 may include a video capture device, such as a video camera, a video archive containing pre-captured video, or a video feed from a video content provider. As a further alternative, the video transmitter 18 can generate computer graphics data as source video or a combination of live video and computer generated video. In some cases, the source device 12 may be a mobile phone or a video phone, in which case the video transmission device 18 may be a video camera mounted on the phone. In each case, the captured, pre-captured, or computer generated video code is transmitted from the source device 12 to the destination device 14 via the transmitter 22 and communication channel 16. It can be encoded by the device 20.

  Video encoder 20 receives video data from video transmitter 18. The video data received from the video transmission device 18 may be a series of video frames. Video encoder 20 divides a series of frames into coding units and processes these coding units to encode a series of video frames. A coding unit may be, for example, an entire frame or a part of a frame (eg a slice). Thus, in some cases, a frame can be divided into slices. Video encoder 20 divides each coding unit into blocks of pixels (referred to herein as video blocks or blocks) and processes the video blocks within individual coding units to encode video data. To do. Thus, a coding unit (eg, frame or slice) may include a plurality of video blocks. In other words, a video sequence may include multiple frames, a frame may include multiple slices, and a slice may include multiple video blocks.

  Each video block may have a constant size or a variable size, and may have a different size depending on the specified coding standard. As an example, International Telecommunication Union Telecommunication Standardization Sector (ITU-T) H.264. H.264 / MPEG-4, Part 10, Advanced Video Coding (AVC) (hereinafter “H.264 / MPEG-4 Part 10 AVC” standard) is 16 × 16 pixels, 8 × 8 pixels, or 4 × It supports intra prediction of various block sizes such as 4 pixels and 8 × 8 pixels for the saturation component. Intra-picture prediction has various block sizes such as 16 × 16 pixels, 16 × 8 pixels, 8 × 16 pixels, 8 × 8 pixels, 8 × 4 pixels, 4 × 8 pixels, and 4 × 4 pixels for luminance components. The saturation component can be executed at the corresponding scale size. H. In H.264, for example, each video block of 16 × 16 pixels is often called a macroblock (MB), and is subdivided into smaller-sized subblocks to perform intra-picture prediction or inter-picture prediction in units of subblocks. Can do. In general, MBs and various sub-blocks can be considered video blocks. Thus, an MB can be considered as a video block, and when partitioned or repartitioned, the MB itself can be considered to form a set of video blocks.

  For each video block, video encoder 20 selects the block type of the block. The block type can indicate whether this block is predicted using inter-picture prediction or intra-picture prediction and the predicted block size of the block. For example, H.M. H.264 / MPEG-4 Part 10 AVC standard is Inter 16 × 16, Inter 16 × 8, Inter 8 × 16, Inter 8 × 8, Inter 8 × 4, Inter 4 × 8, Inter 4 × 4, Intra 16 × 16 Supports several inter-picture and intra-picture prediction block types, including Intra 8 × 8 and Intra 4 × 4. As described in detail below, video encoder 20 may select one of the block types for each video block to be encoded.

  Video encoder 20 also selects a prediction mode for each video block. For intra-picture coded video blocks, the prediction mode can determine how to predict the current video block using one or more pre-coded video blocks. H. In the H.264 / MPEG-4 Part 10 AVC standard, for example, the video encoder 20 has nine possible unidirectional prediction modes for each Intra 4 × 4 block: vertical prediction mode, horizontal prediction mode, DC prediction mode, pair One of a corner lower left prediction mode, a diagonal lower right prediction mode, a vertical right prediction mode, a horizontal lower prediction mode, a vertical left prediction mode, and a horizontal upper prediction mode can be selected. A similar prediction mode is used to predict each Intra 8 × 8 block. For Intra 16 × 16 blocks, video encoder 20 selects one of four possible unidirectional prediction modes: vertical prediction mode, horizontal prediction mode, DC prediction mode, and planar prediction mode. Can do. In some cases, video encoder 20 may select a prediction mode from a set of prediction modes that includes not only a unidirectional prediction mode, but also one or more multi-directional prediction modes that define a combination of unidirectional modes. . For example, the one or more multi-directional prediction modes may be a bi-directional prediction mode that combines two unidirectional prediction modes.

  After selecting the prediction mode of the video block, the video encoder 20 generates a prediction video block using the selected prediction mode. The predicted video block is subtracted from the original video block to form a residual block. The residual block includes a set of pixel difference values as the difference between the pixel value of the original video block and the pixel value of the generated prediction block. The residual block can be represented in a two-dimensional block format (eg, a two-dimensional matrix or array of pixel difference values).

  Video encoder 20 may perform some other operations on the residual block after generating the residual block and before encoding the block. Video encoder 20 may apply a transform such as integer transform, DCT transform, directional transform, or wavelet transform to the residual block of pixel values to generate a block of transform coefficients. The transform coefficient may be a frequency domain representation of the residual block. Accordingly, the video encoder 20 converts the residual pixel value into a transform coefficient (also referred to as a residual transform coefficient). The residual transform coefficients can be called transform blocks or coefficient blocks. The residual transform coefficients may be a one-dimensional representation of the coefficients when a non-separable transform is applied, or may be a two-dimensional representation of the coefficients when a separable transform is applied. Non-separable transformations may include non-separable directional transformations. The separable transforms may include separable directional transforms, DCT transforms, integer transforms, and wavelet transforms.

  After conversion, video encoder 20 performs quantization to generate quantized transform coefficients (also called quantized coefficients or quantized residual coefficients). Again, the quantized coefficients can be expressed in a one-dimensional vector format or a two-dimensional block format. Quantization generally refers to the process of quantizing a coefficient and possibly reducing the amount of data used to represent the coefficient. The quantization process can reduce the bit depth associated with some or all of the coefficients. The term “coefficient” as used herein may represent a transform coefficient, a quantization coefficient, or other type of coefficient. The technique of the present disclosure can be applied to the residual pixel value, the quantized residual pixel value, the transform coefficient, and the quantized transform coefficient in some cases.

  When a separable transform is used and the coefficient block is represented in a two-dimensional block format, video encoder 20 scans the coefficients to convert from the two-dimensional format to the one-dimensional format. In other words, video encoder 20 can scan the coefficients obtained from the two-dimensional block and serialize the coefficients into a one-dimensional vector of coefficients. According to one aspect of the present disclosure, video encoder 20 may adjust the scan order used to transform coefficient blocks to one dimension based on collected statistics. The statistics may comprise an indication that a given coefficient value at each position of the two-dimensional block may be zero or non-zero, and for example, a count, probability or other associated with each coefficient position of the two-dimensional block Statistical criteria may be provided. In some cases, statistics can be collected only for a subset of the coefficient positions of a block. For example, if the scan order is evaluated after a certain number of blocks, the coefficient positions within the block that are determined to have a higher probability of having non-zero coefficients are determined to have a lower probability of having non-zero coefficients. The scanning order can be changed so that scanning is performed before the coefficient positions in the. In this way, the first scan order to group non-zero coefficients more efficiently in the first part of the one-dimensional coefficient vector and more efficiently group zero-value coefficients in the last part of the one-dimensional coefficient vector. Can be adapted. This shortens the zero run between the non-zero coefficients in the first part of the one-dimensional coefficient vector and places one longer run of zeros in the last part of the one-dimensional coefficient vector. The number of bits used for conversion can be reduced.

  Video encoder 20 scans the coefficients and then uses any of a variety of entropy encoding methods such as context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), run length coding, etc. Thus, each video block of the encoding unit is encoded. The transmission source device 12 transmits the encoded video data to the transmission destination device 14 via the transmitter 22 and the channel 16. Communication channel 16 may comprise any wireless or wired communication medium, such as a radio frequency (RF) spectrum, one or more physical transmission lines, or any combination of wireless and wired media. The communication channel 16 may form part of a global network such as a packet network such as a local area network, a wide area network, or the Internet. Communication channel 16 generally represents any suitable communication medium or collection of various communication media for transmitting encoded video data from source device 12 to destination device 14.

  The destination device 14 may include a receiver 24, a video decoder 26, and a display device 28. A receiver 24, which is a means for receiving a video signal, receives an encoded video bitstream from the transmission source device 12 via the channel 16. Video decoder 26 applies entropy decoding to decode the encoded video bitstream to obtain header information, motion vectors, and quantized residual coefficients of the encoded video block in post-encoding units. As described above, the quantization residual coefficient encoded by the transmission source device 12 is encoded as a one-dimensional vector. Accordingly, the video decoder 26 scans the quantized residual coefficients of the encoded video block and converts the one-dimensional vector of coefficients into a two-dimensional block of quantized residual coefficients. Video decoder 26, like video encoder 20, collects statistics indicating the probability that a given coefficient position in the video block may be zero or non-zero, thereby being used in the encoding process. You can adjust the scan order in the same way. Thus, a cross-adaptive scan order can be applied by the video decoder 26 to convert the serialized one-dimensional vector representation of the quantized transform coefficients back into a two-dimensional block of quantized transform coefficients.

  The video decoder 26 reconstructs each block of the coding unit using the decoded header information and the decoded residual information. In particular, video decoder 26 generates prediction video blocks for the current video block using prediction information and motion information included as part of the header information, and combines the prediction block with the corresponding residual video block for each video. Blocks can be reconfigured. The destination device 14 can display the reconstructed video block to the user via the display device 28. The display device 28 comprises any of a variety of display devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, a light emitting diode (LED) display, an organic LED display, and other types of display units. Good.

  In some cases, source device 12 and destination device 14 can operate substantially symmetrically. For example, source device 12 and destination device 14 may each include a video encoding component and a video decoding component. Thus, the system 10 can support one-way or two-way video transmission between devices 12, 14 for, for example, video streaming, video broadcasting, or videophone. An apparatus that includes a video encoding component and a video decoding component may form part of a typical encoded recording and playback device, such as a digital video recorder (DVR).

  The video encoder 20 and the video decoder 26 are ITU-T H.264, a standard defined by the Moving Picture Experts Group (MPEG) in MPEG-1, MPEG-2, and MPEG-4. H.263 standard, H.264. By the H.264 / MPEG4 Part 10 AVC standard, the American Film and Television Engineers Association (SMPTE) 421M Video CODEC standard (commonly referred to as “VC-1”), and the Audio Video Coding Standard Group of China (commonly referred to as “AVS”) It can operate according to any of a variety of video compression standards, such as defined standards, and any other video encoding standard that has been defined by a standards body or developed as a proprietary standard by an organization. Although not shown in FIG. 1, in some aspects, video encoder 20 and video decoder 26 may each be integrated with an audio encoder and decoder, respectively, and a common data stream or separate A suitable MUX-DEMUX unit or other hardware and software that handles both audio and video encoding in the data stream may be included. In this way, the transmission source device 12 and the transmission destination device 14 can process multimedia data. The MUX-DEMUX unit can be installed as required by ITU H.264. The H.223 multiplexer protocol or other protocols such as User Datagram Protocol (UDP) can be followed.

  In some aspects, in the case of video broadcast, the techniques described in this disclosure are extended H.264. Applying to H.264 video coding, Forward Link Only Air Interface Multi-Turrestrable Multi-Turrestrable Multi-Terristral Mestre Ref. Tera-1099 ("FLO Specification") published in July 2007 Can be used to provide real-time video services in terrestrial mobile multimedia multicast (TM3) systems. That is, the communication channel 16 may comprise a wireless information channel used to broadcast wireless video information according to the FLO specification or the like. The FLO specification includes examples defining bitstream syntax and semantics suitable for the FLO radio interface and the decoding process.

  Alternatively, the video can be broadcast according to other standards such as DVB-H (Digital Video Broadcast-Handheld), ISDB-T (Integrated Services Digital Broadcast-Terrestrial), or DMB (Digital Media Broadcast). Accordingly, the transmission source device 12 may be a mobile wireless terminal, a video streaming server, or a video broadcast server. However, the techniques described in this disclosure are not limited to any particular type of broadcast, multicast, or point-to-point system. In the case of broadcasting, the source device 12 can broadcast several channels of video data to a plurality of destination devices, each of which may be a device similar to the destination device 14 of FIG. Thus, although a single destination device 14 is shown in FIG. 1, for a video broadcast application, the source device 12 typically broadcasts video content to multiple destination devices simultaneously.

  In other examples, any wired or wireless communication system including one or more of Ethernet, telephone (eg, POTS), cable, power line, and fiber optic system, and / or code division multiple access ( CDMA or CDMA2000) communication system, frequency division multiple access (FDMA) system, orthogonal frequency division multiple access (OFDM) access system, GSM (Global System for Mobile Communication), GPRS (General Packet Radio Service), EDGE (extended) Time division multiple access (TDMA) system such as data GSM environment), TETRA (Terrestrial Trunked Radio) mobile phone system, wideband code division multiple access (WCDMA) system High data rate 1xEV-DO (First generation Evolution Data Only) or 1xEV-DO Gold Multicast system, IEEE 802.18 system, MediaFLO. TM. Transmitter 22, communication channel 16, and receive for communication by a wireless system including one or more of a system, DMB system, DVB-H system, or other scheme for data communication between two or more devices The device 24 can be configured.

  Video encoder 20 and video decoder 26 are each one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (EPGAs), discrete logic, software, hardware Hardware, firmware, or a combination thereof. Each of video encoder 20 and video decoder 26 can each be integrated as part of a composite encoder / decoder (CODEC) in a respective mobile device, subscriber device, broadcast device, server, etc. It can be included in one or more encoders or decoders. Also, source device 12 and destination device 14 each transmit and receive encoded video as needed, including radio frequency (RF) radio components and antennas sufficient to support wireless communication. Suitable modulation components, demodulation components, frequency conversion components, filtering components, and amplifier components. However, for ease of illustration, such components are briefly described as the transmitter 22 of the source device 12 and the receiver 24 of the destination device 14 of FIG.

  FIG. 2 is a block diagram illustrating an example of the video encoder 20. Video decoder 26 may include similar components as video encoder 20. Video encoder 20 may perform intra-picture and inter-picture coding of blocks within a video frame. Intra-picture coding reduces or eliminates spatial redundancy of video within a given video frame by spatial prediction. Inter-picture coding reduces or eliminates temporal redundancy of video in adjacent frames by temporal prediction. For inter-picture coding, video encoder 20 performs motion estimation to track the movement of matching video blocks between two or more adjacent frames.

  As shown in FIG. 2, video encoder 20 receives a current video block 21 in a video frame to be encoded. In the example of FIG. 2, the video encoder 20 includes a motion prediction unit 23, a reference frame store 25, a block transform unit 29, a quantization unit 31, an inverse quantization unit 33, an inverse transform unit 35, an entropy. An encoding unit 37, a mode determination unit 43, a spatial prediction unit 47, and a deblocking filter 49 are included. Video encoder 20 also includes adder 39, adder 41, and switch 51. The video encoder 20 may also include a scan unit (not shown) for scanning the quantized coefficients. FIG. 2 shows the temporal prediction component of video encoder 20 that inter-codes video blocks and the spatial prediction component that intra-codes video blocks. The switch 51 can be controlled by the mode determination unit 43 and can be used to select a spatial prediction video block or a temporal prediction video block as a prediction video block for an input video block.

  When estimating inter-picture coding, motion prediction unit 23 compares video block 21 with one or more blocks in adjacent video frames to generate one or more motion vectors. One or more adjacent frames can be captured from the reference frame store 25. Motion estimation can be performed on blocks of variable size, eg 16 × 16, 16 × 8, 8 × 16, 8 × 8, or smaller. Motion prediction unit 23 identifies the block in the adjacent frame that best matches the current video block 21 based on, for example, a rate distortion model, and determines the displacement between each block. Based on this, the motion prediction unit 23 creates a motion vector indicating the magnitude and locus of the displacement.

  Motion vectors may have half or quarter pixel accuracy, or even higher accuracy, and better as video encoder 20 tracks motion with higher accuracy than integer pixel locations. It is possible to obtain a predicted block. Using motion vectors having fractional pixel values, the motion prediction unit 23 can perform an interpolation operation. For example, AVC / H. In the H.264 standard, a 6-tap Wiener filter having a coefficient (1, -5, 20, 20, -5, 1) / 32 can be used when obtaining a luminance signal at a half pixel position. In obtaining the luminance signal at the quarter pixel position, bilinear filtering on the value at the integer pixel position and the interpolated value at the half pixel position can be used. Bilinear filters can also be used in fractional pixel interpolation of saturation components that may have up to 1/8 pixel accuracy. Motion prediction unit 23 outputs the predicted video block by motion compensation after identifying the best motion vector of the video block using the velocity distortion model.

  In an alternative, when evaluating intra-picture coding, spatial prediction unit 47 is used to form a predictive video block using previously coded blocks within the same coding unit (eg, the same frame). . For example, the video block 21 can be compared to other previously encoded blocks in the same frame as the video block 21. In some embodiments, already encoded blocks can be retrieved from the reference frame store 25. In some embodiments, various spatial prediction methods can be used. For example, H.M. In H.264 / MPEG-4 AVC, directional spatial prediction can be performed on video blocks of size 4x4, 8x8, and / or 16x16. In addition, a total of nine prediction directions can be used for 4 × 4 luminance blocks and 8 × 8 luminance blocks. A total of four prediction directions can be used for the 16 × 16 luminance block and the 16 × 16 saturation block. Other types of spatial prediction can be performed within the same coding unit. For example, a process similar to motion estimation can be used to identify a video block that matches the current video block within an already encoded portion of the current coding unit. Further, the amount of displacement between the matching video block and the current video block can be determined and then shown as part of the encoded video header data for the current video block. The mode determination unit 43 may select an optimal spatial prediction mode (eg, prediction block size, prediction direction, or displacement of the prediction video block) based on a predetermined criterion such as a Lagrange velocity distortion model. it can.

  The video encoder 20 is a prediction video created by the motion prediction unit 23 or the spatial prediction unit 47 from the original current video block 21 in the adder 39 which is a means for subtracting the prediction block from the original block. A residual video block is formed by subtracting the blocks. A block conversion unit 29, which is one means for applying the conversion, applies the conversion to the remaining blocks. The mode determination unit 43 can indicate to the block conversion unit 29 the size and type of conversion to be used. The quantization unit 31 quantizes the transform coefficient to further reduce the bit rate. An entropy encoding unit 37, which is a means for generating a video signal, entropy encodes the quantized coefficients to further reduce the bit rate. Video decoder 26 performs the inverse operation to reconstruct the encoded video.

  Inverse quantization unit 33 and inverse transform unit 35 reconstruct the residual block by applying inverse quantization and inverse transformation, respectively. The adder 41 adds the reconstructed residual block to the prediction block, and creates a reconstructed video block stored in the reference frame store 25. The reconstructed video block is used by the motion prediction unit 23 or the spatial prediction unit 47 to encode a subsequent video block in the current video frame or a subsequent video frame.

  In performing motion compensation on a given block in the current video frame 21, motion prediction unit 23 can interpolate the reference block obtained from the reference frame using a fixed set of filters. One reference block is required if the current block is predicted in one direction, and two reference blocks are required if the current block is predicted in two directions (bidirectional). H. In H.264, in some cases, multiple reference frames in the forward and reverse directions can be used. The actual filter used in the motion prediction unit 23 depends on the fractional part of the motion vector. For example, if the motion vector points to a half pixel position in a reference frame of a given dimension, to find the value of the half pixel position, (1, -5, 20, 20, -5, 1 A 6-tap filter such as) / 32 is used in the dimension along with a half-pixel motion vector. If both motion vector components point to integer positions, the pixel values obtained from the reference frame in the reference frame store 25 can be used directly without performing an interpolation filtering operation.

  FIG. 7 is a block diagram illustrating an example of the video decoder 26. The encoded bit stream is fed into the system 700. Each part of the bitstream corresponds to a different video block. In addition, some of these video blocks may constitute a single video frame. The portion of the bitstream corresponding to a given video block is entropy decoded by entropy decoding unit 702 to form a residual block comprising quantized residual transform coefficients. Next, the remaining blocks can be reversely scanned by a reverse scanning unit (not shown). The residual block may be dequantized by the inverse quantization unit 706 and inverse transformed by the inverse transform unit 708 to form a decoded residual block. The entropy decoding unit 702 can determine the type and / or size of the inverse transform to be performed based on the received header data, as described below. A predictive video block is generated and added to the decoded residual block at adder unit 710.

  One of two types of prediction methods can be used to form a predictive video block: an intra-picture prediction method and an inter-picture prediction method. Spatial prediction unit 716 generates an intra-picture prediction block using already coded blocks in the same video frame (or the same video slice if a video slice is used as the coding unit). Motion compensation unit 718 generates an inter-picture prediction block using the previous frame and / or the subsequent frame stored in reference frame store 720. Depending on the received header data indicating the encoding mode used to encode the video block, the spatial prediction unit 716 or the motion compensation unit 718 is invoked to generate an intra-picture prediction block or an inter-picture prediction block. The switch 722 can be switched. Next, the prediction block is added to the decoded residual block at an addition unit 710 to generate a decoded video block.

  The resulting reconstructed video block can then be deblocked filtered so that the video block can be filtered at the block edge to prevent blocking artifacts that can be visually detrimental. Transmitted to unit 712. The generated output is the decoded final video block. The decoded final video block can be stored in the reference frame store 720 so that other video blocks in the same or other video frames can be reconstructed.

  The decoder needs to know what kind of transform was used to encode the video data in order to properly decode the encoded video stream. The decoder may then apply an appropriate inverse transform corresponding to the forward transform used in the encoder. Therefore, it is necessary to properly decode the video block by sending data indicating the type of transform used to encode the video block to the decoder as part of the video bitstream.

  As described with respect to FIG. 2, block transform unit 29 applies transforms to the remaining video blocks. Applying the transform to the residual block achieves the desired energy concentration that enables high compression efficiency when combined with quantization and entropy coding. MPEG2 and H.264 Examples of transforms used in common block video coding systems such as H.264 / AVC include 8x8 DCT transforms and 4x4 and 8x8 integer transforms.

  H. The H.264 / AVC standard is the latest video coding standard that provides high coding efficiency. H. H.264 / AVC uses various types of block conversion. For blocks that have undergone intra-picture prediction (spatial prediction) and blocks that have undergone inter-picture prediction (temporal prediction). H.264 / AVC uses a 4 × 4 integer transform based on a 4 × 4 DCT transform or an 8 × 8 integer transform based on an 8 × 8 DCT transform.

  For the saturation signal of the video signal, an additional level of 2 × 2 Hadamard transform is applied to the 2 × 2 DC components in each block.

  For the luminance signal of the video signal, the conversion is selected as follows. First, it is determined whether the block is predicted within a picture or between pictures. If this block is predicted between images, it is next determined whether the block size is smaller than 8 × 8. If the block is smaller than 8x8, a 4x4 integer transform is used. If the block is 8x8 or greater, a 4x4 integer transform or an 8x8 integer transform is used.

  If the block is predicted in-picture, it is determined whether this block is predicted using the INTRA — 16 × 16 mode. If the block is predicted using the INTRA_16 × 16 mode, a 4 × 4 integer transform is applied to the block and an additional level of 4 × 4 Hadamard transform is applied to the 4 × 4 DC component in each block Is done. If the block is not predicted using the INTRA_16x16 mode, a 4x4 integer transform is used if the block is predicted using the INTRA_4x4 mode, and the block uses the INTRA_8x8 mode Therefore, 8 × 8 integer conversion is used.

  If a 4 × 4 or 8 × 8 transform can be used for the block, the transform selection is the H.264 in use. Depends on H.264 / AVC profile. Arbitrary H. other than high profile Under the H.264 profile (eg, baseline profile, extended baseline profile, main profile), only 4 × 4 integer transforms are used. H. Under the H.264 / AVC high profile (ie, fidelity range extension), an 8 × 8 integer transform based on the 8 × 8 DCT transform can also be used for the luminance signal. Whether to select 4 × 4 integer conversion or 8 × 8 integer conversion is indicated by an additional syntax element, conversion_size_8 × 8_flag. If a 4x4 transform or an 8x8 transform can be used (eg, an inter-coded block of size 8x8 or larger), the transform_size_8x8_flag is decoded along with the encoded video data Sent to the instrument. If the transform_size_8 × 8_flag is set to 1, then 8 × 8 integer transform is applied to the remaining block, otherwise (the transform_size_8 × 8_flag is set to 0), the remainder A 4 × 4 integer transform is applied to the block.

  H. In H.264 / AVC, motion is available for various block sizes (ie, motion partitions) such as 16 × 16, 16 × 8, 8 × 16, 8 × 8, 8 × 4, 4 × 8, and 4 × 4. Prediction can be performed. Typically, smaller motion partitions are used around areas of interest and multiple details, while larger motion partitions are usually selected around smoother areas. As a result, the residual blocks after motion prediction are also smoother than usual, i.e., those residual blocks tend to contain lower frequency components. For such signals, applying a larger transformation may result in better energy concentration. Methods for selecting motion partitions and transform sizes and encoders therefor are described in US Pat. No. 5,107,345, US Pat. No. 6,996,283, and US Pat. No. 6,600,286, all incorporated herein by reference. As mentioned above, H.M. H.264 / AVC uses only 4 × 4 integer transforms and 8 × 8 integer transforms for video blocks that have been inter-coded. The 4 × 4 integer transform and the 8 × 8 integer transform are indicated by the value of transform_size_8 × 8_flag, which is currently limited to 1 bit size. Therefore, since the 1-bit conversion_size_8 × 8_flag can indicate only two types of conversion, The current syntax used in H.264 cannot indicate additional transform sizes. The syntax and syntax elements that make it possible to indicate additional transform sizes used by the encoder and decoder are described below. In some embodiments, this syntax element comprises a 2-bit flag value indicating the transform size. The flag value can be included as part of the header information sent to the decoder.

  In the following embodiment, an inter-picture predicted video block or an intra-picture predicted video block may be used with the method described above. That is, a prediction block of a video block can be formed by motion compensation or spatial prediction. In embodiments that use motion compensation, the predicted block size is equal to the motion partition size, so the terms “predicted block” and the term “motion partition” can be used interchangeably. Further, in embodiments where spatial prediction is used, the prediction block size is equal to the size of the spatial prediction block used. Accordingly, the terms “prediction block” and the term “intra-picture prediction block” or the term “spatial prediction block” can be used interchangeably. For example, multiple transform options may be used for video blocks encoded using INTRA — 16 × 16 prediction and INTRA — 8 × 8 prediction. In addition to 4x4 transforms, 16x16 transforms, 16x8 transforms, 8x16 transforms, or 8x8 transforms can be applied to INTRA_16x16 predicted video blocks, and INTRA_8x8 predicted An 8 × 8 transform can be applied to the video block. For the intra-picture predicted block, the transform size can be indicated in the same manner as the inter-picture predicted video block. The transform size flag syntax element can be combined with the predicted block size syntax element, and variable length encoding of the transform size flag syntax element can be used.

  The syntax described below indicates the transform size using both the flag value and the predicted block size for a given block. Combining the predicted block size and flag value of the block can indicate more transform sizes than when using a one-to-one correspondence between flag values and transform sizes. For example, in a one-to-one correspondence between conversion size and flag value, a 2-bit flag only indicates four different conversion sizes, each flag value indicating a single conversion size. However, by further utilizing the predicted block size of the block, the same number of bits can be used for flags to indicate additional transform sizes. For example, if the flag value 00 indicates that the transform size should be equal to the predicted block size of the block, and the predicted block size may be a predicted block size of N different block sizes, a single flag value 00 can indicate N different transform sizes. Thus, in one embodiment, one or more of the flag values can indicate that the transform size in use is equal to the predicted block size of the block. In other embodiments, flag values can be encoded using variable length encoding.

  FIG. 3 is an exemplary embodiment of a process 300 for setting a flag value in the encoder indicating the transform size used by the encoder 20 for a given video block. The predicted block size of each video block can be determined by the mode determination unit 43 and the conversion can be performed by the block conversion unit 29 (see FIG. 2). Selection of the prediction block size and selection of the transform size used in a certain block can be performed by the mode determination unit 43. In a first step 302 of the process 300, it is determined whether the predicted block size for this given block is greater than 8x8. If the predicted block size is 8 × 8 or less, the process proceeds to step 306. Alternatively, if the predicted block size is greater than 8 × 8, the process proceeds to step 338.

  In step 306, it is determined whether the predicted block size is less than 8 × 8. If the predicted block size is less than 8x8, the process 300 proceeds to step 310 where a 4x4 transform is applied to the block. Process 300 then proceeds to step 314 where no flag value is set to be sent to the decoder. Alternatively, if it is determined in step 306 that the predicted block size is greater than or equal to 8 × 8, the process proceeds to step 318 where it is determined whether the transform size to be used in the block is 8 × 8. If it is determined that the 8 × 8 transform size should not be used, the process 300 proceeds to step 322 where a 4 × 4 transform is applied to the block, and then a 1-bit flag with a value of 0 is sent to the decoder. Proceed to step 326 which is set to be transmitted. In an alternative, it is determined in step 318 that an 8 × 8 transform should be used, and the process proceeds to step 330 where an 8 × 8 transform is applied to the block, and then one bit with a value of 1 Proceed to step 334 where the flag is set to be sent to the decoder.

  If it is determined at step 302 that the predicted block size is greater than 8 × 8, the process proceeds to step 338. At step 338, the encoder determines whether to use a transform size greater than 8 × 8 for this given block, either automatically or manually. If a transform size greater than 8x8 should not be used, process 300 proceeds to step 342 where it is determined whether the transform size to use for this given block is 8x8. If the transform size to use is not 8 × 8, the process 300 proceeds to step 346 where a 4 × 4 transform is applied to the block, so that a 1-bit flag value of 0 is sent to the decoder. Proceed to step 350 to be set. In the alternative, if the transform size to be used is 8 × 8, process 300 proceeds to step 354 where an 8 × 8 transform is applied to the block, and then a 2-bit flag value of 10 is sent to the decoder. Proceed to step 358 set as follows.

  If step 338 determines that the transform size to be used is greater than 8 × 8, process 300 proceeds to step 362. In step 362, it is determined whether the predicted block size of this given block is 16x16. If it is determined that the predicted block size is 16 × 16, the process 300 proceeds to step 366 where a 16 × 16 transform is applied to the block, and then proceeds to step 382. Alternatively, if it is determined at step 362 that the predicted block size is not 16 × 16, the process 300 proceeds to step 370 where it is determined whether the predicted block size is 8 × 16. If it is determined that the predicted block size is 8 × 16, the process 300 proceeds to the next step 374 where an 8 × 16 transform is applied to the block, and then proceeds to step 382. Alternatively, if it is determined that the predicted block size is not 8 × 16, the process 300 proceeds to the next step 374 where a 16 × 8 transform is applied to the block, and then proceeds to step 382. In step 382, the 2-bit flag value 11 is set to be sent to the decoder.

According to process 300, flag values correspond to the following conversion types:

  One skilled in the art will recognize that some of the steps of process 300 may be omitted or new steps added to achieve the same result. In addition, some of the steps can be performed in a different order. Note also that the flag values can be reconstructed (eg, 0 is an 8 × 8 transform and 10 is a 4 × 4 transform).

  FIG. 4 is an exemplary embodiment of another process 400 for setting a flag value at the encoder indicating the transform size used by the encoder 20 for a given video block. The predicted block size can be determined by the mode determination unit 43 and the conversion can be performed by the block conversion unit 29. Selection of the prediction block size and selection of the transform size used in a certain block are performed by the mode determination unit 43. In a first step 402 of process 400, it is determined whether the predicted block size for a given block is greater than 8x8. If the predicted block size is 8 × 8 or less, the process proceeds to step 406. Alternatively, if the predicted block size is greater than 8 × 8, the process proceeds to step 438.

  In step 406, it is determined whether the predicted block size is less than 8 × 8. If the predicted block size is less than 8x8, the process 400 proceeds to step 410 where a 4x4 transform is applied to the block. Process 400 then proceeds to step 414 where no flag value is set to be sent to the decoder. Alternatively, if it is determined in step 406 that the predicted block size is greater than or equal to 8 × 8, the process proceeds to step 418 where it is determined whether the transform size to be used in the block is 8 × 8. If it is determined that the 8 × 8 transform size should not be used, the process 400 proceeds to step 422 where a 4 × 4 transform is applied to the block, and then a 1-bit flag with a value of 0 is sent to the decoder. Proceed to step 426 set to transmit. In an alternative, it is determined in step 418 that an 8 × 8 transform should be used, and the process proceeds to step 430 where an 8 × 8 transform is applied to the block, and then one bit with a value of 1 Proceed to step 434 where the flag is set to be sent to the decoder.

  If it is determined at step 402 that the predicted block size is greater than 8 × 8, the process proceeds to step 438. At step 438, it is determined whether the predicted block size is 16x16. If the predicted block size is 16x16, the process 400 proceeds to step 442 where it is determined whether the transform size to apply to the block is 8x8. If it is determined that the transform size to be applied is 8 × 8, the process 400 proceeds to step 446 where the 8 × 8 transform is applied to this given block, and then a 2-bit flag with a value of 00. Proceeds to step 450 where is set to be sent to the decoder. Alternatively, if it is determined at step 442 that the transform size to be applied is not 8 × 8, the process 442 proceeds to step 454 where it is determined whether a 16 × 16 transform should be applied to the block. If a 16x16 transform is to be applied, the process 400 proceeds to step 458 where the 16x16 transform is applied to this given block, and then a 2-bit flag with a value of 01 is sent to the decoder. Proceed to step 462 set to: On the other hand, if it is determined in step 454 that the transform size to be applied is not 16 × 16, then the process 400 determines whether the transform size to be applied to this given block is 16 × 8. Proceed to 466. If the transform size to apply is 16 × 8, process 400 proceeds to step 470 where the 16 × 8 transform is applied to this given block, and then a 2-bit flag with a value of 10 is sent to the decoder. Proceed to step 474 which is set to do so. In an alternative, if it is determined in step 466 that the transform size to be applied to the block is not 16 × 8, the process 400 proceeds to step 478 where an 8 × 16 transform is applied to the given block; The process then proceeds to step 482 where a 2-bit flag with a value of 11 is set to be sent to the decoder.

  If it is determined at step 438 that the predicted block size is not 16 × 16, the process 400 proceeds to step 484 where it is determined whether the transform size to apply to this given block is 8 × 8. If the transform to be applied is 8x8, the process 400 proceeds to step 492 where the 8x8 transform is applied to the block and then set to send a 1-bit flag value 0 to the decoder. The process proceeds to step 426. In an alternative, if it is determined in step 484 that the transform size to be applied to the block is not 8 × 8, the process proceeds to step 486 where it is determined whether the predicted block size is 16 × 8. If the predicted block size is 16 × 8, process 400 proceeds to step 488 where a 16 × 8 transform is performed on the block, and then proceeds to step 434. In the alternative, if it is determined at step 486 that the predicted block size is not 16 × 8, the process 400 proceeds to step 490 where an 8 × 16 transform is performed on the block, and then proceeds to step 434. In step 434, a 1-bit flag with a value of 1 is set to be sent to the decoder.

According to process 400, the flag values correspond to the following conversion types:

  Those skilled in the art will recognize that some of the steps of process 400 may be omitted or new steps added to achieve the same result. In addition, some of the steps can be performed in a different order. Note also that the flag value can be reconstructed (eg, 00 is a 16 × 16 transform and 01 is an 8 × 8 transform).

  FIG. 5 is an exemplary embodiment of a process 500 for performing inverse transformations at the decoder 26 on blocks encoded by the encoder 20 using the process 300. Decoder 6, which may include an entropy decoding unit, a spatial prediction unit, a motion compensation unit, an inverse quantization unit, an inverse transform unit, an entropy decoding unit, and an adder as other components, performs the steps of process 500. It is one means to do. Further, various components of decoder 26 may be used to perform various steps of process 500. In step 502, it is determined whether the predicted block size is greater than 8x8. If the predicted block size is greater than 8 × 8, the process proceeds to step 518 where the decoder looks for a 1-bit or 2-bit flag value and performs an inverse transform based on the flag value and the predicted block size. The types of inverse transformation to be used are shown in Table 1. Alternatively, if it is determined at step 502 that the predicted block size is 8 × 8 or less, the process 500 proceeds to step 506 where it is determined whether the predicted block size is less than 8 × 8. If the predicted block size is less than 8x8, the process 500 proceeds to the next step 510 where a 4x4 inverse transform is performed. On the other hand, if it is determined in step 506 that the predicted block size is greater than or equal to 8 × 8, the process proceeds to step 514 where the decoder looks for a 1-bit flag value and performs an inverse transform based on the flag value. The types of inverse transformation to be used are shown in Table 1.

  FIG. 6 is an exemplary embodiment of a process 600 for performing inverse transforms at the decoder 26 on blocks encoded by the encoder 20 using the process 400. Decoder 26, which may be a processor, is a means for performing the steps of process 600 as another component. In step 602, it is determined whether the predicted block size is greater than 8x8. If the predicted block size is greater than 8x8, the process proceeds to step 618 where it is determined whether the predicted block size is 16x16. If the predicted block size is 16 × 16, process 600 proceeds to step 622 where the decoder looks for a 2-bit flag value and an inverse transform is performed on the block according to this flag value. The types of inverse transformation to be used are shown in Table 2. Alternatively, if it is determined in step 618 that the predicted block size is not 16 × 16, the process 600 causes the decoder to look for a 1-bit flag value and perform an inverse transform based on the 1-bit value and the motion partition size. Proceed to step 626. The types of inverse transformation to be used are shown in Table 2.

  If it is determined at step 602 that the predicted block size is 8 × 8 or less, the process 600 proceeds to step 606 where it is determined whether the predicted block size is less than 8 × 8. If the predicted block size is less than 8x8, the process 600 proceeds to the next step 610 where a 4x4 inverse transform is performed. On the other hand, if it is determined in step 606 that the predicted block size is greater than or equal to 8 × 8, the process proceeds to step 614 where the decoder looks for a 1-bit flag value and performs an inverse transform based on the flag value. The types of inverse transformation to be used are shown in Table 2.

  Processes 300, 400, 500, and 600 represent a specific syntax for determining the size of a transform to use for a block of video. Those skilled in the art will recognize that each process is merely an exemplary process for encoding and decoding each block and setting each flag value. Note that other processes including additional steps, fewer steps, or reconstructed steps can be used to achieve the same syntax as shown in Table 1 or Table 2. Further, those skilled in the art will recognize that the specific flag value assigned to each conversion indication can be changed. Also, a syntax similar to that shown in Tables 1 and 2 can be formed.

  Note also that additional transform sizes (eg, 32 × 32) and predicted block sizes (eg, 32 × 32) can be used to encode and decode each block and set each flag value. I want to be. For example, even if the flag uses only 2 bits for the flag value as described above, the flag can indicate a conversion size of 32 × 32. For example, in process 300, step 362 can determine whether the predicted block size is equal to 32 × 32, and step 370 can determine whether the predicted block size is equal to 16 × 32. Steps 366, 374, and 378 can then be modified so that a 32 × 32 transformation, a 16 × 32 transformation, or a 32 × 16 transformation is performed on the block at each step, respectively. Accordingly, the flag value set in step 358 indicates a 32 × 32, 16 × 32, or 32 × 16 conversion rather than a 16 × 16, 8 × 16, or 16 × 8 conversion. Additional modifications can be made to indicate additional transform sizes using combinations of flag values and predicted block sizes.

  Each bit of the flag value is transmitted along the communication channel 16 as part of the encoded video data. The arrangement of each bit of the flag value may be different along the transmitted bitstream depending on the encoding scheme. The flag value may be part of the header sent to the decoder. The header contains specific characteristics of the current video block such as block type, prediction mode, coded block pattern (CBP) for luminance and saturation, prediction block size, and one or more motion vectors. Additional header syntax elements that can be identified may be included. These header syntax elements can be generated, for example, by entropy encoding unit 37 in video encoder 20.

  In one embodiment, the header includes a bit that indicates whether there are non-zero coefficients in the encoded block. If non-zero coefficients are present, a bit indicating the transform size is also included in the header. If there are no non-zero coefficients, no transform size bits are transmitted. In other embodiments, a transform size element is transmitted in each header regardless of whether non-zero coefficients are present.

  The techniques described in this disclosure may be implemented in hardware, software, firmware, or any combination thereof. Any feature described as a unit or component may be implemented together as an integrated logic device or separately as discrete but interoperable logic devices. Each technique, when implemented in software, can be implemented at least in part by a computer-readable medium comprising instructions that when executed execute one or more of the above-described methods. The computer readable medium may form part of a computer program product that may include packaging material. Computer readable media include synchronous DRAM (SDRAM), read only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic or optical data A random access memory (RAM) such as a storage medium may be provided. In addition to or in the alternative, a computer readable medium capable of carrying or transmitting code in the form of instructions or data structures and accessed, read and / or executed by a computer, at least in part. It can be realized by a possible communication medium.

  The code is like one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits Can be executed by one or more processors. Thus, the term “processor” as used herein may refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. Also, in some aspects, the functionality described herein may be implemented in a dedicated software unit or hardware unit that is configured for encoding and decoding or that is incorporated into a composite video encoder decoder (CODEC). Can be provided. The various features are shown as units in order to emphasize various functional aspects of the illustrated apparatus, and it is not necessarily that such units must be implemented by separate hardware or software components. It is not a suggestion. Rather, functionality associated with one or more units can be integrated within a common or separate hardware or software component.

Various embodiments of the disclosure have been described. These and other embodiments are within the scope of the following claims.
The invention described in the scope of the claims at the beginning of the present application is added below.
(1)
In a method of encoding video data,
Applying spatial prediction or motion compensation to the original video block in the video frame to generate a predictive video block based on the prediction mode;
Subtracting the predicted video block from the original video block in the video frame to form a residual block;
Selecting a transform having a first transform size to apply to the residual block;
Generating header data indicating the selected transform, wherein the header data indicates a first syntax element having a first value indicating at least one transform size and a predicted block size of the predicted video block; Generating the header data such that the first syntax element and the second syntax element together comprise a second syntax element and the second syntax element indicates the first transformation size;
Applying the selected transform to the residual block to generate residual transform coefficients;
Generating a video signal based on the header data and the residual transform coefficients.
(2)
The header data further includes a third syntax element indicating an encoded block pattern, the third syntax element includes a second value, and the first syntax element includes a non-second value. The method according to (1), wherein if zero, the third syntax element is sequentially followed.
(3)
The method of (1), wherein the first value of the first syntax element corresponds to a plurality of transform sizes.
(4)
The method of (3), wherein the first value is mapped to the first transform size based on the predicted block size of the predicted video block.
(5)
The method of (3), wherein the first transform size is size X × Y, where X is not equal to Y.
(6)
The method of (5), wherein at least one of X and Y is equal to 8, and at least one of X and Y is equal to 16.
(7)
The method of (1), wherein the first transform size is equal to the predicted block size of the predicted video block.
(8)
The method according to (1), wherein the first transformation size is N × M, and at least one of M and N is 16 or more.
(9)
Generating header data indicative of the selected transformation;
Determining whether the predicted block size is greater than a first threshold;
The method of (1), comprising determining whether the predicted block size is less than a second threshold.
(10)
The method according to (9), wherein the first threshold value is 8 × 8 and the second threshold value is 8 × 8.
(11)
Generating header data indicative of the selected transformation;
Determining whether the predicted block size is greater than a first threshold;
The method of (1), comprising determining whether the predicted block size is equal to a second value.
(12)
The method according to (11), wherein the first threshold value is 8 × 8 and the second value is 16 × 16.
(13)
The method according to (11), wherein the first threshold value is 8 × 8 and the second value is 16 × 8.
(14)
The method of (1), wherein the selected transformation is an integer transformation.
(15)
The method of (1), wherein the selected transform is a discrete cosine transform.
(16)
The method of (1), wherein the selected transformation is a directional transformation.
(17)
In a method for decoding video data,
Receiving header data for at least one block and residual transform coefficients for the at least one block, receiving a video signal indicative of the at least one block in a frame of video, wherein the header data is A first syntax element having a first value indicating at least one transform size and a second syntax element indicating a predicted block size of the at least one block, the first syntax combined Receiving the video signal, wherein an element and the second syntax element indicate a transform having a first transform size used to encode the at least one block;
Applying spatial prediction or motion compensation to the at least one block to generate a predicted video block of the predicted block size of the at least one block;
Determining the first transform size used to encode the at least one block based on the first syntax element and the second syntax element;
Applying an inverse transform of the determined first transform size to the residual transform coefficients to obtain a decoded residual block;
A method comprising adding the decoded residual block to the predicted video block to obtain a decoded video block.
(18)
The header data further includes a third syntax element indicating an encoded block pattern, the third syntax element includes a second value, and the first syntax element includes the second value. The method according to (17), wherein when non-zero, the third syntax element is sequentially followed.
(19)
The method of (17), wherein the first value of the first syntax element corresponds to a plurality of transform sizes.
(20)
The method of (19), wherein the first value is mapped to the first transform size based on the predicted block size of the at least one block.
(21)
The method of (17), wherein the first transform size is size X × Y, where X is not equal to Y.
(22)
The method of (21), wherein at least one of X and Y is equal to 8, and at least one of X and Y is equal to 16.
(23)
The method of (17), wherein the first transform size is equal to the predicted block size of the at least one block.
(24)
The method according to (17), wherein the first transformation size is N × M, and at least one of M and N is 16 or more.
(25)
Determining the first transform size includes
Determining whether the predicted block size is greater than a first threshold;
Determining whether the predicted block size is less than a second threshold.
(26)
The method according to (25), wherein the first threshold value is 8 × 8 and the second threshold value is 8 × 8.
(27)
Determining the first transform size includes
Determining whether the predicted block size is greater than a first threshold;
The method of (17), comprising determining whether the predicted block size is equal to a second value.
(28)
The method according to (27), wherein the first threshold value is 8 × 8 and the second value is 16 × 16.
(29)
The method according to (27), wherein the first threshold value is 8 × 8 and the second value is 16 × 8.
(30)
The method according to (17), wherein the inverse transformation is an integer transformation.
(31)
The method according to (17), wherein the inverse transform is a discrete cosine transform.
(32)
The method according to (17), wherein the inverse transformation is a directional transformation.
(33)
In an apparatus for encoding video data,
Means for applying spatial prediction or motion compensation to the original video block in the video frame to generate a predictive video block based on the prediction mode;
Means for subtracting the predicted video block from the original video block in the video frame to form a residual block;
Means for selecting a transform having a first transform size to apply to the residual block;
Means for generating header data indicative of the selected transformation comprising:
The header data comprises a first syntax element having a first value indicative of at least one transform size and a second syntax element indicative of a prediction block size of the prediction video block, together with the first syntax element Means for generating the header data, wherein one syntax element and the second syntax element indicate the first transform size;
Means for applying the selected transform to the residual block to generate residual transform coefficients;
An apparatus comprising means for generating a video signal based on the header data and the residual transform coefficients.
(34)
The means for applying spatial prediction or motion compensation comprises a prediction unit, the means for subtracting comprises an adder, the means for selecting the transform size comprises a mode determination unit, and header data The means for generating comprises an entropy coding unit, the means for applying the selected transform comprises a block transform unit, and the means for generating a video signal comprises the entropy coding unit. (33) The apparatus.
(35)
In an apparatus for decoding video data,
Means for receiving a video signal indicative of said at least one block in a frame of video comprising header data for at least one block and a residual transform coefficient for said at least one block, comprising: The data comprises a first syntax element having a first value indicative of at least one transform size and a second syntax element indicative of a motion partition size of the at least one block, the first syntax element together Means for receiving the video signal, wherein the syntax element and the second syntax element indicate a transform having a first transform size used to encode the at least one block;
Means for applying spatial prediction or motion compensation to the at least one block to generate a predictive video block of the predictive block size of the at least one block;
Means for determining the first transform size used to encode the at least one block based on the first syntax element and the second syntax element;
Means for applying an inverse transform of the determined first transform size to the residual transform coefficients to obtain a decoded residual block;
An apparatus comprising means for adding the decoded residual block to the predicted video block to obtain a decoded video block.
(36)
The means for receiving comprises a receiver, the means for applying spatial prediction or motion compensation comprises a prediction unit, and the means for determining the first transform size comprises an entropy decoding unit; The apparatus of (35), wherein said means for applying an inverse transform comprises an inverse transform unit, and said means for adding comprises an adder.
(37)
In a system for encoding video data,
A prediction unit configured to apply spatial prediction or motion compensation to the original video block in the video frame to generate a predictive video block;
An adder configured to subtract the predicted video block from the original video block in the video frame to form a residual block;
A mode determination unit configured to select a transform having a first transform size to apply to the residual block;
A block transform unit configured to apply the selected transform to the residual block to generate a residual transform coefficient;
Generating header data indicating the selected transform, wherein the header data indicates a first syntax element having a first value indicating at least one transform size and a predicted block size of the predicted video block; A first syntax element, the combined first syntax element and the second syntax element indicate the first transform size; and
A system comprising an entropy coding unit configured to generate a video signal based on the header data and the residual transform coefficients.
(38)
The header data further includes a third syntax element indicating an encoded block pattern, the third syntax element includes a second value, and the first syntax element includes the second value. The system according to (37), wherein when non-zero, the third syntax element is sequentially followed.
(39)
The system of (37), wherein the first value of the first syntax element corresponds to a plurality of transform sizes.
(40)
The system of (39), wherein the first value is mapped to the first transform size based on the predicted block size of the predicted video block.
(41)
The system of (37), wherein the first transform size is size X × Y, where X is not equal to Y.
(42)
The system according to (41), wherein at least one of X and Y is equal to 8, and at least one of X and Y is equal to 16.
(43)
The system of (37), wherein the first transform size is equal to the predicted block size of the predicted video block.
(44)
The system according to (37), wherein the first transformation size is N × M, and at least one of M and N is 16 or more.
(45)
The entropy encoding unit is further configured to determine whether the predicted block size is greater than a first threshold and whether the predicted block size is less than a second threshold. The system according to (37).
(46)
The system according to (45), wherein the first threshold value is 8x8 and the second threshold value is 8x8.
(47)
The entropy encoding unit is further configured to determine whether the predicted block size is greater than a first threshold and whether the predicted block size is equal to a second value. (37).
(48)
48. The system of claim 47, wherein the first threshold is 8x8 and the second value is 16x16.
(49)
The system according to (47), wherein the first threshold value is 8 × 8 and the second value is 16 × 8.
(50)
The system of (37), wherein the selected transform is an integer transform.
(51)
The system of (37), wherein the selected transform is a discrete cosine transform.
(52)
The system of (37), wherein the selected transformation is a directional transformation.
(53)
In a system for decoding video data,
A receiver configured to receive a video signal indicative of the at least one block in a frame of video, comprising header data for at least one block and a residual transform coefficient for the at least one block. The header data comprises a first syntax element having a first value indicating at least one transform size and a second syntax element indicating a predicted block size of the at least one block, together The receiver wherein the first syntax element and the second syntax element are indicative of a transform having a first transform size used to encode the at least one block;
A prediction unit configured to apply spatial prediction or motion compensation to the at least one block to generate a prediction video block of the prediction block size of the at least one block;
An entropy decoding unit configured to determine the first transform size used to encode the at least one block based on the first syntax element and the second syntax element;
An inverse transform unit configured to apply an inverse transform of the determined first transform size to the residual transform coefficients to obtain a decoded residual block;
A system comprising an adder configured to add the decoded residual block to the predicted video block to obtain a decoded video block.
(54)
The header data indicates an encoded block pattern and further includes a third syntax element having a second value, and the first syntax element is sequentially updated when the second value is non-zero. The system according to (53), which follows the third syntax element.
(55)
The system according to (53), wherein the first value of the first syntax element corresponds to a plurality of transform sizes.
(56)
The system of claim 55, wherein the first value is mapped to the first transform size based on the predicted block size of the at least one block.
(57)
The system of (53), wherein the first transform size is size X × Y, where X is not equal to Y.
(58)
The system of (57), wherein at least one of X and Y is equal to 8, and at least one of X and Y is equal to 16.
(59)
The system of (53), wherein the first transform size is equal to the predicted block size of the at least one block.
(60)
The system according to (53), wherein the first transformation size is N × M, and at least one of M and N is 16 or more.
(61)
The entropy decoding unit is further configured to determine whether the predicted block size is greater than a first threshold and whether the predicted block size is less than a second threshold. The system according to (53).
(62)
The system of (61), wherein the first threshold is 8x8 and the second threshold is 8x8.
(63)
The entropy decoding unit is further configured to determine whether the predicted block size is greater than a first threshold and determine whether the predicted block size is equal to a second value. The system according to (53).
(64)
The system of (63), wherein the first threshold value is 8x8 and the second value is 16x16.
(65)
The system of (63), wherein the first threshold value is 8x8 and the second value is 16x8.
(66)
The system according to (53), wherein the inverse transform is an integer transform.
(67)
The system according to (53), wherein the inverse transform is a discrete cosine transform.
(68)
The system according to (53), wherein the inverse transformation is a directional transformation.
(69)
At runtime,
Applying spatial prediction or motion compensation to the original video block in the video frame to generate a predictive video block based on the prediction mode;
Subtracting the predicted video block from the original video block in the video frame to form a residual block;
Selecting a transform having a first transform size to apply to the residual block;
Generating header data indicating the selected transform, wherein the header data includes a first syntax element having a first value indicating at least one transform size and a predicted block size of the predicted video block; Generating the header data such that together the first syntax element and the second syntax element indicate the first transform size, the second syntax element indicating:
Applying the selected transform to the residual block to generate a residual transform coefficient;
A computer readable medium comprising instructions for performing a method comprising generating a video signal based on the header data and the residual transform coefficients.
(70)
At runtime,
Receiving a video signal indicative of the at least one block in a frame of video comprising header data for at least one block and a residual transform coefficient for the at least one block, wherein the header data comprises: A first syntax element having a first value indicating at least one transform size and a second syntax element indicating a predicted block size of the at least one block, the first syntax combined Receiving the video signal, wherein an element and the second syntax element indicate a transform having a first transform size used to encode the at least one block;
Applying spatial prediction or motion compensation to the at least one block to generate a predicted video block of the predicted block size of the at least one block;
Determining the first transform size used to encode the at least one block based on the first syntax element and the second syntax element;
Applying an inverse transform of the determined first transform size to the residual transform coefficients to obtain a decoded residual block;
A computer readable medium comprising instructions for performing a method comprising adding the decoded residual block to the predicted video block to obtain a decoded video block.

Claims (62)

In a method for encoding video data by an encoder , the method comprises:
Applying spatial prediction or motion compensation to the original video block in the video frame to generate a predictive video block based on the prediction mode;
Subtracting the predicted video block from the original video block in the video frame to form a residual block;
Selecting a transform having a first transform size to apply to the residual block;
Applying the selected transform to the residual block to generate a residual transform coefficient;
Generating header data indicating the selected transformation, comprising:
The header data comprises a first syntax element having a first value indicating at least three transform sizes and a second syntax element indicating a prediction block size of the prediction video block , wherein the at least three The transform size comprises at least one N × M transform size, and at least one of M and N is greater than or equal to 16, where the first only when combined with the second syntax element The first value of the syntax element indicates the first transform size, where the header data further includes a third syntax element indicating whether the residual transform coefficient includes one or more non-zero coefficients. Prepare
Generating the header data ;
Generating a video signal based on the header data and the residual transform coefficient;
A method comprising: The method of claim 1, wherein the third syntax element comprises a second value, and wherein the first syntax element sequentially follows the third syntax element when the second value is non-zero. .   The method of claim 1, wherein the first value of the first syntax element corresponds to a plurality of transform sizes.   The method of claim 3, wherein the first value is mapped to the first transform size based on the predicted block size of the predicted video block.   The method of claim 1, wherein the first transform size is size X × Y, where X is not equal to Y.   6. The method of claim 5, wherein at least one of X and Y is equal to 8, and at least one of X and Y is equal to 16.   The method of claim 1, wherein the first transform size is equal to the predicted block size of the predicted video block. Generating header data indicative of the selected transformation;
Determining whether the predicted block size is greater than a first threshold;
The method of claim 1, comprising determining whether the predicted block size is less than a second threshold.   The method of claim 8, wherein the first threshold is 8 × 8 and the second threshold is 8 × 8. Generating header data indicative of the selected transformation;
Determining whether the predicted block size is greater than a first threshold;
The method of claim 1, comprising determining whether the predicted block size is equal to a second value.   The method of claim 10, wherein the first threshold is 8 × 8 and the second value is 16 × 16.   The method of claim 10, wherein the first threshold is 8 × 8 and the second value is 16 × 8.   The method of claim 1, wherein the selected transform is an integer transform.   The method of claim 1, wherein the selected transform is a discrete cosine transform. In a method for decoding video data by a decoder , the method comprises:
Receiving header data for at least one block and residual transform coefficients for the at least one block, receiving a video signal indicative of the at least one block in a frame of video, wherein the header data is A first syntax element having a first value indicative of at least three transform sizes and a second syntax element indicative of a predicted block size of the at least one block , wherein the at least three transform sizes are , At least one N × M transform size, and at least one of M and N is greater than or equal to 16, wherein only when combined with the second syntax element is the first syntax element The first value indicates the first transform size, wherein the header data further includes the residual transform coefficient A third syntax element indicating whether to include one or more non-zero coefficients, the method comprising: receiving the video signal,
Applying spatial prediction or motion compensation to the at least one block to generate a predicted video block of the predicted block size of the at least one block;
Determining the first transform size used to encode the at least one block based on the first syntax element and the second syntax element,
Applying an inverse transform of the determined first transform size to the residual transform coefficients to obtain a decoded residual block;
A method comprising adding the decoded residual block to the predicted video block to obtain a decoded video block. Said header data, said third syntax element comprising a second value, the first syntax element, when the second value is non-zero, followed after sequential said third syntax element The method according to claim 15.   The method of claim 15, wherein the first value of the first syntax element corresponds to a plurality of transform sizes.   The method of claim 17, wherein the first value is mapped to the first transform size based on the predicted block size of the at least one block.   The method of claim 15, wherein the first transform size is size X × Y, where X is not equal to Y.   20. The method of claim 19, wherein at least one of X and Y is equal to 8, and at least one of X and Y is equal to 16.   The method of claim 15, wherein the first transform size is equal to the predicted block size of the at least one block. Determining the first transform size includes
Determining whether the predicted block size is greater than a first threshold;
16. The method of claim 15, comprising determining whether the predicted block size is less than a second threshold.   23. The method of claim 22, wherein the first threshold is 8x8 and the second threshold is 8x8. Determining the first transform size includes
Determining whether the predicted block size is greater than a first threshold;
16. The method of claim 15, comprising determining whether the predicted block size is equal to a second value.   25. The method of claim 24, wherein the first threshold is 8x8 and the second value is 16x16.   25. The method of claim 24, wherein the first threshold is 8x8 and the second value is 16x8.   The method of claim 15, wherein the inverse transform is an integer transform.   The method of claim 15, wherein the inverse transform is a discrete cosine transform. In an apparatus for encoding video data,
Means for applying spatial prediction or motion compensation to the original video block in the video frame to generate a predictive video block based on the prediction mode;
Means for subtracting the predicted video block from the original video block in the video frame to form a residual block;
Means for selecting a transform having a first transform size to apply to the residual block;
Means for applying the selected transform to the residual block to generate a residual transform coefficient;
Means for generating header data indicative of the selected transformation comprising:
The header data comprises a first syntax element having a first value indicating at least three transform sizes and a second syntax element indicating a prediction block size of the prediction video block , wherein the at least three The transform size comprises at least one N × M transform size, and at least one of M and N is greater than or equal to 16, where the first only when combined with the second syntax element The first value of the syntax element indicates the first transform size, where the header data further includes a third syntax element indicating whether the residual transform coefficient includes one or more non-zero coefficients. comprising, means for generating the header data,
An apparatus comprising means for generating a video signal based on the header data and the residual transform coefficients.   The means for applying spatial prediction or motion compensation comprises a prediction unit, the means for subtracting comprises an adder, the means for selecting the transform size comprises a mode determination unit, and header data The means for generating comprises an entropy coding unit, the means for applying the selected transform comprises a block transform unit, and the means for generating a video signal comprises the entropy coding unit. 30. The apparatus of claim 29. In an apparatus for decoding video data,
Means for receiving a video signal indicative of said at least one block in a frame of video comprising header data for at least one block and a residual transform coefficient for said at least one block, comprising: The data comprises a first syntax element having a first value indicative of at least three transform sizes and a second syntax element indicative of a predicted block size of the predictive video block , wherein the at least three transform sizes Comprises at least one N × M transform size, and at least one of M and N is greater than or equal to 16, wherein the first syntax element only when combined with the second syntax element The first value of the first transform size used to encode the at least one block Shown, wherein the header data further comprises a third syntax element in which the residual transform coefficients indicate whether to include one or more non-zero coefficients, and means for receiving said video signal,
Means for applying spatial prediction or motion compensation to the at least one block to generate a predictive video block of the predictive block size of the at least one block;
It means for determining on the basis of the first syntax element and the second syntax element, wherein the at least one block is used to encode the first transform size,
Means for applying an inverse transform of the determined first transform size to the residual transform coefficients to obtain a decoded residual block;
An apparatus comprising means for adding the decoded residual block to the predicted video block to obtain a decoded video block.   The means for receiving comprises a receiver, the means for applying spatial prediction or motion compensation comprises a prediction unit, and the means for determining the first transform size comprises an entropy decoding unit; 32. The apparatus of claim 31, wherein the means for applying an inverse transform comprises an inverse transform unit, and the means for adding comprises an adder. In a system for encoding video data,
A prediction unit configured to apply spatial prediction or motion compensation to the original video block in the video frame to generate a predictive video block;
An adder configured to subtract the predicted video block from the original video block in the video frame to form a residual block;
A mode determination unit configured to select a transform having a first transform size to apply to the residual block;
A block transform unit configured to apply the selected transform to the residual block to generate a residual transform coefficient;
An entropy encoding unit that generates header data indicative of the selected transform;
The header data comprises a first syntax element having a first value indicating at least three transform sizes and a second syntax element indicating a prediction block size of the prediction video block , wherein the at least three The transform size comprises at least one N × M transform size, and at least one of M and N is greater than or equal to 16, where the first only when combined with the second syntax element The first value of the syntax element indicates the first transform size, and is configured to generate a video signal based on the header data and the residual transform coefficient , wherein the header data further includes The entropy coding unit comprising a third syntax element that indicates whether the residual transform coefficients include one or more non-zero coefficients ;
A system comprising: The header data includes pre Symbol third syntax element comprising a second value, the first syntax element, when the second value is non-zero, followed by sequential the third syntax element 34. The system of claim 33.   34. The system of claim 33, wherein the first value of the first syntax element corresponds to a plurality of transform sizes.   36. The system of claim 35, wherein the first value is mapped to the first transform size based on the predicted block size of the predicted video block.   34. The system of claim 33, wherein the first transform size is size XxY, where X is not equal to Y.   38. The system of claim 37, wherein at least one of X and Y is equal to 8, and at least one of X and Y is equal to 16.   34. The system of claim 33, wherein the first transform size is equal to the predicted block size of the predicted video block.   The entropy encoding unit is further configured to determine whether the predicted block size is greater than a first threshold and whether the predicted block size is less than a second threshold. 34. The system of claim 33, wherein:   41. The system of claim 40, wherein the first threshold is 8x8 and the second threshold is 8x8.   The entropy encoding unit is further configured to determine whether the predicted block size is greater than a first threshold and whether the predicted block size is equal to a second value. 34. The system of claim 33.   43. The system of claim 42, wherein the first threshold is 8x8 and the second value is 16x16.   43. The system of claim 42, wherein the first threshold is 8x8 and the second value is 16x8.   34. The system of claim 33, wherein the selected transform is an integer transform.   34. The system of claim 33, wherein the selected transform is a discrete cosine transform. In a system for decoding video data,
A receiver configured to receive a video signal indicative of the at least one block in a frame of video, comprising header data for at least one block and a residual transform coefficient for the at least one block. And
The header data comprises a first syntax element having a first value indicating at least three transform sizes and a second syntax element indicating a prediction block size of the prediction video block , wherein the at least three The transform size comprises at least one N × M transform size, and at least one of M and N is greater than or equal to 16, where the first only when combined with the second syntax element The first value of the syntax element indicates a first transform size used to encode the at least one block, wherein the header data further includes one or more non-residue transform coefficients. The receiver comprising a third syntax element indicating whether to include a zero coefficient ;
A prediction unit configured to apply spatial prediction or motion compensation to the at least one block to generate a prediction video block of the prediction block size of the at least one block;
An entropy decoding unit configured to determine the first transform size used to encode the at least one block based on the first syntax element and the second syntax element;
An inverse transform unit configured to apply an inverse transform of the determined first transform size to the residual transform coefficients to obtain a decoded residual block;
A system comprising an adder configured to add the decoded residual block to the predicted video block to obtain a decoded video block. It said header data comprises further a third syntax element comprising a second value, the first syntax element, when the second value is non-zero, followed by sequential the third syntax element 48. The system of claim 47.   48. The system of claim 47, wherein the first value of the first syntax element corresponds to a plurality of transform sizes.   50. The system of claim 49, wherein the first value is mapped to the first transform size based on the predicted block size of the at least one block.   48. The system of claim 47, wherein the first transform size is size XxY, where X is not equal to Y.   52. The system of claim 51, wherein at least one of X and Y is equal to 8, and at least one of X and Y is equal to 16.   48. The system of claim 47, wherein the first transform size is equal to the predicted block size of the at least one block.   The entropy decoding unit is further configured to determine whether the predicted block size is greater than a first threshold and whether the predicted block size is less than a second threshold. 54. The system of claim 53.   55. The system of claim 54, wherein the first threshold is 8x8 and the second threshold is 8x8.   The entropy decoding unit is further configured to determine whether the predicted block size is greater than a first threshold and determine whether the predicted block size is equal to a second value. 48. The system of claim 47.   57. The system of claim 56, wherein the first threshold is 8x8 and the second value is 16x16.   57. The system of claim 56, wherein the first threshold is 8x8 and the second value is 16x8.   48. The system of claim 47, wherein the inverse transform is an integer transform.   48. The system of claim 47, wherein the inverse transform is a discrete cosine transform. A computer-readable storage medium that stores instructions for one or more computers to perform a process, said instructions causing a method to be executed;
Based on the instructions, the method includes:
Applying spatial prediction or motion compensation to the original video block in the video frame to generate a predictive video block based on the prediction mode;
Subtracting the predicted video block from the original video block in the video frame to form a residual block;
Selecting a transform having a first transform size to apply to the residual block;
Applying the selected transform to the residual block to generate a residual transform coefficient;
Generating header data indicating the selected transformation, comprising:
The header data comprises a first syntax element having a first value indicating at least three transform sizes and a second syntax element indicating a prediction block size of the prediction video block , wherein the at least three The transform size comprises at least one N × M transform size, and at least one of M and N is greater than or equal to 16, where the first only when combined with the second syntax element The first value of the syntax element indicates the first transform size; and
Generating a video signal based on the header data and the residual transform coefficient ;
The computer-readable storage medium. A computer-readable storage medium that stores instructions for one or more computers to perform a process, said instructions causing a method to be executed;
Based on the instructions, the method includes:
Receiving a video signal indicative of the at least one block in a frame of video comprising header data for at least one block and a residual transform coefficient for the at least one block, wherein the header data comprises: , A first syntax element having a first value indicating at least three transform sizes, and a second syntax element indicating a predicted block size of the predicted video block , wherein the at least three transform sizes are: At least one N × M transform size, and at least one of M and N is greater than or equal to 16, wherein the first syntax element of the first syntax element only when combined with the second syntax element A first value indicates a first transform size used to encode the at least one block; Wherein said header data further includes said residual transform coefficients comprises a third syntax element indicating whether to include one or more non-zero coefficients, for receiving said video signal,
Applying spatial prediction or motion compensation to the at least one block to generate a predicted video block of the predicted block size of the at least one block;
Determining the first transform size used to encode the at least one block based on the first syntax element and the second syntax element;
Applying an inverse transform of the determined first transform size to the residual transform coefficients to obtain a decoded residual block;
Adding the decoded residual block to the predicted video block to obtain a decoded video block ;
The computer-readable storage medium.

Images