RU2575397C2 - Padding of segments in coded slice network abstraction layer units - Google Patents

Abstract

FIELD: information technology.

SUBSTANCE: method of encoding video data, comprising dividing a picture into a plurality of picture partitions, the picture having a plurality of tree blocks, the picture partitions associated with non-overlapping subsets of the tree blocks of the picture; and generating a coded slice network abstraction layer (NAL) unit which includes encoded representations of the tree blocks associated with a slice of the picture, the encoded representations of the tree blocks grouped within the coded slice NAL unit into segments associated with different picture partitions, wherein one or more of the segments are padded such that each of the segments begins on a byte boundary, said segments including a given segment; and generating a coded slice network abstraction layer (NAL) unit comprises performing a padding operation which appends bits to the given segment if the next tree block is inside the slice and is associated with a different picture partition from the given segment.

EFFECT: high coding efficiency by providing parallel processing.

31 cl, 8 dwg

Description

CROSS RELATIONS TO RELATED APPLICATIONS

This application claims priority to provisional application US No. 61/557259, filed November 8, 2011, the contents of which are fully incorporated here by reference. This application also claims the priority of provisional application US No. 61/555932, filed November 4, 2011.

FIELD OF THE INVENTION

The present invention relates to video coding (i.e., encoding or decoding of video data).

BACKGROUND

Digital video capabilities can be incorporated into a wide range of applications, including digital television sets, direct digital broadcasting systems, wireless broadcasting systems, personal digital assistants (PDAs), laptop computers or desktop computers, digital cameras, digital recording devices , digital media players, video game devices, video game consoles, cellular or satellite radiotelephones, video teleconferencing devices, etc. Digital video devices implement various video compression techniques, which, for example, are described in standards defined as MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4, Part 10, Advanced Standard video coding (AVC), a standard for high-performance video coding (HEVC), which is under development, as well as expanding these standards, for more efficient transmission, reception and storage of digital video information.

According to video compression techniques, spatial (intra-frame) prediction and / or temporal (inter-frame) prediction is performed to reduce or eliminate the redundancy inherent in video sequences. For block-based video coding, a video slice may be divided into video blocks, which may also be called treeblocks, coding units (CUs), and / or coding units. Video blocks in the image slice with intraframe coding (I) are encoded using spatial prediction as applied to reference samples in neighboring blocks of the same image. In video blocks in the image slice with inter-frame coding (P or B), spatial prediction with respect to reference samples in neighboring blocks of the same image or temporal prediction with respect to reference samples of other reference images may be used. Images may be referred to as frames, and reference images may be referred to as reference frames.

SUMMARY OF THE INVENTION

In general, the invention describes techniques for encoding and decoding video data. A video encoder can divide an image into multiple image sections. Sections of the image include non-overlapping subsets of treeblocks of the image. Exemplary types of image sections include tiles and wave parallel processing (WPP) waves. A video encoder can create a block of network abstraction layer (NAL) of a coded slice that contains encoded representations of tree blocks associated with a single slice of a given image. The video encoder creates the NAL block of the encoded slice so that the encoded tree blocks are grouped in the NAL block of the encoded slice according to the sections of the image to which the tree blocks belong. A video encoder can fill one or more segments with insignificant information, so that each of the segments starts at the byte boundary. The video decoder can decode the encoded tree blocks of the NAL block of the encoded slice.

According to one aspect of the invention, a method for encoding video data is described. The method comprises dividing an image into multiple image sections. Image contains many tree blocks. Sections of the image are associated with non-overlapping subsets of tree blocks of the image. The method also comprises creating a NAL block of an encoded slice, which includes encoded representations of tree blocks associated with a slice of a given image, wherein encoded representations of tree blocks are grouped in a NAL block of an encoded slice into segments associated with different sections of the image, where one or more of segments are filled with insignificant information, so that each of the segments begins at the byte boundary.

According to another aspect, a method for decoding video data is described. The method comprises storing a NAL unit of an encoded slice that includes encoded representations of tree blocks associated with an image slice. The specified image is divided into many sections of the image. The encoded representations of these treeblocks are grouped into segments associated with different sections of the image. One or more of these segments is filled with insignificant information, so that each of these segments begins with a byte boundary. The method also comprises decoding the encoded representations of the tree blocks.

According to another aspect, a video coding apparatus is described. A video coding device comprises one or more processors configured to divide an image into multiple image sections. Image contains many tree blocks. Sections of the image are associated with non-overlapping subsets of tree blocks of the image. These one or more processors are also configured to create a NAL unit of an encoded slice that includes encoded representations of tree blocks that are associated with a slice of a given image. The encoded representations of the tree blocks are grouped in the NAL block of the encoded slice into segments associated with different sections of the image. One or more of these segments is filled with insignificant information, so that each of the segments begins at the byte boundary.

According to a further aspect, a video decoding apparatus that decodes video data is described. The video decoding device comprises a memory in which a NAL unit of the encoded slice is stored, including encoded representations of tree blocks associated with the image slice. The specified image is divided into many sections of the image. The encoded representations of these treeblocks are grouped into segments associated with different sections of the image. One or more of these segments is filled with insignificant information, so that each of these segments begins with a byte boundary. The video decoding apparatus also comprises one or more processors that are configured to decode the encoded representations of the tree blocks.

According to yet another aspect, a computer program product is described that comprises one or more computer-readable storage media that store instructions that, when executed by one or more processors, configure a video coding device to divide an image into multiple image sections. Image contains many tree blocks. Sections of the image are associated with non-overlapping subsets of tree blocks of the image. These commands also configure the video coding device to create a NAL unit of the encoded slice, which includes encoded representations of tree blocks that are associated with the slice of the image. The encoded representations of the tree blocks are grouped in the NAL block of the encoded slice into segments associated with different sections of the image. One or more of these segments is filled with insignificant information, so that each of the segments begins at the byte boundary.

According to another aspect, a computer program product is described that comprises one or more computer-readable storage media that store instructions that, when executed by one or more processors, configure a video decoding device to store a NAL unit of the encoded slice, which includes encoded representations of tree blocks associated with image slice. The specified image is divided into many sections of the image. The encoded representations of these treeblocks are grouped into segments associated with different sections of the image. One or more of these segments is filled with insignificant information, so that each of these segments begins with a byte boundary. These commands also configure a video decoding device for decoding encoded representations of treeblocks.

According to yet another aspect, a video coding apparatus that encodes video data is described. The video coding device comprises means for dividing an image into a plurality of image sections. Image contains many tree blocks. Sections of the image are associated with non-overlapping subsets of tree blocks of the image. The video coding apparatus also comprises means for creating a NAL unit of the encoded slice, which includes encoded representations of tree blocks associated with the slice of the image. The encoded representations of the tree blocks are grouped in the NAL block of the encoded slice into segments associated with different sections of the image. One or more of these segments is filled with insignificant information, so that each of the segments begins at the byte boundary.

According to a further aspect, a video decoding apparatus that decodes video data is described. The video decoding device comprises means for storing a NAL unit of the encoded slice, which includes encoded representations of tree blocks associated with the image slice. The specified image is divided into many sections of the image. The encoded representations of these treeblocks are grouped into segments associated with different sections of the image. One or more of these segments is filled with insignificant information, so that each of these segments begins with a byte boundary. The video decoding device also comprises means for decoding the encoded representations of the tree blocks.

Details of one or more examples are presented in the accompanying drawings and are set forth in the description below. Other features, objects, and advantages of the invention will become apparent from the foregoing description, drawings, and from the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an example video coding system in which the techniques described in the present invention can be used;

FIG. 2 is a block diagram illustrating an example video encoder that is configured to implement the techniques disclosed in the present invention;

FIG. 3 is a block diagram illustrating an example video decoder that is configured to implement the techniques disclosed in the present invention;

FIG. 4 is a flowchart illustrating an example operation for creating data for an image slice;

FIG. 5 is a flowchart illustrating an example operation for decoding a NAL unit of an encoded slice;

FIG. 6 is a conceptual diagram illustrating wave parallel processing;

FIG. 7 is a conceptual diagram illustrating an example encoding order when an image is divided into multiple tiles;

FIG. 8 is a conceptual diagram illustrating an exemplary NAL block of an encoded slice.

DETAILED DESCRIPTION OF THE INVENTION

The image includes many tree blocks. Tree blocks are associated with two-dimensional video blocks in this image. A video encoder divides an image into multiple image sections. For example, a video encoder can split an image into tiles or wave parallel processing (WPP) waves. In other words, in this invention, the term "image section" can be used to refer generally to tiles or WPP waves. Image sections are associated with non-overlapping subsets of tree-like image blocks. For example, each tree-like block of a given image can be associated with exactly one of the mentioned image sections.

The video encoder may create a block of network abstraction layer (NAL) of the encoded slice. A NAL unit of an encoded slice may include encoded representations of each tree block associated with a slice of a given image. In this description of the invention, references to a coded representation of a tree block as an encoded tree block are possible. An encoded tree block may include a sequence of bits that represents a video block associated with a tree block. This sequence of bits in a coded treeblock may represent a sequence of syntax elements.

The video encoder can group the encoded tree blocks in the NAL block of the encoded slice into segments. These segments are associated with different sections of the image. Each of these segments may be a sequential series of bits, such as bits representing a sequence consisting of one or more coded treeblocks and associated data. Thus, each NAL unit of the encoded slice may include an encoded tree block associated with the first section of the image, followed by an encoded tree side associated with the second section of the image, followed by an encoded tree block associated with the third section of the image, and t .d.

According to the techniques disclosed herein, a video encoder can fill one or more of these segments with insignificant information so that each of the segments starts at the byte boundary. When a video encoder fills a segment, it can attach fill bits to that segment. The padding bits do not have any semantic meaning, but are intended to ensure that the next segment starts at the byte boundary. In this way, the video encoder can provide byte alignment of WPP tiles or waves when these tiles or WPP waves are included in one NAL unit of the encoded slice for parallel processing purposes.

The video decoder can store the NAL block of the encoded slice in byte-addressable memory. The video decoder may then distribute two or more segments into different decoding streams that operate in parallel. Each decoding stream decodes the encoded tree blocks of a segment allocated to a given decoding stream. Since each segment begins with a byte boundary, a video decoder can provide a memory address for a segment to a decoding stream when allocating a given segment for a given decoding stream. In this way, the start of each segment from the byte boundary is provided, which allows video decoders to decode the segments in parallel in a simpler way than when the segments begin from positions that do not coincide with the byte boundary.

This is their difference from standard video encoders and standard video decoders, which do not provide the beginning of segments from byte boundaries. Since segments may not begin with byte boundaries, a standard video decoder using byte addressing of memory may not be able to decode encoded tree blocks in the indicated segments in parallel. A standard video decoder can use byte memory addressing or byte-by-bit plus bit-by-bit addressing in order to be able to decode coded tree blocks in segments in parallel, but the technical implementation and the calculation process are complicated.

The attached drawings show a number of examples. Elements in the accompanying drawings, denoted by certain reference numbers, correspond to elements indicated under the same reference positions in the following description. Here, elements with names beginning with ordinal numbers (for example, “first”, “second”, “third”, etc.) do not necessarily assume that these elements have a specific order. Rather, on the contrary, these ordinal numbers are used here simply to refer to different elements of the same or similar type. In addition, in the following description, the term “current image” may refer to an image that is currently being encoded or decoded.

In FIG. 1 is a flowchart showing an example of a video coding system 10 in which the techniques described herein may be used. As used herein, the term “video encoder” refers to both video encoders and video decoders. In this description, the terms “video coding” or “coding” may refer generally to video coding and video decoding.

As shown in FIG. 1, a video coding system 10 includes a source device 12 and a destination device 14. The source device 12 creates encoded video data. Accordingly, the source device 12 may be referred to as a video coding device. The destination device 14 may decode the encoded video data generated by the source device 12. Accordingly, the destination device 14 may be referred to as a video decoding device. Source device 12 and destination device 14 may be examples of video coding devices.

The source device 12 and the destination device 14 may contain any of a wide variety of devices, including desktop computers, mobile computing devices, laptop computers (eg, laptops), tablet computers, set-top boxes, handsets, so-called “smartphones”, television sets, cameras, display devices, digital media players, video game consoles, car computers, or the like. In some cases, source device 12 and destination device 14 may be equipped with wireless communications.

The destination device 14 may receive encoded video data from the source device 12 through the channel 16. Channel 16 may contain any medium or device of any type capable of transferring the encoded video data from the source device 12 to the destination device 14. In one example, channel 16 may contain the medium communication, allowing the source device 12 to transmit encoded video data directly to the destination device 14 in real time. The encoded video data may be modulated by the source device 12 in accordance with a communication standard, such as a wireless communication protocol, and may be transmitted to the destination device 14. The communication medium may comprise any wired or wireless communication medium, for example, radio frequency (RF) spectrum or one or more physical transmission lines. A communication medium may form part of a packet network, such as a local area network, a regional network, or a wide area network, such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful for communicating between the source device 12 and the destination device 14.

In another example, channel 16 may correspond to an information medium storing encoded video data created by source device 12. In this example, destination device 14 may have access to said information medium via disks or cards. The storage medium may include a variety of local access media, such as Blu-ray discs, DVD discs, CD discs, flash memory, or any other suitable digital storage medium for storing encoded video data. In yet another example, channel 16 may include a file server or other intermediate storage device that can store encoded video created by source device 12. Destination device 14 in this example can access encoded video data stored on a file server or other intermediate a storage device using streaming or downloading video data. A file server can be any type of server capable of storing and storing encoded video data and transferring it to the destination device 14. Examples of file servers include Web servers (for example, for a Web site), File Transfer Protocol (FTP) servers, network-attached storage devices (NAS) and local disk drives. The destination device 14 may have access to encoded video data through any standard data connection, including an Internet connection. Such connections may include wireless channels (for example, Wi-Fi connections), wired connections (for example, DSL, cable modem, etc.), or a combination of both types of connections, which is suitable for access to encoded video data stored on a file server. The transmission of encoded video data from a file server can be implemented in the form of streaming, uploading, or a combination thereof.

The techniques disclosed herein are not necessarily limited to wireless applications or installations. These techniques can be applied to video coding to support any of a variety of multimedia applications, such as terrestrial television broadcasting, cable television, satellite television, streaming video, for example, via the Internet, digital video encoding for storage on a storage medium, digital video decoding, stored on a storage medium, or other applications. In some examples, video coding system 10 may be configured to support unidirectional or bidirectional video transmission to support applications such as video streaming, video playback, video broadcasting and / or video telephony.

In the example of FIG. 1, source device 12 includes a video source 18, video encoder 20, and output interface 22. In some cases, output interface 22 may include a modulator / demodulator (modem) and / or transmitter. In source device 12, video source 18 may include a source such as a video capture device, for example, a video camera, a video archive containing previously captured video, a video feed interface for receiving video from a video content provider and / or computer graphics system for creating video data or combinations of these sources.

Currently captured, pre-captured, or computer-generated video can be encoded by video encoder 20. Coded video data can be transmitted directly to destination device 14 via output interface 22 of source device 12. Coded video data can also be stored on a storage medium or file server to access in the future from the destination device 14 for decoding and / or reproduction.

In the example of FIG. 1, destination device 14 includes an input interface 28, video decoder 30, and display device 32. In some cases, the input interface 28 may include a receiver and / or modem. The input interface 28 of the destination device 14 receives the encoded video data through channel 16. The encoded video data may include a variety of syntax elements created by video encoder 20 that represent the video data. These syntactic elements may be included in encoded video data transmitted through a communication medium, stored on a storage medium or stored on a file server.

The display device 32 may be an integral part (or external) of the destination device 14. In some examples, the destination device 14 may include an integrated display device, and may also be configured to interact with an external display device. In other examples, destination device 14 may be a display device. In general, the display device 32 displays decoded video data to a user. The display device 32 may include any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

Video encoder 20 and video decoder 30 may operate according to a video compression standard, such as the high performance video coding (HEVC) standard currently being developed, and may conform to the HEVC test model (HM). The latest draft HEVC standard, called “HEVC Working Draft 6” or “WD6,” is described in JCTVC-H1003, Bross et al., “High efficiency video coding (HEVC) text specification draft 6” by the Joint Video Coding Development Team (JCT-VC ) ITU-T SG16 WP3 standards and ISO / IEC JTC1 / SC29 / WG11, 8th Meeting: SanJose, California USA, February, 2012, which as of May 1, 2012 can be downloaded from: http: //phenix.int-evry .fr / jct / doc_end_user / documents / 8_San% 20Jose / wg11 / JCTVC-H1003-v22.zip., the entire contents of which are hereby incorporated by reference. Alternatively, video encoder 20 and video decoder 30 may operate according to other commercial or industrial standards, such as ITU-T H.264, also called MPEG-4, part 10, AVC (Advanced Video Coding), or extensions to these standards when included techniques for dividing the image into tiles or waves of parallel wave processing. However, the techniques disclosed in this invention are not limited to any particular coding standard or technique. Other examples of video compression standards and techniques include MPEG-2 and ITU-T H.263 and proprietary compression formats or open-source compression formats such as VP8 and related formats using techniques for splitting an image into tiles or wave parallel processing.

Although this is in FIG. 1 is not shown, in some aspects, video encoder 20 and video decoder 30 can each be integrated with an audio encoder and audio decoder and may include respective MUX-DMUX units (multiplexer-demultiplexer) or other hardware and software for encoding both audio and video in general data stream or in separate data streams. In some examples, if this is the case, the MUX-DMUX units may conform to the ITU H.223 Multiplexer Protocol or other protocols such as User Datagram Protocol (UDP).

It should be noted that FIG. 1 is just an example, and the techniques disclosed herein can be applied to video encoding settings (eg, video encoding or video decoding), which do not necessarily include any kind of data exchange between the encoding device and the decoding device. In other examples, data may be retrieved from local memory, received as a stream through a network, or the like. An encoding device may encode and store data in memory, and / or a decoding device may extract and decode data from memory. In many examples, encoding and decoding is performed by devices that do not exchange data with each other, but simply encode data into memory and / or extract and decode data from memory.

Video encoder 20 and video decoder 30 can each be implemented as any suitable encoder or decoder circuitry, such as one or more microprocessors, digital signal processors (DSP), ASIC application specific integrated circuits, user programmable gate arrays (FPGAs), discrete logic circuits hardware, or any combination thereof. In the partial implementation of these techniques by software, the device can store software instructions on a suitable computer-readable medium for long-term storage to execute these instructions by hardware using one or more processors that implement the techniques disclosed in the invention. Both video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, or each of them may be integrated in a combined encoder / decoder (“CODEC”) in a corresponding device.

As briefly mentioned above, video encoder 20 encodes video data. This video data may contain one or more images. Each of these images is a still image forming part of the video. In some cases, an image may be called a “frame” of a video or a “field” of video. When video encoder 20 encodes video data, it can create a bitstream. This bitstream may include a sequence of bits that form an encoded representation of the video data. The bitstream may include encoded images and associated data. An encoded image is an encoded representation of an image.

To create a bitstream, video encoder 20 may perform encoding operations for each image in the video data. When video encoder 20 performs encoding operations for images, it can create a series of encoded images and associated data. These image related data may include sequence parameter sets, image parameter sets, adaptation parameter sets, and other syntactic structures. The Sequence Parameter Set (SPS) may contain parameters that apply to zero or more image sequences. Image sequences may also be called coded video sequences, as in the H.264 / AVC and HEVC standards. An Image Parameter Set (PPS) may contain parameters that apply to zero or more images. The Adaptation Parameter Set (APS) may contain parameters that apply to zero or more images. Parameters in the APS set can be parameters whose probability of change is higher than the parameters in the PPS set.

To create an encoded image, video encoder 20 may split the image into video blocks of the same size. The video block may be a two-dimensional matrix of samples. Each of the video blocks is associated with a tree block. In some cases, the treeblock may be called the largest coding unit (LCU) or treeblock coding unit. HEVC treeblocks can in many respects be similar to macroblocks of previous standards such as H.264 / AVC. However, the tree block is not necessarily limited to a specific size and may include one or more coding units (CUs). Video encoder 20 may use quad-tree splitting to isolate video blocks of tree blocks into video blocks associated with CUs, hence the name “tree blocks”.

In some examples, video encoder 20 may split an image into multiple slices. Each of the slices may include an integer number of sequentially encoded tree blocks. In some cases, each of the slices may include an integer number of sequentially encoded CUs. As part of performing an image encoding operation, video encoder 20 may perform encoding operations on each slice of the image. When the video encoder 20 performs an encoding operation on a slice, video encoder 20 may create encoded data associated with the slice. The coded data associated with this slice may be referred to as a “coded slice”.

To create a coded slice, video encoder 20 may perform coding operations on each tree block in the slice. When the video encoder 20 performs an encoding operation on a tree block, it can create an encoded tree block. An encoded tree block may contain data representing an encoded version of the tree block.

When creating an encoded slice, video encoder 20 may perform an encoding operation on treeblocks in this slice according to the raster scan order. In other words, video encoder 20 can encode slice tree blocks in order from left to right through the topmost row of tree blocks in the slice, then left to right through the next row of tree blocks below, and so on, until video encoder 20 encodes all tree blocks in a given slice .

As a result of encoding, according to the order of raster scanning, tree blocks located above and to the left of this tree block will be encoded, and tree blocks located below and to the right of this tree block will remain unencrypted. Then, video encoder 20 can access the information generated by encoding the tree blocks located above and to the left of the tree block when encoding the tree block. However, video encoder 20 cannot access the information generated by encoding the tree blocks located below and to the right of the tree block when encoding this tree block.

To create a coded tree block, video encoder 20 may recursively split a quad tree on a video block of a tree block to divide the video block into successively decreasing video blocks. Each of the successively decreasing video blocks can be associated with its own CU. For example, video encoder 20 can split a video block of a tree block into four subblocks of the same size, split one or more of these subblocks into four elements of a subblock of the same size, etc. The CU thus allocated can be a CU, whose video block is divided into video blocks associated with other CUs. An unbroken CU is a CU whose video block is not divided into video blocks associated with other CUs.

One or more syntax elements in the bitstream may indicate the maximum number of possible partitions of the indicated video block of the tree block that are performed by video encoder 20. The video block of the CU may be in the form of a square. The size of the video block of the CU can be in the range from 8 × 8 pixels to the size of the video block of the tree block (i.e., the size of the tree block), which is a maximum of 64 × 64 pixels or more.

Video encoder 20 may perform coding operations of each CU of the treeblock according to the z-scan order. In other words, video encoder 20 may encode the upper left CU, the upper right CU, the lower left CU, and then the lower right CU in that order. When performing the encoding operation on the dedicated CU, video encoder 20 may encode CUs associated with the sub-blocks of the video block of the dedicated CU according to the z-scan order. In other words, video encoder 20 may encode a CU associated with the lower left sub-block, and then a CU associated with the lower right sub-block in that order.

As a result of encoding the CU blocks of the tree block according to the z-scan order, the CU blocks may be encoded above, above and to the left, above and to the right, to the left and below and to the left of the given CU. Uncoded will remain the CUs below to the right of this CU. Then, video encoder 20 is able to access information created by encoding some CUs that are adjacent to a given CU when encoding a given CU. However, video encoder 20 may be unable to access information created by encoding other CUs that are adjacent to a given CU when encoding a given CU.

When video encoder 20 performs encoding of the non-shared CU, video encoder 20 may create one or more prediction units (PUs) for this CU. Each of the PU blocks of the CU may be associated with a different video block in the video block of the CU. Video encoder 20 may create a predicted video block for each PU of a CU. The predicted video block of the PU may be a block of samples. Video encoder 20 may use intra prediction or inter prediction to create a predicted video block for the PU.

When video encoder 20 uses intra-frame prediction to create a predicted video block of a PU, video encoder 20 can create a predicted video block of a PU based on decoded image samples associated with that PU. If video encoder 20 uses intra-frame prediction to create predicted video blocks of a PU of a CU, the CU is an intra-prediction CU. When a video encoder uses inter-frame prediction to create a predicted video block of a PU block, video encoder 20 can create a predicted video block of a PU block based on decoded samples of one or more images other than the image associated with the PU. If video encoder 20 uses inter prediction to create predicted video blocks of PU blocks of a CU, then the CU is an inter prediction CU.

In addition, when video encoder 20 uses inter-frame prediction to create a predicted video block for a PU, video encoder 20 can create motion information for a given PU. Motion information for a PU may indicate one or more support blocks of a PU. Each support block of the PU block may be a video block in the reference image. The reference image may be an image other than the image associated with the PU. In some cases, the support block of the PU block may also be called the “reference pattern” of the PU block. Video encoder 20 may create a predicted video block for a PU based on the reference blocks of that PU.

After the video encoder 20 has created predicted video blocks for one or more PUs of the CU, video encoder 20 may generate residual data for the CU based on the predicted video blocks for the PUs of the CU. The residual data for the CU may indicate the difference between the samples in the predicted video blocks for the PUs of the CU and the original video block of the CU.

In addition, as part of the process of performing the encoding operation on an unpartitioned CU, video encoder 20 may perform a quad-tree recursive partitioning into residual data of the CU to divide residual data of the CU into one or more residual data blocks (i.e., residual video blocks) associated with transform units (TU) of the CU. Each TU of a CU can be associated with another residual video block.

Video encoder 20 may apply one or more transforms to residual video blocks associated with TUs to create blocks of transform coefficients associated with TUs. Conceptually, a block of transform coefficients can be a two-dimensional (2D) matrix of transform coefficients.

After creating a block of transform coefficients, video encoder 20 may implement a quantization process on a block of transform coefficients. Typically, quantization refers to a process in which quantization of transform coefficients is performed to possibly reduce the amount of data used to represent the transform coefficients, which provides additional compression. The quantization process can reduce the bit depth associated with some or all of the transform coefficients. For example, an n-bit transform coefficient can be rounded to an m-bit transform coefficient in a quantization process, where n is greater than m.

Video encoder 20 may associate each CU with a quantization parameter (QP) value. The QP value associated with the CU may determine how video encoder 20 quantizes the blocks of transform coefficients associated with that CU. Video encoder 20 may adjust the degree of quantization applied to the blocks of transform coefficients associated with the CU, by adjusting the QP value associated with this CU.

After quantizing the transform coefficient block by video encoder 20, it creates sets of syntax elements that represent transform coefficients in the quantized transform coefficient block. Video encoder 20 may apply entropy coding operations, such as context adaptive binary arithmetic coding (CABAC) operations to some of these syntax elements.

The bitstream generated by video encoder 20 may include a number of network abstraction layer (NAL) blocks. Each of the NAL units may be a syntax structure containing an indication of the type of data in the NAL unit, and bytes containing data. For example, a NAL unit may contain data representing a set of sequence parameters, a set of image parameters, an encoded slice, one or more additional extended information (SEI) messages, an access block delimiter, filter data, or another data type. The data in the NAL unit may include various syntax structures.

Video decoder 30 may receive a bitstream generated by video encoder 20. This bitstream may include an encoded representation of the video data encoded by video encoder 20. By receiving the bitstream, video decoder 30 may perform a parsing operation on this bitstream. In a parsing operation, video decoder 30 may extract syntax elements from a bitstream. Video decoder 30 may recover images of video data based on syntax elements extracted from the bitstream. The process of restoring video data based on syntax elements is generally the reverse of the process performed by video encoder 20 to create syntax elements.

After the video decoder 30 has retrieved the syntax elements associated with the CU, the video decoder 30 may create predicted video blocks for the PUs of the CU based on these syntax elements. In addition, video decoder 30 may inverse quantize blocks of transform coefficients associated with TUs of a given CU. Video decoder 30 may reverse transform the blocks of transform coefficients to recover residual video blocks associated with the TUs of a given CU. After creating the predicted video blocks and reconstructing the residual video blocks, video decoder 30 may recover the video block of the CU based on the predicted video blocks and residual video blocks. Video decoder 30 may thus restore the video blocks of the CUs based on the specified syntax elements in the bitstream.

Video encoder 20 may split the current image into a plurality of image sections. Sections of the image may be associated with non-overlapping subsets of tree blocks of the current image. Video encoder 20 may divide the current image into a plurality of image sections in various ways. As described below, video encoder 20 may split the current image into multiple tiles or multiple waves of wave parallel processing (WPP). As used in the present invention, the term “image section” refers generally to both tiles and WPP waves. The process of dividing the current image into image sections may be referred to as “dividing” the current image into image sections.

As mentioned above, video encoder 20 may split the current image into one or more tiles. Each tile can contain an integer number of tree blocks in the current image. Video encoder 20 may divide the current image into tiles by defining two or more vertical tile borders and two or more horizontal tile borders. Each vertical side of the current image can be considered as the vertical border of the tile. Each horizontal side of the current image can be considered the horizontal border of the tile. For example, if video encoder 20 determines four vertical tile borders and three horizontal tile borders for the current image, then the current image is divided into six tiles.

A video encoder, such as video encoder 20 or video decoder 30, may encode tiles of the current image according to a raster scan order. In addition, when encoding a tile with a video encoder, a video encoder can encode each tree block in a given tile according to the raster scan order. The video encoder can encode each tree block of a given tile of the current image in this way before encoding any tree block of another tile of this image. Therefore, the encoding order of the tree blocks of the current image by the video encoder may differ when the video encoder splits the current image into multiple tiles, from the case when the video encoder does not divide the current image into multiple tiles.

In addition, in some cases, the video encoder may use information associated with spatially adjacent CUs to perform intra-frame prediction on a given CU in the current image, as long as the given CU and spatially adjacent CUs belong to the same tile. Space neighboring CUs are CUs related to the current slice of the current image. In some cases, a video encoder may use information associated with spatially adjacent CUs to select a context for encoding a syntax element of a given CU using the CABAC algorithm, as long as a given CU and spatially adjacent CUs are in the same tile. Due to these limitations, the video encoder is capable of parallel coding of tree blocks of multiple tiles.

In other examples, the video encoder may encode the current image using wave parallel processing (WPP). When encoding the current image using WPP, the video encoder can split the tree blocks of the current image into many WPP “waves”. Each WPP wave can correspond to its row of tree blocks in the current image. When encoding the current image using WPP, the video encoder can start encoding from the top line of the tree blocks. Having encoded two or more tree blocks of the top row, the video encoder can begin to encode the second (top) row of tree blocks in parallel with the encoding of the top row of tree blocks. Having encoded two or more tree blocks from the second row from the top, the video encoder can begin to encode the third from the top row of tree blocks in parallel with the encoding of the overlying rows of tree blocks. This encoding order can continue to the bottom rows of tree blocks in the current image.

When a video encoder encodes a current image using WPP, the video encoder can use the information associated with the spatial neighboring CUs outside the current treeblock to perform intra-frame prediction on this CU in the current treeblock, as long as there are neighboring CUs in the space left, left and above , above or to the right and above the current treeblock. If the current treeblock is the leftmost treeblock in a row other than the topmost row, the video encoder can use the information associated with the second treeblock of the immediately superior row to select the context for encoding the syntax element of the current treeblock according to the CABAC algorithm. Otherwise, if the current treeblock is not the leftmost treeblock in this row, the video encoder can use the information associated with the treeblock to the left of the current treeblock to select a context for encoding the syntax element of the current treeblock using the CABAC algorithm . Thus, the video encoder can use the CABAC states of a row based on the CABAC states associated with the immediately superior row after encoding two or more tree blocks of the immediately superior row.

In some examples, when a video encoder encodes a current image using WPP, only the tile boundaries of the current image are horizontal and vertical borders of the current image. Thus, only the tile of the current image can be the same size as the current image. A video encoder can split the current image, which means one tile of the current image into many WPP waves.

As mentioned above, video encoder 20 may create a NAL block of an encoded slice that includes an encoded representation of the slice. A slice can be associated with an integer number of sequentially encoded tree blocks. An encoded slice NAL unit may include a slice header and slice data. Slice data may include encoded representations of each treeblock associated with this slice. Video encoder 20 may create a NAL block of the encoded slice so that the specified encoded representations of the tree blocks are grouped in the slice data in segments according to the sections of the image to which these tree blocks belong. For example, each NAL unit of an encoded slice may include an encoded tree block associated with a first section of an image, followed by a tree block associated with a second section of an image, followed by an encoded tree block associated with a third section of an image, etc. .

According to the techniques disclosed herein, video encoder 20 may populate one or more segments with insignificant information so that each segment begins at the byte boundary. A NAL block of an encoded slice can be divided into a series of bytes. A segment may begin at the byte boundary when its first bit is the first bit of one of the bytes of the NAL block of the encoded slice. In addition, a segment may be byte aligned if the first bit of the segment is the first bit of one of the bytes of the NAL block of the encoded slice. When a video encoder fills a segment, video encoder 20 may attach fill bits to this segment. For example, video encoder 20 may add one or more padding bits to a segment such that the number of bits in the segment is divided by eight without a remainder. Population bits are not required to have semantic meaning; they serve to ensure that the next segment starts at the byte boundary.

Upon receiving the NAL unit of the encoded slice, video decoder 30 may store this NAL unit of the encoded slice in memory. For parallel decoding of sections of an image, video decoder 30 may distribute segments among different decoding streams that are executed in parallel. In order to distribute segments among different decoding streams, video decoder 30 will need to indicate memory addresses associated with the start of each of the segments. Video decoder 30 may use byte addressing of memory. Accordingly, video decoder 30 may not be able to indicate the memory address associated with the beginning of the segment if the beginning of the segment is inside the byte. Therefore, video decoder 30 will not be able in this case to perform parallel decoding of encoded treeblocks in a segment if one or more segments begin within the byte. Alternatively, video decoder 30 may use bitwise memory addressing or bit-by-bit plus bitwise addressing to be able to decode tree blocks in the indicated segments in parallel, but technical implementation and calculations are complicated.

Video encoder 20 may divide an image into a plurality of image sections in this manner. Image contains many tree blocks. Image sections are associated with non-overlapping subsets of tree-like image blocks. Video encoder 20 may create a NAL unit of an encoded slice that includes encoded representations of tree blocks that are associated with an image slice. The encoded representations of the tree blocks in the NAL block of the encoded slice are grouped into segments associated with their image sections. One or more segments are filled with insignificant information, so that each segment begins at the byte boundary.

In addition, video decoder 30 may store a NAL unit of the encoded slice, which includes encoded representations of tree blocks associated with the image slice. An image can be divided into many sections of the image. Coded representations of tree blocks can be grouped into segments, each of which will be associated with its own section of the image. One or more segments are filled with insignificant information so that each segment begins with a byte boundary. Video decoder 30 may decode encoded representations of treeblocks. In some cases, video decoder 30 may decode encoded representations of tree blocks in two or more segments in parallel.

In FIG. 2 is a flowchart illustrating an example video encoder 20 configured to implement the techniques disclosed herein. FIG. 2 is provided for explanatory purposes, and should not be construed as limiting the aforementioned techniques that are widely presented and described here as examples. For the purpose of explaining the invention, video encoder 20 is described herein in the context of encoding according to the HEVC standard. However, the techniques disclosed herein may be applied to other standards or coding methods.

In the example of FIG. 2, video encoder 20 includes many functional components. The functional components of video encoder 20 include a prediction module 100, a remainder data generation module 102, a transform module 104, a quantization module 106, a quantization module 108, an inverse transform module 110, a reconstruction module 112, a filter module 113, a decoded image buffer 114, and a module 116 entropy coding. The prediction module 100 includes an inter prediction module 121, a motion estimation module 122, a motion compensation module 124, and an intra prediction module 126. In other examples, video encoder 20 may include fewer or more functional components or other functional components. In addition, the motion estimation module 122 and the motion compensation module 124 can be implemented as a single module with a high degree of integration, but they are presented in the example of FIG. 2 separately in order to explain their functioning.

Video encoder 20 may receive video data. Reception of video data by video encoder 20 may be from various sources. For example, video encoder 20 may receive video data from a video source 18 (FIG. 1) or another source. Video data may represent a series of images. To encode video data, video encoder 20 may perform an encoding operation for each image. As part of the process of performing an image encoding operation, video encoder 20 may perform encoding operations on each slice of the image. As part of performing an encoding operation on a slice, video encoder 20 may perform encoding operations on tree blocks in the slice.

As part of the encoding operation on the treeblock, the prediction unit 100 may perform quad-tree splitting on the video block of the treeblock to divide the video block into video blocks decreasing with each split. Each of the smaller video blocks may be associated with its own CU. For example, prediction module 100 may split a video block of a tree block into four subblocks of the same size, split one or more of these subblocks into four elements of a subblock of the same size, etc.

The sizes of the video blocks associated with the CUs may range from 8 × 8 samples to the size of a tree block having a maximum of 64 × 64 samples or more. In this description, the expressions “N × N” and “N by N” can be used interchangeably to refer to the dimensions of the video block samples vertically and horizontally, for example, “16 × 16” samples or “16 by 16” samples. In the general case, a 16 × 16 video block has sixteen samples in the vertical direction (y = 16) and sixteen samples in the horizontal direction (x = 16). Similarly, an N × N block typically has N samples in the vertical direction and N samples in the horizontal direction, where N represents a non-negative integer.

In addition, as part of performing the encoding operation on the treeblock, the prediction unit 100 may create a data structure in the form of a hierarchical quad tree for a given treeblock. For example, a treeblock may correspond to a root node of a quad tree data structure. If the prediction module 100 splits the video block of the tree block into four sub-blocks, then this root node will have four child nodes in the data structure in the form of a quad tree. Each of the child nodes corresponds to a CU associated with one of the subunits. If the prediction unit 100 divides one of the subunits into four elements of the subunit, then the node corresponding to the CU associated with this subunit can have four child nodes, each of which corresponds to the CU associated with one of the elements of the subunits.

Each quad-tree data structure node may contain syntax data (eg, syntax elements) for a corresponding treeblock or CU. For example, a node in a quad tree may include a splitting flag, which indicates whether the video block of the CU corresponding to that node is split (i.e., split) into four sub-blocks. The syntax elements for the CU can be determined recursively, and they will depend on whether the given video block of the CU is split into sub-blocks. A CU whose video block is not split can correspond to an end node in a quad tree data structure. The encoded tree block may include data based on structural data such as a quad tree for the corresponding tree block.

Video encoder 20 may perform coding operations on each non-shared unit CU of the treeblock. When the video encoder 20 performs the encoding operation on the non-shared CU, video encoder 20 creates data representing an encoded representation of the non-shared CU.

As part of the encoding operation on the CU, the prediction unit 100 may split the video block of the CU into several (in the extreme case, one) PU blocks of the CU. Video encoder 20 and video decoder 30 may support different PU block sizes. Assuming that the size of a specific CU is 2N × 2N, then video encoder 20 and video decoder 30 may support 2N × 2N or N × N PU sizes and inter-frame prediction for symmetric PU sizes of 2N × 2N, 2N × N, N × 2N, N × N, 2N × nU, nL × 2N, nR × 2N or the like Video encoder 20 and video decoder 30 can also support asymmetric partitioning for PUs with dimensions of 2N × nU, 2N × nD, nL × 2N, and nR × 2N. In some examples, the prediction unit 100 may perform geometric partitioning to split the video block of the CU between the PUs of the given CU along a border that does not intersect the sides of the video block of the CU at right angles.

Inter prediction unit 121 may perform inter prediction on each PU of a CU. Inter prediction can provide temporal compression. To perform inter-frame prediction on the PU, motion estimation module 122 may generate motion information for the PU. Motion compensation module 124 may create a predicted video block for the PU based on motion information and decoded image samples other than the image associated with this CU (i.e., reference images). In this description, the predicted video block created by motion compensation module 124 may be referred to as inter-frame predicted video block.

Slices can be I slices, P slices, or B slices. Motion estimation module 122 and motion compensation module 124 can perform different operations for the PU of the CU, depending on which type of slice (I, P or B) this PU is in. In Slice I, all PUs are obtained using intra-frame prediction. Therefore, if the PU is in I slice, then motion estimation module 122 and motion compensation module 124 do not perform inter-frame prediction on this PU.

If the PU is in the P slice, then the image containing this side of the PU is associated with a list of reference images called “list 0”. Each of the reference images in list 0 contains samples that can be used for inter-frame prediction of other images. When the motion estimation module 122 performs the motion estimation operation with respect to the PU in the P slice, the motion estimation module 122 can search for reference images in list 0 for the reference block for that PU. The reference block of the PU block can be a set of samples, for example, a block of samples, which is closest to the samples in the video block of the PU block. Motion estimation module 122 may use a variety of metrics to determine how close the set of samples in the reference image corresponds to the samples in the video block of the PU. For example, motion estimation module 122 may determine how close a set of samples in a reference image corresponds to samples in a video block of a PU block using an approach based on the sum of the absolute differences (SAD), the sum of the squared deviations (SSD), or other deviation metrics.

After identifying the reference block for the PU block in the P slice, the motion estimation module 122 may create a reference index indicating the reference image in list 0 containing the given reference block and a motion vector that indicates the spatial displacement between the PU block and the reference block. In various examples, motion estimation module 122 may generate variable motion vectors. For example, motion estimation module 122 may create motion vectors with a quarter-frame accuracy, one-eighth of a frame precision, or another fractional accuracy. In the case of fractional accuracy, the values of the reference block can be interpolated based on the integer values of the position samples in the reference image. Motion estimation module 122 may provide a reference index and a motion vector as the motion information of the PU. Motion compensation module 124 may create a predicted video block of a PU block based on a reference block identified by the motion information of the PU block.

If the PU is in the B slice, then the image containing this PU can be associated with two lists of reference images, called "list 0" and "list 1". In some examples, an image containing a B slice may be associated with a combination of lists, which is a combination of list 0 and list 1.

In addition, if the PU is in the B slice, then motion estimation module 122 may perform unidirectional prediction or bidirectional prediction for a given PU. When module 122 performs unidirectional prediction motion estimation for the PU, this module can search for reference images from list 0 or from list 1 for the reference block for the PU. Module 122 can then create a reference index that indicates the reference image in list 0 or list 1 containing the reference block, and a motion vector that indicates the spatial displacement between the PU block and the reference block. Motion estimation module 122 may provide a reference index, a prediction direction indicator, and a motion vector as motion information regarding the PU. The prediction direction indicator may indicate whether the reference index points to the reference image in list 0 or in list 1. Motion compensation module 124 may create a predicted video block of the PU block based on the reference block indicated in the motion information regarding the PU block.

When motion estimation module 122 performs bidirectional prediction for the PU, motion estimation module 122 can search for reference images in list 0 for a reference block for a given PU, and can also search for reference images in list 1 for another reference block for a PU. Then, motion estimation module 122 creates reference indices that indicate reference images in list 0 and in list 1, containing reference blocks and motion vectors indicating spatial displacements between the reference blocks and the PU. The motion estimation module 122 may provide reference indices and motion vectors of the PU unit as the motion information of the PU unit. Motion compensation module 124 may create a predicted video block of the PU block based on the reference blocks indicated in the motion information of the PU block. In some cases, motion estimation module 122 does not provide a complete set of motion information for the PU unit to entropy encoding module 116. Conversely, motion estimation module 122 may report motion information related to a PU, with reference to motion information related to another PU. For example, motion estimation module 122 may determine that the motion information for a given PU is similar to the motion information for a neighboring PU. In this example, motion estimation module 122 may indicate (in the syntax associated with this PU) a value indicating to video decoder 30 that the given PU has the same motion information as the neighboring PU. In another example, motion estimation module 122 may identify (in the syntax associated with this PU) the neighboring PU and their motion vector difference (MVD). The difference of the motion vectors indicates the difference between the motion vector of the PU block and the motion vector of the indicated neighboring PU block. Video decoder 30 may use the motion vector of the indicated neighboring PU and the difference of the motion vectors to determine the motion vector of the PU. Referring to the motion information of the first PU when signaling about the motion of the second PU, the video encoder 20 is able to report motion information for the second PU, using fewer bits.

As part of the encoding operation on the CU, the intra prediction unit 126 may perform intra prediction on the PUs of the CU. Intra frame prediction can provide spatial compression. When performing intra-frame prediction on the PU block, the intra-frame prediction unit 126 may generate prediction data for the PU block based on decoded samples of other PU blocks in the same image. Prediction data for a PU may include a predicted video block and various syntax elements. Intra-frame prediction unit 126 may perform intra-frame prediction on PUs in I slices, P slices, and B slices.

To perform intra-frame prediction on the PU, the intra-frame prediction module 126 may use a plurality of intra-frame prediction modes to create a plurality of prediction data sets for the PU. When using the intra prediction mode to create a prediction data set for a given PU block, the intra prediction module 126 may propagate the samples from the video blocks of neighboring PU blocks through the video block of this PU block in the direction and / or along the gradient associated with the intra prediction mode. Neighboring PUs may be higher, higher, and to the right, higher, and to the left or left of a given PU, assuming that coding for PUs, CUs, and treeblocks is performed from left to right, top to bottom. Intra-frame prediction unit 126 may use a different number of intra-frame prediction modes, for example, 33 directional intra-frame prediction modes, depending on the size of a given PU.

Prediction unit 100 may select prediction data for the PU from among the prediction data created by motion compensation unit 124 for a given PU, or prediction data created by intraframe prediction unit 126 for a given PU. In some examples, prediction module 100 selects prediction data for a given PU based on the bit / distortion rate of the prediction data sets.

If the prediction unit 100 selects the prediction data generated by the intra-frame prediction unit 126, it may signal the intra-frame prediction mode used by it to generate the prediction data for the PUs, i.e., the selected intra-frame prediction mode. Prediction unit 100 may signal the selected intra-frame prediction mode in various ways. For example, it is highly likely that the selected intra prediction mode coincides with the intra prediction mode of the neighboring PU. In other words, the intra prediction mode of the neighboring PU may be the most likely mode for the current PU. Thus, the prediction unit 100 may create a syntax element for indicating that the selected intra-prediction mode is the same as the intra-prediction mode of the neighboring PU.

After the prediction data 100 determines the prediction data for the PU units of the CU unit, the residual data generating unit 102 can create residual data for the CU unit by subtracting the predicted video blocks of the PU units of the CU unit from the video unit of the CU unit. The residual data of the CU may include residual 2D video blocks that correspond to different sample components in the video block of the CU. For example, the residual data may include a residual video block that corresponds to differences between the luminance components of the samples in the predicted video blocks of the PU blocks of the CU and the luminance components of the samples in the original video block of the CU. In addition, the residual data of the CU may include residual video blocks that correspond to differences between the color components of the samples in the predicted video blocks of the PU blocks of the CU and the color components of the samples in the original video block of the CU.

Prediction module 100 may perform quad-tree partitioning to partition the residual video blocks of the CU into subblocks. Each undivided residual video block may be associated with another TU of a CU. The sizes and positions of the residual video blocks associated with the TUs of the CU may or may not depend on the sizes and positions of the video blocks associated with the PUs of the CU. The tree structure, known as the "residual quad tree" (RQT), may include modes associated with each of the residual video blocks. The TUs of the CU can correspond to the end nodes of the RQT tree.

Conversion module 104 may create one or more transform coefficient blocks for each TU of a CU by applying one or more transforms to the residual video block associated with the indicated TU. Each block of transform coefficients may be a 2D matrix of transform coefficients. Conversion module 104 may apply various transformations to the residual video block associated with the TU. For example, transform module 104 may apply a discrete cosine transform (DCT), directional transform, or conceptually similar transform to a residual video block associated with a TU.

After the transform unit 104 creates a transform coefficient block associated with the TU, the quantization module 106 may quantize the transform coefficients in the transform coefficient block. The quantization unit 106 may quantize the transform coefficient block associated with the TU of the CU, based on the QP value associated with this CU.

Video encoder 20 may associate a QP value with a CU in various ways. For example, video encoder 20 may perform bit-distortion consumption analysis on a tree block associated with a CU. In a bit-distortion flow analysis, video encoder 20 can create a plurality of encoded representations of the tree block by repeatedly performing the encoding operation on the tree block. Video encoder 20 may associate different QP values with a CU when creating different coded representations of a given tree block. Video encoder 20 may signal that a given QP value is associated with a CU when the given QP value is associated with a CU in the encoded representation of a tree block having a minimum bit-distortion rate.

The inverse quantization module 108 and the inverse transform module 110 may apply inverse quantization and inverse transform to the block of transform coefficients, respectively, to restore the residual video block from the block of transform coefficients. Recovery module 112 may add the reconstructed residual video block to corresponding samples from one or more predicted video blocks created by prediction module 100 to create a reconstructed video block associated with a TU. By reconstructing in this way the video blocks for each TU of the CU, video encoder 20 may recover the video block of the CU.

After the recovery unit 112 restores the video block of the CU, the filter module 13 may perform a release operation to suppress blocking artifacts in the video block associated with the CU. After performing one or more unlocking operations, the filter module 113 may store the restored video block of the CU in the decoded image buffer 114. Motion estimation module 122 and motion compensation module 124 may use a reference image containing the reconstructed video block to perform inter-frame prediction on subsequent image blocks PU. In addition, intra prediction module 126 may use the reconstructed video blocks in decoded image buffer 114 to perform intra prediction on other PUs in the same image as the CU.

Entropy encoding module 116 may receive data from other functional components of video encoder 20. For example, entropy encoding module 116 may receive blocks of transform coefficients from quantization module 106 and may receive syntax elements from prediction module 100. Upon receipt of the data, entropy encoding unit 116 may perform one or more entropy encoding operations to generate encoded data. For example, video encoder 20 may perform a context-dependent adaptive variable-length coding (CAVLC) operation, a context-adaptive binary arithmetic coding (CABAC) operation, a variable-length word encoding (V2V) coding operation, a syntactically oriented adaptive binary arithmetic coding ( SBAC), an interval division probabilistic entropy coding (PIPE) operation, or another type of entropy coding operation on this data. Entropy encoding unit 116 may output a bitstream that includes data encoded using entropy encoding.

As part of performing an entropy encoding operation on data, entropy encoding module 116 may select a context model. If entropy encoding unit 116 performs the CABAC operation, then this context model may indicate estimates of the probabilities that particular beans have specific meanings. In the context of the CABAC algorithm, the term “bean” is used to refer to a bit of a binary version of a syntax element.

Video encoder 20 may create a NAL block of the encoded slice for each slice of the current image. An encoded slice NAL unit may include a slice header and slice data. Slice data can contain many segments. Each of these segments includes coded treeblocks associated with another section of the image. Video encoder 20 can fill segments with insignificant information so that each segment begins at the byte boundary in the slice data. For example, segments in a NAL unit of an encoded slice may include a given segment. In this example, video encoder 20 can create a NAL block of an encoded slice at least partially by performing an insignificant fill operation in which bits are connected to a given segment if the next tree block is inside the current slice and is associated with an image section other than the given segment.

In some examples, video encoder 20 may create a slice header of a coded slice NAL unit so that the slice header indicates entry points for segments in the slice data of a coded slice NAL unit. These entry points may indicate positions in the slice data of the indicated segments. For example, these entry points may indicate byte offsets of segments. In this example, byte offsets may refer to the first bit of the encoded slice NAL unit, the first bit of the slice data, or another bit in the NAL unit of the encoded slice. In another example, these input points may indicate the number of bits or bytes in each of the segments. In some examples, the slice header does not indicate the insertion point for the first segment in the slice data.

In some examples, video encoder 20 may determine if the flag has a first value (e.g., 1). If the flag has this first value, then video encoder 20 may fill one or more segments with insignificant information so that each segment begins with a byte boundary. When the flag has a second value (for example, 0), video encoder 20 does not populate the segments with insignificant information. As a result, these segments may or may not start from a byte aligned position. In these examples, a set of sequence parameters, a set of image parameters, a set of adaptation parameters or a slice header may include a flag. Thus, in some examples, video encoder 20 may create a parameter set associated with the current image, where this parameter set includes a flag. When this flag has the first value, one or more segments are filled with insignificant information, so that these segments begin with byte boundaries. When this flag has a second meaning, these segments may or may not begin with byte boundaries.

In addition, in some examples, video encoder 20 may split the current image into multiple tiles. If video encoder 20 allows prediction over tile boundaries in the image (that is, when two or more tiles are dependent on each other), video encoder 20 does not perform the indicated segment filling. As a result, these segments may or may not start from a byte aligned position. However, if video encoder 20 does not allow prediction across tile boundaries, it can fill one or more segments with insignificant information so that each segment begins with a byte boundary. Thus, video encoder 20 can create a NAL block of the encoded slice at least partially by performing an operation for filling insignificant information, which ensures that the beginning of each of the segments is on the byte boundary, but only after determining that the said tiles are independent of each other.

In FIG. 3 is a flowchart illustrating an example video decoder 30 configured to implement the techniques disclosed herein. FIG. 3 is provided for clarification only and does not limit these techniques, which are broadly presented as examples and described in detail in the present invention. For convenience of explanation, video decoder 30 is described herein in the context of encoding according to the HEVC standard. However, the techniques disclosed herein may be applied to other coding standards or methods.

In the example of FIG. 3, video decoder 30 includes many functional components. These functional components of video decoder 30 include entropy decoding module 150, prediction module 152, inverse quantization module 154, inverse transform module 156, reconstruction module 158, filter module 159, and decoded image buffer 160. Prediction module 152 includes motion compensation module 162 and intra-frame prediction module 164. In some examples, video decoder 30 may perform a decoding run, which is generally the opposite of the encoding run described with respect to video encoder 20 of FIG. 2. In other examples, video decoder 30 may include more or less functional components or other functional components.

Video decoder 30 may receive a bitstream containing encoded video data. This bitstream may include many syntax elements. When the video decoder 30 receives the bit stream, the entropy decoding unit 150 may perform a parsing operation on that stream. As a result of the parsing operation on the bitstream, entropy decoding module 150 can extract syntax elements from the bitstream. As part of the parsing operation, entropy decoding unit 150 may perform entropy decoding of syntax elements encoded using entropy encoding. Prediction module 152, inverse quantization module 154, inverse transform module 156, reconstruction module 158, and filter module 159 may perform a reconstruction operation that creates decoded video data based on syntax elements extracted from the bitstream.

As discussed above, a bitstream may contain a number of NAL units. The bitstream NAL blocks may include sequence parameter set NAL blocks, image parameter set NAL blocks, SEI NAL blocks, etc. As part of performing a parsing operation on a bitstream, entropy decoding module 150 may perform parsing operations that provide extraction and entropy decoding of sequence parameter sets from sequence parameter NAL blocks, image parameter sets from NAL blocks of image parameter set, image parameter sets, SEI data from blocks SEI NAL etc.

In addition, NAL units of a bitstream may include NAL units of an encoded slice. As part of the parsing operation on the bitstream, video decoder 30 may perform parsing operations to extract and entropy decode the encoded slices from the NAL blocks of the encoded slice. Each of these coded slices may include a slice header and slice data. The slice header may contain syntax elements that belong to the slice. These syntax elements in the header of the slice may include a syntax element identifying a set of image parameters associated with the image that contains the given slice.

The encoded slice NAL unit data may include multiple segments. Each of these segments may include coded treeblocks associated with another section of the image (e.g., tile or WPP wave). One or more of these segments in the slice data may be filled with insignificant information so that each segment begins with a byte boundary. The slice header of the NAL block of the encoded slice may indicate the entry points for the segments. In this case, since segments always begin with byte boundaries, video decoder 30 is able to easily distribute different segments across different decoding streams by using memory byte addressing. Different decoding streams can provide parsing of the encoded tree blocks of segments and reconstruct in parallel the video data associated with the corresponding tree blocks.

As part of extracting the slice data from the NAL units of the encoded slice, the entropy decoding unit 150 may perform parsing operations that extract syntax elements from the encoded CUs. The extracted syntax elements may include syntax elements associated with blocks of transform coefficients. Then, entropy decoding unit 150 may perform CABAC decoding operations on some syntax elements.

After the entropy decoding unit 150 performs parsing operations on the non-shared unit CU, video decoder 30 may perform a restore operation on the non-shared unit CU. To perform a restore operation on an unshared CU, video decoder 30 may perform a restore operation on each TU of a CU. By performing a restore operation for each TU of the CU, video decoder 30 may recover the residual video block associated with this CU.

As part of performing the restore operation on the TU, the inverse quantization unit 154 may inverse quantize, i.e., “dequantize,” the transform coefficient block associated with this TU. The inverse quantization module 154 may inverse quantize the block of transform coefficients using processes similar to the inverse quantization processes proposed for the HEVC standard or the defined H.264 decoding standard. The inverse quantization unit 154 may use the quantization parameter QP computed by the video encoder 20 for the CU of the transform coefficient block to determine the degree of quantization and, likewise, the degree of inverse quantization for the inverse quantization unit 154.

After the inverse quantization module 154 performs the inverse quantization operation for the transform coefficient block, the inverse transform module 156 may create a residual video block for the TU associated with the transform coefficient block. The inverse transform module 156 may apply the inverse transform to the specified block of transform coefficients to create a residual video block for the TU. For example, inverse transform module 156 may apply the inverse DCT transform, the inverse integer transform, the inverse Karhunen-Leve transform (KLT), the inverse rotational transform, the inverse directional transform, or other inverse transform to a block of transform coefficients.

In some examples, the inverse transform module 156 can determine the inverse transform to apply to the block of transform coefficients based on the signaling side of the video encoder 20. In these examples, the inverse transform module 156 can determine such an inverse transform based on the resulting transform signal in the root node of the quad tree for the tree block associated with this block of conversion factors. In other examples, the inverse transform module 156 may determine the inverse transform based on one or more encoding characteristics, such as block size, encoding mode, or the like. In some examples, inverse transform module 156 may apply cascaded inverse transform.

In some examples, motion compensation module 162 may refine the predicted video block of the PU by performing interpolation based on interpolation filters. Identifiers for interpolation filters used to compensate for motion to a fraction of the reference can be included in the syntax elements mentioned. Motion compensation module 162 can use the same interpolation filters that video encoder 20 uses when creating the predicted video block of the PU block to calculate the interpolated values for the fractional samples of the reference block. Motion compensation module 162 can determine the interpolation filters used by video encoder 20 according to the received syntax information and use these interpolation filters to create a predicted video block.

If the PU is encoded using intra-frame prediction, then intra-frame prediction unit 164 may perform intra-frame prediction to create a predicted video block for a given PU. For example, intra prediction module 164 may determine the intra prediction mode for a given PU based on the syntax elements in the bitstream. The bitstream may include syntax elements that the intra prediction module 164 may use to determine the intra prediction mode for a given PU.

In some cases, these syntax elements may indicate that the intra prediction unit 164 should use the intra prediction mode of another PU to determine the intra prediction mode of the current PU. For example, there is a possibility that the intra prediction mode for the current PU is the same as the intra prediction mode for the neighboring PU. In other words, the intra-prediction mode of the neighboring PU may be most likely for the current PU. Therefore, in this example, the bitstream may include a small syntax element that indicates that the intra-prediction mode for a given PU is the same as the intra-prediction mode of a neighboring PU. Then, the intra prediction module 164 may use this intra prediction mode to create prediction data (e.g., predicted samples) for a given PU based on video blocks for spatially adjacent PU blocks.

The recovery unit 158 may use the residual video blocks associated with the TUs of the CU, and the predicted video blocks of the PUs of the CU, that is, either intra prediction data or inter prediction data (depending on what is used), to reconstruct the video block for CU . Thus, video decoder 30 can create a predicted video block and a residual video block based on syntax elements in a bit stream and can create a video block based on a predicted video block and a residual video block.

After restoration by the video block recovery unit 158 of the CU, the filter module 159 may perform a release operation to suppress blocking artifacts associated with the CU. After module 159 performs a release operation to suppress blocking artifacts associated with the CU, video decoder 30 may store the video block of the CU in the decoded image buffer 160. The decoded image buffer 160 may provide reference images for subsequent motion compensation, intra prediction, and presentation on a display device, such as a display device 32 in FIG. 1. For example, video decoder 30 may perform intra-frame prediction or inter-frame prediction operations on PUs of other CUs based on video blocks in decoded image buffer 160.

In FIG. 4 is a flowchart illustrating an example operation 200 for creating data for a slice. Operation 200 may be performed by a video encoder, such as video encoder 20 (figures 1 and 2). The block diagram of FIG. 4 is just one example. In other examples of this operation, it is possible to create slice data in other ways.

At the beginning of operation 200, the video encoder may initialize the address of the tree block, which identifies the initial tree block of the current slice (202). The current slice may be a slice that is currently encoded by a video encoder. The initial treeblock of the current slice may be the first treeblock associated with the current slice according to the coding order of the treeblocks in the current image. To simplify the explanation, here we are talking about a tree block identified by the address of the tree block as the current tree block.

The video encoder may connect syntax elements for the current tree block to the slice data of the NAL block of the encoded slice for the current slice (204). The syntax elements for the current treeblock may include syntax elements in the quadtree of the current treeblock. The syntax elements in the quadtree of the current treeblock may include syntax elements that indicate intra prediction modes, motion information, syntax elements indicating levels of transform coefficients, etc.

In addition, the video encoder can determine if there is more data in the current slice (206). There may still be data in the current slice if the tree block indicated by the address of the tree block is in the current slice. In accordance with the determination of whether there is more data in the current slice (“NO”) in step (206), the video encoder can end operation 200 because it added all the necessary syntax elements to the slice data.

The video encoder can determine if there is more data in the current slice in various ways. For example, a video encoder may activate the coding_tree () function to output syntax elements for a treeblock. In this example, the coding_tree () function can return the moreDataFlag flag, which indicates whether there is more data in the current slice.

In accordance with the determination of whether there is more data associated with the current slice (“YES” at step 206), the video encoder can determine whether the tiles of the current image are independent, and determine whether the next tree block of the current slice is in a tile other than tile of the current treeblock of the current slice (208). As described above, image tiles may be independent if prediction within the image is prohibited (e.g., intra-frame prediction, inter-frame prediction using data in the current image, and CABAC context selection based on data from other tiles in the current image). The video encoder can determine whether the tiles of the current image are independent in various ways. For example, the sequence parameter set associated with the current image may include a tile_boundary_independence_idc syntax element. In this example, if “tile_boundary_independence_idc” is 0, then the tiles of the current image are not independent, and prediction across tile boundaries is allowed. If “tile_boundary_independence_idc” is 0, then prediction across slice boundaries is still prohibited. If “tile_boundary_independence_idc” is 1, then the tiles in the current image are independent, and prediction across tile boundaries is not allowed.

The video encoder can determine in various ways whether the next tree block of the current slice is in a different tile than the tile of the current tree block of the current slice. For example, a video encoder can determine the address of the next tree block of the current slice. In this example, the video encoder can activate the “NewTile (...)” function, which uses the address of the next tree block as a parameter and returns a “newTileflag” value that indicates whether the next tree block is in a tile other than the tile of the current tree block.

If the tiles of the current image are not independent, or the next tree block is in the same tile as the current tree block (“NO” in block 208), then the video encoder can determine whether encoding of the current image using WPP and whether the next tree block of the current slice in the WPP wave, different from the wave of the current tree block of the current slice (210). The video encoder can determine in various ways whether the next tree block of the current slice is not in the WPP wave in which the current tree block of the current slice is located. For example, a video encoder can determine the address of the next tree block of the current slice. In this example, the video encoder can activate the NewWave (...) function, which uses the address of the next tree block as a parameter and returns a value of newWaveFlag, which indicates whether the next tree block is in a WPP wave other than the WPP wave of the current tree block.

According to the definition that the encoding of the current image is performed using WPP, and that the next tree block is in a WPP wave different from the WPP wave of the current tree block (“YES” in block 210), or in accordance with the definition that the tiles the current image are independent, and that the next tree block is in a tile different from the tile of the current tree block (“YES” in block 208), the video encoder can determine if the current segment is aligned bytes. In other words, the video encoder can determine if the current segment ends at the byte boundary. The current segment is the segment associated with the image section (for example, a tile or WPP wave) with which the current treeblock is associated. In accordance with the determination that the current segment is not byte aligned (“NO” at step 212), the video encoder may connect the fill bit to the end of the current segment (214). This fill bit can have different meanings. For example, the fill bit can always have a value of 1. In other examples, the fill bit can always have a value of 0.

After attaching the fill bit to the end of the current segment, the video encoder can again determine whether the current segment is aligned bytes (212). In this way, the video encoder can continue to attach the fill bits to the end of the slice data until the current segment is aligned with bytes.

In accordance with the determination that the slice data is byte aligned (“YES” at step 212), the video encoder can update the address of the tree block (216). The video encoder can update the address of the tree block so that it indicates the next tree block according to the coding order of the tree blocks of the current image. For example, when the video encoder updates the address of the tree block, this address can identify the tree block to the right of the tree block pointed to by the address of the tree block. In FIG. 7, described in detail below, is a conceptual diagram illustrating an example coding order of treeblocks for a single image, which is divided into many tiles.

After updating the address of the treeblock, the video encoder can determine if there is more data in the current slice (218). In accordance with the determination of whether there is more data in the current slice (“YES” at step 218), or in accordance with the determination that encoding of the current image using WPP is not performed and the next tree block is in the same tile as and the current tree block (“NO” at step 210), the video encoder may attach syntax elements for the current tree block to the slice data (204). Thus, the video encoder can attach syntax elements for each tree block of the current slice to the data of the slice and can provide such filling of segments with insignificant information associated with other sections of the image, in which these segments will start from byte boundaries.

According to the determination of whether there is more data in the current slice (“NO” at step 218), the video encoder can complete operation 200 because it can attach all the syntax elements of the current slice to the slice data.

In FIG. 5 is a flowchart illustrating an example operation 250 for decoding a NAL unit of an encoded slice. A video decoder such as video decoder 30 (FIGS. 1 and 3) may perform operation 250. As shown in FIG. 5 is just one example. In other examples, other operations may be used to decode the NAL blocks of the encoded slice.

In the example of FIG. 5, a video decoder can memorize a NAL unit of an encoded slice in a byte-addressable memory (252). An encoded slice NAL unit may include a slice header and slice data. Slice data may include multiple segments. One or more segments may be filled with insignificant information so that each segment begins with a byte boundary.

After storing the encoded slice NAL unit in the memory, the video decoder can identify the position of the segments in the data of the NAL unit encoded slice (254). A video decoder can identify segment positions in various ways. For example, a video decoder can identify the position of segments based on the syntax elements in the slice header of the encoded slice NAL unit, which indicate the byte offsets of the segments. In this example, the slice header may not include a byte offset for the first slice data segment, since the position of the first segment may immediately follow the end of the slice header. In another example, the video decoder can identify the position of the segments based on the labels of the entry points in the slice data. Label entry points can be values located between segments.

After identifying the positions of the segments in the slice data, the video decoder can distribute two or more segments over two or more different decoding streams (256). Each decoding stream can parse the syntax elements of the encoded treeblocks in a segment allocated to a given decoding stream, and reconstruct the video blocks for the corresponding treeblocks, as described above.

In FIG. 6 is a conceptual diagram illustrating parallel wave processing. As described above, the image can be divided into video blocks, each of which is associated with a tree block. In FIG. 6 shows video blocks associated with tree blocks in the form of a grid of white squares. The image includes rows of tree blocks 300A-300E (designated together as "rows of tree blocks 300").

In the first stream, tree blocks are encoded in row 300A of tree blocks. In parallel, coding of tree blocks of other streams in rows 300B, 300C and 300D of tree blocks can be performed. In the example of FIG. 6, in the first stream, the coding of the tree block 302A is performed, in the second stream, the coding of the tree block 302B is performed, in the third stream, the coding of the tree block 302C is performed and in the fourth stream, the coding of the tree block 302D is performed. This description can be attributed to treeblocks 302A, 302B, 302C, and 302D, designated together as “current treeblocks 302”. Since the video encoder can start encoding a row of tree blocks after encoding more than two tree blocks of a row located directly above this row, the current tree blocks 302 are placed horizontally from each other to the width of two tree blocks.

In the example of FIG. 6, these streams can use data from treeblocks indicated by bold gray arrows when performing intra-frame prediction or inter-frame prediction for CUs in current treeblocks 302 (When streams perform inter-frame prediction for CUs, they can also use data from one or more reference frames). When a stream encodes a given treeblock, it can select one or more CABAC contexts based on information associated with previously encoded treeblocks. This thread may use one or more CABAC contexts to perform CABAC encoding on the syntax elements associated with the first CU of this treeblock. If the given treeblock is not the leftmost treeblock of the row, then this thread can select one or more CABAC contexts based on the information associated with the last CU of the treeblock to the left of this treeblock. If a given treeblock is the leftmost treeblock of a row, then the stream may select one or more CABAC contexts based on the information associated with the last CU of the treeblock above and the two treeblocks to the right of the treeblock. These streams may use data from the last CUs of the treeblocks indicated by thin black arrows to select CABAC contexts for the first CUs of the current treeblocks 302.

In FIG. 7 is a conceptual diagram illustrating an example coding order of treeblocks for an image 350 that is partitioned into multiple tiles 352A, 352B, and 352C. Each square white block in image 350 represents a video block associated with a tree block. Bold vertical dashed lines indicate approximate vertical borders of tiles. Bold gray lines indicate the approximate border of the slice.

The numbers in the video blocks indicate the positions of the corresponding treeblocks (LCUs) in the coding order of the treeblocks for the image 350. As shown in the example of FIG. 7, each of the treeblocks in the leftmost tile 352A appears in the order of coding of the treeblocks before any treeblock in the middle tile 352B. Each of the tree blocks in the middle tile 352B appears in the order of coding of the tree blocks in front of any tree block in the far right tile 352C. Within each tile 352A, 352B, and 352C, treeblocks are encoded according to the raster scan order.

The video encoder may create two NAL blocks of the encoded slice for the image 350. The first NAL block of the encoded slice may be associated with the left slice of the image 350. The first NAL block of the encoded slice may include encoded representations of tree blocks 1-23. The slice data of the first NAL block of the encoded slice may include two segments. The first segment may include encoded representations of tree blocks 1-15. The second segment may include encoded representations of the tree blocks 16-30. According to the techniques disclosed herein, the first segment may be filled with insignificant information providing the beginning of the second segment from the byte boundary.

The second NAL block of the encoded slice may be associated with the right slice of the image 350. The second NAL block of the encoded slice may include encoded representations of the tree blocks 24-45. The slice data of the second NAL block of the encoded slice may include two segments. The first segment may include encoded representations of the tree blocks 24-30. The second segment may include encoded representations of treeblocks 31-45. The first segment can be filled with insignificant information so that the second segment starts from the byte boundary.

In FIG. 8 is a conceptual diagram illustrating an example NAL block 400 of an encoded slice. As shown in the example of FIG. 8, an encoded slice NAL unit 400 includes a slice header 402 and slice data 404. Slice data 404 includes a first segment 406 and a second segment 408. Segment 406 includes encoded treeblocks 410A-410N and pad data 412. Segment 408 includes encoded treeblocks 414A-414N.

In one or more examples, the described functions may be implemented in hardware, software, firmware, or any combination thereof. When implemented by software, these functions can be stored or transmitted in the form of one or more instructions or code on a computer-readable medium and executed by a processing hardware unit. A computer-readable storage medium may include a computer-readable storage medium that corresponds to a tangible medium, such as a storage medium or a transmission medium, including any medium that enables a computer program to be transferred from one place to another, for example, according to some protocol communication. In this aspect, a computer-readable storage medium may generally correspond to (1) a tangible computer-readable storage medium that provides long-term storage, or (2) a communication medium such as a signal or carrier wave. The storage medium may be any available medium that can be accessed by one or more computers or one or more processors to extract instructions, code and / or data structures for implementing the techniques disclosed in this invention. The computer program product may include computer-readable media.

By way of example, but not limitation, said computer-readable storage medium may comprise RAM, ROM, electrically erasable programmable ROM (EEPROM), ROM on a compact disc (CD-ROM), or other optical memory device , magnetic disk storage device or other magnetic storage device, flash memory, or any other medium that can be used to store the required program code in the form of instructions or data structures, and which can be accessed by a computer. Also, any connection will correctly be called a computer-readable storage medium. For example, if commands are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable , fiber optic cable, twisted pair, DSL lines or wireless technologies such as infrared, radio and microwave are included in the definition of media. However, it should be understood that the terms “computer-readable storage medium” and “storage medium” do not include connections that carry waves, signals or other temporary media, but refer to a tangible medium of long-term storage. As used herein, the term “disc” refers to a compact disc (CD), a laser disc, an optical disc, a digital versatile disc (DVD), a floppy disc and a Blu-ray disc that reproduce data optically using lasers, while while the term “disk” usually refers to disks that reproduce data magnetically. Combinations of the above should also be included in the concept of a machine-readable storage medium.

Commands may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), user programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Accordingly, the term “processor” as used herein may refer to any of the above structures or any other structure suitable for implementing the above methods. In addition, in some aspects, the functionality described herein may be provided within specialized hardware and / or software modules configured for encoding and decoding, or included in a combination codec. Also, the techniques disclosed herein may be fully implemented in one or more circuits or logic elements.

The techniques disclosed herein may be implemented in a wide variety of devices or installations, including a cordless handset, integrated circuit (IC), or a set of integrated circuits (e.g., microprocessor set). Various components, modules or blocks are described herein in such a way as to emphasize the functional aspects of devices configured to perform the techniques disclosed herein, and this does not require their implementation using various hardware blocks. Rather, on the contrary, as discussed above, the various units can be combined in a hardware codec unit or provided as a set of interacting hardware units, including one or more processors, as described above, in combination with suitable software and / or firmware.

Various examples have been described here. These and other examples are included in the scope of the following claims.

Claims (31)

1. A method of encoding video data, the method comprising:
dividing an image into a plurality of image sections, wherein the image contains a plurality of tree blocks, wherein image sections are associated with non-overlapping subsets of tree blocks of the image; and
creating a block of network abstraction level (NAL) of the encoded slice, which includes encoded representations of tree blocks associated with the slice of the specified image, and the encoded representations of the tree blocks are grouped in the NAL block of the encoded slice into segments associated with different sections of the image, and:
one or more of these segments are filled with insignificant information, so that each of the segments begins at the byte boundary,
said segments include a predetermined segment, and
creating a NAL block of an encoded slice comprises performing an operation for filling insignificant information that attaches bits to a given segment if the next tree block is inside the specified slice and is associated with another section of the image that is different from the specified segment. 2. The method of claim 1, wherein creating the NAL unit of the encoded slice comprises creating a slice header that indicates the entry points of one or more of said segments. 3. The method of claim 2, wherein the input points for said segments indicate byte offsets of the segments. 4. The method of claim 1, wherein the image sections are tiles or waves of wave parallel processing (WPP). 5. The method according to claim 1, further comprising creating a set of parameters associated with the specified image, wherein the set of parameters includes a flag having a first value indicating that one or more of these segments are filled with insignificant information, so that each of these segments begins with a byte boundary, and
in this case, when the flag has a second value, these segments may or may not begin with the boundaries of the byte. 6. The method according to p. 1,
wherein said image sections are tiles; and
in which the creation of the NAL block of the encoded slice comprises the operation of filling insignificant information, which ensures that these segments begin at the byte boundaries only after determining that the indicated tiles are independent of each other. 7. A method for decoding video data, the method comprising:
storing a block of the network abstraction level (NAL) of the encoded slice, which includes encoded representations of tree blocks associated with the image slice, while the specified image is divided into many image sections, and the encoded representations of these tree blocks are grouped into segments associated with different image sections , moreover:
one or more of these segments are filled with insignificant information, so that each of these segments begins with a byte boundary,
said segments include a predetermined segment, and
a NAL block of an encoded slice includes bits attached to a given segment if the next tree block is inside the specified slice and is associated with a different section of the image other than the specified segment; and
decoding encoded representations of treeblocks. 8. The method of claim 7, wherein the NAL unit of the encoded slice comprises a slice header that indicates entry points for one or more of said segments. 9. The method of claim 8, wherein the input points for said segments indicate byte offsets of the segments. 10. The method of claim 7, wherein the image sections are tiles or waves of wave parallel processing (WPP). 11. The method of claim 7, further comprising storing a set of parameters associated with said image, wherein the set of parameters includes a flag having a first value indicating that one or more of said segments are filled with insignificant information, so that each of said segments begins with a byte boundary, and
in this case, when the flag has a second value, these segments may or may not begin with byte boundaries. 12. The method according to p. 7,
wherein said image sections are tiles; and
in which one or more of the indicated segments is filled with insignificant information, so that the indicated segments begin with byte boundaries when the indicated tiles are independent of each other. 13. The method of claim 7, wherein decoding the encoded representations of said treeblocks comprises parallel decoding of encoded representations of the treeblocks in two or more of said segments. 14. A video encoding device that encodes video data, wherein the video encoding device comprises:
a storage medium configured to store video data; and
one or more processors configured for:
dividing the image of the video data into a plurality of image sections, wherein the image contains a plurality of tree blocks, wherein the image sections are associated with non-overlapping subsets of the tree blocks of the image; and
creating a block of network abstraction level (NAL) of the encoded slice, which includes encoded representations of tree blocks that are associated with the slice of a given image, and encoded representations of tree blocks are grouped in the NAL block of the encoded slice into segments associated with different sections of the image, and:
one or more of these segments are filled with insignificant information, so that each of the segments begins at the byte boundary,
said segments include a predetermined segment, and
one or more processors is configured to perform an operation of filling insignificant information that attaches bits to a given segment if the next tree block is inside the specified slice and is associated with another section of the image other than the specified segment. 15. The video coding apparatus of claim 14, wherein the one or more processors are configured to create a coded slice NAL unit, so that the coded slice NAL unit includes a slice header that indicates entry points for one or more of said segments. 16. The video coding apparatus of claim 15, wherein the input points for said segments indicate byte offsets of the segments. 17. The video coding apparatus of claim 14, wherein the image sections are tiles or wave parallel processing (WPP) waves. 18. The video coding apparatus of claim 14, wherein one or more processors are configured to create a parameter set associated with the specified image, wherein the parameter set includes a flag having a first value indicating that one or more of these segments are filled with insignificant information, so that each of these segments begins with a byte boundary, and
in this case, when the flag has a second value, these segments may or may not begin with the boundaries of the byte. 19. The video coding device according to p. 14,
wherein said image sections are tiles; and
in which one or more processors are configured to perform the operation of filling insignificant information, which ensures that these segments begin at byte boundaries only after determining that these tiles are independent of each other. 20. The video encoding device according to claim 14, wherein the video encoding device is a mobile computing device. 21. A video decoding device that decodes video data, wherein the video decoding device comprises:
a memory in which a block of network abstraction level (NAL) block of the encoded slice is stored, which includes encoded representations of tree blocks associated with the image slice, wherein said image is divided into a plurality of image sections, wherein encoded representations of said tree blocks are grouped into segments connected with different sections of the image, moreover:
one or more of these segments are filled with insignificant information, so that each of these segments begins with a byte boundary,
said segments include a predetermined segment, and
a NAL block of an encoded slice includes bits attached to a given segment if the next tree block is inside the specified slice and is associated with a different section of the image other than the specified segment; and
one or more processors that are configured to decode encoded representations of treeblocks. 22. The video decoding apparatus of claim 21, wherein the NAL unit of the encoded slice comprises a slice header that indicates entry points for one or more of said segments. 23. The video decoding apparatus of claim 22, wherein the input points for said segments indicate byte offsets of the segments. 24. The video decoding apparatus of claim 21, wherein the image sections are tiles or wave parallel processing (WPP) waves. 25. The video decoding device according to claim 21, in which a set of parameters associated with the specified image is stored in memory, the set of parameters includes a flag having a first value indicating that one or more of these segments are filled with insignificant information, so that each of these segments begins with a byte boundary, and
in this case, when the flag has a second value, these segments may or may not start from the boundaries of the byte. 26. The video decoding device according to claim 21, wherein the video decoding device is a mobile computing device. 27. The video decoding device according to claim 21, in which one or more processors are configured to parallelly decode the encoded representations of these tree blocks in two or more segments. 28. A permanent computer-readable storage medium that stores instructions that, when executed by one or more processors, configure a video coding device for:
dividing an image into a plurality of image sections, wherein the image contains a plurality of tree blocks, wherein the image sections are associated with non-overlapping subsets of tree blocks of a given image; and
creating a block of network abstraction level (NAL) of the encoded slice, which includes encoded representations of tree blocks associated with the slice of the image, and encoded representations of tree blocks are grouped in the NAL block of the encoded slice into segments associated with different sections of the image, and:
one or more of these segments are filled with insignificant information, so that each of the segments begins at the byte boundary,
said segments include a predetermined segment; and
one or more processors is configured to perform an operation of filling insignificant information that attaches bits to a given segment if the next tree block is inside the specified slice and is associated with another section of the image other than the specified segment. 29. A computer-readable storage medium that stores instructions that, when executed by one or more processors, configure a video decoding device for
storing a network abstraction layer (NAL) block of a coded slice that includes encoded representations of tree blocks associated with an image slice, wherein said image is divided into a plurality of image sections, wherein coded representations of said tree blocks are grouped into segments associated with different image sections , moreover:
one or more of these segments are filled with insignificant information, so that each of these segments begins with a byte boundary,
said segments include a predetermined segment; and
a NAL block of an encoded slice includes bits attached to a given segment if the next tree block is inside the specified slice and is associated with a different section of the image other than the specified segment; and
decoding encoded representations of treeblocks. 30. A video encoding device that encodes video data, wherein the video encoding device comprises:
means for dividing the image into a plurality of image sections, wherein the image comprises a plurality of tree blocks, wherein the image sections are associated with non-overlapping subsets of tree blocks of the image; and
means for creating a block of network abstraction layer (NAL) block of the encoded slice, which includes encoded representations of tree blocks that are associated with a slice of a given image, wherein encoded representations of tree blocks are grouped in a NAL block of the encoded slice into segments associated with different sections of the image, :
one or more of these segments are filled with insignificant information, so that each of the segments begins at the byte boundary,
said segments include a predetermined segment, and
creating a NAL block of an encoded slice comprises performing an operation for filling insignificant information that attaches bits to a given segment if the next tree block is inside the specified slice and is associated with another section of the image that is different from the specified segment. 31. A video decoding device that decodes video data, wherein the video decoding device comprises:
means for storing a block of network abstraction level (NAL) block of the encoded slice, which includes encoded representations of tree blocks associated with the image slice, wherein said image is divided into many image sections, wherein encoded representations of these tree blocks are grouped into segments associated with different sections of the image, moreover:
one or more of these segments are filled with insignificant information, so that each of these segments begins with a byte boundary,
said segments include a predetermined segment, and the NAL block of the encoded slice includes bits attached to the predetermined segment if the next tree block is inside the specified slice and is associated with a different section of the image other than the predetermined segment; and
means for decoding encoded representations of treeblocks.

Images