JP2022516114A - Video encoders, video decoders, and corresponding methods - Google Patents

Abstract

The video coding mechanism is disclosed. The mechanism involves dividing the picture into multiple first level tiles. A subset of first level tiles is divided into multiple second level tiles. One or more first-level tiles and one or more second-level tiles so that all second-level tiles created from a single first-level tile are assigned to the same tile group. Assigned to a tile group. The first level tile and the second level tile are encoded in the bitstream. The bitstream is stored for communication to the decoder.

Description

Cross-reference to related applications This patent application is incorporated herein by reference in a US provisional patent application entitled "Flexible Tiling in Video Coding" filed by Ye-Kui Wang et al. On December 28, 2018. Claim the interests of / 786,167.

The present disclosure relates generally to video coding, and more specifically to flexible video tiling schemes that support multiple tiles with different resolutions in the same picture.

Even the amount of video data required to draw a relatively short video can be considerable, which means that the data is streamed across a communication network with limited bandwidth capacity or otherwise. It can cause difficulties when it comes to communicating on. Therefore, video data is generally compressed before being communicated across modern telecommunications networks. The size of the video can also be an issue when the video is stored on the storage device, as memory resources may be limited. To code video data prior to transmission or storage, thereby reducing the amount of data required to depict digital video images, video compression devices are often software and / or hardware at the source. To use. The compressed data is then received at the destination by a video restore device that decodes the video data. As network resources are limited and the demand for higher video quality is constantly increasing, improved compression and restoration techniques that improve compression ratio with little sacrifice in image quality are desirable.

In one embodiment, the disclosure includes a method performed within an encoder, wherein the method divides the picture into multiple first level tiles by the encoder processor and the first level tile by the processor. Dividing a subset of tiles into multiple second level tiles, each tile group has several first level tiles, each sequence of second level tiles is a single first level One or more first-level tiles and one or more second-level tiles by the processor to include one or more contiguous sequences of second-level tiles, or combinations thereof, that are split from the tiles of Assigning to tile groups, encoding first-level and second-level tiles in the bitstream by the processor, and putting the bitstream in the encoder's memory for communication to the decoder. Including to remember. Some streaming applications (eg, virtual reality (VR) and teleconferencing) can be improved if a single image containing multiple areas encoded at different resolutions can be sent. Some slicing and tiling mechanisms, such as raster scan-based slicing and / or tiling, may not support such features as tiles at different resolutions may be treated differently. For example, tiles at the first resolution may contain a single slice of data, while tiles at the second resolution may carry multiple slices of data due to differences in pixel density. To support this feature, a flexible tiling scheme may be employed that includes first level tiles and second level tiles. Second level tiles are created by partitioning the first level tiles. In this tiling scheme, the first level tiles contain one slice of data at the first resolution, and the first level tiles containing the second level tiles at the second resolution. Allows you to include multiple slices. Such a flexible tiling scheme allows the encoder / decoder (codec) to support pictures containing multiple resolutions, thus enhancing the functionality of both the encoder and the decoder. The present disclosure describes a mechanism for integrating tile groups into a flexible tiling scheme. A tile group can include a first level tile and / or a complete set of second level tiles separated from one or more first level tiles. This technique prevents the second level tiles from a single first level tile from being split into different tile groups. Therefore, the disclosed mechanism allows flexible tiling tiles to be included within a tile group, which allows coding tools to be applied to various tiles based on the tile group. .. Preventing the second level tiles from a single first level tile from being split into different groups reduces the complexity of the resulting flexible tiling scheme with tile groups. To. Accordingly, the present disclosure further enhances the functionality of both encoders and decoders while reducing processor and / or memory resource usage.

In one embodiment, the disclosure includes a method performed within an encoder, wherein the method divides the picture into multiple first level tiles by the encoder processor and the first level tile by the processor. Dividing a subset of tiles into multiple second level tiles and allowing all second level tiles created from a single first level tile to be assigned to the same tile group. The processor assigns first-level and second-level tiles to one or more tile groups, and the processor encodes first-level and second-level tiles into a bitstream. Includes doing and storing the bitstream in the encoder's memory for communication to the decoder. Some streaming applications (eg VR and teleconferencing) can be improved if a single image containing multiple areas encoded at different resolutions can be sent. Some slicing and tiling mechanisms, such as raster scan-based slicing and / or tiling, may not support such features as tiles at different resolutions may be treated differently. For example, tiles at the first resolution may contain a single slice of data, while tiles at the second resolution may carry multiple slices of data due to differences in pixel density. To support this feature, a flexible tiling scheme may be employed that includes first level tiles and second level tiles. Second level tiles are created by partitioning the first level tiles. In this tiling scheme, the first level tiles contain one slice of data at the first resolution, and the first level tiles containing the second level tiles at the second resolution. Allows you to include multiple slices. Such a flexible tiling scheme allows the codec to support pictures with multiple resolutions, thus enhancing the functionality of both the encoder and the decoder. The present disclosure describes a mechanism for integrating tile groups into a flexible tiling scheme. A tile group can include a first level tile and / or a complete set of second level tiles separated from one or more first level tiles. This technique prevents the second level tiles from a single first level tile from being split into different tile groups. Therefore, the disclosed mechanism allows flexible tiling tiles to be included within a tile group, which allows coding tools to be applied to various tiles based on the tile group. .. Preventing the second level tiles from a single first level tile from being split into different groups reduces the complexity of the resulting flexible tiling scheme with tile groups. To. Accordingly, the present disclosure further enhances the functionality of both encoders and decoders while reducing processor and / or memory resource usage.

Optionally, in any of the aforementioned embodiments, another implementation of the embodiment is that the first level tiles outside the subset contain picture data at the first resolution and the second level tiles. Provided to include picture data at a second resolution different from the first resolution.

Optionally, in any of the aforementioned embodiments, another implementation of the embodiment is that each first level tile within a subset of the first level tiles has two or more complete second levels. Offer to include tiles.

Optionally, in any of the aforementioned embodiments, another embodiment of the embodiment allows the first level tiles and the second level tiles to be encoded according to the scan order and encoded according to the scan order. Encoding first-level tiles in raster scan order, and interrupting raster scan order coding of first-level tiles when one of the second-level tiles is encountered. Provided to include encoding all consecutive second level tiles in raster scan order and then continuing raster scan order coding of first level tiles.

Optionally, in any of the aforementioned embodiments, another implementation of the embodiment is that all second level tiles separated from the current first level tile are followed by second level tiles. Provides to be encoded before encoding any second level tile separated from.

Optionally, in any of the aforementioned embodiments, another implementation of the embodiment is such that each of one or more tile groups covers the rectangular portion of the picture with all tiles in the assigned tile group. Provides to be constrained to.

In one embodiment, the disclosure comprises a method carried out within a decoder, wherein a bitstream containing pictures segmented into multiple first level tiles, via a receiver, is a processor of the decoder. By receiving by, a subset of the first level tiles are further subdivided into multiple second level tiles, where each tile group has several first level tiles, the second level. First level tiles and so that each sequence of tiles contains one or more consecutive sequences of second level tiles, or combinations thereof, that are split from a single first level tile. The second level tiles are assigned to one or more tile groups, and the processor decodes the first level tiles and the second level tiles based on the one or more tile groups. Includes the processor generating a reconstructed video sequence for display based on the first level tiles and the second level tiles that have been made. Some streaming applications (eg VR and teleconferencing) can be improved if a single image containing multiple areas encoded at different resolutions can be sent. Some slicing and tiling mechanisms, such as raster scan-based slicing and / or tiling, may not support such features as tiles at different resolutions may be treated differently. For example, tiles at the first resolution may contain a single slice of data, while tiles at the second resolution may carry multiple slices of data due to differences in pixel density. To support this feature, flexible tiling schemes may be employed that include first level tiles and second level tiles. Second level tiles are created by partitioning the first level tiles. In this tiling scheme, the first level tiles contain one slice of data at the first resolution, and the first level tiles containing the second level tiles at the second resolution. Allows you to include multiple slices. Such a flexible tiling scheme allows the codec to support pictures with multiple resolutions, thus enhancing the functionality of both the encoder and the decoder. The present disclosure describes a mechanism for integrating tile groups into a flexible tiling scheme. A tile group can include a first level tile and / or a complete set of second level tiles separated from one or more first level tiles. This technique prevents the second level tiles from a single first level tile from being split into different tile groups. Therefore, the disclosed mechanism allows flexible tiling tiles to be included within a tile group, which allows coding tools to be applied to various tiles based on the tile group. .. Preventing the second level tiles from a single first level tile from being split into different groups reduces the complexity of the resulting flexible tiling scheme with tile groups. To. Accordingly, the present disclosure further enhances the functionality of both encoders and decoders while reducing processor and / or memory resource usage.

In one embodiment, the disclosure comprises a method carried out within a decoder, wherein a bitstream containing pictures segmented into multiple first level tiles, via a receiver, is a processor of the decoder. A subset of the first level tiles is further subdivided into multiple second level tiles and all second levels created from a single first level tile. First level tiles and second level tiles are assigned to one or more tile groups so that the tiles of the first level and the second level tiles are assigned to one or more tile groups, based on receiving and one or more tile groups. 1st level tiles and 2nd level tiles are decoded by the processor, and a reconstructed video sequence for display based on the decoded 1st level tiles and 2nd level tiles. Includes generating by the processor. Some streaming applications (eg VR and teleconferencing) can be improved if a single image containing multiple areas encoded at different resolutions can be sent. Some slicing and tiling mechanisms, such as raster scan-based slicing and / or tiling, may not support such features as tiles at different resolutions may be treated differently. For example, tiles at the first resolution may contain a single slice of data, while tiles at the second resolution may carry multiple slices of data due to differences in pixel density. To support this feature, a flexible tiling scheme may be employed that includes first level tiles and second level tiles. Second level tiles are created by partitioning the first level tiles. In this tiling scheme, the first level tiles contain one slice of data at the first resolution, and the first level tiles containing the second level tiles at the second resolution. Allows you to include multiple slices. Such a flexible tiling scheme allows the codec to support pictures with multiple resolutions, thus enhancing the functionality of both the encoder and the decoder. The present disclosure describes a mechanism for integrating tile groups into a flexible tiling scheme. A tile group can include a first level tile and / or a complete set of second level tiles separated from one or more first level tiles. This technique prevents the second level tiles from a single first level tile from being split into different tile groups. Therefore, the disclosed mechanism allows flexible tiling tiles to be included within a tile group, which allows coding tools to be applied to various tiles based on the tile group. .. Preventing the second level tiles from a single first level tile from being split into different groups reduces the complexity of the resulting flexible tiling scheme with tile groups. To. Accordingly, the present disclosure further enhances the functionality of both encoders and decoders while reducing processor and / or memory resource usage.

Optionally, in any of the aforementioned embodiments, another implementation of the embodiment is that the first level tiles outside the subset contain picture data at the first resolution and the second level tiles. Provided to include picture data at a second resolution different from the first resolution.

Optionally, in any of the aforementioned embodiments, another implementation of the embodiment is that each first level tile within a subset of the first level tiles has two or more complete second levels. Offer to include tiles.

Optionally, in any of the aforementioned embodiments, another embodiment of the embodiment is that the first level tiles and the second level tiles are decoded according to the scan order and are decoded according to the scan order. Decoding tiles at the first level in raster scan order, interrupting raster scan order coding of tiles at the first level when encountering one of the second level tiles, and all in succession Provided to include encoding the second level tiles of the first level tile in raster scan order and then continuing the raster scan order decoding of the first level tiles.

Optionally, in any of the aforementioned embodiments, another implementation of the embodiment is that all second level tiles separated from the current first level tile are followed by second level tiles. Provides decryption before decrypting any second level tile sorted from.

Optionally, in any of the aforementioned embodiments, another implementation of the embodiment is such that each of one or more tile groups covers the rectangular portion of the picture with all tiles in the assigned tile group. Provides to be constrained to.

In one embodiment, the present disclosure comprises a video coding device comprising a processor, a receiver coupled to the processor, and a transmitter coupled to the processor, wherein the processor, receiver, and transmitter are described above. It is configured to perform any of the methods of the embodiment.

In one embodiment, the present disclosure includes a non-temporary computer-readable medium comprising a computer programming product for use by a video coding device, wherein the computer programming product is any of the aforementioned embodiments when executed by a processor. It comprises computer-executable instructions stored on a non-temporary computer-readable medium, such as causing a video coding device to perform such a method.

In one embodiment, the present disclosure comprises a classification means for classifying a picture into a plurality of first level tiles and a subset of the first level tiles into a plurality of second level tiles. , One or more first-level tiles and one or more second-level tiles so that all second-level tiles created from a single first-level tile are assigned to the same tile group. Allocation means for allocating to tile groups, encoding means for encoding first-level and second-level tiles into a bitstream, and bitstreams for communication to the decoder. Includes an encoder with storage means for storage.

Optionally, in any of the aforementioned embodiments, another implementation of the embodiment provides that the encoder is further configured to perform any of the methods of the aforementioned embodiments.

In one embodiment, the present disclosure is a receiving means for receiving a bit stream containing pictures segmented into a plurality of first level tiles, wherein a subset of the first level tiles is a plurality of first level tiles. The first level tiles and the first level tiles are further subdivided into two level tiles so that all second level tiles created from a single first level tile are assigned to the same tile group. Receiving means, where two-level tiles are assigned to one or more tile groups, and decoding means for decoding first-level and second-level tiles based on one or more tile groups. Includes a decoder with a generator for generating a reconstructed video sequence for display based on the decrypted first level tiles and second level tiles.

Optionally, in any of the aforementioned embodiments, another implementation of the embodiment provides that the decoder is further configured to perform any method of any of the aforementioned embodiments. do.

For clarity, any one of the above embodiments may be combined with any one or more of the other embodiments described above to create new embodiments within the scope of the present disclosure. May be done.

These and other features will be more clearly understood from the embodiments for carrying out the following inventions incorporated with the accompanying drawings and claims.

For a more complete understanding of the present disclosure, references are made herein to the following brief description incorporated with respect to the accompanying drawings and embodiments for carrying out the invention, with similar reference numbers representing similar parts.

It is a flowchart of an exemplary method of coding a video signal. FIG. 3 is a schematic of an exemplary coding and decoding (codec) system for video coding. It is a schematic diagram which shows an exemplary video encoder. It is a schematic diagram which shows an exemplary video decoder. FIG. 6 is a schematic diagram showing an exemplary bitstream containing a coded video sequence. A diagram illustrating an exemplary mechanism for creating an extractor track to combine multiple resolution subpictures from different bitstreams into a single picture for use in a virtual reality (VR) application. be. A diagram illustrating an exemplary mechanism for creating an extractor track to combine multiple resolution subpictures from different bitstreams into a single picture for use in a virtual reality (VR) application. be. A diagram illustrating an exemplary mechanism for creating an extractor track to combine multiple resolution subpictures from different bitstreams into a single picture for use in a virtual reality (VR) application. be. A diagram illustrating an exemplary mechanism for creating an extractor track to combine multiple resolution subpictures from different bitstreams into a single picture for use in a virtual reality (VR) application. be. A diagram illustrating an exemplary mechanism for creating an extractor track to combine multiple resolution subpictures from different bitstreams into a single picture for use in a virtual reality (VR) application. be. FIG. 3 illustrates an exemplary video conference application that stitches together multiple resolution pictures from different bitstreams into a single picture for display. FIG. 6 illustrates an exemplary flexible video tiling scheme capable of supporting multiple tiles with different resolutions in the same picture. FIG. 6 illustrates an exemplary flexible video tiling scheme capable of supporting multiple tiles with different resolutions in the same picture. FIG. 6 illustrates an exemplary flexible video tiling scheme capable of supporting multiple tiles with different resolutions in the same picture. FIG. 6 illustrates an exemplary flexible video tiling scheme capable of supporting multiple tiles with different resolutions in the same picture. It is a schematic diagram of an exemplary video coding device. It is a flowchart of an exemplary method of encoding an image by adopting a flexible tiling method. It is a flowchart of an exemplary method of decoding an image by adopting a flexible tiling method. It is a schematic diagram of an exemplary system for coding a video sequence by adopting a flexible tiling scheme.

Initially, exemplary implementations of one or more embodiments are provided below, but any number of disclosed systems and / or methods, whether currently known or present. Please understand that it may be carried out using the technique of. The present disclosure should by no means be limited to the exemplary implementations, drawings, and techniques shown below, including the exemplary designs and implementations illustrated and described herein, and those of equivalents. May be amended within the scope of the appended claims along with the full scope of.

As used herein, a coding tree block (CTB), a coding tree unit (CTU), a coding unit (CU), a coded video sequence (CVS), and a joint. Video Experts Team (JVET), motion constrained tile set (MCTS), maximum transfer unit (MTU), network abstraction layer (NAL), picture order Count (POC: picture order count), raw byte sequence payload (RBSP), sequence parameter set (SPS), versatile video coding (VVC), and working draft (WD). Various acronyms such as: working draft) are adopted.

Many video compression techniques can be employed to reduce the size of video files with minimal loss of data. For example, video compression techniques can include performing spatial (eg, intra-picture) and / or time (eg, inter-picture) predictions to reduce or eliminate data redundancy in a video sequence. For block-based video coding, a video slice (for example, a video picture or part of a video picture) may be divided into video blocks, where the video blocks are tree blocks, coding tree blocks (CTBs), coding tree units (eg, coding tree units). It may also be called a CTU), a coding unit (CU), and / or a coding node. The video blocks in the intracoded (I) slice of the picture are coded using spatial prediction relative to the reference sample in the adjacent blocks in the same picture. A video block in an intercoded unidirectional (P) or bidirectional (B) slice of a picture is a spatial prediction based on a reference sample in an adjacent block in the same picture, or another reference. It may be coded by adopting a time prediction relative to the reference sample in the picture. Pictures may be referred to as frames and / or images, and reference pictures may be referred to as reference frames and / or reference images. Spatial or temporal prediction results in predictive blocks that represent image blocks. The residual data represents the pixel difference between the original image block and the predicted block. Thus, the intercoded block is encoded according to a motion vector pointing to the block of the reference sample forming the predictive block and residual data showing the difference between the coded block and the predictive block. The intracoded block is encoded according to the intracoding mode and the residual data. For further compression, the residual data may be converted from the pixel area to the conversion area. These result in a residual conversion factor, which may be quantized. The quantization conversion coefficients may be initially arranged in a two-dimensional array. The quantized transformation coefficients may be scanned to produce a one-dimensional vector of transformation coefficients. Entropy coding may be applied to achieve even greater compression. Such video compression techniques are described in more detail below.

To ensure that the encoded video may be decoded accurately, the video is encoded and decoded according to the corresponding video coding standard. The video coding standards are the International Telecommunications Union (ITU) Standardization Sector (ITU-T) H.261, International Standardization Organization / International Electrical Standards Conference (ISO / IEC) Motion Picture Expert Group (MPEG) -1 Part 2, ITU- Advanced video coding, also known as T H.262 or ISO / IEC MPEG-2 Part 2, ITU-T H.263, ISO / IEC MPEG-4 Part 2, ITU-T H.264 or ISO / IEC MPEG-4 Part 10. Includes (AVC: Advanced Video Coding), and High Efficiency Video Coding (HEVC), also known as ITU-T H.265 or MPEG-H Part 2. AVC includes Flexible Video Coding (SVC), Multiview Video Coding (MVC) and Multiview Video Coding Plus Depth (MVC + D), and 3D (3D) AVC ( Includes extensions such as 3D-AVC). HEVC includes extensions such as scalable HEVC (SHVC), multi-view HEVC (MV-HEVC), and 3D HEVC (3D-HEVC). The Joint Video Expert Team (JVET) between ITU-T and ISO / IEC has begun to develop a video coding standard called Versatile Video Coding (VVC). VVC is included in the Working Draft (WD), which includes JVET-L1001-v5.

To code a video image, the image is first partitioned and the dividers are coded into the bitstream. Various picture classification methods are available. For example, an image can be divided into regular slices, dependent slices, tiles, and / or according to Wavefront Parallel Processing (WPP). For simplicity, HEVC limits the encoder so that when dividing slices into groups of CTBs for video coding, only normal slices, dependent slices, tiles, WPPs, and combinations thereof can be used. Such divisions can be applied to support maximum transmission unit (MTU) size matching, parallelism, and reduced end-to-end delay. The MTU indicates the maximum amount of data that can be transmitted in a single packet. When the packet payload exceeds the MTU, the payload is split into two packets through a process called fragmentation.

Regular slices, also referred to simply as slices, are segmented parts of the image that can be reconstructed independently of other regular slices in the same picture, despite some interdependencies due to loop filtering behavior. be. Each normal slice is encapsulated within its own network abstraction layer (NAL) unit for transmission. In addition, in-picture predictions (intrasample predictions, motion information predictions, coding mode predictions), and entropy coding dependencies across slice boundaries may be disabled to support independent reconstruction. Such independent reconstruction supports parallelization. For example, slice-based parallelization usually employs minimal inter-processor or inter-core communication. However, each slice is usually independent and each slice is associated with a separate slice header. The use of regular slices can result in significant coding overhead due to the bit cost of the slice header per slice and due to the lack of prediction across slice boundaries. In addition, normal slices may be employed to support alignment to MTU size requirements. In detail, each normal slice can be encapsulated in a separate NAL unit and coded independently, so each normal slice is in the MTU scheme to avoid breaking the slice into multiple packets. Should be smaller than the MTU. Therefore, the parallelization goal and the MTU size matching goal may impose conflicting demands on the slice layout in the picture.

Dependent slices are similar to regular slices, but have a shortened slice header, allowing division of image tree block boundaries without breaking in-picture predictions. Therefore, dependent slices allow a normal slice to be fragmented into multiple NAL units, which means that some of the normal slices are sent out before the coding of the entire normal slice is complete. By enabling it, it results in a reduced end-to-end delay.

A tile is a segmented portion of an image created by the horizontal and vertical boundaries that create the columns and rows of the tile. Tiles may be coded in raster scan order (right-to-left and top-to-bottom). The CTB scan order is local within the tile. Therefore, the CTB in the first tile is coded in raster scan order before proceeding to the CTB in the next tile. Like regular slices, tiles break in-picture predictive and entropy-decoding dependencies. However, tiles do not have to be included within individual NAL units and therefore tiles do not have to be used for MTU size matching. Each tile can be processed by one processor / core, and interprocessor / core communication employed for in-picture prediction between processing units that decode adjacent tiles (adjacent tiles of the same slice). It may be limited to propagating shared slice headers (when in) and performing loop filtering related sharing of reconstructed samples and metadata. When a slice contains more than one tile, the entry point byte offset for each tile other than the first entry point offset in the slice may be signaled in the slice header. For each slice and tile, the following conditions: 1) all coded tree blocks in the slice belong to the same tile, and 2) all coded tree blocks in the tile belong to the same slice. At least one of the belongings should be fulfilled.

In WPP, images are divided into a single line of CTB. The entropy decoding and prediction mechanism may use data from the CTB in other rows. Parallel processing is enabled through parallel decoding of CTB rows. For example, the current row may be decrypted in parallel with the preceding row. However, decoding the current row is delayed by two CTBs from the decoding process of the preceding row. This delay ensures that the data related to the CTB above the current CTB in the current row and the CTB above and to the right are available before the current CTB is coded. This technique looks like a wavefront when represented graphically. This staggered beginning allows parallelization with as many processors / cores as an image contains CTB rows at most. Inter-processor / core-to-core communication to enable intra-picture prediction can be considerable, as intra-picture prediction between adjacent tree block rows in the picture is allowed. WPP classification takes into account NAL unit size. Therefore, WPP does not support MTU size matching. However, to perform MTU size matching as needed, slices can usually be used with WPP with some coding overhead.

Tiles may also include a set of motion constrained tiles. A motion constraint tile set (MCTS) limits the associated motion vector to point to a complete sample location inside the MCTS and a fractional sample location that requires only the complete sample location inside the MCTS for interpolation. It is a tile set designed to be used. In addition, the use of motion vector candidates for temporal motion vector prediction, derived from the outer block of MCTS, is rejected. In this way, each MCTS may be independently decoded without the presence of tiles not contained within the MCTS. Temporal MCTS supplemental enhancement information (SEI) messages may be used to indicate the presence of MCTS in the bitstream and to signal MCTS. The MCTS SEI message provides additional information that can be used in the MCTS subbitstream extraction (specified as part of the SEI message semantics) to generate a matching bitstream for MCTS. The information defines several MCTSs each, and the low-byte sequence payload (RBSP) bytes, sequence parameter set (SPS), and MCTS subbitstream extraction process of the replacement video parameter set (VPS). Contains several extraction information sets, including a picture parameter set (PPS) that should be used in it. When a subbitstream is extracted according to the MCTS subbitstream extraction process, one or all of the slice address related syntax elements (including first_slice_segment_in_pic_flag and slice_segment_address) adopt different values in the extracted subbitstream. The parameter sets (VPS, SPS, and PPS) may be rewritten or replaced, and the slice headers may be updated.

Various tiling schemes may be employed when partitioning the picture for further coding. As a particular example, tiles can be assigned to tile groups that can replace slices in some examples. In some examples, each tile group can be extracted independently of the other tile groups. Therefore, tile grouping may support parallelization by allowing each tile group to be assigned to a different processor. Tile groups can be assigned in raster scan order or constrained to form a rectangular shape of the area in the picture. Explicit tile identifier (ID) signaling may be used to support such tile groups. On some systems, the tile ID is always assigned to be the same as the tile index. Explicit tile ID signaling allows the tile ID to be different from the tile index. Having explicit tile ID signaling supports the extraction of MCTS from the bitstream without the need to update the tile group headers. Note that the explicit signaling of tile IDs and the corresponding use as addresses for tile groups may be specific to HEVC-style tile structure definitions and signaling. If the tile structure definition and / or signaling is modified, the signaling of tile ID by the explicit tile ID mechanism may be inaccurate and / or inapplicable in some examples. Tile grouping and explicit tile ID signaling can be employed, for example, when the decoder may not want to decode the entire image. As a specific example, a video coding scheme may be employed to support virtual reality (VR) video, which may be encoded according to the Omnidirectional Media Application Format (OMAF).

In VR video, one or more cameras may record the environment around the cameras. The user can then watch the VR video as if the user were in the same location as the camera. In VR video, the picture surrounds the entire environment around the user. The user then sees a subpart of the picture. For example, the user may adopt a head-mounted display that changes a sub-part of the displayed picture based on the movement of the user's head. The portion of the video being displayed may be referred to as the viewport.

Therefore, a different feature of omnidirectional video is that only the viewport is displayed at any particular time. This is in contrast to other video applications where the entire video may be displayed. This feature is intended to improve the performance of omnidirectional video systems, for example, through selective delivery according to the user's viewport (or any other criteria such as recommended viewport timed metadata). It may be used. Viewport-dependent delivery may be enabled, for example, by adopting area-by-region packing and / or viewport-dependent video coding. Performance improvements may result in smaller transmit bandwidth, lower decoding complexity, or both when compared to other omnidirectional video systems when adopting the same video resolution / quality.

An exemplary viewport-dependent behavior is a valid equirectangle projection (ERP) with a resolution (5K) of 5000 samples (eg, 5120 x 2560 luma samples) using a HEVC-based viewport-dependent OMAF video profile. It is an MCTS-based method for achieving resolution. This technique is described in more detail below. However, in general, this technique divides VR video into tile groups and encodes the video at multiple resolutions. The decoder can indicate the viewport currently used by the user during streaming. The video server that provides the VR video data can then transfer the tile groups associated with the viewport at a higher resolution and the tile groups that are not seen at a lower resolution. This allows the user to watch the VR video in high resolution without having to send the entire picture in high resolution. Unseen sub-parts are discarded and therefore the user does not have to be aware of the lower resolution. However, if the user changes the viewport, a lower resolution tile group may be displayed to the user. The resolution of the new viewport may then be increased as the video progresses. In order to implement such a system, a picture containing both higher resolution tile groups and lower resolution tile groups should be created.

In another example, the video conference application may be designed to transfer pictures containing multiple resolutions. For example, a video conference may include multiple participants. Participants currently speaking may be displayed in higher resolution and other participants may be displayed in lower resolution. In order to implement such a system, a picture containing both higher resolution tile groups and lower resolution tile groups should be created.

Various flexible tiling mechanisms are disclosed herein to support the creation of pictures with sub-pictures coded at multiple resolutions. For example, video can be coded in multiple resolutions. The video can also be coded by adopting subpictures at each resolution. A sub-picture with a lower resolution is smaller than a sub-picture with a higher resolution. To create a picture with multiple resolutions, the picture can be divided into first level tiles. Subpictures from the highest resolution can be included directly within the first level tiles. Further, the first level tiles can be divided into second level tiles, which are smaller than the first level tiles. Therefore, smaller second level tiles can directly accept lower resolution subpictures. In this way, to use a consistent addressing scheme, slices from each resolution are sliced through the tile index relationship without the need for dynamically readdressing tiles with different resolutions. Can be compressed into a single picture. The first level tiles and the second level tiles may be implemented as MCTS and therefore may accept motion constrained image data at different resolutions. The present disclosure includes many aspects. As a specific example, the first level tile is split into the second level tiles. The first level tile and the second level tile can then be included in the tile group. A tile group is one or more of an integer number of first-level tiles and / or a second-level tile in which each sequence of second-level tiles is split from a single first-level tile. Can be constrained to contain a contiguous sequence of. This technique may ensure that all second level tiles created from a single first level tile are assigned to the same tile group. Another particular example describes a scan sequence for coding a flexible tiling scheme. In this example, the first level tiles are coded in raster scan order for picture and / or tile group boundaries. When a second level tile is encountered, the first level tile scan is interrupted. A contiguous sequence of second level tiles is then scanned in raster scan order against the first level tiles from which such second level tiles are partitioned. The scan order then proceeds to the next sequence of consecutive second level tiles, if any. Otherwise, the first level tile scan is continued. This process continues until the tile group and / or picture is encoded or decrypted, as is the case.

FIG. 1 is a flowchart of an exemplary operation method 100 for coding a video signal. Specifically, the video signal is encoded in the encoder. The coding process compresses the video signal by adopting various mechanisms to reduce the video file size. The smaller file size allows compressed video files to be sent to the user while reducing the associated bandwidth overhead. The decoder then decodes the compressed video file and reconstructs the original video signal for display to the end user. The decoding process generally reflects the coding process so that the decoder can reconstruct the video signal consistently.

In step 101, a video signal is input into the encoder. For example, the video signal may be an uncompressed video file stored in memory. As another example, the video file may be captured by a video capture device such as a video camera and encoded to support live streaming of the video. The video file may contain both audio and video components. The video component contains a series of image frames that give a visual impression of movement when viewed in sequence. The frame includes light, referred to herein as a luma component (or luma sample), and pixels, which are expressed in terms of color, referred to as a chroma component (or color sample). In some examples, the frame may also contain depth values to support 3D viewing.

In step 103, the video is divided into blocks. Separation involves subdividing the pixels in each frame into square and / or rectangular blocks for compression. For example, in High Efficiency Video Coding (HEVC) (also known as High Efficiency Video Coding and MPEG-H Part 2), frames can first be split into coding tree units (CTUs), which have a default size (eg, for example). , 64 pixels x 64 pixels) block. The CTU includes both luma and chroma samples. A coding tree may be employed to divide the CTU into blocks and then recursively subdivide the blocks until a configuration that supports further encoding is achieved. For example, the luma component of the frame may be subdivided until the individual blocks contain relatively homogeneous light values. In addition, the chroma component of the frame may be subdivided until the individual blocks contain relatively homogeneous color values. Therefore, the partitioning mechanism varies depending on the content of the video frame.

In step 105, various compression mechanisms are employed to compress the image blocks partitioned in step 103. For example, inter-prediction and / or intra-prediction may be adopted. Inter-prediction is designed to take advantage of the fact that objects in a common scene tend to appear in continuous frames. Therefore, the block indicating the object in the reference frame does not need to be repeatedly described in the adjacent frame. In particular, objects such as tables may stay in place across multiple frames. Therefore, the table is described once, and adjacent frames can refer back to the reference frame. A pattern matching mechanism may be employed to align objects across multiple frames. Further, moving objects may be depicted across multiple frames, for example due to object movement or camera movement. As a particular example, a video may show a car moving across a screen across multiple frames. Motion vectors can be employed to represent such movements. A motion vector is a two-dimensional vector that provides an offset from the coordinates of an object in a frame to the coordinates of an object in a reference frame. Thus, inter-prediction can encode an image block in the current frame as a set of motion vectors indicating an offset from the corresponding block in the reference frame.

Intra-prediction encodes blocks in a common frame. Intra prediction takes advantage of the fact that the luma and chroma components tend to cluster within the frame. For example, a green fragment within a piece of a tree tends to be placed adjacent to a similar green fragment. Intra prediction employs multiple directional prediction modes (eg 33 for HEVC), planar mode, and direct current (DC) mode. The directional mode indicates that the current block is similar / same as the sample of the adjacent block in the corresponding direction. Plane mode indicates that a set of blocks along a row / column (eg, a plane) can be interpolated based on adjacent blocks at the edges of the row. Planar mode actually shows a smooth transition of light / color across rows / columns by adopting a relatively constant gradient as the values change. DC mode is adopted for boundary smoothing and indicates that the block is similar / same to the mean value associated with the samples of all adjacent blocks associated with the angular orientation of the directional prediction mode. Therefore, the intra prediction block can represent the image block as various prediction mode values that indicate the relationship rather than the actual value. In addition, the inter-prediction block can represent the image block as a motion vector value rather than an actual value. In either case, the predictive block does not have to be an exact representation of the image block in some cases. Any difference is stored in the residual block. Transformations may be applied to the residual blocks to further compress the file.

In step 107, various filtering techniques may be applied. In HEVC, filters are applied according to the in-loop filtering method. The block-based prediction described above may result in the generation of block-like images in the decoder. In addition, block-based prediction schemes may encode the block and then reconstruct the coded block for later use as a reference block. The in-loop filtering method iteratively applies a noise suppression filter, a deblocking filter, an adaptive loop filter, and a sample adaptive offset (SAO) filter to the block / frame. These filters mitigate such blocking artifacts so that the encoded file can be reconstructed accurately. In addition, these filters mitigate the artifacts in the reconstructed reference block, so that the artifacts add additional artifacts in subsequent blocks that are encoded based on the reconstructed reference block. It is unlikely to be generated.

Once the video signal has been partitioned, compressed, and filtered, the resulting data is encoded in the bitstream in step 109. The bitstream contains the data described above, as well as any signaling data desired to support proper video signal reconstruction in the decoder. For example, such data may include partitioned data, predictive data, residual blocks, and various flags that provide coding instructions to the decoder. The bitstream may be stored in memory for transmission to the decoder on request. Bitstreams may also be broadcast and / or multicast to multiple decoders. Creating a bitstream is an iterative process. Therefore, steps 101, 103, 105, 107, and 109 may be performed continuously and / or simultaneously over many frames and blocks. The order shown in FIG. 1 is presented for clarity and ease of description and does not limit the video coding process to any particular order.

At step 111, the decoder receives the bitstream and initiates the decoding process. Specifically, the decoder employs an entropy decoding method to convert the bitstream into the corresponding syntax and video data. In step 111, the decoder employs syntax data from the bitstream to determine the division for the frame. The division should be consistent with the result of the block division in step 103. Entropy coding / decoding as adopted in step 111 is described below. The encoder makes many choices during the compression process, such as choosing a block partitioning scheme from several possible options, based on the spatial positioning of the values in the input image. Signaling the exact choices may employ a large number of bins. The bin used herein is a binary value treated as a variable (eg, a bit value that may change depending on the context). Entropy coding allows the encoder to discard any options that are not explicitly feasible for a particular case, leaving an acceptable set of options. Each acceptable option is then assigned a codeword. Codeword length is based on the number of options you can tolerate (for example, 1 bin for 2 options, 2 bins for 3-4 options, and so on). The encoder then encodes the codeword for the selected option. Codewords are as much desired to uniquely indicate choices from a small subset of acceptable options, as opposed to uniquely showing choices from a potentially large set of all possible options. Because of the size of, this method reduces the size of the codeword. The decoder then decodes the choices by determining an acceptable set of options in a manner similar to an encoder. By determining the acceptable set of options, the decoder can read the codeword and determine the choices made by the encoder.

At step 113, the decoder performs block decoding. In particular, the decoder employs an inverse transformation to generate a residual block. The decoder then employs a residual block and a corresponding predictive block to reconstruct the image block according to the division. The prediction block may include both an intra-prediction block and an inter-prediction block as generated in the encoder in step 105. The reconstructed image block is then placed in the frame of the reconstructed video signal according to the segmented data determined in step 111. The syntax for step 113 may also be signaled in the bitstream via entropy coding as described above.

In step 115, filtering is performed on the frames of the reconstructed video signal in the same manner as in step 107 in the encoder. For example, noise suppression filters, deblocking filters, adaptive loop filters, and SAO filters may be applied to the frame to remove blocking artifacts. Once the frames have been filtered, the video signal may be output to the display in step 117 for viewing by the end user.

FIG. 2 is a schematic diagram of an exemplary coding and decoding (codec) system 200 for video coding. In particular, the codec system 200 provides functionality to support the implementation of method 100. The codec system 200 is generalized to indicate the components adopted in both the encoder and the decoder. The codec system 200 receives and partitions the video signal as described with respect to steps 101 and 103 in the operating method 100, which results in the segmented video signal 201. The codec system 200 then compresses the partitioned video signal 201 into a coded bitstream when acting as an encoder as described for steps 105, 107, and 109 in Method 100. When acting as a decoder, the codec system 200 produces an output video signal from the bitstream, as described for steps 111, 113, 115, and 117 in method 100. The codec system 200 includes general-purpose coder control component 211, transformation scaling and quantization component 213, intra-picture estimation component 215, intra-picture prediction component 217, motion compensation component 219, motion estimation component 221, scaling and vice versa. Includes transformation component 229, filter control analysis component 227, in-loop filter component 225, decryption picture buffer component 223, and header formatting and context adaptive binary arithmetic coding (CABAC) component 231. .. Such components are combined as shown. In FIG. 2, the black line shows the movement of data to be encoded / decoded, and the dashed line shows the movement of control data that controls the operation of other components. All components of the codec system 200 may reside within the encoder. The decoder may include a subset of the components of the codec system 200. For example, the decoder may include an intra-picture prediction component 217, a motion compensation component 219, a scaling and inverse transformation component 229, an in-loop filter component 225, and a decoding picture buffer component 223. Next, these components will be described.

The segmented video signal 201 is a captured video sequence segmented into blocks of pixels by the coding tree. The coding tree employs various split modes to subdivide a block of pixels into smaller blocks of pixels. These blocks can then be further subdivided into smaller blocks. The block may be called a node on the coding tree. The larger parent node is split into smaller child nodes. The number of times a node is subdivided is called the node / coding tree depth. The divided blocks may be contained in a coding unit (CU) in some cases. For example, the CU can be a subpart of the CTU, including a luma block, a red differential chroma (Cr) block, and a blue differential chroma (Cb) block along with the corresponding syntax instructions for the CU. The split mode is a binary tree (BT) that is used to divide a node into two, three, or four child nodes, each of which has a shape that changes depending on the split mode adopted. , Ternary tree (TT: triple tree), and quadtree (QT: quad tree) may be included. The segmented video signal 201 is assigned to the general purpose coder control component 211, the transformation scaling and quantization component 213, the intra-picture estimation component 215, the filter control analysis component 227, and the motion estimation component 221 for compression. Transferred.

The general-purpose coder control component 211 is configured to make decisions related to coding the image of the video sequence into the bitstream according to application constraints. For example, general-purpose coder control component 211 manages bitrate / bitstream size vs. reconstruction quality optimization. Such decisions may be made based on storage space / bandwidth availability and image resolution requirements. General-purpose coder control component 211 also manages buffer utilization in the light of transmit speed to mitigate buffer underrun and overrun problems. To manage these issues, general-purpose coder control component 211 manages partitioning, prediction, and filtering by other components. For example, general purpose coder control component 211 may dynamically increase compression complexity to increase resolution and bandwidth usage, or compress compression complexity to reduce resolution and bandwidth usage. May be reduced. Therefore, the general purpose coder control component 211 controls the other components of the codec system 200 in order to balance the video signal reconstruction quality with the bit rate problem. The general-purpose coder control component 211 creates control data that controls the operation of other components. The control data is also transferred to header formatting and CABAC component 231 and encoded in the bitstream to signal the parameters for decoding in the decoder.

The segmented video signal 201 is also sent to motion estimation component 221 and motion compensation component 219 for inter-prediction. The frame or slice of the divided video signal 201 may be divided into a plurality of video blocks. Motion estimation component 221 and motion compensation component 219 perform interpredictive coding of received video blocks for one or more blocks in one or more reference frames to make time predictions. The codec system 200 may perform multiple coding paths, for example, to select the appropriate coding mode for each block of video data.

The motion estimation component 221 and the motion compensation component 219 may be highly integrated, but are shown separately for conceptual purposes. The motion estimation performed by the motion estimation component 221 is a process of generating a motion vector for estimating motion for a video block. The motion vector may indicate, for example, the displacement of the coded object with respect to the prediction block. A predictive block is a block that is considered to be closely aligned with the block to be coded in terms of pixel difference. The predictive block may be referred to as a reference block. Such pixel differences may be determined by sum of absolute difference (SAD), sum of square difference (SSD), or other difference metrics. HEVC employs several coded objects, including CTU, Coding Tree Block (CTB), and CU. For example, the CTU can be split into CTBs, which can then be split into CBs for inclusion in the CU. The CU can be encoded as a prediction unit (PU) containing the prediction data and / or a transform unit (TU) containing the transformed residual data for the CU. Motion estimation component 221 generates motion vectors, PUs, and TUs by using rate strain analysis as part of the rate strain optimization process. For example, the motion estimation component 221 may determine multiple reference blocks, multiple motion vectors, etc. for the current block / frame, selecting the reference block, motion vector, etc. with the best rate distortion characteristics. good. The best rate distortion characteristics balance both the quality of the video reconstruction (eg, the amount of data loss due to compression) and the coding efficiency (eg, the size of the final coding).

In some examples, the codec system 200 may calculate the value for the sub-integer pixel position of the reference picture stored in the decrypted picture buffer component 223. For example, the video codec system 200 may interpolate values at 1/4 pixel position, 1/8 pixel position, or other fractional pixel position of the reference picture. Therefore, the motion estimation component 221 may perform a motion search for a perfect pixel position and a fractional pixel position and output a motion vector with fractional pixel accuracy. The motion estimation component 221 calculates the motion vector for the PU of the video block in the intercoded slice by comparing the position of the PU with the position of the predicted block of the reference picture. The motion estimation component 221 outputs the motion to the header formatting and CABAC component 231 for encoding the calculated motion vector as motion data, and to the motion compensation component 219.

The motion compensation performed by the motion compensation component 219 may involve fetching or generating a predictive block based on the motion vector determined by the motion estimation component 221. Again, motion estimation component 221 and motion compensation component 219 may be functionally integrated in some examples. Upon receiving the motion vector for the PU of the current video block, the motion compensation component 219 may locate the predicted block to which the motion vector points. The residual video block is then formed by subtracting the pixel value of the predicted block from the pixel value of the current video block being coded to form the pixel difference value. In general, motion estimation component 221 performs motion estimation for the luma component, and motion compensation component 219 uses a motion vector calculated based on the luma component for both chroma and luma components. The prediction block and the residual block are transferred to the transformation scaling and quantization component 213.

The partitioned video signal 201 is also sent to the intra-picture estimation component 215 and the intra-picture prediction component 217. Similar to motion estimation component 221 and motion compensation component 219, intra-picture estimation component 215 and intra-picture prediction component 217 may be highly integrated, but are shown separately for conceptual purposes. The intra-picture estimation component 215 and the intra-picture prediction component 217 are currently in the frame as an alternative to the inter-prediction performed by the motion estimation component 221 and the motion compensation component 219 between frames as described above. Intra-predict the current block for the block. In particular, the intra-picture estimation component 215 determines the intra-prediction mode currently to be used to encode the block. In some examples, the intra-picture estimation component 215 selects the appropriate intra-prediction mode for currently encoding the block from a plurality of tested intra-prediction modes. The selected intra-prediction mode is then transferred to header formatting and CABAC component 231 for encoding.

For example, the intra-picture estimation component 215 uses rate strain analysis to calculate rate strain values for the various intra-prediction modes tested, and the intra-prediction with the best rate strain characteristics among the tested modes. Select a mode. Rate strain analysis generally produces the amount of strain (or error) between the coded block and the original uncoded block encoded to produce the coded block, as well as the coded block. Determine the bit rate used for (for example, the number of bits). Intra-picture estimation component 215 calculates the ratio from strain and rate for various coded blocks to determine which intra-prediction mode shows the best rate strain value for the block. In addition, the intra-picture estimation component 215 now uses the depth modeling mode (DMM) to code depth blocks in the depth map based on rate-distortion optimization (RDO). It may be configured.

When the intra-picture prediction component 217 is implemented on the encoder, it may generate a residual block from the prediction block based on the selected intra-prediction mode determined by the intra-picture estimation component 215, or on the decoder. When implemented in, the residual block may be read from the bitstream. The residual block contains the difference in value between the predicted block and the original block, represented as a matrix. The residual block is then transferred to the transformation scaling and quantization component 213. The intra-picture estimation component 215 and the intra-picture prediction component 217 may operate for both the luma component and the chroma component.

The transformation scaling and quantization component 213 is configured to further compress the residual block. Transform scaling and quantization component 213 applies transformations to the residual block, such as the Discrete Cosine Transform (DCT), Discrete Cosine Transform (DST), or conceptually similar transform, and has a residual transform coefficient value. Create a video block. Wavelet transforms, integer transforms, subband transforms, or other types of transforms may also be used. The conversion may convert the residual information from a pixel value area to a conversion area such as a frequency domain. The transformation scaling and quantization component 213 is also configured to scale the transformed residual information, for example, based on frequency. Such scaling involves applying a scale factor to the residual information, resulting in different frequency information being quantized at different particle sizes, which in turn leads to the final visual quality of the reconstructed video. May have an impact. The transformation scaling and quantization component 213 is also configured to quantize the transformation coefficients to further reduce the bit rate. The quantization process can reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting the quantization parameters. In some examples, the transformation scaling and quantization component 213 may then perform a scan of the matrix containing the quantization transformation coefficients. The quantization conversion factor is transferred to header formatting and CABAC component 231 and encoded in the bitstream.

Scaling and inverse transformation component 229 applies the inverse behavior of transformation scaling and quantization component 213 to support motion estimation. Scaling and inverse transformation component 229, for example, reverse scaling to reconstruct the residual block in the pixel area for later use as a reference block, which may be a predictive block for another current block. , Transformation, and / or apply quantization. The motion estimation component 221 and / or the motion compensation component 219 calculates the reference block by adding the residual block back to the corresponding predicted block for use in later block / frame motion estimation. It's okay. Filters are applied to the reconstructed reference blocks to mitigate the artifacts produced during scaling, quantization, and transformation. Such artifacts can in some cases cause inaccurate predictions (and can produce additional artifacts) when subsequent blocks are predicted.

The filter control analysis component 227 and the in-loop filter component 225 apply the filter to the residual block and / or the reconstructed image block. For example, to reconstruct the original image block, the converted residual block from the scaling and inverse transformation component 229 is the corresponding prediction block from the intra-picture prediction component 217 and / or the motion compensation component 219. May be combined with. The filter may then be applied to the reconstructed image block. In some examples, the filter may be applied to the residual block instead. Like the other components in Figure 2, the filter control analysis component 227 and the in-loop filter component 225 may be highly integrated and implemented together, but separately for conceptual purposes. Shown. Filters applied to reconstructed reference blocks are applied to specific spatial regions and contain multiple parameters for adjusting how such filters are applied. Filter control analysis component 227 analyzes the reconstructed reference block and sets the corresponding parameters to determine where such a filter should be applied. Such data is transferred to header formatting and CABAC component 231 as filter control data for encoding. In-loop filter component 225 applies such a filter based on the filter control data. Filters may include deblocking filters, noise suppression filters, SAO filters, and adaptive loop filters. Such filters may be applied in the spatial / pixel domain (eg, for reconstructed pixel blocks) or in the frequency domain, as is the case.

When acting as an encoder, the filtered reconstructed image blocks, residual blocks, and / or predictive blocks are decoded picture buffer components 223 for later use in motion estimation as described above. It is remembered in. When acting as a decoder, the decode picture buffer component 223 stores the reconstructed filtered block and transfers it towards the display as part of the output video signal. The decoded picture buffer component 223 may be any memory device capable of storing predictive blocks, residual blocks, and / or reconstructed image blocks.

Header formatting and CABAC component 231 receives data from various components of the codec system 200 and encodes such data into a coded bitstream for transmission to the decoder. In particular, header formatting and CABAC component 231 generate various headers for encoding control data such as general purpose control data and filter control data. In addition, all prediction data, including intra-prediction and motion data, as well as residual data in the form of quantization conversion coefficient data are encoded in the bitstream. The final bitstream contains all the information desired by the decoder to reconstruct the original separated video signal 201. Such information may also include an intra-prediction mode index table (also called a codeword mapping table), definitions of coding contexts for various blocks, display of the most probable intra-prediction mode, display of partitioning information, and the like. Such data may be encoded by adopting entropy coding. For example, the information includes context adaptive variable length coding (CAVLC), CABAC, syntax-based context-adaptive binary arithmetic coding (SBAC), and probability interval segmentation entropy. It may be encoded by (PIPE: probability interval partitioning entropy) coding, or by adopting another entropy coding technique. Following entropy coding, the coded bitstream may be sent to another device (eg, a video decoder) or archived for later transmission or retrieval.

FIG. 3 is a block diagram showing an exemplary video encoder 300. The video encoder 300 may be employed to perform the coding function of the codec system 200 and / or to perform steps 101, 103, 105, 107, and / or 109 of the operating method 100. The encoder 300 divides the input video signal and obtains the divided video signal 301 which is substantially similar to the divided video signal 201. The partitioned video signal 301 is then compressed by the components of the encoder 300 and encoded into a bitstream.

Specifically, the partitioned video signal 301 is transferred to the intra-picture prediction component 317 for intra-prediction. The intra-picture prediction component 317 may be substantially similar to the intra-picture estimation component 215 and the intra-picture prediction component 217. The partitioned video signal 301 is also transferred to motion compensation component 321 for interprediction based on the reference block in the decoded picture buffer component 323. The motion compensation component 321 may be substantially similar to the motion estimation component 221 and the motion compensation component 219. The prediction and residual blocks from the intra-picture prediction component 317 and the motion compensation component 321 are transferred to the conversion and quantization component 313 for the conversion and quantization of the residual block. The transformation and quantization component 313 may be substantially similar to the transformation scaling and quantization component 213. The transformed and quantized residual blocks as well as the corresponding predictive blocks (along with the associated control data) are transferred to the entropy coding component 331 for coding into the bitstream. Entropy coding component 331 may be substantially similar to header formatting and CABAC component 231.

The transformed and quantized residual block and / or the corresponding predictive block is also inversely transformed and inversely transformed from the transformed and quantized component 313 for reconstruction into a reference block for use by the motion compensation component 321. Transferred to the quantization component 329. The inverse transformation and quantization component 329 may be substantially similar to the scaling and inverse transformation component 229. The in-loop filter in the in-loop filter component 325 is also applied to the residual block and / or the reconstructed reference block as usual. The in-loop filter component 325 may be substantially similar to the in-loop filter component 227 and the in-loop filter component 225. The in-loop filter component 325 may include a plurality of filters as described for the in-loop filter component 225. The filtered block is then stored in the decoded picture buffer component 323 for use as a reference block by the motion compensation component 321. The decode picture buffer component 323 may be substantially similar to the decode picture buffer component 223.

FIG. 4 is a block diagram showing an exemplary video decoder 400. The video decoder 400 may be employed to perform the decoding function of the codec system 200 and / or to perform steps 111, 113, 115, and / or 117 of method 100. The decoder 400 receives, for example, a bitstream from the encoder 300 and produces a reconstructed output video signal based on the bitstream for display to the end user.

The bitstream is received by the entropy decoding component 433. Entropy decoding component 433 is configured to perform an entropy decoding scheme, such as CAVLC, CABAC, SBAC, PIPE coding, or other entropy coding techniques. For example, the entropy decoding component 433 may employ header information to provide context for interpreting additional data encoded as codewords in the bitstream. The information to be decoded includes any desired information for decoding the video signal, such as general purpose control data, filter control data, partitioning information, motion data, prediction data, and quantization conversion coefficients from residual blocks. .. The quantized transformation coefficients are transferred to the inverse transformation and quantization component 429 for reconstruction into the residual block. The inverse transformation and quantization component 429 may be similar to the inverse transformation and quantization component 329.

The reconstructed residual block and / or prediction block is transferred to the intra-picture prediction component 417 for reconstruction into an image block based on the intra-prediction operation. The intra-picture prediction component 417 may be similar to the intra-picture estimation component 215 and the intra-picture prediction component 217. Specifically, the intra-picture prediction component 417 employs a prediction mode to locate the reference block within the frame and applies a residual block to the result to reconstruct the intra-prediction image block. .. The reconstructed intra-predicted image block and / or the residual block and the corresponding inter-predicted data may be substantially similar to the decoded picture buffer component 223 and the in-loop filter component 225, respectively. Transferred to buffer component 423 via in-loop filter component 425. In-loop filter component 425 filters the reconstructed image blocks, residual blocks, and / or predictive blocks, and such information is stored in the decoded picture buffer component 423. The reconstructed image block from the decoded picture buffer component 423 is transferred to the motion compensation component 421 for interprediction. The motion compensation component 421 may be substantially similar to the motion estimation component 221 and / or the motion compensation component 219. Specifically, motion compensation component 421 employs a motion vector from a reference block to generate a predictive block and apply a residual block to the result to reconstruct the image block. The resulting reconstructed block may also be transferred to the decode picture buffer component 423 via the in-loop filter component 425. The decrypted picture buffer component 423 continues to store additional reconstructed image blocks, which may be reconstructed into frames via partitioning information. Such frames may also be placed in the sequence. The sequence is output towards the display as a reconstructed output video signal.

FIG. 5 is a schematic diagram showing an exemplary bitstream 500 containing a coded video sequence. For example, the bitstream 500 may be generated by the codec system 200 and / or the encoder 300 for decoding by the codec system 200 and / or the decoder 400. As another example, the bitstream 500 may be generated by the encoder in step 109 of method 100 for use by the decoder in step 111.

Bitstream 500 includes a sequence parameter set (SPS) 510, a plurality of picture parameter sets (PPS) 512, a tile group header 514, and image data 520. The SPS510 contains sequence data common to all pictures in the video sequence contained within the bitstream 500. Such data can include picture sizing, bit depth, coding tool parameters, bit rate constraints, and so on. The PPS512 contains parameters specific to one or more corresponding pictures. Therefore, each picture in the video sequence may refer to one PPS512. The PPS512 can show coding tools, quantization parameters, offsets, picture-specific coding tool parameters (eg, filter control), etc. that are available for the tiles in the corresponding picture. The tile group header 514 contains parameters specific to each tile group in the picture. Therefore, there may be one tile group header 514 for each tile group in the video sequence. The tile group header 514 may include tile group information, a picture order count (POC), a reference picture list, a predicted weight, a tile entry point, a deblocking parameter, and the like. Note that some systems refer to tile group header 514 as a slice header and use such information to support slices rather than tile groups.

Image data 520 includes video data encoded according to inter-prediction and / or intra-prediction, as well as corresponding transformation and quantized residual data. Such image data 520 is sorted according to the classification used to classify the image prior to encoding. For example, the image in the image data 520 is divided into tiles 523. Tile 523 is further subdivided into coding tree units (CTUs). The CTU is further subdivided into coding blocks based on the coding tree. The coding block can then be encoded / decoded according to the prediction mechanism. The image / picture can include one or more tiles 523.

Tile 523 is a segmented portion of the picture created by horizontal and vertical boundaries. The tile 523 may be rectangular and / or square. In particular, tile 523 includes four sides that are connected at right angles. The four sides contain two pairs of parallel sides. In addition, the sides of the parallel side pair are equal in length. Therefore, the tile 523 may be of any rectangular shape, where the square is a special case of a rectangle, where all four sides are of equal length. The picture may be placed in the rows and columns of tile 523. A tile row is a set of tiles 523 arranged horizontally adjacent to each other to create a continuous line from the left border to the right border of the picture (or vice versa). A tile column is a set of tiles 523 arranged vertically adjacent to each other to create a continuous line from the top border to the bottom border of the picture (or vice versa). Tile 523 may or may not allow predictions based on other tiles 523, as is the case. Each tile 523 may have a unique tile index within the picture. A tile index is a procedurally selected numeric identifier that can be used to distinguish one tile 523 from another. For example, the tile index may increase numerically in raster scan order. The raster scan order is left-to-right and top-to-bottom. Note that in some examples tile 523 may also be assigned a tile identifier (ID). The tile ID is an assigned identifier that can be used to distinguish one tile 523 from another tile 523. In some examples, the calculation may employ the tile ID instead of the tile index. Further, in some examples, the tile ID may be assigned to have the same value as the tile index. The tile index and / or tile ID may be signaled to indicate a tile group containing tile 523. For example, the tile index and / or tile ID may be employed to map the picture data associated with tile 523 to the appropriate location for display. A tile group is an associated set of tiles 523 that can be extracted and coded separately, for example, to support display of the area of interest and / or to support parallelism. The tile 523 inside the tile group can be coded without reference to the tile 523 outside the tile group. Each tile 523 may be assigned to a corresponding tile group, and thus the picture can contain multiple tile groups.

Figures 6A-6E are for creating an extractor track 610 to combine multiple resolution subpictures from different bitstreams into a single picture for use in a virtual reality (VR) application. An exemplary mechanism 600 is shown. Mechanism 600 may be employed to support exemplary use cases of Method 100. For example, mechanism 600 may be employed to generate a bitstream 500 for transmission from the codec system 200 and / or the encoder 300 to the codec system 200 and / or the decoder 400. As a specific example, the mechanism 600 can be employed for use with VR, OMAF, 360 degree video, and the like.

In VR, only a portion of the video is visible to the user. For example, VR video may be shot to include a sphere that surrounds the user. Users may employ a head-mounted display (HMD) to watch VR video. The user may point the HMD toward the target area. The target area is displayed to the user, and other video data is discarded. In this way, the user sees only the part of the VR video selected by the user at any moment. This technique mimics the user's perception and thus allows the user to experience the virtual environment in a way that mimics the real environment. One of the problems with this technique is that the entire VR video may be sent to the user, but only the current viewport of the video is actually used and the rest is discarded. To increase signaling efficiency for streaming applications, the user's current viewport may be transmitted at a higher first resolution and other viewports may be transmitted at a lower second resolution. In this way, viewports that may be discarded occupy less bandwidth than viewports that may be seen by the user. When the user selects a new viewport, lower resolution content may be shown until the decoder may require that different current viewports be transmitted at a higher first resolution. To support this feature, mechanism 600 may be employed to create the extractor track 610 as shown in FIG. 6E. The extractor track 610 is a track of image data that encapsulates a picture at multiple resolutions for use as described above.

Mechanism 600 encodes the same video content at a first resolution of 611 and a second resolution of 612, respectively, as shown in FIGS. 6A and 6B, respectively. As a specific example, the first resolution 611 may be a 5120 x 2560 luma sample and the second resolution 612 may be a 2560 x 1280 luma sample. The picture of the video may be divided into tile 601 at the first resolution 611 and tile 603 at the second resolution 612, respectively. In the illustrated example, tiles 601 and 603 are each divided into 4x2 grids. In addition, MCTS can be coded for the positions of tiles 601 and 603. The pictures at the first resolution 611 and the second resolution 612 each yield an MCTS sequence that represents the video over time at the corresponding resolution. Each coded MCTS sequence is stored as a sub-picture track or tile track. Mechanism 600 can then use the picture to create a segment that should support viewport adaptive MCTS selection. For example, each range of viewing orientation is considered, which causes different choices between high resolution MCTS and low resolution MCTS. In the illustrated example, four tiles 601 containing the MCTS at the first resolution 611 and four tiles 603 containing the MCTS at the second resolution 612 are obtained.

Mechanism 600 can then create an extractor track 610 for each possible viewport adaptive MCTS selection. Figures 6C and 6D show exemplary viewport-adaptive MCTS selections. Specifically, the set of tiles 605 and 607 selected is selected at the first resolution 611 and the second resolution 612, respectively. The tiles 605 and 607 selected are shown in gray shading. In the illustrated example, the selected tile 605 is the tile 601 at first resolution 611, which will be shown to the user, and the selected tile 607 may be discarded, but the user is new. Tile 603 at a second resolution of 612, which is retained to support display when selecting a different viewport. The selected tiles 605 and 607 are then combined into a single picture containing image data at both the first resolution 611 and the second resolution 612. Such pictures are combined to create an extractor track 610. FIG. 6E shows a single picture from the corresponding extractor track 610 for illustrative purposes. As shown, the pictures in the extractor track 610 include tiles 605 and 607 selected from the first resolution 611 and the second resolution 612. As mentioned above, FIGS. 6C-6E show a single viewport adaptive MCTS selection. An extractor track 610 should be created for each possible combination of tiles 605 and 607 selected to allow user selection of any viewport.

In the illustrated example, each selection of tile 603 that encapsulates the content from the second resolution 612 bitstream contains two slices. A RegionWisePackingBox may be included within the extractor track 610 to create a mapping between the packed picture and the projected picture in equirectangular projection (ERP) format. In the example presented, the bitstream resolved from the extractor track 610 has a resolution of 3200 x 2560. Therefore, a 4000 sample (4K) capable decoder may decode content from which the viewport is extracted from a coded bitstream with 5000 sample 5K (5120 x 2560) resolution.

As shown, the extractor track 610 contains two rows of high resolution tile 601 and four rows of low resolution tile 603. Therefore, the extractor track 610 contains two slices of high resolution content and four slices of low resolution content. Uniform tiling does not have to support such use cases. Uniform tiling is defined by a set of tile columns and a set of tile rows. The tile rows extend from the top of the picture to the bottom of the picture. Similarly, tile rows extend from the left of the picture to the right of the picture. Although such a structure can be easily defined, it is virtually incapable of supporting advanced use cases, such as the use cases described by Mechanism 600. In the illustrated example, different numbers of rows are employed in different sections of the extractor track 610. If uniform tiling is adopted, the tiles on the right side of the extractor track 610 should be rewritten to accept two slices each. This method is inefficient and computationally complex.

The present disclosure includes a flexible tiling scheme as described below that does not require the tiles to be rewritten to contain different numbers of slices. The flexible tiling scheme allows tile 601 to contain content at a first resolution of 611. The flexible tiling scheme also allows the tile 601 to be subdivided into smaller tiles, each of which can be directly mapped to the tile 603 at a second resolution 612. This direct mapping is more efficient because such techniques do not require tiles to be rewritten / readdressed when different resolutions are combined as described above.

FIG. 7 illustrates an exemplary video conference application 700 that stitches together multiple resolution pictures from different bitstreams into a single picture for display. Application 700 may be employed to support exemplary use cases for Method 100. For example, application 700 may be employed in codec system 200 and / or decoder 400 to display video content from bitstream 500 from codec system 200 and / or encoder 300. The video conference application 700 displays the video sequence to the user. The video sequence includes pictures showing the talking participant 701 and other participants 703. The speaking participant 701 is displayed at a higher / larger first resolution, and the other participants 703 are displayed at a lower / smaller second resolution. To code such a picture, the picture should include a part with a single row and a part with three rows. To support such scenarios with uniform tiling, the picture is divided into left tile and right tile. The tile on the right is then rewritten / readdressed to contain three rows. Such readdressing presents both compression and performance disadvantages. The flexible tiling scheme described below allows a single tile to be divided into smaller tiles and mapped to tiles in the subpicture bitstream associated with other participants 703. In this way, the talking participant 701 can be mapped directly to the first level tile, and the other participants 703 can be from the first tile without such rewriting / readdressing. It can be mapped to a split second level tile.

8A-8D are schematics illustrating an exemplary flexible video tiling scheme 800 capable of supporting multiple tiles with different resolutions in the same picture. Flexible video tiling method 800 may be adopted to support the more efficient coding mechanism 600 and application 700. Therefore, the flexible video tiling method 800 can be adopted as part of the method 100. In addition, the flexible video tiling scheme 800 may be employed by the codec system 200, encoder 300, and / or decoder 400. The result of the flexible video tiling method 800 may be stored in the bitstream 500 for transmission between the encoder and the decoder.

As shown in FIG. 8A, pictures (eg, frames, images, etc.) can be divided into first level tiles 801 also known as level 1 tiles. As shown in FIG. 8B, the first level tile 801 may be selectively partitioned to create a second level tile 803, also known as a level 2 tile. The first level tile 801 and the second level tile 803 can then be employed to create a picture with subpictures coded at multiple resolutions. The first level tile 801 is a tile generated by completely dividing a picture into a set of columns and a set of rows. The second level tile 803 is a tile generated by partitioning the first level tile 801.

As described above, in various scenarios, for example in VR and / or teleconferencing, video can be coded in multiple resolutions. The video can also be coded by adopting slices at each resolution. Slices with lower resolution are smaller than slices with higher resolution. To create a picture with multiple resolutions, the picture can be divided into tiles 801 of the first level. Slices from the highest resolution can be included directly within tile 801 of the first level. Further, the first level tile 801 can be divided into second level tiles 803, which are smaller than the first level tile 801. Therefore, the smaller second level tile 803 can directly accept lower resolution slices. In this way, slices from each resolution are, for example, tile index related, without the need for dynamically readdressing tiles with different resolutions in order to use a consistent addressing scheme. Can be compressed into a single picture via. The first level tile 801 and the second level tile 803 may be implemented as MCTS and therefore may accept motion constrained image data at different resolutions.

The present disclosure includes many aspects. As a specific example, the first level tile 801 is split into the second level tile 803. The second level tile 803 may then be constrained to each contain a single rectangular slice of picture data (eg, at a lower resolution). Rectangular slices are slices that are constrained to maintain a rectangular shape and are therefore coded based on horizontal and vertical picture boundaries. Therefore, rectangular slices are not coded based on raster scan groups (which may contain a CTU in the left-to-right line and the top-to-bottom line and may not maintain a rectangular shape). A slice is a spatially separate area of a picture / frame that is encoded separately from any other area within the same frame / picture. In another example, the first level tile 801 can be divided into two or more complete second level tiles 803. In such cases, the first level tile 801 may not include the partial second level tile 803. In another example, the configuration of the first level tile 801 and the second level tile 803 is in a set of parameters in the bitstream, such as the PPS associated with the picture segmented to create the tile. Can be signaled with. In one example, a split display, such as a flag, could be coded in the parameter set for each first level tile 801. The display shows which first level tile 801 is further subdivided into second level tile 803. In another example, the configuration of the second level tile 803 can be signaled as the number of second level tile columns and the number of second level tile rows.

In another example, the first level tile 801 and the second level tile 803 can be assigned to a tile group. Such a tile group can be constrained so that all tiles in the corresponding tile group are constrained to cover a rectangular area of the picture (for example, as opposed to a raster scan). For example, some systems may add tiles to a tile group in raster scan order. This goes to adding the initial tiles in the current row, adding each tile in the row until the left picture border of the current row is reached, the right border of the next row. Includes going to, and adding each tile in the next row until the last tile is reached. This technique can result in a non-rectangular shape that extends beyond the picture. Such shapes may not be useful for creating pictures with multiple resolutions as described herein. Alternatively, in this example, any first level tile 801 and / or second level tile 803 may be added to the tile group (eg, in any order), but the resulting tile. Tile groups may be constrained so that the group must be rectangular or square (for example, including four sides connected at right angles). This constraint may ensure that the second level tile 803, which is separated from the single first level tile 801, is not placed in a different tile group.

In another example, when the tile width of the first level is less than twice the minimum width threshold and the tile height of the first level is less than twice the minimum height threshold, the first Data that explicitly indicates the number of second-level tile columns and the number of second-level tile rows can be excluded from the bitstream. The reason is that the first level tiles 801 that satisfy such conditions do not have to be divided into two or more columns or one row, respectively, so such information can be inferred by the decoder. Because. In another example, a split view showing which first level tile 801 is divided into second level tiles 803 is excluded from the bitstream for some first level tiles 801. sell. For example, when the first level tile 801 has a first level tile width less than the minimum width threshold and the first level tile height is less than the minimum height threshold. , Such data can be excluded. The reason is that the first level tile 801 that meets such conditions is too small to be split into the second level tile 803, so such information can be inferred by the decoder. be.

In another aspect, the flexible video tiling scheme 800 may employ tile group 805 as shown in FIG. 8C. Tile group 805 is shown to be separated by a thick line. The picture is divided into tiles 801 of the first level. A subset of the first level tile 801 is divided into the second level tile 803. The first level tile 801 and the second level tile 803 can then be assigned to tile group 805. Tile group 805 is a related set of tiles that can be extracted and coded separately, for example, to support display of the area of interest and / or to support parallelism. The tile group 805 may be generated as a raster scan tile group or a rectangular tile group. Raster scan tile groups include tiles assigned in raster scan order, from left to right and from top to bottom. For example, the raster scan order may be from the first tile to the right picture border, then from the left picture border to the right picture border in the next row until the last tile is reached. .. In contrast, a rectangular tile group contains a rectangular subpart of the picture. Tile group 805 is a rectangular tile group, but raster scan tile groups may also be used in some examples.

In some embodiments, the first level tile 801 and the second level tile 803 are such that each tile group 805 is one of several first level tiles 801 or second level tile 803. It can be assigned to tile group 805 to contain one or more consecutive sequences. The sequence of second level tiles 803 used herein is a group of second level tiles 803 divided from a single first level tile 801 and thus a contiguous tile index. Have. This technique allows all second level tiles 803 created from a single first level tile 801 to be assigned to the same tile group 805. In the example shown in Figure 8C, one tile group 805 each contains four first level tiles 801 and a second level tile 803 split from a single first level tile 801. Includes 4 other tile groups 805. However, many combinations of tiles and tile groups 805 may be used, depending on the type of video to be encoded and decoded.

In another aspect, the flexible video tiling scheme 800 may employ a scan sequence 807 as shown in FIG. 8D. Scanning sequence 807 is, by way of example, the sequence in which the tiles are encoded in the encoder and / or decoded in the decoder or the virtual reference decoder (in the encoder). Within the illustrated scan order 807, the first level tile 801 is coded in the raster scan order. When one of the second level tiles 803 is encountered, the raster scan order coding of the first level tiles 801 is interrupted. All consecutive second level tiles 803 are then coded in raster scan order before continuing raster scan order coding of first level tile 801. This process continues until all tiles have been coded. In the illustrated example, the second level tile 803, labeled tile 1, is first encountered. Therefore, the raster scan order coding of tile 801 of the first level is interrupted and consecutive second level tiles 803 labeled tiles 1 and 2 are coded. Once all consecutive second level tiles 803 have been coded, the raster scan order coding of the first level tiles 801 continues. Therefore, the first level tile 801 labeled tiles 3 and 4 is then coded. Encounter a second level tile 803 labeled 5. Consecutive second level tiles 803 are then coded. Therefore, tiles labeled 5-8 are coded. Once all consecutive second level tiles 803 have been coded, the raster scan order coding of the first level tiles 801 continues again. This results in coding the first level tile 801 labeled tiles 9 and 10. Encounter a second level tile 803 labeled 11. Consecutive second level tiles 803 are then coded. Therefore, tiles labeled 11 and 12 are coded. In more formal terms, the first level tile 801 is coded in a raster scan order with respect to the boundaries of the picture and / or tile group. In addition, all second level tiles 803 (eg, a sequence of consecutive second level tiles 803) separated from the current first level tile 801 will be followed by second level tiles 803. Coded before encoding any second level tile 803 segmented from. In addition, all second level tiles 803 separated from the current first level tile 801 are encoded in raster scan order with respect to the boundaries of the current first level tile 801.

As described above, the flexible video tiling scheme 800 supports merging subpictures from different bitstreams into pictures containing multiple resolutions. The following describes various embodiments that support such a function. In general, the present disclosure describes a method for signaling and coding tiles in video coding that separates pictures in a more flexible way than the tiling method in HEVC. More specifically, the present disclosure describes some tiling schemes, where tile columns do not have to extend uniformly from top to bottom of the coded picture, as well as tile rows of the coded picture. It does not have to extend uniformly from left to right.

For example, based on the HEVC tiling technique, some tiles should be further subdivided into multiple tile rows to support the features described in FIGS. 6A-6E and 7. In addition, the tiles should be further subdivided into tile columns, depending on how the tiles are arranged. For example, in FIG. 7, in some cases Participants 2-4 may be placed under Participant 1, which can be supported by dividing the tile into columns. To satisfy these scenarios, the first level tiles may be divided into tile rows and columns of second level tiles as described below.

For example, the tile structure can be relaxed as follows: Tiles in the same picture do not need to be a certain number of tile rows. Moreover, the tiles in the same picture do not need to be a certain number of tile columns. The following steps may be used for flexible tile signaling. The first level tile structure may be defined by tile columns and rows as defined in HEVC. The tile columns and rows may or may not be uniform in size. Each of these tiles may be referred to as a first level tile. Flags may be signaled to specify whether each first level tile is further divided into one or more tile columns and one or more tile rows. If the first level tiles are further subdivided, the tile columns and rows may be either uniform in size or non-uniform in size. The new tiles that result from the splitting of the first level tiles are called the second level tiles. Flexible tile structures may be limited to second level tiles only, and therefore, in some examples, no further division of any second level tiles is allowed. In another example, further division of the second level tile may be applied to create subsequent level tiles in the same way as creating a second level tile from a first level tile. ..

In the above example, the picture may include zero or more first level tiles that are not further divided, and zero or more second level tiles. The first level tiles that are further subdivided can only exist conceptually and do not have to be counted in the total number of tiles in the picture. An exemplary scan order is defined as follows. For simplicity, a tile group may be constrained to contain either some complete first level tiles, or a complete subset of first level tiles. The first level tiles may be ordered according to the tile raster scan of the picture. If the first level tiles that are further subdivided are referenced, the set of second level tiles obtained from such subdivisions may be referenced collectively. The second level tile of any current first level tile refers to any second level tile of subsequent first level tiles that follow the current first level tile. May be referenced before. The second level tiles of the current first level tile are referenced in raster scan order within the current first level tile. The CTUs in any current tile may be referenced in the CTU raster scan order within the current tile.

For simplicity, when a first level tile is split into two or more second level tiles, the split may always use uniform tile columns and uniform tile rows of uniform size. In such an example, it is not necessary to signal a flag that specifies whether the level 2 tile column and level 2 tile row are uniform. Moreover, the syntax element does not have to specify the tile row height and tile row width. In some examples, first level tiles may be constrained to always use a uniform size for tile columns and rows. In such an example, it is not necessary to signal a flag that specifies whether the level 1 tile column and level 1 row are uniform. Moreover, the syntax element does not have to specify the tile row height and tile row width. In another example, the first level tiles and the second level tiles may be constrained to always use tile columns and rows of tiles of uniform size. In such an example, it is not necessary to signal a flag that specifies whether the level 1 and level 2 tile columns and rows are uniform. Moreover, the syntax element does not have to specify the tile row height and tile row width.

Derivation of tile location, size, index, and scan order of flexible tiles defined by this technique is described below. In some examples, when such a flexible tile structure is used, the tile group is complete with only a few complete first level tiles, or one single first level tile. It may only contain a contiguous sequence of second level tiles. Further, when such a flexible tile structure is used, the tile group may be constrained to include one or more complete first level tiles. In this example, when a tile group contains second level tiles, all second level tiles resulting from the same first level tile split should be included in the tile group. When such a flexible tile structure is used, it may be further constrained that the tile group contains one or more tiles and together all tiles belong to a tile group that covers the rectangular area of the picture. In another aspect, when such a flexible tile structure is used, the tile group contains one or more first level tiles, together all tiles cover the rectangular area of the picture. Belongs to.

In one example, the signaling of flexible tiles could be: The minimum tile width and minimum tile height are defined values. The first level tile structure can be defined by tile columns and tile rows. The tile columns and rows may or may not be uniform in size. Each of these tiles may be referred to as a first level tile. Flags may be signaled to specify whether any of the first level tiles may be further subdivided. This flag does not exist when the width of each first level tile is less than or equal to twice the minimum tile width and the height of each first level tile is less than or equal to twice the minimum tile height. It's okay. When not present, the value of the flag is presumed to be equal to 0.

In one example, the following applies to each first level tile: Flags can be signaled to specify whether first level tiles are further divided into one or more tile columns and one or more tile rows. The presence of the flag can be constrained as follows: If the tile width of the first level is greater than the minimum tile width, or if the tile height of the first level is greater than the minimum tile height, the flag is present / signaled. Otherwise, the flag does not exist and the value of the flag is presumed to be equal to 0, which indicates that the tiles at the first level will not be split any further.

If the first level tile is further divided, the number of tile columns and the number of tile rows for this division may be further signaled. The tile columns and rows may be either uniform in size or non-uniform in size. The tiles obtained from the division of the first level tiles are called the second level tiles. The existence of the number of tile columns and the number of tile rows can be constrained as follows: When the tile width of the first level is less than twice the minimum tile width, the number of tile columns need not be signaled and the number of tile column values can be inferred to be equal to one. The signaling may employ the _minus1 syntax element so that the signaled syntax element value may be 0 and the number of tile columns is the syntax element value + 1. This technique may further compress the signaling data. When the tile height of the first level is less than twice the minimum tile height, the number of tile rows need not be signaled and the value of the number of tile rows can be inferred to be equal to zero. The signaled syntax element value can be 0 and the number of tile rows can be the syntax element value + 1 to further compress the signaling data. The tiles obtained from the division of the first level tiles can be referred to as the second level tiles. Flexible tile structures may be limited to second level tiles only so that further division of any second level tile is not allowed. In another example, further division of the second level tile may be applied in a similar manner to dividing the first level tile into the second level tile.

In one example, the signaling of the flexible tile structure could be: When a picture contains more than one tile, signals such as flags can be adopted in a parameter set that is directly or indirectly referenced by the corresponding tile group. The flag can specify whether the corresponding tile structure is a uniform tile structure or a non-uniform tile structure (eg, a flexible tile structure as described herein). The flag may be called uniform_tile_structure_flag. When uniform_tile_structure_flag is equal to 1, HEVC style uniform tile structure signaling is adopted, for example by signaling num_tile_columns_minus1 and num_tile_rows_minus1 to indicate a single level of uniform tiles. When uniform_tile_structure_flag is equal to 0, the following information may also be signaled. The number of tiles in a picture can be signaled by the syntax element num_tiles_minus2, which indicates that the number of tiles in the picture (NumTilesInPic) is equal to num_tiles_minus2 + 2. This can result in bit savings during signaling, as by default pictures may be considered tiles. For each tile, except for the last one, the address of the tile's first coding block (for example, CTU) and the last coding block is signaled. The address of the coding block can be the index of the block in the picture (eg, the index of the CTU in the picture). The syntax elements for such coding blocks may be tile_first_block_address [i] and tile_last_block_address [i]. These syntax elements may be coded as ue (v) or u (v). When the syntax element is coded as u (v), the number of bits used to represent each of the syntax elements is ceil (log2 (maximum number of coding blocks in the picture)). The addresses of the first and last coding blocks of the last tile do not have to be signaled, even if they are instead derived based on the picture size in the room sample, and in the set of all other tiles in the picture. good.

In one example, for each tile, the address of the first coding block of the tile, as well as the width and height of the tile, is signaled instead of signaling the addresses of the first and last coding blocks of the tile, except for the last one. May be done. In another example, for each tile, except for the last one, instead of signaling the addresses of the first and last coding blocks of the tile, the upper left point of the tile with respect to the original picture (for example, the upper left of the picture). ) Offset, as well as tile width and height may be signaled. In yet another example, for each tile, except for the last one, the following information may be signaled instead of signaling the addresses of the first and last coding blocks of the tile. The width and height of the tile may be signaled. Also, the location of each tile does not have to be signaled. Instead, a flag may be signaled to specify whether the tile should be placed just to the right of or just below the previous tile. This flag may not be present if the tile can only be to the right of or below the previous tile. The upper left offset of the first tile may be set to always be the origin / upper left of the picture (eg x = 0 and y = 0).

For signaling efficiency, a unique set of tile sizes (eg, width and height) may be signaled. This list of unique tile sizes may be referenced by the index from a loop containing signaling for each tile size. In some examples, tile locations and tile sizes, such as those derived from signaled tile structures, must constrain the partition to ensure that there are no gaps or overlaps between any tiles. Must be.

The following constraints may also apply. The tile shape may be required to be rectangular (eg, not a raster scan shape). The units of tiles in the picture must cover the picture without any gaps and overlaps between the tiles. When decoding is done using only one core, the left adjacent coding block is decoded before the current coding block for coding the current coding block (eg CTU) that is not on the left edge of the picture. There must be. When decoding is done using only one core, the adjacent coding block above is decoded before the current coding block for coding the current coding block (eg CTU) that is not on the top edge of the picture. There must be. When two tiles have tile indexes adjacent to each other (eg idx3 and idx4), one of the following is true: When the two tiles share a vertical edge and / or the first tile has an upper left location in (Xa, Ya) with size (Wa and Ha representing its width and height), and the first. When the second tile has an upper left location in (Xb, Yb), then Yb = Ya + Ha.

The following constraints may also apply. When a tile has two or more left adjacent tiles, the height of the tile must be equal to the sum of the heights of all its left adjacent tiles. When a tile has two or more right adjacent tiles, the height of the tile must be equal to the sum of the heights of all its left adjacent tiles. When a tile has two or more adjacent tiles above it, the width of the tile must be equal to the sum of the widths of all adjacent tiles above it. When a tile has two or more adjacent tiles below it, the width of the tile must be equal to the sum of the widths of all adjacent tiles below it. In addition, tile ID signaling, including the mapping between tile index and tile ID, may be based on the number of tiles in the picture. Therefore, the mapping may be based on the number of tiles in the picture rather than on the tile columns and tile rows. For example, loop so that each tile index is assigned a tile ID (for example, the first index is 0 and the last index is the number of tiles in the picture-1 from the first index to the last index). May be adopted.

The following are specific exemplary embodiments of the above embodiments. The CTB raster and tile scanning process may be as follows. The list ColWidth [i] for i from 0 to num_level1_tile_columns_minus1 containing the double-ended values, which specifies the width of the first level tile column of the first i in CTB units, can be derived as follows.
if (uniform_level1_tile_spacing_flag)
for (i = 0; i <= num_level1_tile_columns_minus1; i ++)
ColWidth [i] = ((i + 1) * PicWidthInCtbsY) / (num_level1_tile_columns_minus1 + 1)-(i * PicWidthInCtbsY) / (num_level1_tile_columns_minus1 + 1)

else {
ColWidth [num_level1_tile_columns_minus1] = PicWidthInCtbsY (6-1)
for (i = 0; i <num_level1_tile_columns_minus1; i ++) {
ColWidth [i] = tile_level1_column_width_minus1 [i] +1
ColWidth [num_tile_level1_columns_minus1]-= ColWidth [i]
}
}

The list RowHeight [j] for j from 0 to num_level1_tile_rows_minus1 containing the double-ended values, which specifies the height of the jth tile row in CTB units, can be derived as follows.
if (uniform_level1_tile_spacing_flag)
for (j = 0; j <= num_level1_tile_rows_minus1; j ++)
RowHeight [j] = ((j + 1) * PicHeightInCtbsY) / (num_level1_tile_rows_minus1 + 1)-(j * PicHeightInCtbsY) / (num_level1_tile_rows_minus1 + 1)

else {
RowHeight [num_level1_tile_rows_minus1] = PicHeightInCtbsY (6-2)
for (j = 0; j <num_level1_tile_rows_minus1; j ++) {
RowHeight [j] = tile_level1_row_height_minus1 [j] +1
RowHeight [num_level1_tile_rows_minus1]-= RowHeight [j]
}
}

The list colBd [i] for i from 0 to num_level1_tile_columns_minus1 + 1, which specifies the location of the boundary of the i-th tile column in CTB units, can be derived as follows.
for (colBd [0] = 0, i = 0; i <= num_level1_tile_columns_minus1; i ++)
colBd [i + 1] = colBd [i] + ColWidth [i] (6-3)

The list rowBd [j] for j from 0 to num_level1_tile_rows_minus1 + 1, which specifies the location of the jth tile row boundary in CTB units, can be derived as follows:
for (rowBd [0] = 0, j = 0; j <= num_level1_tile_rows_minus1; j ++)
rowBd [j + 1] = rowBd [j] + RowHeight [j] (6-4)

The variable NumTilesInPic, which refers to the PPS to specify the number of tiles in the picture, as well as the location of the i-th tile column boundary in CTB units and the location of the i-th tile row boundary in CTB units. List TileColBd [i], TileRowBd [i] for i from 0 to NumTilesInPic-1, including both-end values, specifying the width of the tile column in CTB units and the height of the i-th tile column in CTB units. , TileWidth [i], and TileHeight [i] can be derived as follows.
for (tileIdx = 0, i = 0; i <NumLevel1Tiles; i ++) {
tileX = i% (num_level1_tile_columns_minus1 + 1)
tileY = i / (num_level1_tile_columns_minus1 + 1)
if (! Level2_tile_split_flag [i]) {(6-5)
TileColBd [tileIdx] = colBd [tileX]
TileRowBd [tileIdx] = rowBd [tileY]
TileWidth [tileIdx] = ColWidth [tileX]
TileHeight [tileIdx] = RowHeight [tileY]
tileIdx ++
} else {
for (k = 0; k <= num_level2_tile_columns_minus1 [i]; k ++)
colWidth2 [k] = ((k + 1) * ColWidth [tileX]) / (num_level2_tile_columns_minus1 [i] +1)-(k * ColWidth [tileX]) / (num_level2_tile_columns_minus1 [i] +1)

for (k = 0; k <= num_level2_tile_rows_minus1 [i]; k ++)
rowHeight2 [k] = ((k + 1) * RowHeight [tileY]) / (num_level2_tile_rows_minus1 [i] +1)-(k * RowHeight [tileY]) / (num_level2_tile_rows_minus1 [i] +1)

for (colBd2 [0] = 0, k = 0; k <= num_level2_tile_columns_minus1 [i]; k ++)
colBd2 [k + 1] = colBd2 [k] + colWidth2 [k]
for (rowBd2 [0] = 0, k = 0; k <= num_level2_tile_rows_minus1 [i]; k ++)
rowBd2 [k + 1] = rowBd2 [k] + rowHeight2 [k]
numSplitTiles = (num_level2_tile_columns_minus1 [i] +1) * (num_level2_tile_rows_minus1 [i] +1)

for (k = 0; k <numSplitTiles; k ++) {
tileX2 = k% (num_level2_tile_columns_minus1 [i] +1)
tileY2 = k / (num_level2_tile_columns_minus1 [i] +1)
TileColBd [tileIdx] = colBd [tileX] + colBd2 [tileX2]
TileRowBd [tileIdx] = rowBd [tileY] + rowBd2 [tileY2]
TileWidth [tileIdx] = colWidth2 [tileX2]
TileHeight [tileIdx] = rowHeight2 [tileY2]
tileIdx ++
}
}
}
NumTilesInPic = tileIdx

The list CtbAddrRsToTs [ctbAddrRs] for ctbAddrRs from 0 to PicSizeInCtbsY-1, including the double-ended values, that specifies the conversion of the CTB address in the CTB raster scan of the picture to the CTB address in the tile scan can be derived as follows.
for (ctbAddrRs = 0; ctbAddrRs <PicSizeInCtbsY; ctbAddrRs ++) {
tbX = ctbAddrRs% PicWidthInCtbsY
tbY = ctbAddrRs / PicWidthInCtbsY tileFound = FALSE
for (tileIdx = NumTilesInPic-1, i = 0; i <NumTilesInPic-1 &&! TileFound; i ++) {(6-6)
tileFound = tbX <(TileColBd [i] + TileWidth [i]) && tbY <(TileRowBd [i] + TileHeight [i])
if (tileFound)
tileIdx = i
}
CtbAddrRsToTs [ctbAddrRs] = 0
for (i = 0; i <tileIdx; i ++)
CtbAddrRsToTs [ctbAddrRs] + = TileHeight [i] * TileWidth [i]
CtbAddrRsToTs [ctbAddrRs] + = (tbY-TileRowBd [tileIdx]) * TileWidth [tileIdx] + tbX-TileColBd [tileIdx]

}

The list CtbAddrTsToRs [ctbAddrTs] for ctbAddrTs from 0 to PicSizeInCtbsY-1, including the double-ended values, that specifies the conversion of the CTB address in the tile scan to the CTB address in the CTB raster scan of the picture can be derived as follows.
for (ctbAddrRs = 0; ctbAddrRs <PicSizeInCtbsY; ctbAddrRs ++) (6-7)
CtbAddrTsToRs [CtbAddrRsToTs [ctbAddrRs]] = ctbAddrRs

List TileId [ctbAddrTs] for ctbAddrTs from 0 to PicSizeInCtbsY-1, including the double-ended values, and CTB in the tile scan of the first CTB in the tile, specifying the conversion of the CTB address to the tile ID in the tile scan. The list FirstCtbAddrTs [tileIdx] for tileIdx from 0 to NumTilesInPic-1 with double-ended values that specify the translation to the address can be derived as follows.
for (i = 0, tileIdx = 0; i <= NumTilesInPic; i ++, tileIdx ++) {
for (y = TileRowBd [i]; y <TileRowBd [i + 1]; y ++) (6-8)
for (x = TileColBd [i]; x <TileColBd [i + 1]; x ++)
TileId [CtbAddrRsToTs [y * PicWidthInCtbsY + x]] = tileIdx
FirstCtbAddrTs [tileIdx] = CtbAddrRsToTs [TileColBd [tileIdx]] * PicWidthInCtbsY + TileColBd [tileIdx]]
}

The list NumCtusInTile [tileIdx] for tileIdx from 0 to NumTilesInPic-1 with double-ended values, which specifies the conversion from the tile index to the number of CTUs in the tile, can be derived as follows:
for (i = 0, tileIdx = 0; i <NumTilesInPic; i ++, tileIdx ++) (6-9)
NumCtusInTile [tileIdx] = TileColWidth [tileIdx] * TileRowHeight [tileIdx]

An exemplary picture parameter set RBSP syntax is as follows.

An exemplary picture parameter set RBSP semantic is as follows. num_level1_tile_columns_minus1 + 1 specifies the number of level 1 tile columns that divide the picture. num_level1_tile_columns_minus1 must be in the range 0 to PicWidthInCtbsY-1 including the values at both ends. When not present, the value of num_level1_tile_columns_minus1 is presumed to be equal to 0. num_level1_tile_rows_minus1 + 1 specifies the number of level 1 tile rows that separate the pictures. num_level1_tile_rows_minus1 must be in the range 0 to PicHeightInCtbsY-1 including the values at both ends. When not present, the value of num_level1_tile_rows_minus1 is presumed to be equal to 0. The variable NumLevel1Tiles is set equal to (num_level1_tile_columns_minus1 + 1) * (num_level1_tile_rows_minus1 + 1). NumTilesInPic must be greater than 1 when single_tile_in_pic_flag is equal to 0. uniform_level1_tile_spacing_flag is set equal to 1 to specify that level 1 tile column boundaries and similarly level 1 tile row boundaries are uniformly distributed across the picture. uniform_level1_tile_spacing_flag is explicitly signaled using the syntax elements level1_tile_column_width_minus1 [i] and level1_tile_row_height_minus1 [i], although level 1 tile column boundaries and similarly level 1 tile row boundaries are not uniformly distributed across the picture. Equal to 0 to specify that. When not present, the value of uniform_level1_tile_spacing_flag is presumed to be equal to 1.

level1_tile_column_width_minus1 [i] + 1 specifies the width of the i-th level 1 tile column in CTB units. level1_tile_row_height_minus1 [i] + 1 specifies the height of one row of the i-th tile level in CTB units. level2_tile_present_flag specifies that one or more Level 1 tiles will be split into more Level 2 tiles. level2_tile_split_flag [i] specifies that the i-th level 1 tile is split into two or more tiles. num_level2_tile_columns_minus1 [i] + 1 specifies the number of tile columns that divide the i-th tile. num_level2_tile_columns_minus1 [i] must be in the range 0-ColWidth [i] including both ends values. When not present, the value of num_level2_tile_columns_minus1 [i] is presumed to be equal to 0. num_level2_tile_rows_minus1 [i] + 1 specifies the number of tile rows that divide the i-th tile. num_level2_tile_rows_minus1 [i] must be in the range of 0 to RowHeight [i] including the double-ended value. When not present, the value of num_level2_tile_rows_minus1 [i] is presumed to be equal to 0.

When level2_tile_split_flag [i] is equal to 1, the (num_level2_tile_columns_minus1 [i] +1) * (num_level2_tile_rows_minus1 [i] +1) value must be greater than 1. The picture may contain 0 or more level 1 tiles with level2_tile_split_flag [i] equal to 0, and 0 or more level 2 tiles. Level 1 tiles with level2_tile_split_flag [i] equal to 0 do not have to be counted in the total number of tiles in the picture. When such tiles are referenced, the set of Level 2 tiles obtained from such splits may be referenced collectively.

By calling the CTB raster and tile scan transformation process, the following variables, ie, the list ColWidth [i] for i from 0 to num_level1_tile_columns_minus1 containing both-end values, that specifies the width of the i-th level 1 tile column in CTB units. , Specify the height of the jth level 1 tile row in CTB units, list RowHeight [j] for j from 0 to num_level1_tile_rows_minus1 including the double-ended values, specify the number of tiles in the picture with reference to PPS Variable NumTilesInPic, specify the width of the i-th tile in CTB units, list TileWidth [i] for i from 0 to NumTilesInPic including both ends values, specify the height of the i-th tile in CTB units, List TileHeight [i] for i from 0 to NumTilesInPic including both-end values, List TileColBd [i] for i from 0 to NumTilesInPic including both-ended values, specifying the location of the boundary of the third tile column in CTB units, List TileRowBd [i] for j from 0 to NumTilesInPic, including both-end values, specifying the location of the i-th tile row boundary in units of CTB, conversion from CTB address in CTB raster scan of picture to CTB address in tile scan CtbAddrRsToTs [ctbAddrRs], a list for ctbAddrRs ranging from 0 to PicSizeInCtbsY-1, containing the both-end value, specifying the conversion of the CTB address in the tile scan to the CTB address in the CTB raster scan of the picture, from 0 containing the two-ended value. List CtbAddrTsToRs [ctbAddrTs] for ctbAddrTs spanning PicSizeInCtbsY-1, List TileId [ctbAddrTs] for ctbAddrTs spanning from 0 to PicSizeInCtbsY-1 that specify the conversion of CTB address to tile ID in tile scan, from tile index List NumCtusInTile [t] for tileIdx from 0 to PicSizeInCtbsY-1 with both-ended values that specifies the conversion to the number of CTUs in the tile. ileIdx], and a list FirstCtbAddrTs [tileIdx] for tileIdx from 0 to NumTilesInPic-1, including the double-ended values, that specifies the translation of the tile ID to the CTB address in the tile scan of the first CTB in the tile.

An exemplary tile group header semantic is: A tile group may contain only some complete Level 1 tiles, or only some complete Level 2 tiles in a single Level 1 tile. tile_group_address specifies the tile address of the first tile in the tile group, where the tile address is equal to TileId [firstCtbAddrTs] as specified by Equation 6-8, where firstCtbAddrTs is in the tile group. The CTB address in the tile scan of the CTB of the first CTU. The length of tile_group_address is Ceil (Log2 (NumTilesInPic)) bits. The value of tile_group_address must be in the range 0 to NumTilesInPic-1 including the double-ended value, and the value of tile_group_address will be the value of tile_group_address of any other coded tilegroup NAL unit in the same coded picture. Must not be equal. If tile_group_address does not exist, it is presumed to be equal to 0.

The following is a second specific exemplary embodiment of the above embodiment. An exemplary CTB raster and tile scanning process is as follows. The variable NumTilesInPic, which refers to the PPS to specify the number of tiles in the picture, as well as the location of the i-th tile column boundary in CTB units and the location of the i-th tile row boundary in CTB units. List TileColBd [i], TileRowBd [i] for i from 0 to NumTilesInPic-1, including both-end values, specifying the width of the tile column in CTB units and the height of the i-th tile column in CTB units. , TileWidth [i], and TileHeight [i] are derived as follows.
for (tileIdx = 0, i = 0; i <NumLevel1Tiles; i ++) {
tileX = i% (num_level1_tile_columns_minus1 + 1)
tileY = i / (num_level1_tile_columns_minus1 + 1)
if (! Level2_tile_split_flag [i]) {(6-5)
TileColBd [tileIdx] = colBd [tileX]
TileRowBd [tileIdx] = rowBd [tileY]
TileWidth [tileIdx] = ColWidth [tileX]
TileHeight [tileIdx] = RowHeight [tileY]
tileIdx ++
} else {
if (uniform_level2_tile_spacing_flag [i]) {
for (k = 0; k <= num_level2_tile_columns_minus1 [i]; k ++)
colWidth2 [k] = ((k + 1) * ColWidth [tileX]) / (num_level2_tile_columns_minus1 [i] +1)-(k * ColWidth [tileX]) / (num_level2_tile_columns_minus1 [i] +1)


for (k = 0; k <= num_level2_tile_rows_minus1 [i]; k ++)
rowHeight2 [k] = ((k + 1) * RowHeight [tileY]) / (num_level2_tile_rows_minus1 [i] +1)-(k * RowHeight [tileY]) / (num_level2_tile_rows_minus1 [i] +1)


} else {
colWidth2 [num_level2_tile_columns_minus1 [i]] = ColWidth [tileX])
for (k = 0; k <= num_level2_tile_columns_minus1 [i]; k ++) {
colWidth2 [k] = tile_level2_column_width_minus1 [k] + 1
colWidth2 [k]-= colWidth2 [k]
}
rowHeight2 [num_level2_tile_rows_minus1 [i]] = RowHeight [tileY])
for (k = 0; k <= num_level2_tile_rows_minus1 [i]; k ++) {
rowHeigh2 [k] = tile_level2_column_width_minus1 [k] +1
rowHeight2 [k]-= rowHeight2 [k]
}
}
for (colBd2 [0] = 0, k = 0; k <= num_level2_tile_columns_minus1 [i]; k ++)
colBd2 [k + 1] = colBd2 [k] + colWidth2 [k]
for (rowBd2 [0] = 0, k = 0; k <= num_level2_tile_rows_minus1 [i]; k ++)
rowBd2 [k + 1] = rowBd2 [k] + rowHeight2 [k]
numSplitTiles = (num_level2_tile_columns_minus1 [i] +1) * (num_level2_tile_rows_minus1 [i] +1)

for (k = 0; k <numSplitTiles; k ++) {
tileX2 = k% (num_level2_tile_columns_minus1 [i] +1)
tileY2 = k / (num_level2_tile_columns_minus1 [i] +1)
TileColBd [tileIdx] = colBd [tileX] + colBd2 [tileX2]
TileRowBd [tileIdx] = rowBd [tileY] + rowBd2 [tileY2]
TileWidth [tileIdx] = colWidth2 [tileX2]
TileHeight [tileIdx] = rowHeight2 [tileY2]
tileIdx ++
}
}
}
NumTilesInPic = tileIdx

An exemplary picture parameter set RBSP syntax is as follows.

An exemplary picture parameter set RBSP semantic is: uniform_level2_tile_spacing_flag [i] is used to specify that the level 2 tile column boundaries of the i-th level 1 tile and similarly the level 2 tile row boundaries of the i-th level 1 tile are uniformly distributed across the picture. Set equal to 1. uniform_level2_tile_spacing_flag [i] is a syntax element level2_tile_column_width_minus1 [i] that the level 2 tile column boundaries of the i-th level 1 tile and the level 2 tile row boundaries of the i-th level 1 tile are not evenly distributed across the picture. Can be set equal to 0 to specify that it is explicitly signaled using j] and level2_tile_row_height_minus1 [j]. When not present, the value of uniform_level2_tile_spacing_flag [i] is presumed to be equal to 1. level2_tile_column_width_minus1 [j] + 1 specifies the width of the jth level 2 tile column of the ith level 1 tile in CTB units. level2_tile_row_height_minus1 [j] + 1 specifies the height of the jth tile level 2 row of the ith level 1 tile in CTB units.

The following is a third specific exemplary embodiment of the above embodiment. The CTB raster and tile scanning process is as follows. The list ColWidth [i] for i from 0 to num_level1_tile_columns_minus1 containing the double-ended values, which specifies the width of the first level tile column of the first i in CTB units, may be derived as follows.
for (i = 0; i <= num_level1_tile_columns_minus1; i ++)
ColWidth [i] = ((i + 1) * PicWidthInCtbsY) / (num_level1_tile_columns_minus1 + 1)-(i * PicWidthInCtbsY) / (num_level1_tile_columns_minus1 + 1)

The list RowHeight [j] for j from 0 to num_level1_tile_rows_minus1 containing the double-ended values, which specifies the height of the jth tile row in CTB units, may be derived as follows.
for (j = 0; j <= num_level1_tile_rows_minus1; j ++)
RowHeight [j] = ((j + 1) * PicHeightInCtbsY) / (num_level1_tile_rows_minus1 + 1)-(j * PicHeightInCtbsY) / (num_level1_tile_rows_minus1 + 1)

An exemplary picture parameter set RBSP syntax is as follows.

The following is a fourth specific exemplary embodiment of the above embodiment. An exemplary picture parameter set RBSP syntax is as follows.

An exemplary picture parameter set RBSP semantic is: Bitstream conformance may require the following constraints to apply: The value MinTileWidth specifies the minimum tile width and must be equal to 256 luma samples. The value MinTileHeight specifies the minimum tile height and must be equal to 64 luma samples. The minimum tile width and minimum tile height values may vary according to profile and level definitions. The variable Level1TilesMayBeFurtherSplit may be derived as follows.
Level1TilesMayFurtherBeSplit = 0
for (i = 0,! Level1TilesMayFurtherBeSplit && i = 0; i <NumLevel1Tiles; i ++)
if ((ColWidth [i] * CtbSizeY> = (2 * MinTileWidth)) || (RowHeight [i] * CtbSizeY> = (2 * MinTileHeight)))

Level1TilesMayFurtherBeSplit = 1

level2_tile_present_flag specifies that one or more level tiles will be split into more tiles. When not present, the value of level2_tile_present_flag is presumed to be equal to 0. level2_tile_split_flag [i] + 1 specifies that the i-th level 1 tile is split into two or more tiles. When not present, the value of level2_tile_split_flag [i] is presumed to be equal to 0.

The following is a fifth specific exemplary embodiment of the above embodiment. Each tile location and each tile size may be signaled. The syntax for supporting such tile-structured signaling can be as represented below. tile_top_left_address [i] and tile_bottom_right_address [i] are CTU indexes in the picture that indicate the rectangular area covered by the tile. The number of bits for signaling these syntax elements should be sufficient to represent the maximum number of CTUs in the picture.

Each tile location and each tile size may be signaled. The syntax for supporting such tile-structured signaling can be as represented below. tile_top_left_address [i] is the CTU index of the first CTU in the tile in the order of the CTU raster scans of the picture. The tile width and tile height specify the size of the tile. By signaling units with a common tile size first, some bits can be saved when signaling these two syntax elements.

Alternatively, the signaling could be:

In another example, each tile size could be signaled as follows: The location of each tile does not have to be signaled to signal the flexible tile structure. Instead, a flag may be signaled to specify whether the tile should be placed just to the right or just below the previous tile. This flag may not be present if the tile can only be to the right of or below the current tile.

The values of tile_x_offset [i] and tile_y_offset [i] can be derived by the following ordered steps.
tile_x_offset [0] and tile_y_offset [0] are set equal to 0.
maxWidth is set equal to tile_width [0] and maxHeight is set equal to tile_height [0].
runningWidth is set equal to tile_width [0] and runningHeight is set equal to tile_height [0].
lastNewRowHeight is set equal to 0.
TilePositionCannotBeInferred = false.
For i> 0, the following applies:
Set the value isRight as follows.
If runningWidth + tile_width [i] <= PictureWidth, isRight == 1 and
In other cases, isRight == 0.
Set the value isBelow as follows.
If runningHeight + tile_height [i] <= PictureHeight, isBelow == 1 and
In other cases, isBelow == 0.
If isRight == 1 && isBelow == 1, TilePositionCannotBeInferred = true.
If isRight == 1 && isBelow == 0, then the following, that is,
right_tile_flag [i] = 1,
tile_x_offset [i] = runningWidth,
tile_y_offset [i] = (runningWidth == maxWidth)? 0: lastNewRowHeight,
lastNewRowHeight = (runningWidth == maxWidth)? 0: lastNewRowHeight is applied,
Instead, if isRight == 0 && isBelow == 1, then the following, that is,
right_tile_flag [i] = 0,
tile_y_offset [i] = runningHeight,
tile_x_offset [i] = (runningHeight == maxHeight)? 0: tile_x_offset [i-1],
lastNewRowHeight = (runningHeight == maxHeight && runningWidth == maxWidth)? runningHeight: lastNewRowHeight is applied,
Instead, if isRight == 1 && isBelow == 1 && right_tile_flag [i] == 1, then the following, that is,
tile_x_offset [i] = runningWidth,
tile_y_offset [i] = (runningWidth == maxWidth)? 0: lastNewRowHeight,
lastNewRowHeight = (runningWidth == maxWidth)? 0: lastNewRowHeight is applied,
In other cases (ie isRight == 1 && isBelow == 1 && right_tile_flag [i] == 0),
tile_y_offset [i] = runningHeight,
tile_x_offset [i] = (runningHeight == maxHeight)? 0: tile_x_offset [i-1],
lastNewRowHeight = (runningHeight == maxHeight && runningWidth == maxWidth)? runningHeight: lastNewRowHeight is applied,
When right_tile_flag [i] == 1, the following, that is,
runningWidth = runningWidth + tile_width [i] is applied,
If runningWidth> maxWidth, set maxWidth equal to runningWidth and
runningHeight is equal to tile_y_offset [i] + tile_height [i],
In other cases (ie right_tile_flag [i] == 0), the following, i.e.
runningHeight = runningHeight + tile_height [i] is applied,
If runningHeight> maxHeight, set maxHeight equal to runningHeight and set it to runningHeight.
runningWidth is equal to tile_x_offset [i] + tile_width [i].

The above can be described in pseudocode as:
tile_x_offset [0] = 0
tile_y_offset [0] = 0
maxWidth = tile_width [0]
maxHeight = tile_height [0]
runningWidth = tile_width [0]
runningHeight = tile_height [0]
lastNewRowHeight = 0
isRight = false
isBelow = false
TilePositionCannotBeInferred = false
for (i = 1; i <num_tiles_minus2 + 2; i ++) {
TilePositionCannotBeInferred = false
isRight = (runningWidth + tile_width [i] <= PictureWidth)? true: false
isbelow = (runningHeight + tile_height [i] <= PictureHeight)? true: false
if (! isRight &&! isBelow)
//error. This case should not occur.
if (isRight && isBelow)
TilePositionCannotBeInferred = true
if (isRight &&! isBelow) {
right_tile_flag [i] = true
tile_x_offst [i] = runningWidth
tile_y_offset [i] = (runningWidth == maxWidth)? 0: lastNewRowHeight
lastNewRowHeight = tile_y_offset [i]
}
else if (! isRight && isBelow) {
right_tile_flag [i] = false
tile_y_offset [i] = runningHeight
tile_x_offset [i] = (runningHeight == maxHeight)? 0: tile_x_offset [i-1]
lastNewRowHeight = (runningHeight == maxHeight && runningWidth == maxWidth)? runningHeight: lastNewRowHeight

}
else if (right_tile_flag [i]) {
tile_x_offst [i] = runningWidth
tile_y_offset [i] = (runningWidth == maxWidth)? 0: lastNewRowHeight
lastNewRowHeight = tile_y_offset [i]
}
else {
tile_y_offset [i] = runningHeight
tile_x_offset [i] = (runningHeight == maxHeight)? 0: tile_x_offset [i-1]
lastNewRowHeight = (runningHeight == maxHeight && runningWidth == maxWidth)? runningHeight: lastNewRowHeight

}
}
if (right_tile_flag [i]) {
runningWidth + = tile_width [i]
if (runningWidth> maxWidth) maxWidth = runningWidth
runningHeight = tile_y_offset [i] + tile_height [i]
}
else {
runningHeight + = tile_height [i]
if (runningHeight> maxHeight) maxHeight = runningHeight
runningWidth = tile_x_offset [i] + tile_width [i]
}

The following is an implementation in pseudocode that derives the size of the last tile.
tile_x_offset [0] = 0
tile_y_offset [0] = 0
maxWidth = tile_width [0]
maxHeight = tile_height [0]
runningWidth = tile_width [0]
runningHeight = tile_height [0]
lastNewRowHeight = 0
isRight = false
isBelow = false
TilePositionCannotBeInferred = false
for (i = 1; i <num_tiles_minus2 + 2; i ++) {
currentTileWidth = (i == num_tiles_minus2 + 1)? (PictureWidth-runningWidth)% PictureWidth: tile_width [i]

currentTileHeight = (i == num_tiles_minus2 + 1)? (PictureHeight-runningHeight)% PictureHeight: tile_Height [i]

isRight = (runningWidth + currentTileWidth <= PictureWidth)? True: false
isbelow = (runningHeight + currentTileHeight <= PictureHeight)? true: false
if (! isRight &&! isBelow)
//error. This case should not occur.
if (isRight && isBelow)
TilePositionCannotBeInferred = true
if (isRight &&! isBelow) {
right_tile_flag [i] = true
tile_x_offst [i] = runningWidth
tile_y_offset [i] = (runningWidth == maxWidth)? 0: lastNewRowHeight
lastNewRowHeight = tile_y_offset [i]
}
else if (! isRight && isBelow) {
right_tile_flag [i] = false
tile_y_offset [i] = runningHeight
tile_x_offset [i] = (runningHeight == maxHeight)? 0: tile_x_offset [i-1]
lastNewRowHeight = (runningHeight == maxHeight && runningWidth == maxWidth)? runningHeight: lastNewRowHeight

}
else if (right_tile_flag [i]) {
tile_x_offst [i] = runningWidth
tile_y_offset [i] = (runningWidth == maxWidth)? 0: lastNewRowHeight
lastNewRowHeight = tile_y_offset [i]
}
else {
tile_y_offset [i] = runningHeight
tile_x_offset [i] = (runningHeight == maxHeight)? 0: tile_x_offset [i-1]
lastNewRowHeight = (runningHeight == maxHeight && runningWidth == maxWidth)? runningHeight: lastNewRowHeight

}
}
if (right_tile_flag [i]) {
runningWidth + = currentTileWidth
if (runningWidth> maxWidth) maxWidth = runningWidth
runningHeight = tile_y_offset [i] + currentTileHeight
}
else {
runningHeight + = currentTileHeight
if (runningHeight> maxHeight) maxHeight = runningHeight
runningWidth = tile_x_offset [i] + currentTileWidth
}

For further signaling bit savings, a number of unique tile sizes may be signaled to support unit tile size tabulation. The tile size can then be referenced only by the index.

The following is a sixth specific exemplary embodiment of the above embodiment. An exemplary CTB raster and tile scanning process is as follows. List TileId [ctbAddrTs] for ctbAddrTs from 0 to PicSizeInCtbsY-1, including the double-ended values, and CTB in the tile scan of the first CTB in the tile, specifying the conversion of the CTB address to the tile ID in the tile scan. The list FirstCtbAddrTs [tileIdx] for tileIdx from 0 to NumTilesInPic-1, including double-ended values, that specifies the translation to the address may be derived as follows:
for (i = 0, tileIdx = 0; i <= NumTilesInPic; i ++, tileIdx ++) {
for (y = TileRowBd [i]; y <TileRowBd [i + 1]; y ++) (6-8)
for (x = TileColBd [i]; x <TileColBd [i + 1]; x ++)
TileId [CtbAddrRsToTs [y * PicWidthInCtbsY + x]] = explicit_tile_id_flag? tile_id_val [i]: tileIdx

FirstCtbAddrTs [tileIdx] = CtbAddrRsToTs [TileColBd [tileIdx]] * PicWidthInCtbsY + TileColBd [tileIdx]]

}

The set TileIdToIdx [tileId] for a set of NumTilesInPic tileId values that specify the conversion from tile ID to tile index may be derived as follows:
for (ctbAddrTs = 0, tileIdx = 0, tileStartFlag = 1; ctbAddrTs <PicSizeInCtbsY; ctbAddrTs ++) {
if (tileStartFlag) {
TileIdToIdx [TileId [ctbAddrTs]] = tileIdx
tileStartFlag = 0
}
tileEndFlag = ctbAddrTs == PicSizeInCtbsY-1 || TileId [ctbAddrTs + 1]! = TileId [ctbAddrTs]

if (tileEndFlag) {
tileIdx ++
tileStartFlag = 1
}
}

An exemplary picture parameter set RBSP syntax and semantics are as follows.

In some examples, the semantics can be: tile_id_len_minus1 + 1 refers to the PPS and specifies the number of bits used to represent the syntax element tile_id_val [i] and the syntax element tile_group_address in the tile group header. The value of tile_id_len_minus1 can be in the range Ceil (Log2 (NumTilesInPic) ~ 15 including both ends value. In other examples, the semantics can be: tile_id_len_minus1 + 1 refers to the tile ID value. You may specify the number of bits used to represent the syntax element tile_id_val [i] and the syntax element in the tile group header with reference to the PPS to be used. The value of tile_id_len_minus1 is Ceil (including the double-ended value). It may be in the range of Log2 (NumTilesInPic) to 15.

The explicit_tile_id_flag may be set equal to 1 to specify that the tile ID for each tile is explicitly signaled. The explicit_tile_id_flag may be set equal to 0 to specify that the tile ID is not explicitly signaled. tile_id_val [i] may specify the tile ID of the tile of the i-th tile in the picture. The length of tile_id_val [i] may be tile_id_len_minus 1 + 1 bits. Tile_id_val [i] does not have to be equal to tile_id_val [j] and j is greater than i for any integer m in the range 0 to NumTilesInPic-1, including the two-ended value, when i is not equal to j When tile_id_val [i] may be smaller than tile_id_val [j].

An exemplary tile group header RBSP syntax and semantics are as follows.

tile_group_address may specify the tile address of the first tile in the tile group. The length of tile_group_address may be tile_id_len_minus1 + 1 bits. The value of tile_group_address can be in the range ^{0-2 tile_id_len_minus1 + 1} -1, including both ends, and the value of tile_group_address can be the tile_group_address of any other coded tilegroup NAL unit in the same coded picture. It does not have to be equal to the value. tile_group_address may be estimated to be equal to 0 when it does not exist.

FIG. 9 is a schematic diagram of an exemplary video coding device 900. The video coding device 900 is suitable for implementing the disclosed examples / embodiments as described herein. The video coding device 900 includes a transceiver unit (Tx) including a downstream port 920, an upstream port 950, and / or a transmitter and / or a receiver for communicating data upstream and / or downstream over a network. /Rx) 910 is provided. The video coding device 900 also includes a processor 930, including a logical unit and / or a central processing unit (CPU) for processing the data, and a memory 932 for storing the data. The video coding device 900 is also an electrical component, an optical-to-electrical (OE) component, an electrical-to-optical (EO) component, and / or a telecommunications network, an optical communication network, Alternatively, it may include wireless communication components coupled to upstream port 950 and / or downstream port 920 for the communication of data over a wireless communication network. The video coding device 900 may also include an input and / or output (I / O) device 960 for communicating data with the user. The I / O device 960 may include an output device such as a display for displaying video data and a speaker for outputting audio data. The I / O device 960 may also include an input device such as a keyboard, mouse, trackball, and / or a corresponding interface for interacting with such an output device.

Processor 930 is implemented by hardware and software. The processor 930 may be implemented as one or more CPU chips, cores (eg, as a multi-core processor), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), and digital signal processors (DSPs). Processor 930 communicates with downstream port 920, Tx / Rx910, upstream port 950, and memory 932. The processor 930 includes a coding module 914. Coding module 914 may employ images segmented according to Bitstream 500 and / or Flexible Video Tiling Method 800, such as Methods 100, 1000, and 1100, Mechanism 600, and / or Application 700. Implement the disclosed embodiments described in the document. Coding module 914 may also implement any other method / mechanism described herein. Further, the coding module 914 may implement a codec system 200, an encoder 300, and / or a decoder 400. For example, the coding module 914 can divide the picture into first level tiles and the first level tiles into second level tiles. Coding module 914 also includes a first level tile, such that each tile group contains a contiguous sequence of several first level tiles, or a single first level tile, a second level tile. Level tiles and second level tiles can be assigned to tile groups, so that all second level tiles created from a single first level tile will be assigned to the same tile group. Will be. Coding module 914 can also encode and / or decode such tiles in a scan order such that the first level tiles are coded in raster scan order, and the second level tiles can be that. Second level tiles such as are coded in raster scan order within the first level tiles from which they are separated. Coding module 914 further supports the adoption of such a mechanism for combining sub-pictures at different resolutions into a single picture for various use cases as described herein. .. Therefore, the coding module 914 improves the functionality of the video coding device 900 and addresses issues specific to video coding technology. In addition, the coding module 914 provides the conversion of the video coding device 900 to different states. Alternatively, the coding module 914 can be implemented as an instruction stored in memory 932 (eg, as a computer program product stored on a non-temporary medium) and executed by processor 930.

Memory 932 is one of disk, tape drive, solid state drive, read-only memory (ROM), random access memory (RAM), flash memory, ternary associative memory (TCAM), static random access memory (SRAM), etc. It has the above memory types. Memory 932 may also be used as an overflow data storage device to store such programs when they are selected for execution, and to store instructions and data read during program execution. good.

FIG. 10 is a flowchart of an exemplary method 1000 that encodes an image by adopting a flexible tiling method such as the flexible tiling method 800. Method 1000 may be employed by an encoder, such as Codec System 200, Encoder 300, and / or Video Coding Device 900, when running Method 100, Mechanism 600, and / or supporting application 700. Further, method 1000 may be employed to generate a bitstream 500 for transmission to a decoder such as the decoder 400.

Method 1000 may be initiated when the encoder receives a video sequence containing multiple images and, for example, determines that the video sequence should be encoded into a bitstream based on user input. As an example, a video sequence, and thus an image, can be encoded at multiple resolutions. In step 1001, the picture is divided into a plurality of first level tiles. A subset of first level tiles may also be divided into multiple second level tiles. In some examples, the first level tiles outside the subset may contain picture data at the first resolution. Further, the second level tile may contain picture data at a second resolution different from the first resolution. In some examples, each first level tile within a subset of first level tiles may contain two or more complete second level tiles.

In step 1003, the first level tile and the second level tile are assigned to one or more tile groups. Allocation is one of the second level tiles where each tile group is divided into several first level tiles, each sequence of second level tiles from a single first level tile. It may be executed to include the above consecutive sequences or combinations thereof. As a specific example, the assignment may be performed so that all second level tiles created from a single first level tile are assigned to the same tile group. In some examples, each of one or more tile groups may be constrained so that all tiles in the assigned tile group cover the rectangular portion of the picture.

In step 1005, the first level tile and the second level tile are encoded in the bitstream. For example, the first level tiles and the second level tiles may be encoded according to the scan order. In certain examples, encoding according to scan order may include encoding first level tiles in raster scan order. Raster scan order coding of the first level tiles may be interrupted when one of the second level tiles is encountered. All consecutive second level tiles may be encoded in raster scan order before continuing raster scan order coding of the first level tiles. For example, all second level tiles separated from the current first level tile before encoding any second level tile separated from subsequent second level tiles. It may be encoded. At step 1007, the bitstream may be stored for communication towards the decoder.

FIG. 11 is a flowchart of an exemplary method 1100 that decodes an image by adopting a flexible tiling method such as the flexible video tiling method 800. Method 1100 may be employed by a decoder, such as Codec System 200, Decoder 400, and / or Video Coding Device 900, when running Method 100, Mechanism 600, and / or supporting application 700. Further, method 1100 may be employed upon receiving a bitstream 500 from an encoder such as the encoder 300.

Method 1100 may start when the decoder begins to receive, for example, a bitstream of coded data representing a video sequence as a result of method 1000. The bitstream may contain video data from a video sequence coded at multiple resolutions. At step 1101, the bitstream is received. The bitstream contains pictures segmented into multiple first level tiles. The subset of first level tiles may be further subdivided into multiple second level tiles. In some examples, the first level tiles outside the subset may contain picture data at the first resolution. Further, the second level tile may contain picture data at a second resolution different from the first resolution. In another example, each first level tile within a subset of first level tiles may contain two or more complete second level tiles. First level tiles and second level tiles are assigned to one or more tile groups. Allocation is one of the second level tiles where each tile group is divided into several first level tiles, each sequence of second level tiles from a single first level tile. It may be executed to include the above consecutive sequences or combinations thereof. As a specific example, the assignment may be performed so that all second level tiles created from a single first level tile are assigned to the same tile group. In some examples, each of one or more tile groups may be constrained so that all tiles in the assigned tile group cover the rectangular portion of the picture.

In step 1105, the first level tiles and the second level tiles may be decrypted based on one or more tile groups. In some examples, the first level tiles and the second level tiles are decoded according to the scan order. In certain examples, decoding according to scan order may include decoding first level tiles in raster scan order. Raster scan order coding of the first level tiles may be interrupted when one of the second level tiles is encountered. All consecutive second level tiles may then be encoded in raster scan order before continuing raster scan order decoding of the first level tiles. For example, all second level tiles separated from the current first level tile will be decrypted before decrypting any second level tile separated from subsequent second level tiles. May be done.

In step 1107, a reconstructed video sequence may be generated for display based on the decrypted first level tiles and second level tiles.

FIG. 12 is a schematic diagram of an exemplary system 1200 for coding a video sequence by adopting a flexible tiling scheme such as the flexible video tiling scheme 800. System 1200 may be implemented by encoders and decoders such as codec system 200, encoder 300, decoder 400, and / or video coding device 900. In addition, system 1200 may be employed when implementing methods 100, 1000, 1100, mechanism 600, and / or application 700. The system 1200 may also encode the data into a bitstream such as the bitstream 500 and decode such a bitstream for display to the user.

System 1200 includes video encoder 1202. The video encoder 1202 comprises a partition module 1201 for partitioning a picture into a plurality of first level tiles and a subset of the first level tiles into a plurality of second level tiles. Video Encoder 1202 has first level tiles and second level tiles so that all second level tiles created from a single first level tile are assigned to the same tile group. Also includes an allocation module 1203 for assigning to one or more tile groups. The video encoder 1202 further comprises a coding module 1205 for encoding first level tiles and second level tiles into a bitstream. The video encoder 1202 further comprises a storage module 1207 for storing a bitstream for communication to the decoder. The video encoder 1202 further comprises a transmit module 1209 for transmitting a bitstream towards the decoder. The video encoder 1202 may be further configured to perform any of the steps of Method 1000.

The system 1200 also includes a video decoder 1210. The video decoder 1210 comprises a receive module 1211 for receiving a bitstream containing pictures segmented into multiple first level tiles, with a subset of the first level tiles being multiple second level tiles. The first level tiles and the second level tiles are further subdivided into tiles so that all second level tiles created from a single first level tile are assigned to the same tile group. Tiles are assigned to one or more tile groups. The video decoder 1210 further comprises a decoding module 1213 for decoding first level tiles and second level tiles based on one or more tile groups. The video decoder 1210 further comprises a generation module 1215 for generating a reconstructed video sequence for display based on the decoded first level tiles and the second level tiles. The video decoder 1210 may be further configured to perform any of the steps of method 1100.

When there is no intervening component between the first component and the second component except for lines, traces, or other media, the first component is directly coupled to the second component. .. When there is an intervening component other than a line, trace, or other medium between the first component and the second component, the first component is indirectly combined with the second component. To. The term "coupled" and its variants include both direct and indirect coupling. The use of the term "about" means a range that includes ± 10% of the subsequent number, unless otherwise stated.

It should also be appreciated that the steps of the exemplary methods described herein are not necessarily required to be performed in the order described, and the order of the steps in such methods is merely an example. Should be understood as being. Similarly, additional steps may be included in such methods, and some steps may be excluded or combined in a method consistent with the various embodiments of the present disclosure.

Although some embodiments are provided in the present disclosure, the disclosed systems and methods may be embodied in many other specific embodiments without departing from the spirit or scope of the present disclosure. May be understood. This example should be considered exemplary rather than limiting, and its intent should not be limited to the details given herein. For example, various elements or components may be combined, integrated within another system, or some features may be excluded or not implemented.

In addition, the techniques, systems, subsystems, and methods described and illustrated individually or separately in various embodiments may be combined or without departing from the scope of the present disclosure of other systems, components. , Technique, or method. Modifications, substitutions, and other examples of modifications can be ascertained by one of ordinary skill in the art and may be made without departing from the spirit and scope disclosed herein.

100 How to operate
200 Coding and decoding (codec) system
201 segmented video signal
211 General purpose coder control component
213 Transformation scaling and quantization components
215 Intra-picture estimation component
217 Intrapicture Forecasting Component
219 Motion compensation component
221 Motion estimation component
223 Decrypted picture buffer component
225 In-loop filter component
227 Filter Control Analysis Component
229 Scaling and inverse transformation components
231 Header Formatting and Context Adaptive Binary Arithmetic Coding (CABAC) Components
300 video encoder
301 segmented video signal
313 Transformation and Quantization Components
317 Intra-picture prediction component
321 Motion compensation component
323 Decrypted picture buffer component
325 In-loop filter component
329 Inverse transformation and quantization components
331 Entropy coding component
400 video decoder
417 Intra-picture prediction component
421 motion compensation component
423 Decrypted picture buffer component
425 In-loop filter component
429 Inverse transformation and quantization components
433 Entropy decoding component
500 bitstream
510 Sequence Parameter Set (SPS)
512 Picture Parameter Set (PPS)
514 tile group header
520 Image data
523, 601, 603, 605, 607 tiles
600 mechanism
610 Extractor truck
611 First resolution
612 Second resolution
700 Video Conference Application
701 Talking participants
703 Other participants
800 Flexible video tiling method
801 First level tile
803 Second level tile
805 tile group
807 Scanning order
900 video coding device
910 Transceiver unit (Tx / Rx)
914 Coding module
920 downstream port
930 processor
932 memory
950 upstream port
960 Input and / or Output (I / O) Devices
1000 How to operate
1100 How it works
1200 system
1201 division module
1202 video encoder
1203 allocation module
1205 coding module
1207 Storage module
1209 Transmitter module, transmitter
1210 video decoder
1211 receiver module, receiver
1213 Decryption module
1215 generation module

Claims (20)

It ’s a method implemented in an encoder,
A step of dividing a picture into multiple first level tiles by the encoder processor.
A step of dividing a subset of the first level tiles into a plurality of second level tiles by the processor.
Each tile group consists of several first level tiles, one or more sequences of second level tiles in which each sequence of second level tiles is split from a single first level tile. A step of assigning the first level tile and the second level tile to one or more tile groups by the processor so as to include the sequence to be performed, or a combination thereof.
A step of encoding the first level tile and the second level tile into a bitstream by the processor.
A method comprising storing the bitstream in memory of the encoder for communication towards the decoder. It ’s a method implemented in an encoder,
A step of dividing a picture into multiple first level tiles by the encoder processor.
A step of dividing a subset of the first level tiles into a plurality of second level tiles by the processor.
The first level tile and the second level tile by the processor so that all second level tiles created from a single first level tile are assigned to the same tile group. And the steps to assign to one or more tile groups,
A step of encoding the first level tile and the second level tile into a bitstream by the processor.
A method comprising storing the bitstream in memory of the encoder for communication towards the decoder. A first level tile outside the subset contains picture data at a first resolution, and a second level tile contains picture data at a second resolution different from the first resolution. The method according to item 1 or 2. The method of any one of claims 1 to 3, wherein each first level tile in the subset of first level tiles comprises two or more complete second level tiles. .. The first level tile and the second level tile are encoded according to the scanning order, and the step of encoding according to the scanning order is
The step of coding the first level tiles in raster scan order,
When one of the second level tiles is encountered, the step of interrupting the raster scan order coding of the first level tile, and
A step of encoding all consecutive second level tiles in raster scan order and then continuing said raster scan order coding of said first level tiles.
The method according to any one of claims 1 to 4. All second level tiles separated from the current first level tile are encoded before any second level tiles separated from subsequent second level tiles are encoded. The method according to any one of claims 1 to 5. 10. One of claims 1-6, wherein each of the one or more tile groups is constrained so that all tiles in the assigned tile group cover the rectangular portion of the picture. Method. It ’s a method that is carried out in the decoder.
A step of receiving a bit stream containing a picture divided into a plurality of first level tiles by the processor of the decoder through a receiver, wherein a subset of the first level tiles is a plurality of first level tiles. Further subdivided into two level tiles, each tile group is divided into several first level tiles, each sequence of second level tiles from a single first level tile. The first level tile and the second level tile are assigned to one or more tile groups so as to include one or more consecutive sequences of tiles of the first level, or a combination thereof. ,
A step of decoding the first level tile and the second level tile by the processor based on the one or more tile groups.
A method comprising the step of generating a reconstructed video sequence for display by the processor based on the decoded first level tiles and the second level tiles. It ’s a method that is carried out in the decoder.
A step of receiving a bitstream containing pictures segmented into a plurality of first level tiles via a receiver by the decoder's processor, wherein a subset of the first level tiles is a plurality of first level tiles. The first level tiles and the above first level tiles are further subdivided into two level tiles so that all second level tiles created from a single first level tile are assigned to the same tile group. A step and a step in which the second level tile is assigned to one or more tile groups.
A step of decoding the first level tile and the second level tile by the processor based on the one or more tile groups.
A method comprising the step of generating a reconstructed video sequence for display by the processor based on the decoded first level tiles and the second level tiles. A first level tile outside the subset contains picture data at a first resolution, and a second level tile contains picture data at a second resolution different from the first resolution. The method according to item 8 or 9. The method of any one of claims 8-10, wherein each first level tile in the subset of first level tiles comprises two or more complete second level tiles. .. The first level tile and the second level tile are decoded according to the scanning order, and the step of decoding according to the scanning order is
The step of decoding the first level tiles in raster scan order,
When one of the second level tiles is encountered, the step of interrupting the raster scan order coding of the first level tile, and
A step of encoding all consecutive second level tiles in raster scan order and then continuing said raster scan order decoding of said first level tiles.
The method according to any one of claims 8 to 11. All second level tiles separated from the current first level tile are decrypted before decrypting any second level tiles separated from subsequent second level tiles. , The method according to any one of claims 8 to 12. 10. One of claims 8-13, wherein each of the one or more tile groups is constrained so that all tiles in the assigned tile group cover the rectangular portion of the picture. Method. It ’s a video coding device,
13. Configured to perform the method of
Video coding device. The method according to any one of claims 1 to 14, which is a non-temporary computer-readable medium comprising a computer programming product for use by a video coding device, wherein the computer programming product is executed by a processor. A non-temporary computer-readable medium comprising computer-executable instructions stored on the non-temporary computer-readable medium, such as causing the video coding device to execute. It ’s an encoder,
To divide a picture into multiple first level tiles and a subset of the first level tiles into multiple second level tiles.
Classification means and
One of the first level tiles and one of the second level tiles so that all second level tiles created from a single first level tile are assigned to the same tile group. Allocation means for allocating to the above tile groups and
Coding means for encoding the first level tile and the second level tile into a bitstream, and
An encoder comprising a storage means for storing the bitstream for communication to the decoder. 17. The encoder according to claim 17, further configured to perform the method of any one of claims 1-7. It ’s a decoder,
A receiving means for receiving a bitstream containing a picture divided into a plurality of first level tiles, wherein a subset of the first level tiles is further divided into a plurality of second level tiles. The first level tile and the second level tile are so that all second level tiles created from a single first level tile are assigned to the same tile group. Receiving means and which are assigned to one or more tile groups,
Decoding means for decoding the first level tile and the second level tile based on the one or more tile groups.
A decoder with a generation means for generating a reconstructed video sequence for display based on the decoded first level tiles and the second level tiles. 19. The decoder according to claim 19, further configured to perform the method of any one of claims 9-16.

Images