WO2021027773A1 - Subpicture size definition in video processing - Google Patents

Subpicture size definition in video processing Download PDF

Info

Publication number
WO2021027773A1
WO2021027773A1 PCT/CN2020/108159 CN2020108159W WO2021027773A1 WO 2021027773 A1 WO2021027773 A1 WO 2021027773A1 CN 2020108159 W CN2020108159 W CN 2020108159W WO 2021027773 A1 WO2021027773 A1 WO 2021027773A1
Authority
WO
WIPO (PCT)
Prior art keywords
picture
video
sub
luma
samples
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/108159
Other languages
French (fr)
Inventor
Kai Zhang
Zhipin DENG
Hongbin Liu
Li Zhang
Jizheng Xu
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing ByteDance Network Technology Co Ltd
ByteDance Inc
Original Assignee
Beijing ByteDance Network Technology Co Ltd
ByteDance Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing ByteDance Network Technology Co Ltd, ByteDance Inc filed Critical Beijing ByteDance Network Technology Co Ltd
Priority to CN202080056646.5A priority Critical patent/CN114270831B/en
Publication of WO2021027773A1 publication Critical patent/WO2021027773A1/en
Priority to US17/667,380 priority patent/US11533513B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/423—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/55—Motion estimation with spatial constraints, e.g. at image or region borders
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/11—Selection of coding mode or of prediction mode among a plurality of spatial predictive coding modes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/119—Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/167—Position within a video image, e.g. region of interest [ROI]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/1883—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit relating to sub-band structure, e.g. hierarchical level, directional tree, e.g. low-high [LH], high-low [HL], high-high [HH]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/513—Processing of motion vectors
    • H04N19/517—Processing of motion vectors by encoding
    • H04N19/52—Processing of motion vectors by encoding by predictive encoding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/537—Motion estimation other than block-based
    • H04N19/54—Motion estimation other than block-based using feature points or meshes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/577—Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
    • H04N19/82—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop

Definitions

  • This document is related to video and image coding and decoding technologies.
  • Digital video accounts for the largest bandwidth use on the internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, it is expected that the bandwidth demand for digital video usage will continue to grow.
  • the disclosed techniques may be used by video or image decoder or encoder embodiments for in which sub-picture based coding or decoding is performed.
  • a method of video processing includes determining, for a conversion between a current block of a first picture of a video and a bitstream representation of the video, a motion candidate based on motion information from a second picture according to a rule.
  • the rule specifies that a position from which the motion information is accessed is constrained to be within a specific subpicture of the second picture.
  • the method also includes performing the conversion based on the determining.
  • a method of video processing includes determining, for a conversion of a current block of a first picture of a video and a bitstream representation of the video, an integer sample from a second picture according to a rule.
  • the second picture comprises a reference picture that is not used in an interpolation process.
  • the rule specifies that a position from which the integer sample is accessed is constrained to be within a specific subpicture of the second picture.
  • the method also includes performing the conversion based on the determining.
  • a method of video processing includes determining, for a conversion of a current block of a video and a bitstream representation of the video, a position at which a reconstructed luma sample is accessed according to a rule.
  • the rule specifies that the position is constrained to be within a specific subpicture of a video picture.
  • the method also includes performing the conversion based on the determining.
  • a method of video processing includes determining, for a conversion of a current block of a video and a bitstream representation of the video, a position at which a picture boundary check is performed according to a rule.
  • the rule specifies that the position is constrained to be within a specific subpicture of a video picture.
  • the method also includes performing the conversion based on the determining.
  • a method of video processing includes resetting, after a conversion of a sub-picture of a video picture of a video and a bitstream representation of the video, a table of motion candidates derived based on past conversions and performing a conversion of a subsequent sub-picture of the video picture and the bitstream representation using the table after the resetting.
  • a method of video processing includes performing a conversion between a video comprising a video picture that includes multiple sub-pictures and multiple video blocks and a coded representation of the video according to a rule.
  • the rule specifies that a boundary between any two sub-pictures is also a boundary between two video blocks.
  • a video block in the video picture is covered by a single subpicture of the video picture.
  • a method of video processing includes performing a conversion between a video unit of a video and a coded representation of the video using at least a video picture, where only one of a sub-picture coding mode or a resolution-changing coding mode is enabled for the video unit.
  • the sub-picture coding mode is a mode in which the video picture is divided into multiple sub-pictures
  • the resolution-changing coding mode is a mode in which a resolution of the video picture is adjusted during the conversion.
  • a method of video processing includes performing a conversion between a video unit of a video and a coded representation of the video using at least a video picture, where both a sub-picture coding mode and a resolution-changing coding mode are enabled for the video unit.
  • the sub-picture coding mode is a mode in which the video picture is divided into multiple sub-pictures
  • the resolution-changing coding mode is a mode in which a resolution of the video picture is adjusted during the conversion.
  • a method of video processing includes performing a conversion between a video comprising one or more video pictures and a coded representation of the video, where a dimension of an individual video picture is constrained to be greater than or equal to 8. In some embodiments, the dimension is a width of the individual video picture.
  • a method of video processing includes performing a conversion between a video picture of a video and a coded representation of the video according to a rule.
  • the video picture comprises at least one sub-picture
  • the rule specifies that a characteristic of a sub-picture is represented as at lease one syntax element in the coded representation, the at least one syntax element being different than an index value of the sub-picture grid in the video picture.
  • a method of video processing includes performing a conversion between a video picture of a video and a coded representation of the video according to a rule.
  • the video picture comprises multiple sub-pictures, each sub-picture comprising multiple elements.
  • the rule specifies that a dimension of individual elements in a sub-picture satisfies a constraint.
  • a method of video processing includes performing a conversion between a video comprising a picture that includes multiple sub-pictures and a coded representation of the video using a coding mode according to a rule.
  • the rule specifies that certain stored information about a previous sub-picture is reset prior to processing each next sub-picture of the multiple sub-pictures.
  • a method of video processing includes performing a temporal filtering operation in a conversion between a video and a coded representation of the video according to a rule.
  • the video comprises multiple video pictures, each comprising multiple sub-pictures.
  • the rule specifies that, for a temporal filtering a current sample in a current sub-picture of a video picture, only samples within the same current sub-picture or a sub-picture in a different video picture corresponding to the current sub-picture are available.
  • a method of video processing includes determining, for a conversion between a block in a video picture of a video and a coded representation of the video, a manner of applying a partitioning method to the block based on whether the block crosses one or more sub-picture boundaries of the video picture. The method also includes performing the conversion based on the determining.
  • a method of video processing includes for a conversion between a video picture of a video and a coded representation of the video, two sub-regions of the video picture.
  • a first sub-region comprises multiple sub-pictures of the video picture and a second sub-region comprises remaining samples in the video picture.
  • the method also includes performing the conversion based on the determining.
  • a method of video processing includes determining, for a video block in a first video region of a video, whether a position at which a temporal motion vector predictor is determined for a conversion between the video block and a bitstream representation of the current video block using an affine mode is within a second video region; and performing the conversion based on the determining.
  • another method of video processing includes determining, for a video block in a first video region of a video, whether a position at which an integer sample in a reference picture is fetched for a conversion between the video block and a bitstream representation of the current video block is within a second video region, wherein the reference picture is not used in an interpolation process during the conversion; and performing the conversion based on the determining.
  • another method of video processing includes determining, for a video block in a first video region of a video, whether a position at which a reconstructed luma sample value is fetched for a conversion between the video block and a bitstream representation of the current video block is within a second video region; and performing the conversion based on the determining.
  • another method of video processing includes determining, for a video block in a first video region of a video, whether a position at which a check regarding splitting, depth derivation or split flag signaling for the video block is performed during a conversion between the video block and a bitstream representation of the current video block is within a second video region; and performing the conversion based on the determining.
  • another method of video processing includes performing a conversion between a video comprising one or more video pictures comprising one or more video blocks, and a coded representation of the video, wherein the coded representation complies with a coding syntax requirement that the conversion is not to use sub-picture coding/decoding and a dynamic resolution conversion coding/decoding tool or a reference picture resampling tool within a video unit.
  • another method of video processing includes performing a conversion between a video comprising one or more video pictures comprising one or more video blocks, and a coded representation of the video, wherein the coded representation complies with a coding syntax requirement that a first syntax element subpic_grid_idx [i] [j] is not larger than a second syntax element max_subpics_minus1.
  • the above-described method may be implemented by a video encoder apparatus that comprises a processor.
  • the above-described method may be implemented by a video decoder apparatus that comprises a processor.
  • these methods may be embodied in the form of processor-executable instructions and stored on a computer-readable program medium.
  • FIG. 1 shows an example of region constraint in temporal motion vector prediction (TMVP) and sub-block TMVP.
  • FIG. 2 shows an example of a hierarchical motion estimation scheme.
  • FIG. 3 is a block diagram of an example of a hardware platform used for implementing techniques described in the present document.
  • FIG. 4 is a flowchart for an example method of video processing.
  • FIG. 5 is a block diagram showing an example video processing system in which various techniques disclosed herein may be implemented.
  • FIG. 6 is a flowchart representation of a method for video processing in accordance with the present technology.
  • FIG. 7 is a flowchart representation of another method for video processing in accordance with the present technology.
  • FIG. 8 is a flowchart representation of another method for video processing in accordance with the present technology.
  • FIG. 9 is a flowchart representation of another method for video processing in accordance with the present technology.
  • FIG. 10 is a flowchart representation of another method for video processing in accordance with the present technology.
  • FIG. 11 is a flowchart representation of another method for video processing in accordance with the present technology.
  • FIG. 12 is a flowchart representation of another method for video processing in accordance with the present technology.
  • FIG. 13 is a flowchart representation of another method for video processing in accordance with the present technology.
  • FIG. 14 is a flowchart representation of another method for video processing in accordance with the present technology.
  • FIG. 15 is a flowchart representation of another method for video processing in accordance with the present technology.
  • FIG. 16 is a flowchart representation of another method for video processing in accordance with the present technology.
  • FIG. 17 is a flowchart representation of another method for video processing in accordance with the present technology.
  • FIG. 18 is a flowchart representation of another method for video processing in accordance with the present technology.
  • FIG. 19 is a flowchart representation of another method for video processing in accordance with the present technology.
  • FIG. 20 is a flowchart representation of yet another method for video processing in accordance with the present technology.
  • the present document provides various techniques that can be used by a decoder of image or video bitstreams to improve the quality of decompressed or decoded digital video or images.
  • video is used herein to include both a sequence of pictures (traditionally called video) and individual images.
  • a video encoder may also implement these techniques during the process of encoding in order to reconstruct decoded frames used for further encoding.
  • Section headings are used in the present document for ease of understanding and do not limit the embodiments and techniques to the corresponding sections. As such, embodiments from one section can be combined with embodiments from other sections.
  • This document is related to video coding technologies. Specifically, it is related to palette coding with employing base colors based representation in video coding. It may be applied to the existing video coding standard like HEVC, or the standard (Versatile Video Coding) to be finalized. It may be also applicable to future video coding standards or video codec.
  • Video coding standards have evolved primarily through the development of the well-known ITU-T and ISO/IEC standards.
  • the ITU-T produced H. 261 and H. 263, ISO/IEC produced MPEG-1 and MPEG-4 Visual, and the two organizations jointly produced the H. 262/MPEG-2 Video and H. 264/MPEG-4 Advanced Video Coding (AVC) and H. 265/HEVC standards [1, 2] .
  • AVC H. 264/MPEG-4 Advanced Video Coding
  • H. 265/HEVC standards [1, 2] .
  • the video coding standards are based on the hybrid video coding structure wherein temporal prediction plus transform coding are utilized.
  • Joint Video Exploration Team JVET was founded by VCEG and MPEG jointly in 2015.
  • JVET Joint Exploration Model
  • FIG. 1 illustrates example region constraint in TMVP and sub-block TMVP.
  • TMVP and sub-block TMVP it is constrained that a temporal MV can only be fetched from the collocated CTU plus a column of 4 ⁇ 4 blocks as shown in FIG. 1.
  • sub-picture-based coding techniques based on flexible tiling approach can be implemented. Summary of the sub-picture-based coding techniques includes the following:
  • Whether in-loop filtering across sub-picture boundaries is disabled can be controlled by the bitstream for each sub-picture.
  • the DBF, SAO, and ALF processes are updated for controlling of in-loop filtering operations across sub-picture boundaries.
  • the sub-picture width, height, horizontal offset, and vertical offset are signalled in units of luma samples in SPS.
  • Sub-picture boundaries are constrained to be slice boundaries.
  • Treating a sub-picture as a picture in the decoding process is specified by slightly updating the coding_tree_unit () syntax, and updates to the following decoding processes:
  • Sub-picture IDs are explicitly specified in the SPS and included in the tile group headers to enable extraction of sub-picture sequences without the need of changing VCL NAL units.
  • OSPS Output sub-picture sets
  • subpics_present_flag 1 indicates that subpicture parameters are present in the present in the SPS RBSP syntax.
  • subpics_present_flag 0 indicates that subpicture parameters are not present in the present in the SPS RBSP syntax.
  • max_subpics_minus1 plus 1 specifies the maximum number of subpictures that may be present in the CVS.
  • max_subpics_minus1 shall be in the range of 0 to 254. The value of 255 is reserved for future use by ITU-T
  • subpic_grid_col_width_minus1 plus 1 specifies the width of each element of the subpicture identifier grid in units of 4 samples.
  • the length of the syntax element is Ceil (Log2 (pic_width_max_in_luma_samples /4) ) bits.
  • NumSubPicGridCols (pic_width_max_in_luma_samples + subpic_grid_col_width_minus1 *4 + 3) / (subpic_grid_col_width_minus1 *4 + 4) (7-5)
  • subpic_grid_row_height_minus1 plus 1 specifies the height of each element of the subpicture identifier grid in units of 4 samples.
  • the length of the syntax element is Ceil (Log2 (pic_height_max_in_luma_samples /4) ) bits.
  • NumSubPicGridRows (pic_height_max_in_luma_samples + subpic_grid_row_height_minus1 *4 + 3) / (subpic_grid_row_height_minus1 *4 + 4) (7-6)
  • subpic_grid_idx [i] [j] specifies the subpicture index of the grid position (i, j) .
  • the length of the syntax element is Ceil (Log2 (max_subpics_minus1 + 1) ) bits.
  • SubPicWidth [subpic_grid_idx [i] [j] ] , SubPicHeight [subpic_grid_idx [i] [j] ] , and NumSubPics are derived as follows:
  • subpic_treated_as_pic_flag [i] 1 specifies that the i-th subpicture of each coded picture in the CVS is treated as a picture in the decoding process excluding in-loop filtering operations.
  • subpic_treated_as_pic_flag [i] 0 specifies that the i-th subpicture of each coded picture in the CVS is not treated as a picture in the decoding process excluding in-loop filtering operations.
  • the value of subpic_treated_as_pic_flag [i] is inferred to be equal to 0.
  • loop_filter_across_subpic_enabled_flag [i] 1 specifies that in-loop filtering operations may be performed across the boundaries of the i-th subpicture in each coded picture in the CVS.
  • loop_filter_across_subpic_enabled_flag [i] 0 specifies that in-loop filtering operations are not performed across the boundaries of the i-th subpicture in each coded picture in the CVS.
  • the value of loop_filter_across_subpic_enabled_pic_flag [i] is inferred to be equal to 1.
  • any coded NAL unit of subPicA shall succeed any coded NAL unit of subPicB in decoding order.
  • each subpicture when decoded, shall have its entire left boundary and entire top boundary consisting of picture boundaries or consisting of boundaries of previously decoded subpictures.
  • CtbToSubPicIdx [ctbAddrRs] for ctbAddrRs ranging from 0 to PicSizeInCtbsY-1, inclusive, specifying the conversion from a CTB address in picture raster scan to a subpicture index, is derived as follows:
  • num_bricks_in_slice_minus1 when present, specifies the number of bricks in the slice minus 1.
  • the value of num_bricks_in_slice_minus1 shall be in the range of 0 to NumBricksInPic-1, inclusive.
  • SubPicIdx SubPicLeftBoundaryPos
  • SubPicTopBoundaryPos SubPicRightBoundaryPos
  • SubPicBotBoundaryPos are derived as follows:
  • variable cbWidth specifying the width of the current coding block in luma samples
  • variable cbHeight specifying the height of the current coding block in luma samples
  • variable currCb specifies the current luma coding block at luma location (xCb, yCb) .
  • slice_temporal_mvp_enabled_flag is equal to 0 or (cbWidth *cbHeight) is less than or equal to 32, both components of mvLXCol are set equal to 0 and availableFlagLXCol is set equal to 0.
  • the bottom right collocated motion vector and the bottom and right boundary sample locations are derived as follows:
  • botBoundaryPos subpic_treated_as_pic_flag [SubPicIdx] ?
  • SubPicBotBoundaryPos pic_height_in_luma_samples-1 (8-424)
  • the variable colCb specifies the luma coding block covering the modified location given by ( (xColBr >> 3) ⁇ 3, (yColBr >> 3) ⁇ 3) inside the collocated picture specified by ColPic.
  • the luma location (xColCb, yColCb) is set equal to the top-left sample of the collocated luma coding block specified by colCb relative to the top-left luma sample of the collocated picture specified by ColPic.
  • both components of mvLXCol are set equal to 0 and availableFlagLXCol is set equal to 0.
  • Output of this process is a predicted luma sample value predSampleLX L
  • variable picW is set equal to pic_width_in_luma_samples and the variable picH is set equal to pic_height_in_luma_samples.
  • the luma interpolation filter coefficients fb L [p] for each 1/16 fractional sample position p equal to xFrac L or yFrac L are specified in Table 8-10.
  • subpic_treated_as_pic_flag [SubPicIdx] is equal to 1, the following applies:
  • xInt i Clip3 (SubPicLeftBoundaryPos, SubPicRightBoundaryPos, xInt L + i) (8-460)
  • yInt i Clip3 (SubPicTopBoundaryPos, SubPicBotBoundaryPos, yInt L + i) (8-461)
  • variable cbWidth specifying the width of the current coding block in luma samples
  • variable cbHeight specifying the height of the current coding block in luma samples.
  • the availability flag availableFlagSbCol is derived as follows.
  • availableFlagSbCol is set equal to 0.
  • slice_temporal_mvp_enabled_flag is equal to 0.
  • the location (xCtb, yCtb) of the top-left sample of the luma coding tree block that contains the current coding block and the location (xCtr, yCtr) of the below-right center sample of the current luma coding block are derived as follows:
  • xCtr xCb + (cbWidth /2) (8-544)
  • the luma location (xColCtrCb, yColCtrCb) is set equal to the top-left sample of the collocated luma coding block covering the location given by (xCtr, yCtr) inside ColPic relative to the top-left luma sample of the collocated picture specified by ColPic.
  • the derivation process for subblock-based temporal merging base motion data as specified in clause 8.5.5.4 is invoked with the location (xCtb, yCtb) , the location (xColCtrCb, yColCtrCb) , the availability flag availableFlagA 1 , and the prediction list utilization flag predFlagLXA 1 , and the reference index refIdxLXA 1 , and the motion vector mvLXA 1 , with X being 0 and 1 as inputs and the motion vectors ctrMvLX, and the prediction list utilization flags ctrPredFlagLX of the collocated block, with X being 0 and 1, and the temporal motion vector tempMv as outputs.
  • variable availableFlagSbCol is derived as follows:
  • availableFlagSbCol is set equal to 1.
  • the luma location (xSb, ySb) specifying the top-left sample of the current coding subblock relative to the top-left luma sample of the current picture is derived as follows:
  • xSb xCb + xSbIdx *sbWidth + sbWidth /2 (8-551)
  • ySb yCb + ySbIdx *sbHeight + sbHeight /2 (8-552)
  • the location (xColSb, yColSb) of the collocated subblock inside ColPic is derived as follows.
  • subpic_treated_as_pic_flag [SubPicIdx] is equal to 1, the following applies:
  • variable tempMv is set as follows:
  • variable currPic specifies the current picture.
  • tempMv is set equal to mvL0A 1 :
  • tempMv is set equal to mvL1A 1 :
  • DiffPicOrderCnt (ColPic, RefPicList [1] [refIdxL1A 1 ] ) is equal to 0.
  • the location (xColCb, yColCb) of the collocated block inside ColPic is derived as follows.
  • subpic_treated_as_pic_flag [SubPicIdx] is equal to 1, the following applies:
  • xSbInt L , ySbInt L a luma location in full-sample units (xSbInt L , ySbInt L ) specifying the top-left sample of the bounding block for reference sample padding relative to the top-left luma sample of the reference picture
  • a luma location (xSb, ySb) specifying the top-left sample of the current subblock relative to the top-left luma sample of the current picture
  • Output of this process is a predicted luma sample value predSampleLX L
  • variable shift1 is set equal to Min (4, BitDepth Y -8)
  • variable shift2 is set equal to 6
  • variable shift3 is set equal to Max (2, 14-BitDepth Y ) .
  • variable picW is set equal to pic_width_in_luma_samples and the variable picH is set equal to pic_height_in_luma_samples.
  • the luma interpolation filter coefficients f L [p] for each 1/16 fractional sample position p equal to xFrac L or yFrac L are derived as follows:
  • the luma interpolation filter coefficients f L [p] are specified in Table 8-11 depending on hpelIfIdx.
  • subpic_treated_as_pic_flag [SubPicIdx] is equal to 1, the following applies:
  • xInt i Clip3 (SubPicLeftBoundaryPos, SubPicRightBoundaryPos, xInt L + i-3) (8-771)
  • yInt i Clip3 (SubPicTopBoundaryPos, SubPicBotBoundaryPos, yInt L + i-3) (8-772)
  • chroma location in full-sample units (xSbIntC, ySbIntC) specifying the top-left sample of the bounding block for reference sample padding relative to the top-left chroma sample of the reference picture
  • Output of this process is a predicted chroma sample value predSampleLX C
  • variable shift1 is set equal to Min (4, BitDepth C -8)
  • variable shift2 is set equal to 6
  • variable shift3 is set equal to Max (2, 14-BitDepth C ) .
  • the variable picW C is set equal to pic_width_in_luma_samples /SubWidthC and the variable picH C is set equal to pic_height_in_luma_samples /SubHeightC.
  • chroma interpolation filter coefficients f C [p] for each 1/32 fractional sample position p equal to xFrac C or yFrac C are specified in Table 8-13.
  • the variable xOffset is set equal to (sps_ref_wraparound_offset_minus1 + 1) *MinCbSizeY) /SubWidthC.
  • subpic_treated_as_pic_flag [SubPicIdx] is equal to 1, the following applies:
  • xInt i Clip3 (SubPicLeftBoundaryPos /SubWidthC, SubPicRightBoundaryPos /SubWidthC, xInt L + i) (8-785)
  • yInt i Clip3 (SubPicTopBoundaryPos /SubHeightC, SubPicBotBoundaryPos /SubHeightC, yInt L + i) (8-786)
  • an encoder-only temporal filter can be implemented.
  • the filtering is done at the encoder side as a pre-processing step.
  • Source pictures before and after the selected picture to encode are read and a block based motion compensation method relative to the selected picture is applied on those source pictures.
  • Samples in the selected picture are temporally filtered using sample values after motion compensation.
  • the overall filter strength is set depending on the temporal sub layer of the selected picture as well as the QP. Only pictures at temporal sub layers 0 and 1 are filtered and pictures of layer 0 are filter by a stronger filter than pictures of layer 1.
  • the per sample filter strength is adjusted depending on the difference between the sample value in the selected picture and the co-located samples in motion compensated pictures so that small differences between a motion compensated picture and the selected picture are filtered more strongly than larger differences.
  • a temporal filter is introduced directly after reading picture and before encoding. Below are the steps described in more detail.
  • the overall filter strength, s o is set according to the equation below for RA.
  • n is the number of pictures read.
  • Operation 3 Two pictures before and/or after the selected picture (referred to as original picture further down) are read. In the edge cases e.g. if is the first picture or close to the last picture, only the available pictures are read.
  • Operation 4 Motion of the read pictures before and after, relative to the original picture is estimated per 8x8 picture block.
  • Ahierarchical motion estimation scheme is used and the layers L0, L1 and L2, are illustrated in FIG. 2.
  • Subsampled pictures are generated by averaging each 2x2 block for all read pictures and the original picture, e.g. L1 in FIG. 1.
  • L2 is derived from L1 using the same subsampling method.
  • FIG. 2 shows examples of different layers of the hierarchical motion estimation.
  • L0 is the original resolution.
  • L1 is a subsampled version of L0.
  • L2 is a subsampled version of L1.
  • motion estimation is done for each 16x16 block in L2.
  • the squared difference is calculated for each selected motion vector and the motion vector corresponding to the smallest difference is selected.
  • the selected motion vector is then used as initial value when estimating the motion in L1.
  • the same is done for estimating motion in L0.
  • subpixel motion is estimated for each 8x8 block by using an interpolation filter on L0.
  • VTM 6-tap interpolation filter can used:
  • Operation 5 Motion compensation is applied on the pictures before and after the original picture according to the best matching motion for each block, e.g., so that the sample coordinates of the original picture in each block have the best matching coordinates in the referenced pictures.
  • Operation 6 The samples of the processed one by one for the luma and chroma channels as described in the following steps.
  • Operation 7 The new sample value, I n , is calculated using the following formula.
  • I o is the sample value of the original sample
  • I r (i) is the intensity of the corresponding sample of motion compensated picture i
  • w r (i, a) is the weight of motion compensated picture i when the number of available motion compensated pictures is a.
  • the weights, w r (i, a) is defined as follows:
  • weights, w r (i, a) is defined as follows:
  • Operation 8 The filter is applied for the current sample.
  • the resulting sample value is stored separately.
  • Operation 9 The filtered picture is encoded.
  • A. TMVP in the affine constructed candidates may fetch a MV in the collocated picture out of the range of the current sub-picture.
  • B When deriving gradients in Bi-Directional Optical Flow (BDOF) and Prediction Refinement Optical Flow (PROF) , two extended rows and two extended columns of integer reference samples are required to be fetched. These reference samples may be out of the range of the current sub-picture.
  • BDOF Bi-Directional Optical Flow
  • PROF Prediction Refinement Optical Flow
  • the accessed reconstructed luma samples may be out of the range of the range of the current sub-picture.
  • the neighboring block may be out of the range of the current sub-picture, when deriving the luma intra prediction mode, reference samples for intra prediction, reference samples for CCLM, neighboring block availability for spatial neighboring candidates for merge/AMVP/CIIP/IBC/LMCS, quantization parameters, CABAC initialization process, ctxInc derivation using left and above syntax elements, and ctxIncfor the syntax element mtt_split_cu_vertical_flag.
  • the representation of subpicture may lead to subpicture with incomplete CTUs.
  • the CTU partitions and CU splitting process may need to consider incomplete CTUs.
  • the signaled syntax elements related to sub-picture may be arbitrarily large, which may cause an overflow problem.
  • the subpicture and subpicture grid is defined in units of 4 samples. And the length of syntax element is dependent on the picture height divided by 4. However, since the current pic_width_in_luma_samples and pic_height_in_luma_samples shall be an integer multiple of Max (8, MinCbSizeY) , the subpicture grid may need to be defined in units of 8 samples.
  • the SPS syntax, pic_width_max_in_luma_samples and pic_height_max_in_luma_samples may need to be restricted to be no smaller than 8.
  • temporal filter is used to represent filters that require samples in other pictures.
  • Max (x, y) returns the larger one of x and y.
  • Min (x, y) returns the smaller one of x and y.
  • the position (named position RB) at which a temporal MV predictor is fetched in a picture to generate affine motion candidates must be in a required sub-picture, assuming the top-left corner coordinate of the required sub-picture is (xTL, yTL) and bottom-right coordinate of the required sub-picture is (xBR, yBR) .
  • the required sub-picture is the sub-picture covering the current block.
  • the temporal MV predictor is treated as unavailable.
  • position RB is out of the required sub-picture if x > xBR.
  • position RB is out of the required sub-picture if y > yBR.
  • position RB is out of the required sub-picture if x ⁇ xTL.
  • position RB is out of the required sub-picture if y ⁇ yTL.
  • position RB if outside of the required sub-picture, a replacement of RB is utilized.
  • the replacement position shall be in the required sub-picture.
  • position RB is clipped to be in the required sub-picture.
  • the position RB may be the bottom right position inside the corresponding block of current block in the collocated picture.
  • the proposed method may be utilized in other coding tools which require to access motion information from a picture different than the current picture.
  • whether the above methods are applied may depend on one or more syntax elements signaled in VPS/DPS/SPS/PPS/APS/slice header/tile group header.
  • the syntax element may be subpic_treated_as_pic_flag [SubPicIdx] , where SubPicIdx is the sub-picture index of sub-picture covering the current block.
  • the position (named position S) at which an integer sample is fetched in a reference not used in the interpolation process must be in a required sub-picture, assuming the top-left corner coordinate of the required sub-picture is (xTL, yTL) and the bottom-right coordinate of the required sub-picture is (xBR, yBR) .
  • the required sub-picture is the sub-picture covering the current block.
  • position S is out of the required sub-picture if x > xBR.
  • position S is out of the required sub-picture if y > yBR.
  • position S is out of the required sub-picture if x ⁇ xTL.
  • position S is out of the required sub-picture if y ⁇ yTL.
  • position S is clipped to be in the required sub-picture.
  • whether position S must be in a required sub-picture may depend on one or more syntax elements signaled in VPS/DPS/SPS/PPS/APS/slice header/tile group header.
  • the syntax element may be subpic_treated_as_pic_flag [SubPicIdx] , where SubPicIdx is the sub-picture index of sub-picture covering the current block.
  • the fetched integer sample is used to generate gradients in BDOF and/or PORF.
  • the position (named position R) at which the reconstructed luma sample value is fetched may be in a required sub-picture, assuming the top-left corner coordinate of the required sub-picture is (xTL, yTL) and the bottom-right coordinate of the required sub-picture is (xBR, yBR) .
  • the required sub-picture is the sub-picture covering the current block.
  • position R is out of the required sub-picture if x > xBR.
  • position R is out of the required sub-picture if y > yBR.
  • position R is out of the required sub-picture if x ⁇ xTL.
  • position R is out of the required sub-picture if y ⁇ yTL.
  • position R is clipped to be in the required sub-picture.
  • whether position R must be in a required sub-picture may depend on one or more syntax elements signaled in VPS/DPS/SPS/PPS/APS/slice header/tile group header.
  • the syntax element may be subpic_treated_as_pic_flag [SubPicIdx] , where SubPicIdx is the sub-picture index of sub-picture covering the current block.
  • the fetched luma sample is used to derive the scaling factor for the chroma component (s) in LMCS.
  • the position (named position N) at which the picture boundary check for BT/TT/QT splitting, BT/TT/QT depth derivation, and/or the signaling of CU split flag must be in a required sub-picture, assuming the top-left corner coordinate of the required sub-picture is (xTL, yTL) and the bottom-right coordinate of the required sub-picture is (xBR, yBR) .
  • the required sub-picture is the sub-picture covering the current block.
  • position N is out of the required sub-picture if x > xBR.
  • position N is out of the required sub-picture if y > yBR.
  • position N is out of the required sub-picture if x ⁇ xTL.
  • position N is out of the required sub-picture if y ⁇ yTL.
  • position N is clipped to be in the required sub-picture.
  • whether position N must be in a required sub-picture (e.g. to do as claimed in 5.
  • a and/or 5. b) may depend on one or more syntax elements signaled in VPS/DPS/SPS/PPS/APS/slice header/tile group header.
  • the syntax element may be subpic_treated_as_pic_flag [SubPicIdx] , where SubPicIdx is the sub-picture index of sub-picture covering the current block.
  • History -based Motion Vector Prediction (HMVP) table may be reset before decoding a new sub-picture in one picture.
  • the HMVP table used for IBC coding may be reset
  • the HMVP table used for inter coding may be reset
  • the HMVP table used for intra coding may be reset
  • the width of each element of the subpicture identifier grid in units of N samples.
  • the height of each element of the subpicture identifier grid in units of N samples.
  • N is set to the width and/or height of CTU.
  • the picture width may need to be restricted to be no smaller than 8.
  • the picture height may need to be restricted to be no smaller than 8.
  • a conformance bitstream shall satisfy that sub-picture coding and Adaptive resolution conversion (ARC) /Dynamic resolution conversion (DRC) /Reference picture resampling (RPR) are disallowed to be enabled for one video unit (e.g., sequence) .
  • ARC sub-picture coding and Adaptive resolution conversion
  • DRC Dynamic resolution conversion
  • RPR Reference picture resampling
  • signaling of enabling sub-picture coding may be under the conditions of disallowing ARC/DRC/RPR.
  • subpicture_present_flag 1
  • pic_width_in_luma_samples for all pictures for which this SPS is active is equal to max_width_in_luma_samples.
  • sub-picture coding and ARC/DRC/RPR may be both enabled for one video unit (e.g., sequence) .
  • a conformance bitstream shall satisfy that the donwsampled sub-picture due to ARC/DRC/RPR shall still be in the form of K CTUs in width and M CTUs in height wherein K and M are both integers.
  • a conformance bitstream shall satisfy that for sub-pictures not located at picture boundaries (e.g., right boundary and/or bottom boundary) , the donwsampled sub-picture due to ARC/DRC/RPR shall still be in the form of K CTUs in width and M CTUs in height wherein K and M are both integers.
  • CTU sizes may be adaptively changed based on the picture resolution.
  • a max CTU size may be signaled in SPS. For each picture with less resolution, the CTU size may be changed accordingly based on the reduced resolution.
  • CTU size may be signaled in SPS and PPS, and/or sub-picture level.
  • syntax element subpic_grid_col_width_minus1 and subpic_grid_row_height_minus1 may be constrained.
  • subpic_grid_col_width_minus1 must be no larger (or must be smaller) than T1.
  • subpic_grid_row_height_minus1 must be no larger (or must be smaller) than T2.
  • subpic_grid_col_width_minus1 and/or subpic_grid_row_height_minus1 must follow the constraint such as bullet 3. a or 3. b.
  • T1 in 3. a and/or T2 in 3. b may depend on profiles/levels/tiers of a video coding standard.
  • T1 in 3. a may depend on the picture width.
  • T1 is equal to pic_width_max_in_luma_samples/4 or pic_width_max_in_luma_samples/4+Off. Off may be 1, 2, -1, -2, etc.
  • T2 in 3. b may depend on the picture width.
  • T2 is equal to pic_height_max_in_luma_samples/4 or pic_height_max_in_luma_samples /4-1+Off. Off may be 1, 2, -1, -2, etc.
  • a CTU cannot be covered by more than one sub-picture.
  • the unit of subpic_grid_col_width_minus1 may be the CTU width (such as 32, 64, 128) , instead of 4 as in VVC.
  • the sub-picture grid width should be (subpic_grid_col_width_minus1+1) *CTU width.
  • the unit of subpic_grid_col_height_minus1 may be the CTU height (such as 32, 64, 128) , instead of 4 as in VVC.
  • the sub-picture grid height should be (subpic_grid_col_height_minus1+1) *CTU height.
  • the constraint in a conformance bit-stream, the constraint must be satisfied if the sub-picture approach is applied.
  • the constraint in a conformance bit-stream, the constraint must be satisfied if the sub-picture approach is applied.
  • Sub-picture may only contain rectangular slices. For example, in a conformance bit-stream, the constraint must be satisfied if the sub-picture approach is applied.
  • the constraint in a conformance bit-stream, the constraint must be satisfied if the sub-picture approach is applied.
  • two sub-pictures may be overlapped with each other.
  • the constraint in a conformance bit-stream, the constraint must be satisfied if the sub-picture approach is applied.
  • one sample may not belong to any sub-picture.
  • one sample may belong to more than one sub-pictures.
  • the width and height of a sub-picture defined in the SPS mapped to a resolution presented in the same sequence should be integer multiple times of N (such as 8, 16, 32) luma samples.
  • sub-pictures may be defined for certain layer and may be mapped to other layers.
  • sub-pictures may be defined for the layer with the highest resolution in the sequence.
  • sub-pictures may be defined for the layer with the lowest resolution in the sequence.
  • Which layer the sub-pictures are defined for may be signaled in SPS/VPS/PPS/slice header.
  • all resolutions may be integer multiple of a given resolution.
  • the width and/or height of a sub-picture defined in the SPS may be integer multiple times (e.g., M) of the CTU size.
  • sub-pictures and different resolutions in a sequence may not be allowed simultaneously.
  • Sub-pictures may only apply to a certain layer (s)
  • sub-pictures defined in a SPS may only apply to the layer with the highest resolution in a sequence.
  • sub-pictures defined in a SPS may only apply to the layer with the lowest temporal id in a sequence.
  • Which layer (s) that sub-pictures may be applied to may be indicated by one or multiple syntax elements in SPS/VPS/PPS.
  • Which layer (s) that sub-picture cannot be applied to may be indicated by one or multiple syntax elements in SPS/VPS/PPS.
  • the position and/or dimensions of a sub-picture may be signaled without using subpic_grid_idx.
  • the top-left position of a sub-picture may be signaled.
  • the bottom-right position of a sub-picture may be signaled.
  • the width of sub-picture may be signaled.
  • the height of a sub-picture may be signaled.
  • temporal filter when performing the temporal filtering of a sample, only samples within the same sub-picture that the current sample belongs to may be used. The required samples may be in the same picture that the current sample belongs to or in other pictures.
  • whether to and/or how to apply a partitioning method may depend on whether the current block (or partition) crosses one or multiple boundary of a sub-picture.
  • the picture boundary handling method for partitioning in VVC may also be applied when a picture boundary is replaced by a sub-picture boundary.
  • whether to parse a syntax element (e.g. a flag) which represents a partitioning method (such as QT, horizontal BT, vertical BT, horizontal TT, vertical TT, or not split, etc. ) may depend on whether the current block (or partition) crosses one or multiple boundary of a sub-picture.
  • a sample in the second set is not in any sub-pictures.
  • the second set may be encoded/decoded based on the information of the first set.
  • a default value may be utilized to mark whether a sample/MxK sub-region belonging to the second set.
  • the default value may be set equal to (max_subpics_minus1 +K) wherein K is an integer greater than 1.
  • the default value may be assigned to subpic_grid_idx [i] [j] to indicate that grid belongs to the second set.
  • subpic_grid_idx [i] [j] cannot be larger than max_subpics_minus1.
  • the codeword to code subpic_grid_idx [i] [j] cannot be larger than max_subpics_minus1.
  • any integer number from 0 to max_subpics_minus1 must be equal to at least one subpic_grid_idx [i] [j] .
  • IBC virtual buffer may be reset before decoding a new sub-picture in one picture.
  • all the samples in the IBC virtual buffer may be reset to -1.
  • Palette entry list may be reset before decoding a new sub-picture in one picture.
  • PredictorPaletteSize may be set equal to 0 before decoding a new sub-picture in one picture.
  • the newly added texts are bold italicized and the deleted texts are marked by “ [ [] ] ” .
  • Embodiment 1 sub-picture constraint on affine constructed merge candidates
  • a luma location (xCb, yCb) specifying the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture
  • the fourth (collocated bottom-right) control point motion vector cpMvLXCorner [3] , reference index refIdxLXCorner [3] , prediction list utilization flag predFlagLXCorner [3] and the availability flag availableFlagCorner [3] with X being 0 and 1 are derived as follows:
  • the reference indices for the temporal merging candidate, refIdxLXCorner [3] , with X being 0 or 1, are set equal to 0.
  • slice_temporal_mvp_enabled_flag is equal to 0
  • both components of mvLXCol are set equal to 0 and availableFlagLXCol is set equal to 0.
  • xColBr xCb + cbWidth (8-601)
  • SubPicRightBoundaryPos pic_width_in_luma_samples-1
  • botBoundaryPos subpic_treated_as_pic_flag [SubPicIdx] ?
  • SubPicBotBoundaryPos pic_height_in_luma_samples-1
  • the variable colCb specifies the luma coding block covering the modified location given by ( (xColBr >> 3) ⁇ 3, (yColBr >> 3) ⁇ 3) inside the collocated picture specified by ColPic.
  • the luma location (xColCb, yColCb) is set equal to the top-left sample of the collocated luma coding block specified by colCb relative to the top-left luma sample of the collocated picture specified by ColPic.
  • both components of mvLXCol are set equal to 0 and availableFlagLXCol is set equal to 0.
  • Embodiment 2 sub-picture constraint on affine constructed merge candidates
  • a luma location (xCb, yCb) specifying the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture
  • the fourth (collocated bottom-right) control point motion vector cpMvLXCorner [3] , reference index refIdxLXCorner [3] , prediction list utilization flag predFlagLXCorner [3] and the availability flag availableFlagCorner [3] with X being 0 and 1 are derived as follows:
  • the reference indices for the temporal merging candidate, refIdxLXCorner [3] , with X being 0 or 1, are set equal to 0.
  • slice_temporal_mvp_enabled_flag is equal to 0
  • both components of mvLXCol are set equal to 0 and availableFlagLXCol is set equal to 0.
  • xColBr xCb + cbWidth (8-601)
  • SubPicRightBoundaryPos pic_width_in_luma_samples-1
  • botBoundaryPos subpic_treated_as_pic_flag [SubPicIdx] ?
  • SubPicBotBoundaryPos pic_height_in_luma_samples –1
  • xColBr Min (r ightBoundaryPos , xColBr)
  • yColBr Min (botBoundaryPos , yColBr)
  • the variable colCb specifies the luma coding block covering the modified location given by ( (xColBr >> 3) ⁇ 3, (yColBr >> 3) ⁇ 3) inside the collocated picture specified by ColPic.
  • the luma location (xColCb, yColCb) is set equal to the top-left sample of the collocated luma coding block specified by colCb relative to the top-left luma sample of the collocated picture specified by ColPic.
  • both components of mvLXCol are set equal to 0 and availableFlagLXCol is set equal to 0.
  • Embodiment 3 Fetching integer samples under the sub-picture constraint
  • Output of this process is a predicted luma sample value predSampleLX L
  • variable shift is set equal to Max (2, 14-BitDepth Y ) .
  • variable picW is set equal to pic_width_in_luma_samples and the variable picH is set equal to pic_height_in_luma_samples.
  • subpic_treated_as_pic_flag [SubPicIdx] is equal to 1, the following applies:
  • xInt Clip3 (SubPicLeftBoundaryPos, SubPicRightBoundaryPos, xInt)
  • yInt Clip3 (SubPicTopBoundaryPos, SubPicBotBoundaryPos, yInt)
  • the predicted luma sample value predSampleLX L is derived as follows:
  • predSampleLX L refPicLX L [xInt] [yInt] ⁇ shift3 (8-784)
  • Embodiment 4 deriving the variable invAvgLuma in chroma residual scaling of LMCS
  • nCurrSw specifying the chroma transform block width
  • an (nCurrSw) x (nCurrSh) array predSamples specifying the chroma prediction samples of the current block
  • Output of this process is a reconstructed chroma picture sample array recSamples.
  • variable sizeY is set equal to Min (CtbSizeY, 64) .
  • variable currPic specifies the array of reconstructed luma samples in the current picture.
  • variable invAvgLuma is derived as follows:
  • variablerightBoundaryPos and botBoundaryPos are derived as follows:
  • SubPicRightBoundaryPos pic_width_in_luma_samples-1
  • botBoundaryPos subpic_treated_as_pic_flag [SubPicIdx] ?
  • SubPicBotBoundaryPos pic_height_in_luma_samples–1
  • variable invAvgLuma is derived as follows:
  • subpic_grid_col_width_minus1 plus 1 specifies the width of each element of the subpicture identifier grid in units of 4 N samples.
  • the length of the syntax element is Ceil (Log2 (pic_width_max_in_luma_samples / 4 N ) ) ) bits.
  • NumSubPicGridCols (pic_width_max_in_luma_samples + subpic_grid_col_width_minus1 * [ [4+ 3] ] N +N-1 ) / (subpic_grid_col_width_minus1 * [ [4+ 3] ] N + N-1 ) (7-5)
  • subpic_grid_row_height_minus1 plus 1 specifies the height of each element of the subpicture identifier grid in units of 4 samples.
  • the length of the syntax element is Ceil (Log2 (pic_height_max_in_luma_samples / 4 N ) ) bits.
  • NumSubPicGridRows (pic_height_max_in_luma_samples + subpic_grid_row_height_minus1 * 4 N + N-1 ) / (subpic_grid_row_height_minus1 * [ [4+ 3] N + N-1 )
  • SubPicIdx SubPicLeftBoundaryPos
  • SubPicTopBoundaryPos SubPicRightBoundaryPos
  • SubPicBotBoundaryPos are derived as follows:
  • Embodiment 6 restrict the picture width and the picture height to be equal or larger than 8
  • pic_width_max_in_luma_samples specifies the maximum width, in units of luma samples, of each decoded picture referring to the SPS.
  • pic_width_max_in_luma_samples shall not be equal to 0 and shall be an integer multiple of [ [MinCbSizeY] ] Max (8, MinCbSizeY) .
  • pic_height_max_in_luma_samples specifies the maximum height, in units of luma samples, of each decoded picture referring to the SPS.
  • pic_height_max_in_luma_samples shall not be equal to 0 and shall be an integer multiple of [ [MinCbSizeY] ] Max (8, MinCbSizeY) .
  • Embodiment 7 subpicture boundary check for BT/TT/QT splitting, BT/TT/QT depth derivation, and/or the signaling of CU split flag
  • variable allowBtSplit is derived as follows:
  • allowBtSplit is set equal to FALSE
  • allowBtSplit is set equal to FALSE
  • allowBtSplit is set equal to FALSE
  • allowBtSplit is set equal to FALSE
  • allowBtSplit is set equal to FALSE
  • cbHeight is less than or equal to [ [pic_height_in_luma_samples] ] subpic_treated_as_pic_flag [SubPicIdx] ? SubPicBotBoundaryPos + 1: pic_height_in_luma_samples.
  • variable allowTtSplit is derived as follows:
  • allowTtSplit is set equal to FALSE:
  • treeType is equal to DUAL_TREE_CHROMA and (cbWidth /SubWidthC) * (cbHeight /SubHeightC) is less than or equal to 32
  • treeType is equal to DUAL_TREE_CHROMA and modeType is equal to MODE_TYPE_INTRA
  • allowTtSplit is set equal to TRUE.
  • FIG. 3 is a block diagram of a video processing apparatus 300.
  • the apparatus 300 may be used to implement one or more of the methods described herein.
  • the apparatus 300 may be embodied in a smartphone, tablet, computer, Internet of Things (IoT) receiver, and so on.
  • the apparatus 300 may include one or more processors 302, one or more memories 304 and video processing hardware 306.
  • the processor (s) 302 may be configured to implement one or more methods described in the present document.
  • the memory (memories) 304 may be used for storing data and code used for implementing the methods and techniques described herein.
  • the video processing hardware 306 may be used to implement, in hardware circuitry, some techniques described in the present document.
  • FIG. 4 is a flowchart for a method 400 of processing a video.
  • the method 1800 includes determining (402) , for a video block in a first video region of a video, whether a position at which a temporal motion vector predictor is determined for a conversion between the video block and a bitstream representation of the current video block using an affine mode is within a second video region, and performing (404) the conversion based on the determining.
  • a method of video processing comprising: determining, for a video block in a first video region of a video, whether a position at which a temporal motion vector predictor is determined for a conversion between the video block and a bitstream representation of the current video block using an affine mode is within a second video region; and performing the conversion based on the determining.
  • a method of video processing comprising: determining, for a video block in a first video region of a video, whether a position at which an integer sample in a reference picture is fetched for a conversion between the video block and a bitstream representation of the current video block is within a second video region, wherein the reference picture is not used in an interpolation process during the conversion; and performing the conversion based on the determining.
  • a method of video processing comprising: determining, for a video block in a first video region of a video, whether a position at which a reconstructed luma sample value is fetched for a conversion between the video block and a bitstream representation of the current video block is within a second video region; and performing the conversion based on the determining.
  • a method of video processing comprising: determining, for a video block in a first video region of a video, whether a position at which a check regarding splitting, depth derivation or split flag signaling for the video block is performed during a conversion between the video block and a bitstream representation of the current video block is within a second video region; and performing the conversion based on the determining.
  • a method of video processing comprising: performing a conversion between a video comprising one or more video pictures comprising one or more video blocks, and a coded representation of the video, wherein the coded representation complies with a coding syntax requirement that the conversion is not to use sub-picture coding/decoding and a dynamic resolution conversion coding/decoding tool or a reference picture resampling tool within a video unit.
  • a method of video processing comprising: performing a conversion between a video comprising one or more video pictures comprising one or more video blocks, and a coded representation of the video, wherein the coded representation complies with a coding syntax requirement that a first syntax element subpic_grid_idx [i] [j] is not larger than a second syntax element max_subpics_minus1.
  • a video decoding apparatus comprising a processor configured to implement a method recited in one or more of solutions 1 to 31.
  • a video encoding apparatus comprising a processor configured to implement a method recited in one or more of solutions 1 to 31.
  • a computer program product having computer code stored thereon, the code, when executed by a processor, causes the processor to implement a method recited in any of solutions 1 to 31.
  • FIG. 5 is a block diagram showing an example video processing system 500 in which various techniques disclosed herein may be implemented.
  • the system 500 may include input 502 for receiving video content.
  • the video content may be received in a raw or uncompressed format, e.g., 8 or 10 bit multi-component pixel values, or may be in a compressed or encoded format.
  • the input 502 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interface include wired interfaces such as Ethernet, passive optical network (PON) , etc. and wireless interfaces such as Wi-Fi or cellular interfaces.
  • PON passive optical network
  • the system 500 may include a coding component 504 that may implement the various coding or encoding methods described in the present document.
  • the coding component 504 may reduce the average bitrate of video from the input 502 to the output of the coding component 504 to produce a coded representation of the video.
  • the coding techniques are therefore sometimes called video compression or video transcoding techniques.
  • the output of the coding component 504 may be either stored, or transmitted via a communication connected, as represented by the component 506.
  • the stored or communicated bitstream (or coded) representation of the video received at the input 502 may be used by the component 508 for generating pixel values or displayable video that is sent to a display interface 510.
  • the process of generating user-viewable video from the bitstream representation is sometimes called video decompression.
  • certain video processing operations are referred to as “coding” operations or tools, it will be appreciated that the coding tools or operations are used at an encoder and corresponding decoding tools or operations that reverse the results of the coding will be performed by
  • peripheral bus interface or a display interface may include universal serial bus (USB) or high definition multimedia interface (HDMI) or Displayport, and so on.
  • storage interfaces include SATA (serial advanced technology attachment) , PCI, IDE interface, and the like.
  • FIG. 6 is a flowchart representation of a method 600 for video processing in accordance with the present technology.
  • the method 600 includes, at operation 610, determining, for a conversion between a current block of a first picture of a video and a bitstream representation of the video, a motion candidate based on motion information from a second picture according to a rule.
  • the rule specifies that a position from which the motion information is accessed is constrained to be within a specific subpicture of the second picture.
  • the method 600 also includes at operation 620, performing the conversion based on the determining.
  • the motion information comprises a motion vector and the motion candidate comprises an affine motion candidate.
  • the rule specifies that the position is used as a replacement for a first position in case the first position is outside of the specific subpicture.
  • the position is at a bottom-right corner of a block in the video picture that corresponds to the current block.
  • FIG. 7 is a flowchart representation of a method 700 for video processing in accordance with the present technology.
  • the method 700 includes, at operation 710, determining, for a conversion of a current block of a first picture of a video and a bitstream representation of the video, an integer sample from a second picture according to a rule.
  • the second picture comprises a reference picture that is not used in an interpolation process.
  • the rule specifies that a position from which the integer sample is accessed is constrained to be within a specific subpicture of the second picture.
  • the method 700 also includes, at operation 720, performing the conversion based on the determining.
  • the integer sample is used to generate one or more gradients in a Bi-Directional Optical Flow or a Prediction Refinement Optical Flow process.
  • FIG. 8 is a flowchart representation of a method 800 for video processing in accordance with the present technology.
  • the method 800 includes, at operation 810, determining, for a conversion of a current block of a video and a bitstream representation of the video, a position at which a reconstructed luma sample is accessed according to a rule.
  • the rule specifies that the position is constrained to be within a specific subpicture of a video picture.
  • the method 800 also includes, at operation 820, performing the conversion based on the determining.
  • the reconstructed luma sample is accessed to derive a scaling factor for a chroma component in a luma mapping chroma scaling process.
  • FIG. 9 is a flowchart representation of a method 900 for video processing in accordance with the present technology.
  • the method 900 includes, at operation 910, determining, for a conversion of a current block of a video and a bitstream representation of the video, a position at which a picture boundary check is performed according to a rule.
  • the rule specifies that the position is constrained to be within a specific subpicture of a video picture.
  • the method 900 also includes, at operation 920, performing the conversion based on the determining.
  • the picture boundary check is performed for at least one of: (1) a splitting of a binary tree, a tertiary tree, or a quad tree, (2) a depth derivation for a binary tree, a tertiary tree, or a quad tree, (3) or a signaling of a split flag for the current block.
  • the specific subpicture is a collocated subpicture that covers the current block.
  • the rule specifies that information at the position treated as unavailable in case the position is outside of a specific subpicture.
  • the position is represented as (x, y)
  • a top-left corner of the specific subpicture is represented as (xTL, yTL)
  • a bottom-right corner of the specific subpicture is represented as (xBR, yBR) .
  • the position is outside of the specific subpicture in case x > xBR, y > yBR, x ⁇ xTL, or y ⁇ yTL.
  • the rule specifies that the position is clipped to be within a specific subpicture of the video picture.
  • the position is represented as (x, y) , a top-left corner of the specific subpicture is represented as (xTL, yTL) , and a bottom-right corner of the specific subpicture is represented as (xBR, yBR) .
  • x is clipped to be a smaller value of x and xBR.
  • y is clipped to be a smaller value of y and yBR.
  • x is clipped to be a larger value of x and xTL.
  • y is clipped to be a larger value of y and yTL.
  • whether the rule is applicable is based on a syntax element in the bitstream representation.
  • the syntax element is signaled in a video parameter set, a dependency parameter set, a slice parameter set, a picture parameter set, an active parameter set, a slice header, or a tile group header.
  • the syntax element comprises subpic_treated_as_pic_flag [SubPicIdx] , where SubPicIdx is a subpicture index of the specific subpicture that covers the current block.
  • the conversion generates the current block from the bitstream representation. In some embodiments, the conversion generates the bitstream representation from the current block.
  • FIG. 10 is a flowchart representation of a method 1000 for video processing in accordance with the present technology.
  • the method 1000 includes, at operation 1010, resetting, after a conversion of a sub-picture of a video picture of a video and a bitstream representation of the video, a table of motion candidates derived based on past conversions.
  • the method 1000 also includes, at operation 1020, performing a conversion of a subsequent sub-picture of the video picture and the bitstream representation using the table after the resetting.
  • the table of motion candidates comprises motion candidates for an intra-block coding mode. In some embodiments, the table of motion candidates comprises motion candidates for an inter coding mode. In some embodiments, the table of motion candidates comprises motion candidates for an intra coding mode.
  • the conversion generates the sub-picture or the subsequent sub-picture from the bitstream representation. In some embodiments, the conversion generates the bitstream representation from the sub-picture or the subsequent sub-picture.
  • FIG. 11 is a flowchart representation of a method 1100 for video processing in accordance with the present technology.
  • the method 1100 includes, at operation 1110, performing a conversion between a video comprising a video picture that includes multiple sub-pictures and multiple video blocks and a coded representation of the video according to a rule.
  • the rule specifies that a boundary between any two sub-pictures is also a boundary between two video blocks.
  • a video block in the video picture is covered by a single subpicture of the video picture.
  • any position in the video picture is covered by at most one subpicture of the video picture. In some embodiments, a position in the video picture is not covered by any subpicture of the video picture. In some embodiments, the two sub-pictures of the video picture have no overlapping area.
  • a dimension of a subpicture of the video picture is determined based on a dimension of a video block.
  • a sub-picture comprises multiple elements.
  • a syntax element indicating a width of an element in the sub-picture is represented as N samples, and a width of the sub-picture is determined based on the N samples.
  • a width of the video block comprises N samples.
  • a sub-picture comprises multiple elements.
  • a syntax element indicating a height of an element in the sub-picture is represented as N samples, and a height of the sub-picture is determined based on the N samples.
  • a height of the video block comprises N samples.
  • the video block is a coding tree block (CTB) or a coding tree unit (CTU) .
  • a sub-picture has a rectangular shape.
  • the sub-picture comprises rectangular slices.
  • a sub-picture is applicable to only selected one or more layers of the video.
  • the sub-picture is defined in a sequence parameter set in the bitstream representation, and the sub-picture is applicable to a layer with a highest resolution in a corresponding sequence.
  • the sub-picture is defined in a sequence parameter set in the bitstream representation, and the sub-picture is applicable to a layer with a lowest resolution in a corresponding sequence.
  • the selected one or more layers to which the sub-picture is applicable are signaled in one or more syntax elements in the bitstream representation.
  • one or more layers to which the sub-picture is inapplicable are signaled in one or more syntax elements in the bitstream representation.
  • the one or more syntax elements are signaled in a sequence parameter set, a video parameter set, or a picture parameter set in the bitstream representation.
  • FIG. 12 is a flowchart representation of a method 1200 for video processing in accordance with the present technology.
  • the method 1200 includes, at operation 1210, performing a conversion between a video unit of a video and a coded representation of the video using at least a video picture, where only one of a sub-picture coding mode or a resolution-changing coding mode is enabled for the video unit.
  • the sub-picture coding mode is a mode in which the video picture is divided into multiple sub-pictures
  • the resolution-changing coding mode is a mode in which a resolution of the video picture is adjusted during the conversion.
  • the video picture comprises a current picture or a reference picture.
  • the resolution-changing coding mode comprises a Reference Picture Resampling (PRP) mode.
  • the resolution-changing coding mode comprises a Dynamic Resolution Conversion (DRC) mode.
  • the resolution-changing coding mode comprises an Adaptive Resolution Conversion (ARC) mode.
  • the video unit comprises a video sequence.
  • a syntax element is included in the coded representation to indicate that the sub-picture coding mode is enabled for the coding unit in case the resolution-changing coding mode is disallowed.
  • the resolution-changing coding mode is disallowed in case a syntax element is included in the coded representation to indicate that the sub-picture coding mode is enabled.
  • the syntax element comprises subpics_present_flag.
  • a width of the video picture is set to be equal to a maximum width allowed for video pictures in the video unit in case the syntax element indicates that the sub-picture coding mode is enabled.
  • FIG. 13 is a flowchart representation of a method 1300 for video processing in accordance with the present technology.
  • the method 1300 includes, at operation 1310, performing a conversion between a video unit of a video and a coded representation of the video using at least a video picture, where both a sub-picture coding mode and a resolution-changing coding mode are enabled for the video unit.
  • the sub-picture coding mode is a mode in which the video picture is divided into multiple sub-pictures
  • the resolution-changing coding mode is a mode in which a resolution of the video picture is adjusted during the conversion.
  • the video unit comprises a video sequence.
  • the resolution-changing coding mode comprises an Adaptive Resolution Conversion (ARC) mode, a Dynamic Resolution Conversion (DRC) mode, a Reference Picture Resampling (PRP) mode.
  • ARC Adaptive Resolution Conversion
  • DRC Dynamic Resolution Conversion
  • PRP Reference Picture Resampling
  • the video picture includes multiple video blocks, each having a dimension of W ⁇ H.
  • a sub-picture adjusted according to the resolution-changing coding mode has a width of K ⁇ W and a height of M ⁇ H, K and M being integers.
  • the sub-picture is not located at a boundary of the video picture.
  • the boundary comprises a right boundary or a bottom boundary.
  • the video picture includes multiple video blocks, and a dimension of individual video blocks is adjusted based on a resolution of the video picture.
  • the coded representation comprises a syntax element indicating a maximum dimension for a video block, and the dimension of an individual video block is adjusted based on the maximum dimension and the resolution of the video picture.
  • the dimension of the individual video blocks is signaled in a sequence parameter set, a picture parameter set, or at a sub-picture level in the coded representation.
  • FIG. 14 is a flowchart representation of a method 1400 for video processing in accordance with the present technology.
  • the method 1400 includes, at operation 1410, performing a conversion between a video comprising one or more video pictures and a coded representation of the video, where a dimension of an individual video picture is constrained to be greater than or equal to 8.
  • the dimension is a width of the individual video picture.
  • the dimension is a height of the individual video picture.
  • FIG. 15 is a flowchart representation of a method 1500 for video processing in accordance with the present technology.
  • the method 1500 includes, at operation 1510, performing a conversion between a video picture of a video and a coded representation of the video according to a rule.
  • the video picture comprises at least one sub-picture
  • the rule specifies that a characteristic of a sub-picture is represented as at lease one syntax element in the coded representation, the at least one syntax element being different than an index value of the sub-picture grid in the video picture.
  • the characteristic comprises a top-left position of the sub-picture. In some embodiments, the characteristic comprises a bottom-right position of the sub-picture. In some embodiments, the characteristic comprises a width of the sub-picture. In some embodiments, the characteristic comprises a height of the sub-picture. In some embodiments, the index value of the sub-picture in the video picture is smaller than a maximum number of subpictures in the video picture. In some embodiments, an integer value in a range of [0, the maximum number of subpictures-1] has a one-to-one correspondence with indices values of subpictures in the video picture.
  • FIG. 16 is a flowchart representation of a method 1600 for video processing in accordance with the present technology.
  • the method 1600 includes, at operation 1610, performing a conversion between a video picture of a video and a coded representation of the video according to a rule.
  • the video picture comprises multiple sub-pictures, each sub-picture comprising multiple elements.
  • the rule specifies that a dimension of individual elements in a sub-picture satisfies a constraint.
  • the constraint specifies that a width of an individual element is smaller than T1. In some embodiments, the constraint specifies that a height of an individual element is smaller than T2. In some embodiments, the video picture comprises multiple video blocks, and the constraint specifies that the sub-picture determined based on the dimension of the individual elements is covered by a current video block being processed during the conversion. In some embodiments, a sample outside of the sub-picture covered by the current video block is considered as unavailable for the conversion. In some embodiments, the constraint specifies that the dimension of the individual elements is determined based on a profile, a level, or a tier of a video coding standard.
  • the constraint specifies that the dimension of the individual elements is determined based on a width of the video picture.
  • a width of the individual elements is equal to pic_width_max_in_luma_samples/4+offset, where pic_width_max_in_luma_samples represents a maximum picture width in luma samples and offset is zero or a non-zero integer.
  • the constraint specifies that the dimension of the individual elements is determined based on a height of the video picture.
  • a height of the individual elements is equal to pic_height_max_in_luma_samples/4+offset, where pic_height_max_in_luma_samples represents a maximum picture height in luma samples and offset is zero or a non-zero integer.
  • the conversion generates video picture from the bitstream representation. In some embodiments, the conversion generates the bitstream representation from the video picture.
  • FIG. 17 is a flowchart representation of a method 1700 for video processing in accordance with the present technology.
  • the method 1700 includes, at operation 1710, performing a conversion between a video comprising a picture that includes multiple sub-pictures and a coded representation of the video using a coding mode according to a rule.
  • the rule specifies that certain stored information about a previous sub-picture is reset prior to processing each next sub-picture of the multiple sub-pictures.
  • the certain stored information comprises a virtual buffer used for an intra-block copy coding mode in which a current block in the video picture is coded with samples from the video picture. In some embodiments, the certain stored information comprises a list of entries used for a palette coding mode in which a current block in the video picture is coded using a palette of representative sample values.
  • FIG. 18 is a flowchart representation of a method 1800 for video processing in accordance with the present technology.
  • the method 1800 includes, at operation 1810, performing a temporal filtering operation in a conversion between a video and a coded representation of the video according to a rule.
  • the video comprises multiple video pictures, each comprising multiple sub-pictures.
  • the rule specifies that, for a temporal filtering a current sample in a current sub-picture of a video picture, only samples within the same current sub-picture or a sub-picture in a different video picture corresponding to the current sub-picture are available.
  • FIG. 19 is a flowchart representation of a method 1900 for video processing in accordance with the present technology.
  • the method 1900 includes, at operation 1910, determining, for a conversion between a block in a video picture of a video and a coded representation of the video, a manner of applying a partitioning method to the block based on whether the block crosses one or more sub-picture boundaries of the video picture.
  • the method 1900 also includes, at operation 1920, performing the conversion based on the determining.
  • the partitioning method comprising at least one of: a quad-tree partitioning method, a horizonal binary tree partitioning method, a vertical binary tree partitioning method, a horizontal tertiary tree partitioning method, a vertical tertiary tree partitioning method, or a no-splitting method.
  • the manner further specifies whether a boundary handling method is applicable to the block.
  • the method includes determining a manner of processing a syntax element in the coded representation indicating the partitioning method based on whether the block is located across the one or more boundaries of the sub-picture of the video picture.
  • FIG. 20 is a flowchart representation of a method 2000 for video processing in accordance with the present technology.
  • the method 2000 includes, at operation 2010, determining, for a conversion between a video picture of a video and a coded representation of the video, two sub-regions of the video picture.
  • a first sub-region comprises multiple sub-pictures of the video picture and a second sub-region comprises remaining samples in the video picture.
  • the method 2000 also includes, at operation 2020, performing the conversion based on the determining.
  • a sample in the second sub-region is not located in any sub-picture of the video picture.
  • the second sub-region is processed for the conversion based on information about the first sub-region.
  • a default value is used in the conversion to indicate whether a sample or an area of the video picture is located within the second sub-region.
  • the default value is set to (max_subpics_minus1 + K) , wherein max_subpics_minus1 indicates a maximum number of sub-pictures in the video picture, and K is an integer greater than 1.
  • the default value is assigned to each element in an array of index values representing the sub-pictures in the video picture.
  • the conversion generates the video from the coded representation. In some embodiments, the conversion generates the coded representation from the video.
  • Some embodiments of the disclosed technology include making a decision or determination to enable a video processing tool or mode.
  • the encoder when the video processing tool or mode is enabled, the encoder will use or implement the tool or mode in the processing of a block of video, but may not necessarily modify the resulting bitstream based on the usage of the tool or mode. That is, a conversion from the block of video to the bitstream representation of the video will use the video processing tool or mode when it is enabled based on the decision or determination.
  • the decoder when the video processing tool or mode is enabled, the decoder will process the bitstream with the knowledge that the bitstream has been modified based on the video processing tool or mode. That is, a conversion from the bitstream representation of the video to the block of video will be performed using the video processing tool or mode that was enabled based on the decision or determination.
  • Some embodiments of the disclosed technology include making a decision or determination to disable a video processing tool or mode.
  • the encoder will not use the tool or mode in the conversion of the block of video to the bitstream representation of the video.
  • the decoder will process the bitstream with the knowledge that the bitstream has not been modified using the video processing tool or mode that was enabled based on the decision or determination.
  • the disclosed and other solutions, examples, embodiments, modules and the functional operations described in this document can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this document and their structural equivalents, or in combinations of one or more of them.
  • the disclosed and other embodiments can be implemented as one or more computer program products, e.g., one or more modules of computer program instructions encoded on a computer readable medium for execution by, or to control the operation of, data processing apparatus.
  • the computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more them.
  • data processing apparatus encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers.
  • the apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.
  • a propagated signal is an artificially generated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus.
  • a computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.
  • a computer program does not necessarily correspond to a file in a file system.
  • a program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document) , in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code) .
  • a computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.
  • the processes and logic flows described in this document can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output.
  • the processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit) .
  • processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer.
  • a processor will receive instructions and data from a read only memory or a random-access memory or both.
  • the essential elements of a computer are a processor for performing instructions and one or more memory devices for storing instructions and data.
  • a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks.
  • mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks.
  • a computer need not have such devices.
  • Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks; and CD ROM and DVD-ROM disks.
  • semiconductor memory devices e.g., EPROM, EEPROM, and flash memory devices
  • magnetic disks e.g., internal hard disks or removable disks
  • magneto optical disks e.g., CD ROM and DVD-ROM disks.
  • the processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Television Signal Processing For Recording (AREA)

Abstract

A method of video processing includes performing a conversion between a video comprising a video picture that includes multiple sub-pictures and multiple video blocks, and a coded representation of the video according to a rule. The rule specifies that a boundary between any two sub-pictures is also a boundary between two video blocks.

Description

SUBPICTURE SIZE DEFINITION IN VIDEO PROCESSING
CROSS REFERENCE TO RELATED APPLICATIONS
Under the applicable patent law and/or rules pursuant to the Paris Convention, this application is made to timely claim the priority to and benefits of International Patent Application No. PCT/CN2019/100114, filed on August 10, 2019. For all purposes under the law, the entire disclosure of the aforementioned application is incorporated by reference as part of the disclosure of this application.
TECHNICAL FIELD
This document is related to video and image coding and decoding technologies.
BACKGROUND
Digital video accounts for the largest bandwidth use on the internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, it is expected that the bandwidth demand for digital video usage will continue to grow.
SUMMARY
The disclosed techniques may be used by video or image decoder or encoder embodiments for in which sub-picture based coding or decoding is performed.
In one example aspect a method of video processing is disclosed. The method includes determining, for a conversion between a current block of a first picture of a video and a bitstream representation of the video, a motion candidate based on motion information from a second picture according to a rule. The rule specifies that a position from which the motion information is accessed is constrained to be within a specific subpicture of the second picture. The method also includes performing the conversion based on the determining.
In another example aspect a method of video processing is disclosed. The method includes determining, for a conversion of a current block of a first picture of a video and a bitstream representation of the video, an integer sample from a second picture according to a rule. The second picture comprises a reference picture that is not used in an interpolation process. The rule specifies that a position from which the integer sample is accessed is constrained to be within a specific  subpicture of the second picture. The method also includes performing the conversion based on the determining.
In another example aspect a method of video processing is disclosed. The method includes determining, for a conversion of a current block of a video and a bitstream representation of the video, a position at which a reconstructed luma sample is accessed according to a rule. The rule specifies that the position is constrained to be within a specific subpicture of a video picture. The method also includes performing the conversion based on the determining.
In another example aspect a method of video processing is disclosed. The method includes determining, for a conversion of a current block of a video and a bitstream representation of the video, a position at which a picture boundary check is performed according to a rule. The rule specifies that the position is constrained to be within a specific subpicture of a video picture. The method also includes performing the conversion based on the determining.
In another example aspect a method of video processing is disclosed. The method includes resetting, after a conversion of a sub-picture of a video picture of a video and a bitstream representation of the video, a table of motion candidates derived based on past conversions and performing a conversion of a subsequent sub-picture of the video picture and the bitstream representation using the table after the resetting.
In another example aspect a method of video processing is disclosed. The method includes performing a conversion between a video comprising a video picture that includes multiple sub-pictures and multiple video blocks and a coded representation of the video according to a rule. The rule specifies that a boundary between any two sub-pictures is also a boundary between two video blocks. A video block in the video picture is covered by a single subpicture of the video picture.
In another example aspect a method of video processing is disclosed. The method includes performing a conversion between a video unit of a video and a coded representation of the video using at least a video picture, where only one of a sub-picture coding mode or a resolution-changing coding mode is enabled for the video unit. The sub-picture coding mode is a mode in which the video picture is divided into multiple sub-pictures, and the resolution-changing coding mode is a mode in which a resolution of the video picture is adjusted during the conversion.
In another example aspect a method of video processing is disclosed. The method includes performing a conversion between a video unit of a video and a coded representation of the video using at least a video picture, where both a sub-picture coding mode and a resolution-changing  coding mode are enabled for the video unit. The sub-picture coding mode is a mode in which the video picture is divided into multiple sub-pictures, and the resolution-changing coding mode is a mode in which a resolution of the video picture is adjusted during the conversion.
In another example aspect a method of video processing is disclosed. The method includes performing a conversion between a video comprising one or more video pictures and a coded representation of the video, where a dimension of an individual video picture is constrained to be greater than or equal to 8. In some embodiments, the dimension is a width of the individual video picture.
In another example aspect a method of video processing is disclosed. The method includes performing a conversion between a video picture of a video and a coded representation of the video according to a rule. The video picture comprises at least one sub-picture, and the rule specifies that a characteristic of a sub-picture is represented as at lease one syntax element in the coded representation, the at least one syntax element being different than an index value of the sub-picture grid in the video picture.
In another example aspect a method of video processing is disclosed. The method includes performing a conversion between a video picture of a video and a coded representation of the video according to a rule. The video picture comprises multiple sub-pictures, each sub-picture comprising multiple elements. The rule specifies that a dimension of individual elements in a sub-picture satisfies a constraint.
In another example aspect a method of video processing is disclosed. The method includes performing a conversion between a video comprising a picture that includes multiple sub-pictures and a coded representation of the video using a coding mode according to a rule. The rule specifies that certain stored information about a previous sub-picture is reset prior to processing each next sub-picture of the multiple sub-pictures.
In another example aspect a method of video processing is disclosed. The method includes performing a temporal filtering operation in a conversion between a video and a coded representation of the video according to a rule. The video comprises multiple video pictures, each comprising multiple sub-pictures. The rule specifies that, for a temporal filtering a current sample in a current sub-picture of a video picture, only samples within the same current sub-picture or a sub-picture in a different video picture corresponding to the current sub-picture are available.
In another example aspect a method of video processing is disclosed. The method includes  determining, for a conversion between a block in a video picture of a video and a coded representation of the video, a manner of applying a partitioning method to the block based on whether the block crosses one or more sub-picture boundaries of the video picture. The method also includes performing the conversion based on the determining.
In another example aspect a method of video processing is disclosed. The method includes for a conversion between a video picture of a video and a coded representation of the video, two sub-regions of the video picture. A first sub-region comprises multiple sub-pictures of the video picture and a second sub-region comprises remaining samples in the video picture. The method also includes performing the conversion based on the determining.
In another example aspect a method of video processing is disclosed. The method includes determining, for a video block in a first video region of a video, whether a position at which a temporal motion vector predictor is determined for a conversion between the video block and a bitstream representation of the current video block using an affine mode is within a second video region; and performing the conversion based on the determining.
In another example aspect, another method of video processing is disclosed. The method includes determining, for a video block in a first video region of a video, whether a position at which an integer sample in a reference picture is fetched for a conversion between the video block and a bitstream representation of the current video block is within a second video region, wherein the reference picture is not used in an interpolation process during the conversion; and performing the conversion based on the determining.
In another example aspect, another method of video processing is disclosed. The method includes determining, for a video block in a first video region of a video, whether a position at which a reconstructed luma sample value is fetched for a conversion between the video block and a bitstream representation of the current video block is within a second video region; and performing the conversion based on the determining.
In another example aspect, another method of video processing is disclosed. The method includes determining, for a video block in a first video region of a video, whether a position at which a check regarding splitting, depth derivation or split flag signaling for the video block is performed during a conversion between the video block and a bitstream representation of the current video block is within a second video region; and performing the conversion based on the determining.
In another example aspect, another method of video processing is disclosed. The method includes performing a conversion between a video comprising one or more video pictures comprising one or more video blocks, and a coded representation of the video, wherein the coded representation complies with a coding syntax requirement that the conversion is not to use sub-picture coding/decoding and a dynamic resolution conversion coding/decoding tool or a reference picture resampling tool within a video unit.
In another example aspect, another method of video processing is disclosed. The method includes performing a conversion between a video comprising one or more video pictures comprising one or more video blocks, and a coded representation of the video, wherein the coded representation complies with a coding syntax requirement that a first syntax element subpic_grid_idx [i] [j] is not larger than a second syntax element max_subpics_minus1.
In yet another example aspect, the above-described method may be implemented by a video encoder apparatus that comprises a processor.
In yet another example aspect, the above-described method may be implemented by a video decoder apparatus that comprises a processor.
In yet another example aspect, these methods may be embodied in the form of processor-executable instructions and stored on a computer-readable program medium.
These, and other, aspects are further described in the present document.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 shows an example of region constraint in temporal motion vector prediction (TMVP) and sub-block TMVP.
FIG. 2 shows an example of a hierarchical motion estimation scheme.
FIG. 3 is a block diagram of an example of a hardware platform used for implementing techniques described in the present document.
FIG. 4 is a flowchart for an example method of video processing.
FIG. 5 is a block diagram showing an example video processing system in which various techniques disclosed herein may be implemented.
FIG. 6 is a flowchart representation of a method for video processing in accordance with the present technology.
FIG. 7 is a flowchart representation of another method for video processing in accordance with the present technology.
FIG. 8 is a flowchart representation of another method for video processing in accordance with the present technology.
FIG. 9 is a flowchart representation of another method for video processing in accordance with the present technology.
FIG. 10 is a flowchart representation of another method for video processing in accordance with the present technology.
FIG. 11 is a flowchart representation of another method for video processing in accordance with the present technology.
FIG. 12 is a flowchart representation of another method for video processing in accordance with the present technology.
FIG. 13 is a flowchart representation of another method for video processing in accordance with the present technology.
FIG. 14 is a flowchart representation of another method for video processing in accordance with the present technology.
FIG. 15 is a flowchart representation of another method for video processing in accordance with the present technology.
FIG. 16 is a flowchart representation of another method for video processing in accordance with the present technology.
FIG. 17 is a flowchart representation of another method for video processing in accordance with the present technology.
FIG. 18 is a flowchart representation of another method for video processing in accordance with the present technology.
FIG. 19 is a flowchart representation of another method for video processing in accordance with the present technology.
FIG. 20 is a flowchart representation of yet another method for video processing in accordance with the present technology.
DETAILED DESCRIPTION
The present document provides various techniques that can be used by a decoder of image or video bitstreams to improve the quality of decompressed or decoded digital video or images. For brevity, the term “video” is used herein to include both a sequence of pictures (traditionally called video) and individual images. Furthermore, a video encoder may also implement these techniques during the process of encoding in order to reconstruct decoded frames used for further encoding.
Section headings are used in the present document for ease of understanding and do not limit the embodiments and techniques to the corresponding sections. As such, embodiments from one section can be combined with embodiments from other sections.
1. Summary
This document is related to video coding technologies. Specifically, it is related to palette coding with employing base colors based representation in video coding. It may be applied to the existing video coding standard like HEVC, or the standard (Versatile Video Coding) to be finalized. It may be also applicable to future video coding standards or video codec.
2. Initial Discussion
Video coding standards have evolved primarily through the development of the well-known ITU-T and ISO/IEC standards. The ITU-T produced H. 261 and H. 263, ISO/IEC produced MPEG-1 and MPEG-4 Visual, and the two organizations jointly produced the H. 262/MPEG-2 Video and H. 264/MPEG-4 Advanced Video Coding (AVC) and H. 265/HEVC standards [1, 2] . Since H. 262, the video coding standards are based on the hybrid video coding structure wherein temporal prediction plus transform coding are utilized. To explore the future video coding technologies beyond HEVC, Joint Video Exploration Team (JVET) was founded by VCEG and MPEG jointly in 2015. Since then, many new methods have been adopted by JVET and put into the reference software named Joint Exploration Model (JEM) . In April 2018, the Joint Video Expert Team (JVET) between VCEG (Q6/16) and ISO/IEC JTC1 SC29/WG11 (MPEG) was created to work on the VVC standard targeting at 50%bitrate reduction compared to HEVC.
2.1 The region constraint in TMVP and sub-block TMVP in VVC
FIG. 1 illustrates example region constraint in TMVP and sub-block TMVP. In TMVP and sub-block TMVP, it is constrained that a temporal MV can only be fetched from the collocated CTU plus a column of 4×4 blocks as shown in FIG. 1.
2.2 Example Sub-picture
In some embodiments, sub-picture-based coding techniques based on flexible tiling approach can be implemented. Summary of the sub-picture-based coding techniques includes the following:
(1) Pictures can be divided into sub-pictures.
(2) The indication of existence of sub-pictures is indicated in the SPS, along with other sequence-level information of sub-pictures.
(3) Whether a sub-picture is treated as a picture in the decoding process (excluding in-loop filtering operations) can be controlled by the bitstream.
(4) Whether in-loop filtering across sub-picture boundaries is disabled can be controlled by the bitstream for each sub-picture. The DBF, SAO, and ALF processes are updated for controlling of in-loop filtering operations across sub-picture boundaries.
(5) For simplicity, as a starting point, the sub-picture width, height, horizontal offset, and vertical offset are signalled in units of luma samples in SPS. Sub-picture boundaries are constrained to be slice boundaries.
(6) Treating a sub-picture as a picture in the decoding process (excluding in-loop filtering operations) is specified by slightly updating the coding_tree_unit () syntax, and updates to the following decoding processes:
- The derivation process for (advanced) temporal luma motion vector prediction
- The luma sample bilinear interpolation process
- The luma sample 8-tap interpolation filtering process
- The chroma sample interpolation process
(7) Sub-picture IDs are explicitly specified in the SPS and included in the tile group headers to enable extraction of sub-picture sequences without the need of changing VCL NAL units.
(8) Output sub-picture sets (OSPS) are proposed to specify normative extraction and conformance points for sub-pictures and sets thereof.
2.3 Example Sub-pictures in Versatile Video Coding
Sequence parameter set RBSP syntax
Figure PCTCN2020108159-appb-000001
Figure PCTCN2020108159-appb-000002
subpics_present_flag equal to 1 indicates that subpicture parameters are present in the present in the SPS RBSP syntax. subpics_present_flag equal to 0 indicates that subpicture parameters are not present in the present in the SPS RBSP syntax.
NOTE 2–When a bitstream is the result of a sub-bitstream extraction process and contains only a subset of the subpictures of the input bitstream to the sub-bitstream extraction process, it might be required to set the value of subpics_present_flag equal to 1 in the RBSP of the SPSs.
max_subpics_minus1 plus 1 specifies the maximum number of subpictures that may be present in the CVS.
max_subpics_minus1 shall be in the range of 0 to 254. The value of 255 is reserved for future use by ITU-T|ISO/IEC.
subpic_grid_col_width_minus1 plus 1 specifies the width of each element of the subpicture identifier grid in units of 4 samples. The length of the syntax element is Ceil (Log2 (pic_width_max_in_luma_samples /4) ) bits.
The variable NumSubPicGridCols is derived as follows:
NumSubPicGridCols = (pic_width_max_in_luma_samples + subpic_grid_col_width_minus1 *4 + 3) / (subpic_grid_col_width_minus1 *4 + 4)          (7-5)
subpic_grid_row_height_minus1 plus 1 specifies the height of each element of the subpicture identifier grid in units of 4 samples. The length of the syntax element is Ceil (Log2 (pic_height_max_in_luma_samples /4) ) bits.
The variable NumSubPicGridRows is derived as follows:
NumSubPicGridRows = (pic_height_max_in_luma_samples + subpic_grid_row_height_minus1 *4 + 3) / (subpic_grid_row_height_minus1 *4 + 4)          (7-6)
subpic_grid_idx [i] [j] specifies the subpicture index of the grid position (i, j) . The length of the syntax element is Ceil (Log2 (max_subpics_minus1 + 1) ) bits.
The variables SubPicTop [subpic_grid_idx [i] [j] ] , SubPicLeft [subpic_grid_idx [i] [j] ] ,
SubPicWidth [subpic_grid_idx [i] [j] ] , SubPicHeight [subpic_grid_idx [i] [j] ] , and NumSubPics are derived as follows:
Figure PCTCN2020108159-appb-000003
subpic_treated_as_pic_flag [i] equal to 1 specifies that the i-th subpicture of each coded picture in the CVS is treated as a picture in the decoding process excluding in-loop filtering operations. subpic_treated_as_pic_flag [i] equal to 0 specifies that the i-th subpicture of each coded picture in the CVS is not treated as a picture in the decoding process excluding in-loop filtering operations. When not present, the value of subpic_treated_as_pic_flag [i] is inferred to be equal to 0.
loop_filter_across_subpic_enabled_flag [i] equal to 1 specifies that in-loop filtering operations may be performed across the boundaries of the i-th subpicture in each coded picture in the CVS.
loop_filter_across_subpic_enabled_flag [i] equal to 0 specifies that in-loop filtering operations are not performed  across the boundaries of the i-th subpicture in each coded picture in the CVS. When not present, the value of loop_filter_across_subpic_enabled_pic_flag [i] is inferred to be equal to 1.
It is a requirement of bitstream conformance that the following constraints apply:
- For any two subpictures subpicA and subpicB, when the index of subpicA is less than the index of subpicB, any coded NAL unit of subPicA shall succeed any coded NAL unit of subPicB in decoding order.
- The shapes of the subpictures shall be such that each subpicture, when decoded, shall have its entire left boundary and entire top boundary consisting of picture boundaries or consisting of boundaries of previously decoded subpictures.
The list CtbToSubPicIdx [ctbAddrRs] for ctbAddrRs ranging from 0 to PicSizeInCtbsY-1, inclusive, specifying the conversion from a CTB address in picture raster scan to a subpicture index, is derived as follows:
Figure PCTCN2020108159-appb-000004
num_bricks_in_slice_minus1, when present, specifies the number of bricks in the slice minus 1. The value of num_bricks_in_slice_minus1 shall be in the range of 0 to NumBricksInPic-1, inclusive. When rect_slice_flag is equal to 0 and single_brick_per_slice_flag is equal to 1, the value of num_bricks_in_slice_minus1 is inferred to be equal to 0. When single_brick_per_slice_flag is equal to 1, the value of num_bricks_in_slice_minus1 is inferred to be equal to 0.
The variable NumBricksInCurrSlice, which specifies the number of bricks in the current slice, and SliceBrickIdx [i] , which specifies the brick index of the i-th brick in the current slice, are derived as follows:
Figure PCTCN2020108159-appb-000005
Figure PCTCN2020108159-appb-000006
The variables SubPicIdx, SubPicLeftBoundaryPos, SubPicTopBoundaryPos, SubPicRightBoundaryPos, and SubPicBotBoundaryPos are derived as follows:
Figure PCTCN2020108159-appb-000007
…
Derivation process for temporal luma motion vector prediction
Inputs to this process are:
– a luma location (xCb, yCb) of the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture,
– a variable cbWidth specifying the width of the current coding block in luma samples,
– a variable cbHeight specifying the height of the current coding block in luma samples,
– a reference index refIdxLX, with X being 0 or 1.
Outputs of this process are:
– the motion vector prediction mvLXCol in 1/16 fractional-sample accuracy,
– the availability flag availableFlagLXCol.
The variable currCb specifies the current luma coding block at luma location (xCb, yCb) .
The variables mvLXCol and availableFlagLXCol are derived as follows:
– If slice_temporal_mvp_enabled_flag is equal to 0 or (cbWidth *cbHeight) is less than or equal to 32, both components of mvLXCol are set equal to 0 and availableFlagLXCol is set equal to 0.
– Otherwise (slice_temporal_mvp_enabled_flag is equal to 1) , the following ordered steps apply:
1. The bottom right collocated motion vector and the bottom and right boundary sample locations are derived as follows:
xColBr = xCb + cbWidth     (8-421)
yColBr = yCb + cbHeight     (8-422)
rightBoundaryPos = subpic_treated_as_pic_flag [SubPicIdx] ? SubPicRightBoundaryPos : pic_width_in_luma_samples-1           (8-423)
botBoundaryPos = subpic_treated_as_pic_flag [SubPicIdx] ? SubPicBotBoundaryPos : pic_height_in_luma_samples-1        (8-424)
– If yCb >> CtbLog2SizeY is equal to yColBr >> CtbLog2SizeY, yColBr is less than or equal to botBoundaryPos and xColBr is less than or equal to rightBoundaryPos, the following applies:
– The variable colCb specifies the luma coding block covering the modified location given by ( (xColBr >> 3) << 3, (yColBr >> 3) << 3) inside the collocated picture specified by ColPic.
– The luma location (xColCb, yColCb) is set equal to the top-left sample of the collocated luma coding block specified by colCb relative to the top-left luma sample of the collocated picture specified by ColPic.
– The derivation process for collocated motion vectors as specified in clause 8.5.2.12 is invoked with currCb, colCb, (xColCb, yColCb) , refIdxLX and sbFlag set equal to 0 as inputs, and the output is assigned to mvLXCol and availableFlagLXCol.
Otherwise, both components of mvLXCol are set equal to 0 and availableFlagLXCol is set equal to 0.
…
Luma sample bilinear interpolation process
Inputs to this process are:
– a luma location in full-sample units (xInt L, yInt L) ,
– a luma location in fractional-sample units (xFrac L, yFrac L) ,
– the luma reference sample array refPicLX L.
Output of this process is a predicted luma sample value predSampleLX L
The variables shift1, shift2, shift3, shift4, offset1, offset2 and offset3 are derived as follows:
shift1 = BitDepth Y-6                    (8-453)
offset1 = 1 << (shift1-1)                (8-454)
shift2 = 4                  (8-455)
offset2 = 1 << (shift2-1)                 (8-456)
shift3=10-BitDepth Y             (8-457)
shift4 = BitDepth Y-10                   (8-458)
offset4 = 1 << (shift4-1)                 (8-459)
The variable picW is set equal to pic_width_in_luma_samples and the variable picH is set equal to pic_height_in_luma_samples.
The luma interpolation filter coefficients fb L [p] for each 1/16 fractional sample position p equal to xFrac L or yFrac L are specified in Table 8-10.
The luma locations in full-sample units (xInt i, yInt i) are derived as follows for i = 0.. 1:
– If subpic_treated_as_pic_flag [SubPicIdx] is equal to 1, the following applies:
xInt i = Clip3 (SubPicLeftBoundaryPos, SubPicRightBoundaryPos, xInt L + i)         (8-460)
yInt i = Clip3 (SubPicTopBoundaryPos, SubPicBotBoundaryPos, yInt L + i)         (8-461)
– Otherwise (subpic_treated_as_pic_flag [SubPicIdx] is equal to 0) , the following applies:
Figure PCTCN2020108159-appb-000008
yInt i = Clip3 (0, picH-1, yInt L + i)            (8-463)
…
Derivation process for subblock-based temporal merging candidates
Inputs to this process are:
– a luma location (xCb, yCb) of the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture,
– a variable cbWidth specifying the width of the current coding block in luma samples,
– a variable cbHeight specifying the height of the current coding block in luma samples.
– the availability flag availableFlagA 1 of the neighbouring coding unit,
– the reference index refIdxLXA 1 of the neighbouring coding unit with X being 0 or 1,
– the prediction list utilization flag predFlagLXA 1 of the neighbouring coding unit with X being 0 or 1,
– the motion vector in 1/16 fractional-sample accuracy mvLXA 1 of the neighbouring coding unit with X being 0 or 1.
Outputs of this process are:
– the availability flag availableFlagSbCol,
– the number of luma coding subblocks in horizontal direction numSbX and in vertical direction numSbY,
– the reference indices refIdxL0SbCol and refIdxL1SbCol,
– the luma motion vectors in 1/16 fractional-sample accuracy mvL0SbCol [xSbIdx] [ySbIdx] and mvL1SbCol [xSbIdx] [ySbIdx] with xSbIdx = 0.. numSbX-1, ySbIdx = 0 .. numSbY-1,
– the prediction list utilization flags predFlagL0SbCol [xSbIdx] [ySbIdx] and predFlagL1SbCol [xSbIdx] [ySbIdx] with xSbIdx = 0.. numSbX-1, ySbIdx = 0 .. numSbY-1.
The availability flag availableFlagSbCol is derived as follows.
– If one or more of the following conditions is true, availableFlagSbCol is set equal to 0.
– slice_temporal_mvp_enabled_flag is equal to 0.
– sps_sbtmvp_enabled_flag is equal to 0.
– cbWidth is less than 8.
– cbHeight is less than 8.
– Otherwise, the following ordered steps apply:
1. The location (xCtb, yCtb) of the top-left sample of the luma coding tree block that contains the current coding block and the location (xCtr, yCtr) of the below-right center sample of the current luma coding block are derived as follows:
xCtb = (xCb >> CtuLog2Size) << CtuLog2Size            (8-542)
yCtb = (yCb >> CtuLog2Size) << CtuLog2Size           (8-543)
xCtr = xCb + (cbWidth /2)             (8-544)
yCtr = yCb + (cbHeight /2)                (8-545)
2. The luma location (xColCtrCb, yColCtrCb) is set equal to the top-left sample of the collocated luma coding block covering the location given by (xCtr, yCtr) inside ColPic relative to the top-left luma sample of the collocated picture specified by ColPic.
3. The derivation process for subblock-based temporal merging base motion data as specified in clause 8.5.5.4 is invoked with the location (xCtb, yCtb) , the location (xColCtrCb, yColCtrCb) , the availability flag availableFlagA 1, and the prediction list utilization flag predFlagLXA 1, and the reference index refIdxLXA 1, and the motion vector mvLXA 1, with X being 0 and 1 as inputs and the motion vectors ctrMvLX, and the prediction list utilization flags ctrPredFlagLX of the collocated block, with X being 0 and 1, and the temporal motion vector tempMv as outputs.
4. The variable availableFlagSbCol is derived as follows:
– If both ctrPredFlagL0 and ctrPredFlagL1 are equal to 0, availableFlagSbCol is set equal to 0.
– Otherwise, availableFlagSbCol is set equal to 1.
When availableFlagSbCol is equal to 1, the following applies:
– The variables numSbX, numSbY, sbWidth, sbHeight and refIdxLXSbCol are derived as follows:
numSbX = cbWidth >> 3                  (8-546)
numSbY = cbHeight >> 3                  (8-547)
sbWidth = cbWidth /numSbX                   (8-548)
sbHeight = cbHeight /numSbY                  (8-549)
refIdxLXSbCol = 0                 (8-550)
– For xSbIdx = 0.. numSbX-1 and ySbIdx = 0 .. numSbY-1, the motion vectors mvLXSbCol [xSbIdx] [ySbIdx] and prediction list utilization flags predFlagLXSbCol [xSbIdx] [ySbIdx] are derived as follows:
– The luma location (xSb, ySb) specifying the top-left sample of the current coding subblock relative to the top-left luma sample of the current picture is derived as follows:
xSb = xCb + xSbIdx *sbWidth + sbWidth /2                   (8-551)
ySb = yCb + ySbIdx *sbHeight + sbHeight /2                (8-552)
– The location (xColSb, yColSb) of the collocated subblock inside ColPic is derived as follows.
– The following applies:
Figure PCTCN2020108159-appb-000009
– If subpic_treated_as_pic_flag [SubPicIdx] is equal to 1, the following applies:
Figure PCTCN2020108159-appb-000010
– Otherwise (subpic_treated_as_pic_flag [SubPicIdx] is equal to 0) , the following applies:
Figure PCTCN2020108159-appb-000011
…
Derivation process for subblock-based temporal merging base motion data
Inputs to this process are:
– the location (xCtb, yCtb) of the top-left sample of the luma coding tree block that contains the current coding block,
– the location (xColCtrCb, yColCtrCb) of the top-left sample of the collocated luma coding block that covers the below-right center sample.
– the availability flag availableFlagA 1 of the neighbouring coding unit,
– the reference index refIdxLXA 1 of the neighbouring coding unit,
– the prediction list utilization flag predFlagLXA 1 of the neighbouring coding unit,
– the motion vector in 1/16 fractional-sample accuracy mvLXA 1 of the neighbouring coding unit.
Outputs of this process are:
– the motion vectors ctrMvL0 and ctrMvL1,
– the prediction list utilization flags ctrPredFlagL0 and ctrPredFlagL1,
– the temporal motion vector tempMv.
The variable tempMv is set as follows:
tempMv [0] = 0                 (8-558)
tempMv [1] = 0                  (8-559)
The variable currPic specifies the current picture.
When availableFlagA 1 is equal to TRUE, the following applies:
– If all of the following conditions are true, tempMv is set equal to mvL0A 1:
– predFlagL0A 1 is equal to 1,
– DiffPicOrderCnt (ColPic, RefPicList [0] [refIdxL0A 1] ) is equal to 0,
– Otherwise, if all of the following conditions are true, tempMv is set equal to mvL1A 1:
– slice_type is equal to B,
– predFlagL1A 1 is equal to 1,
– DiffPicOrderCnt (ColPic, RefPicList [1] [refIdxL1A 1] ) is equal to 0.
The location (xColCb, yColCb) of the collocated block inside ColPic is derived as follows.
– The following applies:
Figure PCTCN2020108159-appb-000012
– If subpic_treated_as_pic_flag [SubPicIdx] is equal to 1, the following applies:
Figure PCTCN2020108159-appb-000013
– Otherwise (subpic_treated_as_pic_flag [SubPicIdx] is equal to o, the following applies:
Figure PCTCN2020108159-appb-000014
…
Luma sample interpolation filtering process
Inputs to this process are:
– a luma location in full-sample units (xInt L, yInt L) ,
– a luma location in fractional-sample units (xFrac L, yFrac L) ,
– a luma location in full-sample units (xSbInt L, ySbInt L) specifying the top-left sample of the bounding block for reference sample padding relative to the top-left luma sample of the reference picture,
– the luma reference sample array refPicLX L,
– the half sample interpolation filter index hpelIfIdx,
– a variable sbWidth specifying the width of the current subblock,
– a variable sbHeight specifying the height of the current subblock,
– a luma location (xSb, ySb) specifying the top-left sample of the current subblock relative to the top-left luma sample of the current picture,
Output of this process is a predicted luma sample value predSampleLX L
The variables shift1, shift2 and shift3 are derived as follows:
– The variable shift1 is set equal to Min (4, BitDepth Y-8) , the variable shift2 is set equal to 6 and the variable shift3 is set equal to Max (2, 14-BitDepth Y) .
– The variable picW is set equal to pic_width_in_luma_samples and the variable picH is set equal to pic_height_in_luma_samples.
The luma interpolation filter coefficients f L [p] for each 1/16 fractional sample position p equal to xFrac L or yFrac L are derived as follows:
– If MotionModelIdc [xSb] [ySb] is greater than 0, and sbWidth and sbHeight are both equal to 4, the luma interpolation filter coefficients f L [p] are specified in Table 8-12.
– Otherwise, the luma interpolation filter coefficients f L [p] are specified in Table 8-11 depending on hpelIfIdx.
The luma locations in full-sample units (xInt i, yInt i) are derived as follows for i = 0.. 7:
– If subpic_treated_as_pic_flag [SubPicIdx] is equal to 1, the following applies:
xInt i = Clip3 (SubPicLeftBoundaryPos, SubPicRightBoundaryPos, xInt L + i-3)          (8-771)
yInt i = Clip3 (SubPicTopBoundaryPos, SubPicBotBoundaryPos, yInt L + i-3)          (8-772)
– Otherwise (subpic_treated_as_pic_flag [SubPicIdx] is equal to 0) , the following applies:
Figure PCTCN2020108159-appb-000015
yInt i = Clip3 (0, picH-1, yInt L + i-3)            (8-774)
…
Chroma sample interpolation process
Inputs to this process are:
– a chroma location in full-sample units (xInt C, yInt C) ,
– a chroma location in 1/32 fractional-sample units (xFrac C, yFrac C) ,
– a chroma location in full-sample units (xSbIntC, ySbIntC) specifying the top-left sample of the bounding block for reference sample padding relative to the top-left chroma sample of the reference picture,
– a variable sbWidth specifying the width of the current subblock,
– a variable sbHeight specifying the height of the current subblock,
– the chroma reference sample array refPicLX C.
Output of this process is a predicted chroma sample value predSampleLX C
The variables shift1, shift2 and shift3 are derived as follows:
– The variable shift1 is set equal to Min (4, BitDepth C-8) , the variable shift2 is set equal to 6 and the variable shift3 is set equal to Max (2, 14-BitDepth C) .
– The variable picW C is set equal to pic_width_in_luma_samples /SubWidthC and the variable picH C is set equal to pic_height_in_luma_samples /SubHeightC.
The chroma interpolation filter coefficients f C [p] for each 1/32 fractional sample position p equal to xFrac C or yFrac C are specified in Table 8-13.
The variable xOffset is set equal to (sps_ref_wraparound_offset_minus1 + 1) *MinCbSizeY) /SubWidthC.
The chroma locations in full-sample units (xInt i, yInt i) are derived as follows for i = 0.. 3:
– If subpic_treated_as_pic_flag [SubPicIdx] is equal to 1, the following applies:
xInt i = Clip3 (SubPicLeftBoundaryPos /SubWidthC, SubPicRightBoundaryPos /SubWidthC, xInt L + i)     (8-785)
yInt i = Clip3 (SubPicTopBoundaryPos /SubHeightC, SubPicBotBoundaryPos /SubHeightC, yInt L + i)     (8-786)
– Otherwise (subpic_treated_as_pic_flag [SubPicIdx] is equal to 0) , the following applies:
Figure PCTCN2020108159-appb-000016
yInt i = Clip3 (0, picH C-1, yInt C + i-1)           (8-788)
2.4 Example Encoder-only GOP-based temporal filter
In some embodiments, an encoder-only temporal filter can be implemented. The filtering is done at the encoder side as a pre-processing step. Source pictures before and after the selected picture to encode are read and a block based motion compensation method relative to the selected picture is applied on those source pictures. Samples in the selected picture are temporally filtered using sample values after motion compensation.
The overall filter strength is set depending on the temporal sub layer of the selected picture as well as the QP. Only pictures at temporal sub layers 0 and 1 are filtered and pictures of layer 0 are filter by a stronger filter than pictures of layer 1. The per sample filter strength is adjusted depending on the difference between the sample value in the selected picture and the co-located samples in motion compensated pictures so that small differences between a motion compensated picture and the selected picture are filtered more strongly than larger differences.
GOP Based Temporal Filter
A temporal filter is introduced directly after reading picture and before encoding. Below are the steps described in more detail.
Operation 1: Pictures are read by the encoder
Operation 2: If a picture is low enough in the coding hierarchy, it is filtered before encoding. Otherwise the picture is encoded without filtering. RA pictures with POC %8 == 0 are filtered as well as LD pictures with POC %4 == 0. AI pictures are never filtered.
The overall filter strength, s o, is set according to the equation below for RA.
Figure PCTCN2020108159-appb-000017
where n is the number of pictures read.
For the LD case, s o (n)=0.95 is used.
Operation 3: Two pictures before and/or after the selected picture (referred to as original picture further down) are read. In the edge cases e.g. if is the first picture or close to the last picture, only the available pictures are read.
Operation 4: Motion of the read pictures before and after, relative to the original picture is estimated per 8x8 picture block.
Ahierarchical motion estimation scheme is used and the layers L0, L1 and L2, are illustrated in FIG. 2. Subsampled pictures are generated by averaging each 2x2 block for all read pictures and the original picture, e.g. L1 in FIG. 1. L2 is derived from L1 using the same subsampling method.
FIG. 2 shows examples of different layers of the hierarchical motion estimation. L0 is the original resolution. L1 is a subsampled version of L0. L2 is a subsampled version of L1.
First, motion estimation is done for each 16x16 block in L2. The squared difference is calculated for each selected motion vector and the motion vector corresponding to the smallest difference is selected. The selected motion vector is then used as initial value when estimating the motion in L1. Then the same is done for estimating motion in L0. As a final step, subpixel motion is estimated for each 8x8 block by using an interpolation filter on L0.
The VTM 6-tap interpolation filter can used:
Figure PCTCN2020108159-appb-000018
Operation 5: Motion compensation is applied on the pictures before and after the original picture according to the best matching motion for each block, e.g., so that the sample coordinates of the original picture in each block have the best matching coordinates in the referenced pictures.
Operation 6: The samples of the processed one by one for the luma and chroma channels as described in the following steps.
Operation 7: The new sample value, I n, is calculated using the following formula.
Figure PCTCN2020108159-appb-000019
Where I o is the sample value of the original sample, I r (i) is the intensity of the corresponding sample of motion compensated picture i and w r (i, a) is the weight of motion compensated picture i when the number of available motion compensated pictures is a.
In the luma channel, the weights, w r (i, a) , is defined as follows:
Figure PCTCN2020108159-appb-000020
Where
s l＝0.4
Figure PCTCN2020108159-appb-000021
Figure PCTCN2020108159-appb-000022
For all other cases of i, and a: s r (i, a) ＝0.3
σ l(QP) ＝3* (QP-10)
ΔI (i) ＝I r (i) -I o
For the chroma channels, the weights, w r (i, a) , is defined as follows:
Figure PCTCN2020108159-appb-000023
Where s c=0.55 and σ c=30
Operation 8: The filter is applied for the current sample. The resulting sample value is stored separately.
Operation 9: The filtered picture is encoded.
3. Examples of technical problems solved by disclosed embodiments
(1) There are some designs that can violate the sub-picture constrain.
A. TMVP in the affine constructed candidates may fetch a MV in the collocated picture out of the range of the current sub-picture.
B. When deriving gradients in Bi-Directional Optical Flow (BDOF) and Prediction Refinement Optical Flow (PROF) , two extended rows and two extended columns of integer reference samples are required to be fetched. These reference samples may be out of the range of the current sub-picture.
C. When deriving the chroma residual scaling factor in luma mapping chroma scaling (LMCS) , the accessed reconstructed luma samples may be out of the range of the range of the current sub-picture.
D. The neighboring block may be out of the range of the current sub-picture, when deriving the luma intra prediction mode, reference samples for intra prediction, reference samples for CCLM, neighboring block availability for spatial neighboring candidates for merge/AMVP/CIIP/IBC/LMCS, quantization parameters, CABAC initialization process, ctxInc derivation using left and above syntax elements, and ctxIncfor the syntax element mtt_split_cu_vertical_flag. The representation of subpicture may lead to subpicture with incomplete CTUs. The CTU partitions and CU splitting process may need to consider incomplete CTUs.
(2) The signaled syntax elements related to sub-picture may be arbitrarily large, which may cause an overflow problem.
(3) The representation of sub-pictures may lead to non-rectangular sub-pictures.
(4) Currently the subpicture and subpicture grid is defined in units of 4 samples. And the length of syntax element is dependent on the picture height divided by 4. However, since the current pic_width_in_luma_samples and pic_height_in_luma_samples shall be an integer multiple of Max (8, MinCbSizeY) , the subpicture grid may need to be defined in units of 8 samples.
(5) The SPS syntax, pic_width_max_in_luma_samples and pic_height_max_in_luma_samples may need to be restricted to be no smaller than 8.
(6) Interaction between reference picture resampling/scalability and subpicture is not considered in the current design.
(7) In temporal filtering, samples across different sub-pictures may be required.
4. Example techniques and embodiments
The detailed listing below should be considered as examples to explain general concepts. These items should not be interpreted in a narrow way. Furthermore, these items can be combined in any manner. Hereinafter, temporal filter is used to represent filters that require samples in other pictures. Max (x, y) returns the larger one of x and y. Min (x, y) returns the smaller one of x and y.
1. The position (named position RB) at which a temporal MV predictor is fetched in a picture to generate affine motion candidates (e.g. a constructed affine merge candidate) must be in a required sub-picture, assuming the top-left corner coordinate of the required sub-picture is (xTL, yTL) and bottom-right coordinate of the required sub-picture is (xBR, yBR) .
a. In one example, the required sub-picture is the sub-picture covering the current block.
b. In one example, if position RB with a coordinate (x, y) is out of the required sub-picture, the temporal MV predictor is treated as unavailable.
i. In one example, position RB is out of the required sub-picture if x > xBR.
ii. In one example, position RB is out of the required sub-picture if y > yBR.
iii. In one example, position RB is out of the required sub-picture if x < xTL.
iv. In one example, position RB is out of the required sub-picture if y < yTL.
c. In one example, position RB, if outside of the required sub-picture, a replacement of RB is utilized.
i. Alternatively, furthermore, the replacement position shall be in the required sub-picture.
d. In one example, position RB is clipped to be in the required sub-picture.
i. In one example, x is clipped as x = Min (x, xBR) .
ii. In one example, y is clipped as y = Min (y, yBR) .
iii. In one example, x is clipped as x = Max (x, xTL) .
iv. In one example, y is clipped as y = Max (y, yTL) .
e. In one example, the position RB may be the bottom right position inside the corresponding block of current block in the collocated picture.
f. The proposed method may be utilized in other coding tools which require to access motion information from a picture different than the current picture.
g. In one example, whether the above methods are applied (e.g., position RB must be in a required sub-picture (e.g. to do as claimed in 1. a and/or 1. b) ) may depend on one or more syntax elements signaled in VPS/DPS/SPS/PPS/APS/slice header/tile group header. For example, the syntax element may be subpic_treated_as_pic_flag [SubPicIdx] , where SubPicIdx is the sub-picture index of sub-picture covering the current block.
2. The position (named position S) at which an integer sample is fetched in a reference not used in the interpolation process must be in a required sub-picture, assuming the top-left corner coordinate of the required sub-picture is (xTL, yTL) and the bottom-right coordinate of the required sub-picture is (xBR, yBR) .
a. In one example, the required sub-picture is the sub-picture covering the current block.
b. In one example, if position S with a coordinate (x, y) is out of the required sub-picture, the reference sample is treated as unavailable.
i. In one example, position S is out of the required sub-picture if x > xBR.
ii. In one example, position S is out of the required sub-picture if y > yBR.
iii. In one example, position S is out of the required sub-picture if x < xTL.
iv. In one example, position S is out of the required sub-picture if y < yTL.
c. In one example, position S is clipped to be in the required sub-picture.
i. In one example, x is clipped as x = Min (x, xBR) .
ii. In one example, y is clipped as y = Min (y, yBR) .
iii. In one example, x is clipped as x = Max (x, xTL) .
iv. In one example, y is clipped as y = Max (y, yTL) .
d. In one example, whether position S must be in a required sub-picture (e.g. to do as claimed in 2. a and/or 2. b) may depend on one or more syntax elements signaled in VPS/DPS/SPS/PPS/APS/slice header/tile group header. For example,  the syntax element may be subpic_treated_as_pic_flag [SubPicIdx] , where SubPicIdx is the sub-picture index of sub-picture covering the current block.
e. In one example, the fetched integer sample is used to generate gradients in BDOF and/or PORF.
3. The position (named position R) at which the reconstructed luma sample value is fetched may be in a required sub-picture, assuming the top-left corner coordinate of the required sub-picture is (xTL, yTL) and the bottom-right coordinate of the required sub-picture is (xBR, yBR) .
a. In one example, the required sub-picture is the sub-picture covering the current block.
b. In one example, if position R with a coordinate (x, y) is out of the required sub-picture, the reference sample is treated as unavailable.
i. In one example, position R is out of the required sub-picture if x > xBR.
ii. In one example, position R is out of the required sub-picture if y > yBR.
iii. In one example, position R is out of the required sub-picture if x < xTL.
iv. In one example, position R is out of the required sub-picture if y < yTL.
c. In one example, position R is clipped to be in the required sub-picture.
i. In one example, x is clipped as x = Min (x, xBR) .
ii. In one example, y is clipped as y = Min (y, yBR) .
iii. In one example, x is clipped as x = Max (x, xTL) .
iv. In one example, y is clipped as y = Max (y, yTL) .
d. In one example, whether position R must be in a required sub-picture (e.g. to do as claimed in 4. a and/or 4. b) may depend on one or more syntax elements signaled in VPS/DPS/SPS/PPS/APS/slice header/tile group header. For example, the syntax element may be subpic_treated_as_pic_flag [SubPicIdx] , where SubPicIdx is the sub-picture index of sub-picture covering the current block.
e. In one example, the fetched luma sample is used to derive the scaling factor for the chroma component (s) in LMCS.
4. The position (named position N) at which the picture boundary check for BT/TT/QT splitting, BT/TT/QT depth derivation, and/or the signaling of CU split flag must be in a  required sub-picture, assuming the top-left corner coordinate of the required sub-picture is (xTL, yTL) and the bottom-right coordinate of the required sub-picture is (xBR, yBR) .
a. In one example, the required sub-picture is the sub-picture covering the current block.
b. In one example, if position N with a coordinate (x, y) is out of the required sub-picture, the reference sample is treated as unavailable.
i. In one example, position N is out of the required sub-picture if x > xBR.
ii. In one example, position N is out of the required sub-picture if y > yBR.
iii. In one example, position N is out of the required sub-picture if x < xTL.
iv. In one example, position N is out of the required sub-picture if y < yTL.
c. In one example, position N is clipped to be in the required sub-picture.
i. In one example, x is clipped as x = Min (x, xBR) .
ii. In one example, y is clipped as y = Min (y, yBR) .
iii. In one example, x is clipped as x = Max (x, xTL) .
d. In one example, y is clipped as y = Max (y, yTL) . In one example, whether position N must be in a required sub-picture (e.g. to do as claimed in 5. a and/or 5. b) may depend on one or more syntax elements signaled in VPS/DPS/SPS/PPS/APS/slice header/tile group header. For example, the syntax element may be subpic_treated_as_pic_flag [SubPicIdx] , where SubPicIdx is the sub-picture index of sub-picture covering the current block.
5. History -based Motion Vector Prediction (HMVP) table may be reset before decoding a new sub-picture in one picture.
a. In one example, the HMVP table used for IBC coding may be reset
b. In one example, the HMVP table used for inter coding may be reset
c. In one example, the HMVP table used for intra coding may be reset
6. The subpicture syntax elements may be defined in units of N (such as N = 8, 32, and etc) samples.
a. In one example, the width of each element of the subpicture identifier grid in units of N samples.
b. In one example, the height of each element of the subpicture identifier grid in units of N samples.
c. In one example, N is set to the width and/or height of CTU.
7. The syntax element of picture width and picture height may be restricted to be no smaller than K (K>=8) .
a. In one example, the picture width may need to be restricted to be no smaller than 8.
b. In one example, the picture height may need to be restricted to be no smaller than 8.
8. A conformance bitstream shall satisfy that sub-picture coding and Adaptive resolution conversion (ARC) /Dynamic resolution conversion (DRC) /Reference picture resampling (RPR) are disallowed to be enabled for one video unit (e.g., sequence) .
a. In one example, signaling of enabling sub-picture coding may be under the conditions of disallowing ARC/DRC/RPR.
i. In one example, when subpicture is enabled, such as subpics_present_flag equal to 1, pic_width_in_luma_samples for all pictures for which this SPS is active is equal to max_width_in_luma_samples.
b. Alternatively, sub-picture coding and ARC/DRC/RPR may be both enabled for one video unit (e.g., sequence) .
i. In one example, a conformance bitstream shall satisfy that the donwsampled sub-picture due to ARC/DRC/RPR shall still be in the form of K CTUs in width and M CTUs in height wherein K and M are both integers.
ii. In one example, a conformance bitstream shall satisfy that for sub-pictures not located at picture boundaries (e.g., right boundary and/or bottom boundary) , the donwsampled sub-picture due to ARC/DRC/RPR shall still be in the form of K CTUs in width and M CTUs in height wherein K and M are both integers.
iii. In one example, CTU sizes may be adaptively changed based on the picture resolution.
1) In one example, a max CTU size may be signaled in SPS. For each picture with less resolution, the CTU size may be changed accordingly based on the reduced resolution.
2) In one example, CTU size may be signaled in SPS and PPS, and/or sub-picture level.
9. The syntax element subpic_grid_col_width_minus1 and subpic_grid_row_height_minus1 may be constrained.
a. In one example, subpic_grid_col_width_minus1 must be no larger (or must be smaller) than T1.
b. In one example, subpic_grid_row_height_minus1 must be no larger (or must be smaller) than T2.
c. In one example, in a conformance bit-stream, subpic_grid_col_width_minus1 and/or subpic_grid_row_height_minus1 must follow the constraint such as bullet 3. a or 3. b.
d. In one example, T1 in 3. a and/or T2 in 3. b may depend on profiles/levels/tiers of a video coding standard.
e. In one example, T1 in 3. a may depend on the picture width.
i. For example, T1 is equal to pic_width_max_in_luma_samples/4 or pic_width_max_in_luma_samples/4+Off. Off may be 1, 2, -1, -2, etc.
f. In one example, T2 in 3. b may depend on the picture width.
i. For example, T2 is equal to pic_height_max_in_luma_samples/4 or pic_height_max_in_luma_samples /4-1+Off. Off may be 1, 2, -1, -2, etc.
10. It is constrained that a boundary between two sub-pictures must be a boundary between two CTUs.
a. In other words, a CTU cannot be covered by more than one sub-picture.
b. In one example, the unit of subpic_grid_col_width_minus1 may be the CTU width (such as 32, 64, 128) , instead of 4 as in VVC. The sub-picture grid width should be (subpic_grid_col_width_minus1+1) *CTU width.
c. In one example, the unit of subpic_grid_col_height_minus1 may be the CTU height (such as 32, 64, 128) , instead of 4 as in VVC. The sub-picture grid height should be (subpic_grid_col_height_minus1+1) *CTU height.
d. In one example, in a conformance bit-stream, the constraint must be satisfied if the sub-picture approach is applied.
11. It is constrained that the shape of a sub-picture must be rectangular.
a. In one example, in a conformance bit-stream, the constraint must be satisfied if the sub-picture approach is applied.
b. Sub-picture may only contain rectangular slices. For example, in a conformance bit-stream, the constraint must be satisfied if the sub-picture approach is applied.
12. It is constrained that two sub-pictures cannot be overlapped.
a. In one example, in a conformance bit-stream, the constraint must be satisfied if the sub-picture approach is applied.
b. Alternatively, two sub-pictures may be overlapped with each other.
13. It is constrained that any position in the picture must be covered by one and only one sub-picture.
a. In one example, in a conformance bit-stream, the constraint must be satisfied if the sub-picture approach is applied.
b. Alternatively, one sample may not belong to any sub-picture.
c. Alternatively, one sample may belong to more than one sub-pictures.
14. It may be constrained that sub-pictures defined in a SPS mapped to every resolution presented in the same sequence should obey the location and/or size constrained mentioned above.
a. In one example, the width and height of a sub-picture defined in the SPS mapped to a resolution presented in the same sequence, should be integer multiple times of N (such as 8, 16, 32) luma samples.
b. In one example, sub-pictures may be defined for certain layer and may be mapped to other layers.
i. For example, sub-pictures may be defined for the layer with the highest resolution in the sequence.
ii. For example, sub-pictures may be defined for the layer with the lowest resolution in the sequence.
iii. Which layer the sub-pictures are defined for may be signaled in SPS/VPS/PPS/slice header.
c. In one example, when sub-pictures and different resolutions are both applied, all resolutions (e.g., width or/and height) may be integer multiple of a given resolution.
d. In one example, the width and/or height of a sub-picture defined in the SPS may be integer multiple times (e.g., M) of the CTU size.
e. Alternatively, sub-pictures and different resolutions in a sequence may not be allowed simultaneously.
15. Sub-pictures may only apply to a certain layer (s)
a. In one example, sub-pictures defined in a SPS may only apply to the layer with the highest resolution in a sequence.
b. In one example, sub-pictures defined in a SPS may only apply to the layer with the lowest temporal id in a sequence.
c. Which layer (s) that sub-pictures may be applied to may be indicated by one or multiple syntax elements in SPS/VPS/PPS.
d. Which layer (s) that sub-picture cannot be applied to may be indicated by one or multiple syntax elements in SPS/VPS/PPS.
16. In one example, the position and/or dimensions of a sub-picture may be signaled without using subpic_grid_idx.
a. In one example, the top-left position of a sub-picture may be signaled.
b. In one example, the bottom-right position of a sub-picture may be signaled.
c. In one example, the width of sub-picture may be signaled.
d. In one example, the height of a sub-picture may be signaled.
17. For temporal filter, when performing the temporal filtering of a sample, only samples within the same sub-picture that the current sample belongs to may be used. The required samples may be in the same picture that the current sample belongs to or in other pictures.
18. In one example, whether to and/or how to apply a partitioning method (such as QT, horizontal BT, vertical BT, horizontal TT, vertical TT, or not split, etc. ) may depend on whether the current block (or partition) crosses one or multiple boundary of a sub-picture.
a. In one example, the picture boundary handling method for partitioning in VVC may also be applied when a picture boundary is replaced by a sub-picture boundary.
b. In one example, whether to parse a syntax element (e.g. a flag) which represents a partitioning method (such as QT, horizontal BT, vertical BT, horizontal TT, vertical TT, or not split, etc. ) may depend on whether the current block (or partition) crosses one or multiple boundary of a sub-picture.
19. Instead of splitting one picture into multiple sub-pictures with independent coding of each sub-picture, it is proposed to split a picture into at least two sets of sub-regions, with the first set including multiple sub-pictures and the second set including all the remaining samples.
a. In one example, a sample in the second set is not in any sub-pictures.
b. Alternatively, furthermore, the second set may be encoded/decoded based on the information of the first set.
c. In one example, a default value may be utilized to mark whether a sample/MxK sub-region belonging to the second set.
i. In one example, the default value may be set equal to (max_subpics_minus1 +K) wherein K is an integer greater than 1.
ii. The default value may be assigned to subpic_grid_idx [i] [j] to indicate that grid belongs to the second set.
20.  It is proposed that the syntax element subpic_grid_idx [i] [j] cannot be larger than max_subpics_minus1.
a. For example, it is constrained that in a conformance bit-stream, subpic_grid_idx [i] [j] cannot be larger than max_subpics_minus1.
b. For example, the codeword to code subpic_grid_idx [i] [j] cannot be larger than max_subpics_minus1.
21. It is proposed that, any integer number from 0 to max_subpics_minus1 must be equal to at least one subpic_grid_idx [i] [j] .
22. IBC virtual buffer may be reset before decoding a new sub-picture in one picture.
a. In one example, all the samples in the IBC virtual buffer may be reset to -1.
23. Palette entry list may be reset before decoding a new sub-picture in one picture.
a. In one example, PredictorPaletteSize may be set equal to 0 before decoding a new sub-picture in one picture.
5. Embodiments
In the following embodiments, the newly added texts are bold italicized and the deleted texts are marked by “ [ [] ] ” .
5.1 Embodiment 1: sub-picture constraint on affine constructed merge candidates
8.5.5.6 Derivation process for constructed affine control point motion vector merging candidates
Inputs to this process are:
– a luma location (xCb, yCb) specifying the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture,
– two variables cbWidth and cbHeight specifying the width and the height of the current luma coding block,
– the availability flags availableA 0, availableA 1, availableA 2, availableB 0, availableB 1, availableB 2, availableB 3,
– the sample locations (xNbA 0, yNbA 0) , (xNbA 1, yNbA 1) , (xNbA 2, yNbA 2) , (xNbB 0, yNbB 0) , (xNbB 1, yNbB 1) , (xNbB 2, yNbB 2) and (xNbB 3, yNbB 3) .
Output of this process are:
– the availability flag of the constructed affine control point motion vector merging candidiates availableFlagConstK, with K = 1.. 6,
– the reference indices refIdxLXConstK, with K = 1.. 6, X being 0 or 1,
– the prediction list utilization flags predFlagLXConstK, with K = 1.. 6, X being 0 or 1,
– the affine motion model indices motionModelIdcConstK, with K = 1.. 6,
– the bi-prediction weight indices bcwIdxConstK, with K = 1.. 6,
– the constructed affine control point motion vectors cpMvLXConstK [cpIdx] with cpIdx = 0.. 2, K = 1.. 6 and X being 0 or 1.
…
The fourth (collocated bottom-right) control point motion vector cpMvLXCorner [3] , reference index refIdxLXCorner [3] , prediction list utilization flag predFlagLXCorner [3] and the availability flag availableFlagCorner [3] with X being 0 and 1 are derived as follows:
– The reference indices for the temporal merging candidate, refIdxLXCorner [3] , with X being 0 or 1, are set equal to 0.
– The variables mvLXCol and availableFlagLXCol, with X being 0 or 1, are derived as follows:
– If slice_temporal_mvp_enabled_flag is equal to 0, both components of mvLXCol are set equal to 0 and availableFlagLXCol is set equal to 0.
– Otherwise (slice_temporal_mvp_enabled_flag is equal to 1) , the following applies:
xColBr = xCb + cbWidth     (8-601)
yColBr = yCb + cbHeight       (8-602)
rightBoundaryPos = subpic_treated_as_pic_flag [SubPicIdx] ?
SubPicRightBoundaryPos : pic_width_in_luma_samples-1
botBoundaryPos = subpic_treated_as_pic_flag [SubPicIdx] ?
SubPicBotBoundaryPos : pic_height_in_luma_samples-1
– If yCb >> CtbLog2SizeY is equal to yColBr >> CtbLog2SizeY,  yColBr is less than or equal to  botBoundaryPos and xColBr is less than or equal to rightBoundaryPos, the following applies:
– The variable colCb specifies the luma coding block covering the modified location given by ( (xColBr >> 3) << 3, (yColBr >> 3) << 3) inside the collocated picture specified by ColPic.
– The luma location (xColCb, yColCb) is set equal to the top-left sample of the collocated luma coding block specified by colCb relative to the top-left luma sample of the collocated picture specified by ColPic.
– The derivation process for collocated motion vectors as specified in clause 8.5.2.12 is invoked with currCb, colCb, (xColCb, yColCb) , refIdxLXCorner [3] and sbFlag set equal to 0 as inputs, and the output is assigned to mvLXCol and availableFlagLXCol.
– Otherwise, both components of mvLXCol are set equal to 0 and availableFlagLXCol is set equal to 0.
…
5.2 Embodiment 2: sub-picture constraint on affine constructed merge candidates
8.5.5.6 Derivation process for constructed affine control point motion vector merging candidates
Inputs to this process are:
– a luma location (xCb, yCb) specifying the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture,
– two variables cbWidth and cbHeight specifying the width and the height of the current luma coding block,
– the availability flags availableA 0, availableA 1, availableA 2, availableB 0, availableB 1, availableB 2, availableB 3,
– the sample locations (xNbA 0, yNbA 0) , (xNbA 1, yNbA 1) , (xNbA 2, yNbA 2) , (xNbB 0, yNbB 0) , (xNbB 1, yNbB 1) , (xNbB 2, yNbB 2) and (xNbB 3, yNbB 3) .
Output of this process are:
– the availability flag of the constructed affine control point motion vector merging candidiates availableFlagConstK, with K = 1.. 6,
– the reference indices refIdxLXConstK, with K = 1.. 6, X being 0 or 1,
– the prediction list utilization flags predFlagLXConstK, with K = 1.. 6, X being 0 or 1,
– the affine motion model indices motionModelIdcConstK, with K = 1.. 6,
– the bi-prediction weight indices bcwIdxConstK, with K = 1.. 6,
– the constructed affine control point motion vectors cpMvLXConstK [cpIdx] with cpIdx = 0.. 2, K = 1.. 6 and X being 0 or 1.
…
The fourth (collocated bottom-right) control point motion vector cpMvLXCorner [3] , reference index refIdxLXCorner [3] , prediction list utilization flag predFlagLXCorner [3] and the availability flag availableFlagCorner [3] with X being 0 and 1 are derived as follows:
– The reference indices for the temporal merging candidate, refIdxLXCorner [3] , with X being 0 or 1, are set equal to 0.
– The variables mvLXCol and availableFlagLXCol, with X being 0 or 1, are derived as follows:
– If slice_temporal_mvp_enabled_flag is equal to 0, both components of mvLXCol are set equal to 0 and availableFlagLXCol is set equal to 0.
– Otherwise (slice_temporal_mvp_enabled_flag is equal to 1) , the following applies:
xColBr = xCb + cbWidth      (8-601)
yColBr = yCb + cbHeight      (8-602)
rightBoundaryPos = subpic_treated_as_pic_flag [SubPicIdx] ?
SubPicRightBoundaryPos : pic_width_in_luma_samples-1
botBoundaryPos = subpic_treated_as_pic_flag [SubPicIdx] ?
SubPicBotBoundaryPos : pic_height_in_luma_samples –1
xColBr = Min (r ightBoundaryPos , xColBr)
yColBr = Min (botBoundaryPos , yColBr)
– If yCb >> CtbLog2SizeY is equal to yColBr >> CtbLog2SizeY, [ [yColBr is less than pic_height_in_luma_samples and xColBr is less than pic_width_in_luma_samples, the following applies] ] :
– The variable colCb specifies the luma coding block covering the modified location given by ( (xColBr >> 3) << 3, (yColBr >> 3) << 3) inside the collocated picture specified by ColPic.
– The luma location (xColCb, yColCb) is set equal to the top-left sample of the collocated luma coding block specified by colCb relative to the top-left luma sample of the collocated picture specified by ColPic.
– The derivation process for collocated motion vectors as specified in clause 8.5.2.12 is invoked with currCb, colCb, (xColCb, yColCb) , refIdxLXCorner [3] and sbFlag set equal to 0 as inputs, and the output is assigned to mvLXCol and availableFlagLXCol.
– Otherwise, both components of mvLXCol are set equal to 0 and availableFlagLXCol is set equal to 0.
…
5.3 Embodiment 3: Fetching integer samples under the sub-picture constraint
8.5.6.3.3 Luma integer sample fetching process
Inputs to this process are:
– a luma location in full-sample units (xInt L, yInt L) ,
– the luma reference sample array refPicLX L,
Output of this process is a predicted luma sample value predSampleLX L
The variable shift is set equal to Max (2, 14-BitDepth Y) .
The variable picW is set equal to pic_width_in_luma_samples and the variable picH is set equal to pic_height_in_luma_samples.
The luma locations in full-sample units (xInt, yInt) are derived as follows:
– If subpic_treated_as_pic_flag [SubPicIdx] is equal to 1, the following applies:
xInt = Clip3 (SubPicLeftBoundaryPos, SubPicRightBoundaryPos, xInt)
yInt = Clip3 (SubPicTopBoundaryPos, SubPicBotBoundaryPos, yInt)
– Otherwise:
Figure PCTCN2020108159-appb-000024
yInt = Clip3 (0, picH-1, yInt L)         (8-783)
The predicted luma sample value predSampleLX L is derived as follows:
predSampleLX L = refPicLX L [xInt] [yInt] << shift3         (8-784)
5.4 Embodiment 4: deriving the variable invAvgLuma in chroma residual scaling of LMCS
8.7.5.3 Picture reconstruction with luma dependent chroma residual scaling process for chroma samples
Inputs to this process are:
– a chroma location (xCurr, yCurr) of the top-left chroma sample of the current chroma transform block relative to the top-left chroma sample of the current picture,
– a variable nCurrSw specifying the chroma transform block width,
– a variable nCurrSh specifying the chroma transform block height,
– a variable tuCbfChroma specifying the coded block flag of the current chroma transform block,
– an (nCurrSw) x (nCurrSh) array predSamples specifying the chroma prediction samples of the current block,
– an (nCurrSw) x (nCurrSh) array resSamples specifying the chroma residual samples of the current block,
Output of this process is a reconstructed chroma picture sample array recSamples.
The variable sizeY is set equal to Min (CtbSizeY, 64) .
The reconstructed chroma picture sample recSamples is derived as follows for i = 0.. nCurrSw-1, j = 0.. nCurrSh-1:
– …
– Otherwise, the following applies:
– …
– The variable currPic specifies the array of reconstructed luma samples in the current picture.
– For the derivation of the variable varScale the following ordered steps apply:
1. The variable invAvgLuma is derived as follows:
– The array recLuma [i] with i=0.. (2 *sizeY-1) and the variable cnt are derived as follows:
– The variable cnt is set equal to 0.
– The variablerightBoundaryPos and botBoundaryPos are derived as follows:
rightBoundaryPos = subpic_treated_as_pic_flag [SubPicIdx] ?
SubPicRightBoundaryPos : pic_width_in_luma_samples-1
botBoundaryPos = subpic_treated_as_pic_flag [SubPicIdx] ?
SubPicBotBoundaryPos : pic_height_in_luma_samples–1
– When availL is equal to TRUE, the array recLuma [i] with i = 0.. sizeY-1 is set equal to currPic [xCuCb-1] [Min (yCuCb + i, [ [pic_height_in_luma_samples –1] ]  botBoundaryPos) ] with i = 0.. sizeY-1, and cnt is set equal to sizeY
– When availT is equal to TRUE, the array recLuma [cnt + i] with i = 0.. sizeY –1 is set equal to currPic [Min (xCuCb + i, [ [pic_width_in_luma_samples-1] ]  rightBoundaryPos) ] [yCuCb -1] with i = 0.. sizeY-1, and cnt is set equal to (cnt + sizeY)
– The variable invAvgLuma is derived as follows:
– If cnt is greater than 0, the following applies:
Figure PCTCN2020108159-appb-000025
– Otherwise (cnt is equal to 0) , the following applies:
invAvgLuma = 1 << (BitDepth Y–1)           (8-1014)
5.5 Embodiment 5: an example of defining the subpicture element in unit of N (such as N=8 or 32) other than 4 samples
7.4.3.3 Sequence parameter set RBSP semantics
subpic_grid_col_width_minus1 plus 1 specifies the width of each element of the subpicture identifier grid in units of  4 N samples. The length of the syntax element is Ceil (Log2 (pic_width_max_in_luma_samples / 4 N) ) ) bits.
The variable NumSubPicGridCols is derived as follows:
NumSubPicGridCols = (pic_width_max_in_luma_samples + subpic_grid_col_width_minus1 * [ [4+ 3] ]  N  +N-1) / (subpic_grid_col_width_minus1 * [ [4+ 3] ]  N + N-1)            (7-5)
subpic_grid_row_height_minus1 plus 1 specifies the height of each element of the subpicture identifier grid in units of 4 samples. The length of the syntax element is Ceil (Log2 (pic_height_max_in_luma_samples / 4 N) ) bits.
The variable NumSubPicGridRows is derived as follows:
NumSubPicGridRows = (pic_height_max_in_luma_samples + subpic_grid_row_height_minus1 * 4 N + N-1) / (subpic_grid_row_height_minus1 * [ [4+ 3]  N + N-1)
7.4.7.1 General slice header semantics
The variables SubPicIdx, SubPicLeftBoundaryPos, SubPicTopBoundaryPos, SubPicRightBoundaryPos, and SubPicBotBoundaryPos are derived as follows:
Figure PCTCN2020108159-appb-000026
Figure PCTCN2020108159-appb-000027
5.6 Embodiment 6: restrict the picture width and the picture height to be equal or larger than 8
7.4.3.3 Sequence parameter set RBSP semantics
pic_width_max_in_luma_samples specifies the maximum width, in units of luma samples, of each decoded picture referring to the SPS. pic_width_max_in_luma_samples shall not be equal to 0 and shall be an integer multiple of [ [MinCbSizeY] ]  Max (8, MinCbSizeY) .
pic_height_max_in_luma_samples specifies the maximum height, in units of luma samples, of each decoded picture referring to the SPS. pic_height_max_in_luma_samples shall not be equal to 0 and shall be an integer multiple of [ [MinCbSizeY] ]  Max (8, MinCbSizeY) .
5.7 Embodiment 7: subpicture boundary check for BT/TT/QT splitting, BT/TT/QT depth derivation, and/or the signaling of CU split flag
6.4.2 Allowed binary split process
The variable allowBtSplit is derived as follows:
– …
– Otherwise, if all of the following conditions are true, allowBtSplit is set equal to FALSE
– btSplit is equal to SPLIT_BT_VER
– y0 + cbHeight is greater than [ [pic_height_in_luma_samples] ]  subpic_treated_as_pic_flag [SubPicIdx] ?  SubPicBotBoundaryPos + 1: pic_height_in_luma_samples.
– Otherwise, if all of the following conditions are true, allowBtSplit is set equal to FALSE
– btSplit is equal to SPLIT_BT_VER
– cbHeight is greater than MaxTbSizeY
– x0 + cbWidth is greater than [ [pic_width_in_luma_samples] ]  subpic_treated_as_pic_flag [SubPicIdx] ?  SubPicRightBoundaryPos + 1 : pic_width_in_luma_samples
– Otherwise, if all of the following conditions are true, allowBtSplit is set equal to FALSE
– btSplit is equal to SPLIT_BT_HOR
– cbWidth is greater than MaxTbSizeY
– y0 + cbHeight is greater than [ [pic_height_in_luma_samples] ]  subpic_treated_as_pic_flag [SubPicIdx] ?  SubPicBotBoundaryPos + 1: pic_height_in_luma_samples.
– Otherwise, if all of the following conditions are true, allowBtSplit is set equal to FALSE
– x0 + cbWidth is greater than [ [pic_width_in_luma_samples] ]  subpic_treated_as_pic_flag [SubPicIdx] ?  SubPicRightBoundaryPos + 1 : pic_width_in_luma_samples
– y0 + cbHeight is greater than [ [pic_height_in_luma_samples] ]  subpic_treated_as_pic_flag [SubPicIdx] ?  SubPicBotBoundaryPos + 1: pic_height_in_luma_samples.
– cbWidth is greater than minQtSize
– Otherwise, if all of the following conditions are true, allowBtSplit is set equal to FALSE
– btSplit is equal to SPLIT_BT_HOR
– x0 + cbWidth is greater than [ [pic_width_in_luma_samples] ]  subpic_treated_as_pic_flag [SubPicIdx] ?  SubPicRightBoundaryPos + 1 : pic_width_in_luma_samples
– y0 + cbHeight is less than or equal to [ [pic_height_in_luma_samples] ]  subpic_treated_as_pic_flag [SubPicIdx] ? SubPicBotBoundaryPos + 1: pic_height_in_luma_samples.
6.4.2 Allowed ternary split process
The variable allowTtSplit is derived as follows:
– If one or more of the following conditions are true, allowTtSplit is set equal to FALSE:
– cbSize is less than or equal to 2 *MinTtSizeY
– cbWidth is greater than Min (MaxTbSizeY, maxTtSize)
– cbHeight is greater than Min (MaxTbSizeY, maxTtSize)
– mttDepth is greater than or equal to maxMttDepth
– x0 + cbWidth is greater than [ [pic_width_in_luma_samples] ]  subpic_treated_as_pic_flag [SubPicIdx] ?  SubPicRightBoundaryPos + 1 : pic_width_in_luma_samples
– y0 + cbHeight is greater than [ [pic_height_in_luma_samples] ]  subpic_treated_as_pic_flag [SubPicIdx] ?  SubPicBotBoundaryPos + 1: pic_height_in_luma_samples.
– treeType is equal to DUAL_TREE_CHROMA and (cbWidth /SubWidthC) * (cbHeight /SubHeightC) is less than or equal to 32
– treeType is equal to DUAL_TREE_CHROMA and modeType is equal to MODE_TYPE_INTRA
– Otherwise, allowTtSplit is set equal to TRUE.
7.3.8.2 Coding tree unit syntax
Figure PCTCN2020108159-appb-000028
7.3.8.4 Coding tree syntax
Figure PCTCN2020108159-appb-000029
FIG. 3 is a block diagram of a video processing apparatus 300. The apparatus 300 may be used to implement one or more of the methods described herein. The apparatus 300 may be embodied in a smartphone, tablet, computer, Internet of Things (IoT) receiver, and so on. The apparatus 300 may include one or more processors 302, one or more memories 304 and video processing hardware 306. The processor (s) 302 may be configured to implement one or more methods described in the present document. The memory (memories) 304 may be used for storing data and code used for implementing the methods and techniques described herein. The video processing hardware 306 may be used to implement, in hardware circuitry, some techniques described in the present document.
FIG. 4 is a flowchart for a method 400 of processing a video. The method 1800 includes determining (402) , for a video block in a first video region of a video, whether a position at which a temporal motion vector predictor is determined for a conversion between the video block and a bitstream representation of the current video block using an affine mode is within a second video region, and performing (404) the conversion based on the determining.
The following solutions may be implemented as preferred solutions in some embodiments.
The following solutions may be implemented together with additional techniques described in items listed in the previous section (e.g., item 1) .
1. A method of video processing, comprising: determining, for a video block in a first video region of a video, whether a position at which a temporal motion vector predictor is determined for a conversion between the video block and a bitstream representation of the current video block using an affine mode is within a second video region; and performing the conversion based on the determining.
2. The method of solution 1, wherein the video block is covered by the first region and the second region.
3. The method of any of solutions 1-2, wherein, in case that the position of the temporal motion vector predictor is outside of the second video region, then the temporal motion vector predictor is marked as unavailable and is unused in the conversion.
The following solutions may be implemented together with additional techniques described in items listed in the previous section (e.g., item 2) .
4. A method of video processing, comprising: determining, for a video block in a first video region of a video, whether a position at which an integer sample in a reference picture is fetched for a conversion between the video block and a bitstream representation of the current video block is within a second video region, wherein the reference picture is not used in an interpolation process during the conversion; and performing the conversion based on the determining.
5. The method of solution 4, wherein the video block is covered by the first region and the second region.
6. The method of any of solutions 4-5, wherein, in case that the position of the sample is outside of the second video region, then the sample is marked as unavailable and is unused in the conversion.
The following solutions may be implemented together with additional techniques described in items listed in the previous section (e.g., item 3) .
7. A method of video processing, comprising: determining, for a video block in a first video region of a video, whether a position at which a reconstructed luma sample value is fetched for a conversion between the video block and a bitstream representation of the current video block is within a second video region; and performing the conversion based on the determining.
8. The method of solution 7, wherein the luma sample is covered by the first region and the second region.
9. The method of any of solutions 7-8, wherein, in case that the position of the luma sample is outside of the second video region, then the luma sample is marked as unavailable and is unused in the conversion.
The following solutions may be implemented together with additional techniques described in items listed in the previous section (e.g., item 4) .
10. A method of video processing, comprising: determining, for a video block in a first video region of a video, whether a position at which a check regarding splitting, depth derivation or split flag signaling for the video block is performed during a conversion between the video block and a bitstream representation of the current video block is within a second video region; and performing the conversion based on the determining.
11. The method of solution 10, wherein the position is covered by the first region and the second region.
12. The method of any of solutions 10-11, wherein, in case that the position is outside of the second video region, then the luma sample is marked as unavailable and is unused in the conversion.
The following solutions may be implemented together with additional techniques described in items listed in the previous section (e.g., item 8) .
13. A method of video processing, comprising: performing a conversion between a video comprising one or more video pictures comprising one or more video blocks, and a coded representation of the video, wherein the coded representation complies with a coding syntax requirement that the conversion is not to use sub-picture coding/decoding and a dynamic resolution conversion coding/decoding tool or a reference picture resampling tool within a video unit.
14. The method of solution 13, wherein the video unit corresponds to a sequence of the one or more video pictures.
15. The method of any of solutions 13-14, wherein the dynamic resolution conversion coding/decoding tool comprises an adaptive resolution conversion coding/decoding tool.
16. The method of any of solutions 13-14, wherein the dynamic resolution conversion coding/decoding tool comprises a dynamic resolution conversion coding/decoding tool.
17. The method of any of solutions 13-16, wherein the coded representation indicates that the video unit complies with the coding syntax requirement.
18. The method of solution 17, wherein the coded representation indicates that the video unit uses sub-picture coding.
19. The method of solution 17, wherein the coded representation indicates that the video unit uses the dynamic resolution conversion coding/decoding tool or the reference picture resampling tool.
The following solutions may be implemented together with additional techniques described in items listed in the previous section (e.g., item 10) .
20. The method of any of solutions 1-19, wherein the second video region comprises a video sub-picture and wherein boundaries of the second video region and another video region is also a boundary between two coding tree units.
21. The method of any of solutions 1-19, wherein the second video region comprises a video sub-picture and wherein boundaries of the second video region and another video region is also a boundary between two coding tree units.
The following solutions may be implemented together with additional techniques described in items listed in the previous section (e.g., item 11) .
22. The method of any of solutions 1-21, wherein the first video region and the second video region have rectangular shapes.
The following solutions may be implemented together with additional techniques described in items listed in the previous section (e.g., item 12) .
23. The method of any of solutions 1-22, wherein the first video region and the second video region are non-overlapping.
The following solutions may be implemented together with additional techniques described in items listed in the previous section (e.g., item 13) .
24. The method of any of solutions 1-23, wherein the video picture is divided into video regions such that a pixel in the video picture is covered by one and only one video region.
The following solutions may be implemented together with additional techniques described in items listed in the previous section (e.g., item 15) .
25. The method of any of solutions 1-24, wherein the video picture is split into the first video region and the second video region due to the video picture being in a specific layer of the video sequence.
The following solutions may be implemented together with additional techniques described in items listed in the previous section (e.g., item 10) .
26. A method of video processing, comprising: performing a conversion between a video comprising one or more video pictures comprising one or more video blocks, and a coded representation of the video, wherein the coded representation complies with a coding syntax requirement that a first syntax element subpic_grid_idx [i] [j] is not larger than a second syntax element max_subpics_minus1.
27. The method of solution 26, wherein a codeword representing the first syntax element is not larger than a codeword representing the second syntax element.
28. The method of any of solutions 1-27, wherein the first video region comprises a video sub-picture.
29. The method of any of solutions 1-28, wherein the second video region comprises a video sub-picture.
30. The method of any of solutions 1 to 29, wherein the conversion comprises encoding the video into the coded representation.
31. The method of any of solutions 1 to 29, wherein the conversion comprises decoding the coded representation to generate pixel values of the video.
32. A video decoding apparatus comprising a processor configured to implement a method recited in one or more of solutions 1 to 31.
33. A video encoding apparatus comprising a processor configured to implement a method recited in one or more of solutions 1 to 31.
34. A computer program product having computer code stored thereon, the code, when executed by a processor, causes the processor to implement a method recited in any of solutions 1 to 31.
35. A method, apparatus or system described in the present document.
FIG. 5 is a block diagram showing an example video processing system 500 in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of the system 500. The system 500 may include input 502 for receiving video content. The video content may be received in a raw or uncompressed format, e.g., 8 or 10 bit multi-component pixel values, or may be in a compressed or encoded format. The input 502 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interface include wired interfaces such as Ethernet, passive optical network (PON) , etc. and wireless interfaces such as Wi-Fi or cellular interfaces.
The system 500 may include a coding component 504 that may implement the various coding or encoding methods described in the present document. The coding component 504 may reduce the average bitrate of video from the input 502 to the output of the coding component 504 to produce a coded representation of the video. The coding techniques are therefore sometimes called video compression or video transcoding techniques. The output of the coding component 504 may be either stored, or transmitted via a communication connected, as represented by the component 506. The stored or communicated bitstream (or coded) representation of the video received at the input 502 may be used by the component 508 for generating pixel values or displayable video that is sent to a display interface 510. The process of generating user-viewable video from the bitstream representation is sometimes called video decompression. Furthermore, while certain video processing operations are referred to as “coding” operations or tools, it will  be appreciated that the coding tools or operations are used at an encoder and corresponding decoding tools or operations that reverse the results of the coding will be performed by a decoder.
Examples of a peripheral bus interface or a display interface may include universal serial bus (USB) or high definition multimedia interface (HDMI) or Displayport, and so on. Examples of storage interfaces include SATA (serial advanced technology attachment) , PCI, IDE interface, and the like. The techniques described in the present document may be embodied in various electronic devices such as mobile phones, laptops, smartphones or other devices that are capable of performing digital data processing and/or video display.
FIG. 6 is a flowchart representation of a method 600 for video processing in accordance with the present technology. The method 600 includes, at operation 610, determining, for a conversion between a current block of a first picture of a video and a bitstream representation of the video, a motion candidate based on motion information from a second picture according to a rule. The rule specifies that a position from which the motion information is accessed is constrained to be within a specific subpicture of the second picture. The method 600 also includes at operation 620, performing the conversion based on the determining.
In some embodiments, the motion information comprises a motion vector and the motion candidate comprises an affine motion candidate. In some embodiments, the rule specifies that the position is used as a replacement for a first position in case the first position is outside of the specific subpicture. In some embodiments, the position is at a bottom-right corner of a block in the video picture that corresponds to the current block.
FIG. 7 is a flowchart representation of a method 700 for video processing in accordance with the present technology. The method 700 includes, at operation 710, determining, for a conversion of a current block of a first picture of a video and a bitstream representation of the video, an integer sample from a second picture according to a rule. The second picture comprises a reference picture that is not used in an interpolation process. The rule specifies that a position from which the integer sample is accessed is constrained to be within a specific subpicture of the second picture. The method 700 also includes, at operation 720, performing the conversion based on the determining. In some embodiments, the integer sample is used to generate one or more gradients in a Bi-Directional Optical Flow or a Prediction Refinement Optical Flow process.
FIG. 8 is a flowchart representation of a method 800 for video processing in accordance  with the present technology. The method 800 includes, at operation 810, determining, for a conversion of a current block of a video and a bitstream representation of the video, a position at which a reconstructed luma sample is accessed according to a rule. The rule specifies that the position is constrained to be within a specific subpicture of a video picture. The method 800 also includes, at operation 820, performing the conversion based on the determining. In some embodiments, the reconstructed luma sample is accessed to derive a scaling factor for a chroma component in a luma mapping chroma scaling process.
FIG. 9 is a flowchart representation of a method 900 for video processing in accordance with the present technology. The method 900 includes, at operation 910, determining, for a conversion of a current block of a video and a bitstream representation of the video, a position at which a picture boundary check is performed according to a rule. The rule specifies that the position is constrained to be within a specific subpicture of a video picture. The method 900 also includes, at operation 920, performing the conversion based on the determining.
In some embodiments, the picture boundary check is performed for at least one of: (1) a splitting of a binary tree, a tertiary tree, or a quad tree, (2) a depth derivation for a binary tree, a tertiary tree, or a quad tree, (3) or a signaling of a split flag for the current block. In some embodiments, the specific subpicture is a collocated subpicture that covers the current block.
In some embodiments, the rule specifies that information at the position treated as unavailable in case the position is outside of a specific subpicture. The position is represented as (x, y) , a top-left corner of the specific subpicture is represented as (xTL, yTL) , and a bottom-right corner of the specific subpicture is represented as (xBR, yBR) . In some embodiments, the position is outside of the specific subpicture in case x > xBR, y > yBR, x < xTL, or y < yTL. In some embodiments, the rule specifies that the position is clipped to be within a specific subpicture of the video picture. The position is represented as (x, y) , a top-left corner of the specific subpicture is represented as (xTL, yTL) , and a bottom-right corner of the specific subpicture is represented as (xBR, yBR) . In some embodiments, x is clipped to be a smaller value of x and xBR. In some embodiments, y is clipped to be a smaller value of y and yBR. In some embodiments, x is clipped to be a larger value of x and xTL. In some embodiments, y is clipped to be a larger value of y and yTL.
In some embodiments, whether the rule is applicable is based on a syntax element in the bitstream representation. In some embodiments, the syntax element is signaled in a video  parameter set, a dependency parameter set, a slice parameter set, a picture parameter set, an active parameter set, a slice header, or a tile group header. In some embodiments, the syntax element comprises subpic_treated_as_pic_flag [SubPicIdx] , where SubPicIdx is a subpicture index of the specific subpicture that covers the current block.
In some embodiments, the conversion generates the current block from the bitstream representation. In some embodiments, the conversion generates the bitstream representation from the current block.
FIG. 10 is a flowchart representation of a method 1000 for video processing in accordance with the present technology. The method 1000 includes, at operation 1010, resetting, after a conversion of a sub-picture of a video picture of a video and a bitstream representation of the video, a table of motion candidates derived based on past conversions. The method 1000 also includes, at operation 1020, performing a conversion of a subsequent sub-picture of the video picture and the bitstream representation using the table after the resetting.
In some embodiments, the table of motion candidates comprises motion candidates for an intra-block coding mode. In some embodiments, the table of motion candidates comprises motion candidates for an inter coding mode. In some embodiments, the table of motion candidates comprises motion candidates for an intra coding mode.
In some embodiments, the conversion generates the sub-picture or the subsequent sub-picture from the bitstream representation. In some embodiments, the conversion generates the bitstream representation from the sub-picture or the subsequent sub-picture.
FIG. 11 is a flowchart representation of a method 1100 for video processing in accordance with the present technology. The method 1100 includes, at operation 1110, performing a conversion between a video comprising a video picture that includes multiple sub-pictures and multiple video blocks and a coded representation of the video according to a rule. The rule specifies that a boundary between any two sub-pictures is also a boundary between two video blocks. A video block in the video picture is covered by a single subpicture of the video picture.
In some embodiments, any position in the video picture is covered by at most one subpicture of the video picture. In some embodiments, a position in the video picture is not covered by any subpicture of the video picture. In some embodiments, the two sub-pictures of the video picture have no overlapping area.
In some embodiments, a dimension of a subpicture of the video picture is determined based on a dimension of a video block. In some embodiments, a sub-picture comprises multiple elements. A syntax element indicating a width of an element in the sub-picture is represented as N samples, and a width of the sub-picture is determined based on the N samples. In some embodiments, a width of the video block comprises N samples.
In some embodiments, a sub-picture comprises multiple elements. A syntax element indicating a height of an element in the sub-picture is represented as N samples, and a height of the sub-picture is determined based on the N samples. In some embodiments, a height of the video block comprises N samples.
In some embodiments, the video block is a coding tree block (CTB) or a coding tree unit (CTU) . In some embodiments, a sub-picture has a rectangular shape. In some embodiments, the sub-picture comprises rectangular slices. In some embodiments, a sub-picture is applicable to only selected one or more layers of the video. In some embodiments, the sub-picture is defined in a sequence parameter set in the bitstream representation, and the sub-picture is applicable to a layer with a highest resolution in a corresponding sequence. In some embodiments, the sub-picture is defined in a sequence parameter set in the bitstream representation, and the sub-picture is applicable to a layer with a lowest resolution in a corresponding sequence. In some embodiments, the selected one or more layers to which the sub-picture is applicable are signaled in one or more syntax elements in the bitstream representation. In some embodiments, one or more layers to which the sub-picture is inapplicable are signaled in one or more syntax elements in the bitstream representation. In some embodiments, the one or more syntax elements are signaled in a sequence parameter set, a video parameter set, or a picture parameter set in the bitstream representation.
FIG. 12 is a flowchart representation of a method 1200 for video processing in accordance with the present technology. The method 1200 includes, at operation 1210, performing a conversion between a video unit of a video and a coded representation of the video using at least a video picture, where only one of a sub-picture coding mode or a resolution-changing coding mode is enabled for the video unit. The sub-picture coding mode is a mode in which the video picture is divided into multiple sub-pictures, and the resolution-changing coding mode is a mode in which a resolution of the video picture is adjusted during the conversion.
In some embodiments, the video picture comprises a current picture or a reference picture.  In some embodiments, the resolution-changing coding mode comprises a Reference Picture Resampling (PRP) mode. In some embodiments, the resolution-changing coding mode comprises a Dynamic Resolution Conversion (DRC) mode. In some embodiments, the resolution-changing coding mode comprises an Adaptive Resolution Conversion (ARC) mode.
In some embodiments, the video unit comprises a video sequence. In some embodiments, a syntax element is included in the coded representation to indicate that the sub-picture coding mode is enabled for the coding unit in case the resolution-changing coding mode is disallowed. In some embodiments, the resolution-changing coding mode is disallowed in case a syntax element is included in the coded representation to indicate that the sub-picture coding mode is enabled. In some embodiments, the syntax element comprises subpics_present_flag. In some embodiments, a width of the video picture is set to be equal to a maximum width allowed for video pictures in the video unit in case the syntax element indicates that the sub-picture coding mode is enabled.
FIG. 13 is a flowchart representation of a method 1300 for video processing in accordance with the present technology. The method 1300 includes, at operation 1310, performing a conversion between a video unit of a video and a coded representation of the video using at least a video picture, where both a sub-picture coding mode and a resolution-changing coding mode are enabled for the video unit. The sub-picture coding mode is a mode in which the video picture is divided into multiple sub-pictures, and the resolution-changing coding mode is a mode in which a resolution of the video picture is adjusted during the conversion.
In some embodiments, the video unit comprises a video sequence. In some embodiments, the resolution-changing coding mode comprises an Adaptive Resolution Conversion (ARC) mode, a Dynamic Resolution Conversion (DRC) mode, a Reference Picture Resampling (PRP) mode.
In some embodiments, the video picture includes multiple video blocks, each having a dimension of W×H. A sub-picture adjusted according to the resolution-changing coding mode has a width of K×W and a height of M×H, K and M being integers. In some embodiments, the sub-picture is not located at a boundary of the video picture. In some embodiments, the boundary comprises a right boundary or a bottom boundary.
In some embodiments, the video picture includes multiple video blocks, and a dimension of individual video blocks is adjusted based on a resolution of the video picture. In some embodiments, the coded representation comprises a syntax element indicating a maximum dimension for a video block, and the dimension of an individual video block is adjusted based on  the maximum dimension and the resolution of the video picture. In some embodiments, the dimension of the individual video blocks is signaled in a sequence parameter set, a picture parameter set, or at a sub-picture level in the coded representation.
FIG. 14 is a flowchart representation of a method 1400 for video processing in accordance with the present technology. The method 1400 includes, at operation 1410, performing a conversion between a video comprising one or more video pictures and a coded representation of the video, where a dimension of an individual video picture is constrained to be greater than or equal to 8. In some embodiments, the dimension is a width of the individual video picture. In some embodiments, the dimension is a height of the individual video picture.
FIG. 15 is a flowchart representation of a method 1500 for video processing in accordance with the present technology. The method 1500 includes, at operation 1510, performing a conversion between a video picture of a video and a coded representation of the video according to a rule. The video picture comprises at least one sub-picture, and the rule specifies that a characteristic of a sub-picture is represented as at lease one syntax element in the coded representation, the at least one syntax element being different than an index value of the sub-picture grid in the video picture.
In some embodiments, the characteristic comprises a top-left position of the sub-picture. In some embodiments, the characteristic comprises a bottom-right position of the sub-picture. In some embodiments, the characteristic comprises a width of the sub-picture. In some embodiments, the characteristic comprises a height of the sub-picture. In some embodiments, the index value of the sub-picture in the video picture is smaller than a maximum number of subpictures in the video picture. In some embodiments, an integer value in a range of [0, the maximum number of subpictures-1] has a one-to-one correspondence with indices values of subpictures in the video picture.
FIG. 16 is a flowchart representation of a method 1600 for video processing in accordance with the present technology. The method 1600 includes, at operation 1610, performing a conversion between a video picture of a video and a coded representation of the video according to a rule. The video picture comprises multiple sub-pictures, each sub-picture comprising multiple elements. The rule specifies that a dimension of individual elements in a sub-picture satisfies a constraint.
In some embodiments, the constraint specifies that a width of an individual element is  smaller than T1. In some embodiments, the constraint specifies that a height of an individual element is smaller than T2. In some embodiments, the video picture comprises multiple video blocks, and the constraint specifies that the sub-picture determined based on the dimension of the individual elements is covered by a current video block being processed during the conversion. In some embodiments, a sample outside of the sub-picture covered by the current video block is considered as unavailable for the conversion. In some embodiments, the constraint specifies that the dimension of the individual elements is determined based on a profile, a level, or a tier of a video coding standard. In some embodiments, the constraint specifies that the dimension of the individual elements is determined based on a width of the video picture. In some embodiments, a width of the individual elements is equal to pic_width_max_in_luma_samples/4+offset, where pic_width_max_in_luma_samples represents a maximum picture width in luma samples and offset is zero or a non-zero integer. In some embodiments, the constraint specifies that the dimension of the individual elements is determined based on a height of the video picture. In some embodiments, a height of the individual elements is equal to pic_height_max_in_luma_samples/4+offset, where pic_height_max_in_luma_samples represents a maximum picture height in luma samples and offset is zero or a non-zero integer.
In some embodiments, the conversion generates video picture from the bitstream representation. In some embodiments, the conversion generates the bitstream representation from the video picture.
FIG. 17 is a flowchart representation of a method 1700 for video processing in accordance with the present technology. The method 1700 includes, at operation 1710, performing a conversion between a video comprising a picture that includes multiple sub-pictures and a coded representation of the video using a coding mode according to a rule. The rule specifies that certain stored information about a previous sub-picture is reset prior to processing each next sub-picture of the multiple sub-pictures.
In some embodiments, the certain stored information comprises a virtual buffer used for an intra-block copy coding mode in which a current block in the video picture is coded with samples from the video picture. In some embodiments, the certain stored information comprises a list of entries used for a palette coding mode in which a current block in the video picture is coded using a palette of representative sample values.
FIG. 18 is a flowchart representation of a method 1800 for video processing in accordance  with the present technology. The method 1800 includes, at operation 1810, performing a temporal filtering operation in a conversion between a video and a coded representation of the video according to a rule. The video comprises multiple video pictures, each comprising multiple sub-pictures. The rule specifies that, for a temporal filtering a current sample in a current sub-picture of a video picture, only samples within the same current sub-picture or a sub-picture in a different video picture corresponding to the current sub-picture are available.
FIG. 19 is a flowchart representation of a method 1900 for video processing in accordance with the present technology. The method 1900 includes, at operation 1910, determining, for a conversion between a block in a video picture of a video and a coded representation of the video, a manner of applying a partitioning method to the block based on whether the block crosses one or more sub-picture boundaries of the video picture. The method 1900 also includes, at operation 1920, performing the conversion based on the determining.
In some embodiments, the partitioning method comprising at least one of: a quad-tree partitioning method, a horizonal binary tree partitioning method, a vertical binary tree partitioning method, a horizontal tertiary tree partitioning method, a vertical tertiary tree partitioning method, or a no-splitting method. In some embodiments, the manner further specifies whether a boundary handling method is applicable to the block. In some embodiments, the method includes determining a manner of processing a syntax element in the coded representation indicating the partitioning method based on whether the block is located across the one or more boundaries of the sub-picture of the video picture.
FIG. 20 is a flowchart representation of a method 2000 for video processing in accordance with the present technology. The method 2000 includes, at operation 2010, determining, for a conversion between a video picture of a video and a coded representation of the video, two sub-regions of the video picture. A first sub-region comprises multiple sub-pictures of the video picture and a second sub-region comprises remaining samples in the video picture. The method 2000 also includes, at operation 2020, performing the conversion based on the determining.
In some embodiments, a sample in the second sub-region is not located in any sub-picture of the video picture. In some embodiments, the second sub-region is processed for the conversion based on information about the first sub-region. In some embodiments, a default value is used in the conversion to indicate whether a sample or an area of the video picture is located within the second sub-region. In some embodiments, the default value is set to  (max_subpics_minus1 + K) , wherein max_subpics_minus1 indicates a maximum number of sub-pictures in the video picture, and K is an integer greater than 1. In some embodiments, the default value is assigned to each element in an array of index values representing the sub-pictures in the video picture.
In some embodiments, the conversion generates the video from the coded representation. In some embodiments, the conversion generates the coded representation from the video.
Some embodiments of the disclosed technology include making a decision or determination to enable a video processing tool or mode. In an example, when the video processing tool or mode is enabled, the encoder will use or implement the tool or mode in the processing of a block of video, but may not necessarily modify the resulting bitstream based on the usage of the tool or mode. That is, a conversion from the block of video to the bitstream representation of the video will use the video processing tool or mode when it is enabled based on the decision or determination. In another example, when the video processing tool or mode is enabled, the decoder will process the bitstream with the knowledge that the bitstream has been modified based on the video processing tool or mode. That is, a conversion from the bitstream representation of the video to the block of video will be performed using the video processing tool or mode that was enabled based on the decision or determination.
Some embodiments of the disclosed technology include making a decision or determination to disable a video processing tool or mode. In an example, when the video processing tool or mode is disabled, the encoder will not use the tool or mode in the conversion of the block of video to the bitstream representation of the video. In another example, when the video processing tool or mode is disabled, the decoder will process the bitstream with the knowledge that the bitstream has not been modified using the video processing tool or mode that was enabled based on the decision or determination.
The disclosed and other solutions, examples, embodiments, modules and the functional operations described in this document can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this document and their structural equivalents, or in combinations of one or more of them. The disclosed and other embodiments can be implemented as one or more computer program products, e.g., one or more modules of computer program instructions encoded on a computer readable medium for  execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more them. The term “data processing apparatus” encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A propagated signal is an artificially generated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus.
A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document) , in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code) . A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.
The processes and logic flows described in this document can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit) .
Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a  read only memory or a random-access memory or both. The essential elements of a computer are a processor for performing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks. However, a computer need not have such devices. Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks; and CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.
While this patent document contains many specifics, these should not be construed as limitations on the scope of any subject matter or of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular techniques. Certain features that are described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.
Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Moreover, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.
Only a few implementations and examples are described and other implementations, enhancements and variations can be made based on what is described and illustrated in this patent document.

Claims (23)

  1. A method of video processing, comprising:
    performing a conversion between a video comprising a video picture that includes multiple sub-pictures and multiple video blocks, and a coded representation of the video according to a rule,
    wherein the rule specifies that a boundary between any two sub-pictures is also a boundary between two video blocks.
  2. The method of claim 1, wherein a video block in the video picture is covered by a single subpicture of the video picture.
  3. The method of claim 1 or 2, wherein any position in the video picture is covered by at most one subpicture of the video picture.
  4. The method of claim 1 or 2, wherein a position in the video picture is not covered by any subpicture of the video picture.
  5. The method of any of claims 1 to 4, wherein the two sub-pictures of the video picture have no overlapping area.
  6. The method of any of claims 1 to 5, wherein a dimension of a subpicture of the video picture is determined based on a dimension of a video block.
  7. The method of claim 6, wherein a sub-picture comprises multiple elements, wherein a syntax element indicating a width of an element in the sub-picture is represented as N samples, and wherein a width of the sub-picture is determined based on the N samples.
  8. The method of claim 6 or 7, wherein a width of the video block comprises N samples.
  9. The method of claim 6, wherein a sub-picture comprises multiple elements, wherein a syntax element indicating a height of an element in the sub-picture is represented as N samples, and wherein a height of the sub-picture is determined based on the N samples.
  10. The method of claim 6 or 9, wherein a height of the video block comprises N samples.
  11. The method of any of claims 6 to 10, wherein the video block is a coding tree block (CTB) or a coding tree unit (CTU) .
  12. The method of any of claims 1 to 11, wherein a sub-picture has a rectangular shape.
  13. The method of claim 12, wherein the sub-picture comprises rectangular slices.
  14. The method of any of claims 1 to 13, wherein a sub-picture is applicable to only selected one or more layers of the video.
  15. The method of claim 14, wherein the sub-picture is defined in a sequence parameter set in the bitstream representation, and wherein the sub-picture is applicable to a layer with a highest resolution in a corresponding sequence.
  16. The method of claim 14, wherein the sub-picture is defined in a sequence parameter set in the bitstream representation, and wherein the sub-picture is applicable to a layer with a lowest resolution in a corresponding sequence.
  17. The method of claim 14, wherein the selected one or more layers to which the sub-picture is applicable are signaled in one or more syntax elements in the bitstream representation.
  18. The method of claim 14, wherein one or more layers to which the sub-picture is inapplicable are signaled in one or more syntax elements in the bitstream representation.
  19. The method of claim 17 or 18, wherein the one or more syntax elements are signaled in a sequence parameter set, a video parameter set, or a picture parameter set in the bitstream representation.
  20. The method of any of claims 1 to 19, wherein the conversion generates video picture from the bitstream representation.
  21. The method of any of claims 1 to 19, wherein the conversion generates the bitstream representation from the video picture.
  22. A video processing apparatus comprising a processor configured to implement a method recited in any one or more of claims 1 to 21.
  23. A computer-readable medium having code stored thereon, the code, when executed, causing a processor to implement a method recited in any one or more of claims 1 to 21.
PCT/CN2020/108159 2019-08-10 2020-08-10 Subpicture size definition in video processing Ceased WO2021027773A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202080056646.5A CN114270831B (en) 2019-08-10 2020-08-10 Sub-picture size definition in video processing
US17/667,380 US11533513B2 (en) 2019-08-10 2022-02-08 Subpicture size definition in video processing

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CNPCT/CN2019/100114 2019-08-10
CN2019100114 2019-08-10

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/667,380 Continuation US11533513B2 (en) 2019-08-10 2022-02-08 Subpicture size definition in video processing

Publications (1)

Publication Number Publication Date
WO2021027773A1 true WO2021027773A1 (en) 2021-02-18

Family

ID=74570268

Family Applications (4)

Application Number Title Priority Date Filing Date
PCT/CN2020/108142 Ceased WO2021027767A1 (en) 2019-08-10 2020-08-10 Position restriction for inter coding mode
PCT/CN2020/108159 Ceased WO2021027773A1 (en) 2019-08-10 2020-08-10 Subpicture size definition in video processing
PCT/CN2020/108182 Ceased WO2021027776A1 (en) 2019-08-10 2020-08-10 Buffer management in subpicture decoding
PCT/CN2020/108175 Ceased WO2021027774A1 (en) 2019-08-10 2020-08-10 Subpicture dependent signaling in video bitstreams

Family Applications Before (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/108142 Ceased WO2021027767A1 (en) 2019-08-10 2020-08-10 Position restriction for inter coding mode

Family Applications After (2)

Application Number Title Priority Date Filing Date
PCT/CN2020/108182 Ceased WO2021027776A1 (en) 2019-08-10 2020-08-10 Buffer management in subpicture decoding
PCT/CN2020/108175 Ceased WO2021027774A1 (en) 2019-08-10 2020-08-10 Subpicture dependent signaling in video bitstreams

Country Status (8)

Country Link
US (5) US11553177B2 (en)
EP (3) EP3997869A4 (en)
JP (3) JP7478225B2 (en)
KR (2) KR20220042125A (en)
CN (5) CN114208196B (en)
MX (1) MX2022001465A (en)
MY (1) MY209640A (en)
WO (4) WO2021027767A1 (en)

Families Citing this family (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020233660A1 (en) 2019-05-21 2020-11-26 Beijing Bytedance Network Technology Co., Ltd. Syntax-based motion candidate derivation in sub-block merge mode
CN114208186B (en) 2019-07-25 2023-12-22 北京字节跳动网络技术有限公司 Intra-block copy virtual buffer size limit
EP3997869A4 (en) 2019-08-10 2022-10-26 Beijing Bytedance Network Technology Co., Ltd. SUBPICTURE DEPENDENT SIGNALING IN VIDEO STREAMS
KR20250016463A (en) * 2019-09-03 2025-02-03 엘지전자 주식회사 Method and device for encoding/decoding image using sub-picture, and bit stream transmission method
CN114342410B (en) 2019-09-05 2025-03-21 北京字节跳动网络技术有限公司 Range constraints of block vectors in intra block copy mode
GB201913403D0 (en) * 2019-09-17 2019-10-30 Canon Kk Method and apparatus for encoding and decoding a video stream with subpictures
JP7731348B2 (en) * 2019-09-19 2025-08-29 インターデジタル マディソン パテント ホールディングス， エスアーエス Systems and methods for versatile video encoding
KR102735189B1 (en) * 2019-09-23 2024-11-28 두인 비전 컴퍼니 리미티드 Setting up an intra-block copy virtual buffer based on a virtual pipeline data unit
CN119450036B (en) * 2019-09-24 2025-10-14 Oppo广东移动通信有限公司 Method and apparatus for sub-picture-based image encoding/decoding and method for transmitting bitstream
JP7322290B2 (en) 2019-10-02 2023-08-07 北京字節跳動網絡技術有限公司 Syntax for Subpicture Signaling in Video Bitstreams
CN114631321B (en) 2019-10-18 2024-04-12 北京字节跳动网络技术有限公司 Interaction between sub-pictures and loop filtering
CN115280774B (en) 2019-12-02 2025-08-19 抖音视界有限公司 Method, apparatus, and non-transitory computer readable storage medium for visual media processing
WO2021173552A1 (en) 2020-02-24 2021-09-02 Bytedance Inc. Interaction between subpicture and tile row signaling
EP4097980A4 (en) 2020-03-03 2023-04-19 ByteDance Inc. Low frequency non-separable transform signaling in video coding
WO2021185306A1 (en) 2020-03-18 2021-09-23 Beijing Bytedance Network Technology Co., Ltd. Intra block copy buffer and palette predictor update
KR20220157950A (en) 2020-03-23 2022-11-29 베이징 바이트댄스 네트워크 테크놀로지 컴퍼니, 리미티드 Prediction refinement for affine merge and affine motion vector prediction modes
US12309433B2 (en) * 2021-05-20 2025-05-20 Lemon Inc. On padding methods for neural network-based in-loop filter
US11949863B2 (en) * 2021-08-02 2024-04-02 Tencent America LLC Geometric partition mode with intra block copy
EP4138401A1 (en) * 2021-08-17 2023-02-22 Nokia Technologies Oy A method, an apparatus and a computer program product for video encoding and video decoding
CN118451705A (en) * 2021-09-28 2024-08-06 欧芬诺有限责任公司 Adaptive offset multi-reference line coding
WO2025132145A1 (en) * 2023-12-21 2025-06-26 Interdigital Ce Patent Holdings, Sas Temporal filtering strengh for reference picture resampling
US20250330569A1 (en) * 2024-01-07 2025-10-23 Alibaba (China) Co., Ltd. Cross-component prediction for chroma prediction

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3468190A1 (en) * 2016-06-30 2019-04-10 Huawei Technologies Co., Ltd. Coding and decoding method and apparatus

Family Cites Families (243)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6047112A (en) * 1992-06-30 2000-04-04 Discovision Associates Technique for initiating processing of a data stream of encoded video information
GB2382940A (en) 2001-11-27 2003-06-11 Nokia Corp Encoding objects and background blocks
FI114679B (en) * 2002-04-29 2004-11-30 Nokia Corp Direct access points for video coding
US20040177383A1 (en) 2003-01-24 2004-09-09 Chyron Corporation Embedded graphics metadata
JP3967691B2 (en) * 2003-03-31 2007-08-29 株式会社東芝 Information storage medium, information reproducing apparatus, and information recording / reproducing apparatus
KR100983268B1 (en) * 2003-07-21 2010-09-24 엘지전자 주식회사 How to display subpictures during trick play
CN100502511C (en) 2004-09-14 2009-06-17 华为技术有限公司 Method for organizing interpolation image memory for fractional pixel precision predication
CN101317459A (en) 2005-10-11 2008-12-03 诺基亚公司 Efficient decoded picture buffer management for scalable video coding
US7688333B2 (en) 2007-02-28 2010-03-30 Lsi Corporation Method and/or apparatus for color space reduction for transcoding subpicture elements
JP5232854B2 (en) 2007-04-26 2013-07-10 ポリコム，インク． Deblocking filter array
KR100891768B1 (en) * 2007-05-02 2009-04-07 삼성전자주식회사 Motion vector navigation method
CN101668219B (en) * 2008-09-02 2012-05-23 华为终端有限公司 Communication method, transmitting equipment and system for 3D video
WO2010041857A2 (en) 2008-10-06 2010-04-15 Lg Electronics Inc. A method and an apparatus for decoding a video signal
US8699562B2 (en) * 2008-10-06 2014-04-15 Lg Electronics Inc. Method and an apparatus for processing a video signal with blocks in direct or skip mode
KR20150013776A (en) 2010-04-09 2015-02-05 미쓰비시덴키 가부시키가이샤 Video encoding device and video decoding device
US8988531B2 (en) 2010-07-08 2015-03-24 Texas Instruments Incorporated Method and apparatus for sub-picture based raster scanning coding order
US10327008B2 (en) 2010-10-13 2019-06-18 Qualcomm Incorporated Adaptive motion vector resolution signaling for video coding
CN101990103B (en) 2010-11-10 2013-01-16 香港应用科技研究院有限公司 Method and device for multi-view video coding
US20120294353A1 (en) 2011-05-16 2012-11-22 Mediatek Inc. Apparatus and Method of Sample Adaptive Offset for Luma and Chroma Components
US9641866B2 (en) 2011-08-18 2017-05-02 Qualcomm Incorporated Applying partition-based filters
CN107483928B (en) 2011-09-09 2020-05-12 株式会社Kt Method for decoding video signal
US10034018B2 (en) 2011-09-23 2018-07-24 Velos Media, Llc Decoded picture buffer management
US9699457B2 (en) 2011-10-11 2017-07-04 Qualcomm Incorporated Most probable transform for intra prediction coding
US9807403B2 (en) 2011-10-21 2017-10-31 Qualcomm Incorporated Adaptive loop filtering for chroma components
US9584819B2 (en) 2011-10-24 2017-02-28 Qualcomm Incorporated Grouping of tiles for video coding
CN103891292B (en) 2011-10-24 2018-02-02 寰发股份有限公司 Video data loop filtering processing method and device
US9253481B2 (en) 2012-01-13 2016-02-02 Qualcomm Incorporated Determining contexts for coding transform coefficient data in video coding
US9451252B2 (en) * 2012-01-14 2016-09-20 Qualcomm Incorporated Coding parameter sets and NAL unit headers for video coding
US9332259B2 (en) 2012-01-18 2016-05-03 Qualcomm Incorporated Indication of use of wavefront parallel processing in video coding
US9538200B2 (en) 2012-01-19 2017-01-03 Qualcomm Incorporated Signaling of deblocking filter parameters in video coding
US10244246B2 (en) * 2012-02-02 2019-03-26 Texas Instruments Incorporated Sub-pictures for pixel rate balancing on multi-core platforms
US9967563B2 (en) 2012-02-03 2018-05-08 Hfi Innovation Inc. Method and apparatus for loop filtering cross tile or slice boundaries
US9525861B2 (en) 2012-03-14 2016-12-20 Qualcomm Incorporated Disparity vector prediction in video coding
US10200709B2 (en) 2012-03-16 2019-02-05 Qualcomm Incorporated High-level syntax extensions for high efficiency video coding
US9565431B2 (en) 2012-04-04 2017-02-07 Qualcomm Incorporated Low-delay video buffering in video coding
US20130294524A1 (en) 2012-05-04 2013-11-07 Qualcomm Incorporated Transform skipping and lossless coding unification
US9762903B2 (en) 2012-06-01 2017-09-12 Qualcomm Incorporated External pictures in video coding
US20130336406A1 (en) 2012-06-14 2013-12-19 Qualcomm Incorporated Redundancy removal for merge/skip mode motion information candidate list construction
US9838685B2 (en) 2012-06-15 2017-12-05 Google Technology Holdings LLC Method and apparatus for efficient slice header processing
US9635369B2 (en) 2012-07-02 2017-04-25 Qualcomm Incorporated Video parameter set including HRD parameters
US10110890B2 (en) * 2012-07-02 2018-10-23 Sony Corporation Video coding system with low delay and method of operation thereof
US20140003504A1 (en) * 2012-07-02 2014-01-02 Nokia Corporation Apparatus, a Method and a Computer Program for Video Coding and Decoding
US10021394B2 (en) 2012-09-24 2018-07-10 Qualcomm Incorporated Hypothetical reference decoder parameters in video coding
SG11201501400QA (en) 2012-09-26 2015-04-29 Panasonic Ip Corp America Image decoding method, image coding method, image decoding apparatus, image coding apparatus, and image coding and decoding apparatus
US8989508B2 (en) 2012-09-28 2015-03-24 Sharp Kabushiki Kaisha Electronic device for signaling a sub-picture buffer parameter
US9491457B2 (en) 2012-09-28 2016-11-08 Qualcomm Incorporated Signaling of regions of interest and gradual decoding refresh in video coding
CN104718760B (en) 2012-10-05 2019-04-05 寰发股份有限公司 Method and apparatus for three-dimensional and multi-view video coding
US20150304678A1 (en) 2012-12-26 2015-10-22 Sony Corporation Image processing device and method
US9674542B2 (en) 2013-01-02 2017-06-06 Qualcomm Incorporated Motion vector prediction for video coding
CN116320392A (en) 2013-01-04 2023-06-23 Ge视频压缩有限责任公司 Efficient Scalable Coding Concept
KR101825575B1 (en) * 2013-01-07 2018-02-05 노키아 테크놀로지스 오와이 Method and apparatus for video coding and decoding
US20140218473A1 (en) * 2013-01-07 2014-08-07 Nokia Corporation Method and apparatus for video coding and decoding
WO2014107853A1 (en) 2013-01-09 2014-07-17 Mediatek Singapore Pte. Ltd. Methods for disparity vector derivation
US9445130B2 (en) 2013-01-09 2016-09-13 Qualcomm Incorporated Blockiness metric for large HEVC block artifacts
WO2014128197A1 (en) * 2013-02-20 2014-08-28 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Apparatus and method for encoding or decoding an audio signal using a transient-location dependent overlap
US9467707B2 (en) * 2013-03-05 2016-10-11 Qualcomm Incorporated Parallel processing for video coding
JP6149151B2 (en) 2013-04-02 2017-06-14 ヴィド スケール インコーポレイテッド Extended temporal motion vector prediction for scalable video coding
US9591321B2 (en) 2013-04-07 2017-03-07 Dolby International Ab Signaling change in output layer sets
US9565437B2 (en) 2013-04-08 2017-02-07 Qualcomm Incorporated Parameter set designs for video coding extensions
US10015515B2 (en) 2013-06-21 2018-07-03 Qualcomm Incorporated Intra prediction from a predictive block
US9648326B2 (en) 2013-07-02 2017-05-09 Qualcomm Incorporated Optimizations on inter-layer prediction signalling for multi-layer video coding
JP2015015575A (en) 2013-07-04 2015-01-22 シャープ株式会社 Image decoding apparatus, image encoding apparatus, image decoding method, image encoding method, image decoding program, and image encoding program
EP3020186B1 (en) 2013-07-09 2022-12-07 Nokia Technologies Oy Method and apparatus for video coding involving syntax for signalling motion information
US20160173887A1 (en) 2013-07-10 2016-06-16 Sharp Kabushiki Kaisha Scaling list signaling and parameter sets activation
WO2015003383A1 (en) 2013-07-12 2015-01-15 Mediatek Singapore Pte. Ltd. Methods for inter-view motion prediction
WO2015008479A1 (en) 2013-07-14 2015-01-22 Sharp Kabushiki Kaisha Signaling indications and constraints
CN105359525A (en) * 2013-07-14 2016-02-24 夏普株式会社 Tile alignment signaling and consistency constraints
GB2516424A (en) 2013-07-15 2015-01-28 Nokia Corp A method, an apparatus and a computer program product for video coding and decoding
WO2015006951A1 (en) 2013-07-18 2015-01-22 Mediatek Singapore Pte. Ltd. Methods for fast encoder decision
GB2516824A (en) * 2013-07-23 2015-02-11 Nokia Corp An apparatus, a method and a computer program for video coding and decoding
US20150078457A1 (en) 2013-09-13 2015-03-19 Qualcomm Incorporated Representation format signaling in multi-layer video coding
CN105556975A (en) 2013-10-11 2016-05-04 夏普株式会社 Signaling informs information for encoding
US20150103924A1 (en) 2013-10-13 2015-04-16 Sharp Laboratories Of America, Inc. On operation of decoded picture buffer for interlayer pictures
WO2015056941A1 (en) * 2013-10-14 2015-04-23 한국전자통신연구원 Multilayer-based image encoding/decoding method and apparatus
KR20150057591A (en) 2013-11-20 2015-05-28 주식회사 디오텍 Method and apparatus for controlling playing video
WO2015101716A1 (en) * 2014-01-03 2015-07-09 Nokia Technologies Oy Parameter set coding
US10567804B2 (en) 2014-01-08 2020-02-18 Qualcomm Incorporated Carriage of HEVC extension bitstreams and buffer model with MPEG-2 systems
US10531116B2 (en) * 2014-01-09 2020-01-07 Qualcomm Incorporated Adaptive motion vector resolution signaling for video coding
US20150271515A1 (en) * 2014-01-10 2015-09-24 Qualcomm Incorporated Block vector coding for intra block copy in video coding
US20150215631A1 (en) 2014-01-23 2015-07-30 Broadcom Corporation Parallel Coding with Overlapped Tiles
JP6280679B2 (en) * 2014-02-24 2018-02-14 日本放送協会 Frame thinning device, frame interpolation device, video encoding device, video decoding device, and programs thereof
KR102355224B1 (en) 2014-03-16 2022-01-25 브이아이디 스케일, 인크. Method and apparatus for the signaling of lossless video coding
EP3120553B1 (en) * 2014-03-17 2022-02-23 Nokia Technologies Oy An apparatus, a method and a computer program for video coding and decoding
CN108632629B9 (en) 2014-03-19 2021-06-15 株式会社Kt Method of generating merge candidate list for multi-view video signal and decoding apparatus
US10390087B2 (en) 2014-05-01 2019-08-20 Qualcomm Incorporated Hypothetical reference decoder parameters for partitioning schemes in video coding
US10750198B2 (en) 2014-05-22 2020-08-18 Qualcomm Incorporated Maximum palette parameters in palette-based video coding
US10477204B2 (en) 2014-06-20 2019-11-12 Sharp Kabushiki Kaisha Harmonized palette coding
WO2016008157A1 (en) 2014-07-18 2016-01-21 Mediatek Singapore Pte. Ltd. Methods for motion compensation using high order motion model
WO2016048834A1 (en) 2014-09-26 2016-03-31 Vid Scale, Inc. Intra block copy coding with temporal block vector prediction
CA2961681C (en) 2014-09-30 2022-08-09 Hfi Innovation Inc. Method of adaptive motion vetor resolution for video coding
EP3206400A4 (en) 2014-10-06 2018-05-16 Sony Corporation Image processing device and method
US9800898B2 (en) 2014-10-06 2017-10-24 Microsoft Technology Licensing, Llc Syntax structures indicating completion of coded regions
US9918105B2 (en) * 2014-10-07 2018-03-13 Qualcomm Incorporated Intra BC and inter unification
GB2564731B (en) 2014-10-14 2019-05-29 Canon Kk Description of image composition with HEVC still image file format
CN106797229B (en) * 2014-11-20 2019-06-21 寰发股份有限公司 Video coding method
KR102150979B1 (en) 2014-12-19 2020-09-03 에이치에프아이 이노베이션 인크. Methods of palette based prediction for non-444 color format in video and image coding
US11477477B2 (en) 2015-01-26 2022-10-18 Qualcomm Incorporated Sub-prediction unit based advanced temporal motion vector prediction
CN107211122B (en) 2015-01-29 2020-05-05 佳能株式会社 Palette predictor initialization program for encoding or decoding self-contained coding structure
US20160234522A1 (en) * 2015-02-05 2016-08-11 Microsoft Technology Licensing, Llc Video Decoding
CN104702963B (en) 2015-02-13 2017-11-24 北京大学 The boundary processing method and device of a kind of adaptive loop filter
CN107534783B (en) 2015-02-13 2020-09-08 联发科技股份有限公司 A palette index map encoding and decoding method for blocks in an image
SG10201900582XA (en) * 2015-03-02 2019-02-27 Hfi Innovation Inc Method and apparatus for intrabc mode with fractional-pel block vector resolution in video coding
US11677967B2 (en) * 2015-04-21 2023-06-13 Arris Enterprises Llc Scalable video coding system with parameter signaling
US10200713B2 (en) * 2015-05-11 2019-02-05 Qualcomm Incorporated Search region determination for inter coding within a particular picture of video data
CN109005407B (en) 2015-05-15 2023-09-01 华为技术有限公司 Video image encoding and decoding method, encoding device and decoding device
JP6642573B2 (en) 2015-06-24 2020-02-05 ソニー株式会社 Encoding device, decoding device, and transmission control method
JP2017017629A (en) * 2015-07-03 2017-01-19 富士通株式会社 Moving picture coding apparatus, moving picture coding method, and program
CN107852490B (en) * 2015-07-27 2021-01-26 联发科技股份有限公司 Video coding and decoding method and system
WO2017029400A1 (en) 2015-08-20 2017-02-23 Koninklijke Kpn N.V. Forming one or more tile streams on the basis of one or more video streams
US10375413B2 (en) 2015-09-28 2019-08-06 Qualcomm Incorporated Bi-directional optical flow for video coding
CN109274974B (en) 2015-09-29 2022-02-11 华为技术有限公司 Method and device for image prediction
WO2017083784A1 (en) 2015-11-11 2017-05-18 Apple Inc. Adaptive chroma downsampling and color space conversion techniques
US20190158870A1 (en) 2016-01-07 2019-05-23 Mediatek Inc. Method and apparatus for affine merge mode prediction for video coding system
MY201069A (en) 2016-02-05 2024-02-01 Hfi Innovation Inc Method and apparatus of motion compensation based on bi-directional optical flow techniques for video coding
TWI669946B (en) * 2016-02-09 2019-08-21 弗勞恩霍夫爾協會 Technology for image/video data streaming that allows for efficient scalability or efficient random access
US10349067B2 (en) 2016-02-17 2019-07-09 Qualcomm Incorporated Handling of end of bitstream NAL units in L-HEVC file format and improvements to HEVC and L-HEVC tile tracks
WO2017147765A1 (en) 2016-03-01 2017-09-08 Mediatek Inc. Methods for affine motion compensation
EP3414900B1 (en) 2016-03-15 2025-08-06 HFI Innovation Inc. Method and apparatus of video coding with affine motion compensation
US20170272758A1 (en) 2016-03-16 2017-09-21 Mediatek Inc. Video encoding method and apparatus using independent partition coding and associated video decoding method and apparatus
US11095898B2 (en) 2016-03-28 2021-08-17 Lg Electronics Inc. Inter-prediction mode based image processing method, and apparatus therefor
CN116170588A (en) 2016-03-30 2023-05-26 韩国电子通信研究院 Method and device for encoding and decoding video using picture division information
US10834419B2 (en) 2016-04-13 2020-11-10 Qualcomm Incorporated Conformance constraint for collocated reference index in video coding
KR102873107B1 (en) 2016-04-25 2025-10-17 엘지전자 주식회사 Inter prediction method and device in a video coding system
US10560718B2 (en) 2016-05-13 2020-02-11 Qualcomm Incorporated Merge candidates for motion vector prediction for video coding
US10560712B2 (en) 2016-05-16 2020-02-11 Qualcomm Incorporated Affine motion prediction for video coding
US10419755B2 (en) 2016-05-16 2019-09-17 Qualcomm Incorporated Confusion of multiple filters in adaptive loop filtering in video coding
US11089323B2 (en) 2016-05-28 2021-08-10 Mediatek Inc. Method and apparatus of current picture referencing for video coding
CN107801039B (en) * 2016-08-30 2020-04-14 华为技术有限公司 Motion compensation prediction method and device
US10419757B2 (en) 2016-08-31 2019-09-17 Qualcomm Incorporated Cross-component filter
US10462462B2 (en) 2016-09-29 2019-10-29 Qualcomm Incorporated Motion vector difference coding technique for video coding
US20180098090A1 (en) 2016-10-04 2018-04-05 Mediatek Inc. Method and Apparatus for Rearranging VR Video Format and Constrained Encoding Parameters
US10448010B2 (en) 2016-10-05 2019-10-15 Qualcomm Incorporated Motion vector prediction for affine motion models in video coding
CN109983773A (en) 2016-11-29 2019-07-05 联发科技股份有限公司 The video coding-decoding method and device derived for merging patterns
AU2016273973A1 (en) * 2016-12-16 2018-07-05 Canon Kabushiki Kaisha Transcode PCL delta-row compressed image to edges
WO2018113658A1 (en) 2016-12-22 2018-06-28 Mediatek Inc. Method and apparatus of motion refinement for video coding
US10911761B2 (en) 2016-12-27 2021-02-02 Mediatek Inc. Method and apparatus of bilateral template MV refinement for video coding
WO2018125972A1 (en) 2016-12-28 2018-07-05 Arris Enterprises Llc Adaptive unequal weight planar prediction
US10931969B2 (en) 2017-01-04 2021-02-23 Qualcomm Incorporated Motion vector reconstructions for bi-directional optical flow (BIO)
US20180199057A1 (en) 2017-01-12 2018-07-12 Mediatek Inc. Method and Apparatus of Candidate Skipping for Predictor Refinement in Video Coding
US10523964B2 (en) 2017-03-13 2019-12-31 Qualcomm Incorporated Inter prediction refinement based on bi-directional optical flow (BIO)
US10701390B2 (en) 2017-03-14 2020-06-30 Qualcomm Incorporated Affine motion information derivation
US20180310017A1 (en) 2017-04-21 2018-10-25 Mediatek Inc. Sub-prediction unit temporal motion vector prediction (sub-pu tmvp) for video coding
KR102435881B1 (en) 2017-05-26 2022-08-24 에스케이텔레콤 주식회사 Apparatus and Method for Video Encoding or Decoding
CN107277316B (en) 2017-06-12 2020-05-12 京东方科技集团股份有限公司 Display device and method of manufacturing the same
US10904565B2 (en) 2017-06-23 2021-01-26 Qualcomm Incorporated Memory-bandwidth-efficient design for bi-directional optical flow (BIO)
EP3422724B1 (en) * 2017-06-26 2024-05-01 Nokia Technologies Oy An apparatus, a method and a computer program for omnidirectional video
US11184636B2 (en) 2017-06-28 2021-11-23 Sharp Kabushiki Kaisha Video encoding device and video decoding device
EP3649780B1 (en) 2017-07-03 2025-06-25 InterDigital VC Holdings, Inc. Motion-compensation prediction based on bi-directional optical flow
WO2019008174A1 (en) 2017-07-06 2019-01-10 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Portioned video streaming concepts
US10483493B2 (en) 2017-08-01 2019-11-19 Apple Inc. Electronic device having display with thin-film encapsulation
US10856003B2 (en) 2017-10-03 2020-12-01 Qualcomm Incorporated Coding affine prediction motion information for video coding
WO2019073112A1 (en) * 2017-10-09 2019-04-18 Nokia Technologies Oy An apparatus, a method and a computer program for video coding and decoding
US11877001B2 (en) 2017-10-10 2024-01-16 Qualcomm Incorporated Affine prediction in video coding
US20190116376A1 (en) 2017-10-12 2019-04-18 Qualcomm Incorporated Motion vector predictors using affine motion model in video coding
JP2021016016A (en) 2017-10-20 2021-02-12 シャープ株式会社 Video encoding device and video decoding device
US10776688B2 (en) 2017-11-06 2020-09-15 Nvidia Corporation Multi-frame video interpolation using optical flow
US20190141320A1 (en) 2017-11-07 2019-05-09 Qualcomm Incorporated Enhanced reference picture management in video coding
US11889100B2 (en) 2017-11-14 2024-01-30 Qualcomm Incorporated Affine motion vector prediction in video coding
CN109996072B (en) * 2018-01-03 2021-10-15 华为技术有限公司 Video image processing method and device
WO2019136657A1 (en) 2018-01-11 2019-07-18 Qualcomm Incorporated Video coding using local illumination compensation
US10757417B2 (en) 2018-01-20 2020-08-25 Qualcomm Incorporated Affine motion compensation in video coding
KR102769888B1 (en) 2018-01-25 2025-02-20 프라운호퍼 게젤샤프트 쭈르 푀르데룽 데어 안겐반텐 포르슝 에. 베. Efficient sub-picture extraction
US11012715B2 (en) * 2018-02-08 2021-05-18 Qualcomm Incorporated Intra block copy for video coding
CA3092638A1 (en) 2018-03-01 2019-09-06 Arris Enterprises Llc System and method of motion information storage for video coding and signaling
KR20250020687A (en) 2018-04-01 2025-02-11 광동 오포 모바일 텔레커뮤니케이션즈 코포레이션 리미티드 Image coding method based on affine motion prediction, and device for same
KR20190116101A (en) 2018-04-03 2019-10-14 인텔렉추얼디스커버리 주식회사 Method and apparatus for image encoding/decoding based on affine model
JP2021520710A (en) 2018-04-06 2021-08-19 ヴィド スケール インコーポレイテッド Bidirectional optical flow method with simplified gradient derivation
US11457233B2 (en) 2018-04-12 2022-09-27 Arris Enterprises Llc Motion information storage for video coding and signaling
US10873748B2 (en) 2018-05-12 2020-12-22 Qualcomm Incorporated Storage of high precision motion vectors in video coding
CN110636298B (en) 2018-06-21 2022-09-13 北京字节跳动网络技术有限公司 Unified constraints for Merge affine mode and non-Merge affine mode
WO2020005719A1 (en) 2018-06-27 2020-01-02 Vid Scale, Inc. Methods and apparatus for reducing the coding latency of decoder-side motion refinement
CN110662037B (en) 2018-06-29 2022-06-28 北京字节跳动网络技术有限公司 Limitations on Athletic Information Sharing
CA3105072A1 (en) 2018-06-29 2020-01-02 Vid Scale, Inc. Adaptive control point selection for affine motion model based video coding
CN110662046B (en) 2018-06-29 2022-03-25 北京字节跳动网络技术有限公司 Video processing method, device and readable storage medium
CN110677679B (en) 2018-07-01 2022-07-26 北京字节跳动网络技术有限公司 Shape dependent intra coding
CN110677675B (en) 2018-07-01 2022-02-08 北京字节跳动网络技术有限公司 Method, device and storage medium for efficient affine Merge motion vector derivation
CN110677669B (en) 2018-07-02 2021-12-07 北京字节跳动网络技术有限公司 LUT with LIC
KR102545728B1 (en) 2018-07-16 2023-06-20 엘지전자 주식회사 Inter prediction method and apparatus for predicting temporal motion information in units of subblocks
WO2020017892A1 (en) 2018-07-17 2020-01-23 엘지전자 주식회사 Method for predicting subblock-based temporal motion vector and apparatus therefor
GB2590228B (en) 2018-08-04 2023-04-05 Beijing Bytedance Network Tech Co Ltd Clipping of updated MV or derived MV
JP7212150B2 (en) 2018-09-19 2023-01-24 北京字節跳動網絡技術有限公司 Using Syntax for Affine Modes with Adaptive Motion Vector Resolution
KR20210058947A (en) 2018-09-20 2021-05-24 샤프 가부시키가이샤 Systems and methods for signaling parameters in video coding
CN118474363A (en) 2018-09-20 2024-08-09 韩国电子通信研究院 Image encoding/decoding method and apparatus, and recording medium storing bit stream
GB2577318B (en) 2018-09-21 2021-03-10 Canon Kk Video coding and decoding
WO2020058962A1 (en) 2018-09-23 2020-03-26 Beijing Bytedance Network Technology Co., Ltd. Signaling of motion vector precision indication with adaptive motion vector resolution
WO2020058959A1 (en) 2018-09-23 2020-03-26 Beijing Bytedance Network Technology Co., Ltd. Motion vector derivation for sub-block in affine mode
WO2020058955A1 (en) 2018-09-23 2020-03-26 Beijing Bytedance Network Technology Co., Ltd. Multiple-hypothesis affine mode
TW202029755A (en) 2018-09-26 2020-08-01 美商Ｖｉｄ衡器股份有限公司 Bi-prediction for video coding
US10999589B2 (en) 2018-10-04 2021-05-04 Tencent America LLC Method and apparatus for video coding
JP7230189B2 (en) 2018-10-08 2023-02-28 エルジー エレクトロニクス インコーポレイティド Syntax design method and device for coding using syntax
WO2020084470A1 (en) 2018-10-22 2020-04-30 Beijing Bytedance Network Technology Co., Ltd. Storage of motion parameters with clipping for affine mode
WO2020084461A1 (en) 2018-10-22 2020-04-30 Beijing Bytedance Network Technology Co., Ltd. Restrictions on decoder side motion vector derivation based on coding information
CN111083489B (en) 2018-10-22 2024-05-14 北京字节跳动网络技术有限公司 Multiple iteration motion vector refinement
CN112913247B (en) 2018-10-23 2023-04-28 北京字节跳动网络技术有限公司 Video processing using local illumination compensation
EP4300965A3 (en) 2018-11-05 2024-01-17 Beijing Bytedance Network Technology Co., Ltd. Interpolation for inter prediction with refinement
WO2020094057A1 (en) 2018-11-06 2020-05-14 Beijing Bytedance Network Technology Co., Ltd. Position based intra prediction
CN109600611B (en) 2018-11-09 2021-07-13 北京达佳互联信息技术有限公司 Loop filtering method, loop filtering device, electronic device and readable medium
WO2020098802A1 (en) 2018-11-15 2020-05-22 Beijing Bytedance Network Technology Co., Ltd. Merge with mvd for affine
CN113170105B (en) 2018-11-16 2024-11-05 北京字节跳动网络技术有限公司 History-based affine parameter inheritance
CN113039780B (en) 2018-11-17 2023-07-28 北京字节跳动网络技术有限公司 Merge using motion vector difference in video processing
WO2020103877A1 (en) 2018-11-20 2020-05-28 Beijing Bytedance Network Technology Co., Ltd. Coding and decoding of video coding modes
US11146810B2 (en) 2018-11-27 2021-10-12 Qualcomm Incorporated Decoder-side motion vector refinement
CN113170181B (en) 2018-11-29 2023-12-08 北京字节跳动网络技术有限公司 Affine inheritance methods in intra-block copy mode
WO2020140221A1 (en) 2019-01-02 2020-07-09 北京大学 Candidate mv list construction method and device
US11778171B2 (en) 2019-01-02 2023-10-03 Nokia Technologies Oy Apparatus, a method and a computer program for video coding and decoding
US11627310B2 (en) 2019-01-02 2023-04-11 Lg Electronics Inc. Affine motion prediction-based video decoding method and device using subblock-based temporal merge candidate in video coding system
PL4221213T3 (en) 2019-01-09 2024-11-18 Huawei Technologies Co., Ltd. A video encoder, a video decoder and corresponding methods
WO2020156576A1 (en) 2019-02-02 2020-08-06 Beijing Bytedance Network Technology Co., Ltd. Multi-hmvp for affine
GB201902829D0 (en) 2019-03-01 2019-04-17 Canon Kk Method and apparatus for encoding and decoding a video bitsream for merging regions of interest
KR102825177B1 (en) 2019-03-11 2025-06-26 후아웨이 테크놀러지 컴퍼니 리미티드 Encoders, decoders, and corresponding methods
US20200296405A1 (en) 2019-03-14 2020-09-17 Qualcomm Incorporated Affine motion compensation refinement using optical flow
JP7669278B2 (en) 2019-03-17 2025-04-28 北京字節跳動網絡技術有限公司 Calculating optical flow based prediction refinement
SG11202109031TA (en) 2019-03-18 2021-09-29 Tencent America LLC Method and apparatus for video coding
JP7307192B2 (en) 2019-04-02 2023-07-11 北京字節跳動網絡技術有限公司 Derivation of motion vectors on the decoder side
KR20230165888A (en) 2019-04-02 2023-12-05 베이징 바이트댄스 네트워크 테크놀로지 컴퍼니, 리미티드 Bidirectional optical flow based video coding and decoding
WO2020211867A1 (en) 2019-04-19 2020-10-22 Beijing Bytedance Network Technology Co., Ltd. Delta motion vector in prediction refinement with optical flow process
CN113711608B (en) 2019-04-19 2023-09-01 北京字节跳动网络技术有限公司 Suitability of predictive refinement procedure with optical flow
CN118540470A (en) 2019-04-19 2024-08-23 北京字节跳动网络技术有限公司 Region-based gradient calculation in different motion vector refinements
CN120769050A (en) 2019-04-30 2025-10-10 韦勒斯标准与技术协会公司 A method and device for processing video signals using adaptive motion vector resolution
US11523109B2 (en) 2019-05-02 2022-12-06 Tencent America LLC Method and apparatus for improvements of affine prof
CN118101931B (en) 2019-05-15 2025-02-21 华为技术有限公司 Bidirectional optical flow method and apparatus for reference image resampling in video coding
WO2020233660A1 (en) 2019-05-21 2020-11-26 Beijing Bytedance Network Technology Co., Ltd. Syntax-based motion candidate derivation in sub-block merge mode
US12483702B2 (en) 2019-06-03 2025-11-25 Nokia Technologies Oy Apparatus, a method and a computer program for video coding and decoding
US12143631B2 (en) 2019-06-23 2024-11-12 Sharp Kabushiki Kaisha Systems and methods for performing an adaptive resolution change in video coding
EP3987813A4 (en) 2019-06-24 2023-03-08 Sharp Kabushiki Kaisha SYSTEMS AND METHODS FOR REDUCING RECONSTRUCTION ERROR IN VIDEO CODING BASED ON INTER-COMPONENT CORRELATION
EP3987771A4 (en) 2019-07-08 2022-08-24 Huawei Technologies Co., Ltd. IMAGE CONSTRAINTS OF MIXED NAL UNITS IN A VIDEO CODING MECHANISM
US11641465B2 (en) 2019-07-25 2023-05-02 Hfi Innovation Inc. Method and apparatus of cross-component adaptive loop filtering with virtual boundary for video coding
BR112022002195A2 (en) * 2019-08-06 2022-10-18 Op Solutions Llc ADAPTIVE RESOLUTION MANAGEMENT SIGNALING
US11363307B2 (en) 2019-08-08 2022-06-14 Hfi Innovation Inc. Video coding with subpictures
EP3997869A4 (en) 2019-08-10 2022-10-26 Beijing Bytedance Network Technology Co., Ltd. SUBPICTURE DEPENDENT SIGNALING IN VIDEO STREAMS
GB2586484B (en) 2019-08-20 2023-03-08 Canon Kk A filter
WO2021049586A1 (en) 2019-09-13 2021-03-18 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ Encoding device, decoding device, encoding method, and decoding method
US11202068B2 (en) 2019-09-16 2021-12-14 Mediatek Inc. Method and apparatus of constrained cross-component adaptive loop filtering for video coding
US11451834B2 (en) 2019-09-16 2022-09-20 Tencent America LLC Method and apparatus for cross-component filtering
GB201913403D0 (en) 2019-09-17 2019-10-30 Canon Kk Method and apparatus for encoding and decoding a video stream with subpictures
US11240507B2 (en) 2019-09-24 2022-02-01 Qualcomm Incorporated Simplified palette predictor update for video coding
JP7322290B2 (en) 2019-10-02 2023-08-07 北京字節跳動網絡技術有限公司 Syntax for Subpicture Signaling in Video Bitstreams
CN116016936B (en) 2019-10-10 2025-01-14 北京达佳互联信息技术有限公司 Method and device for video encoding and decoding using palette mode
CN114631321B (en) 2019-10-18 2024-04-12 北京字节跳动网络技术有限公司 Interaction between sub-pictures and loop filtering
CN121531144A (en) 2019-10-25 2026-02-13 夏普株式会社 Systems and methods for sending signals to notify image information in video encoding
US11539982B2 (en) 2019-11-01 2022-12-27 Qualcomm Incorporated Merge estimation region for multi-type-tree block structure
CN115280774B (en) 2019-12-02 2025-08-19 抖音视界有限公司 Method, apparatus, and non-transitory computer readable storage medium for visual media processing
US11212523B2 (en) 2020-01-12 2021-12-28 Mediatek Inc. Video processing methods and apparatuses of merge number signaling in video coding systems

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3468190A1 (en) * 2016-06-30 2019-04-10 Huawei Technologies Co., Ltd. Coding and decoding method and apparatus

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
B. CHOI (TENCENT), S. WENGER (STEWE), S. LIU (TENCENT): "AHG8/AHG12: On sub-picture partitioning support with layers", 15. JVET MEETING; 20190703 - 20190712; GOTHENBURG; (THE JOINT VIDEO EXPLORATION TEAM OF ISO/IEC JTC1/SC29/WG11 AND ITU-T SG.16 ), 27 June 2019 (2019-06-27), XP030219282 *
J. BOYCE (INTEL), LIDONG XU (INTEL): "AHG12: Sub-pictures and sub-picture sets with level derivation", 15. JVET MEETING; 20190703 - 20190712; GOTHENBURG; (THE JOINT VIDEO EXPLORATION TEAM OF ISO/IEC JTC1/SC29/WG11 AND ITU-T SG.16 ), 25 June 2019 (2019-06-25), XP030219894 *
Y. HE (INTERDIGITAL), A. HAMZA (INTERDIGITAL): "AHG12: On picture and sub-picture signaling", 15. JVET MEETING; 20190703 - 20190712; GOTHENBURG; (THE JOINT VIDEO EXPLORATION TEAM OF ISO/IEC JTC1/SC29/WG11 AND ITU-T SG.16 ), 25 June 2019 (2019-06-25), XP030218771 *
Y.-K. WANG (FUTUREWEI), HENDRY (FUTUREWEI), J. CHEN (FUTUREWEI), R. SKUPIN (FRAUNHOFER), Y. SANCHEZ (FRAUNHOFER), K. SUEHRING (HHI: "AHG12: Sub-picture based motion-constrained independent regions", 15. JVET MEETING; 20190703 - 20190712; GOTHENBURG; (THE JOINT VIDEO EXPLORATION TEAM OF ISO/IEC JTC1/SC29/WG11 AND ITU-T SG.16 ), 25 June 2019 (2019-06-25), XP030218718 *
Y.-K. WANG (HUAWEI), HENDRY (HUAWEI), J. CHEN (HUAWEI), R. SKUPIN (FRAUNHOFER), Y. SANCHEZ (FRAUNHOFER), K. SUEHRING (FRAUNHOFER),: "AHG12: Harmonized proposal for sub-picture-based coding for VVC", 14. JVET MEETING; 20190319 - 20190327; GENEVA; (THE JOINT VIDEO EXPLORATION TEAM OF ISO/IEC JTC1/SC29/WG11 AND ITU-T SG.16 ), 22 March 2019 (2019-03-22), XP030204580 *

Also Published As

Publication number Publication date
EP4307665A2 (en) 2024-01-17
KR20250004179A (en) 2025-01-07
JP7478225B2 (en) 2024-05-02
WO2021027776A1 (en) 2021-02-18
US12047558B2 (en) 2024-07-23
MX2022001465A (en) 2022-02-22
JP7632851B2 (en) 2025-02-19
CN117336468A (en) 2024-01-02
US11553177B2 (en) 2023-01-10
US20220166971A1 (en) 2022-05-26
WO2021027767A1 (en) 2021-02-18
US20230118260A1 (en) 2023-04-20
CN114208165B (en) 2023-12-15
MY209640A (en) 2025-07-28
EP3997868A1 (en) 2022-05-18
JP2022543706A (en) 2022-10-13
JP2023182800A (en) 2023-12-26
CN114208165A (en) 2022-03-18
JP2022543705A (en) 2022-10-13
WO2021027774A1 (en) 2021-02-18
EP3997869A1 (en) 2022-05-18
EP4307665A3 (en) 2024-07-10
CN114208166B (en) 2024-04-09
US20220166985A1 (en) 2022-05-26
US20220159246A1 (en) 2022-05-19
KR20220044271A (en) 2022-04-07
CN114270831B (en) 2024-07-30
CN114270831A (en) 2022-04-01
US11533513B2 (en) 2022-12-20
KR20220042125A (en) 2022-04-04
JP7372443B2 (en) 2023-10-31
CN114208196B (en) 2023-11-14
BR112022002493A2 (en) 2022-04-26
US11523108B2 (en) 2022-12-06
US20220174322A1 (en) 2022-06-02
CN114208166A (en) 2022-03-18
CN114208196A (en) 2022-03-18
EP3997869A4 (en) 2022-10-26
US12075030B2 (en) 2024-08-27
EP3997868A4 (en) 2023-02-22

Similar Documents

Publication Publication Date Title
WO2021027773A1 (en) Subpicture size definition in video processing
WO2021073630A1 (en) Syntax constraints in parameter set signaling of subpictures
JP7322290B2 (en) Syntax for Subpicture Signaling in Video Bitstreams
WO2021129805A1 (en) Signaling of parameters at sub-picture level in a video bitstream
HK40063729A (en) Sub-picture related signaling notification in video bitstreams

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20851929

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20851929

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 20851929

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 27-01-2022)

122 Ep: pct application non-entry in european phase

Ref document number: 20851929

Country of ref document: EP

Kind code of ref document: A1