WO2020243881A1

WO2020243881A1 - Simplified intra block copy

Info

Publication number: WO2020243881A1
Application number: PCT/CN2019/089834
Authority: WO
Inventors: Yuchen SUN; Xuguang ZUO; Tsuishan CHANG; Jian Lou
Original assignee: Alibaba Group Holding Limited
Priority date: 2019-06-03
Filing date: 2019-06-03
Publication date: 2020-12-10

Abstract

Systems and methods provided herein are directed to simplifying and improving efficiency of a video decoding process. The video decoder complexity for generating a reconstructed block may be reduced by refraining from using the residual block refinement for an intra block copy process for a coding unit or selectively applying a palette-based prediction and residual block combination to a predictor of the coding unit.

Description

SIMPLIFIED INTRA BLOCK COPY

BACKGROUND

The state-of-the-art video coding standard, such as H. 264/AVC (Advanced Video Coding) , H. 264/MPEG-4 AVC, and H. 265/HEVC (High Efficiency Video Coding) standard, is based on a hybrid coding scheme using block-based prediction and transform coding.

To compress a video frame, a video compression system applies prediction methods, such as intra-prediction or inter-prediction, to derive a prediction frame. Based on the prediction frame, the video compression system further encodes residual information to refine the prediction frame. The residual information is usually compressed by transform and quantization techniques. A decoder is able to reconstruct the video frame from the prediction frame and the residual information. For a typical screen content, repetitive patterns may be found within the same picture and an intra block copy (IBC) mode may utilize this characteristic. In the IBC mode, a prediction unit (PU) may be predicted from a previously reconstructed block within the same picture.

In a video decoder, the residual block decoding is a complicated module that may consume a large amount of computing resources. By simplifying the decoding process of the IBC, the video decoder design may be simplified.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description is set forth with reference to the accompanying figures. In the figures, the left-most digit (s) of a reference number identifies the figure in which the reference number first appears. The use of the same reference numbers in different figures indicates similar or identical items or features.

FIG. 1 illustrates an example decoding process of a video decompression system using prediction methods.

FIG. 2 illustrates an example prediction block, an example residual block, and an example reconstructed block derived by the video decompression system of FIG. 1.

FIG. 3 illustrates an example graphical representation of an intra block copy (IBC) .

FIG. 4 illustrates an example diagram of the palette mode applied to an image.

FIG. 5 illustrates an example decoding process of the video decompression system using HEVC SCC palette mode.

FIG. 6 illustrates an example palette index mapping using the HEVC SCC palette mode.

FIG. 7 illustrates an example decoding process of the video decompression system using palette-based residual refinement method.

FIG. 8 illustrates an example prediction block, an example palette-based residual block, and an example reconstructed block derived by the video decompression system of FIG. 7 using the palette-based residual decoding.

FIG. 9 illustrates an example palette index map coding for the palette-based residual refinement method.

FIG. 10 illustrates another example palette index map coding for the palette-based residual refinement method.

FIG. 11 illustrates an example video decompression system for generating a reconstructed block by adaptively selecting an IBC process with or without the palette-based residual decoding.

FIG. 12 illustrates an example process of the video decompression system illustrated in FIG. 11.

FIG. 13 illustrates an example system for implementing the processes and methods described for simplifying the IBC decoder.

DETAILED DESCRIPTION

Systems and methods discussed herein are directed to simplifying a video decoding process, and more specifically to reducing the video decoder complexity by refraining from using the residual block refinement for of an intra block copy process or selectively utilizing a palette-based prediction and residual block combination to generate a reconstructed block.

FIG. 1 illustrates an example decoding process 100 of a video decompression system 102 using prediction methods.

The video decompression system 102 first derives a prediction block 104 by applying a prediction method, such as intra-prediction 106 or inter-prediction 108, and then decodes a residual block 110 by applying a residual decoding method 112. The residual block 110 and the prediction block 104 may be implemented in parallel. The video decompression system 102 combines the prediction block 104 and the residual block 110 to generate a reconstructed block 114.

FIG. 2 illustrates example prediction block 104, residual block 110, and reconstructed block 114 derived by the video decompression system 102 of FIG. 1.

The prediction block 104 and the residual block 110 are illustrated with the pixel values of coding units (CUs) . As discussed above with reference to FIG. 1, the corresponding pixel values of the prediction block 104 and the residual block 110 are simply added to generate corresponding pixel values of the reconstructed block 114.

FIG. 3 illustrates an example graphical representation 300 of an intra block copy (IBC) .

For a typical screen content, repetitive patterns may be found within the same picture as illustrated in FIG. 3. The IBC mode may utilize this characteristic. In the IBC mode, a prediction unit (PU) may be predicted from a previously reconstructed block within the same picture. Similar to a PU in motion compensation, a displacement vector, called a block vector or a BV, may be used to signal the relative displacement from the position of the current PU to that of the reference block. The prediction errors after the IBC compensation may then be coded using transformation, quantization and entropy coding. Because the IBC mode and the HEVC inter mode share many similarities, the block level IBC operations and HEVC inter mode the HEVC SCC are unified, specifically, the current (partially decoded) picture may be treated as a reference picture for decoding the current slice (302 and 304) .

The CPR, or the IBC, has been shown to be efficient when coding screen content video materials. In the VVC/H. 266, the CPR mode is coded in the same way as the regular inter mode. For each block, the use of the CPR mode may be signaled by using a reference picture index pointing to the current picture itself. The current (partially) decoded picture may be considered as a reference picture. This current picture may be put in the last position of a reference picture list 0. Therefore, for a slice using the current picture as the only reference picture, its slice type may be considered as a P slice (predicted picture) . The bitstream syntax in this approach may follow the same syntax structure for the inter coding while the decoding process may be unified with the inter coding. The difference may be that the block vector (which is the motion vector pointing to the current picture) always uses an integer-pel resolution. For example, in an encoder search for this mode, both block width and height are smaller than or equal to 16; a chroma interpolation is enabled when a luma block vector is an odd integer number; and an adaptive motion vector resolution (AMVR) for the CPR mode is enabled when the sequence parameter set (SPS) flag is on. In this case, when the AMVR is used, a block vector may be switched between 1-pel integer and 4-pel integer resolution at a block level.

In the HEVC, a coding tree unit (CTU) may be split into CUs by using a quaternary-tree structure denoted as coding tree to adapt to various local characteristics. The decision whether to code a picture area using an inter-picture (temporal) or an intra-picture (spatial) prediction may be made at a leaf CU level. Each leaf CU may be further split into one, two or four PUs according to the PU splitting type. Inside one PU, the same prediction process may be applied, and relevant information may be transmitted to the decoder on a PU basis. After obtaining the residual block by applying the prediction process based on the PU splitting type, a leaf CU may be partitioned into transform units (TUs) according to another quaternary-tree structure similar to the coding tree for the CU.

In the VVC, a quadtree with nested multi-type tree using binary and ternary splits segmentation structure may replace the concepts of multiple partition unit types. That is, the separation of the CU, PU and TU concepts may be removed except as needed for CUs having a size too large for the maximum transform length, and more flexibility for CU partition shapes may be supported. In the coding tree structure, a CU may be either a square or rectangular in shape. A CTU may be first partitioned by a quaternary tree (a. k. a. quadtree) structure. Then the quaternary tree leaf nodes may be further partitioned by a multi-type tree structure.

A spatial, or temporal, prediction may result in a predictive block for an original block to be coded. A residual data for the spatial prediction may represent pixel differences between the original block and a corresponding predictive block. An IBC block may be encoded according to a block vector that points to a block of the reconstructed samples in the current frame forming the predictive block and the residual data that indicates the difference between the coded block and the predictive block.

A transform-based residual coding may achieve further compression by transforming the residual data from a pixel domain to a transform domain to generate residual transform coefficients, quantizing the residual transform coefficients, scanning the quantized transform coefficients, which are initially arranged in a two-dimensional array, to generate a one-dimensional vector of transform coefficients. An entropy coding may be applied to the one-dimensional vector to achieve further compression. In addition to the transform-based residual coding, a transform skip residual coding may encode residual pixels by quantization and entropy coding but without transforming the residual data from the pixel domain to the transform domain.

In the HEVC and other video compression standards, a flag (transform_skip flag) may be signaled for each residual block to determine whether the transform-based residual coding method or transform skip residual coding method is used for the block.

Referring back to FIG. 1, the video decompression system may decode the residual block 110 by applying block 112, which includes at least one of the transform-based, the transform-skip, or the HEVC residual decoding method.

FIG. 4 illustrates an example diagram 400 of the palette mode applied to a CU 402. For simplicity, a pixel or a palette index is shown to correspond to only one value. However, in HEVC SCC, a pixel or a palette index may represent three color component values, such as YCbCr or GBR.

In the HEVC SCC palette mode, a flag is transmitted for each CU to indicate whether the palette mode is used for that CU, such as the CU 402. If the palette mode is used for the CU 402, the pixels, having pixel values close to palette colors, such as color A 404, color B 406, and color C 408, are represented by the palette color values 410 as shown in a color histogram 412. For example, the color A 404 is shown to be close to the palette color value of 50, the color B 406 is shown to be close to the palette color value of 80, and the color C 408 is shown to be close to the palette color value of 410. The palette color values 410 of the CU 402 are denoted by palette indices 414, where the palette color values 410 are representative colors of the CU 402 as shown in a palette table 416. The remaining pixels are presented by a special palette index value, denoted as an escape index, and their pixel values are signaled directly. Note that a color is a 3-value, for example, YCbCr or GBR, vector. Palette indices of all pixels in the CU 402 form a palette index map 418.

FIG. 5 illustrates an example decoding process 500 of the video decompression system 502 using HEVC SCC palette mode.

Compared to the intra/inter-prediction methods described with reference to FIG. 2, if a block, or a CU, is encoded by the HEVC SCC palette mode 504, it is decoded independently to generate a reconstructed block 506 without using any prediction method or residual refinement. To decode a palette encoded block, a video decoder of the video decompression system 502 decodes palette colors and indices. As described above with reference to FIG. 4, palette colors are described by a palette table, such as the palette table 416, and encoded by palette table coding tools. Based on the palette table, a pixel of three color components is represented by a palette index, such as the palette index 414. Palette indices of all pixels in the CU form a palette index map, such as the palette index map 418, and are encoded by palette index map coding tools.

FIG. 6 illustrates an example the palette index map coding 600.

The video decompression system 502 constructs a palette table predictor to predict a palette table, such as the palette table 416, of the current CU by applying predictive coding where the palette of the last palette coded CU is used as the predictor. After the palette table is encoded, the pixels in the current CU are represented by palette indices 602 corresponding to the colors in the palette as shown in a parameter table 604 along with other parameters of the palette index map coding. The indices form a palette index map 606 which is divided into several runs and then encoded in horizontal or vertical traverse scan order. In this example, the horizontal scan order is assumed, and the direction of the scan is shown as arrows in the palette index map 606. In the palette index map 606, 16 indices are encoded by run indices 608, three runs R0, R1, and R2, with horizontal traverse scan. There are two run modes 610, copy index mode and copy above mode, and for each starting position of a run, a flag is transmitted to indicate which run mode is used.

If the copy above mode is used, a run of pixels 612 will copy the palette indices from their above pixels. In this example, a run, R2, is a copy above run with a run value of 10. If the copy index mode is used, a palette_index_idc syntax element is first signaled, followed by a piece of PaletteIndexRun information. The palette_index_idc syntax describes whether the run is copy index mode (palette_index_idc=0) or copy above mode (palette_index_idc=1) , and the PaletteIndexRun syntax describes the length of the run. A run of pixels 612 shares the same palette index 602, where the palette index 602 and the run value of the run of pixels 612 are derived from the palette_index_idc and PaletteIndexRun, respectively.

FIG. 7 illustrates an example decoding process 700 of the video decompression system 702 using the palette-based residual refinement method.

The video decompression system 702 may include an option of combining the palette mode and the traditional prediction methods as described above with reference to FIGs. 5 and 6, in addition to the HEVC residual decoding as described above with reference to FIGs. 1 and 2. The video decompression system 702 may derive a palette-based residual block 704 by applying the palette-based residual decoding 706 to palette-based residual information. The video decompression system 702 may then construct a reconstructed block 708 by applying the palette-based prediction and residual block combination 710 to information from the prediction block 104 and the palette-based residual block 1004.

FIG. 8 illustrates the prediction block 104, the palette-based residual block 704, and the reconstructed block 708 derived by the video decompression system 702 of FIG. 7 using the palette-based residual decoding.

The video decompression system 702 may derive the prediction block 104 as described above with reference to FIGs. 1 and 2 and may then decode a palette-based residual block 704. Before reconstructing the reconstructed block 708, the video decompression system 702 may derive and use the palette-based residual block 704 as described above with reference to FIG. 7 to refine the prediction block 104. The video decompression system 702 may classify pixels in the palette-based residual block 704 into two groups, major color pixels and prediction pixels. The major color pixels may be pixels with pixel values, for example C0 and C1 as shown, where the pixel values are associated with major color values in a decoded palette table, such as the palette table 416 of FIG. 4. The prediction pixels may be special pixels, shown as “P” in the palette-based residual block 704. If a pixel is encoded as “P” , the video decompression system 702 may directly use its corresponding prediction pixel value from the prediction block 104 as the corresponding reconstructed pixel value for the reconstructed block 708. If a pixel in the palette-based residual block 704 were a major color pixel having a corresponding major color value, such as C0 or C1, in the decoded palette table, the video decompression system 702 would use the corresponding major color value to reconstruct the reconstructed block 708 as shown.

FIG. 9 illustrates an example palette index map coding 900 for the palette-based residual refinement method.

To signal whether a pixel in the palette-based residual block 704 is a major color pixel or a prediction pixel, the video decompression system 702 may reserve a special color index, for example, 0, as shown under a palette index 902 of a parameter table 904, the maximal index value, or a pre-defined value. The video decompression system 702 may then use a conventional palette mode to decode a palette index map 906 as described above with reference to FIG. 6. The indices form the palette index map 906 which is divided into several runs and then encoded in horizontal or vertical traverse scan order. In this example, the horizontal scan order is assumed, and the direction of the scan is shown as arrows in the palette index map 906. In the palette index map 906, 16 indices are encoded by run indices, four runs R0, R1, R2, and R3 shown under a run index 908, with a horizontal traverse scan. There are two run modes, copy index mode and copy above mode, as shown under a run mode 910, and for each starting position of a run, a flag is transmitted to indicate which run mode is used.

If a pixel in the palette index map 906 were identified to have the reserved color index, it would be identified as a prediction pixel. In FIG. 9, the bottom four pixels are identified as prediction pixels. The video decompression system 702 may reserve a special color index, “0” in this example, for these prediction pixels. The video decompression system 702 may use the copy index run, R3, for the four prediction pixels as indicated by a run of pixels 912 to signal those four prediction pixels.

FIG. 10 illustrates another example palette index map coding 1000 for the palette-based residual refinement method.

To signal whether a pixel in the palette-based residual block 704 is a major color pixel or a prediction pixel, the video decompression system 702 may utilize a special run, such as a prediction run as shown “prediction” under a run mode 1002 of a parameter table 1004 instead of reserving a special color index under a palette index 1006. The video decompression system 702 may identify a prediction pixel of the palette-based residual block 704 as a pixel encoded by the prediction run. Similar to the palette index map coding of FIG. 9, the indices form the palette index map 1008 which is divided into four runs and then encoded in a horizontal scan order shown as arrows. In the palette index map 1008, 16 indices are encoded by run indices 1010, four runs R0, R1, R2, and R3 with horizontal traverse scan. There are three, copy index mode, copy above mode, and prediction mode, under the run mode 1002, and for each starting position of a run, a flag is transmitted to indicate which run mode is used. If a pixel in the palette index map 1008 were identified to have been encoded by the prediction run, it would be identified as a prediction pixel. In FIG. 10, the video decompression system 702 may use the prediction run, R3, for the four bottom pixels to encode as prediction pixels as indicated by a run of pixels 1012 and signal those four pixels as prediction pixels.

FIG. 11 illustrates an example video decompression system 1100 for generating a reconstructed block 1102 by adaptively selecting an IBC encoding 1104 with or without the palette-based residual decoding.

In the video decompression system 1100, a current CU may be encoded by the intra-prediction 106 or the inter-prediction 108 and be refined using the residual block 110 by applying the residual decoding method 112, which may include the HEVC, the transform-based, or transform-skip residual decoding method, to generate a reconstructed block 1102 (A) . Additionally, or alternatively, the video decompression system 1100 may encode the current CU using an IBC encoder 1104, and a predictor of the IBC encoded CU may be directly used to generate the reconstructed block 1102 as shown (B) , be refined using the palette-based prediction and residual block combination 710 as shown (C) . Because the IBC applies the conventional residual coding, both the transform-based and the transform-skip residual coding are allowed. Thus, by forcing the CU to be coded by the transform-skip encoding method for the IBC, i.e., without using a transform_skip flag, and refining the prediction block using the residual decoding as shown (D) , a transform module used to transform the residual data may be eliminated and the decoding process may be simplified. The decision to apply, bypass or disable the residual block refinement for the IBC encoded CU may be based on a system setup, such as a default setup to force the transform-skip encoding and apply the residual or to bypass the refinement, signaling to bypass the refinement, such as sending a flag to indicate whether to bypass the refinement, analyzing a characteristic of the palette index value, such as a maximal palette index value of 0 to indicate bypassing the refinement, and the like.

By adaptively selecting the IBC method 1104 with (C) or without (B) the palette-based prediction and residual block combination 710, or applying the residual decoding 1106 without signaling the transform_skip flag and eliminating the transform module (D) , the complexity of the video decompression system 1100 may be simplified the efficiency may be improved.

FIG. 12 illustrates an example process 1200 of the video decompression system 1100 illustrated in FIG. 11.

At block 1202, whether a first indicator associated with a current CU is an IBC flag or a palette-based residual flag may be determined. Upon determining that the first indicator is the IBC flag at block 1202, whether the transform-skip encoding is enforced may be determined at block 1204. Upon determining that the transform-skip encoding is enforced at block 1204, the transform-skip encoding may be applied to the CU at block 1206, an IBC predictor may be refined by applying the residual block at block 1208, and a reconstruction block, such as the reconstructed block 1102 via (D) as shown in FIG 11, may be generated using the refined IBC predictor at block 1210.

Upon determining that the transform-skip encoding is not enforced at block 1204, whether a default refinement of an IBC predictor of the current CU is set may be determined at block 1212. Upon determining that the default refinement is set at block 1212, a reconstructed block, such as the reconstructed block 1102 via (B) as shown in FIG. 11, may be generated using the IBC predictor of the current CU at block 1214. Upon determining that the default refinement is not set at block 1212 and that a second indicator associated with the current CU is not on at block 1216, then a reconstructed block using the IBC predictor of the current CU may be generated at block 1214. The second indicator associated with the current CU may be the palette-based residual flag having a value of on or off. The second indicator associated with the current CU may also be a maximal palette index having a value associated with the current CU, and the second indicator may be determined not to be on when the value of the maximal palette index is 0 and to be on when the value of the maximal palette index is not 0.

Upon determining that the default refinement is not set at block 1212 and that the second indicator associated with the current CU is on at block 1216, the IBC predictor of the current CU may be refined at block 1218 by applying a palette-based prediction and residual block combination, such as the palette-based prediction and residual block combination 710 as described in reference to FIG. 11, and a reconstruction block using the refined IBC predictor may be generated at block 1210.

Upon determining that the first indicator is the palette-based residual flag at block 1202, whether a value of the palette-based residual flag is on or off may be determined at block 1220, and upon determining that the value of the palette-based residual flag is off at block 1220, a reconstructed block using a predictor of the current CU may be generated at block 1214. Upon determining that the value of the palette-based residual flag is on at block 1220, a reconstructed block may be generated using the predictor of the current CU at block 1214 if the maximal palette index associated with the current is determined to equal to 0 a block 1220.

Upon determining that the value of the palette-based residual flag is on at block 1220 and that the maximal palette index associated with the current is determined not equal to 0 at block 1222, the predictor of the current CU may be refined at block 1218 by applying a palette-based prediction and residual block combination, and a reconstruction block using the refined predictor may be generated at block 1210.

FIG. 13 illustrates an example system 1300 for implementing the processes and methods described for simplifying the IBC decoder.

The techniques and mechanisms described herein may be implemented by multiple instances of the system 1300 as well as by any other computing device, system, and/or environment. The system 1300 shown in FIG. 13 is only one example of a system and is not intended to suggest any limitation as to the scope of use or functionality of any computing device utilized to perform the processes and/or procedures described above. Other well-known computing devices, systems, environments and/or configurations that may be suitable for use with the embodiments include, but are not limited to, personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, game consoles, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, implementations using field programmable gate arrays ( “FPGAs” ) and application specific integrated circuits ( “ASICs” ) , and/or the like.

The system 1300 may include one or more processors 1302 and system memory 1304 communicatively coupled to the processor (s) 1302. The processor (s) 1302 may execute one or more modules and/or processes to cause the processor (s) 1302 to perform a variety of functions. In embodiments, the processor (s) 1302 may include a central processing unit (CPU) , a graphics processing unit (GPU) , both CPU and GPU, or other processing units or components known in the art. Additionally, each of the processor (s) 1302 may possess its own local memory, which also may store program modules, program data, and/or one or more operating systems.

Depending on the exact configuration and type of the system 1300, the system memory 1304 may be volatile, such as RAM, non-volatile, such as ROM, flash memory, miniature hard drive, memory card, and the like, or some combination thereof. The system memory 1304 may include one or more computer-executable modules (modules) 1306 that are executable by the processor (s) 1302.

The modules 1306 may include, but are not limited to, a first indicator module 1308 configured to determine whether a first indicator associated with a current coding unit (CU) is an intra block copy (IBC) flag or a palette-based residual flag, a default setup module 1310 configured to determine whether a transform-skip encoding is enforced, and upon determining that the transform-skip encoding is not enforced, determine whether a default refinement of an IBC predictor of the current CU is set upon the first indicator module determining that the first indicator is the IBC flag, and a reconstruction block module 1312 configured to generate a reconstructed block, such as the reconstruction block 1102 as described above with reference to FIG. 11, using the IBC predictor of the current CU upon the default setup module determining that the default refinement is set. The modules 1306 may further comprise a second indicator module 1314 configured to determine whether a second indicator associated with the current CU is on or off. Upon the default setup module 1310 determining that the default refinement is not set and the second indicator module 1314 determining that the second indicator is off, the reconstruction block module 1312 may be further configured to generate a reconstructed block using the IBC predictor of the current CU.

The module 1306 may further comprise a refinement module 1316 configured to refine the IBC predictor of the current CU by applying a palette-based prediction and residual block combination, such as the palette-based prediction and residual block combination 710 as described above with reference to FIG. 11. The reconstruction block module 1312 may be further configured to generate a reconstruction block using the refined IBC predictor from the refinement module 1316 upon the default setup module 1310 determining that the default refinement is not set and the second indicator module 1314 determining that the second indicator is on. The second indicator associated with the current CU may be a palette-based residual flag having a value of on or off or a maximal palette index having a value associated with the current CU. The second indicator module 1314 may be further configured to determine that the second indicator is not on when the value of the maximal palette index is 0, and the second indicator is on when the value of the maximal palette index is not 0.

The module 1306 may further comprise an encoder 1318 configured to encode the current CU without transformation to generate the IBC predictor. Upon the default setup module 1310 determines that the transform-skip encoding is enforced, the refinement module 1316 may be further configured to refine the IBC predictor by applying a residual block, and the reconstruction block module 1312 may be further configured to generate a reconstructed block using the refined IBC predictor.

The module 1306 may further comprise a palette flag status module 1320 configured to determine whether a value of the palette-based residual flag is on or off, and the reconstruction block module 1312 may be further configured to generate a reconstructed block using a predictor of the current CU upon the first indicator module 1308 determining that the first indicator is the palette-based residual flag and the palette flag status module 1320 determining that the value of the palette-based residual flag is off. The second indicator module 1314 may be further configured to determine whether a maximal palette index associated with the current is equal to 0, and the reconstruction block module 1312 may be further configured to generate a reconstructed block using the predictor of the current CU upon the palette flag status module 1320 determining that the value of the palette-based residual flag is on and the second indicator module 1314 determining that the maximal palette index associated with the current is equal to 0.

Upon the palette flag status module 1320 determining that the value of the palette-based residual flag is on and the second indicator module 1314 determining the maximal palette index associated with the current is not 0, the refinement module 1316 may refine the predictor of the current CU by applying a palette-based prediction and residual block combination 1210, and the reconstruction block module 1312 may generate a reconstruction block using the refined predictor from the refinement module 1316.

The system 1300 may additionally include an input/output (I/O) interface 1322 for receiving data associated with the CU, such as the encoding mode and for outputting the reconstructed block of the CU. The system 1300 may also include a communication module 1324 allowing the system 1300 to communicate with other devices (not shown) over a network (not shown) . The network may include the Internet, wired media such as a wired network or direct-wired connections, and wireless media such as acoustic, radio frequency (RF) , infrared, and other wireless media.

Some or all operations of the methods described above can be performed by execution of computer-readable instructions stored on a computer-readable storage medium, as defined below. The term “computer-readable instructions” as used in the description and claims, include routines, applications, application modules, program modules, programs, components, data structures, algorithms, and the like. Computer-readable instructions can be implemented on various system configurations, including single-processor or multiprocessor systems, minicomputers, mainframe computers, personal computers, hand-held computing devices, microprocessor-based, programmable consumer electronics, combinations thereof, and the like.

The computer-readable storage media may include volatile memory (such as random-access memory (RAM) ) and/or non-volatile memory (such as read-only memory (ROM) , flash memory, etc. ) . The computer-readable storage media may also include additional removable storage and/or non-removable storage including, but not limited to, flash memory, magnetic storage, optical storage, and/or tape storage that may provide non-volatile storage of computer-readable instructions, data structures, program modules, and the like.

A non-transient computer-readable storage medium is an example of computer-readable media. Computer-readable media includes at least two types of computer-readable media, namely computer-readable storage media and communications media. Computer-readable storage media includes volatile and non-volatile, removable and non-removable media implemented in any process or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media includes, but is not limited to, phase change memory (PRAM) , static random-access memory (SRAM) , dynamic random-access memory (DRAM) , other types of random-access memory (RAM) , read-only memory (ROM) , electrically erasable programmable read-only memory (EEPROM) , flash memory or other memory technology, compact disk read-only memory (CD-ROM) , digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information for access by a computing device. In contrast, communication media may embody computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, or other transmission mechanism. As defined herein, computer-readable storage media do not include communication media.

The computer-readable instructions stored on one or more non-transitory computer-readable storage media that, when executed by one or more processors, may perform operations described above with reference to FIGs. 1-13. Generally, computer-readable instructions include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular abstract data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and/or in parallel to implement the processes.

EXAMPLE CLAUSES

A. A method comprising: determining whether a first indicator associated with a current coding unit (CU) is an intra block copy (IBC) flag or a palette-based residual flag; upon determining that the first indicator is the IBC flag: determining whether a transform-skip encoding is enforced, and upon determining that the transform-skip encoding is not enforced, determining whether a default refinement of an IBC predictor of the current CU is set; and upon determining that the default refinement is set, generating a reconstructed block using the IBC predictor of the current CU.

B. The method as paragraph A recites, further comprising: upon determining that the default refinement is not set and a second indicator associated with the current CU is off, generating a reconstructed block using the IBC predictor of the current CU.

C. The method as paragraph B recites, further comprising: upon determining that the default refinement is not set and the second indicator associated with the current CU is on: refining the IBC predictor of the current CU by applying a palette-based prediction and residual block combination; and generating a reconstruction block using the refined IBC predictor.

D. The method as paragraph B recites, wherein the second indicator associated with the current CU is the palette-based residual flag having a value of on or off.

E. The method as paragraph B recites, wherein: the second indicator associated with the current CU is a maximal palette index having a value associated with the current CU, the second indicator is not on when the value of the maximal palette index is 0, and the second indicator is on when the value of the maximal palette index is not 0.

F. The method as paragraph A recites, further comprising: upon determining that the transform-skip encoding is enforced, generating an IBC predictor by applying the transform skip encoding to the current CU; refining the IBC predictor by applying a residual block; and generating a reconstruction block using the refined IBC predictor.

G. The method as paragraph A recites, further comprising: upon determining that the first indicator is the palette-based residual flag: determining whether a value of the palette-based residual flag is on or off (1214) ; and upon determining that the value of the palette-based residual flag is off, generating a reconstructed block using a predictor of the current CU.

H. The method as paragraph G recites, further comprising: upon determining that the value of the palette-based residual flag is on, generating a reconstructed block using the predictor of the current CU if a maximal palette index associated with the current is equal to 0.

I. The method as paragraph H recites, further comprising: upon determining that the value of the palette-based residual flag is on and that the maximal palette index associated with the current is not equal to 0: refining the predictor of the current CU by applying a palette-based prediction and residual block combination; and generating a reconstruction block using the refined predictor.

J. A system comprising: one or more processors; and memory communicatively coupled to the one or more processors, the memory storing computer-executable modules executable by the one or more processors that, when executed by the one or more processors, perform associated operations, the computer-executable modules including: a first indicator module configured to determine whether a first indicator associated with a current coding unit (CU) is an intra block copy (IBC) flag or a palette-based residual flag; a default setup module configured to: determine whether a transform-skip encoding is enforced, and upon determining that the transform-skip encoding is not enforced, determine whether a default refinement of an IBC predictor of the current CU is set upon the first indicator module determining that the first indicator is the IBC flag; and a reconstruction block module configured to generate a reconstructed block using the IBC predictor of the current CU upon the default setup module determining that the default refinement is set.

K. The system as paragraph J recites, wherein the computer-executable modules further comprise: a second indicator module configured to determine whether a second indicator associated with the current CU is on or off, wherein the reconstruction block module is further configured to generate a reconstructed block using the IBC predictor of the current CU upon the default setup module determining that the default refinement is not set and the second indicator module determining that the second indicator is off.

L. The system as paragraph K recites: wherein the computer-executable modules further comprise: a refinement module configured to refine the IBC predictor of the current CU by applying a palette-based prediction and residual block combination, wherein the reconstruction block module is further configured to generate a reconstruction block using the refined predictor from the refinement module upon the default setup module determining that the default refinement is not set and the second indicator module determining that the second indicator is on.

M. The system as paragraph K recites, wherein the second indicator associated with the current CU is the palette-based residual flag having a value of on or off.

N. The system as paragraph K recites, wherein: the second indicator associated with the current CU is a maximal palette index having a value associated with the current CU, and the second indicator module is further configured to determine that: the second indicator is not on when the value of the maximal palette index is 0, and the second indicator is on when the value of the maximal palette index is not 0.

O. The system as paragraph L recites, further comprising: an encoder configured to encode the current CU without transformation to generate the IBC predictor, wherein, upon the default setup module determining that the transform-skip encoding is enforced, the refinement module is further configured to refine the IBC predictor by applying a residual block, and the reconstruction block module is further configured to generate a reconstructed block using the refined IBC predictor.

P. The system as paragraph J recites, wherein the computer-executable modules further comprise: a palette flag status module configured to determine whether a value of the palette-based residual flag is on or off, wherein the reconstruction block module is further configured to generate a reconstructed block using a predictor of the current CU upon the first indicator module determining that the first indicator is the palette-based residual flag and the palette flag status module determining that the value of the palette-based residual flag is off.

Q. The system as paragraph P recites, wherein the second indicator module is further configured to determine whether a maximal palette index associated with the current is equal to 0, and the reconstruction block module is further configured to generate a reconstructed block using the predictor of the current CU upon the palette flag status module determining that the value of the palette-based residual flag is on and the second indicator module determining that the maximal palette index associated with the current is equal to 0.

R. The system as paragraph Q recites, wherein upon the palette flag status module determining that the value of the palette-based residual flag is on and the second indicator module determining the maximal palette index associated with the current is not 0, the refinement module is further configured to refine the predictor of the current CU by applying a palette-based prediction and residual block combination, and the reconstruction block module is further configured to generate a reconstruction block using the refined predictor from the refinement module.

S. A computer-readable storage medium storing computer-readable instructions executable by one or more processors, that when executed by the one or more processors, cause the one or more processors to perform operations comprising: determining whether a first indicator associated with a current coding unit (CU) is an intra block copy (IBC) flag or a palette-based residual flag; upon determining that the first indicator is the IBC flag: determining whether a transform-skip encoding is enforced, and upon determining that the transform-skip encoding is not enforced, determining whether a default refinement of an IBC predictor of the current CU is set; and upon determining that the default refinement is set, generating a reconstructed block using the IBC predictor of the current CU.

T. The computer-readable storage medium as paragraph S recites, wherein the operations further comprise: upon determining that the default refinement is not set and a second indicator associated with the current CU is off, generating a reconstructed block using the IBC predictor of the current CU.

U. The computer-readable storage medium as paragraph T recites, wherein the operations further comprise: upon determining that the default refinement is not set and the second indicator associated with the current CU is on:refining the IBC predictor of the current CU by applying a palette-based prediction and residual block combination; and generating a reconstruction block using the refined IBC predictor.

V. The computer-readable storage medium as paragraph T recites, wherein the second indicator associated with the current CU is the palette-based residual flag having a value of on or off.

W. The computer-readable storage medium as paragraph T recites, wherein: the second indicator associated with the current CU is a maximal palette index having a value associated with the current CU, the second indicator is not on when the value of the maximal palette index is 0, and the second indicator is on when the value of the maximal palette index is not 0.

X. The computer-readable storage medium as paragraph S recites, wherein the operations further comprise: upon determining that the transform-skip encoding is enforced, generating an IBC predictor by applying the transform skip encoding to the current CU; refining the IBC predictor by applying a residual block; and generating a reconstruction block using the refined IBC predictor.

Y. The computer-readable storage medium as paragraph S recites, wherein the operations further comprise: upon determining that the first indicator is the palette-based residual flag: determining whether a value of the palette-based residual flag is on or off; and upon determining that the value of the palette-based residual flag is off, generating a reconstructed block using a predictor of the current CU.

Z. The computer-readable storage medium as paragraph Y recites, wherein the operations further comprise: upon determining that the value of the palette-based residual flag is on, generating a reconstructed block using the predictor of the current CU if a maximal palette index associated with the current is equal to 0.

AA. The computer-readable storage medium as paragraph Z recites, wherein the operations further comprise: upon determining that the value of the palette-based residual flag is on and that the maximal palette index associated with the current is not equal to 0: refining the predictor of the current CU by applying a palette-based prediction and residual block combination; and generating a reconstruction block using the refined predictor.

CONCLUSION

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as exemplary forms of implementing the claims.

Claims

A method comprising:

determining whether a first indicator associated with a current coding unit (CU) is an intra block copy (IBC) flag or a palette-based residual flag;

upon determining that the first indicator is the IBC flag:

determining whether a transform-skip encoding is enforced, and

upon determining that the transform-skip encoding is not enforced, determining whether a default refinement of an IBC predictor of the current CU is set; and

upon determining that the default refinement is set, generating a reconstructed block using the IBC predictor of the current CU.
The method of claim 1 further comprising:

upon determining that the default refinement is not set and a second indicator associated with the current CU is off, generating a reconstructed block using the IBC predictor of the current CU.
The method of claim 2, further comprising:

upon determining that the default refinement is not set and the second indicator associated with the current CU is on:

refining the IBC predictor of the current CU by applying a palette-based prediction and residual block combination; and

generating a reconstruction block using the refined IBC predictor.
The method of claim 2, wherein the second indicator associated with the current CU is the palette-based residual flag having a value of on or off.
The method of claim 2, wherein:

the second indicator associated with the current CU is a maximal palette index having a value associated with the current CU,

the second indicator is not on when the value of the maximal palette index is 0, and

the second indicator is on when the value of the maximal palette index is not 0.
The method of claim 1, further comprising:

upon determining that the transform-skip encoding is enforced,

generating an IBC predictor by applying the transform skip encoding to the current CU;

refining the IBC predictor by applying a residual block; and

generating a reconstruction block using the refined IBC predictor.
The method of claim 1, further comprising:

upon determining that the first indicator is the palette-based residual flag:

determining whether a value of the palette-based residual flag is on or off; and

upon determining that the value of the palette-based residual flag is off, generating a reconstructed block using a predictor of the current CU.
The method of claim 7, further comprising:

upon determining that the value of the palette-based residual flag is on, generating a reconstructed block using the predictor of the current CU if a maximal palette index associated with the current is equal to 0.
The method of claim 8, further comprising:

upon determining that the value of the palette-based residual flag is on and that the maximal palette index associated with the current is not equal to 0:

refining the predictor of the current CU by applying a palette-based prediction and residual block combination; and

generating a reconstruction block using the refined predictor.
A system comprising:

one or more processors; and

memory communicatively coupled to the one or more processors, the memory storing computer-executable modules executable by the one or more processors that, when executed by the one or more processors, perform associated operations, the computer-executable modules including:

a first indicator module configured to determine whether a first indicator associated with a current coding unit (CU) is an intra block copy (IBC) flag or a palette-based residual flag;

a default setup module configured to:

determine whether a transform-skip encoding is enforced, and

upon determining that the transform-skip encoding is not enforced, determine whether a default refinement of an IBC predictor of the current CU is set upon the first indicator module determining that the first indicator is the IBC flag; and

a reconstruction block module configured to generate a reconstructed block using the IBC predictor of the current CU upon the default setup module determining that the default refinement is set.
The system of claim 10, wherein the computer-executable modules further comprise:

a second indicator module configured to determine whether a second indicator associated with the current CU is on or off, wherein the reconstruction block module is further configured to generate a reconstructed block using the IBC predictor of the current CU upon the default setup module determining that the default refinement is not set and the second indicator module determining that the second indicator is off.
The system of claim 11, wherein the computer-executable modules further comprise:

a refinement module configured to refine the IBC predictor of the current CU by applying a palette-based prediction and residual block combination,

wherein the reconstruction block module is further configured to generate a reconstruction block using the refined IBC predictor from the refinement module upon the default setup module determining that the default refinement is not set and the second indicator module determining that the second indicator is on.
The system of claim 11, wherein the second indicator associated with the current CU is the palette-based residual flag having a value of on or off.
The system of claim 11, wherein:

the second indicator associated with the current CU is a maximal palette index having a value associated with the current CU, and

the second indicator module is further configured to determine that:

the second indicator is not on when the value of the maximal palette index is 0, and

the second indicator is on when the value of the maximal palette index is not 0.
The system of claim 12, further comprising:

an encoder configured to encode the current CU without transformation to generate the IBC predictor,

wherein, upon the default setup module determining that the transform-skip encoding is enforced,

the refinement module is further configured to refine the IBC predictor by applying a residual block, and

the reconstruction block module is further configured to generate a reconstructed block using the refined IBC predictor.
The system of claim 10, wherein the computer-executable modules further comprise:

a palette flag status module configured to determine whether a value of the palette-based residual flag is on or off,

wherein the reconstruction block module is further configured to generate a reconstructed block using a predictor of the current CU upon the first indicator module determining that the first indicator is the palette-based residual flag and the palette flag status module determining that the value of the palette-based residual flag is off.
The system of claim 16, wherein the second indicator module is further configured to determine whether a maximal palette index associated with the current is equal to 0, and

the reconstruction block module is further configured to generate a reconstructed block using the predictor of the current CU upon the palette flag status module determining that the value of the palette-based residual flag is on and the second indicator module determining that the maximal palette index associated with the current is equal to 0.
The system of claim 17, wherein:

upon the palette flag status module determining that the value of the palette-based residual flag is on and the second indicator module determining the maximal palette index associated with the current is not 0,

the refinement module is further configured to refine the predictor of the current CU by applying a palette-based prediction and residual block combination, and

the reconstruction block module is further configured to generate a reconstruction block using the refined predictor from the refinement module.
A computer-readable storage medium storing computer-readable instructions executable by one or more processors, that when executed by the one or more processors, cause the one or more processors to perform operations comprising:

determining whether a first indicator associated with a current coding unit (CU) is an intra block copy (IBC) flag or a palette-based residual flag;

upon determining that the first indicator is the IBC flag:

determining whether a transform-skip encoding is enforced, and

upon determining that the transform-skip encoding is not enforced, determining whether a default refinement of an IBC predictor of the current CU is set; and

upon determining that the default refinement is set, generating a reconstructed block using the IBC predictor of the current CU.
The computer-readable storage medium of claim 19, wherein the operations further comprise:

upon determining that the default refinement is not set and a second indicator associated with the current CU is off, generating a reconstructed block using the IBC predictor of the current CU.
The computer-readable storage medium of claim 20, wherein the operations further comprise:

upon determining that the default refinement is not set and the second indicator associated with the current CU is on:

refining the IBC predictor of the current CU by applying a palette-based prediction and residual block combination; and

generating a reconstruction block using the refined IBC predictor.
The computer-readable storage medium of claim 20, wherein the second indicator associated with the current CU is the palette-based residual flag having a value of on or off.
The computer-readable storage medium of claim 20, wherein:

the second indicator associated with the current CU is a maximal palette index having a value associated with the current CU,

the second indicator is not on when the value of the maximal palette index is 0, and

the second indicator is on when the value of the maximal palette index is not 0.
The computer-readable storage medium of claim 19, wherein the operations further comprise:

upon determining that the transform-skip encoding is enforced,

generating an IBC predictor by applying the transform skip encoding to the current CU;

refining the IBC predictor by applying a residual block; and

generating a reconstruction block using the refined IBC predictor.
The computer-readable storage medium of claim 19, wherein the operations further comprise:

upon determining that the first indicator is the palette-based residual flag:

determining whether a value of the palette-based residual flag is on or off; and

upon determining that the value of the palette-based residual flag is off, generating a reconstructed block using a predictor of the current CU.
The computer-readable storage medium of claim 25, wherein the operations further comprise:

upon determining that the value of the palette-based residual flag is on, generating a reconstructed block using the predictor of the current CU if a maximal palette index associated with the current is equal to 0.
The computer-readable storage medium of claim 26, wherein the operations further comprise:

upon determining that the value of the palette-based residual flag is on and that the maximal palette index associated with the current is not equal to 0:

refining the predictor of the current CU by applying a palette-based prediction and residual block combination; and

generating a reconstruction block using the refined predictor.