CN112219399A - Palette-based residual coding in video compression systems - Google Patents
Palette-based residual coding in video compression systems Download PDFInfo
- Publication number
- CN112219399A CN112219399A CN201880094272.9A CN201880094272A CN112219399A CN 112219399 A CN112219399 A CN 112219399A CN 201880094272 A CN201880094272 A CN 201880094272A CN 112219399 A CN112219399 A CN 112219399A
- Authority
- CN
- China
- Prior art keywords
- palette
- block
- pixels
- prediction
- predicted
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000006835 compression Effects 0.000 title abstract description 49
- 238000007906 compression Methods 0.000 title abstract description 49
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims abstract description 78
- 238000000034 method Methods 0.000 claims abstract description 54
- 238000007670 refining Methods 0.000 claims description 8
- 239000003086 colorant Substances 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/136—Incoming video signal characteristics or properties
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/46—Embedding additional information in the video signal during the compression process
- H04N19/463—Embedding additional information in the video signal during the compression process by compressing encoding parameters before transmission
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
Systems and methods are provided for improving decoding performance of a video compression system by combining a palette mode and a legacy prediction method, wherein the method includes deriving a prediction block for a coding unit; decoding palette-based residual information for the coding unit; deriving a palette-based residual block based on the decoded palette-based residual information; combining the prediction block and the palette-based residual block to construct a reconstructed block.
Description
Background
Recent video coding standards, such as the H.264/AVC (advanced video coding) and H.265/HEVC (high efficiency video coding) standards, are based on hybrid coding schemes using block prediction and transform coding. To compress a video frame, a video compression system applies a prediction method, such as intra-frame prediction or inter-frame prediction, to derive a predicted frame. Based on the predicted frame, the video compression system also encodes residual information to refine the predicted frame. The residual information is typically compressed by transform and quantization techniques. Based on this information, the decoder is able to reconstruct the video frame from the predicted frame and the residual information. The decoder combines the prediction block and the residual block to generate a reconstructed block by simply adding the prediction pixel values and the residual pixel values to generate a final reconstructed pixel value.
A new coding tool, palette mode, is employed to standardize the screen content coding extension for high efficiency video coding (HEVC SCC). The palette mode can efficiently describe all pixels using few selected representative colors in the Coding Unit (CU). The palette mode is designed according to the following observations: the pixel values in a frame of screen content are typically concentrated on a few color values. The encoder analyzes the pixels in the CU and determines several representative colors to build a palette table, i.e. a color mapping table between representative color values and color indices. The palette table is signaled in the bitstream. Pixels having pixel values close to the palette color are quantized to the selected palette color represented by the corresponding palette index. The remaining pixels are called escape pixels (escape pixels). A special palette index value is reserved to represent the escape pixel and the pixel value of the escape pixel is signaled directly. All palette indices in a CU form a palette index map that is sent to the decoder along with escape pixel values. The pixel or palette index may represent a monochrome or tristimulus component value, such as YCbCr or GBR.
However, in HEVC palette mode, if a block is coded by the palette mode, the block will be decoded independently without using any prediction method or residual refinement.
Drawings
The detailed description is set forth with reference to the accompanying drawings. In the drawings, the left-most digit(s) of a reference number identifies the drawing in which the reference number first appears. The use of the same reference numbers in different figures indicates similar or identical items or features.
Fig. 1 illustrates an example decoding process of a video compression system using a prediction method.
Fig. 2 illustrates an example prediction block, an example residual block, and an example reconstructed block derived by the video compression system of fig. 1.
Fig. 3 shows an exemplary diagram of a palette mode applied to an image.
Fig. 4 illustrates an example decoding process for a video compression system using HEVC SCC palette mode.
Fig. 5 illustrates an example palette index mapping using HEVC SCC palette mode.
Fig. 6 illustrates an example flow diagram for palette-based residual coding.
FIG. 7 shows an example flow diagram detailing one of the blocks of FIG. 6.
Fig. 8 illustrates an example decoding process for a video compression system using a palette-based residual coding mode.
Fig. 9 illustrates an example prediction block, an example palette-based residual block, and an example reconstructed block derived by the video compression system of fig. 8 using palette-based residual decoding.
Fig. 10 illustrates example palette index map coding for palette-based residual coding.
Fig. 11 illustrates another example palette index map coding for palette-based residual coding.
FIG. 12 illustrates an example system for implementing processes and methods for improving decoding performance by combining palette mode and prediction methods.
Detailed Description
The systems and methods discussed herein relate to improving video compression system performance in HEVC palette mode, and more particularly to improving decoding performance by combining palette mode and prediction methods.
Fig. 1 illustrates an example decoding process 100 for a video compression system 102 that uses a predictive approach.
The video compression system 102 first derives the prediction block 104 by applying a prediction method such as intra-prediction 106 or inter-prediction 108, and then decodes the residual block 110 by applying a residual decoding method 112. The video compression system 102 combines the prediction block 104 and the residual block 110 to generate a reconstructed block 114.
Fig. 2 illustrates an example prediction block 104, residual block 110, and reconstruction block 114 derived by the video compression system 102 of fig. 1.
The prediction block 104 and the residual block 110 are shown with pixel values of a Coding Unit (CU). As discussed above with reference to fig. 1, corresponding pixel values of the prediction block 104 and the residual block 110 are simply added to generate corresponding pixel values of the reconstructed block 114.
FIG. 3 shows an example diagram 300 of a palette mode applied to a CU 302. For simplicity, the pixel or palette index is shown to correspond to only one value. However, in HEVC SCC, a pixel or palette index may represent three color component values, e.g., YCbCr or GBR.
In HEVC SCC palette mode, a flag is sent for each CU to indicate whether the palette mode is used for that CU, e.g., CU 302. If a palette mode is used for CU302, the pixels (having pixel values close to the palette, e.g., color A304, color B306, and color C308) are represented by palette color values 310, as shown in color histogram 312. For example, where color a 304 is shown near palette color value 50, color B306 is shown near palette color value 80, and color C308 is shown near palette color value 310. Palette color values 310 for CU302 are represented by palette indices 314, where palette color values 310 are representative colors for CU302, as shown in palette table 316. The remaining pixels are represented by special palette index values (denoted as escape indices) and their pixel values are signaled directly. Note that the color is a 3-value vector, such as YCbCr or GBR. The palette indices for all pixels in CU302 form palette index map 318.
Fig. 4 illustrates an example decoding process 400 for a video compression system 402 using HEVC SCC palette mode.
In contrast to the intra/inter prediction method described with reference to fig. 1, if a block or CU is encoded by HEVC SCC palette mode 404, it is decoded independently to generate reconstructed block 406 without using any prediction method or residual refinement. To decode the palette coded blocks, a video decoder of the video compression system 402 decodes the palette colors and indices. The palette colors are described by a palette table, such as palette table 316, and are encoded by a palette table encoding tool. Based on the palette table, the pixels of the three color components are represented by palette indices (e.g., palette index 314). The palette indices for all pixels in the CU form a palette index map (e.g., palette index map 318) and are encoded by a palette index map encoding tool.
Fig. 5 illustrates an example palette index map encoding 500.
The video compression system 102 constructs a palette table predictor to predict a palette table (e.g., palette table 316) of the current CU by applying predictive coding, wherein the palette of the last palette coded CU is used as the predictor. After the palette table is encoded, the pixels in the current CU, along with other parameters encoded with the palette index map, are represented by palette indices 502, which correspond to the colors in the palette shown in the parameter table 504. The indices form a palette index map 506, and the palette index map 506 is divided into several runs and then sequentially encoded in a horizontal direction or a vertical horizontal scan. In this example, a horizontal scanning order is assumed, and the direction of scanning is displayed as an arrow in the palette index map 506. In the palette index map 506, 16 indices are encoded by run index 508, three runs R0, R1, and R2 with horizontal scan. There are two run modes 510, copy index mode (copy index) and copy above mode (copy above), and for each start position of the run, a flag is sent to indicate which run mode is used.
If the copy above mode is used, the run of pixels 512 will copy the palette index from the pixel above it. In this example, the stroke R2, is a copy of an upper stroke having a stroke value of 10. If the copy index mode is used, a palette _ index _ idc syntax element is signaled first, followed by a piece of palette indexpun information. The palette _ index _ idc syntax describes whether the run is a copy index mode (palette _ index _ idc ═ 0) or a copy above mode (palette _ index _ idc ═ 1), and the palette indexrun syntax describes the length of the run. The pixel 512 shares the same palette index 502, wherein the palette index 502 and the run value of the run of the pixel 512 are derived from palette _ index _ idc and palette indendexrun, respectively.
Fig. 6 illustrates an example flow diagram 600 for palette-based residual coding in a video compression system in an example.
At block 602, the video compression system may derive a prediction block of a Coding Unit (CU), such as prediction block 104, which may be derived using conventional prediction methods, such as intra-prediction or inter-prediction methods as discussed above with reference to fig. 1. The video compression system may construct a palette table predictor to predict the palette table of the current CU, and may utilize the palette of the last palette-coded CU as the predictor. The video compression system may predict a palette table of the coding unit using a palette of a non-neighboring coding unit with respect to the coding unit as a palette predictor, and apply a prediction method for the palette table to derive a prediction block. At block 604, the video compression system may decode palette-based residual information for the coding unit, and based on the decoded palette-based residual information, may derive a palette-based residual block for the coding unit at block 606. At 608, the video compression system may combine the predicted block and the palette-based residual block to construct a reconstructed block. The video compression system can refine the prediction block and improve the encoding and/or decoding performance, and thus the decoding performance of the video compression system, by combining the palette mode with the conventional prediction method based on the decoded palette-based residual information.
FIG. 7 shows an example flow chart detailing block 608 of FIG. 6.
The video compression system may refine the prediction block 104 based on the decoded palette-based residual information by based on the palette-based residual block in block 702, and classify pixels of the palette-based residual block into two groups, such as primary color pixels and prediction pixels in block 704. Pixels of the dominant color may have palette indices associated with the dominant color pixel values of the decoded palette-based residual block similar to the association shown in palette table 316. At block 706, the predicted pixel is assigned a predetermined color index, such as C0, C1, etc., and the video compression system may generate a palette index map based on the dominant color pixel and the predicted pixel at block 708.
The video compression system may identify whether a pixel in the palette-based residual block is a dominant color pixel or a predicted pixel by decoding the palette index map and identifying an index having a predetermined color index as the predicted pixel at block 710, or by identifying a pixel to be encoded by a prediction run as the predicted pixel at block 712. At block 714, the video compression system may use dominant color pixel values of the dominant color pixels and pixel values of a prediction block for the prediction pixels in the reconstructed block and transmit pixel values of the prediction block for the prediction pixels, thereby combining the palette mode with conventional prediction methods to improve decoding performance of the video compression system.
Fig. 8 illustrates an example decoding process 800 of a video compression system 802 that uses a palette-based residual coding mode.
In addition to the HEVC residual decoding described above with reference to fig. 4 and 5, the video compression system 802 may include the option of combining the palette mode with a conventional prediction method, as described above with reference to fig. 6 and 7. The video compression system 802 may derive the palette-based residual information by applying the palette-based residual decoding 806 to the palette-based residual information. The video compression system 802 may then construct a reconstructed block 808 by applying the palette-based prediction and residual block combination 810 to the information from the prediction block 104 and the palette-based residual block 804.
Fig. 9 illustrates a prediction block 104, a palette-based residual block 804, and a reconstruction block 808 derived by the video compression system 802 of fig. 8 using palette-based residual decoding.
The video compression system 802 may derive the prediction block 104 as described above with reference to fig. 1, and may then decode the palette-based residual block 804. Prior to reconstructing the reconstruction block 808, the video compression system 802 may derive and refine the prediction block 104 using the palette-based residual block 804 as described above with reference to fig. 8. The video compression system 802 may classify the pixels in the palette-based residual block 804 into two groups, dominant color pixels and predicted pixels. Dominant color pixels may be pixels having pixel values, such as C0 and C1 as shown, where the pixel values are associated with dominant color values in a decoding palette table, such as palette table 316 of FIG. 3. The predicted pixel may be a special pixel, shown as "P" in the palette-based residual block 804. If the pixel is encoded as "P," the video compression system 802 may directly use its corresponding predicted pixel value from the prediction block 104 as the corresponding reconstructed pixel value of the reconstruction block 808. If the pixel in the palette-based residual block 804 is a dominant color pixel in the decoded palette table that has a corresponding dominant color value (e.g., C0 or C1), then the video compression system 802 will reconstruct the reconstructed block 808 using the corresponding dominant color value, as shown.
Fig. 10 illustrates example palette index map coding 1000 for palette-based residual coding.
To identify whether a pixel in the palette-based residual block 804 is a dominant color pixel or a predicted pixel, the video compression system 802 may retain a special color index, e.g., 0, as shown below the palette index 1002 of the parameter table 1004, a larger index value or a predetermined value. The video compression system 802 may then decode the palette index map 1006 using a conventional palette mode, as described above with reference to fig. 5. The indices form a palette index map 1006, and the palette index map 1006 is divided into a plurality of runs and then encoded in a horizontal or vertical traversal scan order. In this example, a horizontal scanning order is assumed and the direction of scanning is shown as an arrow in the palette index map 1006. In the palette index map 1006, 16 indices are encoded by run indices, with four runs R0, R1, R2, R1, and R3 of the horizontal traversal scan shown in the run mode 1008. As shown in the trip mode 1010, there are two trip modes, namely a copy index mode and a copy above mode, and for each start position of the trip, a flag is sent to indicate which mode of operation is used.
If a pixel in the palette index map 1006 is identified as having a reserved color index, it will be identified as a predicted pixel. In fig. 10, the bottom four pixels are identified as predicted pixels. The video compression system 802 may reserve a special color index, in this example "0", for these predicted pixels. The video compression system 802 may use a copy index run R3 for the four predicted pixels, as indicated by the pixel run 1012, to identify the four predicted pixels.
Fig. 11 illustrates another example palette index map encoding 1100 for palette-based residual encoding.
To identify whether a pixel in the palette-based residual block 804 is a dominant color pixel or a predicted pixel, the video compression system 802 may utilize a special run, such as a PREDICTION run displayed as "PREDICTION (PREDICTION)" under the run mode 1102 run of the parameter table 1104, rather than retaining a special color index under the palette index 1106. The video compression system 802 may identify the predicted pixels of the palette-based residual block 804 as pixels encoded by a prediction run. Similar to the palette index map coding of fig. 10, the indices form a palette index map 1006, and the palette index map 1006 is divided into four runs and then coded in a horizontal scanning order as indicated by arrows. In the palette index map 1006, 16 indices are encoded by run index 1108, with four runs of R0, R1, R2, and R3 being scanned horizontally across. In the run mode 1102, there are three in total, namely, a copy index mode, a copy above mode and a predict mode, and for each start position of the run, a flag is sent to indicate which run mode to use. If a pixel in the palette index map 1006 is identified as having been predictive run-length encoded, it will be identified as a predicted pixel. Referring to fig. 11, the video compression system 802 may use a prediction run R3 for the four bottom pixels to encode as predicted pixels, as shown by a pixel run 1110, and identify the four pixels as predicted pixels.
FIG. 12 illustrates an example system 1200 for implementing the above-described processes and methods for improving decoding performance by combining palette mode and prediction methods.
The techniques and mechanisms described herein may be implemented by multiple instances of system 1200 as well as by any other computing device, system, and/or environment. The system 1200 shown in fig. 12 is only one example of a system and is not intended to suggest any limitation as to the scope of use or functionality of any computing device for performing the processes and/or procedures described above. Other well known computing devices, systems, environments, and/or configurations that may be suitable for use with an embodiment include, but are not limited to, personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, game consoles, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, implementations using field programmable gate arrays ("FPGAs"), application specific integrated circuits ("ASICs"), and the like.
The system 1200 may include one or more processors 1202 and a system memory 1204 communicatively connected to the one or more processors 1202. The one or more processors 1202 may execute one or more modules and/or processes to cause the one or more processors 1202 to perform various functions. In some embodiments, the processor 1202 may include a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), or both a CPU and a GPU, or other processing units or components known in the art. In addition, each processor 1202 may have its own local memory, which may also store program modules, program data, and/or one or more operating systems.
Depending on the exact configuration and type of system 1200, the system memory 1204 may be volatile, such as RAM, or non-volatile, such as ROM, flash memory, a miniature hard drive, a memory card, etc., or some combination thereof. The system memory 1204 may include one or more computer-executable modules 1206 that are executable by the one or more processors 1202.
The module 1206 may include, but is not limited to, a prediction module 1208, a residual module 1210, and a reconstruction module 1212. The prediction module 1208 may be configured to derive a prediction block of the coding unit as described above with reference to fig. 1. The prediction module may use a palette of non-adjacent coding units relative to the coding unit as a palette predictor to predict a palette table of the coding unit and apply a prediction method, such as intra prediction 102 or inter prediction 108, to the palette table, e.g., palette table 316, to derive a prediction block (e.g., prediction block 104). The residual module 1210 may be configured to decode palette-based residual information of the coding unit and derive a palette-based residual block, such as the palette-based residual block 804, based on the decoded palette-based residual information. The reconstruction module 1212 may be configured to combine the prediction block 104 and the palette-based residual block 804 to construct a reconstructed block, such as the reconstructed block 808, to improve the decoding performance of the system 1200.
The residual module 1210 may be further configured to decode the palette-based residual block 804 and classify pixels of the palette-based residual block into dominant color pixels and predicted pixels. The dominant color pixels may have palette indices associated with the dominant color pixel values of the decoded palette-based residual block 804. The residual module 1210 may additionally be configured to generate a palette index map, such as the palette index map 1006, based on the primary color pixels and the predicted pixels, and assign predetermined color indices, e.g., C0, C1, etc., to the predicted pixels as shown in fig. 9.
The reconstruction module 1212 may be further configured to identify whether the pixels based on the palette residual block are dominant color pixels or predicted pixels by: by decoding the palette index map 1006 and identifying indices having predetermined color indices as predicted pixels as discussed above with reference to fig. 10, or by identifying pixels encoded by a prediction run as predicted pixels as described above with reference to fig. 11. The reconstruction module 1212 may be configured to use dominant color pixel values of dominant color pixels in the reconstruction block 808 and transmit pixel values of a prediction block used to reconstruct prediction pixels in the block 808, thereby combining the palette mode and a conventional prediction method to improve decoding performance of the video compression system.
Some or all of the operations of the above-described methods may be performed by the execution of computer readable instructions stored on a computer readable storage medium, as described below. The term "computer readable instructions" as used in the specification and claims includes routines, applications, application modules, program modules, programs, components, data structures, algorithms, and the like. Computer-readable instructions can be implemented on various system configurations, including single-processor or multiprocessor systems, minicomputers, mainframe computers, personal computers, hand-held computing devices, microprocessor-based, programmable consumer electronics, combinations thereof, and the like.
The computer-readable storage medium may include volatile memory (e.g., Random Access Memory (RAM)) and/or nonvolatile memory (e.g., Read Only Memory (ROM), flash memory, etc.). Computer-readable storage media may also include other removable and/or non-removable storage, including but not limited to flash memory, magnetic storage, optical storage, and/or tape storage, which may provide non-volatile storage of computer-readable instructions, data structures, program modules, and the like.
Non-transitory computer-readable storage media are examples of computer-readable media. Computer-readable media includes at least two types of computer-readable media, namely computer-readable storage media and communication media. Computer-readable storage media includes volatile and nonvolatile, removable and non-removable media implemented in any process or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Computer-readable storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), Digital Versatile Disks (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information for access by a computing device. In contrast, communication media may embody computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, or other transport mechanism. As defined herein, computer-readable storage media does not include communication media.
Computer-readable instructions stored on one or more non-transitory computer-readable storage media that, when executed by one or more processors, may perform the operations described above with reference to fig. 1-12. Generally, computer readable instructions include routines, programs, objects, components, data structures, etc. that perform particular functions or implement particular abstract data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and/or in parallel to implement a process.
Example clauses
A. A method, comprising: deriving a prediction block for the coding unit; decoding palette-based residual information of the coding unit; deriving a palette-based residual block based on the decoded palette-based residual information; combining the prediction block and the palette-based residual block to construct a reconstructed block.
B. The method of paragraph a, wherein deriving the prediction block of the coding unit comprises: predicting a palette table of the predicted coding unit using a palette of a non-adjacent coding unit with respect to the coding unit as a palette predictor; and applying predictive coding to the palette table to derive the prediction block.
C. The method of paragraph a, wherein combining the prediction block and the palette-based residual block to construct a reconstructed block comprises: refining a prediction block based on the decoded palette-based residual information.
D. The method of paragraph C, wherein refining the prediction block based on the decoded palette-based residual information comprises: decoding the palette-based residual block; classifying pixels of the palette-based residual block into dominant color pixels and predicted pixels, the dominant color pixels having palette indices associated with dominant color pixel values of the decoded palette-based residual block; and generates a palette index map based on the dominant color pixels and the predicted pixels.
E. The method of paragraph D, wherein generating the palette index map based on the dominant color pixels and the predicted pixels comprises: is assigned to the predicted pixel.
F. The method of paragraph E, further comprising: identifying whether a pixel in the palette-based residual block is a dominant color pixel or a predicted pixel comprises: decoding the palette index map and identifying indices having predetermined color indices as predicted pixels; or identify pixels encoded by the prediction run as predicted pixels.
G. The method of paragraph F, wherein combining the prediction block and the palette-based residual block to construct a reconstructed block comprises: using dominant color pixel values of dominant color pixels in the reconstructed block; and transmitting pixel values of the prediction block for prediction pixels in the reconstructed block.
H. A computer-readable storage medium storing computer-readable instructions executable by one or more processors, which, when executed by the one or more processors, cause the one or more processors to perform operations comprising: deriving a prediction block for the coding unit; decoding palette-based residual information of the coding unit; deriving a palette-based residual block based on the decoded palette-based residual information; and combining the prediction block and the palette-based residual block to construct a reconstructed block.
I. The computer-readable storage medium of paragraph H, wherein deriving the prediction block for the coding unit comprises: predicting a palette table of the predicted coding unit using a palette of a non-adjacent coding unit with respect to the coding unit as a palette predictor; and applying predictive coding to the palette table to derive the prediction block.
J. The computer-readable storage medium of paragraph H, wherein combining the prediction block and the palette-based residual block to construct a reconstructed block comprises: refining a prediction block based on the decoded palette-based residual information.
K. The computer-readable storage medium of paragraph J, wherein refining the prediction block based on the decoded palette-based residual information comprises: decoding the palette-based residual block; classifying pixels of the palette-based residual block into dominant color pixels and predicted pixels, the dominant color pixels having palette indices associated with dominant color pixel values of the decoded palette-based residual block; and generates a palette index map based on the dominant color pixels and the predicted pixels.
L. the computer-readable storage medium of paragraph K, wherein generating the palette index map based on the dominant color pixels and predicted pixels comprises: a predetermined color index is assigned to the predicted pixel.
M. the computer-readable storage medium of paragraph L, further comprising identifying whether a pixel in the palette-based residual block is a dominant color pixel or a predicted pixel comprises: decoding the palette index map and identifying indices having predetermined color indices as predicted pixels; or identify pixels encoded by the prediction run as predicted pixels.
The computer-readable storage medium of segment M, wherein combining the prediction block and the palette-based residual block to construct a reconstructed block comprises: using dominant color pixel values of dominant color pixels in the reconstructed block; and transmitting pixel values of the prediction block for prediction pixels in the reconstructed block.
A system, comprising: one or more processors; a memory communicatively connected to the one or more processors, the memory storing computer-executable modules executable by the one or more processors, the computer-executable modules comprising: a prediction module configured to derive a prediction block of a coding unit; a residual module configured to decode palette-based residual information of the coding unit and derive a palette-based residual block based on the decoded palette-based residual information; a reconstruction module configured to combine the prediction block and the palette-based residual block to form a reconstructed block.
P. the system as paragraph O recites, wherein the prediction block is further configured to: predicting a palette table of the predicted coding unit using a palette of a non-adjacent coding unit with respect to the coding unit as a palette predictor; and applying predictive coding to the palette table to derive the prediction block.
The system of paragraph O, wherein the residual module is further configured to: decoding the palette-based residual block; classifying pixels of the palette-based residual block into dominant color pixels and predicted pixels, the dominant color pixels having palette indices associated with dominant color pixel values of the decoded palette-based residual block; and generates a palette index map based on the dominant color pixels and the predicted pixels.
The system of paragraph Q, wherein the residual module is further configured to assign a predetermined color index to the predicted pixel.
The system of paragraph R, wherein the reconstruction module is further configured to: identifying whether a pixel in the palette-based residual block is a dominant color pixel or a predicted pixel by: decoding the palette index map and identifying indices having predetermined color indices as predicted pixels; or identify pixels encoded by the prediction run as predicted pixels.
T. the system of paragraph S, wherein the reconstruction module is further configured to: using dominant color pixel values of dominant color pixels in the reconstructed block; and transmitting pixel values of the prediction block for prediction pixels in the reconstructed block.
Conclusion
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as example forms of implementing the claims.
Claims (20)
1. A method, comprising:
deriving a prediction block for the coding unit;
decoding palette-based residual information for the coding unit;
deriving a palette-based residual block based on the decoded palette-based residual information;
combining the prediction block and the palette-based residual block to construct a reconstructed block.
2. The method of claim 1, wherein deriving a prediction block for a coding unit comprises:
predicting a palette table of the predicted coding unit using a palette of a non-adjacent coding unit with respect to the coding unit as a palette predictor; and
applying predictive coding to the palette table to derive the prediction block.
3. The method of claim 1, wherein combining the prediction block and the palette-based residual block to construct a reconstructed block comprises:
refining a prediction block based on the decoded palette-based residual information.
4. The method of claim 3, wherein refining the prediction block based on the decoded palette-based residual information comprises:
decoding the palette-based residual block;
classifying pixels of the palette-based residual block into dominant color pixels and predicted pixels, the dominant color pixels having palette indices associated with dominant color pixel values of the decoded palette-based residual block; and
a palette index map is generated based on the dominant color pixels and the predicted pixels.
5. The method of claim 4, wherein generating a palette index map based on the dominant color pixels and predicted pixels comprises: is assigned to the predicted pixel.
6. The method of claim 5, further comprising identifying whether a pixel in the palette-based residual block is a dominant color pixel or a predicted pixel comprises:
decoding the palette index map and identifying indices having predetermined color indices as predicted pixels; or
Pixels encoded by the prediction run are identified as predicted pixels.
7. The method of claim 6, wherein combining the prediction block and the palette-based residual block to construct a reconstructed block comprises:
using dominant color pixel values of dominant color pixels in the reconstructed block; and
transmitting pixel values of the prediction block for prediction pixels in the reconstructed block.
8. A computer-readable storage medium storing computer-readable instructions executable by one or more processors, which, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
deriving a prediction block for the coding unit;
decoding palette-based residual information for the coding unit;
deriving a palette-based residual block based on the decoded palette-based residual information; and
combining the prediction block and the palette-based residual block to construct a reconstructed block.
9. The computer-readable storage medium of claim 8, wherein deriving the prediction block for the coding unit comprises:
predicting a palette table of the predicted coding unit using a palette of a non-adjacent coding unit with respect to the coding unit as a palette predictor; and
applying predictive coding to the palette table to derive the prediction block.
10. The computer-readable storage medium of claim 8, wherein combining the prediction block and the palette-based residual block to construct a reconstructed block comprises:
refining a prediction block based on the decoded palette-based residual information.
11. The computer-readable storage medium of claim 10, wherein refining the prediction block based on the decoded palette-based residual information comprises:
decoding the palette-based residual block;
classifying pixels of the palette-based residual block into dominant color pixels and predicted pixels, the dominant color pixels having palette indices associated with dominant color pixel values of the decoded palette-based residual block; and
a palette index map is generated based on the dominant color pixels and the predicted pixels.
12. The computer-readable storage medium of claim 11, wherein generating a palette index map based on the dominant color pixels and predicted pixels comprises: a predetermined color index is assigned to the predicted pixel.
13. The computer-readable storage medium of claim 12, further comprising identifying whether a pixel in the palette-based residual block is a dominant color pixel or a predicted pixel comprises:
decoding the palette index map and identifying indices having predetermined color indices as predicted pixels; or
Pixels encoded by the prediction run are identified as predicted pixels.
14. The computer-readable storage medium of claim 13, wherein combining the prediction block and the palette-based residual block to construct a reconstructed block comprises:
using dominant color pixel values of dominant color pixels in the reconstructed block; and
transmitting pixel values of the prediction block for prediction pixels in the reconstructed block.
15. A system, comprising:
one or more processors;
a memory communicatively connected to the one or more processors, the memory storing computer-executable modules executable by the one or more processors, the computer-executable modules comprising:
a prediction module configured to derive a prediction block of a coding unit;
a residual module configured to decode palette-based residual information of a coding unit and to derive a palette-based residual block based on the decoded palette-based residual information;
a reconstruction module configured to combine the prediction block and the palette-based residual block to form a reconstructed block.
16. The system of claim 15, wherein the prediction block is further configured to:
predicting a palette table of the predicted coding unit using a palette of a non-adjacent coding unit with respect to the coding unit as a palette predictor; and
applying predictive coding to the palette table to derive the prediction block.
17. The system of claim 15, wherein the residual module is further configured to:
decoding the palette-based residual block;
classifying pixels of the palette-based residual block into dominant color pixels and predicted pixels, the dominant color pixels having palette indices associated with dominant color pixel values of the decoded palette-based residual block; and
a palette index map is generated based on the dominant color pixels and the predicted pixels.
18. The system of claim 17, wherein the residual module is further configured to assign a predetermined color index to the predicted pixel.
19. The system of claim 18, wherein the reconstruction module is further configured to:
identifying whether a pixel in the palette-based residual block is a dominant color pixel or a predicted pixel by:
decoding the palette index map and identifying indices having predetermined color indices as predicted pixels; or
Pixels encoded by the prediction run are identified as predicted pixels.
20. The system of claim 19, wherein the reconstruction module is further configured to:
using dominant color pixel values of dominant color pixels in the reconstructed block; and
transmitting pixel values of the prediction block for prediction pixels in the reconstructed block.
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2018/094496 WO2020006707A1 (en) | 2018-07-04 | 2018-07-04 | Palette-based residual coding for video compression system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112219399A true CN112219399A (en) | 2021-01-12 |
Family
ID=69060708
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201880094272.9A Pending CN112219399A (en) | 2018-07-04 | 2018-07-04 | Palette-based residual coding in video compression systems |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112219399A (en) |
| WO (1) | WO2020006707A1 (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015101173A1 (en) * | 2013-12-30 | 2015-07-09 | Mediatek Singapore Pte. Ltd. | Method and apparatus for scaling parameter coding for inter-component residual prediction |
| US20150341643A1 (en) * | 2014-05-23 | 2015-11-26 | Futurewei Technologies, Inc. | Advanced screen content coding with improved palette table and index map coding methods |
| US20160100174A1 (en) * | 2014-10-06 | 2016-04-07 | Futurewei Technologies, Inc. | Palette Mode In High Efficiency Video Coding (HEVC) Screen Content Coding (SCC) |
| CN107005717A (en) * | 2014-11-12 | 2017-08-01 | 寰发股份有限公司 | Pixel decoding method is jumped out in index mapping encoding and decoding |
| CN107431827A (en) * | 2015-03-18 | 2017-12-01 | 寰发股份有限公司 | The method and apparatus of index graph code in video and compression of images |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4515832B2 (en) * | 2004-06-14 | 2010-08-04 | オリンパス株式会社 | Image compression apparatus and image restoration apparatus |
| US10362333B2 (en) * | 2014-01-02 | 2019-07-23 | Qualcomm Incorporated | Color index coding for palette-based video coding |
| EP3055830A4 (en) * | 2014-03-21 | 2017-02-22 | Huawei Technologies Co., Ltd. | Advanced screen content coding with improved color table and index map coding methods |
| US9544607B2 (en) * | 2014-08-25 | 2017-01-10 | Hfi Innovation Inc. | Method of palette index signaling for image and video coding |
-
2018
- 2018-07-04 CN CN201880094272.9A patent/CN112219399A/en active Pending
- 2018-07-04 WO PCT/CN2018/094496 patent/WO2020006707A1/en active Application Filing
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015101173A1 (en) * | 2013-12-30 | 2015-07-09 | Mediatek Singapore Pte. Ltd. | Method and apparatus for scaling parameter coding for inter-component residual prediction |
| US20150341643A1 (en) * | 2014-05-23 | 2015-11-26 | Futurewei Technologies, Inc. | Advanced screen content coding with improved palette table and index map coding methods |
| CN106415607A (en) * | 2014-05-23 | 2017-02-15 | 华为技术有限公司 | Advanced screen content coding with improved palette table and index map coding methods |
| US20160100174A1 (en) * | 2014-10-06 | 2016-04-07 | Futurewei Technologies, Inc. | Palette Mode In High Efficiency Video Coding (HEVC) Screen Content Coding (SCC) |
| CN107005717A (en) * | 2014-11-12 | 2017-08-01 | 寰发股份有限公司 | Pixel decoding method is jumped out in index mapping encoding and decoding |
| CN107431827A (en) * | 2015-03-18 | 2017-12-01 | 寰发股份有限公司 | The method and apparatus of index graph code in video and compression of images |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2020006707A1 (en) | 2020-01-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11290717B2 (en) | Encoding and decoding methods and apparatuses with multiple image block division manners | |
| US9788004B2 (en) | Method of color index coding with palette stuffing | |
| US20170302939A1 (en) | Adaptive screen and video coding scheme | |
| US10652537B1 (en) | Coding unit size adaptive palette mode for video compression system | |
| US10075725B2 (en) | Device and method for image encoding and decoding | |
| US20180249162A1 (en) | Method and apparatus for decoding a video using an intra prediction | |
| JP2017522839A (en) | Alignment palette encoding | |
| KR102586674B1 (en) | Improvement on boundary forced partition | |
| JP6164360B2 (en) | Image encoding device, image decoding device, image encoding method, and image decoding method | |
| US11350106B2 (en) | Method for encoding and decoding images, device for encoding and decoding images and corresponding computer programs | |
| US20230091192A1 (en) | Scanning orders for non-transform coding | |
| US20160269738A1 (en) | Method for encoding and decoding images, device for encoding and decoding images, and corresponding computer programmes | |
| US20170264909A1 (en) | Apparatus and method for image coding, and non-transitory computer-readable storage medium | |
| US20150023416A1 (en) | Image encoding device and image decoding device | |
| WO2020177545A1 (en) | Palette size constraint in palette mode for video compression system | |
| WO2020056757A1 (en) | Method, apparatus, and computer-readable storage medium for block partitioning tree separation under separation node | |
| CN112219399A (en) | Palette-based residual coding in video compression systems | |
| WO2020243881A1 (en) | Simplified intra block copy | |
| KR102031120B1 (en) | Method for coding and decoding intra-prediction, and atparatus for the same | |
| WO2020056760A1 (en) | Transform selection for small size block | |
| WO2020056759A1 (en) | Method, apparatus, and computer-readable storage medium for small size block coding | |
| WO2024010635A1 (en) | System and method for multiple-hypothesis prediction for video coding | |
| WO2022191947A1 (en) | State based dependent quantization and residual coding in video coding |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |