US20230132221A1 - Palette coding for screen content coding - Google Patents
Palette coding for screen content coding Download PDFInfo
- Publication number
- US20230132221A1 US20230132221A1 US18/086,740 US202218086740A US2023132221A1 US 20230132221 A1 US20230132221 A1 US 20230132221A1 US 202218086740 A US202218086740 A US 202218086740A US 2023132221 A1 US2023132221 A1 US 2023132221A1
- Authority
- US
- United States
- Prior art keywords
- palette
- coding
- index
- scan direction
- run
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 238000000034 method Methods 0.000 claims abstract description 54
- 230000011664 signaling Effects 0.000 abstract description 33
- 230000008569 process Effects 0.000 abstract description 16
- 230000001788 irregular Effects 0.000 abstract description 13
- 238000013461 design Methods 0.000 abstract description 8
- 239000003086 colorant Substances 0.000 description 43
- 238000010586 diagram Methods 0.000 description 43
- 238000004891 communication Methods 0.000 description 42
- 238000005516 engineering process Methods 0.000 description 18
- 238000013459 approach Methods 0.000 description 9
- 230000001419 dependent effect Effects 0.000 description 6
- 235000019580 granularity Nutrition 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 239000002356 single layer Substances 0.000 description 5
- 239000013598 vector Substances 0.000 description 5
- 230000003044 adaptive effect Effects 0.000 description 4
- 101150014732 asnS gene Proteins 0.000 description 4
- 230000006835 compression Effects 0.000 description 4
- 238000007906 compression Methods 0.000 description 4
- 238000005192 partition Methods 0.000 description 4
- 238000013139 quantization Methods 0.000 description 4
- 241000760358 Enodes Species 0.000 description 3
- 230000001413 cellular effect Effects 0.000 description 3
- 230000002093 peripheral effect Effects 0.000 description 3
- 230000002123 temporal effect Effects 0.000 description 3
- 235000017274 Diospyros sandwicensis Nutrition 0.000 description 2
- 241000282838 Lama Species 0.000 description 2
- 241000023320 Luma <angiosperm> Species 0.000 description 2
- 241000209094 Oryza Species 0.000 description 2
- 235000007164 Oryza sativa Nutrition 0.000 description 2
- 101100247669 Quaranfil virus (isolate QrfV/Tick/Afghanistan/EG_T_377/1968) PB1 gene Proteins 0.000 description 2
- 101100242901 Quaranfil virus (isolate QrfV/Tick/Afghanistan/EG_T_377/1968) PB2 gene Proteins 0.000 description 2
- 101150025928 Segment-1 gene Proteins 0.000 description 2
- 101150082826 Segment-2 gene Proteins 0.000 description 2
- 101100242902 Thogoto virus (isolate SiAr 126) Segment 1 gene Proteins 0.000 description 2
- 101100194052 Thogoto virus (isolate SiAr 126) Segment 2 gene Proteins 0.000 description 2
- 238000013475 authorization Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 229910001416 lithium ion Inorganic materials 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- OSWPMRLSEDHDFF-UHFFFAOYSA-N methyl salicylate Chemical compound COC(=O)C1=CC=CC=C1O OSWPMRLSEDHDFF-UHFFFAOYSA-N 0.000 description 2
- QELJHCBNGDEXLD-UHFFFAOYSA-N nickel zinc Chemical compound [Ni].[Zn] QELJHCBNGDEXLD-UHFFFAOYSA-N 0.000 description 2
- 210000003254 palate Anatomy 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 235000009566 rice Nutrition 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- HBBGRARXTFLTSG-UHFFFAOYSA-N Lithium ion Chemical compound [Li+] HBBGRARXTFLTSG-UHFFFAOYSA-N 0.000 description 1
- 101100242890 Quaranfil virus (isolate QrfV/Tick/Afghanistan/EG_T_377/1968) PA gene Proteins 0.000 description 1
- 241000700159 Rattus Species 0.000 description 1
- 101150027881 Segment-3 gene Proteins 0.000 description 1
- 101100242891 Thogoto virus (isolate SiAr 126) Segment 3 gene Proteins 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- INJRKJPEYSAMPD-UHFFFAOYSA-N aluminum;silicic acid;hydrate Chemical compound O.[Al].[Al].O[Si](O)(O)O INJRKJPEYSAMPD-UHFFFAOYSA-N 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000004873 anchoring Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- OJIJEKBXJYRIBZ-UHFFFAOYSA-N cadmium nickel Chemical compound [Ni].[Cd] OJIJEKBXJYRIBZ-UHFFFAOYSA-N 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000009849 deactivation Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 239000000446 fuel Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 229910052987 metal hydride Inorganic materials 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 229910052759 nickel Inorganic materials 0.000 description 1
- PXHVJJICTQNCMI-UHFFFAOYSA-N nickel Substances [Ni] PXHVJJICTQNCMI-UHFFFAOYSA-N 0.000 description 1
- -1 nickel metal hydride Chemical class 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000013138 pruning Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000009827 uniform distribution Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/136—Incoming video signal characteristics or properties
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/182—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a pixel
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/146—Data rate or code amount at the encoder output
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
- H04N19/91—Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
- H04N19/93—Run-length coding
Definitions
- Screen content sharing applications have become more popular in recent years with the expanded use of remote desktop, video conferencing, and mobile media presentation applications.
- the screen content may include numerous blocks with several major colors and sharp edges due to, for example, sharp curves and text inside.
- Existing video compression methods may not fully characterize the feature of screen content and may lead to a low compression performance, for example, reconstructed pictures may have quality issues.
- the curves and text may be blurred and/or they may be difficult to recognize. Therefore, a well-designed screen compression method is needed for effectively reconstructing screen content.
- a decoder may perform palette decoding of a coding unit (CU) of a picture.
- the decoder may parse a palette index run value of the CU.
- the palette index run value may indicate an index run or a copy run.
- the palette index run value may indicate a length of consecutive palette indices for corresponding pixel positions in the CU, and at least one of the consecutive palette indices may corresponds to an escape color index.
- the consecutive palette indices may correspond to a first escape color index and a second escape color index.
- the decoder may reconstruct the consecutive palette indices for the corresponding pixel positions of the CU according to the palette index run values.
- the decoder may reconstruct color values for the corresponding pixel positions of the CU according to the consecutive palette indices and one or more palette_escape_val values.
- the decoder may parse, during a first pass, palette syntax values sufficient to reconstruct all palette indices in the CU (e.g., which may include a palette mode flag(s), a palette index value(s), and/or a run values) for index mode, or a palette mode flag(s) and/or a run value(s) for copy mode) other than the one or more palette_escape_val values, and parse, during a second pass, the one or more palette_escape_val values for the corresponding pixel positions in the CU for which the palette index parsed from the first pass corresponds to the escape color index.
- palette syntax values sufficient to reconstruct all palette indices in the CU (e.g., which may include a palette mode flag(s), a palette index value(s), and/or a run values) for index mode, or a palette mode flag(s) and/or a run value(s) for copy mode) other than the one or more palette_escape_val values
- a decoder may performing palette decoding for a CU of a picture.
- the decoder may determine a palette table size for a coding unit (CU) of a picture. For example, the decoder may determine that the palette table size of the CU is zero.
- the palette table size for the CU being equal to zero may indicate that all pixels of the CU are coded as escape colors. If the decoder determines that the palette table size of the CU is zero, than the decoder may decode the escape values (e.g., by inferring that a palette_escape_val_present_flag for the CU is equal to one). For example, the palette_escape_val_present_flag may not be received for the CU if the palette table size is zero.
- the decoder may parse a palette_escape_val_present_flag for the CU.
- the decoder may reconstruct palette indices for corresponding pixel positions of the CU based on a palette table for the CU and a value associated with the palette_escape_val_present_flag of the CU.
- a decoder may performing a palette decoding for a CU of a picture.
- the decoder may determine that a coding unit (CU) of a picture comprises a single major color (e.g., only one major color) or only escape colors.
- the decoder may determine that the CU is encoded according to an index mode and a palette index mode (e.g., based on not receiving a palette_mode for the CU).
- the decoder may reconstruct color values for the corresponding pixel positions of the CU.
- the decoder may determine that a run value of the index mode for the CU is equal to a size of the CU minus one based on not receiving a syntax value indicative of a palette run value for the C.U.
- the decoder may determine that the palate index value for the CU is equal to zero based on not receive a palette index value for the CU.
- the decoder may determine that a palette_escape_val_present_flag for the CU is equal to zero and/or determine that a palette table size for the CU is equal to one, for example, to determine that the CU comprises a single major color.
- the decoder may determine that a palette_escape_val_present_flag for the CU is equal to one and/or determine that a palette table size for the CU is equal to zero, for example, to determine that the CU comprise only escape colors.
- the decoder may infer the palette_mode of the CU to be equal to zero based on not receiving the palette_mode for the CU.
- the decoder may receive a palette_escape_val for one or more pixel positions (e.g., each pixel position) in the CU, for example, if the CU comprises only escape colors.
- Sketch copy mode may be used to code blocks comprising irregular lines, syntax redundancy may be removed from blocks with special characteristics, and/or run value coding may be simplified.
- the parsing dependencies in palette coding design may be removed.
- the context modeling dependency of the syntax element palette_transpose_flag may be removed, for example, using one single context.
- the context modeling dependency of the syntax element palette_mode may be removed, for example, by using nun length coding without using context or using one single context.
- the syntax parsing dependencies and/or the syntax signaling dependencies that are related with escape color signaling may be removed,
- a palette table generation process may handle input screen content video with high hit depths, for example, at the encoder side.
- FIG. 1 is a diagram that illustrates an example block diagram of a screen content sharing system.
- FIG. 2 is a diagram illustrating an example of a block-based single layer video encoder.
- FIG. 3 is a diagram illustrating an example of a block-based single layer video decoder.
- FIG. 4 is a diagram illustrating an example of eight partition modes.
- FIG. 5 A is a diagram illustrating an example of a screen content block.
- FIG. 5 B is a diagram illustrating n example of a palette index map of the screen content block of FIG. 5 A .
- FIG. 6 is a diagram illustrating an example SCC test sequence for a console.
- FIG. 7 A is a diagram illustrating an example 8 ⁇ 8 block with four colors, where the dashed line illustrates a scanning order in palette coding.
- FIG. 7 B is a diagram illustrating an example 8 ⁇ 8 block with four colors, where the dashed line illustrates a scanning order in palette coding that uses index mode and copy mode.
- FIG. 8 A is a diagram illustrating an example of 16 directions with course granularity.
- FIG. 8 B is a diagram illustrating an example of 16 directions with fine granularity.
- FIG. 9 is a diagram illustrating an example flow chart of the direction coding algorithm for sketch copy mode.
- FIG. 10 is a diagram illustrating an example of palette coding with sketch copy mode.
- FIG. 11 A illustrates an example of a coding order of a horizontal traverse scan.
- FIG. 11 B illustrates an example of a coding order of a vertical traverse scan.
- FIG. 11 C illustrates an example of a coding order in a reverse horizontal traverse scan.
- FIG. 11 D illustrates an example of a coding order in a reverse vertical traverse scan.
- FIG. 12 A is a diagram illustrating an example of a redundancy removal method of palette index coding where the CPI position is coded in index mode.
- FIG. 12 B is a diagram illustrating an example of a redundancy removal method of palette index coding where the CPI position is coded in copy mode.
- FIG. 13 is a diagram of an example of extended copy mode.
- FIG. 14 A is a system diagram of an example communications system in which one or more disclosed embodiments may be implemented.
- FIG. 14 B is a system diagram of an example wireless transmit/receive unit (WTRU) that may be used within the communications system illustrated in FIG. 14 A .
- WTRU wireless transmit/receive unit
- FIG. 14 C is a system diagram of an example radio access network and an example core network that may be used within the communications system illustrated in FIG. 14 A .
- FIG. 14 D is a system diagram of another example radio access network and an example core network that may be used within the communications system illustrated in FIG. 14 A .
- FIG. 14 E is a system diagram of another example radio access network and an example core network that may be used within the communications system illustrated in FIG. 14 A .
- the screen display of mobile devices may be able to display high definition resolution content, ultra-high definition resolution content, and/or the like.
- Video coding tools such as block coding modes and transform, may not be optimized for screen content encoding.
- FIG. 1 is a diagram that illustrates an example block diagram of a screen content sharing system.
- the screen content sharing system may include a receiver, a decoder, and/or a display (renderer).
- FIG. 2 is a diagram illustrating an example of a block-based single layer video encoder.
- FIG. 3 is a diagram illustrating an example of a block-based single layer video decoder.
- the video decoder of FIG. 3 may receive a video bitstream produced by the encoder of FIG. 2 .
- the video decoder may reconstruct the video signal to be displayed.
- the bitstream may be parsed by an entropy decoder.
- the residual coefficients may be inverse quantized and inverse transformed to obtain the reconstructed residual.
- the coding mode and/or prediction information may be used to obtain the prediction signal using spatial prediction and/or temporal prediction.
- the prediction signal and/or the reconstructed residual may be added together to generate the reconstructed video.
- the reconstructed video may go through loop filtering before being stored in a reference picture store.
- the reconstructed video may be displayed and/or to be used to decode future video signal(s).
- a single layer encoder may employ spatial prediction (e.g., intra prediction) and/or temporal prediction (e.g., inter prediction and/or motion compensated prediction) to predict the input video signal.
- the encoder may include mode decision logic that selects a form of prediction, for example, based on rate and/or distortion considerations.
- the encoder may transform and quantize the prediction residual (e.g., the difference signal between the input signal and the prediction signal).
- the quantized residual, the mode information (e.g., intra or inter prediction), and/or prediction information e.g., motion vectors, reference picture indexes, intra prediction modes, and/or the like) may be compressed at the entropy coder and packed into the output video bitstream. As shown in FIG.
- the encoder may generate the reconstructed video signal by applying inverse quantization and inverse transform to the quantized residual to obtain a reconstructed residual, and add the reconstructed residual to the prediction signal.
- the reconstructed video signal may go through loop filter process (e.g., deblocking filter. Sample Adaptive Offsets, Adaptive Loop Filters, and/or the like)
- the reconstructed video signal may be stored in the reference picture store to he used to predict future video signal.
- High Efficiency Video Coding may be a block based hybrid video coding standard, in that its encoder and decoder may operate substantially in accordance with the example encoder and decoder of FIG. 2 and FIG. 3 .
- HEVC may allow the use of larger video blocks and/or may use quadtree partition to signal block coding information.
- the picture or slice may be partitioned into coding tree blocks (CTB) with the same size (e.g 64 ⁇ 64).
- a CTB may be partitioned into CUs with quadtree,
- a CU may be partitioned into prediction units (PU) and/or transform units (TU) with quadtree.
- FIG. 4 is a diagram illustrating an example of eight partition modes.
- its PU may be one of eight partition modes, for example, as shown in FIG. 4 .
- Temporal prediction e.g., motion compensation
- linear filters may be applied to obtain pixel values at fractional positions.
- the interpolation filters may have seven or more taps for lama and/or four taps for chroma.
- the deblocking filter in HEVC may be content based, for example, such that different deblocking filter operations may be applied at the TU and PU boundaries, for example, depending on coding mode difference, motion difference, reference picture difference, pixel value difference, and/or the like.
- HEVC may use context-based adaptive arithmetic binary coding (CABAC) for block level syntax elements (e.g., except high level parameters).
- CABAC coding may include context-based coded regular bins and/or by-pass coded bins without context.
- HEVC may be focused on continuous tone video content in 4:2:0 format.
- the mode decision and transform coding tools may not be optimized for the discrete tone screen content, for example, which may be captured in the format of 4:4:4 video.
- FIG. 5 A is a diagram illustrating an example of a screen content block.
- FIG. 5 B is a diagram illustrating an example of a palette index map of the screen content block of FIG. 5 A .
- screen content blocks may include a limited number of colors, and the color value of each pixel may be repeated from its above and/or left pixel.
- a palette table may be used as a dictionary to record significant color values,
- the corresponding palette index map may be used to represent the color value of each pixel, for example, as shown in FIG. 5 B .
- the run values may be used to indicate the length of consecutive pixels that have the same color (e.g., palette index) to reduce the spatial redundancy.
- Using a palette based coding method may improve the compression performance of coding screen content.
- Palette coding modes may include an index mode (e.g., run mode) and/or a copy mode.
- the color index may be coded first. If the color index is equal to the palette table size, it may indicate that an escape color is not included in the palette table. And the color value may be explicitly coded following the color index. If the color index is less than the palette table size, which may mean a major color in the palette table, then a run value may be coded to signal how many continuous pixels in the scanning order have the same color as that coded color. In the copy mode, the run value may be coded to indicate how many pixels have the same color as its above pixel. Scanning orders may include, but are not limited to, a horizontal traverse scan and/or a vertical traverse scan, for example, as shown in FIG. 11 A and FIG. 11 B and as described herein.
- the palette index of the first position in the run-length chain may be present in the bit-stream.
- Palette indices may be binarized using truncated binary code (TBC) and/or coded in bypass mode.
- TBC may be a variant of fixed length code (FLC) and/or used for alphabets with uniform distributions.
- FLC fixed length code
- TBC may degenerate to FLC when the size of the alphabet is a power of two. For example, for one palette index level pLevel, it may be assumed that its maximum value pMax is known.
- the maximum level pMax may be specified as an input to the TBC process.
- one redundancy removal method may be applied to improve the efficiency of palette index coding by reducing the magnitude of coded palette indices and/or the maximum length of TBC codewords. For example, one or more of the following conditions may be checked before coding one palette index in the index mode. If the left neighbor of the current pixel position is coded in the index mode, then the current palette index may be the same as that of its left neighbor. Otherwise, if the two palette indices are the same, the palette indices may be coded together in index mode with a larger run-length. If the left neighbor of the current pixel position is coded in copy mode, then the current palette index may be the same as that of its above neighbor.
- the palette indices may be coded together in copy mode with a larger run-length. Therefore, if either of the above two conditions are satisfied, the palette index pLevel and/or the maximum TBC value pMax may be reduced (e.g., by one), which may decrease the codeword length used to present the palette index of the current pixel position.
- FIG. 12 A is a diagram illustrating an example of a redundancy removal method of palette index coding where the CPI position is coded in index mode.
- FIG. 12 B is a diagram illustrating an example of a redundancy removal method of palette index coding where the CPI position is coded in copy mode.
- FIG. 12 A and FIG. 12 B illustrate examples of the redundancy removal process when the left neighboring pixel is coded in index mode (e.g., as shown in FIG. 12 A ) and in copy mode (e.g., as shown in FIG. 12 B ).
- the compared palette index (CPI) position k that is referred by the current pixel position i, as enclosed by the dashed black line in FIG. 12 A and FIG. 12 B , may be calculated as follows:
- iWidth is the width of the current CU.
- the palette index may be independently coded rather than coded in index mode or copy mode, for example, although escape colors may be indicated by the palette index which is equal to the palette size in the current palette coding design. In other words, one run-length chain of pixel positions with the same palette index value may be broken when an escape color position is scanned. As the palette index used for escape colors may not be involved in the run-length based coding, the above redundancy removal algorithm used when coding palette indices may be applicable (e.g., only applicable) if the corresponding CPI position is not a position with an escape color.
- palette indices may be predicted from the above decoded indices in the above row.
- the copy mode may be disabled such that they are coded (e.g., always coded) by index mode, for example, because the above reference pixels exceed the CU boundary.
- the copy mode may be enabled for the pixels in the first row (e.g., or the first column when palette_transpose_flag is equal to 1) by using the boundary pixels of the neighboring CUs as reference, for example, in order to improve the efficiency of copy mode.
- FIG. 13 shows one example of the extended copy mode when palette_transpose_flag is equal to 0 (e.g., horizontal scan order).
- the decoder may reconstruct them from the corresponding neighboring reference pixels (NCPs) from the above neighboring CU.
- NCPs neighboring reference pixels
- the color values of NCPs may be used for the copy mode of the first line and the decoder may not have or derive palette indices of NCPs, for example, as shown in FIG. 13 . Therefore, palette index coding redundancy removal method may not be applied to pixels whose CPI pixels make reference to NCPs.
- the index coding redundancy removal method may be disabled.
- the syntax elements may be binarized into a string of bins, in which bypass bins and context-coded bins may be coded separately by the engine of binary arithmetic coding (BAC), for example, for the context-based binary arithmetic coding (CABAC) that is used for palette mode and the other coding modes in HEVC screen content coding extension.
- BAC binary arithmetic coding
- CABAC context-based binary arithmetic coding
- context models e.g., probability models
- syntax elements may be coded without context models.
- palette_index_idc may be bypass coded and/or interleaved with other context-coded syntax elements, such as, palette_run_type and palette_run_msb_id_plus1, for example, for palette design in HEVC screen content coding.
- the parsing of palette_index_idc may be grouped together and put in front of the other context coded syntax elements, for example, to improve the parsing throughput.
- Table 2 is an example of such a syntax table of palette coding.
- the syntax element num_palette_index may specify the number of palette index signalled in the current block.
- the syntax element palette_last_run_type_flag may specify the palette coding mode of the last run in the current block.
- One or more embodiments described herein may be written using the HEVC standard as the underlying video coding standard, however, the embodiments may be applied to other video codecs.
- Palette coding efficiency may be provided.
- the palette coding may code horizontal lines and/or vertical lines. If it is a horizontal line, the index mode may code continuous pixels with a run value (e.g., a large run value). If it is a vertical line, then the vertical line may be treated as a horizontal line in a vertical traverse scan order, for example, as shown in FIG. 11 B . Lines in various directions may be used in screen content.
- FIG. 6 is a diagram illustrating an example SCC test sequence for a console.
- FIG. 7 A is a diagram illustrating an example 8 ⁇ 8 block with four colors, where the dashed line illustrates a scanning order in palette coding.
- FIG. 7 B is a diagram illustrating an example 8 ⁇ 8 block with four colors, where the dashed line illustrates a scanning order in palette coding that uses index mode and copy mode. For example, there may be two irregular lines in one 8 ⁇ 8 block, for example, as shown in FIG. 7 A .
- FIG. 7 A and FIG. 7 B illustrates an example of palette coding with run and copy mode in horizontal traverse scanning order for the 8 ⁇ 8 block.
- 17 runs may be coded with index mode, for example, shown in black dashed lines and copy modes shown in black solid lines.
- Parsing dependencies of palette coding may be provided. For example, given the high coding efficiency of palette coding for screen content, there may be a large percentage of CUs that select palette coding as the optimal coding mode, and thus palette coding may limit data parsing dependencies to enable high-throughput implementations. There may be a plurality of different kinds of dependencies in palette coding design. For example, context modeling dependencies may be used, which may refer to the dependencies in context derivation of one syntax element on previously coded syntax elements. Syntax parsing dependencies may be used, which may refer to the dependencies that the decoded value of one syntax element at a pixel position is used as an input to the parsing of the syntax elements for the following pixel positions. Syntax signaling dependencies may be used, which may refer to the dependencies that the signaling of one syntax element may be dependent on the decoded value of other syntax elements.
- Context modeling dependencies may be provided. Context modeling dependency of a palette_transpose_flag may be provided. Horizontal traverse scans and/or vertical traverse scans may be enabled to scan the palette indices of one palette-coded CU. One flag palette_transpose_flag may be signaled for each CU coded with palette mode to select one of the two scan patterns for the current CU. Two context models may be used to code the flag palette_transpose_flag, which may be selected depending on whether the palette indices of the left neighboring CU of the current CU are scanned horizontally or vertically. The neighbor-based context modeling approach may provide higher coding efficiency.
- the neighbor-based context modeling approach may not allow for a high degree of parallelism because two neighboring palette-coded CUs, which may not be decoded simultaneously.
- the neighbor-based context modeling approach may use one additional buffer to store the status of a palette_transpose_flag for a left neighboring CU.
- Context modeling dependency of palette_mode may be provided.
- a flag palette_mode may be signaled for one or more pixel positions, for example, except for the pixel positons in the first row of one palette-coded CU (e.g., as there may be no above pixels to copy, the copy mode may be invalid for the pixel positions in the first row).
- Two context models may be used to code the palette_mode flag, which may be determined based on the value of the same syntax element of the collocated pixel in the above row. Accordingly, the context modeling dependency of palette_mode approach may reduce the throughput, as the determination of the context at one pixel may be dependent on the palette index coding mode of pixels preceding it in the scan order.
- Dependencies related to escape color signaling may be provided. Syntax parsing dependencies of palette_index may be provided. For pixels coded in index mode, the palette index of the first pixel in the run-length chain may be signaled via the syntax element palette_index.
- the syntax element palette_index may be binarized using TBC, for which the maximum value of palette index (e.g., pMax) may be used as input to the TBC process, The calculation of the input pMax to the TBC process of one pixel position may be dependent on whether the left neighbor and the CPI position of the current pixel position are coded as escape colors or not, for example, because the redundancy removal approach may be applied to palette index coding and the palette index used to indicate escape colors may not be coded in index mode or copy mode.
- pMax may be set to the palette table size Size _PLT of current CU. Otherwise (e.g., if both the left neighbor and the CPI position of the current pixel position are major colors), pMax may be set to Size _PLT ⁇ 1.
- the escape color may be identified by checking if the decoded palette index at one pixel position is equal to the palette table size. Therefore, the decoded value of the current palette_index syntax element may determine the value of the next syntax element to be processed.
- One palette index may be decoded (e.g., only one palette index) after the palette index of its CPI position is completely reconstructed, for example, when the redundancy removal approach is used in palette index coding.
- Syntax signaling dependencies of palette_escape_val may be provided.
- the color values of the position may be quantized (e.g., if lossy coding is used) and transmitted to a decoder.
- escape colors may be present in a bitstream by signaling the syntax element palette_index (e.g., which is equal to Size _PLT when the redundancy removal approach is not applied, and Size _PLT ⁇ 1 when the redundancy removal approach is applied) followed by another syntax element palette_escape_val, which may indicate the quantized color values (e.g., if lossy coding is used).
- the syntax palette_escape_val element may be signaled (e.g., only signaled) when one pixel is identified as an escape color as indicated by palette_index. Therefore, the value of a palette_index syntax element may determine what syntax element is to be processed. For example, if palette_index indicates the current pixel is an escape color, then the next syntax element may be palette_escape_val, and otherwise the next syntax element may be palette_run. Based on the same analysis of the syntax parsing dependencies of palette_index, one pixel may be (e.g., may only be) identified as an escape color after the palette index of its CPI position is fay reconstructed, for example, due to the redundancy removal process applied to palette index coding. Therefore, the syntax signaling dependency of escape colors may cause throughput issues.
- Syntax signaling dependencies of palette_run may be provided.
- the syntax element palette_run may indicate the number of consecutive pixels with the same palette index in index mode and copy mode.
- the signaling of the syntax element palette_run may be signaled (e.g., only signaled) when one pixel is identified as a major color, which for example, may in contrast to palette_escape_val. Therefore, the similar syntax signaling dependency issue that is applicable to the element palette_escape_val may exist for the signaling of palette_run.
- Palette table generation (e.g., at the encoder only) may be provided.
- One color-clustering-based palette table generation method may be used to select the major colors of one CU coded with palette mode by clustering the color values of the current CU into multiple sets and using the centroids of color clusters as the major colors in the palette table.
- one color cluster may be generated by quantizing the pixels, for which the distortion between the color value and the centroid of the cluster may be a pre-defined threshold, to the same major color. This threshold may be set based on the assumption that the bit depths of luma and chroma components are equal to 8 bits. Therefore, the current palette table generation method may not properly handle the case when the bit depths of input screen content video are more than 8 bits.
- One or more embodiments may address the conflicts for the combination of the extend copy mode and grouping palette indices at front.
- the palette index coding bins may be grouped at the front of one palette CU.
- Palette indices (e.g., as specified by the syntax element palette_index_idc) may be binarized by TBC code for which the maximum possible level need to be known. The redundancy removal may not be applied to pixels whose CPI pixels make reference to NCPs. Therefore, different maximum TBC levels may be derived depending on the positions of pixels.
- the maximum TBC level pMax may be kept unchanged (e.g, the size of the palette table minus 1 when there is no escape color pixel in the current CU, or the size of the palette table when there is at least one escape color pixel in the current CU).
- the maximum TBC level pMax may be reduced by 1.
- the parsing of the syntax palette_index_idc for one pixel may be dependent on the knowledge of the palette coding modes of the preceding pixels in the palette scan order, And, this knowledge may be acquired after decoding the palette coding modes (e.g., palette_run_type_flag) and palette runs (e.g., palette_run_msb_id_plus1 and palette_run_refinement_bits) for the preceding pixels of the current pixel. Therefore, when the extended copy mode is enabled, the parsing of the grouped palette indices may not be placed in front of the parsing of palette modes and palette runs.
- palette coding modes e.g., palette_run_type_flag
- palette runs e.g., palette_run_msb_id_plus1 and palette_run_refinement_bits
- a palette signaling method may reduce the maximum TBC level pMax by one, for example, in order to group palette indices at beginning of palette parsing process while enabling extended copy-above.
- the dynamic range of the palette indices for those pixels may be 0 to pMax, for example, since redundancy removal may not be applied to pixels whose CPI pixels refer to the NCPs from the neighboring CU.
- An additional flag may be signaled for those pixels (e.g., the pixels whose CPI pixels refer to the NCPs from the neighboring CU) when the corresponding parsed palette indices are equal pMax ⁇ 1, for example, in order to compensate the reduced dynamic range of palette indices. If the flag is equal to 1, the palette index may be decoded to be pMax. If the flag is equal to 0, the palette index may be decoded to be pMax ⁇ 1.
- Table 3 is an example palette syntax table.
- palette_run_type_flag ! last_palette_run_type_type_
- One or more embodiments may address the unknown TBC maximum problem when combining the extended copy-above mode with grouping indices at front.
- the syntax element palette_index_refinement_flag may be coded as bypass bins and/or interleaved with the syntax elements of signaling palette mode and palette runs, which may be context-coded.
- a decoder may check whether the current pixel has one CPI position that refers to the pixels from the neighboring CU and/or whether the palette index parsed previously is equal to pMax ⁇ 1 or not.
- a coding mode sketch copy mode is proposed to code blocks comprising irregular lines. Syntax redundancy may be removed from blocks with special characteristics.
- the run value coding may be simplified.
- the parsing dependencies in the current palette coding design may be removed.
- the context modeling dependency of the syntax element palette_transpose_flag may be removed, for example, by simplifying the corresponding context model.
- the context modeling of the syntax element palette_mode may be removed, for example, by using run-length coding without using context.
- the syntax parsing dependencies and/or the syntax signaling dependencies that are related with escape color signaling may be removed.
- a palette table generation process may handle input screen content video with high bit depths, for example, at the encoder side.
- Sketch copy mode for palette coding may be provided.
- Sketch copy mode may be performed by copying a palette index from a coded neighboring sample position in one or more directions (e.g., which may include diagonals of any sort, horizontal, and/or vertical directions).
- Sketch copy mode may allow pixel copying in one or more specific directions.
- Coded irregular lines e.g., diagonal lines
- One or more (e.g., three) syntax elements may be coded for sketch mode: color index, direction, and/or run.
- the color index may be an index in the palette table and/or be equal to the size of palette table to indicate an escape color.
- the direction information may be coded following the color index.
- a run value may be coded to indicate the number pixels having the same color index as first pixel in that signaled direction.
- FIG. 8 A is a diagram illustrating an example of 16 directions with course granularity.
- FIG. 8 B is a diagram illustrating an example of 16 directions with fine granularity.
- FIG. 8 A and FIG. 8 B illustrate example definitions for 16 directions in different granularities, where each side may have eight directions respectively.
- the directions in FIG. 8 A may be defined as follows:
- the directions defined in FIG. 8 B may be in a fine granularity, and the directions may be defined as follows:
- the vertical direction may not be considered in a sketch copy mode, for example, because index mode in a vertical traverse scan may code a vertical line efficiently.
- the number of directions may be reduced if the overhead for direction coding affects performance, for example, at low bitrate coding.
- the direction information may be segmented into one or more elements, for example, a flag “is_righ_flag” and the index of direction at one side. “is_right_flag” may be used to indicate which side the direction belongs to.
- a maximum index of a direction may be equal to a plurality of different directions, for example, seven different directions (e.g., to each side) as provided by the examples of FIG. 8 A and FIG. 8 B .
- the direction of a previous coded neighboring pixel may be considered, for example, to further remove the redundancy of direction coding. If a current pixel has the same direction compared to the direction of a previous coded neighboring pixel, then a flag “same_direction_flag” may be coded (e.g., only the flag “same_direction_flag” is coded). If directions are not the same but they have the same value as “is_right_flag”, then the pruning process may be performed.
- FIG. 9 is a diagram illustrating an example flow chart of the direction coding algorithm for sketch mode.
- FIG. 10 is a diagram illustrating an example of palette coding with sketch copy mode.
- FIG. 10 may be compared to the palette coding in FIG. 7 B .
- the irregular lines may be coded with sketch mode.
- the irregular line e.g., the diagonal line going from S 11 to S 39
- the is_right_flag may be set to 1 and the direction index may be 0.
- the irregular line e.g, the diagonal line going from S 12 to S 32
- the is_right_flag may be set to 0 and the direction index may be 0.
- an index mode may code the remaining pixels in the block. Five runs may be coded in total, for example, because the irregular lines are not included. The number of segments in that block may be reduced by enabling sketch copy mode.
- the block prone to apply palette coding may include a plurality of colors.
- Sketch copy mode may be applied (e.g., only applied) for blocks that include one or more irregular lines. If there are no irregular lines within the block, then sketch copy mode may not be selected though, for example, it may be enabled, The signaling overhead for run and copy mode may be increased due to one or more additional modes added. Therefore, the usage of sketch copy mode may be restricted to reduce the signaling overhead.
- a flag may be added in the coding unit coded with a palette to indicate whether the sketch copy mode is enabled or not. For example, if it is not enabled, then no signaling overhead is added.
- sketch copy mode may be used within one block because sketch copy mode will not save too much if there are not many pixels remaining.
- Sketch copy mode may not save signaling overhead for remaining pixel coding, for example, if the coding position is beyond the specific row defined by a threshold.
- the threshold may be adaptive to the size of a coding unit. For example, the threshold may be [4, 7, 12, 16] for 8 ⁇ 8, 16 ⁇ 16, 32 ⁇ 32, and/or 64 ⁇ 64 coding units.
- FIG. 11 A and FIG. 11 B illustrate examples of a coding order of a horizontal traverse scan and a vertical traverse scan, respectively.
- the horizontal traverse scan and/or the vertical traverse scan may be performed in a reverse order, which for example, may be equivalent to the block being rotated by 180 degrees in a clock-wise or counter clock-wise manner.
- FIG. 11 C and FIG. 11 D illustrate examples of a coding order in a reverse horizontal traverse scan and in a reverse vertical traverse scan, respectively.
- palette_escape_val_present_flag may be used to indicate whether an escape color exists in a coding unit (CU).
- the escape color may be signaled with the color index being equal to the palette table size. If there is no escape color, then the maximum color index may be equal to palette table size minus one. Otherwise, the maximum color index may be equal to palette table size.
- This maximum value may affect the color index coding, for example, because the color index may be coded with truncated binary code.
- the number of bits for a variable x in truncated binary code may be provided by the following:
- palette_escape_val_present_flag may not be signal, for example, to remove palette signaling redundancy.
- the decoder may determine that all pixels are escape colors (e.g., there are no major colors in the palette table). As such, when the palette table size fora CU is 0, the palette_escape_val_present_flag may not be signaled and the decoder may infer the value to be 1.
- the palette table size is a value other than 0.
- the palette_escape_val_present_flag may be signaled. Therefore, the palette_escape_val_present_flag may be signaled (e.g., only signaled) when the palette table size is a value other than 0. If there is one color (e.g., only one color) in the coding unit (e.g., the palette table size is I and palette_escape_val_present_flag is 0) or if all pixels are coded as escape colors (e.g., palette table size is 0 and palette_escape_val_present_flag is 1), then the palette mode may be in index mode (e.g., may be inferred to be in index mode (e.g., run mode)) and the run value may be the size of coding unit minus 1 if run is coded.
- index mode e.g., may be inferred to be in index mode (e.g., run mode)
- the run value may be the size of coding unit minus 1 if run is coded
- the decoder may not receive palette_mode and/or may infer palette_mode to be equal to 0 (e.g., inferred to be in index mode (e.g., run mode)) and/or may not receive palette_run and/or may infer palette_run to be equal to the size of the CU minus 1.
- a condition may be added for palette mode signaling and/or run coding, for example, as shown in Table 6.
- the entropy coding of the syntax element Run in palette coding may be provided. Two different codes may be used to code the syntax element Run.
- Table 4 is an example of a bin.arization of the value of Run. There may be three segments for the whole range of Run. The first one, two, or three digits of the bin may be context coded (e.g., those illustrated in bold in Table 4), and the subsequent digits in the bin may be by-pass coded without any context (e.g., those illustrated as not in bold in Table 4). 0 to 2 may be the first segment coded with fixed context for each bin. 3 to 26 may be the second segment.
- the coding of prefix may be the same as segment 1, and the suffix may be coded with Golomb-Rice codes and/or the rice parameter may be equal to 3.
- a value beyond 26 may be a third segment.
- the prefix of the third segment may be coded the same as segment 1.
- the suffix may be coded with Exp-Golomb codes. Run values from 3 to 10, 3 prefix bins. and/or 4 suffix bins may be coded.
- a second order Exp-Golomb may be provided for values beyond 2.
- An example binarization may be provided in Table 5. For small runs, the number of bins may be reduced.
- Parsing dependency improvement for palette coding may be provided. Removing the context modeling dependency of the syntax element “palette_transpose_flag” may be provided.
- One or more may be used to code the syntax element palette_transpose_flag.
- One context model may be selected for palette_transpose_flag, for example, based on whether the palette indices of the left neighboring CU of the current CU are scanned horizontally or vertically. This could complicate parallelism design for practical implementation as the bit-streams of two neighboring palette-coded CUs may not be parsed simultaneously.
- This neighbor-based context modeling may use one binary buffer to store the value of the syntax element for the left neighboring CU.
- the palette_transpose_flag flag may be the CU-level flag that establishes its context models using its spatial neighbors, for example, while the other CU-level flags (e.g., such as palette_share_flag, palette_escape_val_present_flag, num_signalled_palette_entries, and/or the like) may be coded in bypass mode and/or coded in context mode, using one single context.
- the other CU-level flags e.g., such as palette_share_flag, palette_escape_val_present_flag, num_signalled_palette_entries, and/or the like
- the context modeling dependency of the syntax element palette_transpose_flag on the left neighbor of the current CU may be removed, for example, by coding the flag with one single context. This may be performed to improve parallelism capacity and/or to make CU-level signaling desgin of palette coding more consistent.
- the syntax element palette_transpose_flag may be coded in bypass mode.
- the context modeling dependency of the syntax element “palette_mode” may be removed.
- One or more context models may be used to code the palette_mode_flag, which may be determined based on the value of the same syntax element of the collocated pixel in the above row.
- Table 6 illustrates an example of the context modeling used to code the palette_mode flag.
- palette_mode may be decoded for each run-length chain of consecutive pixels with the same palette index.
- this context modeling approach may reduce the throughput of the parsing process, as the determination of the context at one pixel may be dependent on the palette index coding mode of pixels preceding it in the scan order.
- the flags palette_mode of one CU may form a binary vector which may be composed of elements “0” and “1.”
- Run length coding is method of coding binary vector by indicating the number of consecutive 0's between two 1's. RLE may be used for applications that require high throughput, for example, because multiple bins can be generated together within one single circle. For example, to remove the context modeling of palette_mode, RLE may be used to code the syntax element palette_mode.
- the context modeling dependency of the syntax element palette_mode may be removed by coding the flag with a single context.
- the syntax parsing and/or signaling dependencies related with escape color signaling may be removed.
- the parsing dependencies may be removed, for example, for a higher throughput and/or to improve the efficiency of palette coding.
- the palette index used for escape colors may not be coded in index mode or in copy mode, for example, although the current palette coding method may use the maximum palette index to indicate escape colors. For example, one run-length chain of consecutive pixels with the same palette index may be broken when an escape color is encountered. This may compromise the overall coding efficiency of palette coding and/or introduce dependencies to the parsing process for CUs coded with palette mode.
- the palette index of one pixel may be parsed after the palette index of its CPI position is fully reconstructed (e.g., in order to check if the CPI position is coded as escape color or not, which may determine the input parameter pMax for the TBC binarization process).
- the palette index used to indicate escape colors may be used as one normal palette index and it may be coded using an index mode and/or a copy mode.
- a decoder may parse palette index run values of a CU.
- the palette index run values may indicate a length of consecutive palette indices for corresponding pixel positions in the CU.
- One or more of the palette indices may correspond to an escape color.
- the palette indices may correspond to a plurality of different escape values.
- a run of escape colors (e.g., which may be the same or different escape colors) may be coded in index mode and/or copy mode.
- color values for the different pixel positions of a CU may be encoded according to one or more runs, which may include one or more runs of a major color and/or one or more runs of escape colors.
- a run of escape colors may be a run of the same escape color or different escape colors.
- the decoder may reconstruct the consecutive palette indices for the corresponding pixel positions of the CU according to the palette index run values (e.g., which may be a run of escape colors).
- the decoder may reconstruct the color values for the pixel positions of the CU according to the palette indices and/or one or more palette_escape_val values.
- the parsing of the syntax element palette_escape_val may be separated from that of the other syntax elements, for example, in order to remove the syntax signaling dependencies of the element palette_escape_val.
- one or more scan passes may be applied to parse the syntax elements of one palette-coded CU.
- a scan pass e.g., a first scan pass
- a scan pass e.g., a second scan pass
- palette_escape_val may be parsed for one pixel position when its palette index, as derived by the first passes, is equal to the palette table size of the current CU (e.g, the escape color).
- the syntax parsing dependency of palette index may be removed.
- the maximum palette index pMax may be used with a fixed value (e.g., the maximum of palette table size) as the input to the TBC binarization process for palette index coding.
- the redundancy removal method for escape colors may be removed.
- a color-clustering-based palette table generation method may be used to select the major colors of one CU coded with palette mode by clustering the color values of the current CU into multiple sets and using the centroids of color clusters as the major colors in the palette table.
- one color cluster may be generated by quantizing the pixels (e.g., all the pixels) for which the distortion between the color value and the centroid of the cluster is no more than a pre-defined threshold T, to the same major color, for example, as illustrated by the following:
- P_i and C_i may represent the values of the i-th component of the current pixel and the centroid of the cluster, respectively.
- the threshold T may be set dependent upon a. quantization parameter.
- Table 7 illustrates an example of a mapping between the values of T and the quantization parameters.
- the values of the threshold T may be determined assuming that the input bit depths of luma and chroma components are 8 bits. Therefore, the palette table by the current palette table generation method may not properly handle the case when the bit depths of input screen content video are more than 8 bits.
- the 8-bit distortion may be used for the palette table generation process.
- the bit depths of lama and chroma components may be denoted as BitDepth _Y and BitDepth _C, and the distortion calculation equation may be as follows:
- Table 8 illustrates an example of syntax that may be used to enable the syntax redundancy removal method described herein and/or to remove the parsing dependencies as described herein.
- One or more embodiments may comprise parsing throughput for extended copy mode.
- the parsing of the grouped palette indices may not be placed in front of the parsing of palette modes and palette runs, for example, when the extended copy mode is enabled.
- One or more embodiments may be provided that increase the parsing throughput, and which may be jointly applied with extended copy mode.
- the palette indices may be grouped together and placed after the palette modes and palette runs (e.g., as compared to placing the indices before the modes and runs). Escape colors may be separated and placed after palette indices.
- the following palette syntax arrangement may be used.
- the syntax elements related with palette modes and palette runs e.g., which may include palette_run_type_flag, palette_run_msb_id_plus1, palette_run_refinement_bits and/or the like
- palette_index_idc palette_index_idc
- escape colors e.g., palette_escape_val
- Table 9 illustrates an example of a palette syntax table with this arrangement.
- the palette runs may be coded by the HEVC SCC run coding method.
- One or more of the run coding methods described herein may be combined with one or more of the described embodiments.
- Table 10 is an example of a palette syntax that has an arrangement described above with run-to-end coding included.
- palette_index_idc e.g., palette_index_idc
- escape colors e.g., palette_escape_val
- two individual scan loops may be used for the palette indices and escape color values in the current CU.
- One or more embodiments may group palette indices and escape colors together (e.g., within the group, palette_index_idc and palette_escape_val are still interleaved) and put them after the palette modes and palette runs, for example. Both of palette_index_idc and palette_escape_val may be bypass coded.
- palette_index_idc and palette_escape_val may be bypass coded.
- the following syntax element arrangements may be provided: the syntax elements related with palette modes and palette runs palette_run_type_flag, palette_run_msb_id_plus1, and/or palette_run_refinement_bits) are placed first in the current CU. and one or more (e.g., all) of the syntax elements (e.g., palette_index_idc and palette_escape_val) are grouped together and placed after the palette modes and palette runs. An example of which is provided in Table 11.
- a second run coding method may be used.
- Table 12 is an example of a palette syntax table with the arrangement described above with the second run coding (e.g., run-to-end syntax elements) included.
- the maximum value for palette index coding in TBC may not be determined for those palette indices whose above neighboring pixel refers to the pixels of the neighboring CU.
- the index at the above neighboring position may not be available.
- an additional flag may be signaled to indicate whether the coded value should be interpreted as the maximum value minus one, or interpreted as the maximum value.
- palette indices are grouped (e.g., arranged) at the front and extended copy above mode is enabled.
- a CU level indication (e.g., flag) may be provided.
- the CU-level indication may be arranged up front to instruct the decoder how to interpret and/or to entropy decode the palette indices.
- a syntax element use_index_max_for_palette_indices may be sent as a CU level flag, for example, as illustrated in Table 13. If the use_index_max_for_palette_indices flag has the value 1, then the coding of palette indices may use the maximum value for TBC equal to the palette table size if there is no escape color, or the coding of palette indices may use the maximum value for TBC equal to the palette table size plus 1 if there is an escape color.
- the coding of palette indices may use the maximum value for TBC equal to the palette table size minus 1 if there is no escape color, or the coding of palette indices may use the maximum value for TBC equal to the palette table size if there is an escape color.
- the decoder may determine the maximum value for TBC and decode the palette indices accordingly.
- the syntax element use_index_max_for_palette_indices may indicate whether index max is used as the maximum value for TBC encoding of the palette indices.
- the syntax element use_index_max_for_palette_indices having a value of 1 may indicate that the coding of palette indices uses the maximum value for TBC equal to palette table size if there is no escape color, or that the coding of palette indices uses the maximum value for TBC equal to the palette table size plus 1 if there is an escape color.
- the syntax element use_index_max_for_palette_indices having a value of 0 may indicate that the coding of palette indices uses the maximum value for TBC equal to palette table size minus 1 if there is no escape color, or that the coding of palette indices uses the maximum value for TBC equal to the palette table size if there is escape color.
- the syntax element use_index_max_for_palette__indices may be used as a CU level flag, fbr example, as illustrated in Table 14. If the syntax element use_index_max_for_palette_indices has the value 1, then the number of leading palette indices may be encoded using the maximum value for TBC equal to palette table size if there is no escape color, or the number of leading palette indices may be encoded using the maximum value for TBC equal to the palette table size plus 1 if there is escape color. A remaining number of palette indices may be encoded using the maximum value for TBC equal to palette table size minus 1 if there is no escape color, or equal to the palette table size if there is an escape color.
- the number of leading palette indices may be sent to the decoder as the value palette_num_leading_indices, The number of remaining palette indices may be sent to the decoder as the value palette_num_remaining_indices,
- the value of palette_num_leading_indices may be sent conditionally, for example, the value of palette_num_leading_indices may be sent in the bitstream. only in the case where use_index_max_for_palette_indices has the value 1.
- palette_num_leading_indices may not be present in the bitstream, and may be inferred to have a value of zero.
- one or more (e.g., all) of palette indices may be encoded using the maximum value for TBC equal to palette table size minus 1 if there is no escape color, or one or more (e.g., all) of palette indices may be encoded using the maximum value for TBC equal to the palette table size if there is escape color.
- the decoder may use these syntax elements to determine the maximum value for TBC for one or more (e.g., each) palette index, and may decode the palette indices accordingly.
- the syntax element use_index_max_for_palette_indices may indicate whether a nonzero value of palette_num_leading_indices is signaled in the bitstream.
- the syntax element palette_num_leading_indices may indicate the number of leading palette indices which are coded with a maximum value for TBC coding set to the palette table size if there is no escape color coding in the current CU, or set to the palette table size plus 1 if there is escape color coding in the current CU. If not present, the value of palette_num_leading_indices may be inferred to be zero.
- the syntax element palette_num_remaining_indices may indicate the number of palette indices which are coded with a maximum value for TBC coding set to the palette table size minus 1 if there is no escape color coding in the current CU, or set to the palette table size if there is escape color coding in the current CU.
- a palette syntax may be provided where use_index_max_for_palette_indices is not signaled and/or palette_num_leading_indices is present e.g., always present), for example, as illustrated in Table 15.
- the signaling of palette_index_refinement_flag may be moved from the parsing loop of palette mode and palette run to the parsing loop of palette indices,
- the maximum TBC level may be reduced by one (e.g., pMax ⁇ 1) before parsing the palette indices of one palette-coded CU.
- the palette_index_refinement_flag may be signaled (e.g., always signaled) when the parsed palette index of one pixel is equal to pMax ⁇ 1.
- Such syntax may comprise a higher parsing throughput, for example, since the bypass bins of palette index coding may be separated from the context-coded bins of palette modes and palette runs. An example of this syntax is provided in Table 16.
- FIG. 14 A is a diagram of an example communications system 100 in which one or more disclosed embodiments may be implemented.
- the communications system 100 may be a multiple access system that provides content, such as voice, data, video, messaging, broadcast, etc., to multiple wireless users.
- the communications system 100 may enable multiple wireless users to access such content through the sharing of system resources, including wireless bandwidth.
- the communications systems 100 may employ one or more channel access methods, such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), single-carrier FDMA (SC-FDMA), and the like.
- CDMA code division multiple access
- TDMA time division multiple access
- FDMA frequency division multiple access
- OFDMA orthogonal FDMA
- SC-FDMA single-carrier FDMA
- the communications system 100 may include wireless transmit/receive units (WTRUs) 102 a, 102 b, 102 c, and/or 102 d (which generally or collectively may be referred to as WTRU 102 ), a radio access network (RAN) 103 / 104 / 105 , a core network 106 / 107 / 109 , a public switched telephone network (PSTN) 108 , the Internet 110 , and other networks 112 , though it will be appreciated that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and/or network elements.
- WTRUs wireless transmit/receive units
- RAN radio access network
- PSTN public switched telephone network
- Each of the WTRUs 102 a, 102 b, 102 c, 102 d may be any type of device configured to operate and/or communicate in a wireless environment.
- the WTRUs 102 a , 102 b , 102 c , 102 d may be configured to transmit and/or receive wireless signals and may include user equipment (IJE), a mobile station, a fixed or mobile subscriber unit, a pager, a cellular telephone, a personal digital assistant (PDA), a smartphone, a laptop, a netbook, a personal computer, a wireless sensor, consumer electronics, and the like.
- IJE user equipment
- PDA personal digital assistant
- the communications systems 100 may also include a base station 114 a and a base station 114 b.
- Each of the base stations 114 a, 114 b may be any type of device configured to wirelessly interface with at least one of the WTRUs 102 a, 102 b, 102 c, 102 d to facilitate access to one or more communication networks, such as the core network 106 / 107 / 109 , the Internet 110 , and/or the networks 112 .
- the base stations 114 a, 114 b may be a base transceiver station (BTS), a Node-B, an eNode B, a Home Node B, a Home eNode B, a site controller, an access point (AP), a wireless router, and the like. While the base stations 114 a , 114 b are each depicted as a single element, it will be appreciated that the base stations 114 a , 114 b may include any number of interconnected base stations and/or network elements.
- BTS base transceiver station
- AP access point
- the base station 114 a may be part of the RAN 103 / 104 / 105 , which may also include other base stations and/or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), relay nodes, etc.
- BSC base station controller
- RNC radio network controller
- the base station 114 a and/or the base station 114 b may be configured to transmit and/or receive wireless signals within a particular geographic region, which may be referred to as a cell (not shown).
- the cell may further be divided into cell sectors.
- the cell associated with the base station 114 a may be divided into three sectors.
- the base station 114 a may include three transceivers, e.g., one for each sector of the cell.
- the base station 114 a may employ multiple-input multiple output (MIMO) technology and, therefore, may utilize multiple transceivers for each sector of the cell.
- MIMO multiple-input multiple output
- the base stations 114 a , 114 b may communicate with one or more of the WTRUs 102 a , 102 b , 102 c , 102 d over an air interface 115 / 116 / 117 , which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, infrared (IR), ultraviolet (UV), visible light, etc.).
- the air interface 115 / 116 / 117 may be established using any suitable radio access technology (RAT).
- RAT radio access technology
- the communications system 100 may be a multiple access system and may employ one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, and the like.
- the base station 114 a in the RAN 103 / 104 / 105 and the WTRUs 102 a , 102 b , 102 c may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may establish the air interface 115 / 116 / 117 using wideband CDMA (WCDMA).
- WCDMA may include communication protocols such as High-Speed Packet Access (HSPA) and/or Evolved HSPA (HSPA+).
- HSPA may include High-Speed Downlink Packet Access (HSDPA) and/or High-Speed Uplink Packet Access (HSUPA).
- the base station 114 a and the WTRUs 102 a, 102 b , 102 c may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish the air interface 115 / 116 / 117 using Lona Term Evolution (LTE) and/or LTE-Advanced (LTE-A).
- E-UTRA Evolved UMTS Terrestrial Radio Access
- LTE Long Term Evolution
- LTE-A LTE-Advanced
- the base station 114 a and the WTRUs 102 a, 102 b, 102 c may implement radio technologies such as IEEE 802.16 (e.g., Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 1 ⁇ , CDMA2000 EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile communications (GSM), Enhanced Data rates for GSM Evolution (EDGE), GSM EDGE (GERAN), and the like.
- IEEE 802.16 e.g., Worldwide Interoperability for Microwave Access (WiMAX)
- CDMA2000, CDMA2000 1 ⁇ , CDMA2000 EV-DO Code Division Multiple Access 2000
- IS-95 Interim Standard 95
- IS-856 Interim Standard 856
- GSM Global System for Mobile communications
- GSM Global System for Mobile communications
- EDGE Enhanced Data rates for GSM Evolution
- GERAN GSM EDGERAN
- the base station 114 b in FIG. 14 A may be a wireless router, Home Node B. Home eNode B. or access point, for example, and may utilize any suitable RAT for facilitating wireless connectivity in a localized area, such as a place of business, a home, a vehicle, a campus, and the like.
- the base station 114 b and the WTRUs 102 c, 102 d may implement a radio technology such as IEEE 802.11 to establish a wireless local area network (WLAN).
- the base station 114 b and the WTRUs 102 c, 102 d may implement a radio technology such as IEEE 802.15 to establish a wireless personal area network (WPAN).
- WLAN wireless local area network
- WPAN wireless personal area network
- the base station 114 b and the WTRUs 102 c , 102 d may utilize a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, etc.) to establish a picocell or femtocell.
- a cellular-based RAT e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, etc.
- the base station 114 b may have a direct connection to the Internet 110 .
- the base station 114 b may not be required to access the Internet 110 via the core network 106 / 107 / 109 .
- the RAN 103 / 104 / 105 may be in communication with the core network 106 / 107 / 109 , which may be any type of network configured to provide voice, data, applications, and/or voice over internet protocol (VoIP) services to one or more of the WTRUs 102 a, 102 b , 102 c, 102 d .
- the core network 106 / 107 / 109 may provide call control, billing services, mobile location-based services, pre-paid calling, Internet connectivity, video distribution, etc., and/or perform high-level security functions, such as user authentication.
- the RAN 103 / 104 / 105 and/or the core network 106 / 107 / 109 may be in direct or indirect communication with other RANs that employ the same RAT as the RAN 103 / 104 / 105 or a different RAT.
- the core network 106 / 107 / 109 may also be in communication with another RAN (not shown) employing a GSM radio technology.
- the core network 106 / 107 / 109 may also serve as a gateway for the WTRUs 102 a , 102 b, 102 c, 102 d to access the PSTN 108 , the Internet 110 , and/or other networks 112 .
- the PSTN 108 may include circuit-switched telephone networks that provide plain old telephone service (POTS).
- POTS plain old telephone service
- the Internet 110 may include a global system of interconnected computer networks and devices that use common communication protocols, such as the transmission control protocol (TCP), user datagram protocol (UDP) and the internet protocol (IP) in the TCP/IP internet protocol suite.
- the networks 112 may include wired or wireless communications networks owned and/or operated by other service providers.
- the networks 112 may include another core network connected to one or more RANs, which may employ the same RAT as the RAN 103 / 104 / 105 or a different RAT.
- the WTRUs 102 a, 102 b, 102 c, 102 d in the communications system 100 may include multi-mode capabilities, e.g., the WTRUs 102 a , 102 b , 102 c , 102 d may include multiple transceivers for communicating with different wireless networks over different wireless links.
- the WTRU 102 c shown in FIG. 14 A may be configured to communicate with the base station 114 a , which may employ a cellular-based radio technology, and with the base station 114 b, which may employ an IEEE 802 radio technology.
- FIG. 14 B is a system diagram of an example WTRU 102 .
- the WTRU 102 may include a processor 118 , a transceiver 120 , a transmit/receive element 122 , a speaker/microphone 124 , a keypad 126 , a display/touchpad 128 , non-removable memory 130 , removable memory 132 , a power source 134 , a global positioning system (GPS) chipset 136 , and other peripherals 138 .
- GPS global positioning system
- the base stations 114 a and 114 b, and/or the nodes that base stations 114 a and 114 b may represent, such as but not limited to transceiver station (BTS), a Node-B, a site controller, an access point (AP), a home node-B, an evolved home node-B (eNodeB), a home evolved node-B (HeNB), a home evolved node-B gateway, and proxy nodes, among others, may include some or all of the elements depicted in FIG. 14 B and described. herein.
- the processor 118 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Array (FPGAs) circuits, any other type of integrated circuit (IC), a state machine, and the like.
- the processor 118 may perform signal coding, data processing, power control, input/output processing, and/or any other functionality that enables the WTRU 102 to operate in a wireless environment.
- the processor 118 may be coupled to the transceiver 120 , which may be coupled to the transmit/receive element 122 . While FIG. 14 B depicts the processor 118 and the transceiver 120 as separate components, it will be appreciated that the processor 118 and the transceiver 120 may be integrated together in an electronic package or chip.
- the transmit/receive element 122 may be configured to transmit signals to, or receive signals from, a base station (e.g., the base station 114 a ) over the air interface 115 / 116 / 117 .
- a base station e.g., the base station 114 a
- the transmit/receive element 122 may be an antenna configured to transmit and/or receive RF signals.
- the transmit/receive element 122 may be an emitter/detector configured to transmit and/or receive IR, UV, or visible light signals, for example.
- the transmit/receive element 122 may be configured to transmit and receive both RF and light signals. It will be appreciated that the transmit/receive element 122 may be configured to transmit and/or receive any combination of wireless signals.
- the WTRU 102 may include any number of transmit/receive elements 122 . More specifically, the WTRU 102 may employ MIMO technology. Thus, in one embodiment, the WTRU 102 may include two or more transmit/receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 115 / 116 / 117 .
- the transceiver 120 may be configured to modulate the signals that are to be transmitted by the transmit/receive element 122 and to demodulate the signals that are received by the transmit/receive element 122 .
- the WTRU 102 may have multi-mode capabilities.
- the transceiver 120 may include multiple transceivers for enabling the WTRU 102 to communicate via multiple RATs, such as UTRA and IEEE 802.11, for example.
- the processor 118 of the WTRU 102 may be coupled to, and may receive user input data from, the speaker/microphone 124 , the keypad 126 , and/or the display/touchpad 128 (e.g., a liquid crystal display (LCD) display unit or organic light-emitting diode (OLED) display unit).
- the processor 118 may also output user data to the speaker/microphone 124 , the keypad 126 , and/or the display/touchpad 128 .
- the processor 118 may access information from, and store data in, any type of suitable memory, such as the non-removable memory 130 and/or the removable memory 132 .
- the non-removable memory 130 may include random-access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device.
- the removable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like.
- SIM subscriber identity module
- SD secure digital
- the processor 118 may access information from, and store data in, memory that is not physically located on the WTRU 102 , such as on a server or a home computer (not shown).
- the processor 118 may receive power from the power source 134 , and may be configured to distribute and/or control the power to the other components in the WTRU 102 .
- the power source 134 may be any suitable device for powering the WTRU 102 .
- the power source 134 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, and the like.
- the processor 118 may also be coupled to the GPS chipset 136 , which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU 102 .
- location information e.g., longitude and latitude
- the WTRU 102 may receive location information over the air interface 115 / 116 / 117 from a base station (e.g., base stations 114 a , 114 b ) and/or determine its location based on the timing of the signals being received from two or more nearby base stations. It will be appreciated that the WTRU 102 may acquire location information by way of any suitable location-determination method while remaining consistent with an embodiment.
- the processor 118 may further be coupled to other peripherals 138 , which may include one or more software and/or hardware modules that provide additional features, functionality and/or wired or wireless connectivity.
- the peripherals 138 may include an accelerometer, an e-compass, a satellite transceiver, a digital camera (for photographs or video), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands free headset, a Bluetooth® module, a frequency modulated (FM) radio unit, a digital music player, a media player, a video game player module, an Internet browser, and the like.
- the peripherals 138 may include an accelerometer, an e-compass, a satellite transceiver, a digital camera (for photographs or video), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands free headset, a Bluetooth® module, a frequency modulated (FM) radio unit, a digital music player, a media player, a video game
- FIG. 14 C is a system diagram of the RAN 103 and the core network 106 according to an embodiment.
- the RAN 103 may employ a UTRA radio technology to communicate with the WTRUs 102 a, 102 b, 102 c over the air interface 115 .
- the RAN 103 may also be in communication with the core network 106 .
- the RAN 103 may include Node-Bs 140 a , 140 b, 140 c, which may each include one or more transceivers for communicating with the WTRUs 102 a , 102 b , 102 c over the air interface 115 .
- the Node-Bs 140 a , 140 b , 140 c may each be associated with a particular cell (not shown) within the RAN 103 .
- the RAN 103 may also include RNCs 142 a , 142 b . It will be appreciated that the RAN 103 may include any number of Node-Bs and RNCs while remaining consistent with an embodiment.
- the Node-Bs 140 a , 140 b may be in communication with the RNC 142 a. Additionally, the Node-B 140 c may be in communication with the RNC 142 b .
- the Node-Bs 140 a , 140 b , 140 c may communicate with the respective RNCs 142 a , 142 b via an lub interface.
- the RNCs 142 a , 142 b may be in communication with one another via an Iur interface.
- Each of the RNCs 142 a , 142 b may be configured to control the respective Node-Bs 140 a , 140 b, 140 c to which it is connected.
- each of the RNCs 142 a , 142 b may be configured to carry out or support other functionality, such as outer loop power control, load control, admission control, packet scheduling, handover control, macrodiversity, security functions, data encryption, and the like.
- the core network 106 shown in FIG. 14 C may include a media gateway (MGW) 144 , a mobile switching center (MSC) 146 , a serving CPRS support node (SGSN) 148 , and/or a gateway GPRS support node (GGSN) 150 . While each of the foregoing elements are depicted as part of the core network 106 , it will be appreciated that any one of these elements may be owned and/or operated by an entity other than the core network operator.
- MGW media gateway
- MSC mobile switching center
- SGSN serving CPRS support node
- GGSN gateway GPRS support node
- the RNC 142 a in the RAN 103 may be connected to the MSC 146 in the core network 106 via an IuCS interface.
- the MSC 146 may be connected to the MGW 144 .
- the MSC 146 and the MGW 144 may provide the WTRUs 102 a , 102 b , 102 c with access to circuit-switched networks, such as the PSTN 108 , to facilitate communications between the WTRUs 102 a , 102 b , 102 c and traditional land-line communications devices.
- the RNC 142 a in the RAN 103 may also be connected to the SGSN 148 in the core network 106 via an IuPS interface.
- the SGSN 148 may be connected to the GGSN 150 .
- the SGSN 148 and the GGSN 150 may provide the WTRUs 102 a , 102 b , 102 c with access to packet-switched networks, such as the Internet 110 , to facilitate communications between and the WTRUs 102 a , 102 b , 102 c and IP-enabled devices.
- the core network 106 may also be connected to the networks 112 , which may include other wired or wireless networks that are owned and/or operated by other service providers.
- FIG. 14 D is a system diagram of the RAN 104 and the core network 107 according to an embodiment, As noted above, the RAN 104 may employ an E-UTRA radio technology to communicate with the WTRUs 102 a , 102 b , 102 c over the air interface 116 . The RAN 104 may also be in communication with the core network 107 .
- the RAN 104 may include eNode-Bs 160 a , 160 b , 160 c , though it will be appreciated that the RAN 104 may include any number of eNode-Bs while remaining consistent with an embodiment.
- the eNode-Bs 160 a , 160 b , 160 c may each include one or more transceivers for communicating with the WTRUs 102 a , 102 b , 102 c over the air interface 116 .
- the eNode-Bs 160 a , 160 b , 160 c may implement MIMO technology.
- the eNode-B 160 a for example, may use multiple antennas to transmit wireless signals to, and receive wireless signals from, the WTRU 102 a.
- Each of the eNode-Bs 160 a , 160 b , 160 c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in the uplink and/or downlink, and the like. As shown in FIG. 14 D , the eNode-Bs 160 a , 160 b , 160 c may communicate with one another over an X2 interface.
- the core network 107 shown in FIG. 14 D may include a mobility management gateway (MME) 162 , a serving gateway 164 , and a packet data network (PDN) gateway 166 . While each of the foregoing elements are depicted as part of the core network 107 , it will be appreciated that any one of these elements may be owned and/or operated by an entity other than the core network operator.
- MME mobility management gateway
- PDN packet data network
- the MME 162 may be connected to each of the eNode-Bs 160 a , 160 b , 160 c in the RAN 104 via an S1 interface and may serve as a control node.
- the MME 162 may be responsible for authenticating users of the WTRUs 102 a , 102 b , 102 c , bearer activation/deactivation, selecting a particular serving gateway during an initial attach of the WTRUs 102 a , 102 b , 102 c , and the like.
- the MME 162 may also provide a control plane function for switching between the RAN 104 and other RANs (not shown) that employ other radio technologies, such as GSM or WCDMA.
- the serving gateway 164 may be connected to each of the eNode-Bs 160 a , 160 b , 160 c in the RAN 104 via the S1 interface.
- the serving gateway 164 may generally route and forward user data packets to/from the WTRUs 102 a , 102 b , 102 c .
- the serving gateway 164 may also perform other functions, such as anchoring user planes during inter-eNode B handovers, triggering paging when downlink data is available for the WTRUs 102 a , 102 b , 102 c , managing and storing contexts of the WTRUs 102 a , 102 b , 102 c , and the like.
- the serving gateway 164 may also be connected to the PDN gateway 166 , which may provide the WTRUs 102 a, 102 b , 102 c with access to packet-switched networks, such as the Internet 110 , to facilitate communications between the WTRUs 102 a , 102 b , 102 c and IP-enabled devices.
- the PDN gateway 166 may provide the WTRUs 102 a, 102 b , 102 c with access to packet-switched networks, such as the Internet 110 , to facilitate communications between the WTRUs 102 a , 102 b , 102 c and IP-enabled devices.
- the core network 107 may facilitate communications with other networks.
- the core network 107 may provide the WTRUs 102 a , 102 b , 102 c with access to circuit-switched networks, such as the PSTN 108 , to facilitate communications between the WTRUs 102 a , 102 b , 102 c and traditional land-line communications devices.
- the core network 107 may include, or may communicate with, an IP gateway (e.g., an IP multimedia subsystem (IMS) server) that serves as an interface between the core network 107 and the PSTN 108 .
- the core network 107 may provide the WTRUs 102 a , 102 b , 102 c with access to the networks 112 , which may include other wired or wireless networks that are owned and/or operated by other service providers.
- IMS IP multimedia subsystem
- FIG. 14 E is a system diagram of the RAN 105 and the core network 109 according to an embodiment.
- the RAN 105 may be an access service network (ASN) that employs IEEE 802.16 radio technology to communicate with the WTRUs 102 a , 102 b , 102 c over the air interface 117 .
- ASN access service network
- the communication links between the different functional entities of the WTRUs 102 a , 102 b, 102 c , the RAN 105 , and the core network 109 may be defined as reference points.
- the RAN 105 may include base stations 180 a , 180 b , 180 c , and an ASN gateway 182 , though it will be appreciated that the RAN 105 may include any number of base stations and ASN gateways while remaining consistent with an embodiment.
- the base stations 180 a , 180 b , 180 c may each be associated with a particular cell (not shown) in the RAN 105 and may each include one or more transceivers for communicating with the WTRUs 102 a , 102 b , 102 c over the air interface 117 .
- the base stations 180 a , 180 b , 180 c may implement MIMO technology.
- the base station 180 a may use multiple antennas to transmit wireless signals to, and receive wireless signals from. the WTRU 102 a.
- the base stations 180 a , 180 b , 180 c may also provide mobility management functions, such as handoff triggering, tunnel establishment, radio resource management, traffic classification, quality of service (QoS) policy enforcement, and the like.
- the ASN gateway 182 may serve as a traffic aggregation point and may be responsible for paging, caching of subscriber profiles, routing to the core network 109 , and the like.
- the air interface 117 between the WTRUs 102 a , 102 b , 102 c and the RAN 105 may be defined as an R1 reference point that implements the IEEE 802.16 specification.
- each of the WTRUs 102 a , 102 b , 102 c may establish a logical interface (not shown) with the core network 109 .
- the logical interface between the WTRUs 102 a , 102 b , 102 c and the core network 109 may be defined as an R2 reference point, which may be used for authentication, authorization, IP host configuration management, and/or mobility management.
- the communication link between each of the base stations 180 a , 180 b , 180 c may be defined as an R8 reference point that includes protocols for facilitating WTRU handovers and the transfer of data between base stations.
- the communication link between the base stations 180 a , 180 b , 180 c and the ASN gateway 182 may be defined as an R6 reference point.
- the R6 reference point may include protocols for facilitating mobility management based on mobility events associated with each of the WTRUs 102 a , 102 b , 102 c.
- the RAN 105 may be connected to the core network 109 .
- the communication link between the RAN 105 and the core network 109 inay defined as an R3 reference point that includes protocols for facilitating data transfer and mobility management capabilities, for example,
- the core network 109 may include a mobile IP home agent (MIP-HA) 184 , an authentication, authorization, accounting (AAA) server 186 , and a gateway 188 . While each of the foregoing elements are depicted as part of the core network 109 , it will be appreciated that any one of these elements may be owned and/or operated by an entity other than the core network operator.
- MIP-HA mobile IP home agent
- AAA authentication, authorization, accounting
- the MIP-HA may be responsible for IP address management, and may enable the WTRUs 102 a , 102 b , 102 c to roam between different ASNs and/or different core networks.
- the MIP-HA 184 may provide the WTRUs 102 a , 102 b , 102 c with access to packet-switched networks, such as the Internet 110 , to facilitate communications between the WTRUs 102 a , 102 b , 102 c and IP-enabled devices.
- the AAA server 186 may be responsible fbr user authentication and for supporting user services.
- the gateway 188 may facilitate interworking with other networks.
- the gateway 188 may provide the WTRUs 102 a , 102 b , 102 c with access to circuit-switched networks, such as the PSTN 108 , to facilitate communications between the WTRUs 102 a , 102 b , 102 c and traditional land-line communications devices.
- the gateway 188 may provide the WTRUs 102 a , 102 b , 102 c with access to the networks 112 , which may include other wired or wireless networks that are owned and/or operated by other service providers.
- the RAN 105 may be connected to other ASNs and the core network 109 may be connected to other core networks.
- the communication link between the RAN 105 the other ASNs may be defined as an R4 reference point, which may include protocols for coordinating the mobility of the WTRUs 102 a , 102 b , 102 c between the RAN 105 and the other ASNs.
- the communication link between the core network 109 and the other core networks may be defined as an R5 reference, which may include protocols for facilitating interworking between home core networks and visited core networks.
- ROM read only memory
- RAM random access memory
- register cache memory
- semiconductor memory devices magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks, and digital versatile disks (DVDs).
- a processor in association with software may be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Color Television Systems (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
Sketch copy mode may be used to code blocks comprising irregular lines, syntax redundancy may be removed from blocks with special characteristics, and/or run value coding may be simplified. The parsing dependencies in palette coding design may be removed. For example, the context modeling dependency of the syntax element palette_transpose_flag be removed, for example, by simplifying the corresponding context model. The context modeling of the syntax element palette mode may be removed, for example, by using run-length coding without using context. The syntax parsing dependencies and/or the syntax signaling dependencies that are related with escape color signaling may be removed. A palette table generation process may handle input screen content video with high bit depths, for example, at the encoder side.
Description
- This application claims the benefit of U.S. Provisional Patent Application No. 62/060,536, filed Oct. 6, 2014, and U.S. Provisional Patent Application No. 62/120,293, filed Feb. 24, 2014 the disclosures of which are incorporated herein by reference in their entirety.
- Screen content sharing applications have become more popular in recent years with the expanded use of remote desktop, video conferencing, and mobile media presentation applications. Compared to the natural video content, the screen content may include numerous blocks with several major colors and sharp edges due to, for example, sharp curves and text inside. Existing video compression methods may not fully characterize the feature of screen content and may lead to a low compression performance, for example, reconstructed pictures may have quality issues. For example, the curves and text may be blurred and/or they may be difficult to recognize. Therefore, a well-designed screen compression method is needed for effectively reconstructing screen content.
- A decoder may perform palette decoding of a coding unit (CU) of a picture. For example, the decoder may parse a palette index run value of the CU. The palette index run value may indicate an index run or a copy run. The palette index run value may indicate a length of consecutive palette indices for corresponding pixel positions in the CU, and at least one of the consecutive palette indices may corresponds to an escape color index. For example, the consecutive palette indices may correspond to a first escape color index and a second escape color index.
- The decoder may reconstruct the consecutive palette indices for the corresponding pixel positions of the CU according to the palette index run values. The decoder may reconstruct color values for the corresponding pixel positions of the CU according to the consecutive palette indices and one or more palette_escape_val values. For example the decoder may parse, during a first pass, palette syntax values sufficient to reconstruct all palette indices in the CU (e.g., which may include a palette mode flag(s), a palette index value(s), and/or a run values) for index mode, or a palette mode flag(s) and/or a run value(s) for copy mode) other than the one or more palette_escape_val values, and parse, during a second pass, the one or more palette_escape_val values for the corresponding pixel positions in the CU for which the palette index parsed from the first pass corresponds to the escape color index.
- A decoder may performing palette decoding for a CU of a picture. The decoder may determine a palette table size for a coding unit (CU) of a picture. For example, the decoder may determine that the palette table size of the CU is zero. The palette table size for the CU being equal to zero may indicate that all pixels of the CU are coded as escape colors. If the decoder determines that the palette table size of the CU is zero, than the decoder may decode the escape values (e.g., by inferring that a palette_escape_val_present_flag for the CU is equal to one). For example, the palette_escape_val_present_flag may not be received for the CU if the palette table size is zero. If the decoder determines that the palette table size of the CU is a value other than zero, than the decoder may parse a palette_escape_val_present_flag for the CU. The decoder may reconstruct palette indices for corresponding pixel positions of the CU based on a palette table for the CU and a value associated with the palette_escape_val_present_flag of the CU.
- A decoder may performing a palette decoding for a CU of a picture. The decoder may determine that a coding unit (CU) of a picture comprises a single major color (e.g., only one major color) or only escape colors. The decoder may determine that the CU is encoded according to an index mode and a palette index mode (e.g., based on not receiving a palette_mode for the CU). The decoder may reconstruct color values for the corresponding pixel positions of the CU. The decoder may determine that a run value of the index mode for the CU is equal to a size of the CU minus one based on not receiving a syntax value indicative of a palette run value for the C.U. The decoder may determine that the palate index value for the CU is equal to zero based on not receive a palette index value for the CU. The decoder may determine that a palette_escape_val_present_flag for the CU is equal to zero and/or determine that a palette table size for the CU is equal to one, for example, to determine that the CU comprises a single major color. The decoder may determine that a palette_escape_val_present_flag for the CU is equal to one and/or determine that a palette table size for the CU is equal to zero, for example, to determine that the CU comprise only escape colors. The decoder may infer the palette_mode of the CU to be equal to zero based on not receiving the palette_mode for the CU. The decoder may receive a palette_escape_val for one or more pixel positions (e.g., each pixel position) in the CU, for example, if the CU comprises only escape colors.
- Sketch copy mode may be used to code blocks comprising irregular lines, syntax redundancy may be removed from blocks with special characteristics, and/or run value coding may be simplified.
- The parsing dependencies in palette coding design may be removed. For example, the context modeling dependency of the syntax element palette_transpose_flag may be removed, for example, using one single context. The context modeling dependency of the syntax element palette_mode may be removed, for example, by using nun length coding without using context or using one single context. The syntax parsing dependencies and/or the syntax signaling dependencies that are related with escape color signaling may be removed, A palette table generation process may handle input screen content video with high hit depths, for example, at the encoder side.
-
FIG. 1 is a diagram that illustrates an example block diagram of a screen content sharing system. -
FIG. 2 is a diagram illustrating an example of a block-based single layer video encoder. -
FIG. 3 is a diagram illustrating an example of a block-based single layer video decoder. -
FIG. 4 is a diagram illustrating an example of eight partition modes. -
FIG. 5A is a diagram illustrating an example of a screen content block. -
FIG. 5B is a diagram illustrating n example of a palette index map of the screen content block ofFIG. 5A . -
FIG. 6 is a diagram illustrating an example SCC test sequence for a console. -
FIG. 7A is a diagram illustrating an example 8×8 block with four colors, where the dashed line illustrates a scanning order in palette coding. -
FIG. 7B is a diagram illustrating an example 8×8 block with four colors, where the dashed line illustrates a scanning order in palette coding that uses index mode and copy mode. -
FIG. 8A is a diagram illustrating an example of 16 directions with course granularity. -
FIG. 8B is a diagram illustrating an example of 16 directions with fine granularity. -
FIG. 9 is a diagram illustrating an example flow chart of the direction coding algorithm for sketch copy mode. -
FIG. 10 is a diagram illustrating an example of palette coding with sketch copy mode. -
FIG. 11A illustrates an example of a coding order of a horizontal traverse scan. -
FIG. 11B illustrates an example of a coding order of a vertical traverse scan. -
FIG. 11C illustrates an example of a coding order in a reverse horizontal traverse scan. -
FIG. 11D illustrates an example of a coding order in a reverse vertical traverse scan. -
FIG. 12A is a diagram illustrating an example of a redundancy removal method of palette index coding where the CPI position is coded in index mode. -
FIG. 12B is a diagram illustrating an example of a redundancy removal method of palette index coding where the CPI position is coded in copy mode. -
FIG. 13 is a diagram of an example of extended copy mode. -
FIG. 14A is a system diagram of an example communications system in which one or more disclosed embodiments may be implemented. -
FIG. 14B is a system diagram of an example wireless transmit/receive unit (WTRU) that may be used within the communications system illustrated inFIG. 14A . -
FIG. 14C is a system diagram of an example radio access network and an example core network that may be used within the communications system illustrated inFIG. 14A . -
FIG. 14D is a system diagram of another example radio access network and an example core network that may be used within the communications system illustrated inFIG. 14A . -
FIG. 14E is a system diagram of another example radio access network and an example core network that may be used within the communications system illustrated inFIG. 14A . - A detailed description of illustrative embodiments will now be described with reference to the various Figures, Although this description provides a detailed example of possible implementations, it should he noted that the details are intended to be exemplary and in no way limit the scope of the application.
- The screen display of mobile devices may be able to display high definition resolution content, ultra-high definition resolution content, and/or the like. Video coding tools, such as block coding modes and transform, may not be optimized for screen content encoding.
-
FIG. 1 is a diagram that illustrates an example block diagram of a screen content sharing system. The screen content sharing system may include a receiver, a decoder, and/or a display (renderer).FIG. 2 is a diagram illustrating an example of a block-based single layer video encoder.FIG. 3 is a diagram illustrating an example of a block-based single layer video decoder. The video decoder ofFIG. 3 may receive a video bitstream produced by the encoder ofFIG. 2 . The video decoder may reconstruct the video signal to be displayed. At the video decoder, the bitstream may be parsed by an entropy decoder. The residual coefficients may be inverse quantized and inverse transformed to obtain the reconstructed residual. The coding mode and/or prediction information may be used to obtain the prediction signal using spatial prediction and/or temporal prediction. The prediction signal and/or the reconstructed residual may be added together to generate the reconstructed video. The reconstructed video may go through loop filtering before being stored in a reference picture store. The reconstructed video may be displayed and/or to be used to decode future video signal(s). - A single layer encoder may employ spatial prediction (e.g., intra prediction) and/or temporal prediction (e.g., inter prediction and/or motion compensated prediction) to predict the input video signal. The encoder may include mode decision logic that selects a form of prediction, for example, based on rate and/or distortion considerations. The encoder may transform and quantize the prediction residual (e.g., the difference signal between the input signal and the prediction signal). The quantized residual, the mode information (e.g., intra or inter prediction), and/or prediction information (e.g., motion vectors, reference picture indexes, intra prediction modes, and/or the like) may be compressed at the entropy coder and packed into the output video bitstream. As shown in
FIG. 2 , the encoder may generate the reconstructed video signal by applying inverse quantization and inverse transform to the quantized residual to obtain a reconstructed residual, and add the reconstructed residual to the prediction signal. The reconstructed video signal may go through loop filter process (e.g., deblocking filter. Sample Adaptive Offsets, Adaptive Loop Filters, and/or the like) The reconstructed video signal may be stored in the reference picture store to he used to predict future video signal. - High Efficiency Video Coding (HEVC) may be a block based hybrid video coding standard, in that its encoder and decoder may operate substantially in accordance with the example encoder and decoder of
FIG. 2 andFIG. 3 . HEVC may allow the use of larger video blocks and/or may use quadtree partition to signal block coding information. The picture or slice may be partitioned into coding tree blocks (CTB) with the same size (e.g 64×64). A CTB may be partitioned into CUs with quadtree, A CU may be partitioned into prediction units (PU) and/or transform units (TU) with quadtree. -
FIG. 4 is a diagram illustrating an example of eight partition modes. For an inter-coded CU, its PU may be one of eight partition modes, for example, as shown inFIG. 4 . Temporal prediction (e.g., motion compensation) may be applied to reconstruct inter-coded PUs. Depending on the precision of the motion vectors (e.g., which may be up to quarter pixel in HEVC), linear filters may be applied to obtain pixel values at fractional positions. The interpolation filters may have seven or more taps for lama and/or four taps for chroma. The deblocking filter in HEVC may be content based, for example, such that different deblocking filter operations may be applied at the TU and PU boundaries, for example, depending on coding mode difference, motion difference, reference picture difference, pixel value difference, and/or the like. For entropy coding, HEVC may use context-based adaptive arithmetic binary coding (CABAC) for block level syntax elements (e.g., except high level parameters). CABAC coding may include context-based coded regular bins and/or by-pass coded bins without context. - HEVC may be focused on continuous tone video content in 4:2:0 format. The mode decision and transform coding tools may not be optimized for the discrete tone screen content, for example, which may be captured in the format of 4:4:4 video.
-
FIG. 5A is a diagram illustrating an example of a screen content block.FIG. 5B is a diagram illustrating an example of a palette index map of the screen content block ofFIG. 5A . As shown inFIG. 5A , screen content blocks may include a limited number of colors, and the color value of each pixel may be repeated from its above and/or left pixel. Instead of directly coding the pixel value, a palette table may be used as a dictionary to record significant color values, And the corresponding palette index map may be used to represent the color value of each pixel, for example, as shown inFIG. 5B . The run values may be used to indicate the length of consecutive pixels that have the same color (e.g., palette index) to reduce the spatial redundancy. Using a palette based coding method rather than the conventional block coding modes may improve the compression performance of coding screen content. - Palette coding modes may include an index mode (e.g., run mode) and/or a copy mode. In the index mode, the color index may be coded first. If the color index is equal to the palette table size, it may indicate that an escape color is not included in the palette table. And the color value may be explicitly coded following the color index. If the color index is less than the palette table size, which may mean a major color in the palette table, then a run value may be coded to signal how many continuous pixels in the scanning order have the same color as that coded color. In the copy mode, the run value may be coded to indicate how many pixels have the same color as its above pixel. Scanning orders may include, but are not limited to, a horizontal traverse scan and/or a vertical traverse scan, for example, as shown in
FIG. 11A andFIG. 11B and as described herein. - For pixel positions coded in run code, the palette index of the first position in the run-length chain may be present in the bit-stream. Palette indices may be binarized using truncated binary code (TBC) and/or coded in bypass mode. TBC may be a variant of fixed length code (FLC) and/or used for alphabets with uniform distributions. TBC may degenerate to FLC when the size of the alphabet is a power of two. For example, for one palette index level pLevel, it may be assumed that its maximum value pMax is known. As an example, n=pMax+1 and k=floorlog_2 (n)) such that 2{circumflex over ( )}k<n<2{circumflex over ( )}(k+1) and u=2{circumflex over ( )}(k+1)−n. The palette index may be binarized, for example, as follows: if pLevel<u, the codeword may be specified by the binary representation of pLevel with length k; otherwise, the codeword may be specified by the binary representation of pLevrel+u with
length k+ 1. Table 1 provides an example of palette index binarization when pMax=8. -
TABLE 1 Example of TBC binarization with pMax = 8 Level Codeword 0 000 1 001 2 010 3 011 4 100 5 101 6 110 7 1110 8 1111 - As can be seen from Table 1, before binarizing one palette index level, the maximum level pMax may be specified as an input to the TBC process.
- To exploit the correlations between palette indices in one palette-coded CU, one redundancy removal method may be applied to improve the efficiency of palette index coding by reducing the magnitude of coded palette indices and/or the maximum length of TBC codewords. For example, one or more of the following conditions may be checked before coding one palette index in the index mode. If the left neighbor of the current pixel position is coded in the index mode, then the current palette index may be the same as that of its left neighbor. Otherwise, if the two palette indices are the same, the palette indices may be coded together in index mode with a larger run-length. If the left neighbor of the current pixel position is coded in copy mode, then the current palette index may be the same as that of its above neighbor. Otherwise, if the two palette indices are the same, the palette indices may be coded together in copy mode with a larger run-length. Therefore, if either of the above two conditions are satisfied, the palette index pLevel and/or the maximum TBC value pMax may be reduced (e.g., by one), which may decrease the codeword length used to present the palette index of the current pixel position.
-
FIG. 12A is a diagram illustrating an example of a redundancy removal method of palette index coding where the CPI position is coded in index mode.FIG. 12B is a diagram illustrating an example of a redundancy removal method of palette index coding where the CPI position is coded in copy mode.FIG. 12A andFIG. 12B illustrate examples of the redundancy removal process when the left neighboring pixel is coded in index mode (e.g., as shown inFIG. 12A ) and in copy mode (e.g., as shown inFIG. 12B ). The compared palette index (CPI) position k that is referred by the current pixel position i, as enclosed by the dashed black line inFIG. 12A andFIG. 12B , may be calculated as follows: -
- where iWidth is the width of the current CU.
- The palette index may be independently coded rather than coded in index mode or copy mode, for example, although escape colors may be indicated by the palette index which is equal to the palette size in the current palette coding design. In other words, one run-length chain of pixel positions with the same palette index value may be broken when an escape color position is scanned. As the palette index used for escape colors may not be involved in the run-length based coding, the above redundancy removal algorithm used when coding palette indices may be applicable (e.g., only applicable) if the corresponding CPI position is not a position with an escape color.
- For copy mode, palette indices may be predicted from the above decoded indices in the above row. For the pixels in the first row of one palette CU, the copy mode may be disabled such that they are coded (e.g., always coded) by index mode, for example, because the above reference pixels exceed the CU boundary. The copy mode may be enabled for the pixels in the first row (e.g., or the first column when palette_transpose_flag is equal to 1) by using the boundary pixels of the neighboring CUs as reference, for example, in order to improve the efficiency of copy mode.
FIG. 13 shows one example of the extended copy mode when palette_transpose_flag is equal to 0 (e.g., horizontal scan order). To reconstruct the color indices of the first row of the current CU, if the indices are encoded by copy mode, the decoder may reconstruct them from the corresponding neighboring reference pixels (NCPs) from the above neighboring CU. The color values of NCPs may be used for the copy mode of the first line and the decoder may not have or derive palette indices of NCPs, for example, as shown inFIG. 13 . Therefore, palette index coding redundancy removal method may not be applied to pixels whose CPI pixels make reference to NCPs. For example, if the preceding pixel of the current pixel in the scan order is coded in copy mode and the above neighbor of the current pixel (e.g., the CPI position) refers to one NCP, the index coding redundancy removal method may be disabled. - The syntax elements may be binarized into a string of bins, in which bypass bins and context-coded bins may be coded separately by the engine of binary arithmetic coding (BAC), for example, for the context-based binary arithmetic coding (CABAC) that is used for palette mode and the other coding modes in HEVC screen content coding extension. For context-coded bins, context models (e.g., probability models) may be maintained for the BAC at both encoding and decoding. For bypass bins, syntax elements may be coded without context models. Grouping bypass bins together into longer chains may increase the number of bins processed per cycle (e.g., parsing throughout), which may be used for coding multiple syntax elements HEVC, such as motion vector coding and coefficient coding, for example. The syntax element of palette index coding, (e.g., palette_index_idc) may be bypass coded and/or interleaved with other context-coded syntax elements, such as, palette_run_type and palette_run_msb_id_plus1, for example, for palette design in HEVC screen content coding. The parsing of palette_index_idc may be grouped together and put in front of the other context coded syntax elements, for example, to improve the parsing throughput. Table 2 is an example of such a syntax table of palette coding.
-
TABLE 2 Example of Palette Syntax Descriptor palette_coding( x0, y0, nCbS ) { ... if( indexMax > 0) { palette_transpose_flag ae(v) palette_num_indices for( i = 0; i < palette_num_indices; i++ ) ae(v) palette_index_idc last_palette_run_type_flag ae(v) indicesLeft = palette_num_indices ae(v) } scanPos = 0 while( scanPos < nCbS * nCbS ) { ... if( indexMax > 0 && scanPos > = nCbS && palette_run_type_flag[ xcPrev ][ ycPrev ] ! = COPY_ABOVE_MODE && indicesLeft && scanPos < nCbS * nCbS − 1) { palette_run_type_flag[ xC ][ yC ] ae(v) } indicesLeft −= 1 if( indexMax > 0 && (indicesLeft | | palette_run_type_flag != last_palette_run_type_flag)) { maxPaletteRun = nCbS * nCbS − scanPos − 1 ... } } - The syntax element num_palette_index may specify the number of palette index signalled in the current block. The syntax element palette_last_run_type_flag may specify the palette coding mode of the last run in the current block.
- One or more embodiments described herein may be written using the HEVC standard as the underlying video coding standard, however, the embodiments may be applied to other video codecs.
- Palette coding efficiency may be provided. The palette coding may code horizontal lines and/or vertical lines. If it is a horizontal line, the index mode may code continuous pixels with a run value (e.g., a large run value). If it is a vertical line, then the vertical line may be treated as a horizontal line in a vertical traverse scan order, for example, as shown in
FIG. 11B . Lines in various directions may be used in screen content.FIG. 6 is a diagram illustrating an example SCC test sequence for a console. - Lines that are not in a horizontal direction or a vertical direction may be referred to as irregular lines. A block may be partitioned into a plurality of discontinuous segments, for example, if there are irregular lines. It may be difficult to encode the plurality of discontinuous segments with existing palette coding modes in an efficient way.
FIG. 7A is a diagram illustrating an example 8×8 block with four colors, where the dashed line illustrates a scanning order in palette coding.FIG. 7B is a diagram illustrating an example 8×8 block with four colors, where the dashed line illustrates a scanning order in palette coding that uses index mode and copy mode. For example, there may be two irregular lines in one 8×8 block, for example, as shown inFIG. 7A . There may be four kinds of colors, white, grey, blue, and yellow, which may be illustrated inFIG. 7A andFIG. 7B as different hash lines and/or shading.FIG. 7B illustrates an example of palette coding with run and copy mode in horizontal traverse scanning order for the 8×8 block. In total, 17 runs may be coded with index mode, for example, shown in black dashed lines and copy modes shown in black solid lines. - Parsing dependencies of palette coding may be provided. For example, given the high coding efficiency of palette coding for screen content, there may be a large percentage of CUs that select palette coding as the optimal coding mode, and thus palette coding may limit data parsing dependencies to enable high-throughput implementations. There may be a plurality of different kinds of dependencies in palette coding design. For example, context modeling dependencies may be used, which may refer to the dependencies in context derivation of one syntax element on previously coded syntax elements. Syntax parsing dependencies may be used, which may refer to the dependencies that the decoded value of one syntax element at a pixel position is used as an input to the parsing of the syntax elements for the following pixel positions. Syntax signaling dependencies may be used, which may refer to the dependencies that the signaling of one syntax element may be dependent on the decoded value of other syntax elements.
- Context modeling dependencies may be provided. Context modeling dependency of a palette_transpose_flag may be provided. Horizontal traverse scans and/or vertical traverse scans may be enabled to scan the palette indices of one palette-coded CU. One flag palette_transpose_flag may be signaled for each CU coded with palette mode to select one of the two scan patterns for the current CU. Two context models may be used to code the flag palette_transpose_flag, which may be selected depending on whether the palette indices of the left neighboring CU of the current CU are scanned horizontally or vertically. The neighbor-based context modeling approach may provide higher coding efficiency. The neighbor-based context modeling approach may not allow for a high degree of parallelism because two neighboring palette-coded CUs, which may not be decoded simultaneously. The neighbor-based context modeling approach may use one additional buffer to store the status of a palette_transpose_flag for a left neighboring CU.
- Context modeling dependency of palette_mode may be provided. A flag palette_mode may be signaled for one or more pixel positions, for example, except for the pixel positons in the first row of one palette-coded CU (e.g., as there may be no above pixels to copy, the copy mode may be invalid for the pixel positions in the first row). Two context models may be used to code the palette_mode flag, which may be determined based on the value of the same syntax element of the collocated pixel in the above row. Accordingly, the context modeling dependency of palette_mode approach may reduce the throughput, as the determination of the context at one pixel may be dependent on the palette index coding mode of pixels preceding it in the scan order.
- Dependencies related to escape color signaling may be provided. Syntax parsing dependencies of palette_index may be provided. For pixels coded in index mode, the palette index of the first pixel in the run-length chain may be signaled via the syntax element palette_index. The syntax element palette_index may be binarized using TBC, for which the maximum value of palette index (e.g., pMax) may be used as input to the TBC process, The calculation of the input pMax to the TBC process of one pixel position may be dependent on whether the left neighbor and the CPI position of the current pixel position are coded as escape colors or not, for example, because the redundancy removal approach may be applied to palette index coding and the palette index used to indicate escape colors may not be coded in index mode or copy mode. For example, if the left neighbor or the CPI position of the current pixel position is coded as an escape color, pMax may be set to the palette table size Size_PLT of current CU. Otherwise (e.g., if both the left neighbor and the CPI position of the current pixel position are major colors), pMax may be set to Size_PLT−1. According to the escape color signaling in the current palette coding design, the escape color may be identified by checking if the decoded palette index at one pixel position is equal to the palette table size. Therefore, the decoded value of the current palette_index syntax element may determine the value of the next syntax element to be processed. One palette index may be decoded (e.g., only one palette index) after the palette index of its CPI position is completely reconstructed, for example, when the redundancy removal approach is used in palette index coding.
- Syntax signaling dependencies of palette_escape_val may be provided. For an escape color position for each escape color position of a CU), the color values of the position may be quantized (e.g., if lossy coding is used) and transmitted to a decoder. For example, escape colors may be present in a bitstream by signaling the syntax element palette_index (e.g., which is equal to Size_PLT when the redundancy removal approach is not applied, and Size_PLT−1 when the redundancy removal approach is applied) followed by another syntax element palette_escape_val, which may indicate the quantized color values (e.g., if lossy coding is used). The syntax palette_escape_val element may be signaled (e.g., only signaled) when one pixel is identified as an escape color as indicated by palette_index. Therefore, the value of a palette_index syntax element may determine what syntax element is to be processed. For example, if palette_index indicates the current pixel is an escape color, then the next syntax element may be palette_escape_val, and otherwise the next syntax element may be palette_run. Based on the same analysis of the syntax parsing dependencies of palette_index, one pixel may be (e.g., may only be) identified as an escape color after the palette index of its CPI position is fay reconstructed, for example, due to the redundancy removal process applied to palette index coding. Therefore, the syntax signaling dependency of escape colors may cause throughput issues.
- Syntax signaling dependencies of palette_run may be provided. The syntax element palette_run may indicate the number of consecutive pixels with the same palette index in index mode and copy mode. The signaling of the syntax element palette_run may be signaled (e.g., only signaled) when one pixel is identified as a major color, which for example, may in contrast to palette_escape_val. Therefore, the similar syntax signaling dependency issue that is applicable to the element palette_escape_val may exist for the signaling of palette_run.
- Palette table generation (e.g., at the encoder only) may be provided. One color-clustering-based palette table generation method may be used to select the major colors of one CU coded with palette mode by clustering the color values of the current CU into multiple sets and using the centroids of color clusters as the major colors in the palette table. In lossy coding, one color cluster may be generated by quantizing the pixels, for which the distortion between the color value and the centroid of the cluster may be a pre-defined threshold, to the same major color. This threshold may be set based on the assumption that the bit depths of luma and chroma components are equal to 8 bits. Therefore, the current palette table generation method may not properly handle the case when the bit depths of input screen content video are more than 8 bits.
- One or more embodiments may address the conflicts for the combination of the extend copy mode and grouping palette indices at front. For example, the palette index coding bins may be grouped at the front of one palette CU. Palette indices (e.g., as specified by the syntax element palette_index_idc) may be binarized by TBC code for which the maximum possible level need to be known. The redundancy removal may not be applied to pixels whose CPI pixels make reference to NCPs. Therefore, different maximum TBC levels may be derived depending on the positions of pixels. For example, for pixels that refer to CPI pixels associated with the NCPs from the neighboring CU, the maximum TBC level pMax may be kept unchanged (e.g, the size of the palette table minus 1 when there is no escape color pixel in the current CU, or the size of the palette table when there is at least one escape color pixel in the current CU). For pixels that refer to CPI pixels not associated with NCPs from the neighboring CU, the maximum TBC level pMax may be reduced by 1. The parsing of the syntax palette_index_idc for one pixel may be dependent on the knowledge of the palette coding modes of the preceding pixels in the palette scan order, And, this knowledge may be acquired after decoding the palette coding modes (e.g., palette_run_type_flag) and palette runs (e.g., palette_run_msb_id_plus1 and palette_run_refinement_bits) for the preceding pixels of the current pixel. Therefore, when the extended copy mode is enabled, the parsing of the grouped palette indices may not be placed in front of the parsing of palette modes and palette runs.
- A palette signaling method may reduce the maximum TBC level pMax by one, for example, in order to group palette indices at beginning of palette parsing process while enabling extended copy-above. The dynamic range of the palette indices for those pixels may be 0 to pMax, for example, since redundancy removal may not be applied to pixels whose CPI pixels refer to the NCPs from the neighboring CU. An additional flag may be signaled for those pixels (e.g., the pixels whose CPI pixels refer to the NCPs from the neighboring CU) when the corresponding parsed palette indices are equal pMax−1, for example, in order to compensate the reduced dynamic range of palette indices. If the flag is equal to 1, the palette index may be decoded to be pMax. If the flag is equal to 0, the palette index may be decoded to be pMax−1. For example, Table 3 is an example palette syntax table.
-
TABLE 3 Example Palette Syntax Table Descriptor palette_coding( x0, y0, nCbS ) { ... ... if( indexMax > 0 ) { palette_transpose_flag ae(v) palette_num_indices ae(v) for( i = 0; i < palette_num_indices; i++ ) { palette_index_idc ae(v) PaletteIndexIdc[i] = palette_index_idc } last_palette_run_type_flag ae(v) NumIndicesLeft = palette_num_indices } scanPos = 0 while( scanPos < nCbS * nCbS ) { xC = x0 + travScan[ scanPos ][ 0 ] yC = y0 + travScan[ scanPos ][ 1 ] if( scanPos > 0) { xcPrev = x0 + travScan[ scanPos − 1 ][ 0 ] ycPrev = y0 + travScan[ scanPos − 1 ][ 1 ] } if( indexMax > 0 && (scanPos = = 0 | | ( palette_run_type_flag[ xcPrev ][ ycPrev ] ! = COPY_ABOVE_MODE && NumIndicesLeft && scanPos < nCbS * nCbS − 1 ) ) ) { palette_run_type_flag[ xC ][ yC ] ae(v) } if( palette_run_type_flag[ xC ][ yC ] = = COPY_INDEX_MODE && 0indexMax > 1) { if ( adjustedIndexMax = = indexMax − 1 && PaletteIndexIdc[palette_num_indices − NumIndicesLeft] = = (indexMax − 1) ) palette_index_refinement_flag ae(v) NumIndicesLeft −= 1 } if( indexMax > 0 && (NumIndicesLeft | | palette_run_type_flag != last_palette_run_type_flag) ) { maxPaletteRun = nCbS * nCbS − scanPos − 1 if( maxPaletteRun > 0 ) { palette_run_msb_id_plus1 ae(v) if( palette_run_msb_id_plus1 > 1 ) palette_run_refinement_bits ae(v) } } else paletteRun = nCbS * nCbS − 1 − scanPos runPos = 0 while ( runPos < = paletteRun ) { xR = x0 + travScan[ scanPos ][ 0 ] yR = y0 + travScan[ scanPos ][ 1 ] if( palette_run_type_flag[ xC ][ yC ] = = COPY_INDEX_MODE && paletteIndex = = indexMax ) { PaletteSampleMode[ xR ][ yR ] = ESCAPE_MODE PaletteIndexMap[ xR ][ yR ] = paletteIndex for( cIdx = 0; cIdx < 3; cIdx++ ) { palette_escape_val ae(v) PaletteEscapeVal[ cIdx ][ xR ][ yR ] = palette_escape_val } } else if(palette_run_type_flag[ xC ][ yC ] = = COPY_INDEX_MODE ) { PaletteSampleMode[ xR ][ yR ] = COPY_INDEX_MODE PaletteIndexMap[ xR ][ yR ] = paletteIndex } else { PaletteSampleMode[ xR ][ yR ] = COPY_ABOVE_MODE PaletteIndexMap[ xR ][ yR ] = (yR − 1) < y0 ? (bNeighbour ? NCP_INDEX :0) :PaletteIndexMap[ xR ][ yR − 1 ] } runPos++ scanPos++ } } } - One or more embodiments may address the unknown TBC maximum problem when combining the extended copy-above mode with grouping indices at front. The syntax element palette_index_refinement_flag may be coded as bypass bins and/or interleaved with the syntax elements of signaling palette mode and palette runs, which may be context-coded. To parse the value of palette_index_refinement_flag, a decoder may check whether the current pixel has one CPI position that refers to the pixels from the neighboring CU and/or whether the palette index parsed previously is equal to pMax−1 or not.
- A coding mode sketch copy mode is proposed to code blocks comprising irregular lines. Syntax redundancy may be removed from blocks with special characteristics. The run value coding may be simplified. The parsing dependencies in the current palette coding design may be removed. For example, the context modeling dependency of the syntax element palette_transpose_flag may be removed, for example, by simplifying the corresponding context model. The context modeling of the syntax element palette_mode may be removed, for example, by using run-length coding without using context. The syntax parsing dependencies and/or the syntax signaling dependencies that are related with escape color signaling may be removed. A palette table generation process may handle input screen content video with high bit depths, for example, at the encoder side.
- Sketch copy mode for palette coding may be provided. Sketch copy mode may be performed by copying a palette index from a coded neighboring sample position in one or more directions (e.g., which may include diagonals of any sort, horizontal, and/or vertical directions). Sketch copy mode may allow pixel copying in one or more specific directions. Coded irregular lines (e.g., diagonal lines) may be considered during the counting of a run. One or more (e.g., three) syntax elements may be coded for sketch mode: color index, direction, and/or run. The color index may be an index in the palette table and/or be equal to the size of palette table to indicate an escape color. The direction information may be coded following the color index. A run value may be coded to indicate the number pixels having the same color index as first pixel in that signaled direction.
- The directions to be coded may be defined.
FIG. 8A is a diagram illustrating an example of 16 directions with course granularity.FIG. 8B is a diagram illustrating an example of 16 directions with fine granularity.FIG. 8A andFIG. 8B illustrate example definitions for 16 directions in different granularities, where each side may have eight directions respectively. The directions inFIG. 8A may be defined as follows: -
θi=tan−1(x i), x i={1,2, . . . , 8} - The directions defined in
FIG. 8B may be in a fine granularity, and the directions may be defined as follows: -
θi=tan−1(x i), x i={½, 1, . . . , 4} - The vertical direction may not be considered in a sketch copy mode, for example, because index mode in a vertical traverse scan may code a vertical line efficiently. The number of directions may be reduced if the overhead for direction coding affects performance, for example, at low bitrate coding. The direction information may be segmented into one or more elements, for example, a flag “is_righ_flag” and the index of direction at one side. “is_right_flag” may be used to indicate which side the direction belongs to. A maximum index of a direction may be equal to a plurality of different directions, for example, seven different directions (e.g., to each side) as provided by the examples of
FIG. 8A andFIG. 8B . The direction of a previous coded neighboring pixel may be considered, for example, to further remove the redundancy of direction coding. If a current pixel has the same direction compared to the direction of a previous coded neighboring pixel, then a flag “same_direction_flag” may be coded (e.g., only the flag “same_direction_flag” is coded). If directions are not the same but they have the same value as “is_right_flag”, then the pruning process may be performed.FIG. 9 is a diagram illustrating an example flow chart of the direction coding algorithm for sketch mode. -
FIG. 10 is a diagram illustrating an example of palette coding with sketch copy mode.FIG. 10 may be compared to the palette coding inFIG. 7B . The irregular lines may be coded with sketch mode. At the position S11, the irregular line (e.g., the diagonal line going from S11 to S39) may be coded by a sketch mode with a run value being equal to three. The is_right_flag may be set to 1 and the direction index may be 0. At the position S12, the irregular line (e.g, the diagonal line going from S12 to S32) may be coded by a sketch mode with a run being equal to three. The is_right_flag may be set to 0 and the direction index may be 0. After position S12, an index mode may code the remaining pixels in the block. Five runs may be coded in total, for example, because the irregular lines are not included. The number of segments in that block may be reduced by enabling sketch copy mode. - The block prone to apply palette coding may include a plurality of colors. Sketch copy mode may be applied (e.g., only applied) for blocks that include one or more irregular lines. If there are no irregular lines within the block, then sketch copy mode may not be selected though, for example, it may be enabled, The signaling overhead for run and copy mode may be increased due to one or more additional modes added. Therefore, the usage of sketch copy mode may be restricted to reduce the signaling overhead. A flag may be added in the coding unit coded with a palette to indicate whether the sketch copy mode is enabled or not. For example, if it is not enabled, then no signaling overhead is added. The applications of sketch copy mode may be limited, for example, sketch copy mode may be used within one block because sketch copy mode will not save too much if there are not many pixels remaining. Sketch copy mode may not save signaling overhead for remaining pixel coding, for example, if the coding position is beyond the specific row defined by a threshold. The threshold may be adaptive to the size of a coding unit. For example, the threshold may be [4, 7, 12, 16] for 8×8, 16×16, 32×32, and/or 64×64 coding units.
- The scanning method may be extended.
FIG. 11A andFIG. 11B illustrate examples of a coding order of a horizontal traverse scan and a vertical traverse scan, respectively. The horizontal traverse scan and/or the vertical traverse scan may be performed in a reverse order, which for example, may be equivalent to the block being rotated by 180 degrees in a clock-wise or counter clock-wise manner.FIG. 11C andFIG. 11D illustrate examples of a coding order in a reverse horizontal traverse scan and in a reverse vertical traverse scan, respectively. - Redundancy removal for palette table syntax elements may be provided. The syntax element palette_escape_val_present_flag may be used to indicate whether an escape color exists in a coding unit (CU). The escape color may be signaled with the color index being equal to the palette table size. If there is no escape color, then the maximum color index may be equal to palette table size minus one. Otherwise, the maximum color index may be equal to palette table size. This maximum value may affect the color index coding, for example, because the color index may be coded with truncated binary code. The number of bits for a variable x in truncated binary code may be provided by the following:
-
- where M is the maximum value of variable x plus one. From the equation above, it may be determined that additional values may be coded with (n−1) bits with a smaller M. The element palette_escape_val_present_flag may not be signal, for example, to remove palette signaling redundancy. When the palette table size for a CU is 0 (e.g, the palette table is empty), the decoder may determine that all pixels are escape colors (e.g., there are no major colors in the palette table). As such, when the palette table size fora CU is 0, the palette_escape_val_present_flag may not be signaled and the decoder may infer the value to be 1. When the palette table size is a value other than 0. then the palette_escape_val_present_flag may be signaled. Therefore, the palette_escape_val_present_flag may be signaled (e.g., only signaled) when the palette table size is a value other than 0. If there is one color (e.g., only one color) in the coding unit (e.g., the palette table size is I and palette_escape_val_present_flag is 0) or if all pixels are coded as escape colors (e.g., palette table size is 0 and palette_escape_val_present_flag is 1), then the palette mode may be in index mode (e.g., may be inferred to be in index mode (e.g., run mode)) and the run value may be the size of coding unit minus 1 if run is coded. For example, if there is one color (e.g., only one color) in the coding unit (e.g., the palette table size is 1 and palette_escape_val_present_flag is 0), then the decoder may not receive palette_mode and/or may infer palette_mode to be equal to 0 (e.g., inferred to be in index mode (e.g., run mode)) and/or may not receive palette_run and/or may infer palette_run to be equal to the size of the
CU minus 1. A condition may be added for palette mode signaling and/or run coding, for example, as shown in Table 6. - The entropy coding of the syntax element Run in palette coding may be provided. Two different codes may be used to code the syntax element Run. Table 4 is an example of a bin.arization of the value of Run. There may be three segments for the whole range of Run. The first one, two, or three digits of the bin may be context coded (e.g., those illustrated in bold in Table 4), and the subsequent digits in the bin may be by-pass coded without any context (e.g., those illustrated as not in bold in Table 4). 0 to 2 may be the first segment coded with fixed context for each bin. 3 to 26 may be the second segment. The coding of prefix may be the same as
segment 1, and the suffix may be coded with Golomb-Rice codes and/or the rice parameter may be equal to 3. A value beyond 26 may be a third segment. The prefix of the third segment may be coded the same assegment 1. The suffix may be coded with Exp-Golomb codes. Run values from 3 to 10, 3 prefix bins. and/or 4 suffix bins may be coded. A second order Exp-Golomb may be provided for values beyond 2. An example binarization may be provided in Table 5. For small runs, the number of bins may be reduced. -
TABLE 4 Example Binarization of Run value (e.g., in SCC) Run Bin 0 0 Segment1 1 10 2 110 3 111 0000 Segment2: prefix “111” + suffix . . . 111 0xxx Golomb-Rice code with rice parameter 11 111 10000 equal to 3 . . . 111 10xxx 19 111 110000 . . . 111 110xxx 27 111 1110000 Segment3: prefix “111” + suffix . . . 111 1110xxx Exp- Golomb code 35 111 111100000 . . . 111 11110xxxx -
TABLE 5 Example Binarization of Run value Run Bin 0 0 Segment1 1 10 2 110 3 111 000 Segment2: prefix “111” + 2nd order . . . 111 0xx Exp- Golomb code 7 111 10000 . . . 111 10xxx 15 111 1100000 . . . 111 110xxxx 31 111 111000000 . . . 111 1110xxxxx 63 111 11110000000 . . . 111 11110xxxxxx - Parsing dependency improvement for palette coding may be provided. Removing the context modeling dependency of the syntax element “palette_transpose_flag” may be provided. One or more (e.g., two context models) may be used to code the syntax element palette_transpose_flag. One context model may be selected for palette_transpose_flag, for example, based on whether the palette indices of the left neighboring CU of the current CU are scanned horizontally or vertically. This could complicate parallelism design for practical implementation as the bit-streams of two neighboring palette-coded CUs may not be parsed simultaneously. This neighbor-based context modeling may use one binary buffer to store the value of the syntax element for the left neighboring CU. The palette_transpose_flag flag may be the CU-level flag that establishes its context models using its spatial neighbors, for example, while the other CU-level flags (e.g., such as palette_share_flag, palette_escape_val_present_flag, num_signalled_palette_entries, and/or the like) may be coded in bypass mode and/or coded in context mode, using one single context.
- The context modeling dependency of the syntax element palette_transpose_flag on the left neighbor of the current CU may be removed, for example, by coding the flag with one single context. This may be performed to improve parallelism capacity and/or to make CU-level signaling desgin of palette coding more consistent.
- The syntax element palette_transpose_flag may be coded in bypass mode.
- The context modeling dependency of the syntax element “palette_mode” may be removed. One or more context models may be used to code the palette_mode_flag, which may be determined based on the value of the same syntax element of the collocated pixel in the above row. Table 6 illustrates an example of the context modeling used to code the palette_mode flag. In the index mode and/or the copy mode, palette_mode may be decoded for each run-length chain of consecutive pixels with the same palette index. Correspondingly, this context modeling approach may reduce the throughput of the parsing process, as the determination of the context at one pixel may be dependent on the palette index coding mode of pixels preceding it in the scan order.
-
TABLE 6 Example of context modeling of the syntax element palette_mode Condition Context index The above pixel is coded in 0 index mode or escape color The above pixel is coded in 1 copy-above mode - The flags palette_mode of one CU may form a binary vector which may be composed of elements “0” and “1.” Run length coding (RLE) is method of coding binary vector by indicating the number of consecutive 0's between two 1's. RLE may be used for applications that require high throughput, for example, because multiple bins can be generated together within one single circle. For example, to remove the context modeling of palette_mode, RLE may be used to code the syntax element palette_mode.
- The context modeling dependency of the syntax element palette_mode may be removed by coding the flag with a single context.
- The syntax parsing and/or signaling dependencies related with escape color signaling may be removed. There may be syntax parsing dependencies and syntax signaling dependencies associated with the signaling of escape colors when parsing the syntax elements palette_index, palette_escape_val and palette_run, for example, due to the TBC-based binarization and the redundancy removal algorithm that are applied to palate index coding. The parsing dependencies may be removed, for example, for a higher throughput and/or to improve the efficiency of palette coding.
- The palette index used for escape colors may not be coded in index mode or in copy mode, for example, although the current palette coding method may use the maximum palette index to indicate escape colors. For example, one run-length chain of consecutive pixels with the same palette index may be broken when an escape color is encountered. This may compromise the overall coding efficiency of palette coding and/or introduce dependencies to the parsing process for CUs coded with palette mode. After the redundancy removal algorithm is applied, the palette index of one pixel may be parsed after the palette index of its CPI position is fully reconstructed (e.g., in order to check if the CPI position is coded as escape color or not, which may determine the input parameter pMax for the TBC binarization process).
- The palette index used to indicate escape colors may be used as one normal palette index and it may be coded using an index mode and/or a copy mode. For example, a decoder may parse palette index run values of a CU. The palette index run values may indicate a length of consecutive palette indices for corresponding pixel positions in the CU. One or more of the palette indices may correspond to an escape color. For example, the palette indices may correspond to a plurality of different escape values. As such, a run of escape colors (e.g., which may be the same or different escape colors) may be coded in index mode and/or copy mode. For example, color values for the different pixel positions of a CU may be encoded according to one or more runs, which may include one or more runs of a major color and/or one or more runs of escape colors. A run of escape colors may be a run of the same escape color or different escape colors. The decoder may reconstruct the consecutive palette indices for the corresponding pixel positions of the CU according to the palette index run values (e.g., which may be a run of escape colors). The decoder may reconstruct the color values for the pixel positions of the CU according to the palette indices and/or one or more palette_escape_val values. The parsing of the syntax element palette_escape_val may be separated from that of the other syntax elements, for example, in order to remove the syntax signaling dependencies of the element palette_escape_val. For example, one or more scan passes may be applied to parse the syntax elements of one palette-coded CU. For example, a scan pass (e.g., a first scan pass) may parse the existing palette syntax elements except palette_escape_val. A scan pass (e.g., a second scan pass) may parse the syntax element palette_escape_val, for example, based on the decoded palette indices from the first scan pass. In the second scan pass, palette_escape_val may be parsed for one pixel position when its palette index, as derived by the first passes, is equal to the palette table size of the current CU (e.g, the escape color).
- The syntax parsing dependency of palette index may be removed. The maximum palette index pMax may be used with a fixed value (e.g., the maximum of palette table size) as the input to the TBC binarization process for palette index coding. To remove the syntax signaling dependencies of the elements palette_escape_val and palette_run on escape colors, the redundancy removal method for escape colors may be removed.
- Improved palette table generation for high bit-depth may be provided. A color-clustering-based palette table generation method may be used to select the major colors of one CU coded with palette mode by clustering the color values of the current CU into multiple sets and using the centroids of color clusters as the major colors in the palette table. In lossy coding, one color cluster may be generated by quantizing the pixels (e.g., all the pixels) for which the distortion between the color value and the centroid of the cluster is no more than a pre-defined threshold T, to the same major color, for example, as illustrated by the following:
-
- where P_i and C_i may represent the values of the i-th component of the current pixel and the centroid of the cluster, respectively. The threshold T may be set dependent upon a. quantization parameter. Table 7 illustrates an example of a mapping between the values of T and the quantization parameters.
-
TABLE 7 Example mapping between quantization parameter (QP) and threshold T (e.g., in SCM-2.0) QP 0 1 2 3 4 5 6 7 8 9 10 11 12 T 0 0 0 0 1 1 1 2 2 2 3 3 3 QP 13 14 15 16 17 18 19 20 21 22 23 24 25 T 4 4 4 5 5 5 6 6 7 7 8 9 9 QP 26 27 28 29 30 31 32 33 34 35 36 37 38 T 10 11 12 13 14 15 16 17 19 21 22 24 23 QP 39 40 41 42 43 44 45 46 47 48 49 50 52 T 25 26 28 29 31 32 34 36 37 39 41 42 45 - In Table 7, the values of the threshold T may be determined assuming that the input bit depths of luma and chroma components are 8 bits. Therefore, the palette table by the current palette table generation method may not properly handle the case when the bit depths of input screen content video are more than 8 bits. In order to improve the coding efficiency of palette coding for high bit depths, the 8-bit distortion may be used for the palette table generation process. The bit depths of lama and chroma components may be denoted as BitDepth_Y and BitDepth_C, and the distortion calculation equation may be as follows:
-
- Table 8 illustrates an example of syntax that may be used to enable the syntax redundancy removal method described herein and/or to remove the parsing dependencies as described herein.
-
TABLE 8 Example of Palette coding syntax Descriptor palette_coding( x0, y0, nCbS ) { palette_transpose_flag ae(v) palette_share_flag[ x0 ][ y0 ] ae(v) if( palette_share_flag[ x0 ][ y0 ] ) { palette_size = previousPaletteSize for( n = 0; n < palette_size; n++ ) for( cIdx = 0; cIdx < 3; cIdx++ ) palette_entries[ cIdx ][ n ] = previousPaletteEntries[ cIdx ][ n ] } else { numPredPreviousPalette = 0 for( i = 0; i < previousPaletteStuffingSize; i++ ) previous_palette_entry_flag[ i ] = 0 palette_last_group = 0 for( i = 0; i < previousPaletteStuffingSize && !palette_last_group && numPredPreviousPalette < max_palette_size; i++ ) { lastPossibleGroupFlag = ( i + 4 >= previousPaletteStuffingSize ) lastIdx = min( i + 4, previousPaletteStuffingSize ) − 1 if( i > 3 && !lastPossibleGroupFlag ) palette_all_zeros_in_group ae(v) Else palette_all_zeros_in_group = 0 if( palette_all_zeros_in_group ) i += 4 else { numOnesInGroup = 0 for( idx = i; idx <= lastIdx && numPredPreviousPalette < max_palette_size; idx++ ) { if ( idx = = lastIdx && numOnesInGroup = = 0 ) previous_palette_entry_flag[ idx ] = 1 Else previous_palette_entry_flag[ idx ] ae(v) if ( previous_palette_entry_flag[ idx ] ) { for ( cIdx = 0; cIdx < 3; cIdx++ ) palette_entries[ cIdx ][ numPredPreviousPalette ] = previousPaletteEntries[ cIdx ][ idx ] numPredPreviousPalette++ numOnesInGroup++ } } if( !palette_all_zeros_in_group && !lastPossibleGroupFlag && numPredPreviousPalette < max_palette_size ) palette_last_group ae(v) } } if( numPredPreviousPalette < max_palette_size) num_signalled_palette_entries ae(v) for( cIdx = 0; cIdx < 3; cIdx++ ) for( i = 0; i < num_signalled_palette_entries; i++ ) palette_entries[ cIdx ][ numPredPreviousPalette + i ] ae(v) palette_size = numPredPreviousPalette + num_signalled_palette_entries } if(palette_size >0) palette_escape_val_present_flag ae(v) if( palette_escape_val_present_flag ) indexMax = palette_size Else indexMax = palette_size − 1 scanPos = 0 while( scanPos < nCbS * nCbS ) { xC = x0 + travScan[ scanPos ][ 0 ] yC = y0 + travScan[ scanPos ][ 1 ] if( scanPos > 0) { xC_prev = x0 + travScan[ scanPos − 1 ][ 0 ] yC_prev = y0 + travScan[ scanPos − 1 ][ 1 ] } if(indexMax>0 && scanPos > = nCbS && palette_mode[xC_prev][yC_prev] ! = COPY_ABOVE ) palette_mode[ xC ][ yC ] ae(v) if( palette_mode[ xC ][ yC ] ! = COPY_ABOVE ) { adjustedIndexMax = indexMax adjustedRefIndex = indexMax + 1 } if( scanPos > 0) { if( palette_mode[xC_prev][yC_prev] = = INDEX ) { adjustedIndexMax − = 1 adjustedRefIndex = paletteMap[ xC_prev ][ yC_prev ] } if( scanPos > = nCbS && palette_mode[ xC_prev ][yC_prev ] = = COPY_ABOVE ) { adjustedIndexMax − = 1 adjustedRefIndex = paletteMap[ xC ][ yC − 1 ] } } if( palette_mode[ xC ][ yC ] ! = COPY_ABOVE ) { if( adjustedIndexMax > 0 ) palette_index ae(v) } if( indexMax>0) { palette_run ae(v) runPos = 0 runMode = palette_mode[ xC ][ yC ] while ( runPos < = palette_run ) { xC = x0 + travScan[ scanPos ][ 0 ] yC = y0 + travScan[ scanPos ][ 1 ] if( palette_mode[ xC ][ yC ] = = INDEX ) { palette_mode[ xC ][ yC ] = INDEX paletteMap[ xC ][ yC ] = palette_index } else { palette_mode[ xC ][ yC ] = COPY_ABOVE paletteMap[ xC ][ yC ] = paletteMap[ xC ][ y − 1 ] } runPos++ scanPos++ } } } scanPos = 0 while( scanPos < nCbS * nCbS ) { xC = x0 + travScan[ scanPos ][ 0 ] yC = y0 + travScan[ scanPos ][ 1 ] if( palette_index[xC][yC] = = palette_size ) { for( cIdx = 0; cIdx < 3; cIdx++ ) { palette_escape_val ae(v) paletteEscapeVal[ cIdx ][ xC ][ yC ] = palette_escape_val } } scanPos++ } tempPaletteEntries[ cIdx ][ i ] = palette_entries[ cIdx ][ i ] for( i = 0; i < previousPaletteStuffingSize && current_size < max_palette_predictor size; i++ ) if( previous_palette_entry_flag[ i ] = = 0 ) { for ( cIdx = 0; cIdx < 3; cIdx++ ) tempPaletteEntries[ cIdx ][ current_size ] = previousPaletteEntries[ cIdx ][ i ] current_size++ } previousPaletteStuffingSize = current_size previousPaletteEntries = tempPaletteEntries } - One or more embodiments may comprise parsing throughput for extended copy mode. The parsing of the grouped palette indices may not be placed in front of the parsing of palette modes and palette runs, for example, when the extended copy mode is enabled. One or more embodiments may be provided that increase the parsing throughput, and which may be jointly applied with extended copy mode.
- The palette indices may be grouped together and placed after the palette modes and palette runs (e.g., as compared to placing the indices before the modes and runs). Escape colors may be separated and placed after palette indices. For example, the following palette syntax arrangement may be used. The syntax elements related with palette modes and palette runs (e.g., which may include palette_run_type_flag, palette_run_msb_id_plus1, palette_run_refinement_bits and/or the like) may be placed in the current CU (e.g., first in the current CU). One or more (e.g., all) of the palette indices (e.g., palette_index_idc) for one or more (e.g., all) of the pixels that are coded by index mode may be placed after the syntax elements related to palette modes and palette runs. One or more (e.g., all) of the escape colors (e.g., palette_escape_val) for one or more (e.g., all) of the pixels that are coded as escape colors may be placed after the palette indices of the pixels. Table 9 illustrates an example of a palette syntax table with this arrangement.
-
TABLE 9 Example Palette coding syntax Descriptor palette_coding( x0, y0, nCbS ) { ... ... scanPos = 0 while( scanPos < nCbS * nCbS ) { xC = x0 + travScan[ scanPos ][ 0 ] yC = y0 + travScan[ scanPos ][ 1 ] if( scanPos > 0) { xcPrev = x0 + travScan[ scanPos − 1 ][ 0 ] ycPrev = y0 + travScan[ scanPos − 1 ][ 1 ] } if( indexMax > 0 && scanPos > = nCbS && palette_run_type_flag[ xcPrev ][ ycPrev ] ! = COPY_ABOVE_MODE ) { palette_run_type_flag[ xC ][ yC ] ae(v) } if( indexMax > 0 ) { maxPaletteRun = nCbS * nCbS − scanPos − 1 if( maxPaletteRun > 0 ) { palette_run_msb_id_plus1 ae(v) if( palette_run_msb_id_plus1 > 1 ) palette_run_refinement_bits ae(v) } } else paletteRun = nCbS * nCbS − 1 scanPos = scanPos + paletteRun + 1 } scanPos = 0 while( scanPos < nCbS * nCbS ) { xC = x0 + travScan[ scanPos ][ 0 ] yC = y0 + travScan[ scanPos ][ 1 ] if( palette_run_type_flag[ xC ][ yC ] = = COPY_INDEX_MODE && adjustedIndexMax > 0) palette_index_idc ae(v) runPos = 0 scanPosC = scanPos while ( runPos < = PaletteRun[travScan[ scanPosC ][ 0 ]][ travScan[ scanPosC ][ 1 ]] ) { xR = x0 + travScan[ ScanPos ][ 0 ] yR = y0 + travScan[ ScanPos ][ 1 ] if(palette_run_type_flag[ xC ][ yC ] = = COPY_INDEX_MODE ) { PaletteSampleMode[ xR ][ yR ] = COPY_INDEX_MODE PaletteIndexMap[ xR ][ yR ] = PaletteIndex } else { PaletteSampleMode[ xR ][ yR ] = COPY_ABOVE_MODE PaletteIndexMap[ xR ][ yR ] = PaletteIndexMap[ xR ][ yR − 1 ] } runPos++ scanPos++ } } scanPos = 0 while( scanPos < nCbS * nCbS ) { xC = x0 + travScan[ scanPos ][ 0 ] yC = y0 + travScan[ scanPos ][ 1 ] if( PaletteIndexMap[ xC ][ yC ] = = IndexMax && palette_escape_val_present_flag ) { PaletteSampleMode[ xC ][ yC ] = ESCAPE_MODE for( cIdx = 0; cIdx < 3; cIdx++ ) { palette_escape_val ae(v) PaletteEscapeVal[ cIdx ][ xR ][ yR ] = palette_escape_val } } . } } - The palette runs may be coded by the HEVC SCC run coding method. One or more of the run coding methods described herein may be combined with one or more of the described embodiments. For example, Table 10 is an example of a palette syntax that has an arrangement described above with run-to-end coding included.
-
TABLE 10 Example of palette coding syntax Descriptor palette_coding( x0, y0, nCbS ) { ... ... scanPos = 0 while( scanPos < nCbS * nCbS ) { xC = x0 + travScan[ scanPos ][ 0 ] yC = y0 + travScan[ scanPos ][ 1 ] if( scanPos > 0) { xcPrev = x0 + travScan[ scanPos − 1 ][ 0 ] ycPrev = y0 + travScan[ scanPos − 1 ][ 1 ] } if( indexMax > 0 && scanPos > = nCbS && palette_run_type_flag[ xcPrev ][ ycPrev ] ! = COPY_ABOVE_MODE ) { palette_run_type_flag[ xC ][ yC ] ae(v) } if( indexMax > 0 ) { maxPaletteRun = nCbS * nCbS − scanPos − 1 if( maxPaletteRun > 0 ) { palette_run_msb_id_plus1 ae(v) if( scanPos && (1<< palette_run _msb_id_plus1) > MaxPaletteRun ) palette_run_to_end_flag ae(v) if( !palette_run_to_end_flag ) palette_run_refinement_bits ae(v) } } else paletteRun = nCbS * nCbS − 1 scanPos = scanPos + paletteRun + 1 } scanPos = 0 while( scanPos < nCbS * nCbS ) { xC = x0 + travScan[ scanPos ][ 0 ] yC = y0 + travScan[ scanPos ][ 1 ] if( palette_run_type_flag[ xC ][ yC ] = = COPY_INDEX_MODE && adjustedIndexMax > 0) palette_index_idc ae(v) runPos = 0 scanPosC = scanPos while ( runPos < = PaletteRun[travScan[ scanPosC ][ 0 ]][ travScan[ scanPosC ][ 1 ]] ) { xR = x0 + travScan[ ScanPos ][ 0 ] yR = y0 + travScan[ ScanPos ][ 1 ] if(palette_run_type_flag[ xC ][ yC ] = = COPY_INDEX_MODE ) { PaletteSampleMode[ xR ][ yR ] = COPY_INDEX_MODE PaletteIndexMap[ xR ][ yR ] = PaletteIndex } else { PaletteSampleMode[ xR ][ yR ] = COPY_ABOVE_MODE PaletteIndexMap[ xR ][ yR ] = PaletteIndexMap[ xR ][ yR − 1 ] } runPos++ scanPos++ } } scanPos = 0 while( scanPos < nCbS * nCbS ) { xC = x0 + travScan[ scanPos ][ 0 ] yC = y0 + travScan[ scanPos ][ 1 ] if( PaletteIndexMap[ xC ][ yC ] = = IndexMax && palette_escape_val_present_flag ) { PaletteSampleMode[ xC ][ yC ] = ESCAPE_MODE for( cIdx = 0; cIdx < 3; cIdx++ ) { palette_escape_val ae(v) PaletteEscapeVal[ cIdx ][ xR ][ yR ] = palette_escape_val } } . } } - The syntax elements of palette index coding (e.g., palette_index_idc) and escape colors (e.g., palette_escape_val) may be separately grouped, for example, in the example palette coding syntax of Table 10. In such embodiments, two individual scan loops may be used for the palette indices and escape color values in the current CU.
- One or more embodiments may group palette indices and escape colors together (e.g., within the group, palette_index_idc and palette_escape_val are still interleaved) and put them after the palette modes and palette runs, for example. Both of palette_index_idc and palette_escape_val may be bypass coded. For example, the following syntax element arrangements may be provided: the syntax elements related with palette modes and palette runs palette_run_type_flag, palette_run_msb_id_plus1, and/or palette_run_refinement_bits) are placed first in the current CU. and one or more (e.g., all) of the syntax elements (e.g., palette_index_idc and palette_escape_val) are grouped together and placed after the palette modes and palette runs. An example of which is provided in Table 11.
-
TABLE 11 Example of palette coding syntax Descriptor palette_coding( x0, y0, nCbS ) { ... ... scanPos = 0 while( scanPos < nCbS * nCbS ) { xC = x0 + travScan[ scanPos ][ 0 ] yC = y0 + travScan[ scanPos ][ 1 ] if( scanPos > 0) { xcPrev = x0 + travScan[ scanPos − 1 ][ 0 ] ycPrev = y0 + travScan[ scanPos − 1 ][ 1 ] } if( indexMax > 0 && scanPos > = nCbS && palette_run_type_flag[ xcPrev ][ ycPrev ] ! = COPY_ABOVE_MODE ) { palette_run_type_flag[ xC ][ yC ] ae(v) } if( indexMax > 0 ) { maxPaletteRun = nCbS * nCbS − scanPos − 1 if( maxPaletteRun > 0 ) { palette_run_msb_id_plus1 ae(v) if( palette_run_msb_id_plus1 > 1 ) palette_run_refinement_bits ae(v) } } else paletteRun = nCbS * nCbS − 1 scanPos = scanPos + paletteRun + 1 } scanPos = 0 while( scanPos < nCbS * nCbS ) { xC = x0 + travScan[ scanPos ][ 0 ] yC = y0 + travScan[ scanPos ][ 1 ] if( palette_run_type_flag[ xC ][ yC ] = = COPY_INDEX_MODE && adjustedIndexMax > 0) palette_index_idc ae(v) runPos = 0 scanPosC = scanPos while ( runPos < = paletteRun[travScan[ scanPosC ][ 0 ]][ travScan[ scanPosC ][ 1 ]] ) { xR = x0 + travScan[ scanPos ][ 0 ] yR = y0 + travScan[ scanPos ][ 1 ] if( paletteIndex = = palette_size ) { for( cIdx = 0; cIdx < 3; cIdx++ ) { palette_escape_val ae(v) paletteEscapeVal[ cIdx ][ xC ][ yC ] = palette_escape_val } } runPos++ scanPos++ } } } - A second run coding method may be used. For example, Table 12 is an example of a palette syntax table with the arrangement described above with the second run coding (e.g., run-to-end syntax elements) included.
-
TABLE 12 Example of palette coding syntax Descriptor palette_coding( x0, y0, nCbS ) { ... ... scanPos = 0 while( scanPos < nCbS * nCbS ) { xC = x0 + travScan[ scanPos ][ 0 ] yC = y0 + travScan[ scanPos ][ 1 ] if( scanPos > 0) { xcPrev = x0 + travScan[ scanPos − 1 ][ 0 ] ycPrev = y0 + travScan[ scanPos − 1 ][ 1 ] } if( indexMax > 0 && scanPos > = nCbS && palette_run_type_flag[ xcPrev ][ ycPrev ] ! = COPY_ABOVE_MODE ) { palette_run_type_flag[ xC ][ yC ] ae(v) } if( indexMax > 0 ) { maxPaletteRun = nCbS * nCbS − scanPos − 1 if( maxPaletteRun > 0 ) { palette_run_msb_id_plus1 ae(v) if( scanPos && (1<< palette_run _msb_id_plus1) > MaxPaletteRun ) palette_run_to_end_flag ae(v) if( !palette_run_to_end_flag ) palette_run_refinement_bits ae(v) } } else paletteRun = nCbS * nCbS − 1 scanPos = scanPos + paletteRun + 1 } scanPos = 0 while( scanPos < nCbS * nCbS ) { xC = x0 + travScan[ scanPos ][ 0 ] yC = y0 + travScan[ scanPos ][ 1 ] if( palette_run_type_flag[ xC ][ yC ] = = COPY_INDEX_MODE && adjustedIndexMax > 0) palette_index_idc ae(v) runPos = 0 scanPosC = scanPos while ( runPos < = paletteRun[travScan[ scanPosC ][ 0 ]][ travScan[ scanPosC ][ 1 ]] ) { xR = x0 + travScan[ scanPos ][ 0 ] yR = y0 + travScan[ scanPos ][ 1 ] if( paletteIndex = = palette_size ) { for( cIdx = 0; cIdx < 3; cIdx++ ) { palette_escape_val ae(v) paletteEscapeVal[ cIdx ][ xC ][ yC ] = palette_escape_val } } runPos++ scanPos++ } } } - If the palette indices are grouped at front before palette mode, palette run, and escape color, and the extended copy above mode is enabled, the maximum value for palette index coding in TBC may not be determined for those palette indices whose above neighboring pixel refers to the pixels of the neighboring CU. The index at the above neighboring position may not be available. When the coded value is equal to the maximum value (e.g., index_max) minus 1, then an additional flag may be signaled to indicate whether the coded value should be interpreted as the maximum value minus one, or interpreted as the maximum value.
- One or more embodiments may be provided where palette indices are grouped (e.g., arranged) at the front and extended copy above mode is enabled. A CU level indication (e.g., flag) may be provided. For example, the CU-level indication may be arranged up front to instruct the decoder how to interpret and/or to entropy decode the palette indices.
- A syntax element use_index_max_for_palette_indices may be sent as a CU level flag, for example, as illustrated in Table 13. If the use_index_max_for_palette_indices flag has the
value 1, then the coding of palette indices may use the maximum value for TBC equal to the palette table size if there is no escape color, or the coding of palette indices may use the maximum value for TBC equal to the palette table size plus 1 if there is an escape color. If use_index_max_for_palette_indices flag has the value 0, then the coding of palette indices may use the maximum value for TBC equal to the palette table size minus 1 if there is no escape color, or the coding of palette indices may use the maximum value for TBC equal to the palette table size if there is an escape color. The decoder may determine the maximum value for TBC and decode the palette indices accordingly. -
TABLE 13 Example of palette syntax Descriptor palette_coding( x0, y0, nCbS ) { ... use_index_max_for_palette_indices ae(v) if( indexMax > 0) { palette_transpose_flag ae(v) palette_num_indices ae(v) for( i = 0; i < palette_num_indices; i++ ) palette_index_idc last_palette_run_type_flag ae(v) indicesLeft = palette_num_indices ae(v) } scanPos = 0 while( scanPos < nCbS * nCbS ) { ... if( indexMax > 0 && scanPos > = nCbS && palette_run_type_flag[ xcPrev ][ ycPrev ] ! = COPY_ABOVE_MODE && indicesLeft && scanPos < nCbS * nCbS − 1){ palette_run_type_flag[ xC ][ yC ] ae(v) } indicesLeft −= 1 if( indexMax > 0 && (indicesLeft | | palette_run_type_flag != last_palette_run_type_flag)) { maxPaletteRun = nCbS * nCbS − scanPos − 1 ... } } - The syntax element use_index_max_for_palette_indices may indicate whether index max is used as the maximum value for TBC encoding of the palette indices. The syntax element use_index_max_for_palette_indices having a value of 1 may indicate that the coding of palette indices uses the maximum value for TBC equal to palette table size if there is no escape color, or that the coding of palette indices uses the maximum value for TBC equal to the palette table size plus 1 if there is an escape color. The syntax element use_index_max_for_palette_indices having a value of 0 may indicate that the coding of palette indices uses the maximum value for TBC equal to palette table size minus 1 if there is no escape color, or that the coding of palette indices uses the maximum value for TBC equal to the palette table size if there is escape color.
- The syntax element use_index_max_for_palette__indices may be used as a CU level flag, fbr example, as illustrated in Table 14. If the syntax element use_index_max_for_palette_indices has the
value 1, then the number of leading palette indices may be encoded using the maximum value for TBC equal to palette table size if there is no escape color, or the number of leading palette indices may be encoded using the maximum value for TBC equal to the palette table size plus 1 if there is escape color. A remaining number of palette indices may be encoded using the maximum value for TBC equal to palette table size minus 1 if there is no escape color, or equal to the palette table size if there is an escape color. The number of leading palette indices may be sent to the decoder as the value palette_num_leading_indices, The number of remaining palette indices may be sent to the decoder as the value palette_num_remaining_indices, The value of palette_num_leading_indices may be sent conditionally, for example, the value of palette_num_leading_indices may be sent in the bitstream. only in the case where use_index_max_for_palette_indices has thevalue 1. - If the syntax element use_index_max_for_palette_indices has the value of 0, then the value of palette_num_leading_indices may not be present in the bitstream, and may be inferred to have a value of zero. In this case, one or more (e.g., all) of palette indices may be encoded using the maximum value for TBC equal to palette table size minus 1 if there is no escape color, or one or more (e.g., all) of palette indices may be encoded using the maximum value for TBC equal to the palette table size if there is escape color. The decoder may use these syntax elements to determine the maximum value for TBC for one or more (e.g., each) palette index, and may decode the palette indices accordingly.
-
TABLE 14 Example of palette syntax Descriptor palette_coding( x0, y0, nCbS ) { ... use_index_max_for_palette_indices ae(v) if( indexMax > 0) { palette_transpose_flag ae(v) if (use_index_max_for_palette_indices) palette_num_leading_indices ae(v) palette_num_remaining_indices ae(v) for( i = 0; i < palette_num_leading_indices + palette_num_remaining_indices; i++ ) palette_index_idc last_palette_run_type_flag ae(v) indicesLeft = palette_num_indices ae(v) } scanPos = 0 while( scanPos < nCbS * nCbS ) { ... if( indexMax > 0 && scanPos > = nCbS && palette_run_type_flag[ xcPrev ][ ycPrev ] ! = COPY_ABOVE_MODE && indicesLeft && scanPos < nCbS * nCbS − 1) { palette_run_type_flag[ xC ] [ yC ] ae(v) } indicesLeft −= 1 if( indexMax > 0 && (indicesLeft | | palette_run_type_flag != last_palette_run_type_flag)) { maxPaletteRun = nCbS * nCbS − scanPos − 1 ... } } - The syntax element use_index_max_for_palette_indices may indicate whether a nonzero value of palette_num_leading_indices is signaled in the bitstream. The syntax element palette_num_leading_indices may indicate the number of leading palette indices which are coded with a maximum value for TBC coding set to the palette table size if there is no escape color coding in the current CU, or set to the palette table size plus 1 if there is escape color coding in the current CU. If not present, the value of palette_num_leading_indices may be inferred to be zero. The syntax element palette_num_remaining_indices may indicate the number of palette indices which are coded with a maximum value for TBC coding set to the palette table size minus 1 if there is no escape color coding in the current CU, or set to the palette table size if there is escape color coding in the current CU.
- A palette syntax may be provided where use_index_max_for_palette_indices is not signaled and/or palette_num_leading_indices is present e.g., always present), for example, as illustrated in Table 15.
-
TABLE 15 Example of palette syntax Descriptor palette_coding( x0, y0, nCbS ) { ... if( indexMax > 0) { palette_transpose_flag ae(v) palette_num_leading_indices ae(v) palette_num_remaining_indices ae(v) for( i = 0; i < palette_num_leading_indices + palette_num_remaining_indices; i++ ) palette_index_idc last_palette_run_type_flag ae(v) indicesLeft = palette_num_indices ae(v) } scanPos = 0 while( scanPos < nCbS * nCbS ) { ... if( indexMax > 0 && scanPos > = nCbS && palette_run_type_flag[ xcPrev ][ ycPrev ] ! = COPY_ABOVE_MODE && indicesLeft && scanPos < nCbS * nCbS − 1) { palette_run_type_flag[ xC ][ yC ] ae(v) } indicesLeft −= 1 if( indexMax > 0 && (indicesLeft | | palette_run_type_flag != last_palette_run_type_flag)) { maxPaletteRun = nCbS * nCbS − scanPos − 1 ... } } - The signaling of palette_index_refinement_flag may be moved from the parsing loop of palette mode and palette run to the parsing loop of palette indices, The maximum TBC level may be reduced by one (e.g., pMax−1) before parsing the palette indices of one palette-coded CU. The palette_index_refinement_flag may be signaled (e.g., always signaled) when the parsed palette index of one pixel is equal to
pMax− 1. Such syntax may comprise a higher parsing throughput, for example, since the bypass bins of palette index coding may be separated from the context-coded bins of palette modes and palette runs. An example of this syntax is provided in Table 16. -
TABLE 16 Example of palette syntax Descriptor palette_coding( x0, y0, nCbS ) { palettePredictionFinished = 0 paletteNumPredictedEntries = 0 for( i = 0; i < PredictorPaletteSize && !palettePredictionFinished && paletteNumPredictedEntries < palette_max_size; i++ ) { palette_predictor_run ae(v) if( palette_predictor_run != 1 ) if( palette_predictor_run > 1 ) i += palette_predictor_run − 1 PalettePredictorEntryReuseFlag[ i ] = 1 paletteNumPredictedEntries ++ } else palettePredictionFinished = 1 } if( paletteNumPredictedEntries < palette_max_size ) palette_num_signalled_entries ae(v) for( cIdx = 0; cIdx < 3; cIdx++ ) for( i = 0; i < palette_num_signalled_ entries; i++ ) palette_entry ae(v) } if( currentPaletteSize != 0) palette_escape_val_present_flag ae(v) if( palette_escape_val_present_flag ) { if( cu_qp_delta_enabled_flag && !IsCuQpDeltaCoded ) { cu_qp_delta_palette_abs ae(v) if( cu_qp_delta_palette_abs ) cu_qp_delta_palette_sign_flag ae(v) } if( cu_chroma_qp_offset_enabled_flag && !IsCuChromaQpOffsetCoded ) { cu_chroma_qp_palette_offset_flag ae(v) if( cu_chroma_qp_offset_flag && chroma_qp_offset_list_len_minus1 > 0 ) cu_chroma_qp_palette_offset_idx ae(v) } } if( indexMax > 0 ) { palette_transpose_flag ae(v) palette_num_indices ae(v) for( i = 0; i < palette_num_indices; i++ ) { palette_index_idc ae(v) if (palette_index_idc = = (indexMax − 1) ) palette_index_refinement_flag ae(v) } last_palette_run_type_flag ae(v) NumIndicesLeft = palette_num_indices } scanPos = 0 while( scanPos < nCbS * nCbS ) { xC = x0 + travScan[ scanPos ][ 0 ] yC = y0 + travScan[ scanPos ][ 1 ] if( scanPos > 0) { xcPrev = x0 + travScan[ scanPos − 1 ][ 0 ] ycPrev = y0 + travScan[ scanPos − 1 ][ 1 ] } if( indexMax > 0 && (scanPos = = 0 | | ( palette_run_type_flag[ xcPrev ][ ycPrev ] ! = COPY_ABOVE_MODE && NumIndicesLeft && scanPos < nCbS * nCbS − 1 ) ) ) { palette_run_type_flag[ xC ][ yC ] ae(v) } if( palette_run_type_flag[ xC ][ yC ] = = COPY_INDEX_MODE && indexMax > 1) { NumIndicesLeft −= 1 } if( indexMax > 0 && (NumIndicesLeft | | palette_run_type_flag != last_palette_run_type_flag) ) { maxPaletteRun = nCbS * nCbS − scanPos − 1 if( maxPaletteRun > 0 ) { palette_run_msb_id_plus1 ae(v) if( palette_run_msb_id_plus1 > 1 ) palette_run_refinement_bits ae(v) } } else paletteRun = nCbS * nCbS − 1 − scanPos runPos = 0 while ( runPos < = paletteRun ) { xR = x0 + travScan[ scanPos ][ 0 ] yR = y0 + travScan[ scanPos ][ 1 ] if( palette_run_type_flag[ xC ][ yC ] = = COPY_INDEX_MODE && paletteIndex = = indexMax ) { PaletteSampleMode[ xR ][ yR ] = ESCAPE_MODE PaletteIndexMap[ xR ][ yR ] = paletteIndex for( cIdx = 0; cIdx < 3; cIdx++ ) { palette_escape_val ae(v) PaletteEscapeVal[ cIdx ][ xR ][ yR ] = palette_escape_val } } else if(palette_run_type_flag[ xC ][ yC ] = = COPY_INDEX_MODE ) { PaletteSampleMode[ xR ][ yR ] = COPY_INDEX_MODE PaletteIndexMap[ xR ][ yR ] = paletteIndex } else { PaletteSampleMode[ xR ][ yR ] = COPY_ABOVE_MODE PaletteIndexMap[ xR ][ yR ] = (yR − 1) < y0 ? (bNeighbour ? NCP_INDEX :0) :PaletteIndexMap[ xR ][ yR − 1 ] } runPos++ scanPos++ } } } -
FIG. 14A is a diagram of anexample communications system 100 in which one or more disclosed embodiments may be implemented. Thecommunications system 100 may be a multiple access system that provides content, such as voice, data, video, messaging, broadcast, etc., to multiple wireless users. Thecommunications system 100 may enable multiple wireless users to access such content through the sharing of system resources, including wireless bandwidth. For example, thecommunications systems 100 may employ one or more channel access methods, such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), single-carrier FDMA (SC-FDMA), and the like. - As shown in
FIG. 14A , thecommunications system 100 may include wireless transmit/receive units (WTRUs) 102 a, 102 b, 102 c, and/or 102 d (which generally or collectively may be referred to as WTRU 102), a radio access network (RAN) 103/104/105, acore network 106/107/109, a public switched telephone network (PSTN) 108, theInternet 110, andother networks 112, though it will be appreciated that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and/or network elements. Each of theWTRUs WTRUs - The
communications systems 100 may also include abase station 114 a and abase station 114 b. Each of thebase stations WTRUs core network 106/107/109, theInternet 110, and/or thenetworks 112. By way of example, thebase stations base stations base stations - The
base station 114 a may be part of theRAN 103/104/105, which may also include other base stations and/or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), relay nodes, etc. Thebase station 114 a and/or thebase station 114 b may be configured to transmit and/or receive wireless signals within a particular geographic region, which may be referred to as a cell (not shown). The cell may further be divided into cell sectors. For example, the cell associated with thebase station 114 a may be divided into three sectors. Thus, in one embodiment, thebase station 114 a may include three transceivers, e.g., one for each sector of the cell. In another embodiment, thebase station 114 a may employ multiple-input multiple output (MIMO) technology and, therefore, may utilize multiple transceivers for each sector of the cell. - The
base stations WTRUs air interface 115/116/117, which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, infrared (IR), ultraviolet (UV), visible light, etc.). Theair interface 115/116/117 may be established using any suitable radio access technology (RAT). - More specifically, as noted above, the
communications system 100 may be a multiple access system and may employ one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, and the like. For example, thebase station 114 a in theRAN 103/104/105 and theWTRUs air interface 115/116/117 using wideband CDMA (WCDMA). WCDMA may include communication protocols such as High-Speed Packet Access (HSPA) and/or Evolved HSPA (HSPA+). HSPA may include High-Speed Downlink Packet Access (HSDPA) and/or High-Speed Uplink Packet Access (HSUPA). - In another embodiment, the
base station 114 a and theWTRUs air interface 115/116/117 using Lona Term Evolution (LTE) and/or LTE-Advanced (LTE-A). - In other embodiments, the
base station 114 a and theWTRUs CDMA2000 1×, CDMA2000 EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile communications (GSM), Enhanced Data rates for GSM Evolution (EDGE), GSM EDGE (GERAN), and the like. - The
base station 114 b inFIG. 14A may be a wireless router, Home Node B. Home eNode B. or access point, for example, and may utilize any suitable RAT for facilitating wireless connectivity in a localized area, such as a place of business, a home, a vehicle, a campus, and the like. In one embodiment, thebase station 114 b and theWTRUs base station 114 b and theWTRUs base station 114 b and theWTRUs FIG. 14A , thebase station 114 b may have a direct connection to theInternet 110. Thus, thebase station 114 b may not be required to access theInternet 110 via thecore network 106/107/109. - The
RAN 103/104/105 may be in communication with thecore network 106/107/109, which may be any type of network configured to provide voice, data, applications, and/or voice over internet protocol (VoIP) services to one or more of theWTRUs core network 106/107/109 may provide call control, billing services, mobile location-based services, pre-paid calling, Internet connectivity, video distribution, etc., and/or perform high-level security functions, such as user authentication. Although not shown inFIG. 14A , it will be appreciated that theRAN 103/104/105 and/or thecore network 106/107/109 may be in direct or indirect communication with other RANs that employ the same RAT as theRAN 103/104/105 or a different RAT. For example, in addition to being connected to theRAN 103/104/105, which may be utilizing an E-UTRA radio technology, thecore network 106/107/109 may also be in communication with another RAN (not shown) employing a GSM radio technology. - The
core network 106/107/109 may also serve as a gateway for theWTRUs PSTN 108, theInternet 110, and/orother networks 112. ThePSTN 108 may include circuit-switched telephone networks that provide plain old telephone service (POTS). TheInternet 110 may include a global system of interconnected computer networks and devices that use common communication protocols, such as the transmission control protocol (TCP), user datagram protocol (UDP) and the internet protocol (IP) in the TCP/IP internet protocol suite. Thenetworks 112 may include wired or wireless communications networks owned and/or operated by other service providers. For example, thenetworks 112 may include another core network connected to one or more RANs, which may employ the same RAT as theRAN 103/104/105 or a different RAT. - Some or all of the
WTRUs communications system 100 may include multi-mode capabilities, e.g., theWTRUs WTRU 102 c shown inFIG. 14A may be configured to communicate with thebase station 114 a, which may employ a cellular-based radio technology, and with thebase station 114 b, which may employ an IEEE 802 radio technology. -
FIG. 14B is a system diagram of anexample WTRU 102. As shown inFIG. 14B , theWTRU 102 may include aprocessor 118, atransceiver 120, a transmit/receiveelement 122, a speaker/microphone 124, akeypad 126, a display/touchpad 128,non-removable memory 130,removable memory 132, apower source 134, a global positioning system (GPS)chipset 136, andother peripherals 138. It will be appreciated that theWTRU 102 may include any sub-combination of the foregoing elements while remaining consistent with an embodiment. Also, embodiments contemplate that thebase stations base stations FIG. 14B and described. herein. - The
processor 118 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Array (FPGAs) circuits, any other type of integrated circuit (IC), a state machine, and the like. Theprocessor 118 may perform signal coding, data processing, power control, input/output processing, and/or any other functionality that enables theWTRU 102 to operate in a wireless environment. Theprocessor 118 may be coupled to thetransceiver 120, which may be coupled to the transmit/receiveelement 122. WhileFIG. 14B depicts theprocessor 118 and thetransceiver 120 as separate components, it will be appreciated that theprocessor 118 and thetransceiver 120 may be integrated together in an electronic package or chip. - The transmit/receive
element 122 may be configured to transmit signals to, or receive signals from, a base station (e.g., thebase station 114 a) over theair interface 115/116/117. For example, in one embodiment, the transmit/receiveelement 122 may be an antenna configured to transmit and/or receive RF signals. In another embodiment, the transmit/receiveelement 122 may be an emitter/detector configured to transmit and/or receive IR, UV, or visible light signals, for example. In yet another embodiment, the transmit/receiveelement 122 may be configured to transmit and receive both RF and light signals. It will be appreciated that the transmit/receiveelement 122 may be configured to transmit and/or receive any combination of wireless signals. - In addition, although the transmit/receive
element 122 is depicted inFIG. 14B as a single element, theWTRU 102 may include any number of transmit/receiveelements 122. More specifically, theWTRU 102 may employ MIMO technology. Thus, in one embodiment, theWTRU 102 may include two or more transmit/receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over theair interface 115/116/117. - The
transceiver 120 may be configured to modulate the signals that are to be transmitted by the transmit/receiveelement 122 and to demodulate the signals that are received by the transmit/receiveelement 122. As noted above, theWTRU 102 may have multi-mode capabilities. Thus, thetransceiver 120 may include multiple transceivers for enabling theWTRU 102 to communicate via multiple RATs, such as UTRA and IEEE 802.11, for example. - The
processor 118 of theWTRU 102 may be coupled to, and may receive user input data from, the speaker/microphone 124, thekeypad 126, and/or the display/touchpad 128 (e.g., a liquid crystal display (LCD) display unit or organic light-emitting diode (OLED) display unit). Theprocessor 118 may also output user data to the speaker/microphone 124, thekeypad 126, and/or the display/touchpad 128. In addition, theprocessor 118 may access information from, and store data in, any type of suitable memory, such as thenon-removable memory 130 and/or theremovable memory 132. Thenon-removable memory 130 may include random-access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. Theremovable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like. In other embodiments, theprocessor 118 may access information from, and store data in, memory that is not physically located on theWTRU 102, such as on a server or a home computer (not shown). - 101201 The
processor 118 may receive power from thepower source 134, and may be configured to distribute and/or control the power to the other components in theWTRU 102. Thepower source 134 may be any suitable device for powering theWTRU 102. For example, thepower source 134 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, and the like. - The
processor 118 may also be coupled to theGPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of theWTRU 102. In addition to, or in lieu of, the information from theGPS chipset 136, theWTRU 102 may receive location information over theair interface 115/116/117 from a base station (e.g.,base stations WTRU 102 may acquire location information by way of any suitable location-determination method while remaining consistent with an embodiment. - The
processor 118 may further be coupled toother peripherals 138, which may include one or more software and/or hardware modules that provide additional features, functionality and/or wired or wireless connectivity. For example, theperipherals 138 may include an accelerometer, an e-compass, a satellite transceiver, a digital camera (for photographs or video), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands free headset, a Bluetooth® module, a frequency modulated (FM) radio unit, a digital music player, a media player, a video game player module, an Internet browser, and the like. -
FIG. 14C is a system diagram of theRAN 103 and thecore network 106 according to an embodiment. As noted above, theRAN 103 may employ a UTRA radio technology to communicate with theWTRUs air interface 115. TheRAN 103 may also be in communication with thecore network 106. As shown inFIG. 14C , theRAN 103 may include Node-Bs WTRUs air interface 115. The Node-Bs RAN 103. TheRAN 103 may also includeRNCs RAN 103 may include any number of Node-Bs and RNCs while remaining consistent with an embodiment. - As shown in
FIG. 14C , the Node-Bs RNC 142 a. Additionally, the Node-B 140 c may be in communication with theRNC 142 b. The Node-Bs respective RNCs RNCs RNCs Bs RNCs - The
core network 106 shown inFIG. 14C may include a media gateway (MGW) 144, a mobile switching center (MSC) 146, a serving CPRS support node (SGSN) 148, and/or a gateway GPRS support node (GGSN) 150. While each of the foregoing elements are depicted as part of thecore network 106, it will be appreciated that any one of these elements may be owned and/or operated by an entity other than the core network operator. - The
RNC 142 a in theRAN 103 may be connected to theMSC 146 in thecore network 106 via an IuCS interface. TheMSC 146 may be connected to theMGW 144. TheMSC 146 and theMGW 144 may provide the WTRUs 102 a, 102 b, 102 c with access to circuit-switched networks, such as thePSTN 108, to facilitate communications between theWTRUs - The
RNC 142 a in theRAN 103 may also be connected to theSGSN 148 in thecore network 106 via an IuPS interface. TheSGSN 148 may be connected to theGGSN 150. TheSGSN 148 and theGGSN 150 may provide the WTRUs 102 a, 102 b, 102 c with access to packet-switched networks, such as theInternet 110, to facilitate communications between and theWTRUs - As noted above, the
core network 106 may also be connected to thenetworks 112, which may include other wired or wireless networks that are owned and/or operated by other service providers. -
FIG. 14D is a system diagram of theRAN 104 and thecore network 107 according to an embodiment, As noted above, theRAN 104 may employ an E-UTRA radio technology to communicate with theWTRUs air interface 116. TheRAN 104 may also be in communication with thecore network 107. - The
RAN 104 may include eNode-Bs RAN 104 may include any number of eNode-Bs while remaining consistent with an embodiment. The eNode-Bs WTRUs air interface 116. In one embodiment, the eNode-Bs B 160 a, for example, may use multiple antennas to transmit wireless signals to, and receive wireless signals from, theWTRU 102 a. - Each of the eNode-
Bs FIG. 14D , the eNode-Bs - The
core network 107 shown inFIG. 14D may include a mobility management gateway (MME) 162, a servinggateway 164, and a packet data network (PDN)gateway 166. While each of the foregoing elements are depicted as part of thecore network 107, it will be appreciated that any one of these elements may be owned and/or operated by an entity other than the core network operator. - The
MME 162 may be connected to each of the eNode-Bs RAN 104 via an S1 interface and may serve as a control node. For example, theMME 162 may be responsible for authenticating users of theWTRUs WTRUs MME 162 may also provide a control plane function for switching between theRAN 104 and other RANs (not shown) that employ other radio technologies, such as GSM or WCDMA. - The serving
gateway 164 may be connected to each of the eNode-Bs RAN 104 via the S1 interface. The servinggateway 164 may generally route and forward user data packets to/from theWTRUs gateway 164 may also perform other functions, such as anchoring user planes during inter-eNode B handovers, triggering paging when downlink data is available for theWTRUs WTRUs - The serving
gateway 164 may also be connected to thePDN gateway 166, which may provide the WTRUs 102 a, 102 b, 102 c with access to packet-switched networks, such as theInternet 110, to facilitate communications between theWTRUs - The
core network 107 may facilitate communications with other networks. For example, thecore network 107 may provide the WTRUs 102 a, 102 b, 102 c with access to circuit-switched networks, such as thePSTN 108, to facilitate communications between theWTRUs core network 107 may include, or may communicate with, an IP gateway (e.g., an IP multimedia subsystem (IMS) server) that serves as an interface between thecore network 107 and thePSTN 108. In addition, thecore network 107 may provide the WTRUs 102 a, 102 b, 102 c with access to thenetworks 112, which may include other wired or wireless networks that are owned and/or operated by other service providers. -
FIG. 14E is a system diagram of theRAN 105 and thecore network 109 according to an embodiment. TheRAN 105 may be an access service network (ASN) that employs IEEE 802.16 radio technology to communicate with theWTRUs air interface 117. As will be further discussed below, the communication links between the different functional entities of theWTRUs RAN 105, and thecore network 109 may be defined as reference points. - As shown in
FIG. 14E , theRAN 105 may includebase stations ASN gateway 182, though it will be appreciated that theRAN 105 may include any number of base stations and ASN gateways while remaining consistent with an embodiment. Thebase stations RAN 105 and may each include one or more transceivers for communicating with theWTRUs air interface 117. In one embodiment, thebase stations base station 180 a, for example, may use multiple antennas to transmit wireless signals to, and receive wireless signals from. theWTRU 102 a. Thebase stations ASN gateway 182 may serve as a traffic aggregation point and may be responsible for paging, caching of subscriber profiles, routing to thecore network 109, and the like. - The
air interface 117 between theWTRUs RAN 105 may be defined as an R1 reference point that implements the IEEE 802.16 specification. In addition, each of theWTRUs core network 109. The logical interface between theWTRUs core network 109 may be defined as an R2 reference point, which may be used for authentication, authorization, IP host configuration management, and/or mobility management. - The communication link between each of the
base stations base stations ASN gateway 182 may be defined as an R6 reference point. The R6 reference point may include protocols for facilitating mobility management based on mobility events associated with each of theWTRUs - As shown in
FIG. 14E , theRAN 105 may be connected to thecore network 109. The communication link between theRAN 105 and thecore network 109 inay defined as an R3 reference point that includes protocols for facilitating data transfer and mobility management capabilities, for example, Thecore network 109 may include a mobile IP home agent (MIP-HA) 184, an authentication, authorization, accounting (AAA)server 186, and agateway 188. While each of the foregoing elements are depicted as part of thecore network 109, it will be appreciated that any one of these elements may be owned and/or operated by an entity other than the core network operator. - The MIP-HA may be responsible for IP address management, and may enable the WTRUs 102 a, 102 b, 102 c to roam between different ASNs and/or different core networks. The MIP-
HA 184 may provide the WTRUs 102 a, 102 b, 102 c with access to packet-switched networks, such as theInternet 110, to facilitate communications between theWTRUs AAA server 186 may be responsible fbr user authentication and for supporting user services. Thegateway 188 may facilitate interworking with other networks. For example, thegateway 188 may provide the WTRUs 102 a, 102 b, 102 c with access to circuit-switched networks, such as thePSTN 108, to facilitate communications between theWTRUs gateway 188 may provide the WTRUs 102 a, 102 b, 102 c with access to thenetworks 112, which may include other wired or wireless networks that are owned and/or operated by other service providers. - Although not shown in
FIG. 14E , it will be appreciated that theRAN 105 may be connected to other ASNs and thecore network 109 may be connected to other core networks. The communication link between theRAN 105 the other ASNs may be defined as an R4 reference point, which may include protocols for coordinating the mobility of theWTRUs RAN 105 and the other ASNs. The communication link between thecore network 109 and the other core networks may be defined as an R5 reference, which may include protocols for facilitating interworking between home core networks and visited core networks. - Although features and elements are described above in particular combinations, one of ordinary skill in the art will appreciate that each feature or element can be used alone or in any combination with the other features and elements. In addition, the methods described herein may be implemented in a computer program, software, or firmware incorporated in a computer-readable medium for execution by a computer or processor. Examples of computer-readable media include electronic signals (transmitted over wired or wireless connections) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, a read only memory (ROM), a random access memory (RAM), a register, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks, and digital versatile disks (DVDs). A processor in association with software may be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.
Claims (21)
1-20. (canceled)
21. A method for video decoding, the method comprising:
receiving a scan direction indication for a coding unit (CU) in video data, the scan direction indication indicating a palette coding scan direction associated with the CU, wherein the scan direction indication is coded with a single context; and
decoding the CU based on the first scan direction indication.
22. The method of claim 21 , wherein a context value for decoding the scan direction indication for the CU is independent of a palette coding scan direction associated with another CU.
23. The method of claim 21 , wherein the CU is a first CU, and a context value for decoding the scan direction indication for the first CU is independent of a palette coding scan direction associated with a second CU neighboring the first CU.
24. The method of claim 21 , wherein the CU is a first CU, and the scan direction indication for the first CU is coded with a context that does not depend on a palette coding scan direction associated with a second CU neighboring the first CU.
25. The method of claim 21 , wherein the scan direction indication for the CU is a palette transpose flag configured to indicate whether a horizontal traverse scan or a vertical traverse scan is applied for scanning palette indices of samples in the CU.
26. The device for video decoding, comprising:
a processor configured to:
receive a scan direction indication for a coding unit (CU) in video data, the scan direction indication indicating a palette coding scan direction associated with the CU, wherein the scan direction indication is coded with a single context; and
decode the CU based on the scan direction indication.
27. The device of claim 26 , wherein a context value for decoding the scan direction indication for the CU is independent of a palette coding scan direction associated with another CU.
28. The device of claim 26 , wherein the CU is a first CU, and a context value for decoding the scan direction indication for the first CU is independent of a palette coding scan direction associated with a second CU neighboring the first CU.
29. The device of claim 26 , wherein the CU is a first CU, and the scan direction indication for the first CU is coded with a context that does not depend on a palette coding scan direction associated with a second CU neighboring the first CU.
30. The device of claim 26 , wherein the scan direction indication for the CU is a palette transpose flag configured to indicate whether a horizontal traverse scan or a vertical traverse scan is applied for scanning palette indices of samples in the CU.
31. A method for video encoding, the method comprising:
determining a palette coding scan direction for a coding unit (CU); and
encoding a scan direction indication configured to indicate the determined palette coding scan direction associated with the CU, wherein the scan direction indication is coded with a single context.
32. The method of claim 31 , wherein a context value for encoding the scan direction indication for the CU is independent of a palette coding scan direction associated with another CU.
33. The method of claim 31 , wherein the CU is a first CU, and a context value for encoding the scan direction indication for the first CU is independent of a palette coding scan direction associated with a second CU neighboring the first CU.
34. The method of claim 31 , wherein the CU is a first CU, and the scan direction indication for the first CU is coded with a context that does not depend on a palette coding scan direction associated with a second CU that neighbors the first CU.
35. The method of claim 31 , wherein the scan direction indication for the CU is a palette transpose flag configured to indicate whether a horizontal traverse scan or a vertical traverse scan is applied for scanning palette indices of samples in the CU.
36. The device for video encoding, comprising:
a processor configured to:
determine a palette coding scan direction for a coding unit (CU); and
encode a scan direction indication configured to indicate the determined palette coding scan direction associated with the CU, wherein the scan direction indication is coded with a single context.
37. The device of claim 36 , wherein a context value for encoding the scan direction indication for the CU is independent of a palette coding scan direction associated with another CU.
38. The device of claim 36 , wherein the CU is a first CU, and a context value for encoding the scan direction indication for the first CU is independent of a palette coding scan direction associated with a second CU neighboring the first CU.
39. The device of claim 36 , wherein the CU is a first CU, and the scan direction indication for the first CU is coded with a context that does not depend on a palette coding scan direction associated with a second CU that neighbors the first CU.
40. The device of claim 36 , wherein the scan direction indication for the CU is a palette transpose flag configured to indicate whether a horizontal traverse scan or a vertical traverse scan is applied for scanning palette indices of samples in the CU.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US18/086,740 US20230132221A1 (en) | 2014-10-06 | 2022-12-22 | Palette coding for screen content coding |
Applications Claiming Priority (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201462060536P | 2014-10-06 | 2014-10-06 | |
US201562120293P | 2015-02-24 | 2015-02-24 | |
US14/875,735 US20160100179A1 (en) | 2014-10-06 | 2015-10-06 | Palette coding for screen content coding |
US16/258,829 US11558627B2 (en) | 2014-10-06 | 2019-01-28 | Palette coding for screen content coding |
US18/086,740 US20230132221A1 (en) | 2014-10-06 | 2022-12-22 | Palette coding for screen content coding |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/258,829 Continuation US11558627B2 (en) | 2014-10-06 | 2019-01-28 | Palette coding for screen content coding |
Publications (1)
Publication Number | Publication Date |
---|---|
US20230132221A1 true US20230132221A1 (en) | 2023-04-27 |
Family
ID=54360539
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/875,735 Abandoned US20160100179A1 (en) | 2014-10-06 | 2015-10-06 | Palette coding for screen content coding |
US16/258,829 Active 2037-04-11 US11558627B2 (en) | 2014-10-06 | 2019-01-28 | Palette coding for screen content coding |
US18/086,740 Abandoned US20230132221A1 (en) | 2014-10-06 | 2022-12-22 | Palette coding for screen content coding |
Family Applications Before (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/875,735 Abandoned US20160100179A1 (en) | 2014-10-06 | 2015-10-06 | Palette coding for screen content coding |
US16/258,829 Active 2037-04-11 US11558627B2 (en) | 2014-10-06 | 2019-01-28 | Palette coding for screen content coding |
Country Status (9)
Country | Link |
---|---|
US (3) | US20160100179A1 (en) |
EP (2) | EP3723378A1 (en) |
JP (4) | JP2017535169A (en) |
KR (4) | KR20220000419A (en) |
CN (2) | CN110677656A (en) |
AU (3) | AU2015328326B2 (en) |
MX (2) | MX2017004467A (en) |
TW (2) | TW201626798A (en) |
WO (1) | WO2016057444A2 (en) |
Families Citing this family (46)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9924175B2 (en) | 2014-06-11 | 2018-03-20 | Qualcomm Incorporated | Determining application of deblocking filtering to palette coded blocks in video coding |
US10136141B2 (en) * | 2014-06-11 | 2018-11-20 | Qualcomm Incorporated | Determining quantization parameter (QP) values and delta QP values for palette coded blocks in video coding |
CN107079151B (en) * | 2014-09-26 | 2020-06-05 | 株式会社Kt | Method and apparatus for processing video signal |
JP2017535169A (en) | 2014-10-06 | 2017-11-24 | ヴィド スケール インコーポレイテッド | Improved palette coding for screen content coding |
US10477218B2 (en) * | 2014-10-20 | 2019-11-12 | Kt Corporation | Method and apparatus for predicting and restoring a video signal using palette entry |
US9961351B2 (en) * | 2014-11-26 | 2018-05-01 | Qualcomm Incorporated | Palette mode coding |
CN105704491B (en) * | 2014-11-28 | 2020-03-13 | 同济大学 | Image encoding method, decoding method, encoding device, and decoding device |
CN111800632B (en) * | 2015-01-15 | 2024-03-29 | 株式会社Kt | Method for decoding encoded video signal and method for encoding video signal |
CN112135143B (en) * | 2015-01-29 | 2022-08-30 | 株式会社Kt | Method for decoding encoded video signal and method for encoding video signal |
US10477243B2 (en) * | 2015-01-29 | 2019-11-12 | Kt Corporation | Method and apparatus for predicting and restoring a video signal using palette entry and palette mode |
US10057587B2 (en) * | 2015-01-31 | 2018-08-21 | Qualcomm Incorporated | Coding escape pixels for palette mode coding |
KR102691340B1 (en) * | 2015-04-02 | 2024-08-02 | 주식회사 케이티 | Method and apparatus for processing a video signal |
US10448058B2 (en) * | 2015-05-21 | 2019-10-15 | Qualcomm Incorporated | Grouping palette index at the end and index coding using palette size and run value |
KR20160140410A (en) * | 2015-05-27 | 2016-12-07 | 주식회사 케이티 | Method and apparatus for processing a video signal |
CN107637057A (en) * | 2015-06-03 | 2018-01-26 | 联发科技股份有限公司 | The palette decoding method of image and video data |
US10097842B2 (en) | 2015-09-18 | 2018-10-09 | Qualcomm Incorporated | Restriction of escape pixel signaled values in palette mode video coding |
JP6812149B2 (en) * | 2016-06-30 | 2021-01-13 | オリンパス株式会社 | Scanning microscope and control method of scanning microscope |
CN107454425B (en) * | 2017-08-26 | 2019-10-18 | 同济大学 | A kind of SCC intraframe coding unit candidate modes reduction method |
US10916041B2 (en) * | 2018-03-30 | 2021-02-09 | Samsung Electronics Co., Ltd. | Method for depth image di coding |
CN108986180B (en) * | 2018-06-07 | 2022-09-16 | 创新先进技术有限公司 | Palette generation method and device and electronic equipment |
US20210274182A1 (en) * | 2018-07-02 | 2021-09-02 | Interdigital Vc Holdings, Inc. | Context-based binary arithmetic encoding and decoding |
US10848787B2 (en) * | 2018-08-28 | 2020-11-24 | Google Llc | Lossy image compression using palettization of locally mixed colors |
US10951895B2 (en) * | 2018-12-31 | 2021-03-16 | Alibaba Group Holding Limited | Context model selection based on coding unit characteristics |
JP7197720B2 (en) | 2019-02-24 | 2022-12-27 | 北京字節跳動網絡技術有限公司 | Independent encoding of instructions for use of palette mode |
JP7277608B2 (en) | 2019-04-23 | 2023-05-19 | 北京字節跳動網絡技術有限公司 | Context modeling and selection of multiple transformation matrices |
WO2020243295A1 (en) | 2019-05-31 | 2020-12-03 | Bytedance Inc. | Palette mode with intra block copy prediction |
CN113966612A (en) | 2019-06-16 | 2022-01-21 | 北京字节跳动网络技术有限公司 | Screen content codec based on interaction with motion information |
CN117714698A (en) | 2019-06-28 | 2024-03-15 | 字节跳动有限公司 | Chroma intra mode derivation in screen content coding |
KR20220032520A (en) | 2019-07-20 | 2022-03-15 | 베이징 바이트댄스 네트워크 테크놀로지 컴퍼니, 리미티드 | Condition-dependent coding of instructions for using palette mode |
CN114145013B (en) | 2019-07-23 | 2023-11-14 | 北京字节跳动网络技术有限公司 | Mode determination for palette mode coding and decoding |
WO2021018166A1 (en) * | 2019-07-29 | 2021-02-04 | Beijing Bytedance Network Technology Co., Ltd. | Scanning order improvements for palette mode coding |
WO2021023261A1 (en) | 2019-08-06 | 2021-02-11 | Beijing Bytedance Network Technology Co., Ltd. | Size restriction based on color format |
JP7494289B2 (en) | 2019-08-15 | 2024-06-03 | バイトダンス インコーポレイテッド | Palette modes with different partition structures |
WO2021030788A1 (en) | 2019-08-15 | 2021-02-18 | Bytedance Inc. | Entropy coding for palette escape symbol |
WO2021040459A1 (en) * | 2019-08-30 | 2021-03-04 | 주식회사 케이티 | Video signal processing method and device |
JP7381722B2 (en) | 2019-09-02 | 2023-11-15 | 北京字節跳動網絡技術有限公司 | Coding mode determination based on color format |
CN114375581A (en) * | 2019-09-12 | 2022-04-19 | 字节跳动有限公司 | Use of palette predictor in video coding |
CN114424545B (en) | 2019-09-19 | 2024-07-16 | 字节跳动有限公司 | Quantization parameter derivation for palette modes |
CN118055248A (en) | 2019-09-21 | 2024-05-17 | 北京字节跳动网络技术有限公司 | Method, apparatus and computer readable recording medium for processing video data |
CN114208172A (en) * | 2019-09-24 | 2022-03-18 | 北京达佳互联信息技术有限公司 | Method and apparatus for performing rate-distortion analysis for palette mode |
US11076151B2 (en) | 2019-09-30 | 2021-07-27 | Ati Technologies Ulc | Hierarchical histogram calculation with application to palette table derivation |
US12010344B2 (en) | 2019-11-05 | 2024-06-11 | Lg Electronics Inc. | Image or video coding based on quantization-related information |
WO2021091214A1 (en) * | 2019-11-05 | 2021-05-14 | 엘지전자 주식회사 | Video decoding method and device for coding chroma quantization parameter offset-related information |
CN116134820A (en) * | 2020-08-04 | 2023-05-16 | 现代自动车株式会社 | Method and device for encoding and decoding video data based on patch |
US11877011B2 (en) | 2020-09-17 | 2024-01-16 | Lemon Inc. | Picture dimension indication in decoder configuration record |
CN113141509B (en) * | 2021-03-30 | 2022-06-28 | 中山大学 | Color list generation method |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2015176683A1 (en) * | 2014-05-23 | 2015-11-26 | Mediatek Inc. | Method and apparatus for palette table prediction and signaling |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7162080B2 (en) * | 2001-02-23 | 2007-01-09 | Zoran Corporation | Graphic image re-encoding and distribution system and method |
US7511718B2 (en) * | 2003-10-23 | 2009-03-31 | Microsoft Corporation | Media integration layer |
US8953690B2 (en) * | 2011-02-16 | 2015-02-10 | Google Technology Holdings LLC | Method and system for processing video data |
US11259020B2 (en) * | 2013-04-05 | 2022-02-22 | Qualcomm Incorporated | Determining palettes in palette-based video coding |
GB201321851D0 (en) * | 2013-12-10 | 2014-01-22 | Canon Kk | Run length parameters coding for palette mode |
WO2016044974A1 (en) * | 2014-09-22 | 2016-03-31 | Mediatek Singapore Pte. Ltd. | Palette table signalling |
CN106416257B (en) * | 2014-05-23 | 2019-05-10 | 寰发股份有限公司 | Palette size transmits the method transmitted with conditional palette exception mark |
US10687064B2 (en) | 2014-08-04 | 2020-06-16 | Qualcomm Incorporated | Palette mode encoding and decoding with inferred pixel scan order |
US9544607B2 (en) | 2014-08-25 | 2017-01-10 | Hfi Innovation Inc. | Method of palette index signaling for image and video coding |
JP2017535169A (en) | 2014-10-06 | 2017-11-24 | ヴィド スケール インコーポレイテッド | Improved palette coding for screen content coding |
-
2015
- 2015-10-06 JP JP2017518553A patent/JP2017535169A/en active Pending
- 2015-10-06 KR KR1020217042345A patent/KR20220000419A/en active IP Right Grant
- 2015-10-06 KR KR1020177012344A patent/KR20170065648A/en not_active Application Discontinuation
- 2015-10-06 AU AU2015328326A patent/AU2015328326B2/en active Active
- 2015-10-06 KR KR1020207012864A patent/KR20200051066A/en not_active Application Discontinuation
- 2015-10-06 TW TW104132822A patent/TW201626798A/en unknown
- 2015-10-06 US US14/875,735 patent/US20160100179A1/en not_active Abandoned
- 2015-10-06 EP EP20177998.0A patent/EP3723378A1/en not_active Withdrawn
- 2015-10-06 KR KR1020197010850A patent/KR102109154B1/en active IP Right Grant
- 2015-10-06 EP EP15787070.0A patent/EP3205107A2/en not_active Withdrawn
- 2015-10-06 MX MX2017004467A patent/MX2017004467A/en unknown
- 2015-10-06 TW TW108127789A patent/TWI786321B/en active
- 2015-10-06 CN CN201910625476.7A patent/CN110677656A/en active Pending
- 2015-10-06 CN CN201580053500.4A patent/CN106797469A/en active Pending
- 2015-10-06 WO PCT/US2015/054117 patent/WO2016057444A2/en active Application Filing
-
2017
- 2017-04-05 MX MX2021010428A patent/MX2021010428A/en unknown
-
2018
- 2018-08-31 AU AU2018223030A patent/AU2018223030C1/en active Active
-
2019
- 2019-01-23 JP JP2019009581A patent/JP6778770B2/en active Active
- 2019-01-28 US US16/258,829 patent/US11558627B2/en active Active
-
2020
- 2020-07-30 AU AU2020210245A patent/AU2020210245B2/en active Active
- 2020-10-12 JP JP2020171731A patent/JP2021005900A/en active Pending
-
2022
- 2022-02-28 JP JP2022030001A patent/JP2022079463A/en active Pending
- 2022-12-22 US US18/086,740 patent/US20230132221A1/en not_active Abandoned
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2015176683A1 (en) * | 2014-05-23 | 2015-11-26 | Mediatek Inc. | Method and apparatus for palette table prediction and signaling |
Also Published As
Publication number | Publication date |
---|---|
MX2017004467A (en) | 2017-12-14 |
AU2020210245A1 (en) | 2020-08-20 |
JP2019110539A (en) | 2019-07-04 |
KR20170065648A (en) | 2017-06-13 |
KR102109154B1 (en) | 2020-05-12 |
US11558627B2 (en) | 2023-01-17 |
TW201626798A (en) | 2016-07-16 |
KR20190042761A (en) | 2019-04-24 |
US20190158854A1 (en) | 2019-05-23 |
KR20200051066A (en) | 2020-05-12 |
KR20220000419A (en) | 2022-01-03 |
CN106797469A (en) | 2017-05-31 |
AU2018223030C1 (en) | 2020-11-19 |
TW202005377A (en) | 2020-01-16 |
AU2015328326A1 (en) | 2017-05-04 |
WO2016057444A3 (en) | 2016-05-26 |
AU2020210245B2 (en) | 2022-09-15 |
JP2017535169A (en) | 2017-11-24 |
AU2018223030A1 (en) | 2018-09-20 |
AU2015328326B2 (en) | 2018-05-31 |
JP2021005900A (en) | 2021-01-14 |
EP3205107A2 (en) | 2017-08-16 |
WO2016057444A2 (en) | 2016-04-14 |
JP2022079463A (en) | 2022-05-26 |
TWI786321B (en) | 2022-12-11 |
EP3723378A1 (en) | 2020-10-14 |
CN110677656A (en) | 2020-01-10 |
AU2018223030B2 (en) | 2020-05-28 |
US20160100179A1 (en) | 2016-04-07 |
JP6778770B2 (en) | 2020-11-04 |
MX2021010428A (en) | 2021-09-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20230132221A1 (en) | Palette coding for screen content coding | |
US10484686B2 (en) | Palette coding modes and palette flipping | |
TWI735424B (en) | Escape color coding for palette coding mode | |
KR102268090B1 (en) | Palette coding for screen content coding | |
US20180307457A1 (en) | Two-dimensional palette coding for screen content coding |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |
|
AS | Assignment |
Owner name: INTERDIGITAL VC HOLDINGS, INC., DELAWARE Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:VID SCALE, INC.;REEL/FRAME:068284/0031 Effective date: 20240627 |