RU2801430C1 - Method and device for coding mode based on palette under local dual tree structure - Google Patents

Abstract

FIELD: video encoding and decoding.

SUBSTANCE: video decoding apparatus is provided that includes a receiving circuit and a processing circuit that decodes block prediction information from an encoded video bitstream, determines whether to enable a palette-based coding mode for the block, based on comparing the block size with a threshold value, and, based on the prediction information, determines whether the block is under the local dual tree structure. In addition, the processing circuit decodes the block based on determining whether to enable the palette-based coding mode for the block and determining whether the block is under a local dual tree structure.

EFFECT: increased efficiency of image/video encoding.

41 cl, 12 dwg

Description

Inclusion by reference

[1] This application claims priority over US Patent Application No. 17/097 415 "METHOD AND APPARATUS FOR PALETTE-BASED CODING MODE UNDER LOCAL DUAL WOOD STRUCTURE", filed November 13, 2020, which claimed priority over US Provisional Application No. 62 /963,216, "SIMPLIED PALETTE ENCODING WITH A LOCAL DUAL-TREE ENCODING STRUCTURE" filed January 20, 2020. The entire disclosure of prior applications is hereby incorporated by reference in its entirety.

The technical field to which the invention belongs

[2] The present disclosure describes embodiments primarily related to video coding.

State of the art

[3] The description of the prior art provided herein is intended to provide a general overview of the context of the disclosure. The work of the inventors named here, to the extent that it is described in this section of the description of the prior art, and aspects of the description that otherwise cannot be considered prior art at the time of filing, are neither expressly nor impliedly recognized as prior art. techniques in relation to the present disclosure.

[4] Video encoding and decoding may be performed using motion-compensated inter-frame prediction. Uncompressed digital video may include a series of images, each image having a spatial size of, for example, 1920×1080 luma samples and associated chrominance samples. A series of images can have a fixed or variable image rate (informally also known as frame rate), such as 60 images per second or 60 Hz. Uncompressed video has certain bitrate requirements. For example, 1080p60 4:2:0 video at 8 bits per sample (resolution 1920x1080 luma samples at 60Hz frame rate) requires bandwidth close to 1.5Gbps. An hour of such video requires more than 600 GB of disk space.

[5] One of the goals of video encoding and decoding may be to reduce the redundancy of the input video signal through compression. Compression can help reduce the aforementioned bandwidth and/or memory space requirements, in some cases by two orders of magnitude or more. Both lossless and lossy compression, as well as a combination of both, can be used. Lossless compression refers to methods in which an exact copy of the original signal can be restored from a compressed source signal. When lossy compression is used, the reconstructed signal may not be identical to the original signal, but the distortion between the original and reconstructed signals is small enough to make the reconstructed signal useful for the intended application. In the case of video, lossy compression is widely used. The amount of distortion allowed depends on the application; for example, users of certain consumer streaming applications may tolerate higher distortion than users of television distribution applications. The achievable compression ratio may reflect the following: higher distortion resolution/tolerance may result in a higher compression ratio.

[6] The video encoder and video decoder may use techniques from several broad categories including, for example, motion compensation, transform, quantization, and entropy coding.

[7] Video codec technologies may include techniques known as intra-coding. In intra-coding, sample values are presented without reference to samples or other data from previously reconstructed reference pictures. In some video codecs, the image is spatially divided into blocks of samples. When all blocks of samples are encoded in intra mode, this picture may be an intra picture. Intra-pictures and their derivatives, such as independent decoder update pictures, can be used to reset the decoder state and therefore can be used as the first picture in the encoded video bitstream and video session, or as a still picture. The intra-block counts may be transformed and the transform coefficients may be quantized before entropy encoding. Intra-prediction may be a technique that minimizes sample values in a region prior to transformation. In some cases, the smaller the DC value after the transform and the smaller the AC coefficients, the fewer bits are required for a given quantization step size to represent a block after entropy coding.

[8] Conventional intra coding known from, for example, MPEG-2 generation coding technologies does not use intra prediction. However, some new video compression techniques include techniques that attempt, for example, from surrounding sample data and/or metadata obtained during encoding/decoding of spatially adjacent and preceding data blocks in decoding order. Such methods are hereinafter referred to as "intra-prediction" methods. Note that, in at least some cases, intra-prediction only uses reference data from the current image being reconstructed, and not from reference images.

[9] There can be many different forms of intra-prediction. When more than one of such methods can be used in a given video coding technology, the method used can be encoded in intra-prediction mode. In some cases, modes may have sub-modes and/or parameters, and these may be encoded individually or included in the mode codeword. The choice of which codeword to use for a given mode/submode/parameter combination can affect the improvement in coding efficiency due to intra-prediction, as can the entropy coding technology used to convert the codewords into a bit stream.

[10] A specific intra-prediction mode was introduced in H.264, enhanced in H.265, and further enhanced in new coding technologies such as Joint Exploration Model (JEM), Versatile Video Coding (VVC), and Test Suite (BMS). The predictor block can be formed using the values of neighboring samples belonging to already available samples. The sample values of neighboring samples are copied into the prediction block according to the direction. A reference to the direction to be used may be encoded in the bitstream, or may itself be predicted.

[11] FIG. 1, a subset of the nine prediction directions known from the 33 possible H.265 prediction directions (corresponding to 33 angle modes out of 35 intra modes) is depicted in the lower right corner. The point where the arrows (101) converge is the predicted reading. The arrows indicate the direction in which the reading is predicted. For example, an arrow (102) indicates that the reading (101) is predicted based on the reading or readings in the upper right corner at 45 degrees from the horizontal. Similarly, the arrow (103) indicates that the reading (101) is predicted from the reading or readings in the lower left corner of the reading (101) at an angle of 22.5 degrees from the horizontal.

[12] In the upper left corner of FIG. 1 shows a square block (104) of 4x4 samples (denoted by a dotted thick line). The square block (104) includes 16 samples, each labeled with an "S", its position in the Y dimension (eg, row index) and its position in the X dimension (eg, column index). For example, sample S21 is the second sample in the Y dimension (top) and the first (left) sample in the X dimension. Similarly, sample S44 is the fourth sample in block (104) in both the Y and X dimensions. Since the block size is 4x4 samples, S44 is at the bottom right. The following shows reference readings that follow a similar numbering scheme. The reference sample is labeled with the letter R, its Y position (eg row index) and X position (column index) relative to the block (104). In both H.264 and H.265, prediction samples are adjacent to the block being reconstructed; so there is no need to use negative values.

[13] Image intra-prediction can work by copying reference sample values from adjacent samples according to the prediction signal direction. For example, suppose the encoded video bitstream includes signaling that for this block indicates a prediction direction corresponding to arrow (102), i.e., samples are predicted from the prediction sample or samples in the upper right corner at 45 degrees from the horizontal. In this case, samples S41, S32, S23 and S14 are predicted based on the same reference sample R05. Sample S44 is then predicted based on reference sample R08.

[14] In some cases, the values of several reference samples may be combined, for example, by interpolation, to calculate the reference sample; especially when the directions are not divisible by 45 degrees without a remainder.

[15] The number of possible destinations has increased as video coding technology has evolved. H.264 (2003) can present nine different directions. This number has increased to 33 in H.265 (2013) and JEM/VVC/BMS can support up to 65 destinations at the time of this disclosure. Experiments have been made to determine the most likely directions, and certain entropy encoding techniques are used to represent these likely directions in a small number of bits, while taking a certain penalty for the less likely directions. In addition, the directions themselves can sometimes be predicted from adjacent directions used in adjacent, already decoded blocks.

[16] FIG. 2 illustrates a diagram (201) that depicts 65 intra-prediction directions according to JEM to illustrate an increasing number of prediction directions over time.

[17] The mapping of intra-prediction direction bits in the encoded video bitstream that represent the direction may differ from video coding technology to video coding technology; and may vary, for example, from simple direct prediction direction mappings to intra-prediction mode, to codewords, to complex adaptive circuits including most likely modes, and similar techniques. However, in all cases, there may be certain directions that are statistically less likely in video content than some other directions. Because the goal of video compression is to reduce redundancy, these less likely directions in a well-performing video coding technology will be represented by more bits than the more likely directions.

Disclosure of the essence of the invention

[18] Aspects of the disclosure relate to methods and apparatus for encoding/decoding video. In some examples, a video decoding apparatus includes a receiving circuit and a processing circuit. For example, the processing circuit decodes block prediction information from the encoded video bitstream, determines whether to enable the paletted coding mode for the block based on comparing the block size with a threshold value, and determines whether the block is under the local dual tree structure based on the prediction information. . In addition, the processing circuit decodes the block based on determining whether to enable the palette-based coding mode for the block and determining whether the block is under a local dual tree structure.

[19] In some embodiments, the processing scheme disables the paletted coding mode for a block in response to the block size being less than a threshold. In some examples, the threshold value is defined to disable the local dual tree structure in response to a block exceeding the threshold value. In one example, the processing scheme disables the paletted coding mode for the block in response to the block being a chrominance block and the chroma block size being less than a threshold. In some examples, the processing scheme disables the paletted coding mode for a block based on a combination of mode type and block tree type that indicates that the block is a chrominance block under a local dual tree structure.

[20] In some embodiments, in response to enabling the palette-based coding mode, the processing circuit decodes from the encoded video bitstream a flag indicating whether the block uses the palette-based coding mode, and decodes the block using the palette-based coding mode in response to a flag indicating the use of a palette-based coding mode in the block. In addition, in some examples, in response to the block being under the local dual tree structure, the processing scheme skips updating the list of palette predictors. In one example, the processing circuit updates, in response to the lack of triggering a local dual tree structure, a list of palette predictors based on the palette used in decoding the block.

[21] In some other embodiments, in response to enabling the palette-based coding mode, the processing circuit decodes from the encoded video bitstream a flag indicating whether the block uses the palette-based coding mode, and decodes the palette-based block having a reduced the maximum size, in response to a flag indicating the use of the palette-based encoding mode for the block and that the block is under the local dual tree structure. In addition, the processing circuit may update the palette predictor list based on the palette used in block decoding, and the size of the palette predictor list is limited to a reduced maximum size.

[22] Aspects of the disclosure also relate to a non-transitory computer-readable medium that stores instructions that, when executed by a video decoding computer, cause the computer to perform a video decoding method.

Brief description of the drawings

[23] Additional features, nature and various advantages of the disclosed subject matter will be better understood from the following detailed description and the accompanying drawings, which show the following:

[24] FIG. 1 is a schematic illustration of an exemplary subset of intra-prediction modes.

[25] FIG. 2 is an illustration of exemplary intra-prediction directions.

[26] FIG. 3 is a schematic illustration of a simplified block diagram of a communications system (300) in accordance with an embodiment;

[27] FIG. 4 is a schematic illustration of a simplified block diagram of a communications system (400) in accordance with an embodiment.

[28] FIG. 5 is a schematic illustration of a simplified block diagram of a decoder according to an embodiment.

[29] FIG. 6 is a schematic illustration of a simplified block diagram of an encoder according to an embodiment.

[30] FIG. 7 shows a block diagram of an encoder in accordance with another embodiment.

[31] FIG. 8 is a block diagram of a decoder according to another embodiment;

[32] FIG. 9 shows an example syntax for updating a list of palette predictions in some examples.

[33] FIG. 10 shows an example syntax for determining the existence of a palette mode flag in a bitstream in some examples.

[34] FIG. 11 is a flow diagram describing an example of a process according to some embodiments of the disclosure.

[35] FIG. 12 is a schematic illustration of a computer system in accordance with one embodiment. Implementation of the invention

[36] FIG. 3 illustrates a simplified block diagram of a communications system (300) according to an embodiment of the present disclosure. The communication system (300) includes a plurality of terminal devices that can communicate with each other, such as via a network (350). For example, the communication system (300) includes a first pair of terminals (410) and (3320) interconnected via a network (350). In the example of FIG. 3, the first pair of terminals (3410) and (320) performs a unidirectional data transfer. For example, a terminal device (310) may encode video data (eg, a video stream that is captured by a terminal device (310)) for transmission to another terminal device (320) over a network (350). The encoded video data may be transmitted in the form of one or more encoded video bitstreams. The terminal device (320) may receive the encoded video data from the network (350), decode the encoded video data to recover the video images, and display the video images in accordance with the recovered video data. One-way communication may be common in media serving applications and the like.

[37] In another example, the communication system (300) includes a second pair of terminals (330) and (340) that perform bidirectional transmission of encoded video data, which may occur, for example, during a video conference. For bidirectional data transfer, in the example, each terminal of terminals (330) and (340) can encode video data (for example, a video stream that is captured by the terminal) for transmission to another terminal of terminals (330) and (340). ) via the network (350). Each terminal of the terminals (330) and (340) may also receive the encoded video data transmitted by the other terminal of the terminals (330) and (340) and may decode the encoded video data to reconstruct the video images and may display the video images on an available display device. according to the recovered video data.

[38] In the example of FIG. 3 terminal devices (310), (320), (330) and (340) can be illustrated as servers, personal computers and smartphones, but the principles of the present disclosure cannot be limited to this. Embodiments of the present disclosure find application in laptop computers, tablet computers, media players, and/or specialized video conferencing equipment. Network (350) is any number of networks that transmit encoded video data between terminal devices (310), (320), (330) and (340), including, for example, wired (wired) and/or wireless communication networks. The communications network (350) may communicate on circuit-switched and/or packet-switched channels. Representative networks include telecommunications networks, local area networks, wide area networks, and/or the Internet. For the purposes of this description, the architecture and topology of the network (350) may not be essential to the operation of this disclosure, unless explained herein below.

[39] FIG. 4 illustrates, as an example of application for the disclosed subject matter, the placement of a video encoder and a video decoder in a streaming environment. The disclosed subject matter may be equally applicable to other video-enabled applications including, for example, videoconferencing, digital television, storage of compressed video on digital media including CD, DVD, memory card, etc., and so on. .

[40] The streaming system may include a capture subsystem (413), which may include a video source (401), such as a digital camera, producing, for example, a video stream (402) that is not compressed. In an example, the video stream (402) includes samples taken by a digital camera. The video stream (402), shown in bold to emphasize the large amount of data compared to the encoded video data (404) (or encoded video bitstreams), may be processed by an electronic device (420) that includes a video encoder (403) connected to the source (401) video. The video encoder (403) may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter, as described in more detail below. Encoded video data (404) (or encoded video bitstream (404)), depicted with a thin line to emphasize the smaller amount of data compared to the video stream (402), can be stored on the streaming server (405) for future use. One or more streaming client subsystems, such as client subsystems (406) and (408) in FIG. 4 can access the streaming server (405) to extract copies (407) and (409) of the encoded video data (404). The client subsystem (406) may include a video decoder (410), such as in an electronic device (430). The video decoder (410) decodes the incoming copy (407) of the encoded video data and creates an outgoing video stream (411) that can be rendered on a display (412) (eg, a display screen) or other rendering device (not shown). In some streaming systems, encoded video data (404), (407), and (409) (eg, video bitstreams) may be encoded according to certain video coding/compression standards. Examples of these standards include ITU-T Rec. H.265. In one example, the video coding standard under development is informally known as Universal Video Coding (VVC). The expanded expansion object can be used in a VVC context.

[41] It should be noted that the electronic devices (420) and (430) may include other components (not shown). For example, the electronic device (420) may include a video decoder (not shown), and the electronic device (430) may also include a video encoder (not shown).

[42] FIG. 5 shows a block diagram of a video decoder (510) according to an embodiment of the present disclosure. The video decoder (510) may be included in the electronic device (530). The electronic device (530) may include a receiver (531) (eg, receiver circuitry). Video decoder (510) may be used instead of video decoder (410) in the example of FIG. 4.

[43] The receiver (531) may receive one or more encoded video sequences to be decoded by the video decoder (510); in the same or another embodiment, one coded video sequence at a time, where the decoding of each coded video sequence is independent of other coded video sequences. An encoded video sequence may be received from a channel (501), which may be a hardware/software connection to a storage device that stores the encoded video data. The receiver (531) may receive encoded video data with other data, such as encoded audio data and/or ancillary data streams, which may be sent to their respective using entities (not shown). The receiver (531) may separate the encoded video sequence from other data. To combat network noise, a buffer memory (515) may be connected between the receiver (531) and the entropy decoder/analyzer (520) (hereinafter "parser (520)"). In some applications, the buffer memory (515) is part of the video decoder (510). In other cases, it may be outside the video decoder (510) (not shown). In other cases, there may be a buffer memory (not shown) outside the video decoder (510), for example, to combat network jitter, and, in addition, another buffer memory (515) inside the video decoder (510), for example, to handle playback time. When the receiver (531) receives data from a storage/transfer device with sufficient bandwidth and controllability, or from an isosynchronous network, buffer memory (515) may not be needed or may be small. For use in best quality packet networks such as the Internet, a buffer memory (515) may be required, which may be relatively large and may preferably be of an adaptive size, and may be at least partially implemented in an operating system or similar elements (not shown) outside the video decoder (510).

[44] The video decoder (510) may include a parser (520) for recovering symbols (521) from the encoded video sequence. The categories of these symbols include information used to control the operation of the video decoder (510) and potentially information to control a visualization device such as a visualization device (512) (e.g., a display screen) that is not an integral part of the electronic device (530), but may be connected to an electronic device (530) as shown in FIG. 5. The control information for the rendering device(s) may be in the form of additional extended information (SEI messages) or video usability information (VUI) parameter set fragments (not shown). A parser (520) may parse/entropy decode the received encoded video sequence. Encoding of the encoded video sequence may be in accordance with a video coding technology or standard, and may follow various principles including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, and so on. The parser (520) can extract from the encoded video sequence a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder based on at least one parameter corresponding to the group. Subgroups may include groups of pictures (GOPs), pictures, tiles, slices, macroblocks, coding units (CUs), blocks, transform units (TUs), prediction units (PUs), and so on. The parser (520) may also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, and so on.

[45] The parser (520) may perform an entropy decoding/parsing operation of the video sequence obtained from the buffer memory (515) to create symbols (521).

[46] Symbol recovery (521) may involve many different modules depending on the type of encoded video image or parts thereof (such as: inter and intra image, inter and intra block) and other factors. Which modules are enabled and how can be controlled by the subgroup control information parsed from the encoded video sequence by the parser (520). The flow of such subgroup control information between the parser (520) and the plurality of modules is not shown below for clarity.

[47] In addition to the functional blocks already mentioned, the video decoder (510) can be conceptually subdivided into a number of functional modules, as described below. In a practical implementation operating under commercial constraints, many of these modules interact closely with each other and can be at least partially integrated into each other. However, for purposes of describing the disclosed subject matter, the conceptual division into functional modules below is appropriate.

[48] The first module is the scaling/inverse transform module (551). The scaling/inverse transform module (551) receives the quantized transform coefficient, as well as control information, including which transform to use, block size, quantization factor, quantization scaling matrices, and so on, in the form of symbol(s) (521) from the parser (520). The scaling/inverse transform module (551) may output blocks containing sample values that may be input to the aggregator (555).

[49] In some cases, the output samples of the scaling/inverse transform module (551) may refer to an intra-coded block; that is, to a block that does not use prediction information from previously restored images, but can use prediction information from previously restored portions of the current image. Such prediction information may be provided by the intra prediction module (5452). In some cases, the intra-prediction module (552) generates a block of the same size and shape as the block being reconstructed, using the already reconstructed surrounding information retrieved from the current image buffer (558). The current picture buffer (558) buffers, for example, a partially restored current picture and/or a fully restored current picture. The aggregator (555) in some cases adds, for each sample, the prediction information generated by the intra-prediction module (552) to the output sample information provided by the scaling/inverse transform module (551).

[50] In other cases, the output samples of the scaling/inverse transform module (551) may refer to a block with intercoding and potentially with motion compensation. In such a case, the motion compensation prediction module (553) may access the reference picture memory (557) to retrieve samples used for prediction. After compensating for motion of the selected samples in accordance with the symbols (521) associated with the block, these samples can be added by the aggregator (555) to the output of the scaling/inverse transform module (551) (in this case called residual samples or residual signal) to generate output information. The addresses in the reference picture memory (557) from where the motion compensation prediction module (553) selects prediction samples may be controlled by motion vectors available to the motion compensation prediction module (553) in the form of symbols (521), which may have, for example, components X, Y and reference image. Motion compensation may also include interpolation of sample values retrieved from reference picture memory (557) when accurate sub-sample motion vectors are used, motion vector prediction mechanisms, and so on.

[51] The output samples of the aggregator (555) may be subjected to various loop filtering methods in the loop filter module (556). The video compression technologies may include in-loop filtering technologies that are controlled by parameters included in the encoded video sequence (also referred to as the encoded video bitstream) and provided to the loop filtering module (556) as symbols (521) from the parser (520), but may also respond to meta-information received during decoding of the previous (in the order of decoding) parts of the encoded image or encoded video sequence, and also respond to sample values previously restored and filtered by the loop.

[52] The output of the loop filter module (556) may be a stream of samples that can be output to the renderer (512) and also stored in the reference picture memory (557) for use in future inter-picture prediction.

[53] Certain encoded pictures after full recovery can be used as reference pictures for future prediction. For example, after the encoded picture corresponding to the current picture has been completely restored and the encoded picture has been identified as a reference picture (for example, by the parser (520)), the current picture buffer (558) may become part of the reference picture memory (557), and the buffer of the fresh current picture may be reallocated before starting the reconstruction of the next encoded picture.

[54] The video decoder (510) may perform decoding operations according to a predetermined video compression technology in a standard such as ITU-T Rec. H.265. The encoded video sequence may conform to the syntax specified by the video compression technology or standard being used, in the sense that the encoded video sequence conforms to both the syntax of the video compression technology or standard and to the profiles documented in the video compression technology or standard. In particular, a profile may select certain tools as the only tools available for use in that profile, from among all the tools available in a video compression technology or standard. It may also be necessary for compliance that the complexity of the encoded video sequence be within the boundaries defined by the level of technology or video compression standard. In some cases, levels limit the maximum image size, the maximum frame rate, the maximum recovery sampling rate (measured in megasamples per second, for example), the maximum reference image size, and so on. The limits set by the layers may, in some cases, be further constrained by Hypothetical Reference Decoder (HRD) specifications and metadata for managing the HRD buffer signaled in the encoded video sequence.

[55] In an embodiment, the receiver (531) may receive additional (redundant) data with encoded video. Additional data may be included as part of the encoded video sequence(s). The additional data may be used by the video decoder (510) to correctly decode the data and/or to more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant images, forward error correction codes, and so on.

[56] FIG. 6 is a block diagram of a video decoder (603) according to an embodiment of the present disclosure. The video encoder (603) is included in the electronic device (620). The electronic device (620) includes a transmitter (640) (eg, a transmission circuit). Video encoder (603) may be used instead of video encoder (603) in the example of FIG. 4.

[57] The video encoder (603) may receive video samples from a video source (601) (which is not part of the electronic device (620) in the example of FIG. 6) that may capture video image(s) for encoding by the video encoder (603) . In another example, the video source (601) is part of an electronic device (620).

[58] The video source (601) may provide the source video sequence for encoding the video by the encoder (603) in the form of a digital stream of video samples, which may have any suitable bit depth (for example: 8 bit, 10 bit, 12 bit, ...), any color space (eg, BT.601 Y CrCB, RGB, ...) and any suitable sampling structure (eg, Y CrCb 4:2:0, Y CrCb 4:4:4). In a media serving system, the video source (601) may be a storage device storing pre-rendered video. In a videoconferencing system, the video source (601) may be a camera that captures local image information as a video sequence. The video data may be provided as a plurality of individual images that, when viewed sequentially, convey motion. The images themselves may be organized as a spatial array of pixels, where each pixel may contain one or more samples depending on the sampling structure used, color space, and so on. One skilled in the art can easily understand the relationship between pixels and samples. The description below is based on readings.

[59] According to an embodiment, the encoder (603) may encode and compress images of the source video sequence into the encoded video sequence (643) in real time or with any other time constraints as required by the application. Providing an appropriate coding rate is one of the functions of the controller (650). In some embodiments, the implementation of the controller (650) controls other functional modules, as described below, and is functionally associated with other functional modules. The connection is not shown for clarity. Parameters set by the controller (650) may include parameters related to speed control (picture skip, quantizer, speed-distortion optimization methods lambda value, ...), image size, group of pictures (GOP) layout, maximum motion vector search range and so on. The controller (650) may be configured to have other suitable functions that are related to the video encoder (603) optimized for a particular system design.

[60] In some embodiments, the video encoder (603) is configured to operate in an coding loop. As a simplified description, in an example, an encoding loop may include a source encoder (630) (e.g., responsible for creating symbols, such as a stream of symbols, based on an input image for encoding, and reference image(s), and a (local) decoder (633) built into the video encoder (603) The decoder (633) decompresses the symbols to create sample data in a similar way that a (remote) decoder can also create (because any compression between the symbols and the encoded video bitstream is lossless in video compression technologies considered in the disclosed subject matter.) The reconstructed sample stream (sample data) is entered into the reference picture memory 634. Since the decoding of the symbol stream results in bit-accurate results, regardless of the location of the decoder (local or remote), the contents in the reference picture memory 634 is also bit accurate between the local encoder and the remote encoder In other words, the encoder prediction part "sees" as reference picture samples exactly the same sample values that the decoder would "see" if using prediction during decoding. This fundamental principle of reference picture synchronism (and the resulting wander if synchronism cannot be maintained due to, for example, channel errors) is also used in some related fields of technology.

[61] The operation of the "local" decoder (633) may be the same as that of the "remote" decoder, such as the video decoder (510), which has already been described in detail above in connection with FIG. 5. Briefly referring also to FIG. 5, however, since the symbols are available and the encoding/decoding of the symbols into the encoded video sequence by the entropy encoder (645) and the parser (520) can be lossless, the entropy decoding parts of the video decoder (510), including the buffer memory (515), the parser ( 520) cannot be fully implemented in the local decoder (633).

[62] At this point, it can be observed that any decoding technology other than parsing/entropy decoding that is present in a decoder must also necessarily be present in a substantially identical functional form in the corresponding encoder. For this reason, the disclosed object of the disclosure focuses on the operation of the decoder. The description of encoding technologies can be abbreviated as they are the inverse of fully described decoding technologies. Only in some areas a more detailed description is required, which is given below.

[63] During operation, in some examples, source encoder (630) may perform motion-compensated predictive encoding that predictively encodes an input picture with reference to one or more previously encoded pictures from a video sequence that have been designated "reference pictures". Thus, the encoding engine (632) encodes the differences between the pixel blocks of the input image and the pixel blocks of the reference image(s) that can be selected as prediction reference(s) for the input image.

[64] The local video decoder (633) may decode the encoded video data of pictures that can be designated as reference pictures based on the symbols generated by the source encoder (630). The operations of the encoding engine (632) may preferably be lossy processes. When the encoded video data can be decoded in a video decoder (not shown in FIG. 6), the reconstructed video sequence can typically be a copy of the original video sequence with some errors. The local video decoder (633) replicates the decoding processes that may be performed by the video decoder on the reference pictures and may cause the recovered reference pictures to be stored in the reference picture cache (734). Thus, the video encoder (603) can locally store copies of the reconstructed reference pictures that share the same content as the reconstructed reference pictures to be received by the far end video decoder (in the absence of transmission errors).

[65] The predictor (635) may perform predictive searches for the encoding engine (632). That is, for a new picture to be encoded, the predictor (635) may search the reference picture memory (634) for sample data (as reference pixel block candidates) or certain metadata such as reference picture motion vectors, block shapes, and so on. further, which can serve as a suitable prediction reference for new images. The predictor (635) may operate on a block-by-pixel basis of samples to find appropriate prediction references. In some cases, as determined by the search results obtained by the predictor (635), the input image may have prediction references taken from a plurality of reference pictures stored in the reference picture memory (6734).

[66] The controller (650) may control the encoding operations of the source encoder (630), including, for example, setting parameters and subgroup parameters used to encode video data.

[67] The output signals of all the above functional modules can be entropy encoded in an entropy encoder (645). An entropy encoder (645) translates symbols generated by various functional units into an encoded video sequence by lossless symbol compression according to techniques such as Huffman coding, variable length coding, arithmetic coding, and so on.

[68] The transmitter (640) may buffer the encoded video sequence(s) generated by the entropy encoder (645) to prepare for transmission over a communication channel (660), which may be a hardware/software connection to a storage device that will store the encoded video data. The transmitter (640) may combine the encoded video data from the video encoder (630) with other data to be transmitted, such as encoded audio data and/or ancillary data streams (sources not shown).

[69] The controller (650) may control the operation of the video encoder (603). During encoding, the controller (650) may assign to each encoded picture a certain type of encoded picture, which may affect the encoding methods that can be applied to the corresponding picture. For example, images can often be categorized as one of the following image types:

[70] An intra-picture (I-picture) may be one that can be encoded and decoded without using any other picture in the sequence as a prediction source. Some video codecs allow various types of intra-pictures, including, for example, independent decoder update ("IDR") pictures. Those skilled in the art will be aware of these 1-image variations and their respective uses and features.

[71] A predictive picture (P-picture) may be one that can be encoded and decoded using intra-prediction or inter-prediction using at most one motion vector and a reference index to predict sample values of each block.

[72] A bidirectional predictive picture (B-picture) may be one that can be encoded and decoded using intra-prediction or inter-prediction using at most two motion vectors and reference indices to predict the sample values of each block. Likewise, multi-prediction pictures may use more than two reference pictures and associated metadata to reconstruct a single block.

[73] Source images can typically be spatially divided into multiple blocks of samples (eg, blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and encoded on a block-by-block basis. The blocks may be predictively encoded with reference to other (already encoded) blocks, as determined by the coding assignment applied to the corresponding block images. For example, I-picture blocks may be predictively encoded or they may be predictively encoded with reference to already encoded blocks of the same picture (spatial prediction or intra-prediction). The pixel blocks of P-pictures may be predictively encoded, by spatial prediction, or by temporal prediction with reference to one previously encoded reference picture. Blocks of B-pictures may be predictively coded, spatially predicted, or temporally predicted with reference to one or two previously encoded reference pictures.

[74] The video encoder (603) may perform encoding operations in accordance with a predetermined video coding technology or standard, such as ITU-T Rec. H.265. In operation, the video encoder (603) may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancy in the input video sequence. The encoded video data may therefore follow the syntax given by the video coding technology or standard being used.

[75] In an embodiment, the transmitter (640) may transmit additional data with encoded video. The source encoder (630) may include such data as part of the encoded video sequence. The additional data may comprise temporal/spatial/SNR enhancement layers, other forms of redundant data such as redundant images and slices, SEI messages, VUI parameter set fragments, and so on.

[76] The video may be captured as a plurality of source images (video images) in time sequence. Intra-frame prediction (often abbreviated as intra-prediction) uses the spatial correlation in a given image, while inter-frame prediction uses (temporal or other) correlation between images. In the example, a specific image, when encoding/decoding, called the current image, is divided into blocks. When a block in the current picture is similar to a reference block in a previously encoded and still buffered reference picture in a video, the block in the current picture may be encoded with a vector called a motion vector. The motion vector points to a reference block in the reference picture and may have a third dimension identifying the reference picture if multiple reference pictures are used.

[77] In some embodiments, a dual prediction technique may be used in inter-picture prediction. According to the double prediction method, two reference pictures are used, such as a first reference picture and a second reference picture, which both precede the current picture in the video in decoding order (but may be in the past and future, respectively, in display order). A block in the current picture may be encoded with a first motion vector that points to a first reference block in a first reference picture and a second motion vector that points to a second reference block in a second reference picture. The block may be predicted by the combination of the first reference block and the second reference block.

[78] In addition, in inter-picture prediction, a merge mode technique can be used to improve coding efficiency.

[79] According to some embodiments of the disclosure, predictions such as inter predictions and intra predictions are performed in a unit of blocks. For example, according to the HEVC standard, an image in a video sequence is divided into coding tree units (CTUs) for compression, where the CTUs in the image are the same size, such as 64x64 pixels, 32x32 pixels, or 16x16 pixels. In general, the CTU includes three coding tree units (CTBs), which are one luma CTB and two chrominance CTBs. Each CTU may be recursively split into a quadtree into one or multiple coding units (CUs). For example, a CTU of 64x64 pixels can be divided into one CU of 64x64 pixels, or 4 CUs of 32x32 pixels, or 16 CUs of 16x16 pixels. In the example, each CU is parsed to determine a prediction type for the CU, such as an inter-prediction type or an intra-prediction type. The CU is divided into one or more prediction units (PUs) depending on temporal and/or spatial predictability. Typically, each PU includes a Luma Prediction Unit (PB) and two Chroma PBs. In an embodiment, the encoding (encoding/decoding) prediction operation is performed in units of prediction blocks. Using the luminance prediction block as an example of the prediction block, the prediction block includes a matrix of values (e.g., luminance values) for pixels, such as 8x8 pixels, 16x16 pixels, 8x16 pixels, 16x8 pixels, and the like.

[80] FIG. 7 shows a diagram of a video encoder (703) according to another embodiment of the disclosure. The video encoder (703) is configured to receive a processing block (eg, a prediction block) from sample values in the current video image in the video sequence and encoding the processing block into an encoded image that is part of the encoded video sequence. In one example, video encoder (703) is used instead of video encoder (403) in the example of FIG. 4.

[81] In the HEVC example, video encoder (703) receives a matrix of sample values for a processing block, such as an 8x8 sample prediction block, or the like. The video encoder (703) determines whether the processing unit is best encoded when using the intra mode, inter mode, or dual prediction mode using rate-distortion optimization, for example. When a processing block is to be encoded in intra mode, the video encoder (703) may use an intra prediction technique to encode the processing block into an encoded picture; and when the processing block is to be encoded in inter-mode or dual prediction mode, the video encoder (703) may use an inter-prediction or dual prediction method, respectively, to encode the processing block into an encoded picture. In some video coding technologies, the merge mode may be a sub-mode of inter-picture prediction in which a motion vector is derived from one or more motion vector predictors without taking advantage of the encoded motion vector component outside of the predictors. In some other video coding technologies, there may be a motion vector component applicable to the block in question. In one example, the video encoder (703) includes other components such as a mode selector (not shown) to determine the mode of the processing units.

[82] In the example of FIG. 7, the video encoder (703) includes an inter encoder (730), an intra encoder (722), a residual calculator (723), a switch (726), a residual encoder (724), a common controller (721), and an entropy encoder (725). connected together as shown in Fig. 7.

[83] The inter encoder (730) is configured to receive samples of the current block (eg, processing block), compare the block to one or more reference blocks in the reference pictures (eg, blocks in previous pictures and next pictures), generate inter-prediction information (eg, descriptions of redundant information according to the inter-coding method, motion vectors, merge mode information) and calculating inter-prediction results (eg, predicted block) based on the inter-prediction information using any suitable method. In some examples, the reference pictures are decoded reference pictures that are decoded based on the encoded video information.

[84] The intra-encoder (722) is configured to receive samples of the current block (e.g., processing block), in some cases comparing the block with blocks already encoded in the same picture, generating quantized coefficients after the transform, and in some cases also intra -prediction (eg, intra-prediction direction information according to one or more intra-coding methods). In one example, the intra encoder (722) also calculates intra prediction results (eg, predicted block) based on intra prediction information and reference blocks in the same picture.

[85] The common controller (721) is configured to determine common control data and control other components of the video encoder (703) based on the common control data. In one example, the common controller (721) determines the mode of the block and provides a control signal to the switch (726) based on the mode. For example, when the mode is intra mode, the general controller (721) controls the switch (726) to select the result of the intra mode for use by the residual calculator (723), and controls the entropy encoder (725) to select the intra prediction information and turn on intra-prediction information into the bitstream; and when the mode is inter mode, the common controller (721) controls the switch (726) to select the inter prediction result for use by the residual calculator (723), and controls the entropy encoder (725) to select the inter prediction information and turn on the information inter-predictions to bitstream.

[86] The residual calculator (723) is configured to calculate the difference (residual data) between the received block and the prediction results selected from the intra-coder (722) or inter-coder (730). The residual encoder (724) is configured to operate based on the residual data to encode the residual data to generate transform coefficients. In one example, the residual encoder (724) is configured to transform residual data from the spatial domain to the frequency domain and generate transform coefficients. The transform coefficients are then subjected to quantization processing to obtain quantized transform coefficients. In various embodiments, the video encoder (703) also includes a residual decoder (728). The residual decoder (728) is configured to perform inverse transformation and generate decoded residual data. The decoded residual data may be appropriately used by the intra-coder (722) and the inter-coder (730). For example, the inter-coder (730) may generate decoded blocks based on the decoded residual data and inter-prediction information, and the intra-coder (722) may generate decoded blocks based on the decoded residual data and intra-prediction information. The decoded blocks are suitably processed to generate decoded pictures, and the decoded pictures may be buffered in a memory circuit (not shown) and used as reference pictures in some examples.

[87] The entropy encoder (725) is configured to format the bitstream to include a coded block. The entropy encoder (725) is configured to include various information according to an appropriate standard such as the HEVC standard. In one example, entropy encoder (725) is configured to include common control data, selected prediction information (eg, intra-prediction information or inter-prediction information), residual information, and other pertinent information in the bitstream. It should be noted that, according to the disclosed subject matter, when encoding a block in the merge submode of either the inter mode or the dual prediction mode, there is no residual information.

[88] FIG. 8 shows a diagram of a video decoder (9810) according to another embodiment of the disclosure. The video decoder (810) is configured to receive encoded pictures that are part of the encoded video sequence and decode the encoded pictures to create reconstructed pictures. In one example, video decoder (810) is used instead of video decoder (410) in the example of FIG. 4.

[89] In the example of FIG. 8, the video decoder (810) includes an entropy decoder (871), an interdecoder (880), a residual decoder (873), a reconstruction unit (874), and an intradecoder (872) connected together as shown in FIG. 8.

[90] The entropy decoder (871) may be configured to recover from the encoded image certain symbols that represent the syntax elements that make up the encoded image. Such symbols may include, for example, the mode in which the block is encoded (such as, for example, intra mode, inter mode, dual prediction mode, the latter two in a merge submode or another submode), prediction information (such as, e.g., intra-prediction information or inter-prediction information) that can identify a certain sample or metadata that is used for prediction by the intra-decoder (872) or inter-decoder (880), respectively, the residual information in the form of, for example, quantized coefficients transformations, etc. In one example, when the prediction mode is inter mode or bidirectional prediction mode, inter prediction information is provided to the inter decoder (880); and when the prediction type is an intra prediction type, the intra prediction information is provided to the intra decoder (872). The residual information may be inverse quantized and provided to the residual decoder (873).

[91] The inter decoder (880) is configured to receive the inter prediction information and generate inter prediction results based on the inter prediction information.

[92] 8 The intra decoder (772) is configured to receive the intra prediction information and generate prediction results based on the intra prediction information.

[93] The residual decoder (873) is configured to perform inverse quantization to extract the dequantized transform coefficients, and process the dequantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The residual decoder (873) may also require certain control information (to include a quantizer parameter (QP)), and this information may be provided by the entropy decoder (871) (data path not shown as this may only be small control information).

[94] The recovery module (874) is configured to combine in the spatial domain the residual output by the residual decoder (873) and the prediction results (output by the inter- or intra-prediction modules, as the case may be) to form a reconstructed block that may be part of restored image, which, in turn, can be part of the restored video. It should be noted that other suitable operations such as a release operation and the like may be performed to improve the visual quality.

[95] It should be noted that video encoders (403), (603) and (703) and video decoders (410), (510) and (810) may be implemented using any suitable technique. In an embodiment, video encoders (403), (603) and (703) and video decoders (410), (510) and (810) may be implemented using one or more integrated circuits. In another embodiment, video encoders (403), (603), and (603) and video decoders (410), (510), and (810) may be implemented using one or more processors that execute program instructions.

[96] According to some aspects of the disclosure, an encoding mode, which is called a palette-based encoding mode, can be used in addition to the inter-prediction mode and the intra-prediction mode. The present disclosure provides methods for simplifying a palette-based encoding mode with a local dual tree encoding structure.

[97] In some examples, screen content has certain characteristics, and encoding tools may be developed to encode the screen. Screen coding coding tools can greatly improve coding efficiency. A palette-based coding mode may represent a block of pixels using palette indices that include the base colors of the block. The palette and indices can be encoded using spatial redundancy.

[98] In particular, in some embodiments, a palette is defined as a lookup table of entries that can associate indices with pixel values. In some examples, a pixel value includes one or more values corresponding to color components. In one example, a pixel value may include three values, respectively representing three color components. In another example, for a monochrome palette, a pixel value includes a value representing one color component.

[99] In some embodiments, a block of pixels may be palette-based, and the encoding mode using the palette is referred to as the palette-based encoding mode. The palette-based coding mode is adopted in some video coding standards such as VVC and the like.

[100] According to some embodiments, the palette-based coding mode is a separate coding mode. In an embodiment, the palette-based coding mode is implemented as an encoding unit (e.g., palette-based encoder, palette-based decoder, etc.) that is separate from the intra encoder (722), inter encoder (730), intra -decoder (872), inter-decoder (870). In one example, the palette-based coding mode is implemented as a hardware circuit separate from the intra encoder (722), inter encoder (730), intra decoder (872), inter decoder (870). In another example, the palette-based coding mode is implemented as a software module separate from the software modules for intra-encoder (722), inter-encoder (730), intra-decoder (872), inter-decoder (870). The software module for the palette-based coding mode and the other software modules may be executed by the same processor or by different processors.

[101] According to some aspects of the disclosure, a palette-based coding mode typically includes two parts: a first part of encoding methods for the palette and a second part of encoding methods for samples using the palette. The first part may include selecting a primary color and coding the palette. The second part may include palette index coding, run length coding, and outlier pixel coding.

[102] In some examples, a palette-based encoder may encode a block of video data by specifying a palette for the block (e.g., explicitly encoding a palette, predicting a palette, a combination of signaling and prediction, etc.), placing an entry in the palette to represent one or more values. pixels, and encoding the block with index values that specify entries in the palette used to represent the block's pixel values. In some examples, a palette-based encoder may signal the palette and/or index values in the encoded bitstream. In turn, the palette-based decoder can derive from the encoded bitstream the palette for the block, and the index values for the individual pixels of the block. In some other examples, palette may be predicted and a palette-based encoder may signal index values in the encoded bitstream, without palette signaling, and the palette-based encoder may predict palette and obtain index values in the encoded bitstream. The palette-based decoder can then associate pixel index values with palette entries that provide pixel values. The palette-based decoder may reconstruct the various pixels of the block based on the pixel values associated with the index values.

[103] In some embodiments, the palette may include the most dominant pixel values in a given block. For example, the most dominant pixel values may include one or more pixel values that occur most frequently in a given block. In addition, in some examples, a threshold value (eg, the number of pixels having a specified pixel value) may be used to determine whether the pixel value should be included as one of the most dominant pixel values in the block.

[104] In some embodiments, a histogram-based algorithm is used to classify pixel values for pixels in a block to generate a palette. In some examples, the L most significant peaks in the histogram may be chosen as the primary colors (L is a positive integer). In the example, pixel values close to the base color can be quantized to the base color. In some examples, pixels that do not belong to any of the primary colors are called dropout pixels. Drop-down transition pixels can be quantized before encoding. In some embodiments, no quantization processes are performed for lossless coding.

[105] In addition, in some embodiments, a color index is assigned to each pixel to indicate which color the pixel belongs to. In some examples, if L primaries are used, 0 to (L-1) may be used as color indices for the primaries, and 0 to (L-1) may be referred to as the primary color set. For those pixels whose primary color indexes do not exist, they are assigned a special index (for example, an N index other than 0 to (L-1)), and these pixels are called “outliers”.

[106] In general, the palette is implemented as a color lookup table. The color reference table includes many entries. Each entry associates a color index with a color (eg, base color). In some examples, the same palette is used to encode the luminance and chrominance components. Each entry can represent a specific RGB (YUV) color using three color components. For example, a palette might have an entry of 1s (R, G, B) = (0, 0, 0), which represents pure black, and might have an entry of 0s (R, G, B) = (2, 10, 200 ), which is a bluish color. It should be noted that when the video format is 420, the color plane is upsampled to create a color lookup table for the palette.

[107] In some embodiments, palette-based coding is performed based on the CU. The example outputs a reference color table for the current CU that includes the most dominant pixel values in the current CU. In some examples, the size and elements of a color lookup table may be passed. In some other examples, the color lookup table size and entries may be predictively encoded using the size and/or color lookup table entries of other CUs.

[108] According to some aspects of the disclosure, palette prediction may be performed based on a list of palette predictors. In the example, the list of palette predictors may include several palettes that are used for other CUs. To encode the current palette using the palette predictor list, a binary vector is used as a reference to indicate whether each entry in the palette predictor list is reused in the current palette.

[109] In some examples (eg, HEVC SCC), the palette predictor list stores previously encoded palette entries as references for predicting the current palette. The list of palette predictors is updated after each palette mode CU. In one example, the list of palette predictors works similarly to a aging cache. The most recent palette may be inserted at the beginning of the list of palette predictors, and entries from the furthest CUs in the scan order are discarded if the list size exceeds a threshold. In some examples, the maximum allowed palette predictor size and palette size are either signaled in high-level syntax or as specific values agreed upon by both the encoder and decoders. In an embodiment, an upper bound on the size of the palette predictor list is (directly or indirectly) signaled in the SPS. For example, a list of palette predictors might be about twice the size of the palette limit. In some examples, a palette predictor list is also referred to as a retrospective palette predictor list (HPPL). In an example, in SPS, the maximum size of palette predictors is signaled to be 63, the maximum size of palette is signaled to be 31. In another example, these two values are received by the encoder and decoder without signaling.

[110] In some embodiments, entries in the palette prediction list may have reuse flags, respectively. When the reuse flag for an entry is true (ie, "1"), the entry is called a reused entry. The color components of reused entries in the list of palette predictions can be copied into the color lookup table of the current palette. Reused entries are placed at the beginning of the current palette in such a way as to preserve their order in the list of palette predictors. Reused entries in the current palette may be followed by new palette entries that are not in the list of palette predictors. New palette entries can be signaled. Each new palette entry may include, for example, three color components.

[111] In some embodiments, a palette initializer with predefined entries may be used to initialize a list of palette predictors. Using a palette initializer can result in improved coding efficiency in some scenarios. The palette initializer may be signaled, for example, in a Picture Parameter Set (PPS), in a Sequence Parameter Set (SPS), and the like.

[112] Color indices can be encoded as shown below. After classification, the block pixels can be converted into color indices according to the chosen basic set of colors. In some examples, a predictive coding method can be applied to color indices, where a line of pixels can be predicted by several different modes (e.g., three different modes), including horizontal mode (e.g., index copy mode), vertical mode (e.g., copy from above mode) and normal mode (eg drop mode). In some examples, two index scan orders (eg, horizontal stroke scan and vertical stroke scan) are used in encoding color indices. An index rotation flag may be signaled to indicate which of the two index scan orders is used.

[113] In the index copy mode, starting from the first pixel, one or more consecutive indexes may be copied from the first pixel. The color index of the first pixel may be signaled.

[114] In copy-on-top mode, one or more successive color indices may be copied from an upper pixel line, such as from a pixel line that is above the current pixel line.

[115] In dropout mode, when a dropout pixel is encountered, such as signaled by the largest index (eg, N) in the primary color set, the corresponding pixel value may be encoded after the largest index (eg, N). A CU can have multiple dropout pixels with different color values. For different dropout pixel locations, the pixel values for the dropout pixels can be different.

[116] An index value can be signaled for each index copy mode. The index signaling may be clustered in front (or at the beginning), for example, to improve the throughput of context adaptive binary arithmetic coding (CABAC). Likewise, the pixel values of the outlier pixels may be signaled from behind, for example, to improve CABAC throughput. Index copy mode and top copy mode can be signaled between index encoding and drop encoding.

[117] In an embodiment, the coding tree schema supports the ability for the luminance component and the corresponding chrominance component(s) to have separate block tree structures. In the example for sections P and B, the luminance and chroma CTUs in the CTU share the same coding tree structure (eg, one tree). For I-slices, luma and chrominance CTUs in CTUs may have separate block tree structures (eg, dual tree), and the case of splitting CTUs using separate block tree structures is called dual tree splitting. In an example where dual tree partitioning is applied, the luma CTB may be divided into luma CUs by a luminance coding tree structure, and the chrominance CTB may be divided into chrominance CUs by a chroma coding tree structure.

[118] In some examples, such as JVET, the chrominance component samples may have an independent or separate split tree structure (or coding tree structure) compared to the luma component. A single coding tree structure may start at the CTU level, and the dual tree structure from the CTU level is referred to in some examples as the global dual tree structure. In an example, a chroma CU (eg, a CU that includes only two chrominance components) is larger than a similar chroma CU at the corresponding sample location.

[119] In some examples of the CTU level dual tree (global dual tree structure), the maximum allowable palette predictor size and palette size are reduced, for example, by a factor of two, so that for each channel (luminance or chrominance), complexity is reduced. For example, in the case of single tree coding, the luminance and chrominance blocks encoded with a palette are coded together, and the maximum allowed palette predictor size is 63 entries, and the maximum allowed palette size is 31 entries. In the case of dual tree coding, for palette-coded luminance blocks, the maximum allowable palette predictor size is 31 entries, and the maximum allowable palette size is 15 entries; and for palette-coded chrominance blocks, the maximum allowable palette predictor size is 31 entries, and the maximum allowable palette size is 15 entries.

[120] According to some aspects of the disclosure, a technique referred to as local dual tree is used to avoid using small chroma blocks (eg, such as CUs less than 4x4 chroma samples). In one example, when certain conditions are met (eg, the size of the parent CU is less than or equal to some threshold, which may result in a chrominance block having fewer than 4x4 samples), a local dual tree method may be initiated. From this point on, luma and chrominance coding can be separated for CUs in a similar way to the dual tree at the CTU level.

[121] According to an aspect of the disclosure, when partitioning tree structures are used, luminance components may be encoded in one of intra-prediction mode, IBC mode, and palette-based coding mode, and chrominance components may be encoded in one of intra-prediction mode and palette-based coding. However, the use of a local dual tree complicates operations in palette-based coding mode. For example, a block in a local dual tree structure may have neighboring blocks in a single tree structure. The same list of palette predictors can be used by a box in the local dual tree structure and neighboring boxes in the single tree structure. Updating the list of palette predictors can be tricky.

[122] Aspects of the disclosure provide techniques that facilitate palette-based encoding using a local dual encoding tree structure.

[123] According to an aspect of the disclosure, when some conditions are true, the palette predictor list update process is disabled or skipped for palette mode encoding. In some instances, when a local dual tree may be initiated and the blocks of the local dual tree structure are coded in a palette-based coding mode, the process of updating the palette predictor list may be disabled. Thus, under the local dual tree structure, for luminance blocks that are coded in palette-based encoding mode and/or chrominance blocks that are coded in palette-based encoding mode, the palette entries of these blocks are not used to update the palette entries in the list of palette predictors. .

[124] In an embodiment, a local dual tree initiator can be detected. In the example, when the local dual tree is initiated, the palette predictor list is not updated based on the chrominance blocks that are palette-coded in the local dual tree structure, nor is it updated based on the luminance blocks that are palette-coded under structure of a local dual tree. In another example, when the local dual tree is initiated, the list of palette predictors is not updated based on the luma blocks that are palette-coded according to the local binary tree structure, and may be updated based on the chrominance blocks that are color-coded on palette-based in accordance with the structure of the local dual tree. In another example, when the local dual tree is initiated, the palette predictor list is not updated based on the chrominance blocks that are palette-coded in the local dual tree structure, and may be updated based on the luma blocks that are palette-coded. under the structure of a local dual tree.

[125] In particular, in the example, a variable denoted localDualTree is used to determine whether a local dual tree can be initiated. In the example, the localDualTree variable is obtained according to (Eq. 1):

where treeType denotes the split tree structure type (eg, SINGLE_TREE, DUAL_TREE_LUMA, DUAL_TREE_CHROMA, etc.), slice_type denotes the type (eg, I, P, or B) of the current slice, and qtbtt_dual_tree_intra_flag denotes a flag for the global dual tree. When the variable localDualTree is 1, a local dual tree can be initiated and discovered; when the variable localDualTree is 0, the local dual tree is not initialized.

[126] FIG. 9 shows an example (900) syntax for updating a list of palette predictions in some examples. In some examples, when localDualtree is 0, the list of palette predictions may be updated according to syntax example (900); and when localDualtree is equal to 1, the update process according to syntax example (900) is skipped.

[127] In an example, for a block coded by palette under a single tree structure, the list of palette predictions may be updated on a block-by-block basis. For example, treeType is SINGLE_TREE, so localDualtree is 0. Also, the startComp variable (for example, the initial color component) is set to 0, and the numComps variable (for example, the number of color components) is set to 1 when only the luma component is encoded, and is set to 3 when both the luma component and the chrominance components are encoded. Then, the size of the palette predictor list (eg, denoted by PredictorPaletteSize) and the palette predictor list entries (eg, denoted by predictorPaletteEntries) can be updated according to the (900) syntax.

[128] In another example, for a luminance block with palette coding under the global dual tree, the list of palette predictions may be updated on a per-block basis. For example, in CTU (global dual tree) level dual tree cases, treeType is DUAL_TREE_LUMA, slice_type is I, global dual tree flag qtbtt_dual_tree_intra_flag is 1, thus localDualtree is 0. Also, startComp is set to 0 and numComps is set to a value of 1. Then the size of the palette predictor list (eg, denoted by PredictorPaletteSize) and the palette predictor list entries (eg, denoted by predictorPaletteEntries) can be updated according to the (900) syntax.

[129] In another example, for a chrominance block coded by a palette under the global dual tree, the list of palette predictions may be updated on a block-by-block basis. For example, in CTU (global dual tree) level dual tree cases, treeType is DUAL_TREE_CHROMA, slice_type is I, global dual tree flag qtbtt_dual_tree_intra_flag is 1, so localDualtree is 0. Also, startComp is set to 1 and numComps is set to a value of 2. Then the size of the palette predictor list (eg, denoted by PredictorPaletteSize) and the palette predictor list entries (eg, denoted by predictorPaletteEntries) can be updated according to the (900) syntax.

[130] In another example, for a palette-coded chrominance block under the local dual tree, updating the palette prediction list may be omitted. For example, treeType is DUAL TREE CHROMA, slice_type is I, global dual tree flag qtbtt_dual_tree_intra_flag is 0, thus localDualtree is 1. Then, the update process according to syntax example (900) is skipped.

[131] According to another aspect of the disclosure, when coding patterns are used that can trigger the use of a local dual tree, the palette-based coding mode is disabled for small blocks. In this way, the complexity of updating the list of palette predictors in the case of a local dual tree can be avoided.

[132] In an example, when slice_type is not equal to I, or when slice_type is equal to I, but the slice level flag for enabling a (global) dual tree (e.g., qtbtt_dual_tree_intra_flag) is 0, a local dual tree can be initiated (e.g., localDualTree is equal to 1) . When a local dual tree is initiated, small blocks such as a coding unit with width × height < threshold value, etc. cannot be encoded in palette-based encoding mode. In an embodiment, the threshold is determined such that the local dual tree is not used when the block size of the current coding tree is above this threshold. In some examples, the palette-based coding mode is disabled for small luminance blocks and small chrominance blocks in the local dual tree structure. In some examples, the palette-based coding mode is only disabled for small chroma blocks under the local dual tree structure.

[133] FIG. 10 shows an example syntax for determining the presence of a flag (pred_mode_plt_flag) in a bitstream that indicates whether a palette-based coding mode is used for the current block. In FIG. 10, part (1001) is used to apply the size constraint, and part (1002) is used to apply the chrominance block and the local dual tree constraint. For example, when the block is small (for example, width × height is less than 16 for a luma block, or width × height is less than 16 × SubWidthC × SubHeightC for a chrominance block), the (1001) part may be 0, so there is no pre_mode_plt_flag for the block in the bitstream, and the block is disabled in palette-based coding mode.

[134] In the example, for a chroma block in a local dual tree structure, modeType is MODE_TYPE_INTRA and treeType is DUAL_TREE_CHROMA, so part (1002) may result in 0. Thus, there is no pred_mode_plt_flag for the chroma block in the bitstream, and thus , the palette-based coding mode is disabled for the chrominance block. In another example, for a global dual tree chroma box, modeType is MODE_TYPE_ALL and treeType is DUAL_TREE_CHROMA, so the (1002) part may result in 1. Thus, a chroma box in the global dual tree structure can be encoded in palette-based encoding mode. In another example, for a luminance block in the local dual tree structure, modeType is MODE_TYPE_INTRA and treeType is DUAL_TREE_LUMA, so part (1002) may result in 1. Thus, a luma block in the local dual tree structure can be encoded in a coding mode based on palettes.

[135] According to another aspect of the disclosure, when a local dual tree is used, the maximum allowed predictor palette size and the palette size are reduced, for example, by half. Reducing the size of the palette predictors and the size of the palette can reduce the complexity of the update. In some examples, for each local area (less than the CTU) that allows luminance and chrominance blocks to be encoded separately (e.g., local dual tree), the maximum allowable palette predictor size and palette size are set to half the size for the merged palette case (using a palette, containing luma and chrominance components).

[136] In an embodiment, the maximum allowable palette predictor size and palette size are 63 and 31, respectively, for the merged palette case. Then, with the local dual tree structure, the maximum allowable palette predictor size and palette size for the palette-based luma block coding mode are set to 31 and 15, respectively; and the maximum allowable palette predictor size and palette size for the palette-based chrominance block coding mode are set to 31 and 15, respectively.

[137] FIG. 11 is a flow chart describing a process (1100) according to an embodiment of the disclosure. Process (1100) may be used in block recovery. In various embodiments, process (1100) is performed by a processing circuit, for example, a processing circuit in terminal devices (310), (320), (330) and (340), wherein the processing circuit performs the functions of a video encoder (403), the processing circuit performs the functions of a video decoder ( 410), a processing circuit functions as a video decoder (510), a processing circuit functions as a video encoder (603), and the like. In some embodiments, the process (1100) is implemented in program instructions, such that when the processing circuitry executes the program instructions, the processing circuitry executes the process (1100). The process starts at (S1101) and proceeds to (S1110).

[138] In step (S1110), block prediction information is decoded from the encoded video bitstream.

[139] In step (S1120), it is determined whether to enable the palette-based coding mode for the block based on comparing the block size with a threshold value.

[140] In step (S1130), it is determined whether the block is under the local dual tree structure.

[141] In step (S1140), the block is decoded based on determining whether to enable the palette-based coding mode for the block and determining whether the block is under the local dual tree structure.

[142] In some embodiments, the palette-based coding mode for a block is disabled in response to the block size being less than a threshold. For example, when condition (1001) is not satisfied for a block, the palette-based coding mode is disabled. In some examples, the threshold is determined such that the local dual tree structure is not used when the block size exceeds the threshold.

[143] In an embodiment, the palette-based coding mode for a block is disabled in response to the fact that the block is a chroma block and the size of the chroma block is less than a threshold value. In one example, the palette-based coding mode may be used in small luminance blocks, but not in small chrominance blocks.

[144] In another embodiment, a palette-based coding mode is disabled for a block based on a combination of mode type and block tree type, which indicates that the block is a chrominance block under a local dual tree structure. In the example for block decoding, when the mode type is MODE_TYPE_INTRA and the tree type is DUAL_TREE_CHROMA, so that the condition value (1002) is 0, then the palette-based encoding mode for the block is disabled.

[145] In some embodiments, when the palette-based coding mode is enabled, a flag (eg, pred_mode_plt_flag) is decoded indicating whether the palette-based coding mode is used in the block. In response to a flag indicating the use of the palette-based coding mode in the block, the block is decoded using the palette-based coding mode. In some embodiments, in response to the block being under the local dual tree structure, updating the list of palette predictors may be omitted or omitted.

[146] In some examples, the list of palette predictors is updated based on the palette used in block update decoding, in response to the absence of a local dual tree structure initiator. The example uses the variable localDualTree to determine if a local dual tree has been initiated. For example, when the variable localDualTree is 0, the list of palette predictors can be updated based on the block's current palette. In the example, when the localDualTree variable is set to 1, updating the list of palette predictors based on the block's current palette is skipped. In another example, when the localDualTree variable is equal to 1, the palette predictor list may be updated when the block is a luma block and skipped when the block is a chrominance block.

[147] In another embodiment, in the case of blocks in a local dual tree structure, the maximum palette size and the maximum palette predictor list size are reduced, for example, by half. In some examples, when a palette-based encoding mode is enabled for a block, a flag indicating whether the palette-based encoding mode is used in the block is decoded. Then, when the flag indicates the use of a palette-based coding mode in a block, and the block is in the local binary tree structure, the block is decoded based on a palette having a reduced maximum palette size. For example, the normal maximum palette size is 31, and the reduced maximum palette size is 15. In addition, the palette predictor list is updated based on the palette used in block decoding, and the palette predictor list size is limited by the reduced maximum size for the palette predictor list. For example, the normal maximum size of a palette predictor list is 63, but the reduced maximum size of a palette predictor list is 31.

[148] Then, the process proceeds to (S1199) and ends.

[149] The methods described above may be implemented in computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, in FIG. 12 shows a computer system (1200) suitable for implementing certain embodiments of the disclosed subject matter.

[150] Computer software may be coded using any suitable machine code or computer language, which may be the object of assembly, compilation, linking, or similar mechanisms to produce code containing instructions that can be executed directly or through interpretation, execution of microcode, and and the like, one or more computer central processing units (CPUs), graphics processing units (GPUs), and the like.

[151] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, IoT devices, and the like.

[152] The components shown in FIG. 12 for the computer system (1200) are exemplary in nature and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing embodiments of the present disclosure. The configuration of the components is also not to be interpreted as having any dependency or requirement relating to any one or combination of the components illustrated in the exemplary embodiment of the computer system (1200).

[153] The computer system (1200) may include certain human-machine interface input devices. Such a human-machine interface input device may respond to input from one or more human users through, for example, tactile input (e.g., keystrokes, swipes, movement of a control glove), audio input (e.g., voice, handclaps), visual input (for example: gestures), olfactory input (not shown). HMI devices can also be used to capture certain media not necessarily directly related to conscious human input, such as sound (e.g. speech, music, ambient sound), images (e.g. scanned images, photographic images taken from a camera for still images), video (eg 2D video, 3D video including stereoscopic video).

[154] Human interface input devices may include one or more of the following (only one of each is shown): keyboard (1201), mouse (1202), trackpad (1203), touch screen (1210), control glove (not shown), joystick (1205), microphone (1206), scanner (1207), camera (12308).

[155] The computer system (1200) may also include certain human-machine interface output devices. Such human interface output devices can stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell/taste. Such human interface output devices may include tactile output devices (e.g., tactile feedback from a touch screen (1210), operating glove (not shown), or joystick (1205), but there may also be tactile feedback devices that do not serve as input devices), audio output devices (for example: speakers (1209), headphones (not shown)), image output devices (for example, screens (1210), including CRT screens, LCD screens, plasma screens, OLED - screens, each with or without touchscreen input capability, each with or without tactile feedback capability - some of which may be capable of outputting 2D visual output or more than 3D output by means such as stereographic output ; virtual reality goggles (not shown), holographic displays and smoke tanks (not shown)) and printers (not shown).

[156] The computer system (1200) may also include human accessible storage devices and associated media such as optical media including CD/DVD ROM/RW (1220) with CD/DVD or similar media (1221), flash drive (1222), removable hard drive or solid state drive (1223), previously developed magnetic media such as tape and floppy disk (not shown), specialized ROM/ASIC/PLD based devices such as dongles (not shown) and so on.

[157] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the subject matter disclosed herein does not encompass transmission media, carrier waves, or other time signals.

[158] The computer system (1200) may also include an interface to one or more communication networks. Networks can be, for example, wireless, wired, optical. In addition, networks can be local, wide area, metropolitan, automotive, industrial, real-time, latency tolerant, and so on. Example networks include local area networks such as Ethernet, wireless LANs, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., wired television or wireless digital wide area networks including cable television, satellite television and terrestrial TV broadcasting, automotive and industrial equipment, including CAN bus, etc. Some networks typically require external network interface adapters that are connected to certain general purpose data ports or peripheral buses (1249) (for example, computer system USB ports (1200)); others are typically integrated into the computer system core (1200) by being attached to a system bus as described below (eg, an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system (1200) can communicate with other entities. Such communication can be unidirectional, receive-only (for example, broadcast television), unidirectional, send-only (for example, CAN bus to certain CAN-bus devices), or bidirectional, for example, to other computer systems using a local or wide area digital network. . Each of these networks and network interfaces may use specific protocols and protocol stacks as described above.

[159] The aforementioned human-machine interface devices, human-accessible storage devices, and network interfaces may be attached to the core (1240) of the computer system (1200).

[160] The core (1240) may include one or more central processing units (CPUs) (1241), graphics processing units (GPUs) (1242), dedicated programmable gate programmable area (FPGA) processors (1243), hardware ( 1244), etc. These devices, along with read-only memory (ROM) (1245), random access memory (1246), internal storage devices such as internal non-user hard drives, solid state drives, and the like. (1247) can be connected via the system bus (1248). In some computer systems, the system bus (1248) may be available as one or more physical connectors to allow expansion with additional CPUs, GPUs, and the like. Peripherals can be connected either directly to the system bus (1248) of the kernel, or via a peripheral bus (1249). Peripheral bus architectures include PCI, USB, and the like.

[161] CPUs (1241), GPUs (1242), FGPAs (1243), and accelerators (1244) may execute certain instructions, which in combination may constitute the aforementioned computer code. This computer code may be stored in ROM (1245) or RAM (1246). Transient data may also be stored in RAM (1246), while permanent data may be stored, for example, in an internal mass storage (1247). Fast storage and retrieval to any of the storage devices can be enabled through the use of cache memory, which can be tightly coupled to one or more CPU (1241), GPU (1242), mass storage (1247), ROM (1245), RAM (1246) etc.

[162] The computer-readable medium may contain computer code for performing various computer-implemented operations. The media and computer code may be specially designed and constructed for the purposes of this disclosure, or they may be well known and available to those skilled in the art of computer software.

[163] By way of example, and not limitation, a computer system having an architecture (1200), and in particular a core (1240), may provide functionality through processor(s) (including CPUs, GPUs, FPGA accelerators, and the like). .) executing software embodied on one or more tangible computer-readable media. Such computer-readable media may be media associated with user accessible mass storage as described above and certain kernel storage (1240) that are non-transitory in nature, such as internal kernel mass storage (1247) or ROM (1245). Software implementing various embodiments of the present disclosure may be stored on such devices and executed by the kernel (1240). The computer-readable medium may include one or more memory devices or microchips, as appropriate. The software may cause the core (1240), and in particular the processors therein (including CPUs, GPUs, FPGAs, and the like), to execute certain processes, or certain parts of the specific processes described herein, including the definition of data structures stored in RAM (1246) and changing such data structures in accordance with the processes defined by the software. In addition or alternatively, a computer system may provide functionality as a result of logic embedded in or otherwise embodied in hardware circuitry (e.g., an accelerator (1244)) that may operate in place of or in conjunction with software to perform certain processes or individual parts of the specific processes described in this document. The link to the software may include logic and vice versa where applicable. A reference to a computer-readable medium may include circuitry (eg, an integrated circuit (IC)) storing software to execute, circuitry implementing logic to execute, or both, where applicable. The present disclosure includes any suitable combination of hardware and software.

Appendix A: Abbreviations

JEM: Joint Research Model

VVC: Universal Video Coding

BMS: test suite

MV: motion vector

HEVC: High Efficiency Video Coding

SEI: additional extended information

VUI: video property information

GOPs: groups of images

TUs: conversion units,

PUs: prediction units

CTUs: coding tree units

CTBs: coding tree blocks

PBs: prediction blocks

HRD: Hypothetical Reference Decoder

SNR: signal to noise ratio

CPUs (CPUs): central processing units

GPUs (GPU): graphics processing units

CRT (CRT): cathode ray tube

LCD (LCD): liquid crystal display

OLED: organic light emitting diode

CD: CD

DVD: Digital Video Disc

ROM (ROM): Read Only Memory

RAM (RAM): Random Access Memory

ASIC: Application Specific Integrated Circuit

PLD: Programmable Logic Device

LAN: local area network

GSM: Global System for Mobile Communications

LTE: long-term development

CANBus (CAN-bus): LAN bus of the controller

USB: Universal Serial Bus

PCI: Peripheral Component Connection

FPGA (FPGA): Field Programmable Gate Arrays

SSD: solid state drive

IC: integrated circuit

CU: coding unit.

[164] Although this description discloses several exemplary embodiments, there are variations, permutations, and various replacement equivalents that fall within the scope of the disclosure. Thus, it will be appreciated that those skilled in the art will be able to develop numerous systems and methods that, although not expressly shown or described herein, embody the principles of the disclosure and thus are within its spirit and scope.

Claims (97)

1. Method for decoding video in a decoder, comprising: decoding, by the processor, block prediction information from the encoded video bitstream; determining, by the processor, whether to enable the block in a palette-based coding mode based on a comparison of the block size with a threshold value; determining, by the processor, whether the block is under the local dual tree structure based on the prediction information; And decoding, by the processor, the block based on determining whether to enable the paletted encoding mode for the block, and determining whether the block is under a local dual tree structure. 2. The method according to p. 1, further comprising: disabling the paletted coding mode for the block in response to the block size being less than a threshold. 3. The method of claim 1, wherein the threshold is determined to disable the local dual tree structure in response to a block size exceeding the threshold. 4. The method according to p. 1, further comprising: disabling the paletted coding mode for the block in response to the fact that the block is a chrominance block and the size of the chrominance block is less than a threshold value. 5. The method according to p. 1, further comprising: inhibiting a palette-based coding mode for a block based on a combination of mode type and block tree type, which indicates that the block is a chrominance block under a local dual tree structure. 6. The method according to p. 1, further comprising: decoding, from the encoded video bitstream, a flag indicating whether a palette-based coding mode is used in the block, in response to enabling the palette-based coding mode; decoding the block using a palette-based coding mode in response to a flag indicating the use of a palette-based coding mode in the block; And skipping updating the list of palette predictors in response to the block being under the local dual tree structure. 7. The method according to p. 6, further comprising: updating, in response to not initiating a local dual tree structure, a list of palette predictors based on the palette used in block decoding. 8. The method according to p. 1, further comprising: decoding, from the encoded video bitstream, a flag indicating whether a palette-based coding mode is used in the block, in response to enabling the palette-based coding mode; And decoding the block based on a palette having a reduced maximum size, in response to a flag indicating the use of a palette-based coding mode in the block, and finding the block under a local dual tree structure. 9. The method of claim 8, further comprising: updating the palette predictor list based on the palette used in decoding the block, with the size of the palette predictor list limited to a reduced maximum size. 10. A video decoding device, comprising: processing scheme configured for: decoding block prediction information from the encoded video bitstream; determining whether to allow the block to a palette-based coding mode based on comparing the block size with a threshold value; determining whether the block is under the local dual tree structure based on the prediction information; And decoding the block based on determining whether to enable a palette-based encoding mode for the block, and determining whether the block is under a local dual tree structure. 11. The apparatus of claim 10, wherein the processing circuitry is configured to: inhibiting the paletted coding mode for the block in response to the block size being less than a threshold value. 12. The apparatus of claim 10, wherein the threshold is defined to disable the local dual tree structure in response to a block size exceeding the threshold. 13. The apparatus of claim 10, wherein the processing circuitry is configured to: disable the palette-based coding mode for the block in response to the block being a chrominance block and the size of the chrominance block being less than a threshold value. 14. The apparatus of claim 10, wherein the processing circuitry is configured to: prohibiting a palette-based coding mode for a block based on a combination of mode type and block tree type, which indicates that the block is a chrominance block under a local dual tree structure. 15. The apparatus of claim 10, wherein the processing circuitry is configured to: decoding, from the encoded video bitstream, a flag indicating whether a palette-based coding mode is used in the block, in response to enabling the palette-based coding mode; decoding the block using a palette-based coding mode in response to a flag indicating the use of a palette-based coding mode in the block; And skipping updating the list of palette predictors in response to the block being under the local dual tree structure. 16. The apparatus of claim 15, wherein the processing circuitry is configured to: updating, in response to not initiating a local dual tree structure, a list of palette predictors based on the palette used in block decoding. 17. The apparatus of claim 10, wherein the processing circuitry is configured to: decoding, from the encoded video bitstream, a flag indicating whether a palette-based coding mode is used in the block, in response to enabling the palette-based coding mode; And decoding the block based on a palette having a reduced maximum size, in response to a flag indicating that the block is using a palette-based coding mode and that the block is under a local dual tree structure. 18. The apparatus of claim 17, wherein the processing circuitry is configured to: updating the palette predictor list based on the palette used in decoding the block, the palette predictor list being limited to a reduced maximum size. 19. A non-transitory computer-readable medium that stores instructions that, when executed by a video decoding computer, cause the computer to perform: decoding block prediction information from the encoded video bitstream; determining whether to allow the block to a palette-based coding mode based on comparing the block size with a threshold value; determining whether the block is under the local dual tree structure based on the prediction information; And decoding the block based on determining whether to enable a palette-based encoding mode for the block, and determining whether the block is under a local dual tree structure. 20. The non-transitory computer-readable medium of claim 19, wherein the instructions cause the computer to do: inhibiting a palette-based coding mode for a block based on a combination of mode type and block tree type, which indicates that the block is a chrominance block under a local dual tree structure. 21. A method for encoding video in an encoder, comprising: determining, by the processor, whether the block is under a local dual tree structure; determining, by the processor, whether the block size is less than a predetermined threshold; And allowing, by the processor, a palette-based coding mode to encode the block in response to the fact that the block is not under the local dual tree structure and the block size is less than a predetermined threshold. 22. The method of claim 21, further comprising: disabling a palette-based encoding mode for encoding the block in response to the block being under the local dual tree structure and the block size being less than a predetermined threshold. 23. The method of claim 21, further comprising: disabling a palette-based encoding mode for encoding a block in response to the fact that the block is a particular color component and is under a local dual tree structure. 24. The method of claim 23, further comprising: disabling the palette-based encoding mode for encoding the block in response to the block being a chrominance component and being under a local dual tree structure. 25. The method of claim 23, further comprising: disabling a palette-based encoding mode for block encoding in response to the block being a luma component and being under a local dual tree structure. 26. The method of claim 21, further comprising: encoding the block based on determining whether to allow a palette-based encoding mode for encoding the block, and determining whether the block is under a local dual tree structure. 27. The method of claim 21, wherein the predetermined threshold is determined to disable the local dual tree structure in response to a block size exceeding the predetermined threshold. 28. The method of claim 21, further comprising: determining whether the block area is less than a predetermined threshold. 29. The method of claim 21, further comprising: inhibiting a palette-based coding mode for encoding a block based on a combination of mode type and block tree type, which indicates that the block is a chrominance block under a local dual tree structure. 30. The method of claim 21, further comprising: encoding a flag in the video bitstream that carries an encoded block corresponding to the block to indicate whether the block uses a palette-based coding mode in response to enabling the palette-based coding mode. 31. Device for encoding video, containing: processing scheme configured for: determining whether the block is under a local dual tree structure; determining whether the block size is less than a predetermined threshold value; And permitting a palette-based encoding mode to encode the block in response to the fact that the block is not under the local dual tree structure and the block size is less than a predetermined threshold. 32. The apparatus of claim 31, wherein the processing circuitry is configured to: inhibiting a palette-based coding mode for encoding the block in response to the block being under the local dual tree structure and the block size being less than a predetermined threshold. 33. The apparatus of claim 31, wherein the processing circuitry is configured to: prohibiting a palette-based encoding mode for encoding a block in response to the fact that the block is a particular color component and is under a local dual tree structure. 34. The apparatus of claim 33, wherein the processing circuitry is configured to: prohibiting a palette-based encoding mode to encode the block in response to the block being a chrominance component and being under a local dual tree structure. 35. The apparatus of claim 33, wherein the processing circuitry is configured to: prohibiting a paletted encoding mode to encode the block in response to the block being a luma component and being under a local dual tree structure. 36. The apparatus of claim 31, wherein the processing circuitry is configured to: encoding the block based on determining whether to allow a palette-based encoding mode for encoding the block, and determining whether the block is under a local dual tree structure. 37. The apparatus of claim 31, wherein the processing circuitry is configured to: determining whether the block area is less than a predetermined threshold. 38. The apparatus of claim 31, wherein the predetermined threshold is determined to disable the local dual tree structure in response to a block size exceeding the predetermined threshold. 39. The apparatus of claim 31, wherein the processing circuitry is configured to: inhibiting a palette-based coding mode for encoding a block based on a combination of mode type and block tree type, which indicates that the block is a chrominance block under a local dual tree structure. 40. The apparatus of claim 31, wherein the processing circuitry is configured to: encoding a flag in the video bitstream that carries an encoded block corresponding to the block to indicate whether the block uses a palette-based coding mode in response to enabling the palette-based coding mode. 41. A non-transitory computer-readable medium that stores instructions that, when executed by a video encoding computer, instructs the computer to perform the method according to any one of paragraphs. 21-30.

Images