RU2777188C1 - Method and device for video encoding - Google Patents

Abstract

FIELD: video technology.

SUBSTANCE: invention relates to the field of video encoding/decoding. A video encoding/decoding device is proposed, which includes a processing scheme that decodes the encoded information of the encoding unit (CU) from the bit stream of the encoded video. The encoded information indicates the last position of the non-zero conversion coefficients of the first coding block (CB) from the CU. The processing scheme determines whether the index of the secondary transformation in the encoded information is signaled, based on the last position, and whether to carry out the secondary transformation on the second CB, based on whether the index of the secondary transformation in the encoded information is signaled. When it is determined that a secondary transformation is to be carried out, the processing scheme performs a secondary transformation on the second CB and reconstructs the second CB. In response to the determination that the secondary transformation is not subject to implementation, the processing scheme reconstructs the second CB without performing a secondary transformation on the second CB.

EFFECT: an increase in encoding efficiency through internal prediction and, consequently, the entropy encoding technology used to translate code words into a bitstream.

8 cl, 39 dwg, 3 tbl

Description

INCLUSION BY LINK

[0001] The present invention claims priority of U.S. Patent Application No. 16/838,755, "Method and Apparatus for Video Coding", filed April 2, 2020, which claims priority of U.S. Provisional Application No. 62/829,435, "Modifications on the Secondary Transform", filed April 4, 2019. The contents of previous applications are hereby incorporated by reference in their entirety.

FIELD OF TECHNOLOGY TO WHICH THE INVENTION RELATES

[0002] This application describes embodiments generally related to video coding.

BACKGROUND OF THE INVENTION

[0003] The description of the prior art is provided herein for the purpose of presenting the overall context of the invention. The work of the authors of the invention, to the extent that it is described in this section, as well as aspects of the description that cannot be qualified as state of the art at the time of filing, neither expressly nor impliedly recognized as the state of the art underlying the present invention.

[0004] Encoding and decoding of the video signal may be performed using inter-picture prediction with motion compensation. Uncompressed digital video may include a sequence of images, each of which has a spatial size of, for example, 1920×1080 luma samples and associated chrominance samples. The image sequence may have a fixed or variable image rate (informally also referred to as frame rate), such as 60 images per second, or 60 Hz. Uncompressed video has significant bitrate requirements. For example, 1080p60 4:2:0 video with 8 bits per sample (resolution of 1920x1080 luminance samples at 60Hz frame rate) requires about 1.5Gbps of bandwidth. An hour of such video requires more than 600 GB of storage.

[0005] One goal of video encoding and decoding may be to reduce redundancy in the input video signal by compressing. Compression can help to alleviate the aforementioned bandwidth or storage requirements, in some cases by two orders of magnitude or more. You can use both lossless and lossy compression, as well as a combination of both. Lossless compression refers to techniques for reconstructing an exact copy of the original signal from the compressed original signal. When lossy compression is used, the reconstructed signal may not be identical to the original signal, but the discrepancy between the original and reconstructed signals is small enough that the reconstructed signal can be used for the intended application. Lossy compression is widely used for video. The amount of distortion allowed depends on the application; for example, users of some custom streaming applications can tolerate higher distortion than users of broadcast TV applications. The achievable compression ratio may reflect that a higher allowed/allowable distortion may result in a higher compression ratio.

[0006] The video encoder and decoder may use techniques from several broad categories including, for example, motion compensation, transformation, quantization, and entropy coding.

[0007] Video codec technologies may include techniques known as intra coding. With internal (intra-) coding, sample values are represented without reference to samples or other data from previously reconstructed reference pictures. In some video codecs, the image is spatially divided into blocks of samples. When all blocks of samples are encoded in intra-mode, the picture is called an intra-picture. Intra-pictures and their derivatives, such as pictures obtained in the decoder independent update mode, can be used to reset the decoder state, allowing them to be used as the first picture in the encoded video bitstream and video session, or as a still picture. The intra-block samples may be transformed and the transform coefficients may be quantized prior to entropy coding. Intra-prediction may be a technique for minimizing sample values in the pre-transform region. In some cases, the smaller the value of DC (constant component) after transformation, and the smaller the AC (variable components) coefficients, the fewer bits are required for a given quantization step size to represent a block after entropy coding.

[0008] Conventional intra coding known, for example, from MPEG-2 generation coding technologies, does not use intra prediction. However, some more recent video compression techniques include techniques that attempt, for example, from ambient sample data and/or metadata obtained during encoding/decoding of spatially adjacent and preceding data blocks in decoding order. Such methods are hereinafter referred to as "internal prediction" methods. Note that, in at least some cases, intra prediction is only performed using reference data from the current image to be reconstructed, and not from reference images.

[0009] There may be many different forms of intra prediction. When more than one of these methods can be used in a given video coding technology, the method used can be encoded in intra prediction mode. In some cases, modes may have sub-modes and/or parameters that may be encoded separately or included in a mode codeword. The use of a particular codeword for a given combination of modes/submodes/parameters can affect the gain in coding efficiency through intra prediction and hence the entropy coding technology used to translate the codewords into the bitstream.

[0010] A certain mode of intra prediction was first specified in H.264, refined in H.265, and further refined in more modern coding technologies, for example, the joint exploration model (JEM, joint exploration model), universal video coding (VVC, versatile video coding) and a set of standards (BMS, benchmark set). The predictor block can be formed using the values of adjacent samples belonging to already available samples. The values of neighboring samples are copied into the predictor block according to the direction. The reference to the direction to be used may be encoded in the bitstream, or may be self-predictable.

[0011] In the lower right corner of FIG. 1A shows a subset of the nine predictor directions of the set known from 33 possible H.265 predictor directions (corresponding to 33 angular modes out of 35 intra modes). The point (101) where the arrows converge represents the predicted count. The arrows represent the direction from which the reading is predicted. For example, an arrow (102) indicates that the reading (101) is predicted from the reading or readings located at the top right, at an angle of 45 degrees to the horizontal. Likewise, arrow (103) indicates that sample (101) is predicted from the sample(s) located down to the left of sample (101), at an angle of 22.5 degrees to the horizontal.

[0012] In the upper left corner of FIG. 1A depicts a square block (104) of 4×4 samples (indicated by the heavy dotted line). The square block (104) includes 16 samples, each labeled with an "S", a y-axis position (eg, row index) and an x-axis position (eg, column index). For example, sample S21 is the second sample on the Y axis (top) and the first sample (left) on the X axis. Similarly, sample S44 is the fourth sample in block (104) on both the Y and X axes. Since the block has a size of 4×4 samples , S44 is located at the bottom right. Additionally, reference readings are shown that follow a similar numbering scheme. The reference sample is denoted by the letter R, its Y-position (eg, row index) and X-position (column index) relative to the block (104). In both H.264 and H.265, the predicted samples are adjacent to the block to be reconstructed; so there is no need to use negative values.

[0013] Intra-picture prediction may work by copying sample reference values from neighboring samples indicated by the signaled prediction direction. For example, suppose the coded video bitstream includes signaling which, for this block, indicates the prediction direction indicated by arrow (102) - that is, the samples are predicted from the predicted sample(s) located at the top right, at an angle of 45 degrees to the horizontal. In this case, samples S41, S32, S23 and S14 are predicted from the same reference sample R05. Then, the sample S44 is predicted from the reference sample R08.

[0014] In some cases, the values of a plurality of reference samples may be combined, such as by interpolation, to calculate a reference sample; especially when the directions are not multiples of 45 degrees.

[0015] The number of possible directions increases as video coding technologies develop. In H.264 (2003), nine different directions could be represented. In H.265 this has increased to 33 (2013), and JEM/VVC/BMS, at the time of writing this description, can support up to 65 destinations. Experiments have been made to identify the most likely directions, and some entropy encoding techniques are used to represent these likely directions with a small number of bits, at the cost of losing some of the less likely directions. Additionally, the directions themselves can sometimes be predicted from adjacent directions used in adjacent already decoded blocks.

[0016] FIG. 1B is a diagram (180) showing 65 intra prediction directions according to JEM to illustrate an increase in the number of prediction directions over time.

[0017] The mapping of intra prediction direction bits in the encoded video bitstream that represent the direction may differ from one video coding technology to another video coding technology; and may cover, for example, from simple forward prediction direction mapping to intra prediction mode, code words, complex adaptive schemes involving most likely modes, and similar techniques. However, in all cases, there may be some directions that are statistically less likely to occur in video content than for some other directions. Because the goal of video compression is to reduce redundancy, these less likely directions will, in a well-performing video coding technology, be represented by more bits than more likely directions.

[0018] Video encoding and decoding may be performed using inter-picture prediction with motion compensation. Motion compensation may be a lossy compression technique and may refer to techniques where a block of sample data from a previously reconstructed image or part of it (reference image) subjected to a spatial shift in the direction indicated by a motion vector (hereinafter MV, motion vector) is used to predicting a newly reconstructed image or part of an image. In some cases, the reference image may be identical to the image currently being reconstructed. MVs may have two X and Y dimensions, or three dimensions, with the third indicating the reference picture used (the latter, indirectly, may be a temporal dimension).

[0019] In some video compression techniques, the MV applicable to a certain region of sample data can be predicted from other MVs, such as those associated with another region of sample data, spatially adjacent to the reconstructed region and preceding that MV in decoding order. This can significantly reduce the amount of data required for MV encoding, thereby eliminating redundancy and increasing the compression ratio. MV prediction can work efficiently, for example, because when encoding the input video signal received from the camera (known as natural video), there is a statistical probability that areas larger than the area to which a single MV is applied move in similar direction, and therefore in some cases can be predicted using a similar motion vector obtained from the MV of the neighboring area. As a result, the MV found for a given region is similar or identical to the MV predicted from the surrounding MVs, which in turn can be represented, after entropy encoding, in fewer bits than would be used when encoding the MV directly. In some cases, MV prediction can serve as an example of lossless compression of a signal (namely, MV) obtained from the original signal (namely, a stream of samples). In other cases, the MV prediction itself may be lossy, for example, due to rounding errors when computing a predictor from multiple surrounding MVs.

[0020] Various MV prediction mechanisms are described in H.265/HEVC (ITU-T Rec. H.265, "High Efficiency Video Coding", December 2016). Of the many MV prediction mechanisms offered in H.265, a technique hereinafter referred to as "spatial pooling" is described.

[0021] Referring to FIG. 2, the current block (201) contains samples found by the encoder during the motion search process to predict from a previous block of the same size that was spatially shifted. Instead of encoding this MV directly, the MV can be obtained from the metadata associated with one or more reference pictures, e.g., from the most recent (in decoding order) reference picture, using the MV associated with any of the five surrounding samples, labeled A0, A1, and B0, B1, B2 (202-206, respectively). In H.265, MV prediction may use predictors from the same reference picture as the neighboring block uses.

SUMMARY OF THE INVENTION

[0022] Aspects of the invention provide methods and apparatus for encoding/decoding a video signal. In some examples, the video decoding apparatus includes a processing circuit. The processing circuit may decode the encoded coding unit (CU) information from the encoded video bitstream. The encoded information may indicate the last position of the non-zero transform coefficients of the first coding block (CB) in the CU. The processing circuit may determine whether a secondary transform index is signaled in the encoded information based on the last position. The processing circuit may determine whether to perform the secondary transform on the second CB based on whether it is determined that the secondary transform index is to be signaled in the encoded information. In response to the determination that a secondary transformation is to be performed, the processing circuit may perform a secondary transformation on the second CB and a reconstruction of the second CB. In response to the determination that the secondary transformation is not to be performed, the processing circuit may reconstruct the second CB without performing the secondary transformation on the second CB.

[0023] According to an embodiment, the processing circuit may determine whether the horizontal component of the last position is less than the first threshold, and whether the vertical component of the last position is less than the second threshold. In response to determining that the horizontal component is less than the first threshold and determining that the vertical component is less than the second threshold, the processing circuit may determine that the secondary transform index is not signaled in the encoded information.

[0024] According to an embodiment, the processing circuit may determine whether the sum of the horizontal component and the vertical component of the last position is less than a threshold. In response to determining that the sum is less than a threshold, the processing circuit may determine that the secondary transform index is not signaled in the encoded information.

[0025] According to an embodiment, the processing circuit may determine whether at least one of (i) the horizontal component and (ii) the vertical component of the last position is less than a threshold. In response to determining that at least one of them is less than a threshold, the processing circuit may determine that the secondary transform index is not signaled in the encoded information.

[0026] According to an embodiment, the processing circuit may determine whether at most one of (i) the horizontal component and (ii) the vertical component of the last position is less than a threshold. In response to determining that at most one of them is less than a threshold, the processing circuit may determine that the secondary transform index is not signaled in the encoded information.

[0027] According to an embodiment, the first CB is a luminance block, the last position is the last luminance position for the luminance block. The processing circuit may determine whether the secondary transform index is signaled based on the last luma position.

[0028] According to an embodiment, the first CB is a luminance block. The last position is the last brightness position for the brightness block. The CU further includes a chrominance block. The encoding information further indicates the last chroma position of the non-zero transform coefficients for the chrominance block. The processing circuit may determine whether the secondary transform index is signaled based on the last luminance position and the last chrominance position.

[0029] In some examples, the video decoding device includes a processing circuit. The processing circuit may decode the coding unit (CU) encoding information from the encoded video bitstream. The encoding information may indicate the size of the CU. The processing circuit may determine whether the secondary transformation is allowed based on the CU size and the CU size threshold, wherein when the CU size is less than or equal to the CU size threshold, it is determined that the secondary transformation is allowed, and when the CU size is larger than the CU size threshold, it is determined that the secondary conversion is not allowed.

[0030] According to an embodiment, the CU size threshold is the maximum size of a transform unit in the CU.

[0031] According to an embodiment, when the CU size is less than or equal to the CU size threshold, the processing circuitry may determine the number of non-zero transform coefficients for at least one CB in the CU, wherein the size of each of the at least one CB is greater than or equal to the first threshold. When the number of non-zero transform coefficients is less than the second threshold, the processing circuit may determine that the secondary transform index is not signaled in the encoded information. In the example, the encoding information indicates that the color format for the CU is YUV 4:2:0. The CU includes a luminance block and two chrominance blocks. The processing circuit may determine whether the first luminance block dimension is equal to 4 and the second luminance block dimension is equal to N, where N is greater than or equal to 4. In response to determining that the first and second dimensions are 4 and N, respectively, the processing circuit may determine the number of non-zero transform coefficients only from the luminance block, where at least one CB is a luminance block.

[0032] According to an embodiment, the encoding information indicates that the color format for the CU is YUV 4:2:2. The CU includes a luminance block and two chrominance blocks. The processing circuit may determine whether the luminance block size is 4×N, where N is greater than or equal to 4. In response to determining that the luminance block size is 4×N, where N and 4 are the height and width of the luminance block, respectively, the processing circuit can only determine the number of non-zero transform coefficients from the luma block. At least one CB is a luminance block.

[0033] Aspects of the invention also provide a computer-readable medium that stores instructions that, when executed by a video decoding computer, cause the computer to perform any of the video decoding methods.

BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Additional features, nature and various advantages of the disclosed invention will be apparent from the following detailed description and the accompanying drawings, in which:

[0035] FIG. 1A is a diagram of an exemplary subset of intra prediction modes;

[0036] FIG. 1B is a diagram of intra prediction directions;

[0037] FIG. 2 is a diagram of the current block and its surrounding space merge candidates in one example;

[0038] FIG. 3 is a simplified block diagram of a communications system (300) in accordance with an embodiment;

[0039] FIG. 4 is a simplified block diagram of a communications system (400) in accordance with an embodiment;

[0040] FIG. 5 is a simplified block diagram of a decoder according to an embodiment;

[0041] FIG. 6 is a simplified block diagram of an encoder according to an embodiment;

[0042] FIG. 7 is a block diagram of an encoder according to another embodiment;

[0043] FIG. 8 is a block diagram of a decoder according to another embodiment;

[0044] FIG. 9 is an example of a transformation unit syntax according to an embodiment;

[0045] FIG. 10A-10C show an example of a residual encoding syntax according to an embodiment;

[0046] FIG. 11A-11B show examples of primary conversions according to an embodiment;

[0047] FIG. 12A-12E show an example of a conversion process according to an embodiment;

[0048] FIG. 13 is an exemplary transformation encoding process (1300);

[0049] FIG. 14 is an exemplary transformation encoding process (1400);

[0050] FIG. 15A shows an exemplary reduced forward transform process (1501) and an exemplary reduced inverse transform process (1502);

[0051] FIG. 15B-15C are examples of reduced secondary transforms in accordance with some embodiments;

[0052] FIG. 15D is an exemplary transform set selection table (1550);

[0053] FIG. 16A illustrates exemplary intra prediction directions and intra prediction modes according to an embodiment;

[0054] FIG. 16B illustrates exemplary intra prediction directions and corresponding intra prediction modes in accordance with an embodiment;

[0055] FIG. 17 is an example of 4 reference lines for a coding block (1710) in accordance with an embodiment;

[0056] FIG. 18 is a table that relates the number of parts into which a block is divided to its size, according to an embodiment;

[0057] FIG. 19 is an example of splitting a block having a size of 4x8 or 8x4;

[0058] FIG. 20 is another example of splitting a block having a size larger than 4×8 and 8×4;

[0059] FIG. 21A-21D are examples of various YUV formats;

[0060] FIG. 22 is a block diagram of the operations of a process (2200) in accordance with an embodiment;

[0061] FIG. 23 is a flowchart of the operations of a process (2300) in accordance with an embodiment;

[0062] FIG. 24 is a diagram of a computer system in accordance with an embodiment.

DETAILED DESCRIPTION OF EMBODIMENTS

[0063] FIG. 3 shows a simplified block diagram of a communication system (300) according to an embodiment of the present invention. The communication system (300) includes a plurality of terminal devices that can communicate with each other, for example, via a network (350). For example, the communication system (300) includes a first pair of terminals (310) and (320) interconnected via a network (350). In the example shown in FIG. 3, the first pair of terminals (310) and (320) performs a unidirectional data transfer. For example, a terminal device (310) may encode video data (eg, a video stream captured by a terminal device (310)) for transmission to another terminal device (320) via a network (350). The encoded video data may be transmitted in the form of one or more encoded video bitstreams. The terminal device (320) may receive the encoded video data from the network (350), decode the encoded video data to recover the video images, and display the video images according to the recovered video data. One-way communication may be inherent in media service applications and the like.

[0064] In another example, the communication system (300) includes a second pair of terminals (330) and (340) that carry out bi-directional transmission of encoded video data that may occur, for example, during a video conference. For bidirectional data communication, by way of example, each terminal of terminals (330) and (340) may encode video data (eg, a video stream captured by the terminal) for transmission to the other terminal of terminals (330) and (340). ) via the network (350). Each terminal of terminals (330) and (340) may also receive encoded video data transmitted by another terminal from terminals (330) and (340) and may decode the encoded video data to reconstruct video images and display video images on an available display device according to recovered video data.

[0065] In the example shown in FIG. 3, terminal devices (310), (320), (330), and (340) can be illustrated as servers, personal computers, and smartphones, but this does not limit the principles of the present invention. Embodiments of the present invention find application in laptop computers, tablet computers, media players and/or dedicated video conferencing equipment. Network (350) represents any number of networks that carry encoded video data between terminals (310), (320), (330) and (340), including, for example, wired and/or wireless communication networks. The communication network (350) allows data to be exchanged in a circuit-switched and/or packet-switched mode. Exemplary networks include telecommunications networks, local area networks, wide area networks, and/or the Internet. For the purposes of this discussion, the architecture and topology of the network (350) may not be relevant to the present invention unless specifically noted below.

[0066] FIG. 4 shows, as an example of the application of the disclosed invention, the placement of a video encoder and a video decoder in a streaming environment. The disclosed invention may be equally applicable to other video processing applications, including, for example, video conferencing, digital television, storage of compressed video on digital media, including CD, DVD, memory card, etc., etc.

[0067] The streaming system may include a capture subsystem (413), which may include a video source (401), such as a digital camera, producing, for example, an uncompressed video stream (402). For example, the video stream (402) includes samples taken by a digital camera. The video stream (402), shown in bold to highlight the large amount of data compared to the encoded video data (404) (or encoded video bitstreams), may be processed by an electronic device (420) that includes a video encoder (403) connected to source (401) of the video signal. The video encoder (403) may include hardware, software, or a combination thereof for providing or implementing aspects of the disclosed invention, as described in more detail below. The encoded video data (404) (or encoded video bitstream (404)), depicted with a thin line to emphasize the smaller amount of data compared to the video stream (402), may be stored on the streaming server (405) for future use. One or more streaming client subsystems, such as client subsystems (406) and (408) in FIG. 4 may access the streaming server (405) to retrieve copies (407) and (409) of the encoded video data (404). The client subsystem (406) may include a video decoder (410), such as in an electronic device (430). The video decoder (410) decodes the incoming copy (407) of the encoded video data and creates an outgoing video stream (411) that can be rendered on a display (412) (eg, a display screen) or other rendering device (not shown). In some streaming systems, the encoded video data (404), (407), and (409) (eg, video bitstreams) may be encoded according to one or another video coding/compression standard. Examples of these standards include ITU-T Recommendation H.265. For example, a video coding standard is being developed, officially called Universal Video Coding (VVC, Universal Video Coding). The disclosed invention may be used in the context of VVC.

[0068] Note that electronic devices (420) and (430) may include other components (not shown). For example, electronic device (420) may include a video decoder (not shown), and electronic device (430) may also include a video encoder (not shown).

[0069] FIG. 5 shows a block diagram of a video decoder (510) according to an embodiment of the present invention. The video decoder (510) may be included in the electronic device (530). The electronic device (530) may include a receiver (531) (eg, receiver circuits). Video decoder (510) may be used instead of video decoder (410) in the example shown in FIG. four.

[0070] The receiver (531) may receive one or more encoded video sequences for decoding by the video decoder (510); in the same or another embodiment, one coded video sequence at a time, where the decoding of each coded video sequence is independent of other coded video sequences. The encoded video sequence may be received from a channel (501), which may be a hardware/software link to a storage device where the encoded video data is stored. The receiver (531) may receive encoded video data with other data, such as encoded audio data and/or auxiliary data streams, which may be relayed to appropriate users and entities (not shown). The receiver (531) may separate the encoded video sequence from other data. To combat network jitter, a buffer memory (515) may be connected between the receiver (531) and the entropy decoder/analyzer (520) (hereinafter "analyzer (520)"). In some applications, the buffer memory (515) is included in the video decoder (510). In others, it may not be part of the video decoder (510) (not shown). In other embodiments, there may be a buffer memory (not shown) outside the video decoder (510), for example, to combat network jitter, in addition to another buffer memory (515) within the video decoder (510), for example, to control playback timing. When the receiver (531) receives data from a storage/relay device with sufficient bandwidth and controllability, or from an isosynchronous network, buffer memory (515) may not be needed or may be small. For use in best attempt packet networks, such as the Internet, buffer memory (515) may be required, may be relatively large, and may have a predominantly adaptive size, and may at least partially be implemented in an operating system or similar elements (not shown) outside video decoder (510).

[0071] The video decoder (510) may include an analyzer (520) for reconstructing the symbols (521) from the encoded video sequence. The categories of these symbols include information used to control the operation of the video decoder (510), and possibly information to control a visualization device, such as a visualization device (512) (for example, a display screen) that is not an integral part of the electronic device (530), but may be connected to an electronic device (530) as shown in FIG. The control information for the rendering device(s) may be Supplemental Enhancement Information (SEI) messages or Video Usability Information (VUI) parameter set fragments (not shown). The analyzer (520) may analyze/entropy decode the received encoded video sequence. Encoding of the encoded video sequence may be in accordance with a video coding technology or standard, and may follow various principles, including variable run length coding, Huffman coding, arithmetic coding with or without context sensitivity, and so on. The analyzer (520) can extract from the encoded video sequence a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder based on at least one parameter corresponding to the group. Subgroups may include groups of pictures (GOP, Groups of Pictures), images, tiles, slices, macroblocks, coding units (CU, Coding Units), blocks, transformation units (TU, Transform Units), prediction units (PU, Prediction Units ) etc. The analyzer (520) can also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, and so on.

[0072] The analyzer (520) may perform an entropy decoding/analysis operation of the video sequence received from the buffer memory (515) to create symbols (521).

[0073] Several different modules may be used for symbol reconstruction (521) depending on the type of encoded video image or parts thereof (eg: inter- and intra-pictures, inter- and intra-block) and other factors. Which modules are used, and how, may be determined by the subgroup management information extracted from the encoded video sequence by the analyzer (520). The flow of such subgroup management information between the analyzer (520) and multiple modules is not shown hereinafter for simplicity.

[0074] In addition to the previously mentioned functional blocks, the video decoder (510) can be fundamentally divided into several functional modules, as described below. In a practical implementation operating under commercial constraints, many of these modules interact closely with one another and can, at least partially, be nested within one another. However, for the purposes of describing the disclosed invention, a fundamental division into the following functional modules is appropriate.

[0075] The first module is the scaling/inverse transform module (551). The scaling/inverse transform module (551) receives the quantized transform coefficient as well as control information including the transform used, block size, quantization factor, quantization scaling matrices, and so on. as character(s) (521) from the analyzer (520). The scaling/inverse transform module (551) can output blocks containing sample values that can be input to the aggregator (555).

[0076] In some cases, the output samples of the block (551) scaling / inverse transformation may refer to an intra-coded block; that is, a block that does not use predicted information from previously reconstructed images, but may use predicted information from previously reconstructed portions of the current image. Such predicted information may be provided by an intra-picture prediction module (552). In some cases, the intra-image prediction module (552) generates a block of the same size and shape as the block to be reconstructed using the previously reconstructed environment information retrieved from the current image buffer (558). The current picture buffer (558) buffers, for example, a partially reconstructed current picture and/or a fully reconstructed current picture. The aggregator (555) in some cases adds, for each sample, the prediction information generated by the intra prediction module (552) to the output sample information provided by the scaling/inverse transform module (551).

[0077] In other cases, the output samples of the scaling/inverse transform module (551) may refer to an intra-coded block, possibly with motion compensation. In such a case, the motion compensation prediction module (553) may access the reference picture memory (557) to retrieve samples used for prediction. After applying motion compensation to the extracted samples according to the symbols (521) associated with the block, these samples can be added by the aggregator (555) to the output signal of the scaling / inverse transform module (551) (in this case referred to as residual samples or residual signal) for generation of output sample information. The addresses in the reference picture memory (557) from where the motion compensation prediction module (553) retrieves the predicted samples may be governed by the motion vectors available to the motion compensation prediction module (553) in the form of symbols (521), which may have, for example, X, Y components and reference image. Motion compensation may also include interpolation of sample values retrieved from reference picture memory (557) when accurate sub-sample motion vectors, motion vector prediction mechanisms, etc. are used.

[0078] Various loop filtering techniques can be applied to the aggregator (555) output samples in the loop filter module (556). The video compression technologies may include deblocking filter technologies driven by parameters included in the coded video sequence (also referred to as the coded video bitstream) and made available to the loop filter module (556) as symbols (521) from the analyzer (520), but may also respond to metainformation obtained during decoding of the previous (in the order of decoding) parts of the encoded image or encoded video sequence, as well as respond to previously reconstructed and contour filtered sample values.

[0079] The loop filter module (556) may output a stream of samples that may be fed to the renderer (512) as well as stored in the reference picture memory (557) for use in future inter-picture prediction.

[0080] Some encoded pictures, when fully reconstructed, can be used as reference pictures for future prediction. For example, when the encoded picture corresponding to the current picture is completely reconstructed and the encoded picture is identified as a reference picture (e.g., by the analyzer (520)), the current picture buffer (558) may become part of the reference picture memory (557), and a fresh current picture buffer may be re-allocated before the reconstruction of the next encoded picture starts.

[0081] The video decoder (510) may perform decoding operations according to a predetermined video compression technology, such as the ITU-T Rec. H.265. The encoded video sequence may conform to the syntax specified by the used video compression technology or standard, in the sense that the encoded video sequence may adhere to both the syntax of the video compression technology or standard and the profiles documented in the video compression technology or standard. In particular, a profile may select certain tools as tools available for use only under that profile, from among all the tools available in a video compression technology or standard. It may also be necessary for consistency that the complexity of the encoded video sequence remain within the boundaries set by the video compression technology level or standard. In some cases, the levels limit the maximum image size, the maximum frame rate, the maximum sampling rate for reconstruction (measured in megasamples per second, for example), the maximum reference image size, and so on. The limits set by the levels, in some cases, may be further limited by the specifications of the hypothetical reference decoder (HRD, Hypothetical Reference Decoder) and metadata for managing the HRD buffer signaled in the encoded video sequence.

[0082] According to an embodiment, the receiver (531) may receive additional (redundant) data with encoded video. Additional data may be included as part of the encoded video sequence(s). The additional data may be used by the video decoder (510) to correctly decode the data and/or more accurately reconstruct the original video data. The additional data may be, for example, enhancement layers in time, space, or signal-to-noise ratio (SNR), redundant slices, redundant images, forward error correction codes, and so on.

[0083] FIG. 6 is a block diagram of a video encoder (603) according to an embodiment of the present invention. The video encoder (603) is included in the electronic device (620). The electronic device (620) includes a transmitter (640) (eg, a transmission circuit). Video encoder (603) may be used instead of video encoder (403) in the example shown in FIG. four.

[0084] The video encoder (603) may receive video samples from a video source (601) (which is not part of the electronic device (620) in the example shown in FIG. 6) that may capture the video image(s) to be encoding with a video encoder (603). In another example, the video signal source (601) is included in the electronic device (620).

[0085] The video source (601) may provide the source video sequence to be encoded by the video encoder (603) in the form of a digital video sample stream of any suitable bit depth (eg: 8 bit, 10 bit, 12 bit, ...), any color space (eg, BT.601 Y CrCB, RGB, ...) and any suitable sampling structure (eg Y CrCb 4:2:0, Y CrCb 4:4:4). In a media service system, the video source (601) may be a storage device where previously prepared video is stored. In a videoconferencing system, the video source (601) may be a camera that captures local image information as a video sequence. The video data may be provided as a plurality of individual images that give a sense of movement when viewed in sequence. The images themselves may be organized as a spatial array of pixels, where each pixel may contain one or more samples depending on the sampling structure, color space, etc. used. It is not difficult for a person skilled in the art to understand the relationship between pixels and samples. The following description is about readings.

[0086] According to an embodiment, the video encoder (603) may encode and compress images of the original video sequence into the encoded video sequence (643) in real time or subject to any other time constraints imposed by the application. Establishing a proper coding rate is one of the functions of the controller (650). In some embodiments, the controller (650) controls other functional modules, as described below, and is operatively connected to other functional modules. The connection is not shown for simplicity. The parameters set by the controller (650) may include parameters related to bit rate adjustment (picture skip, quantizer, lambda value applied in rate-distortion optimization, ...), picture size, group of pictures (GOP) scheme, maximum area motion vector search, etc. The controller (650) may be configured to have other suitable functions related to the video encoder (603) optimized for a given system design.

[0087] In some embodiments, video encoder (603) is configured to operate in a coding loop. As a very simplified description, by way of example, an encoding loop may include a source encoder (630) (e.g., responsible for creating symbols, e.g., a stream of symbols, based on the input image to be encoded and the reference image(s) )) and a (local) decoder (633) built into the video encoder (603). The decoder (633) reconstructs the symbols to create sample data, in a similar way as a (remote) decoder would (because any compression between the symbols and the encoded video bitstream is lossless in the video compression technologies of the disclosed invention). The reconstructed sample stream (sample data) is entered into the reference picture memory (634). Since the decoding of the symbol stream produces bit-accurate results independent of the position of the decoder (local or remote), the contents of the reference picture memory (634) will also be the same bit-accuracy for the local encoder and the remote encoder. In other words, the predicted portion of the encoder "sees" as reference picture samples exactly the same sample values as the decoder would "see" if using prediction during decoding. This fundamental principle of reference picture synchronism (and eventually drift if synchronism cannot be maintained due to, for example, channel errors) is also used in some related fields of technology.

[0088] A "local" decoder (633) may operate in the same manner as a "remote" decoder, such as the video decoder (510) detailed above with reference to FIG. 5. However, again, referring to FIG. 5, since the symbols are available and encoding/decoding of the symbols into the encoded video sequence by the entropy encoder (645) and analyzer (520) can be lossless, the entropy decoding parts of the video decoder (510) including the buffer memory (515) and the analyzer (520) , may not be fully implemented in the local decoder (633).

[0089] In doing so, it can be concluded that any decoding technology present in a decoder, with the exception of analysis/entropy decoding, must also necessarily be present, in essentially identical functional form, in the corresponding encoder. For this reason, the disclosed invention is focused on the operation of the decoder. The description of encoding technologies can be abbreviated as they are the inverse of the decoded technologies described in detail. Only in some areas a more detailed description is required, which is given below.

[0090] According to some examples, in operation, source encoder (630) may perform motion-compensated predictive coding, in which an input picture is predictively encoded based on one or more previously encoded pictures from a video sequence, referred to as "reference pictures". Thus, the encoding engine (632) encodes the differences between the pixel blocks of the input image and the pixel blocks of the reference image(s) that can be selected as the predicted reference(s) to the input image.

[0091] The local video decoder (633) may decode the encoded video data of pictures that can be specified as reference pictures based on the symbols generated by the source encoder (630). The operations of the encoding engine (632) may be predominantly lossy processes. When the encoded video data can be decoded in a video decoder (not shown in FIG. 6), the reconstructed video sequence can typically be a copy of the original video sequence with some errors. The local video decoder (633) duplicates the decoding processes that may be performed by the video decoder on the reference pictures and may cause the reconstructed reference pictures to be stored in the reference picture cache (634). Thus, the video encoder (603) can locally store copies of the reconstructed reference pictures having the same content as the reconstructed reference pictures to be received by the video decoder at the receiver side (in the absence of transmission errors).

[0092] The predictor (635) may perform prediction searches for the encoding engine (632). Thus, for a new picture to be encoded, the predictor (635) can search the reference picture memory (634) for sample data (as candidates for the role of reference pixel blocks) or some metadata, for example, motion vectors of the reference picture, block shapes etc., which can serve as a proper reference for predicting new images. The predictor (635) may operate on a sample block x pixel block basis to find the proper prediction references. In some cases, according to the search results obtained by the predictor (635), the input picture may have prediction references extracted from the plurality of reference pictures stored in the reference picture memory (634).

[0093] The controller (650) may control the encoding operations of the source encoder (630), including, for example, setting parameters and subgroup parameters used to encode video data.

[0094] The output signal of all the above functional modules can be entropy encoded in an entropy encoder (645). An entropy encoder (645) translates symbols generated by various functional units into a coded video sequence by lossless symbol compression according to techniques such as Huffman coding, variable run length coding, arithmetic coding, and so on.

[0095] The transmitter (640) may buffer the encoded video sequence(s) created by the entropy encoder (645) in preparation for transmission over a communication channel (660), which may be a hardware/software link to a storage device where encoded video data. The transmitter (640) may combine the encoded video data from the video encoder (603) with other data to be transmitted, such as encoded audio data and/or auxiliary data streams (sources not shown).

[0096] The controller (650) may control the operation of the video encoder (603). During encoding, the controller (650) may assign to each encoded picture one or another encoded picture type, which may determine the encoding methods applicable to the corresponding picture. For example, images can often be assigned the following image types:

[0097] An intra-picture (I-picture) that can be encoded and decoded without using any other picture in the sequence as a prediction source. Some video codecs allow different types of intra-pictures, including, for example, pictures in the Independent Decoder Refresh ("IDR" format). The person skilled in the art will be aware of the varieties of I-images and their respective applications and features.

[0098] A predicted picture (P-picture) that can be encoded and decoded using intra prediction or inter prediction using at most one motion vector and a reference index to predict sample values of each block.

[0099] A bi-directionally predicted picture (B-picture) that can be encoded and decoded using intra prediction or inter prediction using no more than two motion vectors and reference indices to predict sample values of each block. Likewise, multi-predicted pictures may use more than two reference pictures and associated metadata to reconstruct a single block.

[0100] Source images typically allow for spatial division into multiple blocks of samples (eg, blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and coding on a block-by-block basis. The blocks may be predictively encoded with reference to other (previously encoded) blocks determined by the coding assignment applied to the respective images of those blocks. For example, blocks of I-pictures may be encoded without prediction or predictively with reference to previously coded blocks of the same picture (spatial prediction or intra prediction). The pixel blocks of P-pictures may be predictively encoded by spatial prediction or temporal prediction with reference to one previously encoded reference picture. Blocks of B-pictures may be predictively encoded by spatial prediction or temporal prediction with reference to one or two previously encoded reference pictures.

[0101] The video encoder (603) may perform encoding operations according to a predetermined video coding technology or standard, such as ITU-T Rec. H.265. In operation, the video encoder (603) may perform various compression operations, including predictive encoding operations that exploit temporal and spatial redundancies in the input video sequence. Therefore, the encoded video data may conform to the syntax given by the video coding technology or standard being used.

[0102] According to an embodiment, the transmitter (640) may transmit additional data with encoded video. The source encoder (630) may include such data as part of the encoded video sequence. The additional data may include temporal/spatial/SNR enhancement layers, other forms of redundant data such as redundant images and slices, SEI messages, VUI parameter set fragments, and so on.

[0103] The video may be captured as a plurality of source images (video images) in time sequence. Intra-image prediction (often abbreviated as intra-image prediction) uses spatial correlation within a given image, and inter-image prediction uses (temporal or otherwise) correlation between images. For example, a specific image to be encoded/decoded, which is called the current image, is divided into blocks. When a block in the current picture is similar to a reference block in a previously encoded and still buffered reference picture in a video, the block in the current picture may be encoded with a vector, which is referred to as a motion vector. The motion vector points to a reference block in the reference picture and may have a third dimension identifying the reference picture if multiple reference pictures are used.

[0104] In some embodiments, a dual prediction technique may be used in inter-picture prediction. According to the double prediction method, two reference pictures are used, for example, a first reference picture and a second reference picture, both of which precede the current picture in the video in decoding order (but may be in the past and future, respectively, in display order). A block in the current picture may be encoded by a first motion vector that points to a first reference block in the first reference picture and a second motion vector that points to a second reference block in the second reference picture. The block may be predicted by a combination of the first reference block and the second reference block.

[0105] Additionally, a combining mode technique may be used in inter-picture prediction to improve coding efficiency.

[0106] According to some embodiments of the invention, predictions, such as inter-picture predictions and intra-picture predictions, are performed block by block. For example, according to the HEVC standard, an image in a video sequence is divided into coding tree units (CTUs) for compression, CTUs in an image are the same size, for example, 64×64 pixels, 32×32 pixels, or 16×16 pixels. In general, the CTU includes three coding tree blocks (CTBs), namely one luma CTB and two chrominance CTBs. Each CTU may recursively divide the quadtree into one or more coding units (CUs). For example, a 64x64 pixel CTU can be divided into one 64x64 pixel CU, or 4 32x32 pixel CUs, or 16 16x16 pixel CUs. For example, each CU is analyzed to determine a prediction type for the CU, such as an inter prediction type or an intra prediction type. The CU is divided into one or more prediction units (PU) depending on temporal and/or spatial predictability. In general, each PU includes a Luma Prediction Unit (PB) and two Chroma PBs. According to an embodiment, the encoding (encoding/decoding) prediction operation is performed in a prediction block unit. Using the luminance prediction block as an example of the prediction block, the prediction block includes a matrix of values (e.g., luminance values) for pixels, e.g., 8x8 pixels, 16x16 pixels, 8x16 pixels, 16x8 pixels, etc. P.

[0107] In FIG. 7 shows a diagram of a video encoder (703) according to another embodiment of the invention. The video encoder (703) is configured to receive a processing block (eg, a prediction block) of sample values in the current video image in the video sequence and to encode the processing block into an encoded image that is part of the encoded video sequence. For example, video encoder (703) is used instead of video encoder (403) in the example shown in FIG. four.

[0108] In the HEVC example, video encoder (703) receives a matrix of sample values for a processing block, such as an 8x8 sample prediction block, or the like. The video encoder (703) determines whether the processing block is best encoded in intra mode, inter mode, or dual prediction mode using, for example, rate-distortion optimization. When the processing block is to be encoded in intra mode, the video encoder (703) may use an intra prediction technique to encode the processing block into an encoded picture; and when the processing block is to be encoded in inter mode or double prediction mode, the video encoder (703) may use an inter prediction or double prediction technique, respectively, to encode the processing block into an encoded picture. In some video coding technologies, the combining mode may be a sub-mode of inter-picture prediction where the motion vector is derived from one or more motion vector predictors without involving the encoded motion vector component outside of the predictors. In some other video coding technologies, there may be a motion vector component applicable to a given block. By way of example, the video encoder (703) includes other components, such as a mode determination module (not shown) for determining the mode of the processing units.

[0109] In the example shown in FIG. 7, the video encoder (703) includes an inter encoder (730), an intra encoder (722), a residual calculator (723), a switch (726), a residual encoder (724), a common controller (721), and an entropy encoder (725). ) connected to each other as shown in Fig. 7.

[0110] The inter-encoder (730) is configured to receive samples of the current block (e.g., processing block), compare the block with one or more reference blocks in reference pictures (e.g., blocks in previous pictures and later pictures), generate external prediction (eg, describing redundant information according to the intra coding method, motion vectors, combining mode information), and calculating inter prediction results (eg, prediction block) based on the inter prediction information using any suitable method. In some examples, reference pictures are decoded reference pictures that are decoded based on encoded video information.

[0111] The intra-encoder (722) is configured to receive samples of the current block (e.g., processing block), in some cases comparing the block with blocks previously encoded in the same picture, generating quantized coefficients after the transform, and in some cases also information intra prediction (eg, intra prediction direction information according to one or more intra coding methods). By way of example, the intra encoder 722 also calculates intra prediction results (eg, prediction block) based on intra prediction information and reference blocks in the same picture.

[0112] The common controller (721) is configured to determine common control data and control other components of the video encoder (703) based on the common control data. For example, the common controller (721) determines the mode of the block and outputs a control signal to the switch (726) based on the mode. For example, when the mode is the intra mode, the common controller (721) controls the switch (726) to select the result of the intra mode for use by the residual calculator (723), and controls the entropy encoder (725) to select the intra prediction information and turn on the intra prediction information into the bit stream; and when the mode is inter mode, the common controller (721) controls the switch (726) to select the inter prediction result to be used by the residual calculator (723), and controls the entropy encoder (725) to select the inter prediction information and include the inter prediction information in the bit flow.

[0113] The residual calculator (723) is configured to calculate the difference (residual data) between the received block and the prediction results selected from the intra-coder (722) or inter-coder (730). The residual encoder (724) is configured to operate on the residual data to encode the residual data to generate transform coefficients. For example, the residual encoder (724) is configured to transform residual data from the spatial domain to the frequency domain and generate transform coefficients. Then, the transform coefficients are subjected to quantization processing to obtain quantized transform coefficients. In various embodiments, the video encoder (703) also includes a residual decoder (728). The residual decoder (728) is configured to perform inverse transformation and generate decoded residual data. The decoded residual data may be appropriately used by the intra-coder (722) and the inter-coder (730). For example, the intercoder (730) may generate decoded blocks based on the decoded residual data and inter prediction information, and the intra encoder (722) may generate decoded blocks based on the decoded residual data and intra prediction information. The decoded blocks are properly processed to generate decoded pictures, and the decoded pictures can be buffered in a memory circuit (not shown) and used as reference pictures in some examples.

[0114] The entropy encoder (725) is configured to format the bit stream so that it includes a coded block. The entropy encoder (725) is configured to include various information according to an appropriate standard, such as the HEVC standard. For example, the entropy encoder (725) is configured to include common control data, selected prediction information (eg, intra prediction information or inter prediction information), residual information, and other appropriate information in the bitstream. Note that, according to the disclosed invention, when encoding a block in the combining sub-mode of any of the inter-mode and the dual-prediction mode, there is no residual information.

[0115] In FIG. 8 shows a diagram of a video decoder (810) according to another embodiment of the invention. The video decoder (810) is configured to receive encoded pictures constituting part of the encoded

video sequence, and decoding the encoded pictures to generate reconstructed pictures. For example, video decoder (810) is used instead of video decoder (410) in the example shown in FIG. four.

[0116] In the example shown in FIG. 8, the video decoder (810) includes an entropy decoder (871), an inter decoder (880), a residual decoder (873), a reconstruction unit (874), and an intra decoder (872) connected to each other as shown in FIG. eight.

[0117] The entropy decoder (871) may be configured to reconstruct, from the encoded image, some symbols that represent the syntax elements that form the encoded image. Such symbols may include, for example, a block coding mode (e.g., intra mode, inter mode, dual prediction mode, the latter two being in a pooling submode or other submode), prediction information (e.g., intra prediction information, or inter prediction information ) that can identify a specific sample or metadata used for prediction by the intra-decoder (872) or inter-decoder (880), respectively, residual information in the form of, for example, quantized transform coefficients, and the like. For example, when the prediction mode is an intra or dual prediction mode, the inter prediction information is input to the inter decoder (880); and when the prediction type is an intra prediction type, the intra prediction information is input to the intra decoder (872). The residual information may be inverse quantized and fed to the residual decoder (873).

[0118] The inter decoder (880) is configured to receive inter prediction information and generate inter prediction results based on the inter prediction information.

[0119] The intra decoder (872) is configured to receive intra prediction information and generate prediction results based on the intra prediction information.

[0120] The residual decoder (873) is configured to perform inverse quantization to extract the dequantized transform coefficients and process the dequantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The residual decoder (873) may also require some control information (include a quantizer parameter (QP, Quantizer Parameter)), and this information may be provided by the entropy decoder (871) (the data path is not shown since this may only be small volume control information).

[0121] The reconstruction module (874) is configured to combine, in the spatial domain, the residual at the output of the residual decoder (873) and prediction results (at the output of the inter- or intra-prediction modules, as appropriate) to form a reconstructed block that may be included into the reconstructed image, which, in turn, can be part of the reconstructed video. Note that other suitable operations, such as a release operation and the like, may be performed to improve the visual quality.

[0122] Note that video encoders (403), (603) and (703) and video decoders (410), (510) and (810) may be implemented using any suitable technique. According to an embodiment, video encoders (403), (603) and (703) and video decoders (410), (510) and (810) may be implemented using one or more integrated circuits. In another embodiment, video encoders (403), (603), and (703) and video decoders (410), (510), and (810) may be implemented using one or more processors that execute program instructions.

[0123] Aspects of the invention relate to modifications of the secondary transform, for example, implementations of the inverse secondary transform.

[0124] In some embodiments, for example, in HEVC, the primary transform may include 4-point, 8-point, 16-point, and 32-point discrete cosine transform (DCT, discrete cosine transform) type 2 (DCT-2 ), and transformation kernel matrices can be represented using 8-bit integers (i.e., an 8-bit transformation kernel). The transformation kernel matrices of the smaller DCT-2 are part of the transformation kernel matrices of the larger DCT-2, as shown in Annex I.

[0125] DCT-2 core matrices exhibit symmetry/antisymmetry characteristics. Therefore, a "partial butterfly" implementation can be supported to reduce the number of operation counters (eg, multiplications, additions, subtractions, shifts, etc.), and identical matrix multiplication results can be obtained using a partial butterfly.

[0126] In some embodiments, for example, in VVC, in addition to the 4-point, 8-point, 16-point, and 32-point DCT-2 transformations described above, additional 2-point and 64-point DCT-2 transformations may be included. An example of a 64-point DCT-2 core, such as that used in VVC, is shown in Appendix II as a 64×64 matrix.

[0127] In addition to DCT-2 and 4x4 DST-7, such as those used in HEVC, the Adaptive Multiple Transform (AMT) scheme (also known as EMT, Enhanced Multiple Transform) or multiple transform selection (MTS, Multiple Transform Selection)) can be used, for example, in VVC to encode the remainder for both inter- and intra-coded blocks. The AMT scheme may use multiple selected transforms from the DCT/DST families, different from the current HEVC transforms. DST-7 and DCT-8 transformation matrices are introduced for the first time. Table 1 shows examples of basis functions of the selected DST/DCT for N-point input.

[0128] Primary transformation matrices used in VVC, for example, can be used with an 8-bit representation. AMT applies transformation matrices to CUs whose width and height are less than or equal to 32. Whether AMT is applied may indicate a flag (eg, mts_flag). When mts_flag is 0, in some examples only DCT-2 is used to encode the residual data. When mts_flag is 1, an index (eg, mts_idx) may be further signaled using 2 bins to identify the horizontal and vertical transformation to be used according to Table 2, where a type value of 1 means DST-7 is used, and a type value of 2 , means using DCT-8. In Table 2, the value of trTypeHor and trTypeVer depends on mts_idx[x][y][cIdx].

[0129] In some embodiments, implicit MTS may be applied when the aforementioned signaling-based MTS (ie, explicit MTS) is not used. With implicit MTS, transform selection is made according to block width and height instead of signaling. For example, with implicit MTS, DST-7 is chosen for the shorter side (i.e., at least one of M and N) of the MxN block, and DCT-2 is chosen for the longer side (i.e., at most one of M and N) block.

[0130] Exemplary transformation kernels, each of which is a matrix formed by basis vectors, for DST-7 and DCT-8 are illustrated in Appendix III.

[0131] In some examples, for example, in VVC, when both the height and the width of the coding block are less than or equal to 64, the size of the TV is the same as the size of the coding block. If the height or width of a coding block is greater than 64, when performing a transformation (e.g., inverse transform, inverse primary transform, etc.) or intra prediction, the coding block is further divided into multiple subblocks, where the width and height of each subblock is less than or equal to 64. One transformation can be performed on each subblock.

[0132] The corresponding MTS syntax and semantics in some examples in VVC can be described below (greyed out) in FIG. 9 and 10A-10C. In FIG. 9 shows an example of the conversion unit syntax. In FIG. 10A-10C show an example of a residual encoding syntax.

[0133] The following is an example of the semantics of a transformation unit. cu_mts_flag[x0][y0] equal to 1 indicates that the selection of multiple transforms is applied to the residual samples of the corresponding luma transform block, cu_mts_flag[x0][y0] equal to 0 indicates that the selection of multiple transforms is not applied to the residual samples of the corresponding transform block brightness. Indexes x0, y0 of the matrix indicate the position (x0, y0) of the upper left brightness value of the considered transformation block relative to the upper left value of the image brightness. When cu_mts_flag[x0][y0] is absent, it is assumed to be 0.

[0134] The following is an example of residual encoding semantics, mts_idx[x0][y0] indicates which transform kernels are applied to the residual luminance samples in the horizontal and vertical direction of the current transform block. Indices x0, y0 of the matrix indicate the position (x0, y0) of the upper left brightness value of the considered transformation block relative to the upper left value of the image brightness. When mts_idx[x0][y0] is absent, it is assumed to be -1.

[0135] In FIG. 11A shows an exemplary direct transform (also referred to as direct primary transform) performed by an encoder. Direct transformation may include direct horizontal transformation and direct vertical transformation. A forward horizontal transform is first applied to a residual block (1110) having residual data to obtain an intermediate block. Then, a direct vertical transform is applied to the intermediate block to obtain a coefficient block (1112) having the transform coefficients.

[0136] FIG. 11B shows an exemplary inverse transform (also referred to as inverse primary transform or inverse transform) performed by a decoder. Generally speaking, the inverse transformation is the same as the direct transformation. The reverse prime may include a reverse primary horizontal transform (also referred to as reverse horizontal) and a reverse primary vertical transform (also referred to as reverse vertical). To match the forward transform, the order of applying the inverse horizontal and vertical transforms is switched in the reverse transform. Accordingly, an inverse vertical transform is first applied to the coefficient block (1122) to obtain an intermediate block. The inverse horizontal transform is then applied to the intermediate block to obtain a residual block (1120).

[0137] Under the primary transformation can be understood as a direct primary transformation or reverse primary transformation. A horizontal transformation can be understood as a reverse horizontal transformation or a forward horizontal transformation. Likewise, a vertical transformation can be understood as an inverse vertical transformation or a forward vertical transformation.

[0138] In an example, for example, in VVC, at the decoder, first a reverse vertical primary transform is performed, then, after applying the reverse vertical transform, a reverse horizontal primary transform is performed, as shown in FIG. 12A-12E in gray texts. In FIG. 12A-12E show an example transform process, for example, for scaled transform coefficients. The texts highlighted in gray are shown in Fig. 12E.

[0139] According to an embodiment, a mode-dependent non-separable secondary transform (NSST) can be used between the forward transform kernel and quantization on the encoder side, and between the dequantization and the inverse transform kernel on the decoder side. For example, to keep complexity low, NSST is applied to the low frequency coefficients after the primary transform (or kernel transform). When both the width (W) and the height (H) of the transform coefficient block are greater than or equal to 8, NSST 8x8 is applied to the top left 8x8 region of the transform coefficient block. Otherwise, when the transform coefficient block width W or height H is 4, NSST 4×4 is applied, and NSST 4×4 is performed on the upper left min(8,W)×min(8,H) region of the transform coefficient block. The above transformation selection method applies to both the luma and chrominance components.

[0140] An implementation of NSST matrix multiplication is described below using a 4×4 input block as an example. The input block X 4×4 is written in eq. (1) as

в ур. (2), где[0141] The input block X can be represented as a vector

in ur. (2), where

где

указывает вектор коэффициентов преобразования, и Т - матрица преобразования 16×16. Затем вектор

16×1 коэффициентов преобразования реорганизуется как блок 4×4 согласно порядку сканирования (например, горизонтальному порядку сканирования, вертикальному порядку сканирования или диагональному порядку сканирования) входного блока X. Коэффициенты с меньшими индексами могут располагаться в блоке 4×4 коэффициентов с меньшими индексами сканирования. В некоторых вариантах осуществления, для уменьшения сложности NSST вместо вышеописанного умножения матриц может использоваться преобразование гиперкуба Гивенса (HyGT) с реализацией бабочки.The non-separable transformation is computed as

where

indicates a vector of transform coefficients, and T is a 16×16 transform matrix. Then the vector

The 16×1 transform coefficients are reorganized as a 4×4 block according to the scan order (e.g., horizontal scan order, vertical scan order, or diagonal scan order) of the input block X. Coefficients with lower indices may be placed in a 4×4 block of coefficients with lower scan indices. In some embodiments, instead of the above described matrix multiplication, the Givens Hypercube Transform (HyGT) with butterfly implementation can be used to reduce the complexity of the NSST.

[0142] In the example, 35x3 non-separable secondary transforms are available for both 4x4 and 8x8 block sizes, where 35 is the number of transform sets associated with intra prediction modes and 3 is the number of NSST candidates for each intra prediction mode. Table 3 shows an exemplary mapping from the intra prediction mode to the corresponding transform set. The set of transforms applied to the luminance/chrominance transform coefficients is specified by the corresponding luma/chrominance intra prediction mode according to Table 3, which shows the mapping from the intra prediction mode to the transform set index. For an intra prediction mode greater than 34, which corresponds to a diagonal prediction direction, the block of transform coefficients is transposed before/after the NSST at the encoder/decoder, respectively.

[0143] For each transform set, the selected NSST candidate may be further indicated by explicitly signaling the NSST index at the CU level. The NSST index at the CU level is signaled in the bitstream for each intra-coded CU after the transform coefficients and truncated unary binarization are used for the NSST index at the CU level. For example, the truncated value is 2 for the flat or DC mode, and 3 for the angular intra prediction mode. In the example, the NSST index at the CU level is only signaled if there is more than one non-zero coefficient in the CU. The default value is zero and is not signaled, indicating that NSST is not applied to the CU. Each of the values 1-3 indicates which NSST candidate is to be applied from the transform set.

[0144] In some embodiments, NSST is not applied to a block encoded in transform skip mode. When the NSST index at the CU level is signaled to the CU and is non-zero, the NSST is not used for the block that is encoded in the CU's transform skip mode. When CUs with blocks of all components are encoded in the transform skip mode, or the number of non-zero CB coefficients in the non-skipping transform mode is less than 2, the NSST index at the CU level is not signaled to the CU.

[0145] In some embodiments, a variation of NSST is used, such as a reduced size transform (RST). RST uses a nulling conversion scheme. In the example, in RST, whether the intra prediction mode is flat or DC is checked for entropy encoding of the NSST transform index.

[0146] In the example, 4 transform sets are applied, and each transform set includes three RST transform kernels. The RST transform kernels may have a size of 16x48 (or 16x64) (used for a transform coefficient block whose height and width is greater than or equal to 8) or 16x16 (used for a transform coefficient block whose height or width is 4). For convenience of notation, the 16x48 (or 16x64) transform is referred to as RST8x8 and the 16x16 transform is referred to as RST4x4.

[0147] In FIG. 13 and FIG. 14 shows examples of two transform encoding processes (1300) and (1400) using a 16×64 transform (or a 64×16 transform depending on whether the transform is a forward or inverse secondary transform) and a 16×48 transform (or a 48× transform). 16 depending on whether the transformation is forward or inverse secondary transformation), respectively. According to FIG. 13, in a process (1300), on the encoder side, a direct primary transform (1310) may first be performed on the residual block to obtain a block (1313) of coefficients. The direct secondary transform (1312) can then be applied to the coefficient block (1313). In the forward secondary transform (1312), the 64 coefficients of the 4×4 A-D subblocks in the upper left corner of the coefficient block (1313) may be represented by a vector of length 64, and the vector of length 64 may be multiplied with a 64×16 transformation matrix (i.e., a width of 64 and height 16) according to Eq.(2), which gives a vector of length 16. The elements of the vector of length 16 are transferred back to the upper left subblock A of the 4×4 block (1313) of coefficients. The coefficients in sub-blocks B-D may be zero. The coefficients obtained after the direct secondary transform (1312) are then quantized in step (1314) and entropy encoded to generate coded bits in the bitstream (1316).

[0148] The encoded bits may be received at the decoder side and entropy decoded and then dequantized (1324) to generate a coefficient block (1323). An inverse secondary transform (1322), eg, inverse RST8x8, may be performed to obtain 64 coefficients, eg, from 16 coefficients on the top left 4×4 subblock E. The 64 coefficients can be carried back to the 4x4 E-H subblocks. Additionally, the coefficients in the block (1323) of coefficients after the inverse secondary transform (1322) may be processed with the inverse primary transform (1320) to obtain a reconstructed residual block.

[0149] The process (1400) shown in FIG. 14 is similar to process (1300) except that fewer (ie, 48) coefficients are processed during the direct secondary transform (1412). In particular, the 48 coefficients in subblocks A-C are processed with a smaller 48x16 transform matrix. By using a smaller 48×16 transformation matrix, the size of the memory for storing the transformation matrix and the amount of calculations (eg, multiplication operations, addition operations, subtraction operations, etc.) can be reduced, and thus the computational complexity can be reduced.

[0150] Reduced Transform (RT, also referred to as RST) can map an N-dimensional vector to an R-dimensional vector in another space, where R/N (R<N) is a reduction factor.

[0151] The RST (or RT) matrix is the following R×N matrix:

where the R strings of the transformation are the R bases of an N-dimensional space. The inverse transformation matrix for RT is obtained by transposing the forward transformation matrix.

[0152] In FIG. 15A shows a reduced forward transform process (1501) and a reduced inverse transform process (1502). T expresses the transformation matrix RST of size R×N, and T ^T expresses the matrix obtained by transposing T, and T ^T has size N×R.

[0153] In RST8×8, a reduction factor of 4 (1/4 size) can be realized. For example, instead of 64x64, which is the size of a conventional 8x8 non-separable transform matrix, a 16x64 direct matrix can be used. The 64×16 inverse RST matrix may be used at the decoder side to generate the kernel (primary) transform coefficients in the upper left 8×8 region. Direct RST8x8 uses 16x64 matrices (or 8x64 for an 8x8 block), whereby direct RST8x8 produces non-zero coefficients only in the top left 4x4 region in a given top left 8x8 region. In other words, when RST is applied, the area in the top left 8x8 area that extends beyond the top left 4x4 area has only zero coefficients. For RST4x4, 16x16 (or 8x16 for a 4x4 block) direct matrix multiplication can be applied.

[0154] In addition, for RST8×8, to further reduce the size of the transformation matrix, instead of using all the top left 8×8 coefficients (64 coefficients in shaded subblocks in Fig. 15B) of the residual block (1510) as input to calculate the secondary transformation , the top left three coefficients of the 4×4 subblock (48 coefficients in shaded subblocks in FIG. 15C) of the residual block (1510) are used as input to calculate the secondary transform. Accordingly, a 16×64 transformation matrix is used in FIG. 15B and a 16x48 transformation matrix is used in FIG. 15C.

[0155] In an example, the reverse RST is applied when the following two conditions are met: (i) the block size (eg, block width W and/or block height H) is greater than or equal to a threshold (eg, W>=4 and H>=4), and (ii) the Transform Skip Mode Flag is zero. For example, if the width (W) and height (H) of the transform coefficient block is greater than 4, then RST8×8 is applied to the top left 8×8 region of the transform coefficient block. Otherwise, RST4×4 is applied to the upper left region min(8, W)×min(8, H) of the transform coefficient block.

[0156] In an example, when the RST index is 0, RST is not applied. Otherwise, the RST is applied and the kernel is selected with the RST index. In the example, RST is applied to intra-CU (eg, intra-prediction or intra-mode encoded CU) in both intra- and inter-slices, and for both luminance and chrominance components. If dual tree is enabled, the RST indices for luma and chrominance are signaled separately. For inter-slice (dual tree forbidden), a single RST index is signaled and used for both luma and chrominance. When ISP mode is selected, RST is disabled and the RST index is not signaled.

[0157] In an example, the RST matrix may be selected from four transform sets, each consisting of two transforms. Which set of transforms to apply may be determined based on the intra prediction mode as follows. When one of the three cross component linear model (CCLM) modes is specified, transform set 0 may be selected. Otherwise, the transform set may be selected according to the table (1550) shown in FIG. 15D. The index (eg, IntraPredMode) for accessing the table (1550) may take values in the range [-14, 80] and is a converted mode index used, for example, for wide-angle intra prediction. An example of intra prediction modes is shown in FIG. 16V. In the example, the index to access the table (1550) may take values in the range [-14, 83] or any suitable range.

[0158] FIG. 16A shows exemplary intra prediction directions and intra prediction modes used in HEVC. In HEVC, there are a total of 35 intra prediction modes (modes 0 ~ 34). Mode 0 and mode 1 are non-directional modes, of which mode 0 is a flat mode (labeled Intra_Planar in FIG. 16A) and mode 1 is a DC mode (labeled Intra_DC in FIG. 16A). Modes 2-34 are directional modes, of which mode 10 is horizontal mode, mode 26 is vertical mode, and mode 2, mode 18 and mode 34 are diagonal modes. In some examples, the intra prediction modes are signaled by the three most probable modes (MPM) and the 32 remaining modes.

[0159] FIG. 16B shows exemplary intra prediction directions and intra prediction modes in some examples (eg, VVC). There are a total of 95 intra prediction modes (modes -14 ~ 80), of which mode 18 is horizontal mode, mode 50 is vertical mode, and mode 2, mode 34 and mode 66 are diagonal modes. Modes -1 ~ -14 and modes 67 ~ 80 are called wide-angle intra prediction (WAIP) modes.

[0160] Multi-linear intra prediction may use more reference lines for intra prediction. The reference line may include multiple samples in an image. In the example, the reference line includes the row samples and the column samples. In an example, the encoder may determine and signal the reference line used to generate the intra-predictor. An index (also referred to as a reference line index) indicating a reference line may be signaled prior to signaling the intra prediction mode(s). In the example, MPMs are only enabled when a non-zero baseline index is signaled. In FIG. 17 shows an example of 4 reference lines for a coding block (1710). In the example shown in FIG. 17, the reference line may include six segments, i. e. segments A - F. Reference line 3 may include an upper left reference point. Segments A and F can be filled with the nearest samples from segments B and E, respectively. In some examples, such as HEVC, a single reference line is used for intra prediction (eg, reference line 0, which is adjacent to the coding block (1710). In some examples, such as VVC, multiple reference lines (eg, reference lines 0, 1, and 3) are used for intra prediction.

[0161] Intra sub-partition (ISP) coding mode may be used. In the ISP coding mode, the intra-predicted luminance block may be divided vertically or horizontally into 2 or 4 parts depending on the block size.

[0162] In FIG. 18 shows table 4 which relates the number of parts into which a block is divided to its size. For example, when the block size is 4×4, block splitting in the ISP coding mode is not performed. When the block size is 4x8 or 8x4, the block is split into two parts in the ISP coding mode. For all other block sizes greater than 4x8 or 8x4, the block is split into four parts. In FIG. 19 shows an example of splitting a block having a size of 4x8 or 8x4. In FIG. 20 shows another example of splitting a block having a size other than 4x8, 8x4, and 4x4, such as a block size greater than 4x8 and 8x4. In the example, all parts satisfy the condition of having at least 16 samples. For the chrominance component, ISP does not apply.

[0163] In some examples, for each of the parts, the decoder may entropy decode the coefficients sent from the encoder to the decoder, then the decoder performs inverse quantization and inverse transformation of the coefficients to generate residuals (or residual data) for the part. Additionally, when the part is intra-predicted by the decoder, the decoder may add the residuals to the results of the intra prediction to obtain reconstructed samples of the part. Therefore, the reconstructed samples of each part may be available to generate a prediction of the next part(s) to be reconstructed. The above process may be repeated for the next part(s), and so on. All parts share the same intra prediction mode, in one example. In some examples, in the ISP, each part can be considered as a TU, since the transformation and reconstruction are performed separately for each part.

[0164] In some examples, the ISP algorithm is only tested with intra prediction modes that are part of the MPM list. For this reason, when a block uses an ISP, the MPM flag is assumed to be one. In addition, when an ISP is used for a certain block, the MPM list may be modified to exclude the DC mode and to give priority to horizontal intra prediction modes for horizontal partitioning (or horizontal partitioning) ISP and vertical intra prediction modes for vertical partitioning (or vertical partitioning) in some examples.

[0165] In FIG. 21A-21D show examples of different YUV formats or chrominance formats. Each color format can specify different downsampling grids for different color components.

[0166] Secondary transformation can be understood as NSST, RST (or RT), and the like. The secondary transformation index can be understood as an NSST index, an RST index, and the like. In the example, the secondary transform index indicates the secondary transform (also referred to as the secondary transform candidate). The secondary conversion index may be signaled at the CU level. For example, an NSST index or an RST index is signaled at the CU level for the CU. Whether the secondary transform index should be signaled may depend on the number of non-zero CU coefficients. Thus, the decoder may cycle through all the TUs included in the CU to determine the number of non-zero CU coefficients. In some embodiments, the process is relatively complex.

[0167] In certain secondary transform constructs (e.g., RST), when a single split tree is used among different color components in a CU, including, for example, a luma component and two chroma components, the number of non-zero coefficients in the CU can be counted to determine whether whether the index of the secondary conversion. However, for a 4xN or Nx4 luma block, the corresponding chrominance block has a size of 2xN/2 or N/2x2 in YUV4:2:0 format. Thus, a secondary transform such as RST is not applied to the chroma block, and the number of non-zero chroma block coefficients need not be counted.

[0168] In some examples, the second transformation, such as NSST or RST, is not allowed for the ISP. This may limit the full benefit of the secondary transforms in terms of coding efficiency.

[0169] The embodiments described herein may be used individually or combined in any order. Additionally, embodiments may be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits) in an encoder, decoder, or the like. In one example, one or more processors may execute a program that is stored on a computer-readable carrier. In some examples, the block may be a prediction block, a coding block, a CU, or the like.

[0170] In the specification, embodiments for MTS candidate DST-7 may apply to DST-4, and embodiments for MTS candidate DCT-8 may apply to DCT-4. Additionally, in some embodiments, references to NSST may also apply to RST, which is an example of an alternative non-separable secondary transform design.

[0171] The high-level syntax element (HLS, high-level syntax) may refer to a video parameter set (VPS, Video Parameter Set), a sequence parameter set (SPS, Sequence Parameter Set), a picture parameter set (PPS, Picture Parameter Set), slice header, tile header, tile group header, etc. The CTU header may refer to the syntax element(s) signaled to the CTU, for example, as header information. In the example, the CTU size is the maximum CU size. The size of a TU may refer to the maximum width and/or height of a TU or TU area.

[0172] In general, when the luma component size (represented by luma samples) of a certain unit (eg, TU, CU) is known, a corresponding chrominance component size can be obtained, which is given by the number of chrominance samples. The example uses YUV format 4:2:0 and the CU has a CU size of 64x64 luminance samples (or 64x64-L). Accordingly, the CU has a CU size of 32x32 chroma samples (or 32x32-C). The CU size may be referred to as 64x64-L, 32x32-C, or 64x64-L/32x32-C. The CU may include a luminance block and two chrominance blocks, where the luma block has 64x64 luminance samples and each of the two chrominance blocks has 32x32 chroma samples. Description can be adapted for TU. For brevity, such description is omitted.

[0173] The size of a TU may be represented using the luminance samples in the TU. For example, a maximum TU size of M samples means a maximum TU size of M luma samples. Likewise, the size of a CU can be represented using the luminance samples in the CU. The TU size and CU size may be represented using chrominance samples or a combination of luma and chrominance samples in other embodiments.

[0174] Size may refer to width, height, and/or area. For example, the maximum TU size may refer to the width, height, and/or area of the maximum TU. In general, TU, CU, etc. may be any suitable shape, including rectangular, square, L-shaped, or any suitable shape. In the case of an irregularly shaped block, such as an L-shape, the block size may indicate the area of the block.

[0175] In some embodiments, a maximum TU size (also referred to as a maximum TU size) may be signaled in an encoded video bitstream, such as in HLS (eg, SPS and PPS). The maximum TU size may be expressed in terms of luma samples or chrominance samples.

[0176] In some embodiments, the maximum TU size may be stored in the encoder and/or decoder, so the maximum TU size is not signaled. In one example, the maximum TU size may be stored in profile and/or layer definitions. The maximum TU size may be stored as a number of luminance or chrominance samples.

[0177] According to aspects of the invention, whether a secondary transform is allowed for a CU may be determined based on the size of the CU. In an example, whether a secondary transform is allowed for a CU may be determined based on the CU size and the CU size threshold. When the CU size is less than or equal to the CU size threshold, it is determined that the secondary transformation is allowed, and when the CU size is greater than the CU size threshold, it is determined that the secondary transformation is not allowed. In the example, when it is determined that the secondary transform is not allowed, the secondary transform index is not signaled. Thus, when the decoder determines that the secondary transform is not allowed, the decoder may also determine that the secondary transform index is not signaled. In an example, the CU size may refer to the width and/or height of the CU, for example, may be 64 counts. In an example, the CU size may refer to the area of the CU, eg 64×64 counts.

[0178] In an example, a CU size threshold (eg, CU width threshold, CU height threshold, or CU area threshold) is constrained by the maximum TU size in the CU. The maximum TU size may be signaled in the HLS. The maximum TU size may also be predetermined and stored in the decoder.

[0179] As described above, in some examples, the decoder may cycle through the TUs in the CU to determine the number of non-zero CU coefficients and then determine whether the secondary transform index is signaled. According to aspects of the invention, instead of counting the number of non-zero CU coefficients, it is possible to determine whether a secondary transform index is signaled based on the last position of the non-zero transform coefficients (or the position of the last non-zero coefficients) of the first CB in the CU. The first CB may be any suitable block, such as a luma block, a chrominance block, or the like, residing in the CU. The secondary transform index may indicate the selected secondary transform for the second CB in the CU.

[0180] According to aspects of the invention, it is possible to determine whether to perform a secondary transform on the second CB in the CU based on a determination that a secondary transform index is signaled. Further, when it is determined that the secondary conversion is to be performed, the second CB may be reconstructed after the secondary conversion indicated by the secondary conversion index is performed on the second CB. Alternatively, when it is determined that the secondary conversion is not to be performed, the sample at the second CB may be reconstructed without performing the secondary conversion at the second CB. The second CB may be any suitable block, such as a luminance block, a chrominance block, or the like, residing in the CU. In the example, the first CB is identical to the second CB. In another example, the first CB is different from the second CB.

[0181] In an example, the CU includes a luma unit. The first CB is the luminance block. The last position is the position of the last non-zero luminance factor for the luminance block. Accordingly, it is determined whether the secondary transform index is signaled based on the last luma position. The second CB is also a luminance block or the first CB.

[0182] In some embodiments, additional information may be included to determine if a secondary transform index is being signaled, as described below.

[0183] In an example, the CU includes a luminance block and a chrominance block. The first CB is the luminance block. The last position is the position of the last non-zero luminance factor for the luminance block. The side information may include the position of the last non-zero chrominance factor for the chrominance block. Accordingly, the determination whether the secondary transform index is signaled is made based on the position of the last non-zero luminance coefficient and the position of the last non-zero chrominance coefficient for the chrominance block. The second CB may be one of a luminance block and a chrominance block.

[0184] In an example, the CU includes a luminance block and two chrominance blocks (eg, a chroma I block and a chroma II block). The first CB is the luminance block. The last position is the position of the last non-zero luma factor for the luma block. The side information may include position I of the last non-zero chrominance factor for chrominance block I and position II of the last non-zero chrominance factor for chrominance block P. Accordingly, whether a secondary transform index is signaled may be determined based on the position of the last non-zero luminance factor, the position I of the last non-zero the chrominance coefficient of non-zero transform coefficients for chrominance block I and position II of the last non-zero chrominance coefficient of non-zero transform coefficients for chrominance block II. The second CB may be one of a luminance block, a chrominance block I, and a chrominance block II.

[0185] As described above, it is possible to determine whether a secondary transform index is signaled based on the position of the last non-zero coefficient of the first CB from the CU. The position of the last non-zero coefficient may include a horizontal component (eg, last_pos_x) and a vertical component (eg, last_pos y), and thus it can be determined whether a secondary transform index is signaled based on the horizontal component and/or the vertical component. The horizontal component and the vertical component can be an integer equal to or greater than 0. The vertical component can be an integer equal to or greater than 0.

[0186] According to an embodiment, the horizontal component can be compared to a first threshold and/or the vertical component can be compared to a second threshold. The first threshold may be identical to the second threshold. Alternatively, the first threshold may be different from the second threshold. The first threshold and/or the second threshold may be a positive integer such as 1, 2, 3, and the like.

[0187] In an example, it may be determined whether the horizontal component is less than the first threshold, and whether the vertical component is less than the second threshold. When it is determined that the horizontal component is less than the first threshold and the vertical component is less than the second threshold, it may be determined that the secondary transform index is not signalled.

[0188] In the example, it may be determined whether the horizontal component is greater than or equal to the first threshold. Additionally, it may be determined whether the vertical component is greater than or equal to the second threshold. When it is determined that the horizontal component is greater than or equal to the first threshold and the vertical component is greater than or equal to the second threshold, it may be determined that the secondary transform index is signaled.

[0189] According to an embodiment, it can be determined whether the sum of the horizontal component and the vertical component of the last position of the third threshold is smaller. When it is determined that the sum is less than the third threshold, it may be determined that the secondary transform index is not signaled. The third threshold may be a positive integer such as 1, 2, 3, or the like. When it is determined that the sum is greater than or equal to the third threshold, it may be determined that the secondary transform index is signaled.

[0190] According to an embodiment, it can be determined whether at least one of the horizontal component and the vertical component is less than the fourth threshold. When it is determined that at least one of the horizontal component and the vertical component is less than the fourth threshold, it is determined that the secondary transform index is not signaled. The fourth threshold may be a positive integer such as 1, 2, 3, or the like.

[0191] According to an embodiment, it can be determined whether at most one of the horizontal component and the vertical component is less than the fifth threshold. When it is determined that at most one is less than the fifth threshold, it is determined that the secondary transform index is not signaled. The fifth threshold may be a positive integer such as 1, 2, 3, or the like. The fourth threshold may be identical to the fifth threshold. Alternatively, the fourth threshold may be different from the fifth threshold.

[0192] According to an embodiment, the first CB is a luminance block in a CU. Additionally, the CU includes a chrominance block. However, whether the secondary transform index is signaled is determined based only on the position of the last non-zero luminance coefficient of the CU luminance block. Therefore, the position of the last non-zero chrominance factor for a chrominance block is not considered when determining whether a secondary transform index is signaled.

[0193] According to an embodiment, the CU includes a luminance block and a chrominance block as described above. Whether a secondary transform index is signaled is determined based on the position of the last non-zero luminance factor for the luminance block and the position of the last non-zero chrominance factor for the chrominance block. Similarly, the position of the last non-zero luminance factor may include a horizontal luminance component and a vertical luminance component, and the position of the last non-zero chrominance factor may include a horizontal chroma component and a vertical chroma component. Thus, it can be determined whether the secondary transform index is signaled based on the horizontal luma component, the vertical luma component, the horizontal chrominance component, and/or the vertical chrominance component.

[0194] According to an embodiment, one or more of the respective horizontal component and vertical component of each of the positions of the last non-zero luminance and chrominance coefficients can be compared with an appropriate threshold, such as 1, 2, 3, or the like. The example may determine if one or more of the respective horizontal component and vertical component of each of the positions of the last non-zero luminance and chrominance coefficients is less than a corresponding threshold. When one or more of the respective horizontal component and the vertical component of each of the positions of the last non-zero luminance and chroma coefficients is less than the corresponding threshold, it may be determined that the secondary transform index is not signaled.

[0195] According to an embodiment, the horizontal sum is obtained by summing the horizontal luma component and the horizontal chrominance component, and the vertical sum is obtained by summing the vertical luminance component and the vertical chrominance component. Can determine whether each of the horizontal sum and vertical sum is less than the corresponding threshold. When each of the horizontal sum and the vertical sum is less than the corresponding threshold, it may be determined that the secondary transform index is not signaled.

[0196] In the example, the first sum of the horizontal luminance component and the vertical luminance component is determined, and the second sum of the horizontal chrominance component and the vertical chrominance component is determined. It may be determined whether each of the first sum and the second sum is less than a corresponding threshold. When it is determined that each of the first sum and the second sum is less than the corresponding threshold, it may be determined that the secondary transform index is not signaled.

[0197] According to an embodiment, the total sum is obtained by summing the first sum and the second sum, when it is determined that the total sum is less than a threshold, it may be determined that the secondary transform index is not signaled.

[0198] According to an embodiment, a first minimum of the horizontal luminance component and a vertical luminance component is determined, and a second minimum of the horizontal chrominance component and the vertical chrominance component is determined. It is possible to determine whether each of the first minimum and second minimum is less than the corresponding threshold. When it is determined that each of the first minimum and second minimum is less than the corresponding threshold, it may be determined that the secondary transform index is not signaled. The above description can be adapted to use the first maximum of the horizontal luminance component and the vertical luminance component and the second maximum of the horizontal chrominance component and the vertical chrominance component to determine whether a secondary transform index is signaled.

[0199] Likewise, the minimum of the horizontal sum and the vertical sum can be used to determine whether a secondary transform index is being signaled. The maximum of the horizontal sum and the vertical sum can be used to determine whether a secondary transform index is being signaled.

[0200] According to aspects of the invention, when determining the number of non-zero transform coefficients for a CU, non-zero transform coefficients in the CB from the CU can be counted when the size of the CB (also referred to as the size of the CB) is greater than or equal to a size threshold, such as 4. In an example, when the size of the CB less than the size threshold, non-zero conversion coefficients in the CB are not calculated, i.e. are not included in the number of non-zero transform coefficients for the CU. The size threshold may be predetermined and stored in the decoder. The size threshold may be signaled explicitly, for example, from an encoder to a decoder. Further, when the number of non-zero transform coefficients in the CU is less than a threshold number, it can be determined that the secondary transform index is not signaled.

[0201] In the example, the color format for the CU is YUV 4:2:0. The CU includes a luminance block and two chrominance blocks that are aligned with the luminance block. The size threshold is 4. When a luminance block has a size of 4×N or N×4, where N may mean the width or height of the luma block and may be greater than or equal to 4, the two chrominance blocks have a size of 2×N/2 or N/2× 2. In the example, N is a positive even number. The number of non-zero transform coefficients for the CU is determined from the luminance block only, without considering the two chrominance blocks. The width or height of each of the two chroma blocks is less than the size threshold. In another example, the size threshold is 4x4 and the size of the CB 2xN/2 or N/2x2 is also less than the size threshold, where N can be a positive even number. Secondary conversion is not performed on two chrominance blocks.

[0202] In the example, the color format for the CU is YUV 4:2:2. The CU includes a luminance block and two chrominance blocks that are aligned with the luminance block. The size threshold is 4. When the luminance block has a size of 4×N, where N is greater than or equal to 4, the two chrominance blocks have a size of 2×N. The number of non-zero transform coefficients for the CU is determined from the luminance block only, without considering the two chrominance blocks. The width (eg, 2) of each of the two chrominance blocks is less than a size threshold. Secondary conversion is not performed on two chrominance blocks.

[0203] A secondary transform may be performed on a first block of coefficients (eg, TV) to obtain a first 4x2 (or 2x4) subblock that includes at least one non-zero coefficient. For example, RST is applied on a first 4×4 TV (eg, the first block of coefficients) to obtain a second TV that includes a first subblock and a second subblock. The first 4x2 (or 2x4) subblock includes at least one non-zero coefficient. The coefficients in the second 4x2 (or 2x4) subblock are set to 0. Thus, the 4x2 (or 2x4) subblock scan for the first subblock (i.e., the scan order of the 4x2 (or 2x4) coefficients )) is used for entropy coding of the second TV. In the example, the scan order of the 4x2 (or 2x4) coefficients is identical to the scan order that is used for entropy coding of a 4x2 (or 2x4) chrominance block.

[0204] Similarly, a secondary transform may be performed on a first coefficient block (e.g., TV) to obtain a first 8x4 (or 4x8) subblock that includes at least one non-zero coefficient, for example, when the first coefficient block is greater than 8x4 (or 4x8). For example, RST is applied on the first 8x8 TV (eg, the first block of coefficients) to obtain a second TV that includes the first subblock and the second subblock. The first 8x4 (or 4x8) subblock includes at least one non-zero coefficient. The coefficients in the second 8x4 (or 4x8) subblock are set to 0. Thus, the 8x4 (or 4x8) block scan for the first subblock (i.e., the scan order of the 8x4 (or 4x8) coefficients )) is used for entropy coding of the second TV. In the example, the scan order of the 8x4 (or 4x8) coefficients is identical to the scan order that is used for entropy coding of the 8x4 (or 4x8) chrominance block.

[0205] In FIG. 22 shows a flowchart of the operations of a process (2200) according to an embodiment of the invention. Process (2200) may be used in reconstructing an intra-coded block to generate a prediction block for the reconstructed block. In some examples, process (2200) may be used in reconstructing an inter-coded block. In various embodiments, process (2200) is performed by a processing circuit, e.g. that performs the functions of a video decoder (410), a processing circuit that performs the functions of a video decoder (510), a processing circuit that performs the functions of a video encoder (603), and the like. In some embodiments, the process (2200) is implemented in program instructions, such that when the processing circuitry executes the program instructions, the processing circuitry executes the process (2200). The process starts at step (S2201) and proceeds to step (S2210).

[0206] In step (S2210), the encoded CU information may be decoded from the encoded video bitstream. The encoded information may indicate the last position of the non-zero transform coefficients (or the position of the last non-zero coefficient) of the first CB from the CU. In an example, the CU may include a luminance block and a chrominance block. The first CB may be a luminance block or a chrominance block.

[0207] In step (S2220), it may be determined whether a secondary transform index is signaled in the encoded information based on the position of the last non-zero coefficient as described above. The secondary conversion index may indicate a secondary conversion to be performed on the second CB in the CU. The second CB may be a luminance block or a chrominance block.

[0208] In an example, the position of the last non-zero coefficient may include a horizontal component and a vertical component, and whether a secondary transform index in the encoded information is signaled may be determined based on the horizontal component and/or the vertical component. As described above, the side information can be used to determine whether the secondary transform index is signaled, and step (S2220) can be appropriately adapted to include the side information. In the example, when it is determined that the secondary transform index is signaled, the process (2200) proceeds to step (S2230). Otherwise, the process (2200) proceeds to step (S2250)

[0209] In step (S2230), whether to perform secondary transform on the second CB may be determined based on whether it is determined that the secondary transform index is signaled in the encoded information. In some examples, when it is determined that a secondary transform index is signaled, the secondary transform is determined to be performed. When it is determined that the secondary conversion is to be performed, the process (2200) proceeds to step (S2240). Otherwise, the process (2200) proceeds to step (S2250).

[0210] In step (S2240), the secondary conversion indicated by the secondary conversion index is performed on the second CB. The secondary transform may be NSST. The secondary transformation may be an RST including a zeroing method. For example, when the second CB is 8x8, RST is applied on the second CB to obtain a transformed block that includes a first 8x4 subblock and a second 8x4 subblock. The first subblock includes at least one non-zero coefficient. The coefficients in the second subblock are not calculated and are set to 0.

[0211] In step (S2250), the count in the second CB may be reconstructed. Then, the process (2200) proceeds to step (S2299) and ends.

[0212] Process (2200) may be appropriately adapted as described above. For example, one or more of the steps may be modified, omitted, or combined. In the example, steps (S2220) and (S2230) are combined. Additional step(s) may also be added. The order in which the process (2200) is executed can also be modified.

[0213] FIG. 23 shows a flowchart of the operations of a process (2300) according to an embodiment of the invention. Process (2300) may be used in reconstructing an intra-coded block to generate a prediction block for the reconstructed block. In some examples, process (2300) may be used in reconstructing an inter-coded block. In various embodiments, process (2300) is performed by a processing circuit, e.g. that performs the functions of a video decoder (410), a processing circuit that performs the functions of a video decoder (510), a processing circuit that performs the functions of a video encoder (603), and the like. In some embodiments, the process (2300) is implemented in program instructions, such that when the processing circuit executes the program instructions, the processing circuit executes the process (2300). The process starts at step (S2301) and proceeds to step (S2310).

[0214] In step (S2310), the encoding information of the CU may be decoded from the encoded video bitstream, where the encoding information indicates the size of the CU.

[0215] In step (S2320), whether the secondary transform is allowed may be determined based on the CU size and the CU size threshold. When the CU size is less than or equal to the CU size threshold, it is determined that the secondary transformation is enabled. The process (2300) proceeds to step (S2330). When the CU size is larger than the CU size threshold, it is determined that the secondary transformation is not allowed, and the process (2300) proceeds to step (S2350).

[0216] In step (S2330), it may be determined whether to perform secondary mapping on the CB in the CU, for example, based on whether the secondary mapping index is signaled as described above. When it is determined that the secondary conversion is to be performed, the process (2300) proceeds to step (S2340). Otherwise, the process (2300) proceeds to step (S2350).

[0217] In step (S2340), the secondary transformation indicated by the secondary transformation index is performed on the CB, similarly to step (S2240).

[0218] In step (S2350), the CB count may be reconstructed. Then, the process (2300) proceeds to step (S2399) and ends.

[0219] Process (2300) can be appropriately adapted. For example, one or more steps may be modified. Additional step(s) may also be added.

[0220] Process (2200) and process (2300) can be combined as appropriate. For example, (S2310) and (S2320) may be performed first, and then (S2210) to (S2250).

[0221] The above methods may be implemented in computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, in FIG. 24 shows a computer system (2400) suitable for implementing some embodiments of the disclosed invention.

[0222] Computer software may be coded using any suitable machine code or computer language that may be assembled, compiled, linked, or similarly processed to create code containing instructions that can be executed directly, or through interpretation, execution of microcode, etc. .p., one or more computer central processing units (CPUs, central processing units), graphic processors (GPUs, Graphics Processing Units), etc.

[0223] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, IoT devices, and the like.

[0224] The computer system components (2400) shown in FIG. 24 are illustrative in nature and are not intended to impose any limitation on the scope of application or functionality of computer software implementing embodiments of the present invention. The configuration of the components should also not be interpreted as having any dependency or requirement in connection with any one or combination of the components shown in the illustrative embodiment of the computer system (2400).

[0225] The computer system (2400) may include some human interface input devices. Such an input device may be responsible for input by one or more users through, for example, tactile input (e.g., keystrokes, waving, glove movements), audio input (e.g., voice, claps), visual input (e.g., gestures), olfactory input (not shown). Interface devices may also be used to capture certain media not necessarily directly related to conscious human input, such as sound (e.g. speech, music, external sound), images (e.g. scanned images, photographic images received from a still camera), video (eg, 2D video, 3D video including stereoscopic video).

[0226] The human interface input devices may include one or more of (one each depicted): a keyboard (2401), a mouse (2402), a touch pad (2403), a touch screen (2410), an information glove (not shown). ), joystick (2405), microphone (2406), scanner (2407), camera (2408).

[0227] The computer system (2400) may also include some human interface output devices. Such output devices may stimulate the senses of one or more users through, for example, tactile output, sound, light, and smell/taste. Such output devices may include tactile output devices (e.g., haptic feedback via touch screen (2410), data glove (not shown), or joystick (2405), but may also be tactile feedback devices that do not serve as input devices) , audio output devices (for example: speakers (2409), headphones (not shown)), visual output devices (for example, screens (2410), including CRT screens, LCD screens, plasma screens, OLED screens, each with touchscreen input capabilities, each with or without tactile feedback capabilities, some of which are capable of 2D visual output or more than 3D output through, for example, stereographic output; virtual reality goggles (not shown), holographic displays, and smoke tanks (not shown)) and printers (not shown).

[0228] The computer system (2400) may also include human accessible storage devices and associated media, such as optical media including CD/DVD ROM/RW (2420) with CD/DVD media (2421), etc. .p., flash memory cards (2422), removable hard drive or solid state drive (243), conventional magnetic media such as tape and floppy disk (not shown), dedicated ROM/ASIC/PLD based devices such as security hardware keys (not shown) and the like.

[0229] Those skilled in the art should also understand that the term "computer-readable media" as used in connection with the invention disclosed herein does not encompass transmission media, carrier waves, or other short-term signals.

[0230] The computer system (2400) may also include an interface to one or more communication networks. Networks can be, for example, wireless, wired, optical. Networks may additionally be local, wide area, metropolitan, transport and industrial, real-time, latency tolerant, and so on. Examples of networks include local area networks, such as Ethernet, wireless LANs, cellular networks, including GSM, 3G, 4G, 5G, LTE, etc., digital wide area networks for wired or wireless television, including cable television, satellite TV and terrestrial TV, transport and industrial include CANBus, etc. Some networks typically require external network interface adapters that are connected to some general purpose data ports or peripheral buses (2449) (for example, computer system USB ports (2400)); others are typically built into the core of the computer system (2400) by connecting to a system bus as described below (eg, an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system (2400) can communicate with other entities. Such communication may be unidirectional with receive-only capability (eg broadcast), unidirectional with transmit-only capability (eg CANbus to some CANbus devices), or bidirectional, eg to other computer systems using a local area network or a wide area digital network. Several protocols and protocol stacks may be used on each of these networks and the network interfaces described above.

[0231] The aforementioned human interface devices, human accessible storage devices, and network interfaces can be connected to the core (2440) of the computer system (2400).

[0232] The core (2440) may include one or more central processing units (CPUs) (2441), graphics processing units (GPUs) (2442), specialized programmable processing modules in the form of Field Programmable Gate Arrays (FPGAs). ) (2443), hardware accelerators (2444) for some tasks, etc. These devices, together with read-only memory (ROM) (2445), random access memory (2446), high-capacity internal storage such as internal hard drives (2447) not accessible to the user, SSD, etc., can be connected via the system bus (2448). In some computer systems, the system bus (2448) may be available in the form of one or more physical connectors to allow for expansions with additional CPUs, GPUs, and the like. Peripherals can be connected either directly to the system bus (2448) of the kernel, or via a peripheral bus (2449). Peripheral bus architectures include PCI, USB, and the like.

[0233] CPUs (2441), GPUs (2442), FPGAs (2443), and accelerators (2444) may execute some instructions that together may constitute the aforementioned computer code. This computer code can be stored in ROM

(2445) or RAM (2446). Transient data may also be stored in RAM (2446), while persistent data may be stored, for example, in an internal mass storage (2447). Fast storage and retrieval from any storage device can be achieved through the use of cache memory, which can be closely associated with one or more CPU (2441), GPU (2442), mass storage (2447), ROM (2445), RAM (2446) etc.

[0234] Computer-readable media may store computer code for performing various computer-implemented operations. The media and computer code may be specially created for the purposes of the present invention or may be well known and available to those skilled in the art of computer software.

[0235] By way of example, and not limitation, a computer system having an architecture (2400), and in particular a core (2440), can provide functionality through execution by processor(s) (including(and) CPU, GPU, FPGAs, accelerators, etc.) software embodied in one or more tangible computer-readable media. Such computer-readable media may be media associated with the user-accessible mass storage described above, as well as some persistent storage in the core (2440), such as internal mass storage (2447) or ROM (2445). Software implementing various embodiments of the present invention may be stored on such devices and executed by the core (2440). The computer-readable medium may include one or more memory devices or microcircuits, in accordance with particular needs. The software may direct the core (2440) and, in particular, its processors (including CPUs, GPUs, FPGAs, etc.) to execute specific processes or specific parts of the specific processes described here, including setting data structures stored in RAM (2446) , and modifying such data structures according to processes defined by the software. Additionally or alternatively, the computer system may provide functionality through logic hardwired or otherwise embodied in circuitry (e.g., an accelerator (2444)) that may operate in place of or in conjunction with software to perform specific processes or specific portions of the specific processes described herein. A reference to software may cover logic, and vice versa when appropriate. Reference to computer-readable media may encompass a circuit (eg, an integrated circuit (IC)) where software is stored for execution, a circuit embodying logic for execution, or both, as appropriate. The present invention encompasses any suitable combination of hardware and software.

Appendix A: Abbreviations

JEM: collaborative research model

VVC: Universal Video Coding

BMS: a set of standards

MV: motion vector

HEVC: High Performance Video Coding

SEI: Additional Improvement Information

VUI: video suitability information

GOP: group of images

TU: conversion unit,

PU: prediction unit

CTU: coding tree unit

CTB: coding tree block

RW: prediction block

HRD: Hypothetical Reference Decoder

SNR: signal to noise ratio

CPU: central processing unit

GPU: GPU

CRT: kinescope

LCD: liquid crystal display

OLED: organic light emitting diode

CD: CD

DVD: Digital Video Disc

ROM: Read Only Memory

RAM: random access memory

ASIC: Application Specific Integrated Circuit

PLD: Programmable Logic Device

LAN: local area network

GSM: Global System for Mobile Communications

LTE: a project for the long-term development of communication systems

CANBus: controller network bus

USB: Universal Serial Bus

PCI: Peripheral Component Interconnect

FPGA: Field Programmable Gate Array

SSD: solid state drive

IC: integrated circuit

CU: coding unit.

[0236] Although several illustrative embodiments are described here, changes, permutations, and various equivalents for substitution are possible within the scope of the invention. Thus, those skilled in the art can provide numerous systems and methods that, although not explicitly shown or described herein, embody the principles of the invention and thus fall within its spirit and scope.

Annex I

4x4 transformation

8×8 transformation

16×16 conversion

Transformation 32×32

Appendix II

Core 64-point DCT-2

where

Appendix III

4-point DST-7

where {a, b, c, d}={29, 55, 74, 84}

8-point DST-7:

where {a, b, c, d, e, f, g, h}={17, 32, 46, 60, 71, 78, 85, 86}

16-point DST-7

where {a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p}={9, 17, 25, 33, 41, 49, 56, 62, 66, 72, 77, 81, 83, 87, 89, 90}

32-point DST-7

where {a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s, t, u, v, w, x, y, z, A, B, C, D, E, F}={4, 9, 13, 17, 21, 26, 30, 34, 38, 42, 45, 50, 53, 56, 60, 63, 66, 68, 72, 74, 77, 78, 80, 82, 84, 85, 86, 88, 88, 89, 90, 90}

4-point DCT-8

where {a, b, c, d}={84, 74, 55, 29}

8-point DCT-8:

where {a, b, c, d, e, f, g, h}={86, 85, 78, 71, 60, 46, 32, 17}

16-point DCT-8

where {a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p}={90, 89, 87, 83, 81, 77, 72, 66, 62, 56, 49, 41, 33, 25, 17, 9}

32-point DCT-8

where {a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s, t, u, v, w, x, y, z, A, B, C, D, E, F}={90, 90, 89, 88, 88, 86, 85, 84, 82, 80, 78, 77, 74, 72, 68, 66, 63, 60, 56, 53, 50, 45, 42, 38, 34, 30, 26, 21, 17, 13, 9, 4}

Claims (32)

1. A video decoding method in a decoder, in which: decoding encoded coding unit (CU) information from the encoded video bitstream, the encoded information indicating the last position of non-zero transform coefficients of the first coding block (CB) in the CU; determining whether a secondary transform index is signaled in the encoded information based on whether the last non-zero transform coefficient position indicates that the first CB in the CU has only zero coefficients or only one non-zero coefficient in this last coefficient position, determining that the secondary transform index no transform is signaled in the coded information if the last non-zero transform coefficient position indicates that the first CB in the CU has only zero coefficients or only one non-zero coefficient in the last coefficient position; determining whether to perform said secondary transform on the second CB in the CU based on whether the secondary transform index is signaled in the encoded information; in response to determining that the secondary transformation is to be performed, performing a secondary transformation on the second CB and reconstructing the second CB; and in response to determining that the secondary transformation is not to be performed, the second CB is reconstructed without performing the secondary transformation on the second CB. 2. The method of claim 1, wherein said last position includes a horizontal component and a vertical component, both of which have a value of zero. 3. The method of claim 2, wherein, at the step of determining whether the secondary transform index is being signaled, one of the following is performed: (a) determining whether the horizontal component of said last position is less than a first threshold and whether the vertical component of said last position is less than a second threshold; and in response to determining that the horizontal component is less than the first threshold and determining that the vertical component is less than the second threshold, determining that the secondary transform index is not signaled in the encoded information; (b) determining if the sum of the horizontal component and the vertical component of the last position is less than a threshold; and in response to determining that the sum is less than a threshold, determining that the secondary transform index is not signaled in the encoded information; (c) determining if at least one of (i) the horizontal component and (ii) the vertical component of the last position is less than a threshold; and in response to determining that at least one of them is less than a threshold, determining that the secondary transform index is not signaled in the encoded information; (d) determining if at most one of (i) the horizontal component and (ii) the vertical component of the last position is less than a threshold; and in response to determining that at most one of them is less than a threshold, it is determined that the secondary transform index is not signaled in the encoded information. 4. The method according to any one of paragraphs. 1-3, in which the first CB is the luma block; the last position is the last brightness position for the brightness block; and in the step of determining whether the second transform index is signalled, determining whether the secondary transform index is signaled based on the last luminance position. 5. The method according to any one of paragraphs. 1, 2, in which the first CB is the luma block; the last position is the last brightness position for the brightness block; The CU also includes a chrominance block; the encoded information also indicates the last chrominance position of the non-zero transform coefficients for the chrominance block; and in the step of determining whether the secondary transform index is signaled, determining whether the secondary transform index is signaled based on the last luminance position and the last chrominance position. 6. The method according to any one of paragraphs. 1-5, in which: decoding the encoded CU information from the encoded video bitstream, the encoded information indicating the size of the CU; and determining whether said secondary transformation is allowed based on the CU size and the CU size threshold, wherein when the CU size is less than or equal to the CU size threshold, it is determined that the secondary transformation is allowed, and when the CU size is greater than the CU size threshold, it is determined that the secondary transformation is not allowed. 7. The method of claim 6, wherein the CU size threshold is a maximum transform unit size in the CU. 8. Device for video decoding, containing a processing circuit configured to perform the method according to any one of paragraphs. 1-7.

Images