RU2785384C1 - Image decoding device using differential encoding - Google Patents

Abstract

FIELD: imaging technology.

SUBSTANCE: invention relates to the field of video encoding and decoding using the differential encoding method performed in the conversion skip mode. A video decoding method is proposed for restoring the current block encoded using an intra-prediction, which includes decoding direction information from the bitstream indicating the direction of differential encoding applied to the residual block of the current block, generating a residual block from residual signals by restoring and using information about residual signals from the bitstream and using direction information, reverse quantization of the modified residual block, generating a prediction block for the current block by means of an intra-prediction and restoring the current block by adding a prediction block and a reverse quantized residual block.

EFFECT: increase in the efficiency of video encoding and decoding.

10 cl, 9 dwg, 4 tbl

Description

The field of technology to which the invention belongs

[1] The present description, using some embodiments as an example, relates to encoding and decoding images or videos. More specifically, the present disclosure relates to video encoding and decoding using a differential coding technique performed in a transform skipping mode.

State of the art

[2] The statements in this section provide only background information relating to the present disclosure and do not necessarily constitute prior art.

[3] Since video data has a large amount of data compared to audio data or still image data, a lot of hardware resources, including memory, are required to store or transmit the data in raw form before being subjected to the compression process.

[4] Accordingly, storage or transmission of video data is typically accompanied by compression using an encoder before the decoding module can receive, decompress and play the compressed video data. Known video compression technologies include the H.264/AVC standard and the High Efficiency Video Coding (HEVC) standard, which improves H.264/AVC coding efficiency by about 40%.

[5] However, the ever-increasing size, resolution, and frame rate of video images, and the consequent increase in the amount of data to be encoded, requires a new and improved compression method with higher coding efficiency and higher image quality compared to known compression methods.

Disclosure of the essence of the invention

Technical problem

[6] The present disclosure, in some embodiments, is directed to an apparatus and method for decoding video using a differential coding technique for a residual signal during decoding in a transform skipping mode.

Technical solution

[7] At least one aspect of the present disclosure provides a video decoding method for recovering a current block that is encoded using intra prediction. The method includes: decoding from the bitstream direction information indicating the direction of the differential encoding applied to the residual block of the current block, generating the residual block by recovering information about the residual signals from the bitstream, and generating a modified residual block by modifying the residual signals in the residual block by direction information. Each target residual signal for modification in the residual block is modified by adding to one or more residual signals in the residual block that are located in the direction indicated by the direction information from the target residual signal. The method further includes inverse quantizing the modified residual block into an inversely quantized residual block, generating a prediction block for the current block using intra prediction, and reconstructing the current block by adding the prediction block and the inversely quantized residual block.

[8] Another aspect of the present disclosure provides a video decoding apparatus for recovering a current block that is encoded using intra prediction. The video decoding apparatus includes an entropy decoding module, an inverse quantization module, an intra-prediction module, and an adder. The entropy decoding module is configured to decode from the bitstream direction information indicating the direction of the differential encoding applied to the residual block of the current block, and to generate the residual block by recovering the residual signal information from the bitstream. The inverse quantizer is configured to generate a modified residual block by modifying the residual signals in the residual block based on the direction information, and to inverse quantize the modified residual block into an inversely quantized residual block. The intra-prediction module is configured to generate a prediction block for the current block by intra-prediction. The adder is configured to add a prediction block and an inversely quantized residual block to the reconstructed current block. Wherein, each target residual signal for modification in the residual block is modified by summing with one or more residual signals in the residual block that are located in the direction indicated by the direction information from the target residual signal.

Benefits of the Invention

[9] As described above, some embodiments of the present disclosure provide a video decoding apparatus and method using a differential coding technique for a residual signal during decoding in a transform skip mode, which provides improved compression performance.

Brief description of the drawings

[10] FIG. 1 is a diagram illustrating a video coding device that can implement the methods of the present invention.

[11] FIG. 2 is a diagram explaining a block splitting method using a QTBTTT structure.

[12] FIG. 3A and 3B are diagrams of a plurality of intra-prediction modes, including wide-angle intra-prediction modes.

[13] FIG. 4 is a diagram illustrating a video decoding apparatus capable of implementing the techniques proposed by the present invention.

[14] FIG. 5 illustrates differential coding techniques according to at least one embodiment of the present disclosure.

[15] FIG. 6 is a flowchart of a video decoding method for an intra prediction encoded block using an explicit differential coding technique according to at least one embodiment of the present disclosure.

[16] FIG. 7 is a flowchart of a video decoding method for an inter prediction encoded block using an explicit differential coding technique according to at least one embodiment of the present disclosure.

[17] FIG. 8 shows a flowchart of a video decoding method for an intra prediction encoded block using an implicit differential coding technique according to another embodiment of the present disclosure.

List of reference symbols

Implementation of the invention

[18] Hereinafter, an embodiment of the present invention will be described in detail with reference to the attached drawings. In the following description, like reference numerals preferably refer to like elements, even though these elements are shown in different drawings. In addition, for the sake of clarity and brevity, in the following description of some embodiments, a detailed description of known components and functions related thereto is omitted when they obscure the subject matter of the present invention.

[19] Embodiments of the present disclosure relate to video encoding and decoding. More specifically, the present disclosure provides an apparatus and method for decoding video using a differential coding technique for a residual signal to improve compression efficiency during coding in a skip transform mode.

[20] FIG. 1 is a diagram illustrating a video coding device that can implement the methods of the present invention. Next, the video encoding device and subcomponents of the device will be described with reference to FIG. one.

[21] The video coding device may be configured to include a block division module 110, a prediction module 120, a subtraction module 130, a transform module 140, a quantization module 145, a reorder module 150, an entropy coding module 155, an inverse quantization module 160, a 165 inverse transform, adder 170, filter module 180 and memory 190.

[22] The respective components of the video encoding device may be implemented as hardware or software, or as a combination of hardware and software. In addition, the function of each component may be implemented in software, and the software function for each component may be implemented to be executed by a microprocessor.

[23] A video image is composed of a plurality of images. Each image is divided into a plurality of sections, and encoding is performed for each section. For example, one image is divided into one or more fragments and/or sections. Here, one or more fragments may be defined as a group of fragments. Each fragment and/or section is divided into one or more coding tree units (CTUs). And each CTU is divided into one or more coding units (CU) in a tree structure. Information applicable to corresponding CUs is encoded as CU syntax, and information normally applicable to CUs included in one CTU is encoded as CTU syntax. In addition, information generally applicable to all blocks in one section is encoded as section header syntax, and information applicable to all blocks constituting one picture is encoded in a picture parameter set (PPS) or picture header. In addition, information commonly referred to by a plurality of pictures is encoded in a sequence parameter set (SPS). In addition, information typically referred to by one or more SPSs is encoded in a Video Parameter Set (VPS). In the same way, information normally applied to a single fragment or group of fragments can be encoded as fragment header or fragment group header syntax.

[24] Block division unit 110 determines the size of a coding tree unit (CTU). CTU size information (CTU size) is encoded as SPS syntax or PPS set and transmitted to the video decoding apparatus.

[25] The image dividing unit 110 divides each image constituting a video image into a plurality of CTUs having a predetermined size, and then uses a tree structure to recursively divide the CTUs. The leaf node in the tree structure becomes the CU coding unit, which is the base coding unit.

[26] The tree structure for can be a quadtree (QuadTree (QT)), in which the top node (or parent node) is divided into four lower nodes of the same size (or child nodes), a binary tree (BinaryTree (BT)), in which the top node is divided into two bottom nodes, by a ternary tree (TernaryTree (TT)), in which the top node is divided into three bottom nodes with a size ratio of 1:2:1, or by a combination of two or more of the QT structure, the BT structure, and the TT structure. For example, a structure including quadtree and binary tree (QTBT) or a structure including quadtree and binary tree ternary tree (QTBTTT) may be used. Here, the BTTT may be collectively referred to as a multiple type tree (MTT).

[27] FIG. 2 shows a tree structure of the QTBTTT partition. As shown in FIG. 2, the CTU can be first divided into a QT structure. The splitting of the quadtree can be repeated until the splitting block size reaches the minimum block size (MinQTSize) of the leaf node allowed in the QT structure. The first flag (QT_split_flag) indicating whether each node of the QT structure is divided into four lower level nodes is encoded by the entropy encoding unit 155 and transmitted to the video decoding apparatus. If the leaf node of the QT structure does not exceed the maximum size (MaxBTSize) of the root node block allowed in the BT structure, it may be further divided into any one or more BT structures or TT structures. In the BT structure and/or the TT structure, there may be a plurality of separation directions. For example, there may be two directions in which the block of the corresponding node splits horizontally and vertically. As shown in FIG. 2, when the MTT split starts, a second flag (mtt_split_flag) indicating whether the nodes are split and, if so, an additional flag indicating the split directions (vertical or horizontal) and/or a flag indicating the split or type of split (binary or ternary ) is encoded by the entropy encoding unit 155 and transmitted to the video decoding apparatus.

[28] Alternatively, before encoding the first flag (QT_split_flag) indicating whether each node is split into four lower level nodes, a CU unit split flag (split_cu_flag) indicating whether the node is split or not may be encoded. When the value of the CU split flag (split_cu_flag) indicates that splitting is not performed, the block of the corresponding node becomes a leaf node in the split tree structure and serves a coding unit (CU), which is a basic coding unit. When the value of the CU split flag (split_cu_flag) indicates that the node has been split, the video encoder starts encoding from the first flag in the manner described above.

[29] As another example of a tree structure that uses the QTBT structure, there are two types of partitioning, including the type in which the block of the corresponding node is horizontally divided into two blocks of the same size (i.e., symmetrical horizontal partitioning), and the type , in which it is divided vertically (i.e. symmetrical vertical separation). The entropy encoding unit 155 to be encoded and transmitted to the video decoding apparatus is a split flag (split_flag) indicating whether each node of the BT structure is divided into lower-level blocks, and split type information indicating its split type. Meanwhile, there may be an additional type in which the block of the corresponding node is split into two asymmetrically shaped blocks. The asymmetric shape may include a block shape of the corresponding node being split into two rectangular blocks having an aspect ratio of 1:3, or a block shape of the corresponding node being split in a diagonal direction.

[30] The CU may have different sizes depending on the structure of the QTBT or QTBTTT partition of the CTU. Hereinafter, a block corresponding to a CU to be encoded or decoded (ie, a QTBTTT end node) is referred to as a "current block". When using the QTBTTT division, the current block can be in the form of not only a square, but also a rectangle.

[31] Prediction module 120 predicts the current block to generate a prediction block. The prediction module 120 includes an intra prediction module 122 and an inter prediction module 124 .

[32] In general, each of the current image blocks may be predictively encoded. Prediction of the current block can usually be performed using an intra-prediction method or an inter-prediction method, wherein the intra-prediction method uses data from the image containing the current block, and the inter-prediction method uses data from the image encoded before the image containing the current block . Inter prediction includes both unidirectional and bidirectional prediction.

[33] The intra-prediction module 122 predicts pixels in the current block using neighboring pixels (reference pixels) located around the current block in the current image. There are many intra-prediction modes according to prediction directions. For example, as shown in FIG. 3A, the multiple intra prediction modes may include 2 non-directional modes including planar mode and DC mode, as well as 65 directional modes. The respective prediction modes provide various respective definitions of neighboring pixels and calculation formulas to be used.

[34] Additional directional modes may be used for efficient directional prediction of a square wave current block, as shown in FIG. 3B, the dotted arrows of intra-prediction modes (numbered 67-80 and numbers -1 to -14. These may be referred to as "wide-angle intra-prediction modes." The arrows in FIG. 3B indicate the respective reference samples used for prediction rather than direction prediction.The prediction direction is opposite to the direction indicated by the arrow.Wide-angle intra-prediction mode is a mode in which prediction is performed in the opposite direction of a certain directional mode without additional bit transmission when the current block is rectangular.In this case, from the wide-angle intra- Some wide-angle intra-prediction modes available for use in the current block may be determined by the ratio of the width to height of the rectangular current block.For example, wide-angle intra-prediction modes that have an angle of less than 45 degrees (intra-prediction modes numbered 67-80) , can be used when the current block is rectangular with height less than width. Wide-angle intra-prediction modes having an angle of -135 degrees or more (intra-prediction modes numbered from -1 to -14) can be used when the current block has a rectangular shape with a height greater than the width.

[35] Intra-prediction module 122 may determine the intra-prediction mode to be used to encode the current block. In some examples, intra-prediction module 122 may encode the current block using multiple intra-prediction modes and select an appropriate intra-prediction mode to use from among the tested modes. For example, intra-prediction module 122 may calculate velocity distortion values by analyzing the rate-distortion of several tested intra-prediction modes and select the intra-prediction mode that has the best rate-distortion performance among the tested modes.

[36] The intra prediction unit 122 selects one intra prediction mode from a plurality of intra prediction modes, and predicts the current block using at least one neighboring pixel (reference pixel) determined according to the selected intra prediction mode and calculation formula. Information about the selected intra-prediction mode is encoded by the entropy encoding module 155 and transmitted to the video decoding device.

[37] The inter prediction unit 124 generates a prediction block for the current block through a motion compensation process. The inter-prediction module 124 searches for a block most similar to the current block in a reference picture that was encoded and decoded earlier than the current picture, and generates a prediction block of the current block using the found block. Next, inter-prediction module 124 generates a motion vector corresponding to an offset between the current block in the current picture and the prediction block in the reference picture. In general, motion estimation is performed for the luminance component, and a motion vector calculated from the luminance component is used for both the luminance component and the chrominance component. Motion information including reference picture information and motion vector information used to predict the current block is encoded by the entropy encoding unit 155 and transmitted to the video decoding apparatus.

[38] The subtractor 130 generates a residual block by subtracting from the current block the prediction block generated by the intra-prediction module 122 or the inter-prediction module 124.

[39] Transform unit 140 converts the residual signal in the residual block having pixel values in the spatial domain into transform coefficients in the frequency domain. The transform unit 140 may transform the residual signals in the residual block using the entire size of the residual block as a transform block, or divide the residual block into a plurality of subblocks and use the subblocks as transform blocks to perform transform. Alternatively, the transform unit 140 may divide the residual block into two sub-blocks, which are a transform region and a non-transform region, and use only the transform region sub-block as a transform block for transforming the residual signals. Here, the transform region sub-box may be one of two rectangular boxes having a 1:1 aspect ratio based on the horizontal axis (or vertical axis). In this case, a flag (cu_sbt_flag) indicating that only one sub-block has been converted, direction information (vertical/horizontal) (cu_sbt_horizontal_flag) and/or position information (cu_sbt_pos_flag) are encoded by the entropy encoding unit 155 and transmitted to the video decoding apparatus. In addition, the sub-block size of the transform area may have an aspect ratio of 1:3 based on the horizontal axis (or vertical axis). In this case, a flag (cu_sbt_quad_flag) specifying the corresponding division is further encoded by the entropy encoder 155 and transmitted to the video decoding apparatus.

[40] Meanwhile, the conversion unit 140 can separately convert the residual block in the horizontal directions and the vertical directions. Various types of transformation functions or transformation matrices may be used for the transformation operation. For example, pairs of transform functions for horizontal transform and vertical transform may be defined as a multiple transform set (MTS). Transform unit 140 may select one pair of transform functions having the highest transform efficiency in the MTS and transform the residual block in the horizontal and vertical directions, respectively. Information (mts_idx) about the pair of transform functions selected in the MTS is encoded by the entropy encoder 155 and transmitted to the video decoder. The size of the transform block to which the MTS is applied may be limited to 32×32 or less.

[41] The quantization unit 145 quantizes the transform coefficients output from the transform unit 140 and outputs the quantized transform coefficients to the entropy encoding unit 155 .

[42] The reordering module 150 may perform reordering of the coefficient values using the quantized transform coefficients.

[43] Reordering module 150 may use a coefficient scan to transform a two-dimensional coefficient matrix into a one-dimensional coefficient sequence. For example, reordering module 150 may scan the DC coefficients to coefficients in the high frequency region by zigzag scanning or diagonal scanning to output a one-dimensional sequence of coefficients. Depending on the size of the transform block and the intra-prediction mode, the zigzag scan used can be replaced by a vertical scan to scan the 2D coefficient array in the column direction and a horizontal scan to scan the coefficients into a 2D block shape in the row direction. In other words, the scanning method to be used can be selected from zigzag scanning, diagonal scanning, vertical scanning and horizontal scanning according to the transform block size and the intra prediction mode.

[44] The entropy encoding unit 155 encodes the sequence of one-dimensional quantized transform coefficients output from the reordering unit 150 using various encoding methods such as context-dependent adaptive binary arithmetic code (CABAC), exponential Golomb algorithm, and the like. encoding to generate the bitstream.

[45] In addition, the entropy encoding unit 155 encodes block division information such as CTU size, CU division flag, QT division flag, MTT division type, and MTT division direction, allowing the video decoding apparatus to divide the block in the same manner as and a video encoding device. In addition, the entropy encoding unit 155 encodes prediction type information indicating whether the current block is encoded by intra prediction or inter prediction, and encodes, depending on the prediction type, intra prediction information (that is, intra prediction mode information). ) or inter-prediction information (ie, reference picture and motion vector information).

[46] The inverse quantizer 160 performs inverse quantization of the quantized transform coefficients output from the quantizer 145 to generate transform coefficients. The inverse transform unit 165 converts the transform coefficients output from the inverse quantization unit 160 from the frequency domain to the spatial domain to recover the residual block.

[47] The adder 170 adds the recovered residual block and the prediction block generated by prediction unit 120 to recover the current block. The pixels in the reconstructed current block are used as reference pixels in intra-prediction of the next block.

[48] The filter unit 180 performs filtering on the reconstructed pixels to reduce blocky artifacts, ringing artifacts, blur artifacts, etc. generated due to block-based prediction and transform/quantization. Filter module 180 may include a deblocking filter 182 and an adaptive sample offset (SAO) filter 184.

[49] The deblocking filter 180 filters the boundary between the reconstructed blocks to remove blocking artifact caused by block-by-block encoding/decoding, and the SAO filter 184 performs additional filtering of the deblocking filtered video. The SAO filter 184 is a filter used to compensate for the difference between the reconstructed pixel and the original pixel caused by lossy coding.

[50] The reconstructed block is filtered through the deblocking filter 182 and the SAO filter 184 and stored in the memory 190. When all blocks in one image are restored, the reconstructed image can be used as a reference image for inter-prediction of blocks in the image to be subsequently encoded.

[51] FIG. 4 is a block diagram illustrating a video decoding apparatus configured to implement the methods of the present disclosure. Hereinafter, the video decoding apparatus and components of the apparatus will be described with reference to FIG. four.

[52] The video decoding apparatus may be configured to include an entropy decoding unit 410, a reordering unit 415, an inverse quantization unit 420, an inverse transform unit 430, a prediction unit 440, an adder 450, a filter unit 460, and a memory 470.

[53] As with the video encoding apparatus of FIG. 1, the respective components of the video decoding apparatus may be implemented as hardware or software, or as a combination of hardware and software. In addition, the function of each component may be implemented in software, and the software function for each component may be implemented to be executed by a microprocessor.

[54] The entropy decoding unit 410 decodes the bitstream generated by the video encoding apparatus and extracts block separation information to determine the current block to be decoded, and extracts prediction information and residual signal information and the like necessary to reconstruct the current block .

[55] The entropy decoding module 410 extracts the CTU size information from the sequence parameter set (SPS) or picture parameter set (PPS), determines the size of the CTU, and divides the image into CTUs of a given size. Then, the entropy decoding module 410 determines the CTU as the highest level, that is, the root node of the tree structure, and extracts partitioning information per CTU, and thereby partitions the CTU using the tree structure.

[56] For example, when splitting a CTU using the QTBTTT structure, the first flag (QT_split_flag) associated with QT splitting is first extracted and each node is split into four lower layer nodes. For a node corresponding to a QT end node, entropy decoding module 410 extracts a second flag (MTT_split_flag) associated with the MTT split and split direction (vertical/horizontal) and/or split type (binary/ternary) information to split the corresponding end node. on the MTT structure. This allows you to recursively split the corresponding nodes below the QT end node into a BT or TT structure.

[57] As another example, when splitting the CTU using the QTBTTT structure, entropy decoding module 410 may first extract a CU split flag (split_cu_flag) indicating whether the CU is split. When the corresponding block is split, it can extract the first flag (QT_split_flag). During the splitting process, each node may have zero or more recursive QT splits followed by zero or more recursive MTT splits. For example, a CTU may immediately enter an MTT division or, conversely, have only a few QT divisions. As another example, when splitting a CTU using the QTBT structure, entropy decoding module 410 extracts the first flag (QT_split_flag) associated with QT split to split each node into four lower layer nodes. And, for a node corresponding to a QT end node, entropy decoding module 410 extracts a split flag (split_flag) indicating whether the node is further split into BT and split direction information.

[58] Meanwhile, when the entropy decoding unit 410 determines the current block to be decoded by dividing the tree structure, it extracts prediction type information indicating whether the current block was predicted by intra-prediction or inter-prediction. When the prediction type information indicates an intra prediction, the entropy decoding unit 410 extracts a syntax element for the intra prediction information (intra prediction mode) of the current block. When the prediction type information indicates an inter prediction, the entropy decoding unit 410 extracts a syntax element for the inter prediction information, that is, information indicating a motion vector and a reference picture referenced by the motion vector.

[59] In addition, the entropy decoding unit 410 extracts information about the quantized transform coefficients of the current block as information about the residual signal.

[60] The reordering unit 415 reverses the coefficient scan performed by the video encoding apparatus, the sequence of one-dimensional quantized transform coefficients entropy decoded by the entropy decoding unit 410 into a two-dimensional array of coefficients, (i.e., a block).

[61] An inverse quantizer 420 performs inverse quantization of the quantized transform coefficients. An inverse transform unit 430 inversely transforms the inverse quantized transform coefficients from the frequency domain to the spatial domain to recover the residual signals and thus generate a reconstructed residual block of the current block.

[62] In addition, when the demapper 430 demaps only a partial area (subblock) of a transform block, it extracts a flag (cusbt_flag) indicating that only the subblock of the transform block has been converted, the directionality of the subblock (vertical/horizontal) information (cu_sbt_horizontal_flag ) and/or subblock position information (cu_sbt_pos_flag) and back converts the subblock transform coefficients from the frequency domain to the spatial domain to recover the residual signals. At the same time, the demapper 430 fills the remaining area that is not demapped with "0" as residual signals, and thereby generates the last residual block for the current block.

[63] In addition, when the MTS is applied, the inverse transform module 430 uses the MTS information (mts_idx) transmitted from the video encoding device to determine the transform functions or transform matrices to be applied in the horizontal and vertical directions, respectively, and uses the determined function conversion to perform inverse conversion of the conversion coefficients to the conversion unit in the horizontal and vertical directions.

[64] The prediction module 440 may include an intra-prediction module 442 and an inter-prediction module 444. The intra prediction module 442 is activated when the prediction type of the current block is intra prediction, and the inter prediction module 444 is activated when the prediction type of the current block is inter prediction.

[65] The intra-prediction module 442 determines, among the plurality of intra-prediction modes, the intra-prediction mode of the current block from the syntax element for the intra-prediction mode extracted by the entropy decoding module 410, and according to the determined intra-prediction mode, it predicts the current block using adjacent reference pixels of the current block.

[66] The inter-prediction module 444 uses the syntax element for the intra-prediction mode extracted by the entropy decoding module 410 to determine the motion vector of the current block and the reference picture referenced by the motion vector, and it uses the motion vector and the reference picture to predict the current block.

[67] The adder 450 adds the residual block output from the inverse transform module and the prediction module output from the inter-prediction module or the intra-prediction module to reconstruct the current block. The pixels in the reconstructed current block are used as reference pixels in intra-prediction of the next block to be decoded.

[68] The filter module 460 may include a deblocking filter 462 and a SAO filter 464. The deblocking filter 462 performs deblocking filtering at a boundary between reconstructed blocks to remove blocking artifact caused by block decoding. The SAO filter 464 performs additional filtering on the reconstructed block after deblocking filtering to compensate for the difference between the reconstructed pixel and the original pixel caused by lossy coding. The reconstructed block is filtered through a deblocking filter 462 and a SAO filter 464 and stored in memory 470. When all blocks in one picture are reconstructed, the reconstructed picture can be used as a reference picture for inter-prediction of blocks in the picture to be subsequently encoded.

[69] As described above, the video encoding apparatus converts the residual signals into frequency domain transform coefficients, encodes and signals the transform coefficients to the video decoding apparatus. The video decoder inversely transforms the transform coefficients into residual signals in the spatial domain. However, depending on the characteristics of the video or residual signals, most of the transform coefficients generated from the transform may be in the high frequency region. In this case, the frequency domain coding transform coefficients may have a lower coding gain than the coding of the spatial domain residual signals.

[70] The skip transform (TS) mode is a method of direct entropy coding of residual signals without their transformation, and generally does not have superior coding performance compared to the discrete cosine transform (DCT). However, since the screen content contains many high frequency residual components that occur at the edge of high color contrast graphic elements, the transform skip mode is useful for video with screen content coding (SCC) or similar characteristics.

[71] When the skip transform mode is applied, a skip transform residual coding process is used to quantize and encode the predicted residual signals. A transform skip coding process is a technique in which the transform coefficient coding process is modified to be suitable for SCC.

[72] Transform skip mode information (transform_skip_flag [x0] [y0] [cIdx], hereinafter transform_skip_flag) may be delivered in units of transform blocks. Here (x0, y0) is the upper left position of the transform block, cIdx is the indicator for color elements, where 0 denotes luma, 1 denotes Cb chroma, and 2 denotes Cr. When the transformation skip mode "transform_skip_flag" is 1, the transformation is skipped, and when it is 0, whether to perform the transformation or skip it may be determined according to another syntax element.

[73] Meanwhile, the size of the transform block to which the transform skipping mode can be applied may be limited to 32×32. In this case, the encoding of the transform_skip_flag can be omitted for a transform block larger than the transform block size to which the transform skip mode can be applied, and the video decoding apparatus does not apply the transform skip mode to the transform block when it is larger than 32x2. Alternatively, the transform block size information to which the transform-skip mode can be applied can be encoded as a high-level syntax, such as a syntax element in an SPS or PPS, and signaled to a video decoder. For a transform block larger than the size defined by the high-level syntax, the transform_skip_flag encoding is omitted. The video decoding apparatus decodes the size information of a transform block to which the transform skip mode can be applied, and it does not decode the transform_skip_flag and does not apply the transform skip mode when the current transform block to be transformed is larger than the size specified in the size information.

[74] Information about the MTS and the skip conversion mode may be combined as shown in Table 1 and expressed in a single syntax.

[75]

[76] As shown in Table 1, MTS and skip conversion mode can be set according to tu_mts_idx. Here, for the range 0 to 5, tu_mts_idx is expressed as truncated unary codes.

[77] The following describes a differential coding method for improving the compression performance in the transform-skip mode with a video encoding apparatus and a video decoding apparatus.

[78] When a transform unit (TU) is encoded in a lossy compression transform skip mode, a differential coding technique may be applied to the residual signal after intra-prediction or inter-prediction. In the transform-skip mode, the differential encoding technique can reduce the total residual component energy for entropy encoding, and thus provide further improved compression performance.

[79] FIG. 5 illustrates differential coding techniques according to at least one embodiment of the present disclosure.

(0≤i<N, i - натуральное число) могут быть выражены Уравнением 1. В частности, остаточный сигнал в позиции (i, j) модифицируется посредством вычитания из него остаточного сигнала в позиции (i, j-1). Здесь (i, j) обозначает i-й ряд и j-й столбец.[80] In the horizontal differential coding method, as shown in FIG. 5 in (A), the current sample is predicted by using, among the samples encoded by the video coding device, the leftmost column residual component in the horizontal direction. By applying the horizontal differential coding method to the residual signals r _i,j of a block of size N × N (N is a natural number), the resulting residual signals

(0≤i<N, i is a natural number) can be expressed by Equation 1. In particular, the residual signal at position (i, j) is modified by subtracting the residual signal at position (i, j-1) from it. Here (i, j) denotes the i-th row and j-th column.

[81]

[82] Here, Q(r) is the reconstructed residual signal including quantization noise.

в устройство декодирования и резервирует их после восстановления для предсказывания остаточных сигналов следующего столбца. Процесс горизонтального предсказания может выполняться последовательно для всех столбцов блока.[83] As shown in Equation 1, in horizontal differential encoding, the encoder entropy encodes and then transmits the residual signals

to the decoder and reserves them after recovery to predict the residual signals of the next column. The horizontal prediction process may be performed sequentially for all columns of the block.

блока N×N, результирующие остаточные сигналы

(0≤i<N, i - натуральное число) могут быть выражены Уравнением 2. В частности, остаточный сигнал в позиции (i, j) модифицируется посредством вычитания из него остаточного сигнала в позиции (i-1, j). Здесь (i, j) также обозначает i-й ряд и j-й столбец.[84] In the horizontal differential coding method, as shown in FIG. 5 in (B), the current sample is predicted by using, among the samples encoded by the video encoding device, the leftmost column residual component in the horizontal direction. After applying the vertical differential coding method to the residual signals

block N×N, the resulting residual signals

(0≤i<N, i is a natural number) can be expressed by Equation 2. In particular, the residual signal at position (i, j) is modified by subtracting the residual signal at position (i-1, j) from it. Here (i, j) also denotes the i-th row and the j-th column.

[85]

на сторону устройства декодирования и резервирует их после восстановления для предсказывания остаточных сигналов следующего ряда. Процесс вертикального предсказания может выполняться последовательно для всех рядов блока.[86] As shown in Equation 2, in vertical differential encoding, the encoder side entropy encodes and then transmits the residual signals

to the side of the decoder and reserves them after recovery to predict the residual signals of the next row. The vertical prediction process may be performed sequentially for all rows of the block.

[87] When the horizontal differential coding method is applied, the video decoder recovers the residual signals as shown in Equation 3. In other words, the residual signals in the residual block recovered by the video decoder from the bitstream are modified according to the horizontal differential coding method. The target residual signal to be changed in the reconstructed residual block is modified such that the left residual signals located in the same row to the left of the target residual signal are added to the target residual signal.

[88]

[89] The video decoding apparatus may sequentially add the recovered residual signals, and thus recover the residual signals of the j-th column. The horizontal restoration process can be performed sequentially for all columns of the block.

[90] When the vertical differential coding method is applied, the video decoder recovers the residual signals as shown in Equation 4. The residual signals in the residual block recovered by the video decoder from the bitstream are modified according to the vertical differential coding method. The target residual signal to be modified in the reconstructed residual block is modified such that the upper residual signals located in the same column above the target residual signal are added to the target residual signal.

[91]

[92] The video decoding apparatus may sequentially add the recovered residual signals and thus recover the i-th row residual signals. The vertical recovery process may be performed sequentially for all rows of the block.

[93] The above description illustrates differential coding in the horizontal and vertical directions, although the differential coding direction is not necessarily limited to only the horizontal and vertical directions. The differential encoding direction may include horizontal and vertical directions, and may further include other directions such as a 45 degree diagonal direction and a 135 degree diagonal direction. Thus, the video decoding apparatus can add to the target residual signal for modification in the residual block reconstructed from the bitstream, one or more residual signals in the residual block, which are located in the differential encoding direction from the target residual signal, and thereby modify the target residual signal. .

[94] In the case of implicit differential coding, the video decoding apparatus may derive prediction direction information for differential coding from the intra prediction information decoded in advance. For example, when the current block uses intra prediction and the intra prediction mode is directional mode #18, horizontal differential coding may be applied to the residual signals. When the current block uses intra prediction and the intra prediction mode is directional mode #50, vertical differential coding may be applied to the residual signals.

[95] In another embodiment of the present disclosure, when the current block uses intra-prediction and the difference between the intra-prediction mode and the directional mode #18 is less than a preset threshold range as shown in Equation 5, a horizontal differential may be applied to the residual signals. coding.

[96]

указывает абсолютное значение.[97] Here predModeIntra is the intra-prediction mode of the current block, and

specifies an absolute value.

[98] In addition, when the current block uses intra prediction and the difference between the intra prediction mode and the directional mode #50 is smaller than the preset threshold range as shown in Equation 6, vertical differential coding can be applied to the residual signals.

[99]

[100] Meanwhile, in the case of explicit differential coding, the video decoding apparatus can determine differential coding prediction mode information by decoding information from the bitstream.

[101] When the transform skip mode, that is, transform_skip_flag, is true, the video decoding apparatus receives differential encoding information diff_cod_idx as shown in Table 2 and determines whether to apply the differential encoding method to the current block.

[102]

[103] In Table 2, DIFF_COD_OFF is a mode in which no additional differential coding is performed after skipping a transform, and DIFF_COD_VER and DIFF_COD_HOR are prediction modes for prediction in the vertical or horizontal direction. The prediction mode indices in the vertical or horizontal direction may be interchangeable. As shown in Table 2, when diff_cod_idx is not equal to 0, the syntax is binarized to 2 bits. The first bit can be expressed as 1 and the second bit can be expressed as 0 or 1 depending on the direction. The video decoding apparatus recognizes from the first bit that differential coding is applied and identifies the direction of the differential coding from the second bit. On the other hand, when diff_cod_idx is 0 (DIFF_COD_OFF), the syntax is binarized to 0. In this case, no second bit is required to indicate the direction of the differential encoding. In the above example, the use or non-use of differential coding and the direction of differential coding are expressed as the same syntax. However, from the binarization results shown in Table 2, it is apparent that the use or non-use of differential coding and the direction of differential coding can be encoded as separate syntaxes. For example, two syntaxes can be used with a 1-bit flag indicating whether or not to use differential coding and a 1-bit flag indicating the direction of the differential coding.

[104] In another embodiment of the present disclosure, transform_skip_flag transform skip mode information and diff_cod_idx differential encoding information may be combined and expressed as shown in Table 3.

[105]

[106] Here, "transform_skip_idx=0" performs the same operation as "transform_skip_flag=1". Also, "transform_skip_idx=-1" works the same as "transform_skip_flag=0".

[107] In another embodiment of the present disclosure, transform_skip_flag transform skip mode information, diff_cod_idx differential encoding information, and tu_mts_idx MTS information may be combined and expressed as shown in Table 4.

[108]

[109] Explicit differential coding according to the present disclosure can be applied to both intra-prediction and inter-prediction modes.

[110] When explicit differential coding is applied to the intra-prediction mode, information about the intra-prediction mode may be omitted, and only the syntax associated with differential coding may be encoded. The intra-prediction mode of the current block is set to the differential encoding direction. As described above, when using 67 intra prediction modes, a large number of bits are required to convey information about the intra prediction mode applied to the current block. The use of explicit differential coding in intra prediction can reduce the number of bits required to encode information about the intra prediction mode by determining the direction of the intra prediction mode from the direction of the differential coding. In other words, by deriving the intra-prediction mode from the syntax regarding the differential coding prediction direction, the number of transmitted codes is reduced.

[111] Alternatively, when applying differential coding, the intra-prediction mode of the current block may be limited to allow the use of intra-prediction modes solely within a threshold range from the direction of the differential coding. For example, when applying horizontal differential coding, only the intra prediction modes satisfying Equation 5 are valid as the intra prediction modes of the current block among all the intra prediction modes. When applying vertical differential coding, only the intra prediction modes satisfying Equation 6 are valid as the intra prediction modes of the current block among all the intra prediction modes. Accordingly, since the selection of intra prediction modes of the current block is made from among some intra prediction modes similar to the direction of differential coding, and not all intra prediction modes, the number of bits required to encode the intra prediction mode of the current block can be reduced. The video decoding device may generate a candidate set composed of intra prediction modes within a threshold range from the differential encoding direction, and may set, as the intra prediction mode of the current block, a candidate which is indicated by the intra prediction mode information received from the device. video encoding, among the set of candidates.

[112] The video encoding device and the video decoding device may not perform deblocking filtering at the boundary between the current block and the adjacent block when the differential encoding method is applied to both the residual block of the current block and the residual block of the adjacent block.

[113] FIG. 6 is a flowchart of a video decoding method for decoding an intra prediction encoded block using an explicit differential coding technique according to at least one embodiment of the present disclosure.

[114] The video decoding apparatus decodes from the bitstream information about the differential coding applied to the residual block of the current block (S610).

[115] The differential encoding information may include a flag indicating whether differential encoding is applied and a flag for indicating a direction of differential encoding. The video decoding apparatus may decode differential coding information applied to the residual block of the current block. The differential coding information may be passed to the video decoder as a diff_cod_idx syntax element as shown in Table 2.

[116] The video decoding apparatus recovers the residual signal information from the bitstream and generates a residual block using the residual signal information and the differential encoding information (S620).

[117] The video decoding apparatus first recovers the residual signal information.

[118] For example, the video decoder recovers a flag (tu_cbf_luma) indicating whether there is a residual signal relative to the current block. When a residual signal is present (tu_cbf_luma is true), the residual signals for the current block are recovered from the bitstream. On the other hand, when tu_cbf_luma is false, all residual signals are set to 0.

[119] Meanwhile, when there is a residual signal (tu_cbf_luma is true), the transform_skip_flag indicating whether a transformation is skipped can be decoded. Here, when transform_skip_flag has not been passed, it is deduced that transform_skip_flag has the same value as the flag indicating whether or not to apply differential encoding. In other words, as described above, since the differential encoding method is applied in the transform-skip mode, the transform_skip_flag is not included in the bitstream. When differential encoding is applied, the transform_skip_flag parameter can be automatically set to "true" ("1") so that the transformation can be skipped.

[120] When an explicit differential coding technique is applied, the video decoding apparatus may use the decoded direction information to generate a residual block from the residual signals recovered from the bitstream. Equation 3 is used in case of horizontal differential coding and Equation 4 is used in case of vertical differential coding.

[121] The video decoding apparatus performs inverse quantization of the residual block (S630).

[122] The video decoding apparatus generates a prediction block of the current block by intra-prediction (S640). The video decoding apparatus may generate a prediction block of the current block using sub-steps S642 and S644.

[123] The video decoding apparatus sets the intra prediction mode of the current block using the differential encoding information (S642).

[124] When a differential encoding technique is applied, the video decoding apparatus may set the intra prediction mode using direction information from the decoded differential encoding information. Specifically, in the case of horizontal differential coding, the intra prediction mode is set to the horizontal intra prediction mode (for example, INTRA_ANGULAR18 in FIGS. 3A and 3B), and in the case of vertical differential coding, the intra prediction mode is set to the vertical intra prediction mode (for example, , INTRA_ANGULAR50 in Figures 3A and 3B).

[125] The video decoding apparatus predicts the current block using previously reconstructed pixels around the current block in accordance with the intra prediction mode (S644).

[126] The video decoding apparatus recovers the current block by adding a prediction block to the inversely quantized residual block (S650).

[127] Since the differential coding method is applied in the transform skipping mode, the video decoding apparatus skips the inverse transform of the inversely quantized residual block. Accordingly, the video decoding apparatus can reconstruct the current block by adding an inversely quantized residual block and an intra-prediction block of the current block. In the case of intra-prediction of the next block to be decoded, the pixels in the reconstructed current block may be used as reference pixels.

[128] In order to remove the blocking artifact caused by block decoding, the video decoding apparatus performs deblocking filtering on a boundary between an adjacent block adjacent to the reconstructed current block and the reconstructed current block. However, when both the residual block of an adjacent block and the residual block of the current block have been differentially encoded, the video decoding apparatus may skip deblocking filtering at a boundary between the adjacent block and the current block.

[129] FIG. 7 is a flowchart of a video decoding method for decoding an inter prediction encoded block using an explicit differential coding technique according to at least one embodiment of the present disclosure.

[130] The video decoding apparatus decodes, from the bitstream, differential coding information applied to the residual block of the current block (S710).

[131] The differential encoding information may include a flag for indicating the use or non-use of the differential encoding and a flag for indicating the direction of the differential encoding. The video decoding apparatus may decode differential coding information applied to the residual block of the current block. The differential coding information may be passed to the video decoder as a diff_cod_idx syntax element as shown in Table 2.

[132] The video decoding apparatus recovers the residual signal information from the bitstream and generates a residual block using the residual signal information and the differential encoding information (S720).

[133] The video decoding apparatus generates residual signals by first recovering the transmitted residual signal information. Since the specific method is the same as that described with reference to FIG. 6, its further description will be omitted.

[134] When an explicit differential coding technique is applied, the video decoding apparatus may use the decoded direction information to generate a residual block from the residual signals recovered from the bitstream. Equation 3 is used in case of horizontal differential coding and Equation 4 is used in case of vertical differential coding.

[135] The video decoding apparatus performs inverse quantization of the residual block (S730).

[136] The video decoding apparatus generates a prediction block of the current block by inter-prediction (S740). The video decoding apparatus may generate a prediction block of the current block using sub-steps S742 and S744.

[137] The video decoding apparatus uses the extracted inter-prediction mode syntax element to determine the motion vector of the current block and the reference picture referenced by the motion vector (S742).

[138] The video decoding apparatus predicts the current block using the motion vector and the reference image (S744).

[139] The video decoding apparatus recovers the current block by adding a prediction block to the inversely quantized residual block (S750).

[140] Since the differential coding method is applied in the transform skipping mode, the video decoding apparatus skips the inverse transform of the inversely quantized residual block. Accordingly, the video decoding apparatus can reconstruct the current block by adding an inversely quantized residual block and an inter-prediction block of the current block.

[141] To remove the blocking artifact caused by block decoding, the video decoding apparatus may perform deblocking filtering on a boundary between an adjacent block adjacent to the reconstructed current block and the reconstructed current block. However, when both the residual block of an adjacent block and the residual block of the current block have been differentially encoded, the video decoding apparatus may skip performing deblocking filtering on a boundary between the adjacent block and the current block. When all blocks in one picture are reconstructed, the reconstructed picture can be used as a reference picture for inter-prediction of blocks in the next picture to be coded.

[142] FIG. 8 is a flowchart of a video decoding method for decoding an intra prediction encoded block using an implicit differential coding technique according to another embodiment of the present disclosure.

[143] The video decoding apparatus decodes the intra prediction information applied to the current block from the bitstream, and uses the decoded intra prediction information to generate a prediction block of the current block (S810). The video decoding apparatus may generate a prediction block of the current block using sub-steps S842 and S844.

[144] The video decoding apparatus uses the decoded intra prediction information to determine the intra prediction mode of the current block (S812).

[145] The video decoding apparatus may determine, using the extracted intra prediction mode syntax element, the intra prediction mode of the current block from the plurality of intra prediction modes.

[146] The video decoding apparatus predicts the current block using previously reconstructed pixels around the current block in accordance with the intra prediction mode (S814).

[147] The video decoding apparatus uses the intra prediction mode to output information about the direction of the differential encoding applied to the residual block of the current block (S820).

[148] When an implicit differential encoding technique is applied, the required information about the differential encoding is the direction of the differential encoding. In the transform skip mode (transform_skip_flag is set to true), the video decoding apparatus can estimate differential encoding prediction direction information from the intra prediction information. For example, when the intra prediction mode is directional mode #18, horizontal differential coding may be applied to the residual block. In addition, when the intra prediction mode is the directional mode #50, vertical differential coding may be applied to the residual block.

[149] In another embodiment of the present disclosure, when the difference between the intra-prediction mode and the directional mode #18 is less than a preset threshold range as shown in Equation 5, horizontal differential coding may be applied to the residual block. In addition, when the difference between the intra prediction mode and the directional mode #50 is smaller than the preset threshold range as shown in Equation 6, vertical differential coding may be applied to the residual block.

[150] The video decoding apparatus recovers the residual signal information from the bitstream and generates a residual block using the residual signal information and the differential encoding information (S830).

[151] When an implicit differential coding technique is applied, the video decoding apparatus may use the decoded information estimated from the intra-prediction mode to generate a residual block from the residual signals recovered from the bitstream. Equation 3 is used in case of horizontal differential coding and Equation 4 is used in case of vertical differential coding.

[152] The video decoding apparatus performs inverse quantization of the residual block (S840).

[153] The video decoding apparatus reconstructs the current block by adding a prediction block and an inversely quantized residual block (S850).

[154] Since the differential coding method is applied in the transform skipping mode, the video decoding apparatus skips the inverse transform of the inversely quantized residual block. Accordingly, the video decoding apparatus can reconstruct the current block by adding an inversely quantized residual block and an intra-prediction block of the current block. For future intra-prediction of the next block to be decoded, the pixels in the reconstructed current block may be used as reference pixels.

[155] To remove the blocking artifact caused by block decoding, the video decoding apparatus may perform deblocking filtering on a boundary between an adjacent block adjacent to the reconstructed current block and the reconstructed current block. However, when both the residual block of the adjacent block and the residual block of the current block have been differentially encoded, the video decoding apparatus is released from performing deblocking filtering at the boundary between the adjacent block and the current block.

[156] Although the steps in the respective flowcharts are described as being performed sequentially, they only embody the technical idea of some embodiments of the present disclosure. Therefore, one skilled in the relevant art can perform the steps by changing the sequences described in the respective drawings or by performing two or more steps in parallel, and therefore the steps in the respective flowcharts are not limited to the illustrated chronological sequences.

[157] Meanwhile, various methods or functions described herein may be implemented as instructions stored on a non-volatile storage medium that can be read and executed by one or more processors. A non-transitory recording medium includes, for example, all types of recording devices in which data is stored in a form readable by a computer system. For example, a non-transitory storage medium may include, but is not limited to, storage media such as an erasable programmable read-only memory (EPROM), a flash drive, an optical drive, a magnetic hard disk, and a solid state drive (SSD).

[158] While the exemplary embodiments of the present invention have been described for illustrative purposes, those skilled in the art will appreciate that various modifications, additions, and substitutions are possible without departing from the spirit and scope of the claimed invention. Therefore, for brevity and clarity, exemplary embodiments of the present invention have been described. The scope of the technical idea of the present embodiments is not limited to the illustrations. Accordingly, one of ordinary skill in the art will appreciate that the scope of the claimed invention is not limited by the embodiments that have been directly described above, but by the claims and their equivalents.

Cross-reference to related application

This application claims priority under 35 USC §119(a) of Patent Application No. 10-2019-0028995 filed March 13, 2019 in Korea, the entire contents of which are incorporated herein by reference. In addition, this non-provisional application claims priority in countries other than the United States for the same reason as the Korean patent application, the entire content of which is hereby incorporated by reference.

Claims (27)

1. A video decoding method for recovering a current block that is encoded using intra-prediction, comprising: decoding from the bitstream direction information indicating the direction of the differential encoding applied to the residual block of the current block; generating a residual block by recovering residual signal information from the bitstream; generating a modified residual block by modifying residual signals in the residual block based on the direction information, wherein each target residual signal for modification in the residual block is modified by adding to one or more residual signals in the residual block that are located in the direction indicated by the direction information. direction from the target residual signal; inverse quantization of the modified residual block into an inversely quantized residual block; deriving an intra-prediction mode of the current block using direction information without decoding the intra-prediction mode information of the current block from the bitstream, and generating a prediction block for the current block by intra-prediction of the current block based on the derived intra-prediction mode; and restoring the current block by adding a prediction block and an inversely quantized residual block. 2. The video decoding method according to claim 1, further comprising: decoding from the bitstream a flag indicating whether differential coding has been applied to the residual block of the current block, wherein the direction information is decoded from the bitstream when the flag indicates that the differential encoding has been applied to the residual block of the current block. 3. The video decoding method according to claim 1, wherein the direction information indicates any one of a horizontal direction and a vertical direction. 4. The video decoding method of claim 3, wherein when the direction information indicates a horizontal direction, the target residual signal is modified by summing with left residual signals located to the left of the target residual signal in the same row as the target residual signal. 5. The video decoding method of claim 3, wherein when the direction information indicates a vertical direction, the target residual signal is modified by summing with upper residual signals located above the target residual signal in the same column as the target residual signal. 6. The video decoding method of claim 1, wherein the intra prediction mode of the current block is set to have a prediction direction that matches the direction indicated by the direction information. 7. The video decoding method of claim 1, wherein the inverse transform for the inversely quantized residual block is omitted. 8. The video decoding method according to claim. 1, in which the information about the residual signals contains: information indicating whether residual signals exist; and information indicating whether the conversion skip mode is used. 9. The video decoding method according to claim 1, further comprising: filtering deblocking the boundary between the reconstructed current block and a neighboring block adjacent to the reconstructed current block, however, deblocking filtering is not performed on the boundary between the reconstructed current block and the neighboring block when differential encoding is applied to both the residual block of the neighboring block and the residual block of the current block. 10. A non-temporal computer-readable storage medium storing a bit stream including encoded data of the current block, which is encoded using intra-prediction, the bit stream being decoded by a method comprising: decoding, from the bitstream, direction information indicating a direction of differential encoding applied to the residual block of the current block, and generating a residual block by recovering the residual signal information from the bitstream; generating a modified residual block by modifying the residual signals in the residual block based on the direction information, and inverse quantizing the modified residual block into an inversely quantized residual block; deriving an intra-prediction mode of the current block using direction information without decoding the intra-prediction mode information of the current block from the bitstream, and generating a prediction block for the current block by intra-prediction of the current block based on the derived intra-prediction mode; and adding a prediction block and an inversely quantized residual block to reconstruct the current block, wherein each target residual signal for modification in the residual block is modified by adding to one or more residual signals in the residual block that are located in the direction indicated by the direction information from the target residual signal.

Images