WO2020184913A1

WO2020184913A1 - Method and apparatus for processing video signal

Info

Publication number: WO2020184913A1
Application number: PCT/KR2020/003187
Authority: WO
Inventors: 최장원; 허진; 유선미; 최정아; 김승환; 이령
Original assignee: 엘지전자 주식회사
Priority date: 2019-03-08
Filing date: 2020-03-06
Publication date: 2020-09-17

Abstract

Embodiments of the present specification provide a method and an apparatus for processing a video signal. A decoding method for a video signal according to an embodiment of the present specification may comprise the steps of: for a current block including data corresponding to a chroma component, identifying conditions for a conversion skip on the basis of a dual tree type of the current block and the size of the current block; when the conditions for the conversion skip are satisfied, acquiring a conversion skip flag of the current block; performing residual coding with respect to the current block on the basis the conversion skip flag; and generating a residual sample of the current block through the residual coding. Efficient encoding/decoding can be performed with respect to a chroma block by identifying conditions for a conversion skip for the chroma block and applying the conversion skip to the chroma block.

Description

Method and apparatus for processing video signals

Embodiments of the present specification relate to a video/video compression coding system, and more particularly, to a method and apparatus for processing residual data in an encoding/decoding process of a video signal.

Compression coding refers to a series of signal processing techniques for transmitting digitized information through a communication line or storing it in a format suitable for a storage medium. Media such as video, image, and audio may be subject to compression encoding. In particular, a technique for performing compression encoding on an image is referred to as video image compression.

Next-generation video content will be characterized by high spatial resolution, high frame rate, and high dimensionality of scene representation. In order to process such content, it will bring a tremendous increase in terms of memory storage, memory access rate, and processing power.

In a hybrid coding system, a prediction signal is generated through prediction of a video signal, and a transform is applied to a residual signal generated by subtracting the prediction signal from an original video signal. In this case, transformation skip may be applied, and various techniques for efficiently performing transformation or transformation skip are being discussed.

An embodiment of the present specification provides a method and apparatus for efficiently performing transformation/transformation skipping and residual data coding for a color difference block including color difference component data.

The technical problems to be achieved in the embodiments of the present specification are not limited to the technical problems mentioned above, and other technical problems that are not mentioned are those of ordinary skill in the technical field to which the embodiments of the present specification belong from the following description. Will be clearly understood.

Embodiments of the present specification provide a method and apparatus for processing a video signal. The encoding method of a video signal according to an embodiment of the present specification includes, for a current block including data corresponding to a chroma component, based on a dual tree type of the current block and a size of the current block. Checking a condition for transform skipping, and if the condition for transform skipping is satisfied, determining whether to apply the transform skip to the current block, and encoding a transform skip flag for the current block Includes steps.

In one embodiment, the checking of the condition for the transform skip includes the transform skip when the size of the current block is smaller than the transform skip maximum size and the tree type of the current block corresponds to a color difference dual tree or a single tree. It may include determining that the condition for is satisfied.

In an embodiment, the transform skip maximum size is determined based on transform skip maximum size information, and the transform skip maximum size information may be transmitted to a decoder as a syntax element for an upper layer component including the current block.

In an embodiment, the maximum transform skip size may be determined based on a type of a tile group to which the current block belongs.

In an embodiment, if the tile group corresponds to an I-tile group, the transform skip maximum size is determined as a first reference size, and if the tile group corresponds to a B-tile group or a P-tile group, the transform skip maximum The size is determined as a second reference size, and the first reference size may correspond to a value larger than the second reference size.

In one embodiment, the transform skip maximum size is determined based on a type of a tile group to which the current block belongs and a prediction type of the current block, and the prediction type is one of an intra prediction type or an inter prediction type. It may include.

The decoding method of a video signal according to an embodiment of the present specification is for skipping a transform based on a tree type of the current block and a size of the current block for a current block including data corresponding to a chroma component. Checking a condition; if the condition for the transform skip is satisfied, obtaining a transform skip flag of the current block; and performing residual coding for the current block based on the transform skip flag; and And generating a residual sample of the current block through the residual coding.

In one embodiment, the checking of the condition for the transform skip includes the transform skip when the size of the current block is smaller than the transform skip maximum size and the tree type of the current block corresponds to a color difference dual tree or a single tree. It can be determined that the condition for is satisfied.

In one embodiment, the transform skip maximum size is determined based on transform skip maximum size information transmitted from an encoder, and the transform skip maximum size information may be transmitted as a syntax element for an upper layer component including the current block. have.

An embodiment of the present specification provides an apparatus for processing a video signal. A video signal encoding apparatus according to an embodiment of the present specification includes a memory for storing the video signal, and a processor coupled to the memory and for processing the video signal. The processor, for a current block including data corresponding to a chroma component, checks a condition for skipping transformation based on a tree type of the current block and a size of the current block, and the transformation When a condition for skipping is satisfied, it is determined whether to apply the transform skip to the current block, and set to encode a transform skip flag for the current block.

A video signal decoding apparatus according to an exemplary embodiment of the present specification includes a memory for storing the video signal, and a processor coupled to the memory and for processing the video signal. The processor, for a current block including data corresponding to a chroma component, checks a condition for skipping transformation based on a tree type of the current block and a size of the current block, and the transformation When the condition for skipping is satisfied, a transform skip flag of the current block is obtained, residual coding of the current block is performed based on the transform skip flag, and residual coding of the current block is performed through the residual coding. It is set up to generate samples.

An embodiment of the present specification provides a non-transitory computer-readable medium storing one or more instructions. The one or more instructions executed by one or more processors are, for a current block including data corresponding to a chroma component, a tree type of the current block and a size of the current block. A video to determine whether to apply the transform skip to the current block, and to encode the transform skip flag for the current block if the condition for the transform skip is satisfied and the condition for the transform skip is satisfied. Control the signal processing device.

In addition, the one or more instructions executed by one or more processors are, for a current block including data corresponding to a chroma component, a tree type and a current block of the current block. A condition for transform skip is checked based on the size of, and if the condition for transform skip is satisfied, a transform skip flag of the current block is obtained, and residual coding for the current block is performed based on the transform skip flag. And controlling a video signal processing apparatus to generate a residual sample of the current block through the residual coding.

Through the method of determining whether to apply the transform skip to the color difference block or the method of applying the transform skip to the color difference block provided in the embodiment of the present specification, the encoder or decoder can efficiently perform transform or transform skip on the color difference block. I can.

The effects obtained in the embodiments of the present specification are not limited to the above-mentioned effects, and other effects not mentioned are clearly to those of ordinary skill in the art to which the embodiments of the present specification belong from the following description. It will be understandable.

The accompanying drawings, which are included as part of the detailed description to aid in understanding of the present specification, provide embodiments of the present specification, and describe technical features of the present specification together with the detailed description.

1 shows an example of an image coding system according to an embodiment of the present specification.

2 shows an example of a schematic block diagram of an encoding apparatus in which encoding of a video signal is performed according to an embodiment of the present specification.

3 shows an example of a schematic block diagram of a decoding apparatus for decoding an image signal according to an embodiment of the present specification.

4 shows an example of a content streaming system according to an embodiment of the present specification.

5 shows an example of a video signal processing apparatus according to an embodiment of the present specification.

6 illustrates an example of a picture division structure according to an embodiment of the present specification.

7 illustrates an example of a picture divided into a plurality of coding tree units according to an embodiment of the present specification.

8 shows an example of a multi-type tree structure according to an embodiment of the present specification.

9 shows an example of a mechanism for signaling split information of a quadtree with nested multi-type tree structure according to an embodiment of the present specification.

10 illustrates an example of a coding tree unit divided into a plurality of coding units based on a quadtree and a nested multi-type tree structure according to an embodiment of the present specification.

11 illustrates an example of a case in which division of a ternary tree is limited according to an embodiment of the present specification.

12 illustrates an example in which binary tree division and ternary tree division overlap according to an embodiment of the present specification.

13 illustrates an example of a case in which the ternary tree division and the byte tree division are restricted according to an embodiment of the present specification.

14 is an example of a flowchart for encoding a picture constituting a video signal according to an embodiment of the present specification.

15 is an example of a flowchart for decoding a picture constituting a video signal according to an embodiment of the present specification.

16 illustrates an example of a hierarchical structure for a coded image according to an embodiment of the present specification.

17 is a schematic block diagram of a transform and quantization unit, an inverse quantization and an inverse transform unit in an encoding apparatus according to an embodiment of the present specification.

18 is a schematic block diagram of an inverse quantization and inverse transform unit in a decoding apparatus.

19 illustrates an example of a transformation procedure to which multiple transform selection is applied according to an embodiment of the present specification.

20 illustrates an example of an inverse transform procedure to which multiple transform selection is applied according to an embodiment of the present specification.

21 illustrates an example of an inverse transform procedure using information related to multiple transform selection according to an embodiment of the present specification.

22 is a block diagram for entropy encoding according to an embodiment of the present specification.

23A and 23B illustrate an entropy encoding method and related components according to an embodiment of the present specification.

24A and 24B illustrate an entropy decoding method and related components according to an embodiment of the present specification.

25 illustrates an example of intra-block transform coefficients according to an embodiment of the present specification.

26 illustrates an example of a color difference block and a corresponding luminance block in a dual tree according to an embodiment of the present specification.

27 is an example of a flowchart for encoding a video signal according to an embodiment of the present specification.

28 is an example of a flowchart for decoding a video signal according to an embodiment of the present specification.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. DETAILED DESCRIPTION OF THE INVENTION The detailed description to be disclosed below together with the accompanying drawings is intended to describe exemplary embodiments of the present invention and is not intended to represent the only embodiments in which the present invention may be practiced. The following detailed description includes specific details to provide a thorough understanding of the present invention. However, one of ordinary skill in the art appreciates that the invention may be practiced without these specific details.

In some cases, in order to avoid obscuring the concept of the present invention, well-known structures and devices may be omitted, or may be shown in a block diagram form centering on core functions of each structure and device.

In addition, as for terms used in the present invention, general terms that are currently widely used are selected as far as possible, but specific cases will be described using terms arbitrarily selected by the applicant. In such a case, since the meaning of the term is clearly described in the detailed description of the corresponding part, it should not be interpreted simply by the name of the term used in the description of the present invention, and it should be clarified that the meaning of the term should be understood and interpreted. .

Specific terms used in the following description are provided to aid understanding of the present invention, and the use of these specific terms may be changed in other forms without departing from the technical spirit of the present invention. For example, signals, data, samples, pictures, frames, blocks, etc. may be appropriately replaced and interpreted in each coding process.

Hereinafter, in this specification, a'processing unit' means a unit in which an encoding/decoding process such as prediction, transformation, and/or quantization is performed. Also, the processing unit may be interpreted as including a unit for a luma component and a unit for a chroma component. For example, the processing unit may correspond to a block, a coding unit (CU), a prediction unit (PU), or a transform unit (TU).

Further, the processing unit may be interpreted as a unit for a luminance component or a unit for a color difference component. For example, the processing unit may correspond to a coding tree block (CTB), a coding block (CB), a PU, or a transform block (TB) for a luminance component. Alternatively, the processing unit may correspond to CTB, CB, PU or TB for the color difference component. Further, the present invention is not limited thereto, and the processing unit may be interpreted as including a unit for a luminance component and a unit for a color difference component.

Further, the processing unit is not necessarily limited to a square block, and may be configured in a polygonal shape having three or more vertices.

In addition, in the present specification, pixels or pixels are collectively referred to as samples. In addition, using a sample may mean using a pixel value or a pixel value.

1 shows an example of an image coding system according to an embodiment of the present specification. The image coding system may include a source device 10 and a reception device 20. The source device 10 may transmit the encoded video/video information or data in a file or streaming format to the receiving device 20 through a digital storage medium or a network.

The source device 10 may include a video source 11, an encoding device 12, and a transmitter 13. The receiving device 20 may include a receiver 21, a decoding device 22 and a renderer 23. The encoding device 12 may be referred to as a video/image encoding device, and the decoding device 22 may be referred to as a video/image decoding device. The transmitter 13 may be included in the encoding device 12. The receiver 21 may be included in the decoding device 22. The renderer 23 may include a display unit, and the display unit may be configured as a separate device or an external component.

The video source 11 may acquire a video/image through a process of capturing, synthesizing, or generating a video/image. The video source 11 may include a video/image capturing device and/or a video/image generating device. The video/image capturing device may include, for example, one or more cameras, and a video/image archive including previously captured video/images. Video/image generating devices may include, for example, computers, tablets, and smartphones, and may (electronically) generate video/images. For example, a virtual video/image may be generated through a computer, and in this case, a video/image capturing process may be substituted as a process of generating related data.

The encoding device 12 may encode an input video/video. The encoding apparatus 12 may perform a series of procedures such as prediction, transformation, and quantization for compression and coding efficiency. The encoded data (encoded video/video information) may be output in the form of a bitstream.

The transmitter 13 may transmit the encoded video/video information or data output in the form of a bitstream to the receiver 21 of the receiving device 20 through a digital storage medium or a network in a file or streaming form. Digital storage media include USB (universal serial bus), SD card (secure digital card), CD (compact disc), DVD (digital versatile disc), Blu-ray disc, HDD (hard disk drive), SSD (solid state drive) may include a variety of storage media. The transmitter 13 may include an element for generating a media file through a predetermined file format, and may include an element for transmission through a broadcast/communication network. The receiver 21 may extract the bitstream and transmit it to the decoding device 22.

The decoding device 22 may decode the video/video by performing a series of procedures such as inverse quantization, inverse transformation, and prediction corresponding to the operation of the encoding device 12.

The renderer 23 may render the decoded video/image. The rendered video/image may be displayed through the display unit.

2 shows an example of a schematic block diagram of an encoding apparatus in which encoding of a video signal is performed according to an embodiment of the present specification. The encoding device 100 of FIG. 2 may correspond to the encoding device 12 of FIG. 1.

Referring to FIG. 2, the encoding apparatus 100 includes an image partitioning module 110, a subtraction module 115, a transform module 120, and a quantization module. (130), a de-quantization module (140), an inverse-transform module (150), an addition module (155), a filtering module (160), a memory A (memory) 170, an inter prediction module 180, an intra prediction module 185, and an entropy encoding module 190 may be included. The inter prediction unit 180 and the intra prediction unit 185 may be collectively referred to as a prediction unit. That is, the prediction unit may include an inter prediction unit 180 and an intra prediction unit 185. The transform unit 120, the quantization unit 130, the inverse quantization unit 140, and the inverse transform unit 150 may be included in a residual processing unit. The residual processing unit may further include a subtraction unit 115. The above-described image segmentation unit 110, subtraction unit 115, transform unit 120, quantization unit 130, inverse quantization unit 140, inverse transform unit 150, addition unit 155, filtering unit 160 ), the inter prediction unit 180, the intra prediction unit 185, and the entropy encoding unit 190 may be configured by one hardware component (eg, an encoder or a processor) according to an embodiment. In addition, the memory 170 may include a decoded picture buffer (DPB) 175 and may be configured by a digital storage medium.

The image segmentation unit 110 may divide an input image (or picture, frame) input to the encoding apparatus 100 into one or more processing units. For example, the processing unit may be referred to as a coding unit (CU). In this case, the coding unit may be recursively partitioned from a coding tree unit (CTU) or a largest coding unit (LCU) according to a quad-tree binary-tree (QTBT) structure. For example, one coding unit may be divided into a plurality of coding units of a deeper depth based on a quad tree structure and/or a binary tree structure. In this case, for example, a quad tree structure may be applied first and a binary tree structure may be applied later. Alternatively, the binary tree structure may be applied first. A coding procedure according to an embodiment of the present specification may be performed based on a final coding unit that is no longer divided. In this case, the maximum coding unit may be directly used as the final coding unit based on coding efficiency according to image characteristics. Further, if necessary, the coding unit is recursively divided into coding units of a lower depth, so that a coding unit having an optimal size may be used as a final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration described below. As another example, the processing unit may further include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit may be divided from the above-described coding units, respectively. The prediction unit may be a unit of sample prediction, and the transform unit may be a unit for inducing a transform coefficient or a unit for inducing a residual signal from the transform coefficient.

The term "unit" used in this document may be used interchangeably with terms such as "block" or "area" in some cases. In this document, the MxN block may represent a set of samples or transform coefficients consisting of M columns and N rows. A sample may generally represent a pixel or a value of a pixel, may represent a pixel/pixel value of a luminance component, or a pixel/pixel value of a saturation component. A sample may be used as a term corresponding to one picture (or image) as a pixel or pel.

The encoding apparatus 100 subtracts a prediction signal (predicted block, prediction sample array) output from the inter prediction unit 180 or the intra prediction unit 185 from the input video signal (original block, original sample array) A signal (residual signal, residual block, residual sample array) may be generated. The generated residual signal is transmitted to the conversion unit 120. In this case, as illustrated, a unit that subtracts the prediction signal (prediction block, prediction sample array) from the input image signal (original block, original sample array) in the encoding apparatus 100 may be referred to as a subtraction unit 115. . The prediction unit may perform prediction on a block to be processed (hereinafter, referred to as a current block) and generate a predicted block including prediction samples for the current block. The prediction module may determine whether intra prediction or inter prediction is applied on a per CU basis. The prediction unit may generate information about prediction, such as prediction mode information, as described later in the description of each prediction mode, and may transmit information about prediction to the entropy encoding unit 190. Information about prediction is encoded by the entropy encoding unit 190 and may be output in the form of a bitstream.

The intra prediction unit 185 may predict the current block by referring to samples in the current picture. The referenced samples may be located in the vicinity of the current block or may be located away from each other according to the prediction mode. In intra prediction, prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The non-directional mode may include, for example, a DC mode and a planar mode (Planar mode). The directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes according to a detailed degree of the prediction direction. However, this is an example, and more or less directional prediction modes may be used depending on the setting. The intra prediction unit 185 may determine a prediction mode applied to the current block by using the prediction mode applied to the neighboring block.

The inter prediction unit 180 may derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on the reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the inter prediction unit 180 may predict motion information in units of blocks, subblocks, or samples based on the correlation between motion information between neighboring blocks and the current block have. The motion information may include a motion vector and a reference picture index. The motion information may further include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter prediction, the neighboring block may include a spatial neighboring block existing in the current picture and a temporal neighboring block existing in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be referred to as a collocated reference block or a colCU (colCU), and a reference picture including the temporal neighboring block may be referred to as a collocated picture (colPic). For example, the inter prediction unit 180 constructs a motion information candidate list based on motion information of neighboring blocks, and indicates which candidate is used to derive a motion vector and/or a reference picture index of the current block. Can generate information. Inter prediction may be performed based on various prediction modes. For example, when a skip mode and a merge mode are used, the inter prediction unit 180 may use motion information of a neighboring block as motion information of a current block. In the case of the skip mode, unlike the merge mode, a residual signal is not transmitted. In the case of motion vector prediction (MVP) mode, a motion vector of a neighboring block is used as a motion vector predictor and a motion vector difference (MVD) is signaled to move the current block. Vector can be indicated.

The prediction unit may generate a prediction signal (prediction sample) based on various prediction methods described later. For example, the prediction unit may not only apply intra prediction or inter prediction to predict one block, but also apply intra prediction and inter prediction together (simultaneously). This may be referred to as CIIP (combined inter and intra prediction). Also, the prediction unit may perform intra block copy (IBC) to predict a block. IBC may be used for content (eg, game) video/video coding, such as, for example, screen content coding (SCC). Also, IBC may be referred to as CPR (current picture referencing). IBC basically performs prediction in the current picture, but can be performed similarly to inter prediction in that it derives a reference block in the current picture. That is, the IBC may use at least one of the inter prediction techniques described in this document.

The prediction signal generated by the prediction unit (including the inter prediction unit 180 and/or the intra prediction unit 185) may be used to generate a reconstructed signal or may be used to generate a residual signal. The transform unit 120 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transformation technique uses at least one of DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), KLT (Karhunen-Loeve Transform), GBT (Graph-Based Transform), or CNT (Conditionally Non-linear Transform). Can include. Here, GBT refers to transformation obtained from a graph representing relationship information between pixels. CNT refers to a transformation obtained based on the prediction signal and generating a prediction signal using all previously reconstructed pixels. In addition, the conversion process may be applied to a pixel block having the same size of a square, or may be applied to a block that is not a square or has a variable size.

The quantization unit 130 quantizes the transform coefficients and transmits the quantized transform coefficients to the entropy encoding unit 190. The entropy encoding unit 190 may encode a quantized signal (information on quantized transform coefficients) and output it as a bitstream. Information about the quantized transform coefficients may be referred to as residual information. The quantization unit 130 may rearrange the quantized transform coefficients in a block form into a one-dimensional vector form based on a coefficient scan order, and quantize the quantized transform coefficients based on the characteristics of the quantized transform coefficients in a one-dimensional vector form. It is also possible to generate information about transform coefficients. The entropy encoding unit 190 may perform various encoding techniques such as exponential Golomb, context-adaptive variable length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC). The entropy encoding unit 190 may encode information necessary for video/image restoration (eg, values of syntax elements) in addition to quantized transform coefficients together or separately. The encoded information (eg, video/video information) may be transmitted or stored in a bitstream format in units of network abstraction layer (NAL) units. The video/video information may further include information on various parameter sets, such as adaptation parameter set (APS), picture parameter set (PPS), sequence parameter set (SPS), or video parameter set (VPS). Signaled/transmitted information and/or syntax elements described later in this document may be encoded through the above-described encoding procedure and included in the bitstream. The bitstream may be transmitted over a network or may be stored in a digital storage medium. Here, the network may include a broadcasting network and/or a communication network, and the digital storage medium may include a storage medium such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. For the signal output from the entropy encoding unit 190, a transmission unit (not shown) for transmitting and/or a storage unit (not shown) for storing may be configured as internal/external elements of the encoding apparatus 100, or the transmission unit It may be a component of the entropy encoding unit 190.

The quantized transform coefficients output from the quantization unit 130 may be used to generate a reconstructed signal. For example, a residual signal may be restored by applying inverse quantization and inverse transform through the inverse quantization unit 140 and the inverse transform unit 150 in the loop for the quantized transform coefficients. The addition unit 155 adds the reconstructed residual signal to the prediction signal output from the inter prediction unit 180 or the intra prediction unit 185 to obtain a reconstructed signal (a reconstructed picture, a reconstructed block, and a reconstructed sample array). Can be generated. When there is no residual signal for the block to be processed, such as when the skip mode is applied, the predicted block may be used as a reconstructed block. The addition unit 155 may be referred to as a restoration unit or a restoration block generation unit. The generated reconstructed signal may be used for intra prediction of the next processing target block in the current picture, and may be used for inter prediction of the next picture through filtering as described later.

The filtering unit 160 may improve subjective/objective image quality by applying filtering to the reconstructed signal. For example, the filtering unit 160 may apply various filtering methods to the reconstructed picture to generate a modified reconstructed picture, and may transmit the modified reconstructed picture to the DPB 175 of the memory 170. . Various filtering methods may include, for example, deblocking filtering, sample adaptive offset (SAO), adaptive loop filter (ALF), and bilateral filter. The filtering unit 160 may generate filtering information and transmit the filtering information to the entropy encoding unit 190 as described later in the description of each filtering method. The filtering information may be output in the form of a bitstream through entropy encoding in the entropy encoding unit 190.

The modified reconstructed picture transmitted to the DPB 175 may be used as a reference picture in the inter prediction unit 180. When inter prediction is applied, the encoding apparatus 100 may avoid prediction mismatch between the encoding apparatus 100 and the decoding apparatus 200 by using the modified reconstructed picture, and may improve encoding efficiency. The DPB 175 may store the modified reconstructed picture for use as a reference picture in the inter prediction unit 180. The stored motion information may be transmitted to the inter prediction unit 180 for use as motion information of a spatial neighboring block or motion information of a temporal neighboring block. The memory 170 may store reconstructed samples of reconstructed blocks in the current picture, and transfer information on the reconstructed samples to the intra prediction unit 185.

3 shows an example of a schematic block diagram of a decoding apparatus for decoding an image signal according to an embodiment of the present specification. The decoding device 200 of FIG. 3 may correspond to the decoding device 22 of FIG. 1.

Referring to FIG. 3, the decoding apparatus 200 includes an entropy decoding module 210, a de-quantization module 220, an inverse transform module 230, and an adder. (addition module) 235, filtering module 240, memory 250, inter prediction module 260, and intra prediction module 265 may be included. have. The inter prediction unit 260 and the intra prediction unit 265 may be collectively referred to as a prediction module. That is, the prediction unit may include an inter prediction unit 180 and an intra prediction unit 185. The inverse quantization unit 220 and the inverse transform unit 230 may be collectively referred to as a residual processing module. That is, the residual processing unit may include an inverse quantization unit 220 and an inverse transform unit 230. The entropy decoding unit 210, the inverse quantization unit 220, the inverse transform unit 230, the addition unit 235, the filtering unit 240, the inter prediction unit 260, and the intra prediction unit 265 are implemented. It may be configured by one hardware component (eg, a decoder or a processor) according to an example. Also, the memory 250 may include the DPB 255, and may be configured by one hardware component (eg, a memory or a digital storage medium) according to an embodiment.

When a bitstream including video/image information is input, the decoding apparatus 200 may reconstruct an image in response to a process in which the video/image information is processed by the encoding apparatus 100 of FIG. 2. For example, the decoding apparatus 200 may perform decoding using a processing unit applied by the encoding apparatus 100. Thus, upon decoding, the processing unit may be, for example, a coding unit, and the coding unit may be divided from a coding tree unit or a maximum coding unit according to a quad tree structure and/or a binary tree structure. In addition, the reconstructed image signal decoded and output through the decoding device 200 may be reproduced through the playback device.

The decoding apparatus 200 may receive a signal output from the encoding apparatus 100 of FIG. 2 in the form of a bitstream, and the received signal may be decoded through the entropy decoding unit 210. For example, the entropy decoding unit 210 may parse the bitstream to derive information (eg, video/video information) necessary for image restoration (or picture restoration). The video/video information may further include information on various parameter sets, such as adaptation parameter set (APS), picture parameter set (PPS), sequence parameter set (SPS), or video parameter set (VPS). The decoding apparatus may decode a picture based on information on a parameter set. Signaled/received information and/or syntax elements described later in this document may be decoded through a decoding procedure and obtained from a bitstream. For example, the entropy decoding unit 210 acquires information in the bitstream using a coding technique such as exponential Golomb coding, CAVLC, or CABAC, and a value of a syntax element required for image restoration, and a quantized value of a transform coefficient for a residual. Can be printed. In more detail, in the CABAC entropy decoding method, a bin corresponding to each syntax element is received in a bitstream, and information about the syntax element to be decoded and decoding information of a block to be decoded and a neighbor or a symbol/bin decoded in a previous step The symbol corresponding to the value of each syntax element is determined by determining the context model using the information of, and performing arithmetic decoding of the bin by predicting the probability of occurrence of the bin according to the determined context model. Can be generated. In this case, the CABAC entropy decoding method may update the context model using information of the decoded symbol/bin for the context model of the next symbol/bin after the context model is determined. Among the information decoded by the entropy decoding unit 210, information on prediction is provided to the prediction unit (inter prediction unit 260 and intra prediction unit 265), and the register on which entropy decoding is performed by the entropy decoding unit 210 Dual values, that is, quantized transform coefficients and related parameter information may be input to the inverse quantization unit 220. In addition, information about filtering among information decoded by the entropy decoding unit 210 may be provided to the filtering unit 240. Meanwhile, a receiving unit (not shown) for receiving a signal output from the encoding device 100 may be further configured as an inner/outer element of the decoding device 200, or the receiving unit may be a component of the entropy decoding unit 210. May be. Meanwhile, the decoding apparatus 200 according to the present specification may be referred to as a video/video/picture decoding apparatus. The decoding apparatus 200 may be divided into an information decoder (video/video/picture information decoder) and a sample decoder (video/video/picture sample decoder). The information decoder may include an entropy decoding unit 210, and the sample decoder is an inverse quantization unit 220, an inverse transform unit 230, an addition unit 235, a filtering unit 240, a memory 250, and an inter prediction. It may include at least one of the unit 260 and the intra prediction unit 265.

The inverse quantization unit 220 may output transform coefficients through inverse quantization of the quantized transform coefficients. The inverse quantization unit 220 may rearrange the quantized transform coefficients into a two-dimensional block shape. In this case, the reordering may be performed based on the coefficient scan order performed by the encoding apparatus 100. The inverse quantization unit 220 may perform inverse quantization on quantized transform coefficients by using a quantization parameter (eg, quantization step size information) and obtain transform coefficients.

The inverse transform unit 230 inversely transforms the transform coefficients to obtain a residual signal (residual block, residual sample array).

The prediction unit may perform prediction on the current block and generate a predicted block including prediction samples for the current block. The prediction unit may determine whether intra prediction or inter prediction is applied to the current block based on information on prediction output from the entropy decoding unit 210, and may determine a specific intra/inter prediction mode.

The intra prediction unit 265 may predict the current block by referring to samples in the current picture. The referenced samples may be located near the current block or may be spaced apart according to the prediction mode. In intra prediction, prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The intra prediction unit 265 may determine a prediction mode applied to the current block by using the prediction mode applied to the neighboring block.

The inter prediction unit 260 may derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on the reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, motion information may be predicted in units of blocks, subblocks, or samples based on correlation between motion information between neighboring blocks and current blocks. The motion information may include a motion vector and a reference picture index. The motion information may further include inter prediction direction (L0 prediction, L1 prediction, Bi prediction) information. In the case of inter prediction, the neighboring block may include a spatial neighboring block existing in the current picture and a temporal neighboring block existing in the reference picture. For example, the inter prediction unit 260 may construct a motion information candidate list based on neighboring blocks, and derive a motion vector and/or a reference picture index of the current block based on the received candidate selection information. Inter prediction may be performed based on various prediction modes, and information on prediction may include information indicating a mode of inter prediction for a current block.

The addition unit 235 is reconstructed by adding the obtained residual signal to the prediction signal (predicted block, prediction sample array) output from the prediction unit (including the inter prediction unit 260 and/or the intra prediction unit 265). Signals (restored pictures, reconstructed blocks, reconstructed sample arrays) can be generated. When there is no residual for a block to be processed, such as when the skip mode is applied, the predicted block may be used as a reconstructed block.

The addition unit 235 may be referred to as a restoration unit or a restoration block generation unit. The generated reconstructed signal may be used for intra prediction of the next processing target block in the current picture, and may be used for inter prediction of the next picture through filtering as described later.

The filtering unit 240 may improve subjective/objective image quality by applying filtering to the reconstructed signal. For example, the filtering unit 240 may apply various filtering methods to the reconstructed picture to generate a modified reconstructed picture, and may transmit the modified reconstructed picture to the DPB 255 of the memory 250. . Various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter.

The modified reconstructed picture delivered to the DPB 255 of the memory 250 may be used as a reference picture by the inter prediction unit 260. The memory 250 may store motion information of a block from which motion information in a current picture is derived (or decoded) and/or motion information of blocks in a picture that have already been reconstructed. The stored motion information may be transmitted to the inter prediction unit 260 to be used as motion information of a spatial neighboring block or motion information of a temporal neighboring block. The memory 250 may store reconstructed samples of reconstructed blocks in the current picture, and may be transmitted to the intra prediction unit 265.

In the present specification, embodiments described in the filtering unit 160, the inter prediction unit 180, and the intra prediction unit 185 of the encoding apparatus 100 are respectively the filtering unit 240 and the inter prediction unit 260 of the decoding apparatus. ) And the intra prediction unit 265 may be applied to be the same or correspond to each other.

4 shows an example of a content streaming system according to an embodiment of the present specification. Content streaming systems to which the embodiments of the present specification are applied are largely an encoding server 410, a streaming server 420, a web server 430, and a media storage 440. ), a user equipment 450, and a multimedia input device 460.

The encoding server 410 generates a bitstream by compressing content input from a multimedia input device 460 such as a smartphone, a camera, or a camcorder into digital data, and transmits the generated bitstream to the streaming server 420. As another example, when the multimedia input device 460 such as a smartphone, a camera, or a camcorder directly generates a bitstream, the encoding server 410 may be omitted.

The bitstream may be generated by an encoding method or a bitstream generation method to which an embodiment of the present specification is applied, and the streaming server 420 may temporarily store the bitstream while transmitting or receiving the bitstream.

The streaming server 420 transmits multimedia data to the user device 450 based on a user request through the web server 430, and the web server 430 serves as an intermediary that informs the user of what kind of service exists. When a user requests a desired service from the web server 430, the web server 430 transmits information on the requested service to the streaming server 420, and the streaming server 420 transmits multimedia data to the user. In this case, the content streaming system may include a separate control server, and in this case, the control server serves to control commands/responses between devices in the content streaming system.

The streaming server 420 may receive content from the media storage 440 and/or the encoding server 410. For example, when receiving content from the encoding server 410, the streaming server 420 may receive the content in real time. In this case, in order to provide a smooth streaming service, the streaming server 420 may store the bitstream for a predetermined time.

The user device 450 includes, for example, a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistants (PDA), a portable multimedia player (PMP), a navigation system, and a slate PC ( slate PC), tablet PC, ultrabook, wearable device, e.g., smartwatch, smart glass, head mounted display (HMD)), It can include digital TV, desktop computer, and digital signage.

Each server in the content streaming system may be operated as a distributed server, and in this case, data received from each server may be distributedly processed.

5 shows an example of a video signal processing apparatus according to an embodiment of the present specification. The video signal processing apparatus of FIG. 5 may correspond to the encoding apparatus 100 of FIG. 1 or the decoding apparatus 200 of FIG. 2.

The video signal processing apparatus 500 for processing a video signal includes a memory 520 for storing a video signal, and a processor 510 for processing a video signal while being combined with the memory 520. The processor 510 according to the embodiment of the present specification may be configured with at least one processing circuit for processing a video signal, and may process a video signal by executing instructions for encoding/decoding a video signal. That is, the processor 510 may encode original video data or decode an encoded video signal by executing encoding/decoding methods described below. The processor 510 may be composed of one or more processors corresponding to each of the modules of FIG. 2 or 3. The memory 520 may correspond to the memory 170 of FIG. 2 or the memory 250 of FIG. 3.

Partitioning structure

The video/image coding method according to the present specification may be performed based on a split structure described later. Procedures such as prediction, residual processing (e.g., (inverse) transformation, (inverse) quantization), syntax element coding, and filtering, which will be described later in detail, are CTU (coding tree unit) derived based on the load structure, CU (and/ Alternatively, it may be performed based on TU, PU). The block division procedure may be performed by the video division unit 110 of the encoding apparatus 100 described above, and division-related information is (encoded) processed by the entropy encoding unit 190 and transferred to the decoding apparatus 200 in the form of a bitstream. Can be delivered. The entropy decoding unit 210 of the decoding apparatus 200 derives the block division structure of the current block based on the division-related information obtained from the bitstream, and based on this, a series of procedures (e.g., prediction, registration) for decoding an image. Dual processing, block/picture restoration, and in-loop filtering) can be performed.

In video/image coding according to the embodiment of the present specification, an image processing unit may have a hierarchical structure. One picture may be divided into one or more tiles or tile groups. One tile group may include one or more tiles. One tile may contain more than one CTU. The CTU can be divided into one or more CUs. A tile is a rectangular region of CTUs within a particular tile column and a particular tile row in a picture. The tile group may include an integer number of tiles according to a tile raster scan in a picture. The tile group header may convey information/parameters applicable to the corresponding tile group. When the encoding device 100/decoding device 200 has a multi-core processor, an encoding/decoding procedure for a tile or a group of tiles may be processed in parallel. Here, the tile group may have one type of tile groups including an intra (I) tile group, a predictive (P) tile group, and a bi-predictive (B) tile group. For prediction of blocks in an I tile group, inter prediction is not used and only intra prediction can be used. Of course, even for the I tile group, a coded original sample value may be signaled without prediction. Intra prediction or inter prediction may be used for blocks in a P tile group, and when inter prediction is used, only uni prediction may be used. Meanwhile, intra prediction or inter prediction may be used for blocks in the B tile group, and when inter prediction is used, not only unidirectional prediction but also bi prediction may be used.

6 illustrates an example of a picture division structure according to an embodiment of the present specification. In FIG. 6, a picture having 216 (18 by 12) luminance CTUs is divided into 12 tiles and 3 tile groups.

The encoder determines the size of a tile/tile group and a maximum and minimum coding unit according to a characteristic (e.g., resolution) of a video image or in consideration of coding efficiency or parallel processing, and provides information about this or information for inducing it. It can be included in the bitstream.

The decoder may obtain information indicating whether the tile/tile group of the current picture and the CTU in the tile are divided into a plurality of coding units. Coding efficiency can be increased if such information is not always acquired (decoded) by the decoder, but is acquired (decoded) only under certain conditions.

The tile group header (tile group header syntax) may include information/parameters commonly applicable to the tile group. APS (APS syntax) or PPS (PPS syntax) may include information/parameters that can be commonly applied to one or more pictures. SPS (SPS syntax) may include information/parameters that can be commonly applied to one or more sequences. VPS (VPS syntax) may include information/parameters that can be commonly applied to the entire video. The high-level syntax of the present specification may include at least one of APS syntax, PPS syntax, SPS syntax, and VPS syntax.

Further, for example, information on the division and configuration of a tile/tile group may be configured in an encoder through a higher level syntax and then transmitted to a decoder in the form of a bitstream.

Pictures can be divided into a sequence of coding tree units (CTUs). The CTU may correspond to a coding tree block (CTB). Alternatively, the CTU may include a coding tree block of luma samples and two coding tree blocks of corresponding chroma samples. In other words, for a picture including three sample arrays, the CTU may include an NxN block of luminance samples and two corresponding blocks of chrominance samples. 7 shows an example in which a picture is divided into CTUs.

The maximum allowed size of the CTU for coding and prediction may be different from the maximum allowed size of the CTU for transformation. For example, even if the maximum size of the luminance conversion blocks is 64x64, the maximum allowable size of the luminance block in the CTU may be 128x128.

The CTU may be divided into CUs based on a quad-tree (QT) structure. The quadtree structure can be called a quaternary tree structure. This is to reflect various local characteristics. Meanwhile, in this document, the CTU may be divided based on division of a multi-type tree structure including a binary tree (binary-tree, BT) and a ternary tree (TT) as well as a quad tree. Hereinafter, the QTBT structure may include a quadtree and binary tree-based partitioning structure, and QTBTTT may include a quadtree, binary tree, and ternary tree-based partitioning structure. Alternatively, the QTBT structure may include a quadtree, binary tree, and ternary tree-based partitioning structure. In the coding tree structure, the CU may have a square or rectangular shape. The CTU can be divided into a quadtree structure first. Thereafter, leaf nodes of a quadtree structure may be further divided by a multitype tree structure. For example, as shown in FIG. 8, the multitype tree structure may schematically include four partition types.

The four split types are vertical binary splitting (SPLIT_BT_VER) (310), horizontal binary splitting (SPLIT_BT_HOR) (320), vertical ternary splitting (SPLIT_TT_VER) (330), and horizontal turner. It may include horizontal ternary splitting (SPLIT_TT_HOR) 340. Leaf nodes of a multi-type tree structure can be called CUs. These CUs can be used for prediction and transformation procedures. In this document, in general, CU, PU, and TU may have the same block size. However, if the maximum supported transform length is smaller than the width or height of the color component of the CU, the CU and the TU may have different block sizes.

In Fig. 9, the CTU is treated as the root of a quadtree, and is first partitioned into a quadtree structure. Each quadtree leaf node can then be further partitioned into a multitype tree structure. In a multi-type tree structure, a first flag (eg, mtt_split_cu_flag) is signaled to indicate whether a corresponding node is additionally partitioned. If the corresponding node is additionally partitioned, a second flag (eg, mtt_split_cu_vertical_flag) may be signaled to indicate the splitting direction. Thereafter, a third flag (eg, mtt_split_cu_binary_flag) may be signaled to indicate whether the division type is binary division or ternary division. For example, based on the second flag (mtt_split_cu_vertical_flag) and the third flag (mtt_split_cu_binary_flag), the multi-type tree splitting mode (MttSplitMode) of the CU may be derived as shown in Table 1 below.

In FIG. 10, bold block edges represent quadtree partitioning, and the remaining edges represent multitype tree partitioning. A quadtree partition with a multi-type tree can provide a content-adaptive coding tree structure. The CU may correspond to a coding block (CB). Alternatively, the CU may include a coding block of luminance samples and two coding blocks of corresponding chrominance samples. The size of the CU may be as large as CTU, or may be as small as 4x4 in units of luminance samples. For example, in the case of a 4:2:0 color format (or color difference format), the maximum color difference CB size may be 64x64 and the minimum color difference CB size may be 2x2.

In this document, for example, the maximum allowable luminance TB size may be 64x64, and the maximum allowable color difference TB size may be 32x32. If the width or height of the CB divided according to the tree structure is larger than the maximum transform width or height, the corresponding CB may be automatically (or implicitly) divided until the TB size limit in the horizontal and vertical directions is satisfied.

Meanwhile, for a quadtree coding tree scheme involving a multi-type tree, the following parameters may be defined and identified as SPS syntax elements.

-CTU size: the root node size of a quaternary tree

-MinQTSize: the minimum allowed quaternary tree leaf node size

-MaxBtSize: the maximum allowed binary tree root node size

-MaxTtSize: the maximum allowed ternary tree root node size

-MaxMttDepth: the maximum allowed hierarchy depth of multi-type tree splitting from a quadtree leaf

-MinBtSize: the minimum allowed binary tree leaf node size

-MinTtSize: the minimum allowed ternary tree leaf node size

As an example of a quadtree coding tree structure involving a multitype tree, the CTU size may be set to 128x128 luminance samples and 64x64 blocks of two corresponding color difference samples (in a 4:2:0 chroma format). In this case, MinQTSize is set to 16x16, MaxBtSize is set to 128x128, MaxTtSzie is set to 64x64, MinBtSize and MinTtSize (for width and height mode) may be set to 4x4, and MaxMttDepth may be set to 4. Quart tree partitioning can be applied to CTU to create quadtree leaf nodes. The quadtree leaf node may be referred to as a leaf QT node. Quadtree leaf nodes may have a size of 128x128 (eg, CTU size) from 16x16 size (eg MinOTSize). If the leaf QT node is 128x128, it may not be additionally divided into a binary tree/ternary tree. This is because even if it is split in this case, it exceeds MaxBtsize and MaxTtszie (ie, 64x64). In other cases, the leaf QT node can be further divided into a multi-type tree. Therefore, the leaf QT node is a root node for a multi-type tree, and the leaf QT node may have a multi-type tree depth (mttDepth) of 0. If the multi-type tree depth reaches MaxMttdepth (eg, 4), additional partitioning may not be considered any more. If the width of the multi-type tree node is equal to MinBtSize and less than or equal to 2xMinTtSize, additional horizontal division may not be considered. If the height of the multi-type tree node is equal to MinBtSize and less than or equal to 2xMinTtSize, additional vertical division may not be considered.

In the hardware decoder, in order to allow the pipeline design of 64x64 luminance blocks and 32x32 chrominance blocks, TT segmentation may be forbidden in certain cases. For example, when the width or height of the luma coding block is greater than 64, TT partitioning may be prohibited as shown in FIG. 11. Also, for example, if the width or height of the color difference coding block is greater than 32, TT division may be prohibited.

In this document, the coding tree scheme may support that luminance and color difference (component) blocks have a separate block tree structure. When the luminance and color difference blocks in one CTU have the same block tree structure, it may be indicated as SINGLE_TREE. When the luminance and color difference blocks in one CTU have an individual block tree structure, it may be indicated as DUAL_TREE. In this case, the block tree type for the luminance component may be called DUAL_TREE_LUMA, and the block tree type for the color difference component may be called DUAL_TREE_CHROMA. For P and B slice/tile groups, luminance and color difference CTBs within one CTU may be limited to have the same coding tree structure. However, for I slice/tile groups, the luminance and color difference blocks may have separate block tree structures from each other. If an individual block tree mode is applied, the luminance CTB may be divided into CUs based on a specific coding tree structure, and the color difference CTB may be divided into color difference CUs based on a different coding tree structure. This means that the CU in the I slice/tile group may be composed of a coding block of a luminance component or a coding block of two color difference components, and the CU of a P or B slice/tile group may be composed of blocks of three color components. can do.

Although the quadtree coding tree structure accompanying a multi-type tree has been described above, the structure in which the CU is divided is not limited thereto. For example, the BT structure and the TT structure may be interpreted as a concept included in the Multiple Partitioning Tree (MPT) structure, and the CU may be interpreted as being divided through the QT structure and the MPT structure. In an example in which a CU is divided through a QT structure and an MPT structure, a syntax element (eg, MPT_split_type) including information on how many blocks a leaf node of the QT structure is divided into, and a leaf node of the QT structure are vertical and horizontal. The split structure may be determined by signaling a syntax element (eg, MPT_split_mode) including information on which direction is split.

In another example, the CU may be divided in a different way from the QT structure, BT structure, or TT structure. That is, according to the QT structure, the CU of the lower depth is divided into 1/4 size of the CU of the upper depth, or the CU of the lower depth is divided into 1/2 of the CU of the upper depth according to the BT structure, or according to the TT structure. Unlike CUs of lower depth are divided into 1/4 or 1/2 of CUs of higher depth, CUs of lower depth are 1/5, 1/3, 3/8, 3 of CUs of higher depth depending on the case. It may be divided into /5, 2/3, or 5/8 size, and the method of dividing the CU is not limited thereto.

If a portion of the tree node block exceeds the bottom or right picture boundary, the tree node block restricts all samples of all coded CUs to be located within the picture boundaries. Can be. In this case, for example, the following division rules may be applied.

-If a part of the tree node block is outside the boundary of the lower and right pictures,

· If the block is a QT node and the size of the block is larger than the minimum QT size, the block is forcibly divided into the QT division mode.

Otherwise, the block is forcibly divided into SPLIT_BT_HOR mode.

-Otherwise, if part of the tree node block is outside the lower picture boundary,

· If the block is a QT node and the block size is larger than the minimum QT size and the block size is larger than the maximum BT size, the block is forcibly divided into the QT division mode.

· If the block is a QT node and the size of the block is less than or equal to the minimum QT size, the block is forcibly divided into the QT division mode or the SPLIT_BT_HOR mode.

Otherwise (if the block is a BTT node or the size of the block is less than or equal to the minimum QT size), the block is forcibly split into SPLIT_BT_HOR mode.

-Otherwise, if a part of the tree node block is outside the right picture boundary,

· If the block is a QT node, the size of the block is larger than the minimum QT size, and the size of the block is larger than the maximum BT size, the block is forcibly divided into the QT division mode.

Otherwise, if the block is a QT node, the size of the block is larger than the minimum QT size, and the size of the block is less than or equal to the maximum BT size, the block is forcibly divided into the QT division mode or SPLIT_BT_VER mode.

Otherwise (if the block is a BTT node or the size of the block is less than or equal to the minimum QT size), the block is forcibly divided into SPLIT_BT_VER mode.

A quadtree coding block structure with a multitype tree can provide a very flexible block partitioning structure. Because of the partitioning types supported for the multitype tree, different partitioning patterns can potentially lead to the same coding block structure result in some cases. By limiting the occurrence of such redundant partition patterns, the amount of data of partitioning information can be reduced.

As shown in FIG. 12, two levels of consecutive binary splits in one direction have the same coding block structure as binary division for the center partition after ternary division. . In this case, the binary tree division (in the given direction) for the center partition of the ternary tree division is prohibited. This prohibition can be applied to CUs of all pictures. When such a specific division is prohibited, signaling of corresponding syntax elements may be modified to reflect such a prohibited case, and through this, the number of bits signaled for partitioning may be reduced. For example, as shown in FIG. 12, when binary tree division for the center partition of the CU is prohibited, the mtt_split_cu_binary_flag syntax element indicating whether the division is binary division or ternary division is not signaled, and its value is 0 can be inferred by the decoder.

In addition, virtual pipeline data units (VPDUs) may be defined for intra-picture pipeline processing. VPDUs may be defined as non-overlapping units within one picture. In a hardware decoder, successive VPDUs can be processed simultaneously by multiple pipeline stages. The VPDU size is roughly proportional to the buffer size in most pipeline stages. Therefore, keeping the VDPU size small is important when considering the buffer size from a hardware perspective. In most hardware decoders, the VPDU size can be set equal to the maximum TB size. For example, the VPDU size may be 64x64 (64x64 luminance samples) size. However, this is an example, and the VPDU size may be changed (increased or decreased) in consideration of the TT and/or BT partition described above. In this document, an MxN block may represent a block including samples consisting of M columns and N rows.

13 illustrates an example of a case in which the ternary tree division and the byte tree division are restricted according to an embodiment of the present specification. In order to maintain the VPDU size at 64x64 luminance samples size, at least one of the following restrictions may be applied as shown in FIG. 13.

-TT split is not allowed for a CU with either width or height, or both width and height equal to the width or height, or for a CU with both width and height equal to 128. 128).

-For a 128xN CU with N <= 64 (ie width equal to 128 and height smaller than) 128xN (N <= 64) (i.e., width 128 and height less than 128) CU 128), horizontal BT is not allowed).

-For an Nx128 CU with N <= 64 (ie height equal to 128 and width smaller than) Nx128 (N <= 64) (i.e., the height is 128 and the width is less than 128) 128), vertical BT is not allowed).

Video/video coding procedure

In video/video coding, pictures constituting the video/video may be encoded/decoded according to a series of decoding orders. A picture order corresponding to an output order of a decoded picture may be set differently from a decoding order, and based on this, not only forward prediction but also backward prediction may be performed during inter prediction.

14 is an example of a flowchart for encoding a picture constituting a video signal according to an embodiment of the present specification. In FIG. 14, step S1410 may be performed by the

prediction units

180 and 185 of the encoding apparatus 100 described in FIG. 2, and step S1420 may be performed by the

residual processing units

115, 120, and 130. , S1430 may be performed by the entropy encoding unit 190. Step S1410 may include an inter/intra prediction procedure described in this document, step S1420 may include a residual processing procedure described in this document, and step S1430 includes an information encoding procedure described in this document. can do.

Referring to FIG. 14, the picture encoding procedure is not only a procedure for encoding information for picture restoration (eg, prediction information, residual information, and partitioning information) schematically as described in FIG. 2 to output in a bitstream format, A procedure for generating a reconstructed picture for the current picture and a procedure for applying in-loop filtering to the reconstructed picture (optional) may be included. The encoding apparatus 100 may derive (modified) residual samples from the quantized transform coefficients through the inverse quantization unit 140 and the inverse transform unit 150, and prediction samples corresponding to the output of step S1410 and ( A reconstructed picture may be generated based on the modified) residual samples. The reconstructed picture generated in this way may be the same as the reconstructed picture generated by the decoding apparatus 200 described above. A modified reconstructed picture can be generated through an in-loop filtering procedure for the reconstructed picture, which can be stored in the memory 170 (DPB 175), and, as in the case of the decoding device 200, a subsequent picture It can be used as a reference picture in an inter prediction procedure upon encoding of. As described above, in some cases, some or all of the in-loop filtering procedure may be omitted. When the in-loop filtering procedure is performed, (in-loop) filtering-related information (parameters) may be encoded by the entropy encoding unit 190 and output in the form of a bitstream, and the decoding apparatus 200 The in-loop filtering procedure may be performed in the same manner as the encoding apparatus 100.

Through this in-loop filtering procedure, noise generated during image/video coding such as blocking artifacts and ringing artifacts may be reduced, and subjective/objective visual quality may be improved. In addition, since both the encoding device 100 and the decoding device 200 perform an in-loop filtering procedure, the encoding device 100 and the decoding device 200 can derive the same prediction result, so that the reliability of picture coding is This can be improved and the amount of data transmitted for picture coding can be reduced.

15 is an example of a flowchart for decoding a picture constituting a video signal according to an embodiment of the present specification. Step S1510 may be performed by the entropy decoding unit 210 in the decoding apparatus 200 of FIG. 3, step S1520 may be performed by the

prediction units

260 and 265, and step S1530 may be performed by the residual processing unit ( 220, 230), step S1540 may be performed by the addition unit 235, step S1350 may be performed by the filtering unit 240. Step S1510 may include the information decoding procedure described in this document, step S1520 may include the inter/intra prediction procedure described in this document, and step S1530 includes the residual processing procedure described in this document. In addition, step S1540 may include the block/picture restoration procedure described in this document, and step S1550 may include the in-loop filtering procedure described in this document.

Referring to FIG. 15, the picture decoding procedure is a procedure for obtaining image/video information (through decoding) from a bitstream (S1510), a picture restoration procedure (S1520 to S1540), and a reconstructed picture, as described in FIG. It may include an in-loop filtering procedure (S1550) for. The picture restoration procedure is based on prediction samples and residual samples obtained through the process of inter/intra prediction (S1520) and residual processing (S1530, inverse quantization and inverse transformation of a quantized code or coefficient) described in this document. Can be done. A modified reconstructed picture may be generated through an in-loop filtering procedure for a reconstructed picture generated through a picture restoration procedure, and the modified reconstructed picture may be output as a decoded picture, and the decoding apparatus 200 It is stored in the DPB 255 of and can be used as a reference picture in inter prediction train when decoding a picture later. In some cases, the in-loop filtering procedure may be omitted, and in this case, the reconstructed picture may be output as a decoded picture, stored in the DPB 255 of the decoding device 200, and referenced in the inter prediction train when decoding a subsequent picture. Can be used as a picture. The in-loop filtering procedure S1550 may include a deblocking filtering procedure, a sample adaptive offset (SAO) procedure, an adaptive loop filter (ALF) procedure, and/or a bi-lateral filter procedure as described above. And some or all of them may be omitted. In addition, one or some of the deblocking filtering procedure, the SAO procedure, the ALF procedure, and the bilateral filter procedure may be sequentially applied, or all may be sequentially applied. For example, after the deblocking filtering procedure is applied to the reconstructed picture, the SAO procedure may be performed. Also, for example, after the deblocking filtering procedure is applied to the reconstructed picture, the ALF procedure may be performed. This may be similarly performed in the encoding device 100.

As described above, not only the decoding device 200 but also the encoding device 100 may perform a picture restoration procedure. A reconstructed block may be generated based on intra prediction/inter prediction for each block, and a reconstructed picture including the reconstructed blocks may be generated. When the current picture/slice/tile group is an I picture/slice/tile group, blocks included in the current picture/slice/tile group may be reconstructed based only on intra prediction. In this case, inter prediction may be applied to some blocks in the current picture/slice/tile group, and intra prediction may be applied to the remaining some blocks. The color component of a picture may include a luminance component and a chrominance component, and the methods and embodiments proposed in this document may be applied to the luminance component and the chrominance component unless explicitly limited in this document.

The coded image is a video coding layer (VCL) that deals with the decoding process of the image and itself, a subsystem that transmits and stores encoded information, and a network abstraction (NAL) that exists between the VCL and the subsystem and is responsible for network adaptation. layer).

VCL data including video data (tile group data) compressed in the VCL is generated, or a parameter set including information such as PPS (picture parameter set), SPS (sequence parameter set), VPS (video parameter set), or An additionally required SEI (supplemental enhancement information) message may be generated in the process of decoding an image.

In NAL, header information (NAL unit data) may be added to a raw byte sequence payload (RBSP) generated in VCL to generate a NAL unit. In this case, the RBSP may refer to tile group data, parameter set, and SEI message generated in the VCL. In the NAL unit header, NAL unit type information specified according to RBSP data included in the corresponding NAL unit may be included.

As shown in FIG. 16, the NAL unit may be divided into a VCL NAL unit and a Non-VCL NAL unit according to an RBSP generated from VCL. The VCL NAL unit may mean a NAL unit that includes information about an image (tile group data), and the Non-VCL NAL unit is an NAL that includes information (parameter set or SEI message) necessary for decoding an image. It can mean a unit.

The above-described VCL NAL unit and Non-VCL NAL unit may be transmitted through a network with header information added according to the data standard of the sub-system. For example, the NAL unit may be converted into a data format of a predetermined standard such as an H.266/VVC file format, a real-time transport protocol (RTP), and a transport stream (TS) and then transmitted through various networks.

As described above, the NAL unit type may be specified according to the RBSP data structure included in the corresponding NAL unit, and information on the NAL unit type may be stored in the NAL unit header and signaled.

For example, the NAL unit may be largely classified into a VCL NAL unit type and a Non-VCL NAL unit type according to whether or not information on an image (tile group data) is included. The VCL NAL unit type may be classified according to the nature and type of a picture included in the VCL NAL unit, and the non-VCL NAL unit type may be classified according to the type of the parameter set.

The following is an example of a NAL unit type specified according to the type of a parameter set included in the Non-VCL NAL unit type.

-APS (Adaptation Parameter Set) NAL unit: A type for a NAL unit including APS

-Video Parameter Set (VPS) NAL unit: A type for a NAL unit including a VPS

-SPS (Sequence Parameter Set) NAL unit: a type for a NAL unit including SPS

-PPS (Picture Parameter Set) NAL unit: A type for a NAL unit including PPS

The above-described NAL unit types have syntax information for the NAL unit type, and the syntax information may be stored in the NAL unit header and signaled. For example, syntax information may be nal_unit_type, and NAL unit types may be specified by nal_unit_type values.

The tile group header (tile group header syntax) may include information/parameters commonly applicable to the tile group. APS (APS syntax) or PPS (PPS syntax) may include information/parameters that can be commonly applied to one or more pictures. SPS (SPS syntax) may include information/parameters that can be commonly applied to one or more sequences. VPS (VPS syntax) may include information/parameters that can be commonly applied to the entire video. In this specification, the higher-level syntax may include at least one of APS syntax, PPS syntax, SPS syntax, and VPS syntax.

In this specification, the image/video information encoded by the encoding device 100 by the decoding device 200 and signaled in the form of a bitstream includes intra-picture partitioning-related information, intra/inter prediction information, residual information, and in-loop filtering information. In addition, information included in the APS, information included in the PPS, information included in the SPS, and/or the information included in the VPS may be included.

Transform

In this document, MTS (Multiple Transform Selection, hereinafter referred to as'MTS') may mean a method of performing transformation using at least two or more transformation types. This may also be expressed as AMT (Adaptive Mutliple Transform) or EMT (Explicit Multiple Transform), and likewise, mts_idx may also be expressed as AMT_idx, EMT_idx, AMT_TU_idx EMT_TU_idx, transformation index or transformation combination index, and the like, and embodiments of the present specification are It is not limited to these expressions.

17 is a schematic block diagram of a transform and quantization unit, an inverse quantization and inverse transform unit in an encoding apparatus according to an embodiment of the present specification, and FIG. 18 is a schematic block diagram of an inverse quantization and inverse transform unit in a decoding apparatus.

Referring to FIG. 17, the transform and quantization unit 120/130 may include a primary transform unit 121, a secondary transform unit 122, and a quantization unit 130. have. The inverse quantization and inverse transform unit 140/150 may include an inverse quantization unit 140, an inverse secondary transform unit 151, and an inverse primary transform unit 152. I can.

Referring to FIG. 18, the inverse quantization and inverse transform unit 220/230 includes an inverse quantization unit 220, an inverse secondary transform unit 231, and an inverse primary transform unit ( 232) may be included.

In the present invention, when performing the transformation, the transformation may be performed through a plurality of steps. For example, as shown in FIG. 17, two steps of a primary transform and a secondary transform may be applied, or a higher transform step may be used depending on an algorithm. Here, the first-order transform may be referred to as a core transform.

The first-order transform unit 121 may apply a first-order transform to the residual signal, where the first-order transform may be predefined as a table in the encoder and/or decoder.

The second-order transform unit 122 may apply a second-order transform to the first-order transformed signal, where the second-order transform may be predefined as a table in the encoder and/or decoder.

In one embodiment, a non-separable secondary trans-form (NSST) may be conditionally applied as a secondary transform. For example, NSST is applied only to an intra prediction block, and may have a transform set applicable to each prediction mode group.

Here, the prediction mode group may be set based on symmetry with respect to the prediction direction. For example, since prediction mode 52 and prediction mode 16 are symmetric with respect to prediction mode 34 (diagonal direction), one group may be formed and the same transform set may be applied. In this case, when applying the transform for the prediction mode 52, the input data is transposed and then applied, because the prediction mode 16 and the transform set are the same.

Meanwhile, in the case of the planar mode and the DC mode, since there is no symmetry with respect to a direction, each transform set may be formed, and the corresponding transform set may be composed of two transforms. For the remaining directional modes, three transforms may be configured for each transform set.

The quantization unit 130 may perform quantization on the second-order transformed signal.

The inverse quantization and inverse transform unit 140/150 performs the above-described process in reverse, and redundant descriptions will be omitted.

In this specification, an embodiment of a separable transform in which a transform is applied by separating a horizontal direction and a vertical direction is basically described, but the transform combination may also be composed of non-separable transforms. .

Alternatively, a transform combination may be configured by mixing separable transforms and non-separable transforms. In this case, when non-separated transformation is used, selection of transformation for each row/column or selection for each horizontal/vertical direction becomes unnecessary, and transformation combinations can be used only when a separable transformation is selected.

In addition, the schemes proposed in the present specification can be applied regardless of the first-order transformation or the second-order transformation. That is, there is no restriction that it should be applied to either of the two, and both can be applied. Here, the first-order transform may mean a transform for firstly transforming the residual block, and the second-order transform may mean a transform for applying a transform to a block generated as a result of the first-order transform.

First, the encoding apparatus 100 may determine a transform group corresponding to the current block (S1910). Here, the transform group may refer to the transform group shown in Table 2 below, but the present invention is not limited thereto and may be composed of other transform combinations.

The encoding apparatus 100 may perform transformation on candidate transformation combinations available in the transformation group (S1920). As a result of performing the conversion, the encoding apparatus 100 may determine or select a conversion combination having the lowest RD (rate distortion) cost (S1930). The encoding apparatus 100 may encode a transformation combination index corresponding to the selected transformation combination (S1940).

First, the decoding apparatus 200 may determine a transform group for the current block (S2010). The decoding apparatus 200 may parse the transform combination index, where the transform combination index may correspond to any one of a plurality of transform combinations in the transform group (S2020). The decoding apparatus 200 may induce a transform combination corresponding to the transform combination index (S2030). The decoding apparatus 200 may perform an inverse transform on the current block based on the transform combination (S2040). When the transformation combination is composed of row transformation and column transformation, the row transformation may be applied first and then the column transformation may be applied. However, the present invention is not limited thereto, and the opposite is applied, or when non-separated transforms are configured, the non-separated transform may be directly applied.

The decoding apparatus 200 to which the present invention is applied may acquire sps_mts_intra_enabled_flag or sps_mts_inter_enabled_flag (S2110). Here, sps_mts_intra_enabled_flag indicates whether cu_mts_flag is present in the residual coding syntax of the intra coding unit. For example, if sps_mts_intra_enabled_flag = 0, cu_mts_flag does not exist in the residual coding syntax of the intra coding unit, and if sps_mts_intra_enabled_flag = 1, cu_mts_flag is present in the residual coding syntax of the intra coding unit. In addition, sps_mts_inter_enabled_flag indicates whether cu_mts_flag is present in the residual coding syntax of the inter coding unit. For example, if sps_mts_inter_enabled_flag = 0, cu_mts_flag does not exist in the residual coding syntax of the inter coding unit, and if sps_mts_inter_enabled_flag = 1, cu_mts_flag is present in the residual coding syntax of the inter coding unit.

The decoding apparatus 200 may acquire cu_mts_flag based on sps_mts_intra_enabled_flag or sps_mts_inter_enabled_flag (S2120). For example, when sps_mts_intra_enabled_flag = 1 or sps_mts_inter_enabled_flag = 1, the decoding device 200 may acquire cu_mts_flag. Here, cu_mts_flag indicates whether MTS is applied to the residual samples of the luminance conversion block. For example, if cu_mts_flag = 0, the MTS is not applied to the residual samples of the luminance transform block, and if cu_mts_flag = 1, the MTS is applied to the residual samples of the luminance transform block.

The decoding apparatus 200 may acquire mts_idx based on cu_mts_flag (S2030). For example, when cu_mts_flag = 1, the decoding apparatus 200 may acquire mts_idx. Here, mts_idx indicates which transform kernel is applied to the luminance residual samples along the horizontal and/or vertical directions of the current transform block.

For example, for mts_idx, at least one of the embodiments described herein may be applied.

The decoding apparatus 200 may derive a transform kernel corresponding to mts_idx (S2040). For example, a transformation kernel corresponding to mts_idx may be divided into horizontal transformation and vertical transformation and defined.

For example, when MTS is applied to the current block (ie, cu_mts_flag = 1), the decoding apparatus 200 may configure an MTS candidate based on the intra prediction mode of the current block. In this case, the step of configuring an MTS candidate may be further included in the decoding flowchart of FIG. 21. In addition, the decoding apparatus 200 may determine an MTS candidate applied to the current block using mts_idx from among the configured MTS candidates.

As another example, different transformation kernels may be applied for horizontal transformation and vertical transformation. However, the present invention is not limited thereto, and the same transformation kernel may be applied to the horizontal transformation and the vertical transformation.

As an embodiment, a horizontal transform kernel and a vertical transform kernel according to mts_idx may be defined as shown in Table 2 below.

In Table 2, trTypeHor and trTypeVer represent the type of the horizontal conversion kernel and the type of the vertical conversion kernel. If trTypeHor and trTypeVer are 0, DCT2 is applied, if trTypeHor and trTypeVer are 1, DST7 is applied, and if trTypeHor and trTypeVer are 2, DCT8 can be applied.

Further, the decoding apparatus 200 may perform inverse transformation based on the transformation kernel (S2050).

Unified transform type signaling

Table 3 below shows an example of the syntax for the syntax element tu_mts_idx.

Compared with Example 1, in Example 2, instead of parsing the TS (transform skip) flag after fixed length coding with two bins for the MTS flag and MTS index, the syntax element tu_mts_idx according to Example 2 was truncated. It uses truncated unary binarization. The first bin indicates TS, the second bin indicates MTS, and the rest indicate MTS index. The semantics and binarization methods are shown in Table 4 below.

Alternatively, the semantics and binarization method may be as shown in Table 5.

The number of context models is not changed, and the allocation of the context index increment CtxInc to each bin of tu_mts_idx may be as shown in Table 6 below.

Entropy coding

Some or all of the above-described video/video information may be entropy-encoded by the entropy encoding unit 190, and some or all of the above-described video/video information may be entropy-decoded by the entropy decoding unit 210. Among the above-described entropy coding, CABAC may be employed as a technique for entropy coding, and syntax element(s) included in residual information to be described later may be entropy-coded based on CABAC.

The video/image information may be encoded/decoded in units of syntax elements. In this document, that information is encoded/decoded may include encoding/decoding by the method described in this paragraph.

22 is a block diagram for entropy encoding according to an embodiment of the present specification. 22 shows a block diagram of CABAC for encoding one syntax element. The encoding process of CABAC first converts the input signal into a binary value through binarization when the input signal is a syntax element rather than a binary value. If the input signal is already binary, it is bypassed without going through binarization. Here, each

binary number

0 or 1 constituting the binary value is called a bin. For example, if the binary string (empty string) after binarization is 110, each of 1, 1, and 0 is referred to as one bin. The bin(s) for one syntax element may represent a value of a corresponding syntax element.

The binarized bins are input into a regular coding engine or a bypass coding engine. The regular coding engine allocates a context model that reflects a probability value to the corresponding bin, and codes the corresponding bin based on the assigned context model. The regular coding engine may update the probability model for the corresponding bin after performing coding for each bin. Bins coded in this way are called context-coded bins. The bypass coding engine omits the procedure of estimating the probability for the input bin and the procedure of updating the probability model applied to the corresponding bin after coding. Instead of allocating context, a uniform probability distribution (eg, 50:50) is applied to code the input bins to improve coding speed. Bins coded in this way are called bypass bins. The context model may be allocated and updated for each context-coded (normally coded) bin, and the context model may be indicated based on ctxIdx or ctxInc. ctxIdx may be derived based on ctxInc. Specifically, for example, a context index (ctxIdx) indicating a context model for each of the regularly coded bins may be derived as a sum of a context index increment (ctxInc) and a context index offset (ctxIdxOffset). Here, ctxInc may be derived differently for each bin. ctxIdxOffset may be represented by the lowest value of ctxIdx. The minimum value of ctxIdx may be referred to as an initial value (initValue) of ctxIdx. ctxIdxOffset is a value generally used to distinguish context models for other syntax elements, and a context model for one syntax element may be identified/derived based on ctxInc.

In the entropy encoding procedure, it is determined whether encoding is performed through a regular coding engine or a bypass coding engine, and a coding path may be switched. Entropy decoding performs the same process as entropy encoding in reverse order.

Referring to FIG. 23B, the encoding apparatus 100 (entropy encoding unit 190) performs an entropy coding procedure for image/video information. The image/video information may include partitioning related information, prediction related information (eg, inter/intra prediction classification information, intra prediction mode information, inter prediction mode information), residual information, and in-loop filtering related information, or It can contain various syntax elements for. Entropy coding may be performed in units of syntax elements. Steps S2310 to S2320 of FIG. 23A may be performed by the entropy encoding unit 190 of the encoding apparatus 100 of FIG. 2 described above.

The encoding device 100 performs binarization on the target syntax element (S2310). Here, binarization may be based on various binarization methods such as a Truncated Rice binarization process and a fixed-length binarization process, and a binarization method for the target syntax element may be predefined. The binarization procedure may be performed by the binarization unit 192 in the entropy encoding unit 190.

The encoding device 100 performs entropy encoding on the target syntax element (S2320). The encoding apparatus 100 may encode an empty string of a target syntax element based on an entropy coding technique such as CABAC or CAVLC based on normal coding (context based) or bypass coding, and the output may be included in a bitstream. . The entropy encoding procedure may be performed by the entropy encoding processing unit 193 in the entropy encoding unit 190. As described above, the bitstream can be delivered to a decoding device through a (digital) storage medium or a network.

Referring to FIG. 24B, the decoding apparatus 200 (entropy decoding unit 210) may decode encoded image/video information. The image/video information may include partitioning related information, prediction related information (eg, inter/intra prediction classification information, intra prediction mode information, inter prediction mode information), residual information, and in-loop filtering related information, or It can contain various syntax elements related to it. Entropy coding may be performed in units of syntax elements. Steps S2410 to S2420 may be performed by the entropy decoding unit 210 of the decoding apparatus 200 of FIG. 3 described above.

The decoding apparatus 200 performs binarization on the target syntax element (S2410). Binarization can be based on various binarization methods such as the Truncated Rice binarization process and the Fixed-length binarization process, and the binarization method for the target syntax element can be predefined. The decoding apparatus may derive available bin strings (empty string candidates) for available values of a target syntax element through a binarization procedure. The binarization procedure may be performed by the binarization unit 212 in the entropy decoding unit 210.

The decoding apparatus 200 performs entropy decoding on the target syntax element (S2420). The decoding apparatus 200 sequentially decodes and parses each bin for a relative syntax element from the input bit(s) in the bitstream, and compares the derived bin string with the available bin strings for the corresponding syntax element. If the derived empty string is the same as one of the available empty strings, a value corresponding to the corresponding empty string is derived as a value of the corresponding syntax element. If not, the next bit in the bitstream is further parsed and the above-described procedure is performed again. Through this process, the information can be signaled using variable length bits without using a start bit or an end bit for specific information (specific syntax element) in the bitstream. Through this, relatively fewer bits can be allocated to a low value, and overall coding efficiency can be improved.

The decoding apparatus 200 may decode each bin in the bin string from the bitstream based on an entropy coding technique such as CABAC or CAVLC based on context or bypass. As described above, the bitstream may include various information for video/video decoding. As described above, the bitstream can be delivered to a decoding device through a (digital) storage medium or a network.

As described above, residual samples may be derived into quantized transform coefficients through a transform and quantization process. In this case, the quantized transform coefficients may also be called transform coefficients. In this case, the transform coefficients within the block may be signaled in the form of residual information. The residual information may include residual coding syntax. That is, the encoding apparatus 100 may construct a residual coding syntax with residual information, encode it, and output it in the form of a bitstream, and the decoding apparatus 200 decodes the residual coding syntax from the bitstream to obtain a residual ( Quantized) transform coefficients can be derived. As described later in Table 7, the residual coding syntax is a syntax element indicating where the position of the last effective transform coefficient in the corresponding block is, whether there is an effective transform coefficient in the subblock, and the size/code of the effective transform coefficient ( syntax elements).

For example, the transform coefficients are encoded by using any or all of the transform_skip_flag, last_sig_coeff_x_prefix, last_sig_coeff_y_prefix, last_sig_coeff_x_suffix, last_sig_coeff_y_suffix, coded_sub_block_flag, sig_coeff_flag, par_level_flag, abs_level_gt1_flag, abs_level_gt3_flag, abs_remainder, coeff_sign_flag, and mts_idx syntax element (syntax element). This may be referred to as residual (data) coding or (transform) coefficient coding. Meanwhile, the conversion/quantization process may be omitted. In this case, values of residual samples may be coded and signaled according to a predetermined method. Table 7 shows syntax elements related to residual data encoding.

transform_skip_flag indicates whether transformation is omitted in an associated block. The associated block may be a coding block (CB) or a transform block (TB). Regarding the transform (and quantization) and residual coding procedure, CB and TB may be used interchangeably. For example, it is as described above that residual samples are derived for CB, and (quantized) transform coefficients can be derived through transform and quantization of the residual samples, and through a residual coding procedure. Information (or syntax elements) efficiently representing the (quantized) transform coefficients (position, size, sign, etc.) can be generated and signaled. Quantized transform coefficients can simply be called transform coefficients. In general, when the CB is not larger than the maximum TB, the size of the CB may be the same as the size of the TB. In this case, the target block to be transformed (and quantized) and residual coded may be referred to as CB, and may be referred to as TB. On the other hand, when CB is larger than the maximum TB, the target block to be transformed (and quantized) and residual coded may be referred to as TB. Hereinafter, syntax elements related to residual coding will be described as being signaled in units of transform blocks (TB), but this is an example, as described above, that the TB can be mixed with the coding block (CB).

The (x, y) position information of the last non-zero coefficient in the transform block is encoded as last_sig_coeff_x_prefix, last_sig_coeff_y_prefix, last_sig_coeff_x_suffix, and last_sig_coeff_y_suffix. Specifically, last_sig_coeff_x_prefix represents the prefix of the column position of the last significant coefficient in the scanning order in the transform block, and last_sig_coeff_y_prefix is the scan order in the transform block Represents the prefix of the row position of the last significant coefficient in (scanning order), and last_sig_coeff_x_suffix is the last in the scanning order in the transform block Represents the suffix of the column position of the significant coefficient, and last_sig_coeff_y_suffix is the row position of the last significant coefficient in the scanning order in the transform block Represents the suffix of (row position). Here, the effective coefficient may represent the non-zero coefficient. The scan order may be an upward-right diagonal scan order. Alternatively, the scan order may be a horizontal scan order or a vertical scan order. The scan order may be determined based on whether intra/inter prediction is applied to a target block (CB or CB including TB) and/or a specific intra/inter prediction mode.

Next, after dividing the transform block into 4x4 sub-blocks, a 1-bit syntax element coded_sub_block_flag is used for each 4x4 sub-block to indicate whether a non-zero coefficient exists in the current sub-block. The sub-block may be used interchangeably with a coefficient group (CG).

If coded_sub_block_flag is 0, since there is no more information to be transmitted, the encoding process for the current sub-block ends. Conversely, if the value is 1, the sig_coeff_flag encoding process is continuously performed. The coded_sub_block_flag coding is unnecessary for the last sub-block containing the non-zero coefficient, and the sub-block containing the DC information of the transform block has a high probability of containing the non-zero coefficient, so the coded_sub_block_flag is not encoded, and this value is 1 Assume this.

If coded_sub_block_flag indicates that a non-zero coefficient exists in the current sub-block, sig_coeff_flag having a binary value is encoded according to the reverse scan order. The 1-bit syntax element sig_coeff_flag is encoded for each coefficient according to the scan order. If the coefficient value at the current scan position is not 0, the sig_coeff_flag value becomes 1. Here, in the case of a sub-block including the last non-zero coefficient, since sig_coeff_flag is not required to be encoded for the last non-zero coefficient, the encoding process is omitted. Level information encoding is performed only when sig_coeff_flag is 1, and four syntax elements are used in the level information encoding process. Specifically, each sig_coeff_flag[xC][yC] represents whether the level (value) of the corresponding transform coefficient at each transform coefficient position (xC, yC) in the current TB is non-zero.

The remaining level value after sig_coeff_flag encoding is shown in Equation 1. At this time, the level value remAbsLevel to be encoded is as follows. Here, coeff means an actual transform coefficient value.

abs_level_gt1_flag indicates whether remAbsLevel' at the corresponding scanning position (n) is greater than 1. If the value of abs_level_gt1_flag is 0, the absolute value of the coefficient at the corresponding position is 1. If the value of abs_level_gt1_flag is 1, then the level value remAbsLevel to be encoded is equal to Equation 2 below.

The least significant coefficient (LSB) value of remAbsLevel of Equation 2 is encoded as shown in Equation 3 through par_level_flag. That is, par_level_flag[n] may represent parity of the transform coefficient level (value) at the scanning position n. After par_leve_flag encoding, the level value remAbsLevel to be encoded is updated as shown in Equation 4.

abs_level_gt3_flag indicates whether remAbsLevel' at the corresponding scanning position n is greater than 3. Only when abs_level_gt3_flag is 1, abs_remainder encoding is performed. The relationship between the actual transform coefficient value coeff and each syntax element is shown in Equation 5, and Table 2 shows some examples. Finally, the sign of each coefficient is coded using a 1-bit symbol coeff_sign_flag. | coeff| represents a transform coefficient level (value), and may be expressed as AbsLevel for the transform coefficient.

coeff_sign_flag represents the sign of the transform coefficient level at the corresponding scanning position (n). Table 8 below shows syntax elements of the transform coefficient level |coeff|.

This block may be a 4x4 transform block, or may be an 8x8, 16x16, 32x32 code or a 4x4 subblock of a block. Table 9 shows encoding results for the coefficients scanned inverse diagonally in FIG. 25. In Table 9, scan_pos refers to the position of the coefficient according to the inverse diagonal scan. scan_pos 15 is the coefficient of the first scan in the 4x4 block, that is, the lower right corner, and scan_pos 0 is the coefficient of the last scan, that is, the upper left corner.

On the other hand, CABAC provides high performance, but has a disadvantage of poor throughput performance. This is due to the regular encoding engine of CABAC. Since the regular encoding uses the updated probability state and range through encoding of the previous bin, it shows high data dependence, and it takes a lot of time to read the probability interval and determine the current state. This can solve the CABAC throughput problem by limiting the number of context-coded bins. Therefore, as shown in Table 7, the sum of bins used to express sig_coeff_flag, abs_level_gt1_flag, par_level_flag, and abs_level_gt3_flag may be limited to 32 for 4x4 subblocks and 8 (remBinsPass1) for 2x2 subblocks according to the size of the subblock. . When all the restricted context encoding bins are used to encode the context element, bypass encoding can be performed by binarizing the coefficients for the remaining coefficients without using CABAC.

Residual Coding for Transform Skip

In order to adapt the residual coding to the statistic and signal characteristics of transform skip levels representing the quantized predictive residual (spatial domain), the following methods may be provided.

No last significant coefficient scanning position: Since the residual signal reflects the spatial residual after prediction and energy compaction by transformation for TS (transform skip) is not performed, a high probability or transformation for zero tailings The insignificant at the lower right corner of the block is no longer given. Thus, in this case, signaling of the last significant coefficient scanning position is omitted. Instead, the subblock to be processed first is the lower-right subblock in the transform block.

Subblock CBF: The absence of last significant coefficient scanning position signaling requires that subblock CBF signaling using coded_sub_block_flag for TS is changed as follows.

-Due to quantization, the above-described ineffective sequence still occurs locally inside the transform block. Thus, the last valid scanning position is removed as previously described and coded_sub_block_flag is coded for all subblocks.

-The coded_sub_block_flag for the DC frequency position (upper left subblock) subblock indicates a special case. The coded_sub_block_flag for the DC frequency location subblock is not signaled and is always inferred as 1. When the last valid scanning position is in another subblock, this means that at least one valid level exists outside the DC subblock. As a result, even though it is inferred that the coded_sub_block_flag for the DC subblock is 1, the DC subblock may only include zero/invalid levels. A coded_sub_block_flag for each subblock is signaled together with the absence of last scanning position information in the TS. This includes coded_sub_block_flag for DC subblocks except when all other coded_sub_block_flag syntax elements are already 0. In this case, DC coded_sub_block_flag is deduced as 1 (inferDcSbCbf=1). Since at least one valid level must exist in this DC subblock, if all other sig_coeff_flag syntax elements in the DC subblock are 0, the sig_coeff_flag syntax element for the first position at (0,0) is not signaled and is 1 Is inferred instead (inferSbDcSigCoeffFlag=1).

-The context modeling for coded_sub_block_flag is changed. The context model index is calculated as the sum of the coded_sub_block_flag on the right side of the current subblock and the coded_sub_block_flag on the bottom side and the logical disjunction of both sides.

sig_coeff_flag context modeling: In sig_coeff_flag, the local template context modeling includes only the right neighbor (NB0) and the lower neighbor (NB1) for the current scanning position. The context Moel offset is only the number of valid neighbor locations sig_coeff_flag[NB0] + sig_coeff_flag[NB1]. Thus, the selection of different context sets according to the diagonal d in the current block is removed. This derives a single context model set and three context models for the sig_coeff_flag flag.

abs_level_gt1_flag and par_level_flag context modeling: A single context model is used for abs_level_gt1_flag and par_level_flag.

abs_remainder coding: Although the empirical distribution of the transform skip from skip residual absolute levels is still suitable for Laplacian or geometrical distribution, the instationarity of transform coefficients is greater than the absolute levels. In particular, the variance inside the window of successive realization is higher than the residual absolute levels, hereinafter causing abs_remainder syntax binarization and changes in context modeling.

-Using a high cutoff value in binarization, i.e., a turning point from coding with abs_level_gt1_flag, par_level_flag, and abs_level_gt3_flag to Rice codes for abs_remainder, and dedicated for each bin position Context models require high compression efficiency. Increasing the cutoff increases more "greater than X" flags, e.g. leading to abs_level_gt5_flag, abs_level_gt7_flag, etc. until the cutoff is reached. The cutoff itself is fixed at 5 (numGtFlags=5).

-The template for deriving the Rice parameter is changed, that is, the left neighbor and the lower neighbor at the current scanning position are considered to be similar to the local template for sig_coeff_flag context modeling.

coeff_sign_flag context modeling: even when the global empirical distribution is almost uniformly distributed, the codes are used using context models due to the fact that the predictive residual is often skewed and the instationarities inside the sequences of codes. Is coded. A single dedicated context model is used for coding of codes, and the code is parsed after sig_coeff_flag to keep all context coded bins together.

Reduction of context coded bins: The first scanning pass, i.e. the transmission of sig_coeff_flag, abs_level_gt1_flag, and par_level_flag syntax elements, is not changed. However, the limit on the maximum number of context coded bins (CCBs) per sample is removed and managed differently. Reduction of CCBs may be guaranteed by setting a mode in which CCB> k (k is a positive integer) to be invalid. For example, k = 2 may be applied as a normal level coding mode. This limitation corresponds to the reduction of the quantization space.

Transform skip residual coding syntax may be as shown in Table 10 below.

Definition of the conversion skip size when using the conversion kernel index including conversion status information

Blocks that have not undergone transcoding have different characteristics of residual data from those in which general transcoding has been performed, so efficient residual data coding for blocks to which transcoding has not been performed, that is, a block to which transform skip is applied. I need a way. The conversion status flag indicating whether conversion is performed may be transmitted in units of conversion blocks. However, the size of the transform block may vary and is not limited to a specific value. For example, when the conversion status flag is 1, the residual data encoding proposed in the present invention may be performed, and when the conversion status flag is 0, the existing residual data encoding described in Table 7 may be performed.

Alternatively, the residual data encoding except for coeff_sign_flag below coded_sub_block_flag may follow some or all of the previously described residual data encoding method. When unified transform type signaling is performed, when tu_mts_idx is 1 (or when the value of tu_mts_idx indicates that transformation is not applied to the target block), the proposed residual data encoding is performed. In this case, when tu_mts_idx is 0 (or a value other than 1), residual data encoding based on the syntax elements of Table 7 may be performed. In this case, the transform_skip_flag syntax element and/or the mts_idx syntax element in Table 7 may be omitted.

Alternatively, when tu_mts_idx 0 indicates that no transformation is applied to the target block (transform skip), when tu_mts_idx is 0, the proposed residual data encoding may be performed. In this case, when tu_mts_idx is not 0, residual data encoding based on the syntax elements described in Table 7 (transform_skip_flag syntax element and/or mts_idx syntax element may be omitted) may be performed. In this case "tu_mts_idx == 1?" The determination procedure may be replaced with a "tu_mts_idx == 0" determination procedure.

When the binarization for the transform skip and the transform index is mts_enabled, ts_enabled, mts_enabled, and ts_enabled are all possible, each may be defined differently. Likewise, the size at which the transform skip is defined may be differently defined depending on whether mts_enabled is 0 or 1.

1) If mts_enabled=1, the size of the transform skip may be dependent on the allowed MTS. For example, if the size of the MTS is allowed to be 32 or less, the transform skip may be always defined for a block having a block size of 32 or less.

2) Alternatively, in the case of mts_enabled=1, the maximum size already promised between the encoder and the decoder may be used. And ts_enabled may be defined according to this maximum size. For example, an encoder and a decoder can be defined to use only transform skips of blocks with the size of one side less than or equal to 8. In this case, the binarization table of "Unified Transform Type Signaling" can be effectively applied by defining ts_enabled = 0 for a block with a size of one side of the block larger than 8. The maximum size information of the transform skip block may be expressed as the maximum number of pixels rather than the maximum size of one side of the block.

3) Alternatively, in the case of mts_enabled=1, the maximum size of the transform skip may be defined separately from the size of the MTS. In this case, information on the transform skip size may be transmitted to the decoder to define the size of the MTS. For example, if the size of the MTS is allowed to be less than 32, a flag indicating whether to follow the size of the MTS is transmitted, and if the maximum size of the MTS is not followed, information to allow the maximum size of the transform skip to 16 is provided. Can be transmitted. In this case, when the size of one side of the block is 16 or more, ts_enabled = 0 is defined, so that the binarization table of "unified transformation type signaling" can be effectively applied. Alternatively, the maximum size information of the transform skip block may be expressed as the maximum number of pixels instead of the maximum size of one side of the block.

4) Alternatively, when mts_enabled = 0, the maximum size already promised between the encoder and the decoder can be used. For example, when mts_enabled = 0 and ts_enabled = 1, it may be defined that the encoder and the decoder use only the transform skip of a block whose size of one side is less than or equal to 8. The maximum size information of the transform skip block may be expressed as the maximum number of pixels rather than the maximum size of one side of the block.

5) Alternatively, when mts_enabled = 0, information on the maximum size of the transform skip may be transmitted. For example, information for allowing the maximum size of the transform skip to 16 may be transmitted in a high level syntax. In this case, if the size of the block side is 16 or more, ts_enabled = 0 is defined, so that the binarization table of "unified transform type signaling" can be effectively applied. Alternatively, the maximum size information of the transform skip block may be expressed as the maximum number of pixels instead of the maximum size of one side of the block.

In addition, in encoding the syntax element tu_mts_idx, the context table may be determined according to the size of the block, the width-height ratio of the block, whether intra or inter prediction of the block, and whether to skip surrounding transforms. For example, when the block size is less than 8x8, the context table of index 0 can be used, and when the size of the block is larger than 8x8, the context table of index 1 can be used. As another example, when the width-height ratio of a block is 1, the context table of index 0 can be used, and if it is smaller or larger than that, the context table of index 1 can be used. As another example, when the prediction mode of the current block is an intra mode, index 0 context table may be used, and when an inter mode is an inter mode, index 1 context table may be used for encoding. The number of context tables and tables can be variously defined based on probability and distribution, and the present invention is not limited to the number of specific context tables and context tables.

For example, according to an embodiment of the present invention, a syntax including the following syntax elements may be configured/encoded and signaled to the decoding apparatus 200 through a bitstream, and the decoding apparatus 200 ordered the syntax elements. Can be parsed/decoded according to. What each syntax element specifies is indicated in the relevant semantics.

The picture parameter set RBSP syntax may be as shown in Table 11 below.

In Table 11, transform_skip_enabled_flag indicates whether it is possible to omit transform for residual blocks in a picture. In one embodiment, if transform_skip_enabled_flag is 0, tu_mts_idx does not indicate whether to omit the transform for the residual block coding syntax.

log2_transform_skip_max_size_minus2 represents the maximum block size to which the transform skip can be applied. If transform_skip_enabled_flag is 0 or does not exist, the value of log2_transform_skip_max_size_minus2 is deduced as 0.

The variable MaxTsSize indicating the maximum transform skip size may be set to 1 << (log2_transform_skip_max_size_minus2 + 2).

Hereinafter, a method for applying a transform skip to a luminance and color difference block according to an embodiment of the present specification will be described. The following embodiments provide a method for determining whether to apply a transform skip to a luminance and color difference block and to apply the transform skip.

Example 1

According to an embodiment of the present specification, a transform skip may be applied to a color difference block, and in order to determine whether to apply a transform skip to a color difference block, a transform skip flag indicating whether a transform skip is applied to the color difference block is used. I can. However, a condition for applying the transform skip to the color difference block may be set, and the transform skip condition of the color difference block may be determined based on the size of the current block and the tree type of the current block. That is, the size of the current block must be smaller than or equal to the reference size in consideration of coding complexity, and the tree type (single tree or dual tree chroma) of the current block is a condition for determining whether the current block corresponds to a color difference block. It may be determined whether it corresponds to a block. Here, the reference size may be set equal to the conversion skip condition of the luminance block. For example, the reference size may be determined based on the transform skip maximum size information, and the transform skip maximum size information is commonly applied to luminance and color difference components. Since the reference size for applying the transform skip to the luminance block and the chrominance block is set identically, the encoder does not need to separately transmit information indicating the reference size for the luminance block and the chrominance block, and single information (transform skip maximum The signaling overhead can be reduced by transmitting the size information) to the decoder.

Example 2

As described above, the transform skip and the transcoding related index may be integrated and managed (tu_mts_idx), which may be used only for the luminance block. That is, in the chrominance block, there are no indexes related to transform skip and transcoding, which means that transform skip is not used in the chrominance block.

Transform skips are often used in text and graphics with motion (TGM) images, and result in a lot of improvement in compression performance. Accordingly, this embodiment proposes a method of adaptively applying a transform skip to a color difference block.

Transformation skip may be adaptively applied to the chrominance block according to whether the luminance block co-located to the current chrominance block is skipped. That is, when the luminance block corresponding to the current color difference block is applied with the transform skip, the transform skip is applied to the color difference block in the same manner. In general, a luminance image and a chrominance image have high correlation characteristics. Therefore, when a transformation skip is applied to a luminance block, applying the transformation skip to the corresponding chrominance block is probabilistically improving coding efficiency. It helps. In the case of adaptively applying the transformation skip to the color difference block according to whether or not the luminance block is skipped, as in this embodiment, since there is no need to transmit additional information on whether to apply the transformation skip, the application of the transformation skip to the color difference block is more efficient. I can.

This method can be applied when the tile group type is I, P, and B, and when the tile group type is I, it can be applied regardless of whether or not dual tree is applied. That is, when the dual tree as shown in FIG. 26 is applied, the conversion of the luminance block corresponding to the center position (CR) of the luminance block corresponding to the current color difference block (same as the luminance mode derivation method in the prediction DM mode in the color difference screen) Transformation skip is adaptively applied to the current color difference block through the presence or absence of skip application. In addition to the CR position, it is possible to use whether or not to apply the transform skip of the upper left position (TL), the upper right position (TR), the lower left position (BL), and the lower right position (BR) in the luminance block.

In addition, when determining whether to skip conversion of a color difference block, when the luminance block corresponding to the current color difference block uses the conversion skip, a conversion skip flag indicating whether to skip conversion of the current color difference block may be transmitted. That is, when determining whether to apply the transform skip to the current color difference block, the encoder determines whether to apply the transform skip to the color difference block through calculation of the RD cost, and transmits this information (transform skip flag) to the decoder. The transform skip flag may be encoded/decoded through CABAC, and by-pass coding or context model may be used.

When determining whether to skip conversion of the current color difference block, the encoder/decoder may determine whether to apply it only when the size of the current color difference block is 32x32 or less, such as a luminance block. Also, the encoder/decoder may determine whether to apply the transform skip according to the color format of the current block. For example, when the current color format is 4:2:0, the encoder/decoder may determine whether to apply the color difference block only when the size of the color difference block is 16x16 or less. Also, the encoder/decoder may determine whether to apply the transform skip according to the type of the region to which the current block belongs (eg, picture, slice, tile group). For example, when the type of the current tile group is I, the encoder/decoder may determine whether or not the transform skip is applied only when the block size is 32x32 or 16x16 or less. If the current tile group type is P or B, and the block size is less than 16x16 (if it is 8x8, 4x4, or 2x2 or less), it may be determined whether or not transform skip is applied. Blocks that have not undergone transcoding through the above method may be encoded and decoded through the same method as before.

Example 3

In addition, when determining whether to skip the transform of the current color difference block, the encoder/decoder may adaptively determine whether or not to apply the transform skip according to the intra prediction mode of the color difference block. That is, the luminance block corresponding to the current chrominance block uses the transform skip method, and the intra prediction mode of the chrominance block is the DM mode (direct mode, a method of using the luminance block's intra prediction mode) or CCLM (cross-component linear model). Only in the mode, the transform skip can be applied to the current color difference block. When the intra prediction mode of the current color difference block is the DM mode or the CCLM mode, it means that the correlation with the luminance block is high, and therefore, using information of the luminance block may help improve coding efficiency with a higher probability. This method may be used only when the intra prediction mode of the current color difference block is the DM mode, or may be used only when the CCLM mode is used. Also, this method is applicable when the current luminance block and color difference block are in the intra prediction mode. If the current color difference block is not in the intra prediction mode, the transform skip method is not used, or if the corresponding luminance block uses transform skip as described above, the transform skip may be applied to the color difference block.

In addition, when determining whether to skip transformation of a color difference block, a transformation skip flag indicating whether to skip transformation of a current color difference block may be transmitted. That is, when determining whether to apply the transform skip to the current color difference block, the encoder determines whether to apply the transform skip to the color difference block through calculation of the RD cost, and transmits this information (transform skip flag) to the decoder. The transform skip flag may be encoded/decoded through CABAC, and by-pass coding or context model may be used.

Example 4

As described above, the transform skip and transform encoding related indexes may be integrated and managed (tu_mts_idx), but this may be used only for a luminance block, and is used when the block size is less than or equal to 32x32. (Same as MTS conditions)

The condition for transform skip may be applied to all I, P, and B tile group types, and a flag for transform skip (transform skip flag) may be transmitted when the P/B tile group type is smaller than 32x32. However, since the P/B tile group type has a very small residual signal than the I tile group type, the conversion skip efficiency decreases.

Accordingly, this embodiment proposes a method of more efficiently transmitting transform skip information (transform skip flag) by changing a block size condition for transform skip for a block belonging to a P/B tile group type. That is, as shown in Tables 12 to 14, whether or not to apply the transform skip may be determined according to the tile group type.

In addition, even in the B/P tile group type, conditions may be different according to the prediction mode (Intra or Inter) of the current block. That is, as shown in Tables 15 to 19, whether or not transform skip is applied may be determined according to a tile group tile and a block type (intra or inter).

When the transform skip is not applied to the current block size through the method proposed in the present embodiment, information related to transform skip (eg, transform skip flag) is not transmitted. In addition, the encoder/decoder may perform encoding and decoding through the same method as before for a block for which transcoding is determined through the method described above.

Bitstream

The encoded information (eg, encoded video/video information) derived by the encoding apparatus 100 based on the above-described embodiments of the present specification may be output in a bitstream form. The encoded information may be transmitted or stored in a bitstream form in units of network abstraction layer (NAL) units. The bitstream may be transmitted over a network or may be stored in a non-transitory digital storage medium. In addition, as described above, the bitstream is not directly transmitted from the encoding device 100 to the decoding device 200, but may be provided with a streaming/download service through an external server (eg, a content streaming server). Here, the network may include a broadcasting network and/or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD.

27 is an example of a flowchart for encoding a video signal according to an embodiment of the present specification. The operations of FIG. 27 may be performed by the encoding device 100 or the video signal processing device 500.

In step S2710, the encoder checks a condition for skipping a transform based on a dual tree type of the current block and a size of the current block with respect to the current block including data corresponding to a chroma component.

In one embodiment, when the size of the current block is smaller than the maximum transform skip size (MaxTsSize) and the dual tree type of the current block corresponds to the color difference dual tree (DUAL_TREE_CHROMA), the condition for the transform skip is satisfied. Can be determined to be.

In an embodiment, the transform skip maximum size (MaxTsSize) may be determined based on transform skip maximum size information (eg, log2_transform_skip_max_size_minus2). For example, the maximum transform skip size (MaxTsSize) may be set to 1 << (log2_transform_skip_max_size_minus2 + 2).

Here, the transform skip maximum size information may be transmitted to the decoder as a syntax element for an upper layer component including the current block. For example, the transform skip maximum size information may be transmitted as a syntax element for a picture as shown in Table 11, and may also be transmitted as a syntax element for a sequence.

In an embodiment, the maximum transform skip size may be determined based on a type of a tile group to which the current block belongs. For example, as shown in Tables 12 to 19, the maximum transform skip size may be set according to whether the current tile group is an I tile group or a B/P tile group.

In an embodiment, if the current tile group corresponds to the I-tile group, the maximum transform skip size is determined as the first reference size, and if the current tile group corresponds to the B-tile group or the P-tile group, the transform skip maximum The size may be determined as the second reference size. Here, the first reference size is a value larger than the second reference size. That is, the maximum transform skip size for the tile group corresponding to the B-tile group or the P-tile group may be set to be relatively smaller than that of the I-tile group.

In an embodiment, the maximum transform skip size may be determined based on a type of a tile group to which the current block belongs and a prediction type of the current block. Here, the prediction type includes either an intra prediction type or an inter prediction type. For example, as shown in Tables 15 to 19, the maximum transform skip size may be set based on the prediction type (Intra/Inter) of the current block together with the type of the current tile group.

In step S2720, the encoder determines whether to apply the transform skip to the current block when the condition for transform skip is satisfied. When the condition for skipping the transform is satisfied, the encoder may determine whether to skip the transform through comparison between the case of performing the transform and the case of not performing the transform.

In step S2730, the encoder encodes a transform skip flag (eg, transform_skip_flag) for the current block. For example, when a transform is applied to the current block, the transform skip flag may be encoded as 0, and if the transform is not applied to the current block, the transform skip flag may be encoded as 1. As another example, the encoder may indicate whether to skip transform using the MTS index (tu_mts_idx).

28 is an example of a flowchart for decoding a video signal according to an embodiment of the present specification. The operations of FIG. 28 may be performed by the decoding apparatus 200 or the video signal processing apparatus 500.

In step S2810, the decoder checks a condition for skipping a transform based on a dual tree type of the current block and a size of the current block with respect to the current block including data corresponding to a chroma component.

In one embodiment, when the size of the current block is smaller than the maximum transform skip size (MaxTsSize), and the dual tree type of the current block corresponds to the color difference dual tree (DUAL_TREE_CHROMA), the condition for the transform skip is satisfied. Can be determined to be.

In an embodiment, the transform skip maximum size may be determined based on transform skip maximum size information commonly applied to luminance and color difference components.

Here, the transform skip maximum size information may be transmitted from the encoder as a syntax element for an upper layer component including the current block. For example, the transform skip maximum size information may be transmitted as a syntax element for a picture as shown in Table 11, and may also be transmitted as a syntax element for a sequence.

In step S2820, when the condition for the transform skip is satisfied, the decoder acquires (parses) a transform skip flag (eg, transform_skip_flag) of the current block. In step S2830, the decoder performs residual coding on the current block based on the transform skip flag. For example, if the transform skip flag is 0, an inverse transform may be applied to the current block, and a residual signal derived through the inverse transform may be decoded. If the transform skip flag is 1, decoding of the residual signal of the current block may be performed without applying an inverse transform to the current block. In step S2840, the decoder generates a residual sample of the current block through residual coding.

As described with reference to FIG. 5, the video signal processing apparatus 500 according to the exemplary embodiment of the present specification includes a memory 520 for storing a video signal, and a processor 510 coupled to the memory 520. can do.

For encoding of a video signal, the processor 510 provides for a current block including data corresponding to a chroma component, based on a dual tree type of the current block and a size of the current block. The transform skip condition is checked, and if the transform skip condition is satisfied, it is determined whether to apply the transform skip to the current block, and the transform skip flag for the current block is encoded.

In one embodiment, when the size of the current block is smaller than the maximum transform skip size and the dual tree type of the current block corresponds to a color difference dual tree, the processor 510 determines that the condition for the transform skip is satisfied. I can.

In an embodiment, the transform skip maximum size is determined based on transform skip maximum size information, and the transform skip maximum size information may be transmitted to a decoder as a syntax element of an upper layer component including the current block. .

In an embodiment, if the tile group corresponds to an I-tile group, the transform skip maximum size is determined as a first reference size, and if the tile group corresponds to a B-tile group or a P-tile group, the transform skip maximum The size is determined as a second reference size, and the first reference size may be a value larger than the second reference size.

In one embodiment, the transform skip maximum size is determined based on a type of a tile group to which the current block belongs and a prediction type of the current block, and the prediction type is an intra prediction type or an inter prediction type. It can contain one.

For decoding a video signal, the processor 510 performs a transform skip for a current block including data corresponding to a chroma component based on a dual tree type of the current block and a size of the current block. When a condition is checked and the condition for the transform skip is satisfied, a transform skip flag of the current block is obtained, residual coding for the current block is performed based on the transform skip flag, and the residual coding is performed. Through this, a residual sample of the current block is generated.

Further, the processing method to which the present invention is applied may be produced in the form of a program executed by a computer, and may be stored in a computer-readable recording medium. Multimedia data having a data structure according to the present invention can also be stored in a computer-readable recording medium. The computer-readable recording medium includes all kinds of storage devices and distributed storage devices in which computer-readable data is stored. The computer-readable recording medium includes, for example, Blu-ray disk (BD), universal serial bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical It may include a data storage device. Further, the computer-readable recording medium includes media implemented in the form of a carrier wave (for example, transmission through the Internet). In addition, the bitstream generated by the encoding method may be stored in a computer-readable recording medium or transmitted through a wired or wireless communication network.

In addition, an embodiment of the present invention may be implemented as a computer program product using a program code, and the program code may be executed in a computer according to the embodiment of the present invention. The program code may be stored on a carrier readable by a computer.

A non-transitory computer-readable medium according to an embodiment of the present specification stores one or more instructions executed by one or more processors. The one or more instructions are, for encoding a video signal, for a current block including data corresponding to a chroma component, a dual tree type of the current block and a size of the current block. A video signal to determine whether to apply the transform skip to the current block, and to encode the transform skip flag for the current block when the condition for the transform skip is checked based on the condition for the transform skip is satisfied. It controls the processing device 500 (or the encoding device 100).

In one embodiment, the one or more instructions indicate that the condition for the transform skip is satisfied when the size of the current block is smaller than the maximum transform skip size and the dual tree type of the current block corresponds to the color difference dual tree. Can be set to determine.

The one or more commands are transform skipped based on the dual tree type of the current block and the size of the current block for a current block including data corresponding to a chroma component for decoding a video signal. If the condition for is checked and the condition for the transform skip is satisfied, a transform skip flag of the current block is obtained, residual coding for the current block is performed based on the transform skip flag, and the residual The video signal processing apparatus 500 (or encoding apparatus 100) is controlled to generate a residual sample of the current block through coding.

In one embodiment, the one or more instructions indicate that the condition for the transform skip is satisfied when the size of the current block is smaller than the maximum transform skip size and the dual tree type of the current block corresponds to the color difference dual tree. You can decide.

As described above, the embodiments described in the present invention may be implemented and performed on a processor, microprocessor, controller, or chip. For example, the functional units illustrated in each drawing may be implemented and executed on a computer, processor, microprocessor, controller, or chip.

In addition, the decoder and encoder to which the present invention is applied include a multimedia broadcasting transmission/reception device, a mobile communication terminal, a home cinema video device, a digital cinema video device, a surveillance camera, a video chat device, a real-time communication device such as video communication, a mobile streaming device, Storage media, camcorders, video-on-demand (VoD) service providers, OTT video (Over the top video) devices, Internet streaming service providers, three-dimensional (3D) video devices, video telephony video devices, and medical video devices. And can be used to process video signals or data signals. For example, an OTT video (Over the top video) device may include a game console, a Blu-ray player, an Internet-connected TV, a home theater system, a smartphone, a tablet PC, and a digital video recorder (DVR).

The embodiments described above are those in which components and features of the present invention are combined in a predetermined form. Each component or feature should be considered optional unless explicitly stated otherwise. Each component or feature may be implemented in a form that is not combined with other components or features. In addition, it is also possible to constitute an embodiment of the present invention by combining some components and/or features. The order of operations described in the embodiments of the present invention may be changed. Some configurations or features of one embodiment may be included in other embodiments, or may be replaced with corresponding configurations or features of other embodiments. It is obvious that the embodiments may be configured by combining claims that do not have an explicit citation relationship in the claims or may be included as new claims by amendment after filing.

The embodiment according to the present invention may be implemented by various means, for example, hardware, firmware, software, or a combination thereof. In the case of implementation by hardware, an embodiment of the present invention is one or more ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs ( field programmable gate arrays), processors, controllers, microcontrollers, microprocessors, etc.

In the case of implementation by firmware or software, an embodiment of the present invention may be implemented in the form of a module, procedure, or function that performs the functions or operations described above. The software code can be stored in a memory and driven by a processor. The memory may be located inside or outside the processor, and may exchange data with the processor through various known means.

It is obvious to those skilled in the art that the present invention can be embodied in other specific forms without departing from the essential features of the present invention. Therefore, the above detailed description should not be construed as restrictive in all respects, but should be considered as illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

Above, the above-described preferred embodiments of the present invention are disclosed for the purpose of illustration, and those skilled in the art improve and change various other embodiments within the technical spirit and scope of the present invention disclosed in the appended claims below. , Substitution or addition may be possible.

Claims

As a method for decoding a video signal,

For a current block including data corresponding to a chroma component, checking a condition for skipping transformation based on a tree type of the current block and a size of the current block; and

When the condition for the transform skip is satisfied, obtaining a transform skip flag of the current block; and

Performing residual coding on the current block based on the transform skip flag,

And generating a residual sample of the current block through the residual coding.
The method of claim 1,

Checking the conditions for the conversion skip,

If the size of the current block is smaller than the maximum transform skip size and the tree type of the current block corresponds to a color difference dual tree or a single tree, determining that a condition for the transform skip is satisfied. Way.
The method of claim 2,

The maximum conversion skip size is,

It is determined based on the transform skip maximum size information transmitted from the encoder,

The conversion skip maximum size information,

And transmitted as a syntax element for an upper layer component including the current block.
The method of claim 3,

The maximum conversion skip size is,

A method, characterized in that it is determined based on information on a maximum transform skip size commonly applied to luminance and color difference components.
The method of claim 2,

The maximum conversion skip size is,

And determining based on a type of a tile group to which the current block belongs.
The method of claim 5,

If the tile group corresponds to an I-tile group, the maximum transform skip size is determined as a first reference size,

If the tile group corresponds to a B-tile group or a P-tile group, the maximum transform skip size is determined as a second reference size,

The method according to claim 1, wherein the first reference size is larger than the second reference size.
The method of claim 5,

The maximum conversion skip size is,

It is determined based on a type of a tile group to which the current block belongs and a prediction type of the current block,

The method of claim 1, wherein the prediction type includes one of an intra prediction type or an inter prediction type.
A method for encoding a video signal, comprising:

For a current block including data corresponding to a chroma component, checking a condition for skipping transformation based on a tree type of the current block and a size of the current block; and

If the condition for the transform skip is satisfied, determining whether to apply the transform skip to the current block;

And encoding a transform skip flag for the current block.
The method of claim 8,

Checking the conditions for the conversion skip,

If the size of the current block is smaller than the maximum transform skip size and the tree type of the current block corresponds to a color difference dual tree or a single tree, determining that a condition for the transform skip is satisfied. Way.
The method of claim 9,

The maximum conversion skip size is,

It is determined based on the transform skip maximum size information,

The conversion skip maximum size information,

And transmitted to a decoder as a syntax element for an upper layer component including the current block.
The method of claim 10,

The maximum conversion skip size is,

And determining based on a type of a tile group to which the current block belongs.
The method of claim 11,

If the tile group corresponds to an I-tile group, the maximum transform skip size is determined as a first reference size,

If the tile group corresponds to a B-tile group or a P-tile group, the maximum transform skip size is determined as a second reference size,

The method according to claim 1, wherein the first reference size is larger than the second reference size.
The method of claim 11,

The maximum conversion skip size is,

It is determined based on a type of a tile group to which the current block belongs and a prediction type of the current block,

The method of claim 1, wherein the prediction type includes one of an intra prediction type or an inter prediction type.
An apparatus for decoding a video signal, comprising:

A memory for storing the video signal;

A processor coupled to the memory and processing the video signal,

The processor,

For a current block including data corresponding to a chroma component, a condition for skipping transformation is checked based on a tree type of the current block and a size of the current block,

When the condition for the transformation skip is satisfied, a transformation skip flag of the current block is obtained,

Perform residual coding for the current block based on the transform skip flag,

And generating a residual sample of the current block through the residual coding.
An apparatus for encoding a video signal, comprising:

A memory for storing the video signal;

A processor coupled to the memory and processing the video signal,

The processor,

For a current block including data corresponding to a chroma component, a condition for skipping transformation is checked based on a tree type of the current block and a size of the current block,

When the condition for the transform skip is satisfied, it is determined whether to apply the transform skip to the current block,

And encoding a transform skip flag for the current block.