RU2801265C1 - Method and equipment for image encoding/decoding to perform in-loop filtration based on sub-frame structure and method for transmitting a bit stream - Google Patents

Abstract

FIELD: image encoding.

SUBSTANCE: invention is related to image encoding/decoding method and equipment for performing in-loop filtering based on a subframe structure. The image decoding method implemented by the image decoding equipment comprises decoding the first flag indicating whether or not in-loop filtering can be performed across a subframe boundary from a bitstream, and performing in-loop filtering at a current block boundary based on the first flag. Based on the boundary of the current block coinciding with the boundary of the current subframe including the current block, in-loop filtering on the boundary of the current block is performed only when in-loop filtering is available for both the boundary of the current subframe and the boundary of an adjacent subframe adjacent to the current subframe.

EFFECT: increasing the efficiency of image encoding/decoding.

14 cl, 46 dwg, 2 tbl

Description

The field of technology to which the invention belongs

[1] The present disclosure relates to an image encoding/decoding method and equipment for performing in-loop filtering based on a subframe structure and a bitstream transmission method, and more specifically, an image encoding/decoding method and equipment for performing in-loop filtering across a subframe boundary and to a method for transmitting a bitstream generated by the image encoding method/equipment of the present disclosure.

State of the art

[2] Recently, the demand for high-quality high definition images, such as high definition (HD) images and ultra high definition (UHD) images, has been growing in various fields of technology. As the resolution and quality of image data increases, the amount of information or bits to be transmitted increases relatively compared to existing image data. An increase in the amount of information or bits to be transmitted results in an increase in transmission costs and storage costs.

[3] Accordingly, there is a need for a high-performance image compression technology for efficiently transmitting, storing, and reproducing information regarding high-quality, high-resolution images.

The essence of the invention

Technical task

[4] The purpose of the present disclosure is to provide an image encoding/decoding method and equipment with improved encoding/decoding efficiency.

[5] Another object of the present disclosure is to provide an image encoding/decoding method and equipment for performing in-loop filtering across a current block boundary.

[6] Another object of the present disclosure is to provide an image encoding/decoding method and equipment for performing in-loop filtering based on a subframe structure.

[7] Another object of the present disclosure is to provide an image encoding/decoding method and equipment for performing in-loop filtering considering an adjacent subframe adjacent to a current block boundary.

[8] Another purpose of the present disclosure is to provide a method for transmitting a bit stream generated by an image encoding method or equipment according to the present disclosure.

[9] Another purpose of the present disclosure is to provide a recording medium storing a bit stream generated by an image encoding method or equipment according to the present disclosure.

[10] Another purpose of the present disclosure is to provide a recording medium storing a bitstream received, decoded, and used to reconstruct an image by image decoding equipment according to the present disclosure.

[11] The technical problems solved by the present disclosure are not limited to the above technical problems, and other technical problems that are not described herein should become apparent to those skilled in the art from the description below.

Technical solution

[12] An image decoding method performed by image decoding equipment according to an aspect of the present disclosure may comprise decoding a first flag indicating whether or not in-loop filtering can be performed across a subframe boundary, from a bitstream, and performing in-loop filtering at a current block boundary, on the basis of the first flag. Based on the boundary of the current block, which is the boundary of the current subframe including the current block, in-loop filtering at the boundary of the current block may not be performed, based on the first flag for the current subframe indicating that in-loop filtering cannot be performed across the boundary of the current subframe, or a first flag for an adjacent subframe adjacent to the current block boundary, indicating that in-loop filtering cannot be performed across the boundary of the adjacent subframe.

[13] The image decoding equipment according to an aspect of the present disclosure may comprise a memory and at least one processor. At least one processor can decode a first flag indicating whether or not in-loop filtering can be performed across a sub-frame boundary from the bit stream, and perform in-loop filtering at a current block boundary based on the first flag. Based on the boundary of the current block, which is the boundary of the current subframe including the current block, in-loop filtering at the boundary of the current block may not be performed, based on the first flag for the current subframe indicating that in-loop filtering cannot be performed across the boundary of the current subframe, or a first flag for an adjacent subframe adjacent to the current block boundary, indicating that in-loop filtering cannot be performed across the boundary of the adjacent subframe.

[14] The image encoding method performed by the image encoding equipment according to an aspect of the present disclosure may comprise determining whether or not in-loop filtering can be performed across a subframe boundary, and performing in-loop filtering at a current block boundary based on the result of the determination. Based on the boundary of the current block, which is the boundary of the current subframe including the current block, in-loop filtering at the boundary of the current block may not be performed, based on the first flag for the current subframe indicating that in-loop filtering cannot be performed across the boundary of the current subframe, or a first flag for an adjacent subframe adjacent to the current block boundary, indicating that in-loop filtering cannot be performed across the boundary of the adjacent subframe.

[15] In addition, the transmission method according to another aspect of the present disclosure may transmit a bit stream generated by the image encoding equipment or the image encoding method of the present disclosure.

[16] In addition, a computer-readable recording medium according to another aspect of the present disclosure may store a bitstream generated by the image encoding equipment or the image encoding method of the present disclosure.

[17] The features briefly summarized above with respect to the present disclosure are merely exemplary aspects of the following detailed description of the present disclosure and do not limit the scope of the present disclosure.

Benefits of the Invention

[18] According to the present disclosure, it is possible to provide an image encoding/decoding method and equipment with improved encoding/decoding efficiency.

[19] In addition, according to the present disclosure, an image encoding/decoding method and equipment for performing improved in-loop filtering can be provided.

[20] In addition, according to the present disclosure, an image encoding/decoding method and equipment for performing in-loop filtering based on a subframe structure can be provided.

[21] In addition, according to the present disclosure, it is possible to provide an image encoding/decoding method and equipment for performing in-loop filtering considering an adjacent subframe adjacent to a current block boundary.

[22] In addition, according to the present disclosure, it is possible to provide a method for transmitting a bit stream generated by the image encoding method or equipment according to the present disclosure.

[23] In addition, according to the present disclosure, it is possible to provide a recording medium storing a bit stream generated by the image encoding method or equipment according to the present disclosure.

[24] In addition, according to the present disclosure, it is possible to provide a recording medium storing a bitstream received, decoded, and used to reconstruct an image by the image decoding equipment according to the present disclosure.

[25] Those skilled in the art should appreciate that the benefits that can be achieved through the present disclosure are not limited to those specifically described above, and other benefits of the present disclosure should be more clearly understood from the detailed description.

Brief description of the drawings

[26] FIG. 1 is a view schematically showing a video coding system to which an embodiment of the present disclosure is applicable.

[27] FIG. 2 is a view schematically showing image encoding equipment to which an embodiment of the present disclosure is applicable.

[28] FIG. 3 is a view schematically showing image decoding equipment to which an embodiment of the present disclosure is applicable.

[29] FIG. 4 is a view showing an image segmentation structure according to the embodiment.

[30] FIG. 5a-5c are views illustrating examples of frame segmentation.

[31] FIG. 6 is a view showing an embodiment of a block segmentation type according to the multi-type tree structure.

[32] FIG. 7 is a view showing a signaling mechanism of blocking information in a quadtree with a nested multi-type tree structure according to the present disclosure.

[33] FIG. 8 is a view showing an example in which a CTU is segmented into multiple CUs by applying a multi-type tree after applying the quadtree.

[34] FIG. 9 shows redundant splitting patterns that can occur in binary tree splitting and ternary tree splitting.

[35] FIG. 10 is a view illustrating examples in which CU partition is limited.

[36] FIG. 11 is a flowchart illustrating a video/image coding method based on inter-prediction.

[37] FIG. 12 is a view illustrating the configuration of the inter prediction unit 180 according to the present disclosure.

[38] FIG. 13 is a flowchart illustrating a video/image decoding method based on inter-prediction.

[39] FIG. 14 is a view illustrating the configuration of the inter prediction unit 260 according to the present disclosure.

[40] FIG. 15 is a view illustrating a syntax for signaling a syntax element for subframes in an SPS.

[41] FIG. 16 is a view illustrating an example of an algorithm for extracting a predetermined variable such as SubPicTop.

[42] FIG. 17 is a view illustrating an image encoding method using a subframe by image encoding equipment according to an embodiment of the present disclosure.

[43] FIG. 18 is a view illustrating a method for decoding an image using a subframe by the image decoding equipment according to the embodiment.

[44] FIG. 19 is a view illustrating a method for applying a deblocking filter.

[45] FIG. 20 is a view illustrating a reconstructed sample to which a deblocking filter is applied.

[46] FIG. 21 and 22 are views illustrating examples of reducing an area treated as a frame in a subframe.

[47] FIG. 23 and 24 are views illustrating examples in which a subframe decoding error occurs.

[48] FIG. 25 is a view illustrating the SPS syntax for signaling a syntax element for subframes according to an embodiment of the present disclosure.

[49] FIG. 26 is a view illustrating an example of the luminance sample filtering process based on the SPS syntax of FIG. 25.

[50] FIG. 27 is a view illustrating another example of the luminance sample filtering process based on the SPS syntax of FIG. 25.

[51] FIG. 28 is a view illustrating an example of a chrominance sample filtering process based on the SPS syntax of FIG. 25.

[52] FIG. 29 is a flowchart illustrating a filtering method based on a subframe structure according to an embodiment of the present disclosure.

[53] FIG. 30 is a view illustrating a cross-filtering process according to an embodiment of the present disclosure.

[54] FIG. 31 and 32 are views illustrating a cross-filtering process for luminance samples according to an embodiment of the present disclosure.

[55] FIG. 33 is a view illustrating a cross-filtering process for chrominance samples according to an embodiment of the present disclosure.

[56] FIG. 34 is a flowchart illustrating a filtering method based on a subframe structure according to an embodiment of the present disclosure.

[57] FIG. 35 is a view illustrating a cross-filtering process according to an embodiment of the present disclosure.

[58] FIG. 36 and 37 are views illustrating a cross-filtering process for luminance samples according to an embodiment of the present disclosure.

[59] FIG. 38 is a view illustrating a cross-filtering process for chrominance samples according to an embodiment of the present disclosure.

[60] FIG. 39 is a view illustrating a cross-filtering process according to an embodiment of the present disclosure.

[61] FIG. 40 is a flowchart illustrating a filtering method based on a subframe structure according to an embodiment of the present disclosure.

[62] FIG. 41 and 42 are views illustrating a cross-filtering process for luminance samples according to an embodiment of the present disclosure.

[63] FIG. 43 is a view illustrating a cross-filtering process for chrominance samples according to an embodiment of the present disclosure.

[64] FIG. 44 is a view showing a content streaming system to which an embodiment of the present disclosure is applicable.

Optimal Mode for Carrying Out the Invention

[65] Hereinafter, embodiments of the present disclosure are described in detail with reference to the accompanying drawings so that they can be easily implemented by those skilled in the art. However, the present disclosure may be implemented in all manner of forms and is not limited to the embodiments described herein.

[66] In describing the present disclosure, if it is determined that a detailed description of an associated known function or construct would unduly ambiguize the scope of the present disclosure, the detailed description thereof is omitted. In the drawings, parts not related to the description of the present disclosure are omitted and like reference numbers are attached to like parts.

[67] In the present disclosure, when a component is "connected", "coupled", or "linked" to another component, this may include not only a direct connection-based relationship, but also an indirect connection-based relationship in which an intermediate component is present. In addition, when a component "includes" or "has" other components, it means that other components may be included in addition to, rather than excluding, other components, unless otherwise noted.

[68] In the present disclosure, the terms "first", "second", etc. may be used only for the purpose of distinguishing one component from other components and do not limit the order or importance of components unless otherwise noted. Accordingly, within the scope of this disclosure, the first component in one embodiment may be referred to as the "second component" in another embodiment, and similarly, the second component in one embodiment may be referred to as the "first component" in another embodiment.

[69] In the present disclosure, components that are different from each other are intended to clearly describe each feature and do not mean that the components are necessarily separated. Thus, multiple components may be integrated and implemented in a single hardware or software module, or a single component may be distributed and implemented in multiple hardware or software modules. Therefore, even if not stated otherwise, such embodiments in which components are integrated or a component is distributed are also included within the scope of this disclosure.

[70] In the present disclosure, the components described in various embodiments do not necessarily mean essential components, and some components may be optional components. Accordingly, an embodiment consisting of a subset of the components described in the embodiment is also included within the scope of the present disclosure. In addition, embodiments including other components, in addition to the components described in various embodiments, are included within the scope of the present disclosure.

[71] The present disclosure relates to image encoding and decoding, and the terms used in the present disclosure may have a general meaning widely used in the technical field to which the present disclosure belongs, if not defined for the first time in the present disclosure.

[72] In the present disclosure, "frame" generally means a unit representing one image in a particular time period, and a slice/tile is a coding unit constituting a frame, and one frame may be composed of one or more slices/ tiles. In addition, a slice/tile may include one or more coding tree units (CTUs).

[73] In the present disclosure, "pixel" or "pel" can mean the smallest unit constituting one frame (or image). In addition, "sample" can be used as a term corresponding to a pixel. A sample can generally represent a pixel or a pixel value, and can represent only a pixel/pixel value of the luma component, or only a pixel/pixel value of the chroma component.

[74] In the present disclosure, a "unit" may represent a basic unit of image processing. The unit may include at least one of a specific area of the frame and information associated with the area. The unit can be used interchangeably with terms such as "array of samples", "block" or "zone" in some cases. In general, an MxN block may include samples (or arrays of samples) or a set (or array) of transform coefficients of M columns and N rows.

[75] In the present disclosure, "current block" may mean one of "current coding block", "current coding unit", "target coding block", "target decoding block", or "target processing block". When prediction is performed, "current block" may mean "current prediction block" or "target prediction block". When transformation (inverse transformation)/quantization (dequantization) is performed, "current block" may mean "current transformation block" or "transformation target block". When filtering is performed, "current block" may mean "filtering target block".

[76] In addition, in the present disclosure, "current block" may mean a block including both a luma component block and a chrominance signal component block, or a "current block luminance signal block", unless explicitly specified in as a block of color signals. The chrominance component block of the current block may be expressed by including an explicit description of the luma component block, such as "luminance signal block" or "current luminance signal block". In addition, the chrominance component block of the current block may be explicitly expressed by including an explicit description of the chrominance component block such as "chroma block" or "current chrominance block".

[77] In the present disclosure, the terms "/" and "," are to be interpreted as indicating "and/or". For example, the expression "A/B" and "A, B" can mean "A and/or B". Additionally, "A/B/C" and "A/B/C" may mean "at least one of A, B and/or C".

[78] In the present disclosure, the term "or" is to be interpreted as indicating "and/or". For example, the expression "A or B" may contain 1) only "A", 2) only "B", and/or 3) "both A and B". In other words, in the present disclosure, the term "or" is to be interpreted as indicating "in addition or alternatively".

[79] Overview of the Video Coding System

[80] FIG. 1 is a view showing a video coding system to which an embodiment of the present disclosure is applicable.

[81] The video encoding system according to the embodiment may include encoding equipment 10 and decoding equipment 20. The encoding equipment 10 may deliver the encoded information or video and/or image data to the decoding equipment 20 in the form of a file or streaming via a digital storage medium or a network.

[82] The encoding equipment 10 according to the embodiment may include a video source generator 11, an encoding unit 12, and a transmitter 13. The decoding equipment 20 according to the embodiment may include a receiver 21, a decoding unit 22, and a rendering unit 23. The encoding module 12 may be referred to as a "video/image coding module", and the decoding module 22 may be referred to as a "video/image decoding module". The transmitter 13 may be included in the encoding module 12. The receiver 21 may be included in the module 22 decoding. The rendering module 23 may include a display, and the display may be configured as a separate device or an external component.

[83] The video source generator 11 can acquire video/image through a process of capturing, synthesizing, or generating video/image. The video source generator 11 may include a video/image capture device and/or a video/image generator. The video/image capture device may include, for example, one or more cameras, video/image archives including previously captured videos/images, and the like. The video/image generating apparatus may include, for example, computers, tablet computers, and smartphones, and may (electronically) generate video/images. For example, the virtual video/image may be generated via a computer or the like. In this case, the video/image capturing process may be replaced by a linked data generation process.

[84] The encoding unit 12 may encode the input video/image. The coding unit 12 may perform a series of procedures such as prediction, transformation, and quantization for compression and coding efficiency. The encoding unit 12 may output encoded data (encoded video/image information) in the form of a bit stream.

[85] The transmitter 13 may transmit the encoded information or video/image data output in the form of a bit stream to the receiver 21 of the decoding equipment 20 via a digital storage medium or a network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, and the like. The transmitter 13 may include an element for generating a media file via a predetermined file format and may include an element for transmission via a broadcast/communications network. The receiver 21 can extract/receive a bit stream from a storage medium or a network and transmit the bit stream to the decoding module 22 .

[86] The module 22 decoding can decode the video/image by performing a sequence of procedures such as dequantization, inverse transformation and prediction, corresponding to the operation of the module 12 encoding.

[87] The rendering unit 23 may render the decoded video/image. The rendered video/image can be displayed through the display.

[88] Overview of Image Coding Equipment

[89] FIG. 2 is a view schematically showing image encoding equipment to which an embodiment of the present disclosure is applicable.

[90] As shown in FIG. 2, the image encoding equipment 100 may include an image segmentation unit 110, a subtractor 115, a transform 120, a quantizer 130, a dequantizer 140, an inverse transform 150, an adder 155, a filter 160, a memory 170, an inter prediction unit 180, an intra prediction unit 185. and an entropy encoder 190. The inter prediction module 180 and the intra prediction module 185 may be collectively referred to as a "prediction module". Transformer 120, quantizer 130, dequantizer 140, and inverse transform 150 may be included in the residual processor. The residual processor may further include a subtractor 115.

[91] All or at least a portion of the plurality of components configuring the image encoding hardware 100 may be configured by a single hardware component (eg, encoder or processor) in some embodiments. In addition, the storage device 170 may include a decoded frame buffer (DPB) and may be configured by a digital storage medium.

[92] The image segmentation unit 110 may segment an input image (either a frame or a movie frame) input to the image encoding equipment 100 into more than one processing unit. For example, a processing unit may be referred to as a "coding unit (CU)". A coding unit may be obtained by recursively segmenting a coding tree unit (CTU) or a largest coding unit (LCU) according to a quadtree, binary tree, and ternary tree (QT/BT/TT) structure. For example, one coding unit may be segmented into multiple deeper coding units based on a quadtree structure, a binary tree structure, and/or a ternary structure. To segment a coding unit, a quadtree structure may be applied first, and subsequently a binary tree structure and/or a ternary structure may be applied. The encoding procedure according to the present disclosure may be performed based on a final encoding unit that is no longer segmented. The largest coding unit may be used as the final coding unit, or the larger depth coding unit obtained by segmenting the largest coding unit may be used as the final coding unit. Here, the encoding procedure may include a prediction, transformation, and restoration procedure, which is described below. As another example, the encoding procedure processing unit may be a prediction unit (PU) or a transformation unit (TU). The prediction unit and the transformation unit may be split or segmented from the final coding unit. The prediction unit may be a selective prediction unit, and the transform unit may be a unit for extracting a transform coefficient and/or a unit for extracting a residual signal from a transform coefficient.

[93] The prediction unit (inter prediction unit 180 or intra prediction unit 185) can perform prediction on a block to be processed (current block) and generate a prediction block including prediction samples for the current block. The prediction module may determine whether intra prediction or inter prediction is applied based on the current block or CU. The prediction module may generate various information related to the prediction of the current block and pass the generated information to the entropy encoder 190. The prediction information may be encoded in the entropy encoder 190 and output in the form of a bit stream.

[94] The intra prediction module 185 may predict the current block by referring to the samples in the current frame. The samples for a reference may be located in the surroundings of the current block, or may be spaced apart according to an intra prediction mode and/or an intra prediction technique. The intra prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The non-directional mode may include, for example, a DC mode and a planar mode. The directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes according to the granularity of the prediction direction. However, this is just an example, more or less directional prediction modes may be used depending on the setting. The intra prediction unit 185 may determine the prediction mode applied to the current block by using the prediction mode applied to the adjacent block.

[95] The inter prediction unit 180 may derive a predicted block for the current block based on a reference block (reference sample array) indicated by a motion vector for the reference frame. In this case, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the adjacent block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction direction information (L0 prediction, L1 prediction, bi-prediction, etc.). In the case of inter prediction, an adjacent block may include a spatial adjacent block present in the current frame and a temporal adjacent block present in the reference frame. The reference frame including the reference block and the reference frame including the temporal neighbor block may be the same or different. The temporary neighbor block may be referred to as a "collocated reference block", "collocated CU (colCU)", or the like. A reference frame including a temporal adjacent block may be referred to as a "collocated frame (colPic)". For example, inter prediction module 180 may configure a list of motion information candidates based on adjacent blocks and generate information indicating which candidate is used to derive the motion vector and/or the reference frame index of the current block. Inter prediction may be performed based on various prediction modes. For example, in the case of the skip mode and the merge mode, the inter prediction unit 180 may use the neighboring block motion information as the current block motion information. In the case of the skip mode, unlike the combine mode, the residual signal may not be transmitted. In the case of a motion vector prediction (MVP) mode, the motion vector of an adjacent block may be used as a motion vector predictor, and the motion vector of the current block may be signaled by encoding a motion vector difference and an indicator for the motion vector predictor. The motion vector difference may mean the difference between the motion vector of the current block and the motion vector predictor.

[96] The prediction module may generate a prediction signal based on various prediction methods and prediction technologies described below. For example, the prediction module may not only apply intra prediction or inter prediction, but also apply both intra prediction and inter prediction at the same time to predict the current block. A prediction method that simultaneously applies both intra prediction and inter prediction to predict the current block may be referred to as "combined inter and intra prediction (CIIP)". In addition, the prediction module may perform intra-block copy (IBC) to predict the current block. In-block copying can be used for image/video encoding of game content and the like, such as on-screen content coding (SCC). IBC is a method for predicting the current frame using a previously reconstructed reference block in the current frame at a location separated from the current block by a predetermined distance. When IBC is applied, the location of the reference block in the current frame may be encoded as a vector (block vector) corresponding to a predetermined distance. IBC essentially performs prediction in the current frame, but may be performed similar to inter-prediction in that the reference block is retrieved within the current frame. Thus, the IBC may use at least one of the inter-prediction technologies described in the present disclosure.

[97] The predictive signal generated by the prediction module may be used to generate a recovered signal or generate a residual signal. The subtractor 115 may generate a residual signal (residual block or residual sample array) by subtracting the prediction signal (predicted block or predictive sample array) output from the predictor from the input image signal (original block or original sample array). The generated residual signal may be transmitted to transducer 120.

[98] Converter 120 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technology may include at least one of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen-Loeve Transform (KLT), Graph Based Transform (GBT), or Conditionally Nonlinear Transform (CNT). ). Here, GBT means a transformation obtained from a graph when relationship information between pixels is represented by a graph. CNT means the transformation obtained based on the predictive signal generated using all previously reconstructed pixels. In addition, the transformation process may be applied to square pixel blocks having the same size, or may be applied to blocks having a variable size rather than square.

[99] The quantizer 130 may quantize the transform coefficients and pass them to the entropy encoder 190. The entropy encoder 190 may encode the quantized signal (information regarding the quantized transform coefficients) and output a bit stream. The information regarding the quantized transform coefficients may be referred to as "residual information". The quantizer 130 may repackage the quantized transform coefficients in block form into one-dimensional vector form based on the scan order of the coefficients, and generate information about the quantized transform coefficients based on the quantized transform coefficients in one-dimensional vector form.

[100] The entropy encoder 190 may implement various coding techniques such as exponential Golomb coding, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), and the like, for example. Entropy encoder 190 may encode information needed for video/image reconstruction other than quantized transform coefficients (eg, syntax element values, etc.) together or separately. Encoded information (eg, encoded video/image information) may be transmitted or stored in units of network abstraction layers (NALs) in the form of a bitstream. The video/image information may further include information regarding various parameter sets such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. The signaling information, the signaling information and/or the syntax elements described in the present disclosure may be encoded via the encoding procedure described above and included in the bit stream.

[101] The bit stream may be transmitted over a network or may be stored on a digital storage medium. The network may include a broadcast network and/or a communications network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, and the like. A transmitter (not shown) transmitting a signal output from the entropy encoder 190 and/or a storage module (not shown) storing the signal may be included as an internal/external member of the encoding equipment 100, and alternatively, a transmitter may be included in the entropy encoder 190 .

[102] The quantized transform coefficients output from quantizer 130 may be used to generate a residual signal. For example, the residual signal (residual block or residual samples) may be recovered by applying dequantization and inverse transform to the quantized transform coefficients via dequantizer 140 and inverse transform 150.

[103] The adder 155 adds the recovered residual signal with the predictive signal output from the inter prediction unit 180 or the intra prediction unit 185 so as to generate a recovered signal (frame recovered, block recovered, recovered sample array). If there is no remainder for a block to be processed, such as in a case in which the skip mode is applied, the predicted block may be used as the reconstructed block. The adder 155 may be referred to as a "recovery module" or "recovered block generator". The generated recovered signal may be used to intra-predict the next block to be processed in the current frame, and may be used to inter-predict the next frame through filtering as described below.

[104] The filter 160 can improve subjective/objective image quality by applying filtering to the reconstructed signal. For example, filter 160 may generate a modified reconstructed frame by applying various filtering methods to the reconstructed frame and store the modified reconstructed frame in memory 170, namely the DPB of memory 170. Various filtering methods may include, for example, filtering to deblock , filtering based on a sampled adaptive bias, an adaptive loop filter, a bilateral filter, and the like. The filter 160 may generate various filtering-related information and pass the generated information to the entropy encoder 190 as described below in the description of each filtering method. Filtering related information may be encoded by entropy encoder 190 and output in the form of a bitstream.

[105] The modified reconstructed frame transmitted to the memory 170 may be used as a reference frame in the inter prediction module 180. When inter prediction is applied via the image encoding equipment 100, prediction mismatch between the image encoding equipment 100 and the image decoding equipment can be eliminated, and encoding efficiency can be improved.

[106] The DPB of the memory 170 may store the modified reconstructed frame for use as a reference frame in the inter prediction module 180. The memory 170 may store block motion information from which motion information in the current frame is extracted (or encoded) and/or motion information of blocks in the frame that have already been recovered. The stored motion information may be transmitted to the inter-prediction unit 180 and used as spatial neighbor block motion information or temporal neighbor block motion information. The memory 170 may store the recovered samples of the recovered blocks in the current frame and may pass the recovered samples to the intra prediction module 185 .

[107] Overview of image decoding equipment

[108] FIG. 3 is a view schematically showing image decoding equipment to which an embodiment of the present disclosure is applicable.

[109] As shown in FIG. 3, the image decoding equipment 200 may include an entropy decoder 210, a dequantizer 220, an inverse transform 230, an adder 235, a filter 240, a memory 250, an inter prediction unit 260, and an intra prediction unit 265. The inter prediction module 260 and the intra prediction module 265 may be collectively referred to as a "prediction module". Dequantizer 220 and inverse converter 230 may be included in the residual processor.

[110] All or at least a portion of the plurality of components configuring the image decoding equipment 200 may be configured by a hardware component (eg, decoder or processor) according to the embodiment. In addition, the storage device 250 may include a decoded frame buffer (DPB) or may be configured by a digital storage medium.

[111] The image decoding equipment 200, which receives a bitstream including video/image information, can restore an image by performing a process corresponding to the process performed by the image encoding equipment 100 of FIG. 2. For example, the image decoding equipment 200 may perform decoding using the processing unit used in the image encoding equipment. Thus, the decoding processing unit, for example, may be an encoding unit. The coding unit may be obtained by segmenting the coding tree unit or the largest coding unit. The reconstructed image signal decoded and output by the decoding equipment 200 can be reproduced by the playback equipment.

[112] The image decoding equipment 200 may receive a signal output from the image encoding equipment of FIG. 2 in the form of a bitstream. The received signal may be decoded via entropy decoder 210. For example, entropy decoder 210 may parse the bit stream to extract information (eg, video/image information) needed for image recovery (or frame recovery). The video/image information may further include information regarding various parameter sets such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. The image decoding equipment may further decode the frame based on the parameter set information and/or general constraint information. Signaling/receiving information and/or syntax elements described in the present disclosure may be decoded by a decoding procedure and obtained from a bitstream. For example, entropy decoder 210 decodes information in the bitstream based on an encoding method such as Exponential Golomb, CAVLC, or CABAC encoding, and output syntax element values required for image reconstruction, and quantized transform coefficient values for the remainder. More specifically, the CABAC entropy decoding method may receive a bin corresponding to each syntax element in the bit stream, determine a context model using decoding target syntax element information, neighboring block decoding information, and decoding target block, or symbol/bin information decoded at the previous stage, and perform arithmetic decoding for the permission element by predicting the occurrence probability of the permission element according to the determined context model, and generating a symbol corresponding to the value of each syntax element. In this case, the CABAC entropy decoding method can update the context model by using the decoded symbol/bin information for the context model of the next symbol/bin after the context model is determined. Prediction related information from the information decoded by the entropy decoder 210 may be provided to a prediction unit (inter prediction unit 260 and intra prediction unit 265) and a residual value for which entropy decoding is performed in the entropy decoder 210, i. quantized transform coefficients and related parameter information may be input to the dequantizer 220. In addition, information regarding filtering from the information decoded by the entropy decoder 210 may be provided to the filter 240. Meanwhile, a receiver (not shown) for receiving a signal output from the equipment image encoding equipment may be further configured as an internal/external element of the image decoding equipment 200, or the receiver may be a component of the entropy decoder 210.

[113] Meanwhile, the image decoding equipment according to the present disclosure may be referred to as "video/image/frame decoding equipment". The image decoding equipment can be classified into an information decoder (video/image/frame information decoder) and a sampling decoder (video/image/frame samples decoder). The information decoder may include an entropy decoder 210. The selective decoder may include at least one of a dequantizer 220, an inverse transform 230, an adder 235, a filter 240, a memory 250, an inter prediction module 260, or an intra prediction module 265.

[114] The dequantizer 220 may dequantize the quantized transform coefficients and output the transform coefficients. The dequantizer 220 may recompose the quantized transform coefficients in the form of a two-dimensional block. In this case, the re-arranging may be performed based on the scan order of the coefficients performed in the image encoding equipment. The dequantizer 220 may perform dequantization on the quantized transform coefficients by using a quantization parameter (eg, quantization step size information) and obtain the transform coefficients.

[115] The inverse transform 230 may inversely transform the transform coefficients to obtain a residual signal (residual block, array of residual samples).

[116] The prediction module may perform prediction for the current block and may generate a predicted block including prediction samples for the current block. The prediction module may determine whether intra prediction or inter prediction is applied to the current block based on the prediction information output from the entropy decoder 210, and may determine a specific intra/inter prediction mode.

[117] Identical to that described in the prediction module of the image encoding equipment 100, the prediction module can generate a prediction signal based on various prediction methods (technologies) which are described below.

[118] The intra prediction module 265 may predict the current block by referring to the samples in the current frame. The description of the intra prediction module 185 applies equally to the intra prediction module 265 .

[119] The inter prediction unit 260 may derive a predicted block for the current block based on a reference block (reference sample array) indicated by a motion vector for the reference frame. In this case, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the adjacent block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction direction information (L0 prediction, L1 prediction, bi-prediction, etc.). In the case of inter prediction, an adjacent block may include a spatial adjacent block present in the current frame and a temporal adjacent block present in the reference frame. For example, inter-prediction unit 260 may configure a motion information candidate list based on neighboring blocks, and derive the current block's motion vector and/or reference frame index based on the received candidate selection information. Inter prediction may be performed based on various prediction modes, and the prediction information may include information indicative of the inter prediction mode for the current block.

[120] The adder 235 may generate a reconstructed signal (recovered frame, reconstructed block, reconstructed sample array) by summing the obtained residual signal with the predictive signal (predicted block, predicted sample array) output from the prediction module (including the inter prediction module 260 and /or internal prediction module 265). If there is no remainder for the block to be processed, for example, when the skip mode is applied, the predicted block may be used as the reconstructed block. The description of adder 155 is equally applicable to adder 235. Adder 235 may be referred to as a "recovery module" or "recovered block generator". The generated recovered signal may be used to intra-predict the next block to be processed in the current frame, and may be used to inter-predict the next frame through filtering as described below.

[121] The filter 240 can improve subjective/objective image quality by applying filtering to the reconstructed signal. For example, filter 240 may generate a modified reconstructed frame by applying various filtering methods to the reconstructed frame and store the modified reconstructed frame in memory 250, namely the DPB of storage 250. Various filtering methods may include, for example, filtering to deblock , filtering based on a sampled adaptive bias, an adaptive loop filter, a bilateral filter, and the like.

[122] The (modified) reconstructed frame stored in the DPB of the storage device 250 may be used as a reference frame in the inter prediction module 260 . The memory 250 may store block motion information from which motion information in the current frame is extracted (or decoded) and/or motion information of blocks in the frame that have already been recovered. The stored motion information may be transmitted to the inter-prediction unit 260 so that it is used as spatial neighbor block motion information or temporal neighbor block motion information. The memory 250 may store the recovered samples of the recovered blocks in the current frame and pass the recovered samples to the intra prediction module 265 .

[123] In the present disclosure, the embodiments described in the filter 160, the inter prediction unit 180, and the intra prediction unit 185 of the image coding equipment 100 can be equally or appropriately applied to the filter 240, the inter prediction unit 260, and the intra prediction unit 265 image decoding equipment 200 .

[124] General representation of image segmentation

[125] The video/image coding method according to the present disclosure may be based on an image segmentation structure as follows. In particular, the prediction, residual processing ((inverse) transform, (de)-quantization, etc.), syntax element encoding, and filtering procedures described below may be performed on a CTU, CU (and/or TU, PU) basis. ) extracted based on the image segmentation structure. An image may be segmented in units of blocks, and a block segmentation procedure may be performed in the image segmentation unit 110 of the encoding equipment. The segmentation related information may be encoded by the entropy encoder 190 and transmitted to the decoding equipment in the form of a bit stream. The entropy decoder 210 of the decoding equipment may extract the segmentation structure into blocks of the current frame based on the segmentation-related information obtained from the bitstream, and based on this, may perform a series of procedures (e.g., prediction, residual processing, block/frame recovery, in-loop filtering, and etc.) for image decoding. The CU size and TU size may be identical, or multiple TUs may be present in a CU area. Meanwhile, the CU size may generally represent the CB size of the luminance component (samples). The TU size may generally represent the TB size of the component (samples) luminance signals. The CB or TB size of the chrominance component (samples) can be derived based on the CB or TB size of the luminance component (samples) according to the component ratio according to the chrominance signal format (color format, e.g. 4:4:4, 4:2 :2, 4:2:0, etc.) frame/picture. The TU size may be derived based on maxTbSize indicating the maximum TB size available. For example, when the CU size exceeds maxTbSize, a set of TUs (TB) with maxTbSize may be extracted from the CU, and transformation/inverse transformation may be performed in units of TUs (TB). In addition, for example, when applying intra prediction, the mode/type of intra prediction may be extracted in units of CU (or CB), and the procedure for extracting neighboring reference samples and generating predictive samples may be performed in units of TU (or TB). In this case, one or multiple TUs (or TBs) may be present in the same CU- (or CB-) area, and in this case, multiple TUs or (TBs) may share the same intra prediction mode/type.

[126] In addition, when encoding and decoding images according to the present disclosure, the image processing module may have a hierarchical structure. For example, one frame may be segmented into one or more tiles or groups of tiles. One group of tiles may include one or more tiles. One tile may include one or more CTUs. A CTU may be segmented into one or more CUs as described above. A tile may consist of a rectangular area including CTUs collected in a particular row and particular column in a frame. The tile group may include an integer number of tiles according to a raster scan of the tiles. The tile group header may signal information/parameters applicable to the corresponding tile group. When the encoding/decoding equipment has a multi-core processor, the encoding/decoding procedure for a tile or group of tiles may be performed in parallel. Here, the tile group may have one of the tile group types including an internal (I) tile group, a predictive (P) tile group, and a bi-predictive (B) tile group. For blocks in an I-tile group, inter prediction may not be used and only intra prediction may be used for prediction. Of course, even in this case, the original sample value can be encoded and signaled without prediction. For blocks in a P-tile group, intra-prediction or inter-prediction may be used, and only uni-prediction may be used in inter-prediction. Meanwhile, for blocks in the B-tile group, intra-prediction or inter-prediction may be used, and up to bi-prediction may be used when inter-prediction is used.

[127] In encoding equipment, a tile/group of tiles, a slice, and a size of the maximum and minimum coding unit may be determined according to the characteristics (e.g., resolution) of the image, and considering the coding efficiency or parallel processing, and information for it or information capable of extracting it, may be included in the bitstream.

[128] In the decoding equipment, information indicating whether a slice of the current frame, a tile/tile group, or a CTU in a tile is segmented into a plurality of coding units can be obtained. When this information is received (transmitted) only under specific conditions, efficiency can be improved.

[129] A slice header or tile group header (tile group header syntax) may include information/parameters typically applicable to a slice or tile group. APS (APS Syntax) or PPS (PPS Syntax) may include information/parameters typically applicable to one or more frames. SPS (SPS Syntax) may include information/parameters generally applicable to one or more sequences. VPS (VPS Syntax) may include information/parameters generally applicable to the entire video. In the present disclosure, the high-level syntax may include at least one of APS syntax, PPS syntax, SPS syntax, or VPS syntax.

[130] In addition, for example, information regarding segmentation and construction of a tile/group of tiles may be constructed in an encoding cascade through a high-level syntax and transmitted to the decoding equipment in the form of a bitstream.

[131] Segmentation Structure

[132] Frames may be segmented into a sequence of coding tree units (CTUs). The CTU may correspond to a coding tree block (CTB). Alternatively, the CTU may include a luminance sample coding tree block and two corresponding chrominance samples coding tree blocks. For example, for a frame that contains three arrays of samples, the CTU may include an NxN block of luma samples and two corresponding blocks of chrominance samples. Fig. 4 shows an example in which a frame is segmented into CTUs.

[133] The maximum allowable CTU size for encoding and prediction may be different from the maximum allowable CTU size for transformation. For example, even when the maximum luminance block size in the transform CTU is 64×64, the maximum luminance block size in the CTU for coding and prediction may be 128×128.

[134] In addition, the frame may be segmented into one or more rows of tiles and one or more columns of tiles. A tile may be a sequence of CTUs covering a rectangular area in a frame.

[135] A tile may be segmented into one or more bricks, and each brick may consist of multiple CTU rows in a tile. In the present disclosure, a tile that is not segmented into a plurality of bricks may be referred to as a "brick".

[136] A slice may include a plurality of tiles in a frame or a plurality of bricks in a tile. Two slicing modes can be supported. One may be a raster scan based slicing mode and the other may be a rectangular slicing mode.

[137] In a raster slicing mode, a slice may include a plurality of consecutive tiles in a frame according to a raster scan order. In the present disclosure, a slice according to a raster scan based slicing mode may be referred to as a "raster scan based slicing".

[138] In the rectangular slicing mode, the slice may include a plurality of bricks constructing a rectangular area in the frame. In the present disclosure, a slice according to the rectangular slicing mode may be referred to as a "rectangular slice". The plurality of bricks included in a rectangular slice may exist according to the raster scan order of the slice bricks.

[139] FIG. 5a-5c are views illustrating examples of frame segmentation.

[140] First, referring to FIG. 5a, in a raster scan based slicing mode, a frame can be segmented into 12 tiles and three raster scan based slices.

[141] Referring to FIG. 5b, in the rectangular slicing mode, the frame may be segmented into 24 tiles (ie, six rows of tiles and four columns of tiles) and nine rectangular slices.

[142] Referring to FIG. 5c, the frame can be segmented into four tiles (i.e. two rows of tiles and two columns of tiles), 11 bricks (i.e. one top left brick, five top right bricks, two bottom left bricks, and three bottom right bricks). bricks) and into four rectangular slices.

[143] General Representation of CTU Segmentation

[144] As described above, a coding unit (CU) can be obtained by recursively segmenting a coding tree unit (CTU) or a largest coding unit (LCU) according to a quadtree/binary tree/ternary tree (QT/BT/TT) structure. For example, the CTU may first be segmented into quadtree structures. Thereafter, the leaf nodes of the quadtree structure may further be segmented by a multi-type tree structure.

[145] Quadtree segmentation means that the current CU (or CTU) is segmented equally into four. By segmenting according to the quadtree, the current CU can be segmented into four CUs having the same width and the same height. When the current CU is no longer segmented into a quadtree structure, the current CU corresponds to a leaf node of the quadtree structure. The CU corresponding to the leaf node of the quadtree structure can no longer be segmented and can be used as the final coding unit described above. Alternatively, the CU corresponding to the leaf node of the quadtree structure may further be segmented by a multi-type tree structure.

[146] FIG. 6 is a view showing an embodiment of a block segmentation type according to the multi-type tree structure. The segmentation according to the multi-type tree structure may include two types of partitioning according to the binary tree structure and two types of partitioning according to the ternary tree structure.

[147] The two types of splitting according to the binary tree structure may include vertical binary splitting (SPLIT_BT_VER) and horizontal binary splitting (SPLIT_BT_HOR). Vertical binary splitting (SPLIT_BT_VER) means that the current CU is split equally in half in the vertical direction. As shown in FIG. 5, by vertical binary splitting, two CUs having a height identical to the height of the current CU and having a width that is half the width of the current CU can be formed. Horizontal binary splitting (SPLIT_BT_HOR) means that the current CU is split equally in half in the horizontal direction. As shown in FIG. 5, by horizontal binary splitting, two CUs having a height that is half the height of the current CU and having a width identical to the width of the current CU can be formed.

[148] The two types of splitting according to the ternary tree structure may include vertical ternary splitting (SPLIT_TT_VER) and horizontal ternary splitting (SPLIT_TT_HOR). In vertical ternary splitting (SPLIT_TT_VER), the current CU is split in the vertical direction at a ratio of 1:2:1. As shown in FIG. 5, by vertical ternary partitioning, two CUs having a height identical to the height of the current CU and having a width that is 1/4 of the width of the current CU, and a CU having a height identical to the height of the current CU and having a width that is is half the width of the current CU. In horizontal ternary splitting (SPLIT_TT_HOR), the current CU is split horizontally in a ratio of 1:2:1. As shown in FIG. 5, by horizontal ternary partitioning, two CUs having a height that is 1/4 of the height of the current CU and having a width identical to the width of the current CU, and a CU having a height that is half the height of the current CU and having a width identical to the width of the current CU.

[149] FIG. 7 is a view showing a signaling mechanism of blocking information in a quadtree with a nested multi-type tree structure according to the present disclosure.

[150] Here, the CTU is treated as the root node of the quadtree and is segmented for the first time into a quadtree structure. Information (eg, qt_split_flag) indicating whether or not quadtree splitting is performed for the current CU (CTU or quadtree node (QT_node)) is signaled. For example, when qt_split_flag has a first value (eg, "1"), the current CU may be segmented into a quadtree. In addition, when qt_split_flag has a second value (eg, "0"), the current CU is not segmented into a quadtree, but becomes a leaf node (QT_leaf_node) of the quadtree. Each leaf node of the quadtree can then be further segmented into multi-type tree structures. Thus, a leaf node of a quadtree can become a node (MTT_node) of a multitype tree. In the multi-type tree structure, the first flag (eg, Mtt_split_cu_flag) is signaled to indicate whether or not the current node is further segmented. If the corresponding node is further segmented (eg, if the first flag is 1), a second flag (eg, Mtt_split_cu_vertical_flag) may be signaled to indicate the split direction. For example, the split direction may be the vertical direction if the second flag is 1, and may be the horizontal direction if the second flag is 0. Then, a third flag (eg, Mtt_split_cu_binary_flag) may be signaled to indicate whether the split type is is a binary split type or a ternary split type. For example, the split type may be a binary split type when the third flag is 1, and may be a ternary split type when the third flag is 0. A node of a multitype tree obtained by binary split or ternary split may further be segmented into multitype tree structures. However, a multitype tree node may not be segmented into quadtree structures. If the first flag is 0, the corresponding multitype tree node is no longer broken, but becomes a leaf node (MTT_leaf_node) of the multitype tree. The CU corresponding to the leaf node of the multi-type tree may be used as the final coding unit described above.

[151] Based on the mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag, the multi-type tree split mode (MttSplitMode) of the CU may be derived as shown in Table 1 below. In the description below, the multi-type tree split mode may be referred to as "multi-tree split type" or "partition type".

[152] Tab. 1

MttSplitMode mtt_split_cu_vertical_flag mtt_split_cu_binary_flag SPLIT_TT_HOR 0 0 SPLIT_BT_HOR 0 1 SPLIT_TT_VER 1 0 SPLIT_BT_VER 1 1

[153] FIG. 8 is a view showing an example in which a CTU is segmented into multiple CUs by applying a multi-type tree after applying a quadtree. In FIG. 8, the bold block edges 810 represent quadtree based segmentation, and the remaining edges 820 represent multitype tree based segmentation. A CU may correspond to a coding block (CB). In an embodiment, the CU may include a luminance sample coding block and two chrominance sample coding blocks corresponding to the luminance samples.

[154] The CB or TB size of the chrominance component (samples) may be derived based on the CB or TB size of the luminance component (samples) according to the component ratio according to the color format (chrominance format, e.g., 4:4:4, 4:2:2, 4:2:0, etc.) frame/picture. In the case of a 4:4:4 color format, the CB/TB size of the chrominance components may be set to the CB/TB size of the luminance components. In the case of the 4:2:2 color format, the width of CB/TB of the chroma components may be set to half the width of the CB/TB of the luma components, and the height of the CB/TB of the chrominance components may be set to the height of the CB/TB of the luma components. In the case of a 4:2:0 color format, the CB/TB width of the chroma components may be set to half the width of the CB/TB luma components, and the height of the CB/TB chrominance components may be set to half the height of the CB/TB luminance components.

[155] In an embodiment, when the CTU size is 128 based on the luminance sample unit, the CU size may be 128x128 to 4x4, which is the same size as the CTU. In one embodiment, in the case of a 4:2:0 color format (or chrominance format), the CB-size of the chrominance signals may be 64×64 to 2×2.

[156] Meanwhile, in the embodiment, the CU size and TU size may be identical. Alternatively, a plurality of TUs may be provided in a CU area. TU-size, in General, represents the size of the block transformation (TB) component (samples) of the luminance signals.

[157] The TU size may be derived based on the largest allowable TB size maxTbSize, which is a predetermined value. For example, when the CU size exceeds maxTbSize, a plurality of TUs (TBs) having maxTbSizes can be extracted from the CUs, and mapping/demapping can be performed in units of TUs (TBs). For example, the largest allowable TB size of luminance signals may be 64×64, and the largest allowable TB size of chrominance signals may be 32×32. If the width or height of a CB segmented according to the tree structure is greater than the largest width or height of the transform, the CB may be automatically (or implicitly) segmented until the TB-size limit in the horizontal and vertical directions is satisfied.

[158] In addition, for example, when intra prediction is applied, the mode/type of intra prediction may be extracted in units of CU (or CB), and the procedure for extracting adjacent reference samples and generating predictive samples may be performed in units of TU (or TB). In this case, one or multiple TUs (or TBs) may be provided in one CU- (or CB-) area, and in this case, multiple TUs or (TBs) may share the same intra prediction mode/type.

[159] Meanwhile, for a nested multi-type tree quadtree-based coding tree scheme, the following parameters may be signaled as SPS syntax elements from the encoding equipment to the decoding equipment. For example, at least one of CTU-size, which is a parameter representing the size of the quadtree root nodes, MinQTSize, which is a parameter representing the minimum allowed size of quadtree leaf nodes, MaxBtSize, which is a parameter representing the maximum the allowed size of the root nodes of the binary tree, MaxTtSize, which is a parameter representing the maximum allowed size of the root nodes of the ternary tree, MaxMttDepth, which is the parameter representing the maximum allowed depth of the splitting hierarchy based on the multitype tree from the leaf node of the quadtree, MinBtSize, which is a parameter representing the minimum allowed size of leaf nodes of a binary tree, or MinTtSize, which is a parameter representing the minimum allowed size of leaf nodes of a ternary tree, is signaled.

[160] As an embodiment of using the 4:2:0 chrominance format, the CTU size may be set to 128x128 luma blocks and two 64x64 chrominance blocks corresponding to the luma blocks. In this case, MinOTSize may be set to 16x16, MaxBtSize may be set to 128x128, MaxTtSzie may be set to 64x64, MinBtSize and MinTtSize may be set to 4x4, and MaxMttDepth may be set to 4. Quadtree based segmentation can be applied to CTUs to form leaf nodes of a quadtree. A leaf node of a quadtree may be referred to as a "QT leaf node". The leaf nodes of the quadtree can range in size from 16×16 (eg, MinOTSize) to 128×128 (eg, CTU-size). If the QT-node-leaf is 128x128, it may additionally not be segmented into a binary tree/ternary tree. This is due to the fact that in this case, even with segmentation, it exceeds MaxBtsize and MaxTtszie (for example, 64×64). In other cases, QT leaf nodes can additionally be segmented into a multi-type tree. Therefore, a QT leaf node is the root node for a multitype tree, and a QT leaf node can have a value of 0 for the depth of the multitype tree (mttDepth). If the depth of the multitype tree reaches MaxMttdepth (eg 4), additional segmentation may no longer be considered. If the width of a multitype tree node is equal to MinBtSize and less than or equal to 2xMinTtSize, then additional horizontal segmentation may not be considered. If the height of a multitype tree node is equal to MinBtSize and less than or equal to 2xMinTtSize, additional vertical segmentation may not be considered. When segmentation is not considered, the encoding equipment may skip signaling segmentation information. In this case, the decoding equipment can extract segmentation information with a predetermined value.

[161] Meanwhile, one CTU may include a luminance sample coding unit (hereinafter referred to as a "luminance signal unit") and two chrominance sample coding units corresponding thereto (hereinafter referred to as "chroma units"). The above-described coding tree-based scheme may be equally or separately applied to the luma block and the chrominance block of the current CU. In particular, luminance and chrominance blocks in one CTU may be segmented into an identical block tree structure, in which case, the tree structure is represented as SINGLE_TREE. Alternatively, the luma and chrominance blocks in one CTU may be segmented into separate block tree structures, in which case, the tree structure may be represented as DUAL_TREE. Thus, when the CTU is segmented into dual trees, a block tree structure for the luma block and a block tree structure for the chrominance block may be separately present. In this case, the block tree structure for the luma block may be called "DUAL_TREE_LUMA", and the block tree structure for the chrominance signal component may be called "DUAL_TREE_CHROMA". For groups of P- and B-slices/tiles, the luminance and chrominance blocks in one CTU may be restricted to have the same coding tree structure. However, for groups of I-slices/tiles, the luminance and chrominance blocks may have a separate block tree structure relative to each other. If a separate block tree structure is applied, the luminance CTB may be segmented into CUs based on a particular coding tree structure, and the chroma CTB may be segmented into chrominance CUs based on another coding tree structure. Thus, this means that a CU in an I-slice/tile group to which a separate block tree structure is applied may include a luma component coding block or two chroma component coding blocks, and a CU of a P- or B- group. slices/tiles may include blocks of three color components (luminance component and two chrominance components).

[162] Although a quadtree-based coding tree structure with a nested multi-type tree is described, the structure in which the CU is segmented is not limited to this. For example, a BT framework and a TT framework can be interpreted as a concept included in a multi-segmentation tree (MPT) structure, and a CU can be interpreted as being segmented through the QT framework and the MPT framework. In an example in which a CU is segmented through a QT framework and an MPT framework, a syntax element (for example, MPT_split_type) including information on how many blocks the leaf node of the QT framework is segmented into, and a syntax element (for example, MPT_split_mode) , including information as to which of the vertical and horizontal directions the leaf-node of the QT structure is segmented, may be signaled to determine the segmentation structure.

[163] In another example, the CU may be segmented in a manner different from the QT framework, the BT framework, or the TT framework. Thus, in contrast to the fact that a lesser CU is segmented 1/4 of a greater depth CU according to the QT structure, a lesser CU is segmented 1/2 of a greater depth CU according to the BT structure, or a lesser CU is segmented by 1 /4 or 1/2 of the deeper CU according to the TT structure, the shallower CU may be segmented into 1/5, 1/3, 3/8, 3/5, 2/3 or 5/8 of the deeper CU in some cases, and the CU segmentation method is not limited to this.

[164] When a portion of the node tree block exceeds the lower frame boundary and/or the right frame boundary for a frame, the corresponding tree node block may be constrained such that the samples in all encoded CUs are located on frame boundaries. In this case, for example, the following partitioning rules may apply.

[165] Split Rule 1: As a case in which a portion of the tree node box exceeds both the lower frame boundary and the right frame boundary, when the tree node box is a QT node having a size greater than the minimum QT size, the node box The tree node can be split in QT split mode, and when the tree node block has a size equal to or less than the minimum QT size, or is not a QT node, the tree node block is split in horizontal binary split mode (SPLIT_BT_HOR mode).

[166] Split rule 2: As a case in which split rule 1 is not satisfied and a part of the tree node box exceeds the lower frame boundary, the tree node box is split in the QT split mode when the tree node box is a QT node having a size greater than the minimum QT size and the maximum BT size, and the tree node block is split in the horizontal binary splitting mode, when the tree node block is a BTT node having a size equal to or less than the minimum QT size.

[167] Splitting Rule 3: As a case in which splitting rule 1 and splitting rule 2 are not satisfied and a part of the tree node block exceeds the right frame boundary, the tree node block is split in the QT splitting mode when the tree node block is a QT -a node having a size greater than the minimum QT size and the maximum BT size, and the tree node block is split in the QT split mode or in the horizontal binary split mode, when the tree node block has a size greater than the minimum QT size and equal to or less than maximum BT size. Alternatively, when the tree node block is a BTT node or has a size equal to or less than the minimum QT size, the tree node block is split in horizontal binary splitting mode.

[168] As described above, a multi-type tree quadtree coding block structure can provide a very flexible block segmentation structure. Due to the segmentation types supported in a multi-type tree, different segmentation patterns can potentially result in an identical coding block structure in some cases. In the encoding equipment and the decoding equipment, by limiting the occurrence of such redundant segmentation patterns, the amount of segmentation information data can be reduced.

[169] For example, FIG. 9 shows redundant splitting patterns that can occur in binary tree splitting and ternary tree splitting. As shown in FIG. 9, the continuous binary split 910 and 920 for one direction of the two-stage layers has a coding block structure identical to the coding block structure for the binary split for the center segment after the ternary split. In this case, binary tree splitting for the ternary tree splitting central blocks 930 and 940 may be disabled. this prohibition is applicable to the CU of all frames. When such a particular splitting is disabled, the signaling of the corresponding syntax elements can be modified to reflect this forbidden case, thereby reducing the number of bits signaled for splitting. For example, as shown in the example shown in FIG. 9, when binary tree splitting is disabled for the central CU, the syntax element mtt_split_cu_binary_flag indicating whether the split is binary split or ternary split is not signaled, and its value can be extracted as 0 by the decoding equipment.

[170] Virtual Conveyor Data Unit

[171] Virtual Pipeline Data Units (VPDUs) may be defined for pipelining within a frame. VPDUs may be defined as non-overlapping units in one frame. In decoding equipment, consecutive VPDUs may be simultaneously processed through multiple pipeline stages. In most stages of the pipeline, the VPDU size can be roughly proportional to the buffer size. Accordingly, keeping the VPDU size small is important when considering the buffer size from a hardware point of view. In most types of decoding equipment, the VPDU size may be set to the maximum transform block (TB) size. For example, the VPDU size may be 64x64 (64x64 luma samples). In addition, the VPDU size may change (increase or decrease) in view of the above-described partitioning into a ternary tree (TT) and/or a binary tree (BT).

[172] Meanwhile, in order to keep the VPDU size equal to 64×64, at least one of the following restrictions may be applied.

[173] Constraint 1: A ternary tree split for a CU having at least one of width or height of 128 is not allowed.

[174] Constraint 2: Horizontal binary tree splitting for a CU having a width of 128 and a height of 64 or less (ie, CU 128xN with N≤64) is not allowed.

[175] Constraint 3: Vertical binary tree splitting for a CU having a width of 64 or less or a height of 128 (ie, a CU of Nx128 with N≤64) is not allowed.

[176] Examples of CU partitioning now allowed under the above restrictions are shown in FIG. 10. In FIG. 10, bold solid lines indicate block splits, and the remaining solid lines indicate CUs.

[177] Referring to FIG. 10, according to constraint 1, vertical ternary tree splitting for CU 1010 128x128 is not allowed. In addition, according to constraint 1, splitting into a horizontal ternary tree for CU 1020 128×128 is not allowed. In addition, according to constraint 3, vertical binary tree partitioning for CU 1030 64x128 is not allowed. In addition, according to constraint 2, splitting into a horizontal binary tree for CU 1040 128x64 is not allowed. In addition, according to constraint 1, vertical ternary tree splitting for CU 1050 64×128 is not allowed. In addition, according to constraint 1, splitting into a horizontal ternary tree for CU 1060 128×64 is not allowed. In addition, according to constraint 1, horizontal ternary partitioning for CU 1070 64x128 is not allowed. In addition, according to constraint 1, vertical ternary partitioning for CU 1080 128x64 is not allowed.

[178] Meanwhile, in the dual tree in the intra frame, different segmentation structures may be applied to the luminance coding tree and the chrominance coding tree. In the dual tree, a larger coding pipeline may be introduced, and in the chrominance coding tree, a chrominance block having a small size such as 2×2, 4×2, and 2×4 may be resolved according to a range of QTBT values MinQTSizeC, MinBtSizeY and MinTTSizeY. However, this can make it difficult to design practical decoding hardware. In addition, the multiplication operation is required in cross-component linear model (CCLM) mode, planar mode, angular mode, and so on. In order to solve the above-described problems, in a dual tree, a chroma block having a small size such as 2x2, 4x2, and 2x4 may be limited through segmentation constraints.

[179] Interprediction General Representation

[180] Hereinafter, this document describes mutual prediction according to the present disclosure.

[181] The prediction module of the image coding equipment/image decoding equipment according to the present disclosure may perform inter-prediction in units of blocks to extract a prediction sample. Inter-prediction may represent prediction derived in a manner that depends on data elements (eg, sample values, motion information, etc.) of a frame(s) other than the current frame. When inter-prediction is applied to a current block, a predicted block (prediction block or prediction sample array) for the current block may be derived based on a reference block (reference sample array) indicated by a motion vector for a reference frame indicated by a reference frame index. In this case, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information of the current block may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the adjacent block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction type information (L0 prediction, L1 prediction, bi-prediction, etc.). In the case of inter prediction, an adjacent block may include a spatial adjacent block present in the current frame and a temporal adjacent block present in the reference frame. The reference frame including the reference block and the reference frame including the temporal neighbor block may be the same or different. The temporal neighbor block may be called a "collocated reference block", "collocated CU (colCU)", or "colBlock", and a reference frame including the temporal neighbor block may be referred to as a "collocated frame (colPic)" or "colPicture" . For example, a list of motion information candidates may be constructed based on neighboring blocks with respect to the current block, and flag or index information indicating which candidate is selected (used) may be signaled so as to derive the current block's motion vector and/ or the index of the reference frame.

[182] Mutual prediction may be performed based on various prediction modes. For example, in the case of the skip mode and the merge mode, the motion information of the current block may be equal to the motion information of the selected neighboring block. In the case of the skip mode, the residual signal may not be transmitted, in contrast to the combine mode. In the case of a motion information prediction (MVP) mode, a motion vector of a selected neighbor block may be used as a motion vector predictor, and the motion vector difference may be signaled. In this case, the motion vector of the current block may be extracted using the sum of the motion vector predictor and the motion vector difference. In the present disclosure, the MVP mode may have a meaning identical to that of advanced motion vector prediction (AMVP).

[183] The motion information may include L0 motion information and/or L1 motion information according to the type of mutual prediction (L0 prediction, L1 prediction, bi-prediction, etc.). The motion vector in the L0 direction may be referred to as the "L0 motion vector" or "MVL0", and the motion vector in the L1 direction may be referred to as the "L1 motion vector" or "MVL1". Prediction based on the L0 motion vector may be referred to as "L0 prediction", prediction based on the L1 motion vector may be referred to as "L1 prediction", and prediction based on both the L0 motion vector and the L1 motion vector may be referred to as "biprediction". ". Here, the L0 motion vector may indicate a motion vector associated with the L0 list of reference frames (L0), and the L1 motion vector may indicate a motion vector associated with the L1 list of reference frames (L1). The reference frame list L0 may include frames before the current frame in the output order as reference frames, and the reference frame list L1 may include frames after the current frame in the output order. Previous frames may be referred to as "forward (reference) frames", and subsequent frames may be referred to as "reverse (reference) frames". The reference frame list L0 may further include frames after the current frame in output order as reference frames. In this case, in the reference frame list L0, previous frames may be indexed first, and subsequent frames may then be indexed. The reference frame list L1 may further include frames before the current frame in output order as reference frames. In this case, in the reference frame list L1, subsequent frames may be indexed first, and previous frames may then be indexed. Here, the output order may correspond to the order of the numbers in the sequence of frames (POC).

[184] FIG. 11 is a flowchart illustrating an inter-prediction-based video/image coding method.

[185] FIG. 12 is a view illustrating the configuration of the inter prediction unit 180 according to the present disclosure.

[186] The encoding method of FIG. 11 may be performed by the image encoding equipment of FIG. 2. In particular, step S1110 may be performed by the inter-prediction unit 180, and step S1120 may be performed by the residual processor. Specifically, step S1120 may be performed by subtractor 115. Step S1130 may be performed by entropy encoder 190. Prediction information of step S1130 may be extracted by inter-prediction unit 180, and residual information of step S1130 may be extracted by residual processor. The residual information is information about the residual samples. The residual information may include information regarding quantized transform coefficients for the residual samples. As described above, the residual samples may be extracted as transform coefficients via the image encoding equipment transform 120, and the transform coefficients may be extracted as quantized transform coefficients via the quantizer 130. Information regarding the quantized transform coefficients may be encoded by the entropy encoder 190 via a residual encoding procedure.

[187] Referring to FIG. 11 and 12 together, the image encoding equipment may perform inter prediction for the current block (S1110). The image encoding equipment can extract the inter-prediction mode and motion information of the current block, and generate prediction samples of the current block. Here, the procedures for determining the inter-prediction mode, extracting motion information, and generating prediction samples may be simultaneously executed, or any of them may be performed before other procedures. For example, as shown in FIG. 12, the mutual prediction unit 180 of the image coding equipment may include a prediction mode determination unit 181, a motion information extraction unit 182, and a prediction sample extraction unit 183. The prediction mode determining unit 181 may determine the prediction mode of the current block, the motion information extractor 182 may extract the motion information of the current block, and the prediction sample extractor 183 may extract the prediction samples of the current block. For example, the inter-prediction unit 180 of the image coding equipment may search for a block similar to the current block in a predetermined area (search area) of reference frames through motion estimation, and retrieve a reference block whose difference with respect to the current block is equal to or less than a predetermined criterion or minimum. Based on this, a reference frame index indicating a reference frame in which the reference block is located can be extracted, and a motion vector can be extracted based on a position difference between the reference block and the current block. The image encoding equipment can determine the mode applied to the current block from various prediction modes. The image encoding equipment can compare the distortion cost functions as a function of the transmission rate (RD) for various prediction modes and determine the optimal prediction mode of the current block. However, the method for determining the prediction mode of the current block by the image encoding equipment is not limited to the above example, and various methods can be used.

[188] For example, when the skip mode or merge mode is applied to the current block, the image coding equipment may extract merge candidates from adjacent blocks relative to the current block and construct a merge candidate list using the extracted merge candidates. In addition, the image encoding equipment can extract a reference block whose difference relative to the current block is equal to or less than a predetermined criterion or minimum from the reference blocks indicated by the join candidates included in the join candidate list. In this case, the combining candidate associated with the extracted reference block may be selected, and combining index information indicating the selected combining candidate may be generated and signaled to the image decoding equipment. The motion information of the current block may be extracted using the motion information of the selected join candidate.

[189] As another example, when the MVP mode is applied to the current block, the image encoding equipment may extract motion vector predictor (MVP) candidates from adjacent blocks relative to the current block and construct a list of MVP candidates using the extracted MVP candidates. . In addition, the image encoding equipment may use the motion vector of an MVP candidate selected from among the MVP candidates included in the MVP candidate list as the MVP of the current block. In this case, for example, a motion vector indicating a reference block derived by the above-described motion estimation may be used as the motion vector of the current block, and the MVP candidate with the motion vector having the smallest difference with respect to the motion vector of the current block among the candidate MVP candidates , may represent the selected MVP candidate. A motion vector difference (MVD), which is a difference obtained by subtracting the MVP from the motion vector of the current block, can be extracted. In this case, index information indicating the selected MVP candidate and information regarding the MVD may be signaled to the picture decoding equipment. In addition, when applying the MVP mode, the index value of the key frame may be constructed as the index information of the key frame and separately signaled to the image decoding equipment.

[190] The image encoding equipment may derive residual samples based on the predictive samples (S1120). The image encoding equipment can extract residual samples through a comparison between the original samples of the current block and the predictive samples. For example, the residual sample may be derived by subtracting the corresponding predictive sample from the original sample.

[191] The image encoding equipment may encode image information including prediction information and residual information (S1130). The image encoding equipment 100 may output encoded image information in the form of a bitstream. The prediction information may include prediction mode information (eg, skip flag, merge flag, or mode index, etc.) and motion information as information associated with the prediction procedure. Of the prediction mode information, the skip flag indicates whether or not the skip mode is applied to the current block, and the join flag indicates whether or not the join mode is applied to the current block. Alternatively, the prediction mode information may indicate one of a plurality of prediction modes, such as a mode index. When the skip flag and the merge flag are 0, it may be determined that the MVP mode is applied to the current block. The information on the motion information may include candidate selection information (eg, merge index, MVP flag, or MVP index), which is information for extracting a motion vector. Of the candidate selection information, the merge index may be signaled when the merge mode is applied to the current block, and may be information for selecting one of the merge candidates included in the merge candidate list. Of the candidate selection information, an MVP flag or an MVP index may be signaled when the MVP mode is applied to the current block, and may be information for selecting one of the MVP candidates in the MVP candidate list. In addition, information regarding motion information may include information regarding the above-described MVD and/or reference frame index information. In addition, the information on the motion information may include information indicating whether L0 prediction, L1 prediction, or bi-prediction should be applied. The residual information is information about the residual samples. The residual information may include information regarding quantized transform coefficients for the residual samples.

[192] The output bitstream may be stored in a (digital) storage medium and transmitted to the image decoding equipment, or may be transmitted to the image decoding equipment via a network.

[193] As described above, the image encoding equipment can generate a reconstructed frame (a frame including reconstructed samples and a reconstructed block) based on the reference samples and the residual samples. This serves to extract, by the image encoding equipment, a prediction result identical to that performed by the image decoding equipment, thereby improving the encoding efficiency. Accordingly, the image encoding equipment may store the reconstructed frame (or the reconstructed samples and the reconstructed block) in a memory and use it as a reference frame for inter prediction. As described above, the in-loop filtering procedure may additionally be applied to the reconstructed frame.

[194] FIG. 13 is a flowchart illustrating a video/image decoding method based on inter-prediction.

[195] FIG. 14 is a view illustrating the configuration of the inter prediction unit 260 according to the present disclosure.

[196] The image decoding equipment can perform an operation corresponding to the operation performed by the image encoding equipment. The image decoding equipment may perform prediction for the current block based on the received prediction information and extract prediction samples.

[197] The decoding method of FIG. 13 may be performed by the image decoding equipment of FIG. 3. Steps S1310 to S1330 may be performed by the prediction unit and prediction information of step S1310, and the residual information of step S1340 may be obtained from the bitstream by the entropy decoder 210. The residual processor of the image decoding equipment may derive residual samples for the current block based on the residual information (S1340 ). In particular, the residual processor dequantizer 220 may perform dequantization on the dequantized transform coefficients derived from the residual information to extract the transform coefficients, and the residual processor inverse transform 230 may perform an inverse transform on the transform coefficients to extract the residual samples for the current block. Step S1350 may be performed by an adder 235 or a recovery module.

[198] In particular, the image decoding equipment may determine the prediction mode of the current block based on the received prediction information (S1310). The image decoding equipment can determine which inter-prediction mode is applied to the current block based on the prediction mode information in the prediction information.

[199] For example, it may be determined whether or not the skip mode is applied to the current block based on the skip flag. In addition, it may be determined whether the merge mode or MVP mode is applied to the current block based on the merge flag. Alternatively, one of the various possible inter-prediction modes may be selected based on the mode index. Possible options for inter prediction modes may include skip mode, merge mode, and/or MVP mode, or may include various inter prediction modes, which are described below.

[200] The image decoding equipment may extract motion information of the current block based on the determined inter-prediction mode (S1320). For example, when the skip mode or merge mode is applied to the current block, the image decoding equipment may construct a merge candidate list, which is described below, and select one of the merge candidates included in the merge candidate list. The selection may be performed based on the above-described candidate selection information (combination index). The motion information of the current block may be extracted using the motion information of the selected join candidate. For example, the motion information of the selected join candidate may be used as the motion information of the current block.

[201] As another example, when the MVP mode is applied to the current block, the image decoding equipment may construct a list of MVP candidates and use the motion vector of the MVP candidate selected from among the MVP candidates included in the MVP candidate list. options, as the MVP of the current block. The selection may be performed based on the candidate selection information (MVP flag or MVP index) described above. In this case, the MVD of the current block may be derived based on the information regarding the MVD, and the motion vector of the current block may be derived based on the MVP and MVD of the current block. In addition, the index of the key frame of the current block may be derived based on the information of the index of the key frame. The frame indicated by the index of the reference frame in the reference frame list of the current block may be retrieved as the reference frame referenced for inter-prediction of the current block.

[202] The image decoding equipment may generate prediction samples of the current block based on motion information of the current block (S1330). In this case, the reference frame may be derived based on the reference frame index of the current block, and the prediction samples of the current block may be derived using the reference block samples indicated by the current block motion vector for the reference frame. In some cases, the prediction sample filtering procedure may additionally be performed on all or some of the prediction samples of the current block.

[203] For example, as shown in FIG. 14, the mutual prediction unit 260 of the image decoding equipment may include a prediction mode determination unit 261, a motion information extraction unit 262, and a prediction sample extraction unit 263. In the image decoding equipment mutual prediction unit 260, the prediction mode determination unit 261 may determine the prediction mode of the current block based on the received prediction mode information, the motion information extraction unit 262 may extract motion information (motion vector and/or reference frame index, etc.). ) of the current block based on the received motion information, and the prediction sample extractor 263 may extract the prediction samples of the current block.

[204] The image decoding equipment may generate residual samples of the current block based on the received residual information (S1340). The image decoding equipment may generate reconstructed samples of the current block based on the predictive samples and the residual samples, and generate a reconstructed frame based on this (S1350). Thereafter, the in-loop filtering procedure may be applied to the reconstructed frame as described above.

[205] As described above, the inter-prediction procedure may include a step of determining an inter-prediction mode, a step of extracting motion information according to the determined prediction mode, and a step of performing prediction (predictive sampling) based on the extracted motion information. The inter-prediction procedure may be performed by the image encoding equipment and the image decoding equipment as described above.

[206] Subframe definition

[207] Hereinafter, this document describes a subframe according to the present disclosure.

[208] The subframe may construct a rectangular area of the encoded frame. The subframe size may be defined differently within a frame. For all frames belonging to the same sequence, the size and position of a particular individual subframe may be given the same. An individual subframe sequence may be independently decoded. The tile and slice (and CTB) may be constrained such that they do not overlap the subframe boundary. To this end, the encoding equipment may perform encoding such that the subframes are independently decoded. To this end, semantic restrictions in the bitstream may be required. In addition, for all frames belonging to the same sequence, the arrangement of tiles, slices, and bricks in a subframe may be constructed differently.

[209] The purpose of the subframe design decision

[210] The subframe design aims to abstract or encapsulate a range, a subframe level, or a larger group of slices or tiles. Therefore, a Constant Motion Tile Set Subset (MCTS) NAL NAL unit can be extracted from one VVC bitstream and rebased to another VVC bitstream without complexities such as modification at the VCL level. Here, MCTS is a coding technology for ensuring spatial and temporal independence between tiles, and when MCTS is applied, information regarding tiles that are not included in the MCTS to which the current tile belongs cannot be referred to. When an image is split into MCTS and encoded, independent transmission and decoding of the MCTS is possible.

[211] Such a sub-frame design may have the advantage of changing the viewing orientation in viewport-dependent mixed-resolution 360° streaming schemes.

[212] Subframe use case

[213] The use of a subframe is required in a viewport dependent 360° streaming scheme providing increased real spatial resolution for a viewport. For example, the layout in tiles covering a viewport extracted from a 6K (6144×3072) ERP (equirectangular projection) frame or cube map projection (CMP) resolution having a 4K decoding performance (HEVC layer 5.1) equivalent to it is included in sections D.6.3 and D.6.4 of the OMAF and is used in the industry VR forum standards guide. Such a resolution is known to be suitable for a head-mounted display using a square HD(2560×1440) display panel.

[214] Encoding: The content may be encoded at two spatial resolutions, including a resolution having a cube face size of 1656×1536 and a resolution having a cube face size of 768×768. In this case, in all bitstreams, a 6×4 grid of tiles may be used and the MCTS may be encoded at each tile position.

[215] Streamed MCTS selection: 12 MCTS may be selected from the high resolution bitstream, and 12 additional MCTS may be obtained from the low resolution bitstream. Therefore, a hemisphere (180°x180°) of the streamed content can be generated from a high resolution bit stream.

[216] Decoding using combining MCTS and bitstream: MCTS of one time point are received, which can be combined into an encoded frame having a resolution of 1920×4608 corresponding to the HEVC layer 5.1. In another embodiment for the merged frame, four columns of tiles have a width value of 768, two columns of tiles have a width value of 384, and three rows of tiles have a height value of 768, thereby constructing a frame consisting of luminance samples of 3840×2304 . Here, the units of width and height may be units of the number of luma samples.

[217] Subframe Signaling

[218] Subframe signaling may be performed at the SPS layer, as shown in FIG. 15.

[219] FIG. 15 is a view illustrating the SPS syntax for signaling a syntax element for subframes.

[220] Referring to FIG. 15, the pic_width_max_in_luma_samples syntax element may indicate the maximum width of each decoded frame referring to the SPS in units of luma samples. The value of pic_width_max_in_luma_samples may be greater than 0 and may be an integer multiple of MinCbSizeY. Here, MinCbSizeY may indicate a minimum luminance component coding block size.

[221] The pic_height_max_in_luma_samples syntax element may indicate the maximum height of each decoded frame referring to the SPS in units of luma samples; pic_height_max_in_luma_samples may be greater than 0 and may be an integer multiple of MinCbSizeY.

[222] The subpic_grid_col_width_minus1 syntax element may be used to indicate the width of an individual grid element of subframe identifiers. For example, the value obtained by summing 1 with subpic_grid_col_width_minus1 may indicate the width of an individual subframe ID grid element in units of 4 samples; subpic_grid_col_width_minus1 can have bit length in Ceil(Log2 (pic_width_max_in_luma_samples/4)).

[223] Therefore, the NumSubPicGridCols variable indicating the number of columns in the subframe grid can be retrieved as shown in Equation 1 below.

[224] Equation 1

NumSubPicGridCols=(pic_width_max_in_luma_samples+subpic_grid_col_width_minus1*4+3)/(subpic_grid_col_width_minus1*4+4)

[225] Further, the subpic_grid_row_height_minus1 syntax element may be used to indicate the height of each subframe identifier grid element. For example, the value obtained by summing 1 with subpic_grid_row_height_minus1 may indicate the height of an individual subframe ID grid element in units of 4 samples; subpic_grid_row_height_minus1 can have bit length in Ceil(Log2(pic_height_max_in_luma_samples/4)).

[226] Therefore, the NumSubPicGridRows variable indicating the number of rows in the subframe grid can be retrieved as shown in Equation 2 below.

[227] Equation 2

NumSubPicGridRows=(pic_height_max_in_luma_samples+subpic_grid_row_height_minus1*4+3)/(subpic_grid_row_height_minus1*4+4)

[228] Then, the subpic_grid_idx[i][j] syntax element may indicate the index of the subframe at position (i, j) of the grid; subpic_grid_idx[i][j] can have bit length in Ceil(Log2(max_subpics_minus1+1)).

[229] The variables SubPicTop[subpic_grid_idx[i][j]], SubPicLeft[subpic_grid_idx[i][j]], SubPicWidth[subpic_grid_idx[i][j]], SubPicHeight[subpic_grid_idx[i][j]], and NumSubPic can retrieved using the algorithm of FIG. 16.

[230] Then, the subpic_treated_as_pic_flag[i] syntax element may indicate whether or not the subframe is treated as identical to the general frame during the decoding process. For example, the first value (eg, 0) of subpic_treated_as_pic_flag[i] may indicate that the i-th subframe included in a single CVS frame to be encoded is not treated as a frame in the decoding process. The second value (eg, 1) of subpic_treated_as_pic_flag[i] may indicate that the i-th subframe included in the individual CVS frame to be encoded is treated as a frame in the decoding process. When the value of subpic_treated_as_pic_flag[i] is not obtained from the bitstream, the value of subpic_treated_as_pic_flag[i] may be inferred as the first value (eg, 0).

[231] The loop_filter_across_subpic_enabled_flag[i] syntax element may indicate whether or not in-loop filtering can be performed across the boundary of the i-th subframe included in a single CVS encoded frame. For example, the first value (eg, 0) of loop_filter_across_subpic_enabled_flag[i] may indicate that in-loop filtering cannot be performed across the boundary of the i-th subframe included in a single CVS encoded frame. The second value (eg, 1) of loop_filter_across_subpic_enabled_flag[i] may indicate that in-loop filtering can be performed across the boundary of the i-th subframe included in a single CVS encoded frame. When the value of subpic_treated_as_pic_flag[i] is not obtained from the bitstream, the value of subpic_treated_as_pic_flag[i] may be inferred as the first value (eg, 0).

[232] Meanwhile, in order to match the bitstream, the following restrictions may be applied. For any two subframes subpicA and subpicB, when the index of subpicA is less than the index of subpicB, all coded NAL units of subpicA shall have a lower decoding order than all coded NAL units of subpicB. Alternatively, after decoding is performed, the shape of the subframes should have a perfect left boundary and a perfect upper boundary, constructing a frame boundary or a previously decoded subframe boundary.

[233] Hereinafter, an image encoding/decoding method based on a subframe structure will be described in detail with reference to FIG. 17 and 18.

[234] FIG. 17 is a view illustrating an image encoding method using a subframe by image encoding equipment according to an embodiment of the present disclosure.

[235] The image encoding equipment may encode the current frame based on the structure of the subframe. Alternatively, the image encoding equipment may encode at least one subframe constructing the current frame and output a (sub)bitstream including (encoded) information about at least one (encoded) subframe.

[236] Referring to FIG. 17, the image encoding equipment may partition the input frame into a plurality of subframes (S1710). In addition, the image encoding equipment may generate information regarding the subframe (S1720). Here, the information regarding the subframe may include information regarding the area of the subframe and/or information regarding the grid spacing to be used for the subframe. In addition, the subframe information may include information indicating whether or not each subframe can be treated as a frame, and/or information indicating whether or not in-loop filtering can be performed across a boundary of each subframe.

[237] The image encoding equipment may encode at least one subframe based on information regarding the subframe. For example, each subframe may be independently encoded based on information regarding the subframe. In addition, the image encoding equipment can encode image information including information regarding a subframe and output a bit stream (S1730). Here, the bitstream for a subframe may be referred to as a "substream" or "subbitstream".

[238] FIG. 18 is a view illustrating a method for decoding an image using a subframe by the image decoding equipment according to the embodiment.

[239] The image decoding equipment may decode at least one subframe belonging to the current frame using (encoded) information of at least one (encoded) subframe obtained from the (sub)bitstream.

[240] Referring to FIG. 18, the image decoding equipment may obtain information regarding the subframe from the bit stream (S1810). Here, the bitstream may include a substream or a subbitstream for a subframe. Information regarding the subframe may be constructed in the HLS of the bitstream. In addition, the image decoding equipment can extract at least one subframe based on information regarding the subframe (S1820).

[241] The image decoding equipment may decode at least one subframe based on information regarding the subframe (S1830). For example, when the current subframe including the current block is treated as a frame, the current subframe can be independently decoded. In addition, when in-loop filtering may be performed across a boundary of the current subframe, in-loop filtering (eg, deblocking filtering) may be performed on a boundary of the current subframe and a boundary of an adjacent subframe adjacent to the boundary. In addition, when the boundary of the current subframe coincides with the boundary of the frame, in-loop filtering across the boundary of the current subframe may not be performed. The image decoding equipment can decode a subframe based on a CABAC method, a prediction method, a residual processing (transform and quantization) method, an in-loop filtering method, and so on. In addition, the image decoding equipment may output at least one decoded subframe or a current frame including at least one subframe. The decoded subframe may be output in the form of an output subframe set (OPS). For example, with respect to a 360-degree view image or an omnidirectional image, when only a part of the current frame is rendered, only some of all subframes in the current frame may be decoded, and all or some of the decoded subframes may be rendered according to the user's viewport.

[242] Filtering General Representation

[243] Hereinafter, this document describes a filtering method according to the present disclosure.

[244] Filtering may be performed on the reconstructed frame generated by the image encoding/decoding equipment. As a result of performing the filtering, a modified reconstructed frame may be generated, and the image decoding equipment may determine the modified reconstructed frame as the final decoded frame. In addition, in the image encoding/decoding equipment, the modified reconstructed frame may be stored in a decoded frame buffer (DPB) or memory, and then may be used as a reference frame in frame encoding/decoding.

[245] In the present disclosure, filtering according to some embodiments of the present disclosure may be used with the same meaning as in-loop filtering. The filter used for filtering may include at least one of a deblocking filter, a sampled adaptive offset (SAO) filter, an adaptive loop filter (SLF), or a two-sided filter. At least one of the deblocking filter, SAO filter, ALF, and/or two-way filter may be sequentially applied to the reconstructed frame, thereby generating a modified reconstructed frame. The filter application order may be preset in the image encoding/decoding equipment. In the example, after the deblocking filter is applied to the reconstructed frame, the SAO filter may be applied. In another example, after the deblocking filter is applied to the reconstructed frame, ALF may be applied. In the present disclosure, filtering may be performed by at least one of the filter 160 of FIG. 2 and/or filter 240 of FIG. 3.

[246] The deblocking filter may eliminate distortion generated at a boundary of the reconstructed frame. For example, the deblocking filter may extract a target boundary, which is a boundary between blocks in a reconstructed frame, and set a boundary intensity (BS) for the target boundary. The image encoding/decoding equipment may perform deblocking filtering for a target boundary by applying filtering based on a predetermined boundary intensity to the target boundary. Here, the boundary intensity may be determined based on at least one of the prediction modes of the two blocks adjacent to the target boundary, the motion vector difference, whether or not the reference frames are identical, or the presence/absence of a non-zero effective coefficient.

[247] As another example, the SAO filter may compensate for the offset difference between the reconstructed frame and the original frame in units of samples. The SAO filter may be implemented by a filter type such as a bandpass based filter or an offset based filter. When the SAO filter is used, the samples may be classified into different categories according to the type of SAO filter, and a predetermined bias value may be added to each sample based on the category. The information regarding the SAO filter may include at least one of information regarding whether or not to apply the SAO filter, SAO filter type information, and/or SAO offset value information. In an example, the SAO filter may be limited to being applied to a reconstructed frame to which the deblocking filter is applied.

[248] The ALF may be a unit-sample filter for applying filter coefficients according to the shape of the filter to the reconstructed frame. The image coding equipment may signal at least one of whether or not to apply the ALF, the shape of the ALF, and/or the filter factor by comparing the reconstructed frame with the original frame. Thus, the ALF information may include at least one of information regarding whether or not to apply ALF, ALF filter shape information, and/or ALF filter coefficient information. In an example, ALF may be limited to applying to a reconstructed frame to which a deblocking filter is applied.

[249] General Representation of the Deblocking Filter

[250] FIG. 19 is a view illustrating a method for applying a deblocking filter.

[251] As described above, the deblocking filter may be applied to the reconstructed frame. The deblocking filter may be applied to the boundary of each CU (or TU) included in the current block based on the coding/decoding order of the frame.

[252] In some embodiments, the deblocking filter may first be applied to the vertical boundary of the CU and then applied to the horizontal boundary of the encoding/decoding block. Alternatively, the deblocking filter may first be applied to the horizontal border of the CU and then applied to the vertical border of the CU. The deblocking filter may be applied to all CU boundaries, all subunit boundaries in all CUs, all TU boundaries, and/or all subunit boundaries in all TUs.

[253] Referring to FIG. 19, the image encoding/decoding equipment can extract a target boundary to which deblocking filtering is to be applied (S1910). In addition, the image encoding/decoding equipment can determine the boundary intensity of the target boundary (S1920). In an example, the boundary intensity bS may be determined according to the condition of the transform block adjacent to the target boundary. In the present disclosure, the boundary intensity may also be referred to as "edge filtering intensity" or "filtering intensity". In the description below, when the target boundary is a vertical boundary, based on the target boundary, the left block may be defined as a P-box, and the right block may be defined as a Q-box. In addition, when the target boundary is a horizontal boundary, based on the target boundary, the upper block may be defined as a P-box and the lower block may be defined as a Q-box. In addition, a sample of P-boxes may be denoted by p ₀ , and a sample of Q-boxes may be denoted by q ₀ . Here, p and q may represent samples facing the target boundary in the P-box and Q-box. For example, p ₀ may represent a sample of the left or top block adjacent to the target boundary, and q ₀ may represent a sample of the right or bottom block adjacent to the target boundary.

[254] In an example where BDPCM is applied to a sample of P-blocks and a sample of Q-blocks included in one block of luminance signals, the boundary intensity for the target boundary may be determined as the first value (for example, 0).

[255] In another example, when a P-block sample and a Q-block sample included in the same CU are intra-predicted, the boundary intensity for the target boundary may be determined as a third value (eg, 2). As another example, when the target boundary is a TU boundary and combined inter-intra prediction (CIIP) is applied to a sample of P-blocks and a sample of Q-blocks included in one CU, the boundary intensity for the target boundary may be determined as the third value ( for example 2). As another example, when the target edge is a TU edge, and at least one of the P-block sample and the Q-block sample included in one TU has a non-zero transform coefficient level, the edge intensity for the target edge may be defined as component of the second value (for example, 1). As another example, when the prediction mode of the sub-CU including P-block sampling is different from the prediction mode of the CU sub-unit including Q-block sampling, the boundary intensity for the target boundary may be determined to be a component of the second value (e.g., 1) .

[256] In another example, when the current block is a block of luminance signals, and at least one of the following conditions is satisfied, the boundary intensity for the target boundary may be determined as a component of the second value (eg, 1). Meanwhile, when all of the above-described conditions are not satisfied, the boundary intensity for the target boundary may be determined as a component of the first value (eg, 0).

[257] For example, when both a sub-block in a CU including a P-block sample and a sub-block in a CU including a Q-block sample are encoded/decoded in IBC mode, and the difference in horizontal or vertical motion vector values for of each subblock is equal to or greater than a value of 4 units per unit per 1/4 luminance sample, the edge intensity for the target edge may be determined to be a component of the second value (eg, 1).

[258] Alternatively, when a sub-unit in a CU including a P-block sample and a sub-unit in a CU including a Q-block sample refer to different reference frames or have different numbers of motion vectors, the boundary intensity for the target boundary may be determined as a component of the second value (for example, 1).

[259] Alternatively, when one motion vector is used to predict a sub-block in a CU including a P-block sample and a sub-block in a CU including a Q-block sample, or the difference between horizontal values or vertical values of the motion vector for each subblock equal to or greater than 4 units per unit per 1/4 luminance sample, the edge intensity for the target edge may be determined to be a component of the second value (eg, 1).

[260] Alternatively, when two motion vectors and two different reference frames are used to predict a subblock in a CU including a P-block sample, two motion vectors for the same reference frame are used to predict a subblock in a CU including into the Q-block sample, and the difference between the horizontal values or the vertical values of the motion vectors for the same reference frame is equal to or greater than a value of 4 units per unit per 1/4 luma sample, the edge intensity for the target edge may be determined as a component of the second value.

[261] Alternatively, when two motion vectors for the same reference frame are used to predict a subunit of a CU including a sample of P-blocks, and two motion vectors for an identical reference frame are used to predict a subunit of a CU including a sample of Q-blocks, the boundary intensity for the target boundary may be determined to be a component of the second value (eg, 1) if the following first and second conditions are satisfied. Here, the first condition may mean a case in which the absolute value difference between the horizontal components or the vertical components is equal to or greater than a value of 4 units per unit per 1/4 sample luminance signals for the list 0 motion vectors used to predict each sub-block, or a difference in the absolute value between the horizontal components or the vertical components is equal to or greater than the value of 4 units per unit in 1/4 luminance samples for the list 1 motion vectors used to predict each sub-block. In addition, the second condition may mean a case in which the difference in absolute value between the horizontal or vertical components is equal to or greater than a value of 4 units per unit per 1/4 sample luma for list 0 motion vectors for subblock prediction in a CU including the list 1 motion vector for sub-block prediction in a CU including the Q-block sample, or the difference in absolute value between the horizontal components or the vertical components between the list 1 motion vector for sub-block prediction of the CU including the P sample -blocks, and a list 0 motion vector for predicting a CU subblock including a Q-block sample is equal to or greater than a value of 4 units per unit per 1/4 luminance sample.

[262] The image encoding/decoding equipment may apply the deblocking filter to the target boundary based on the boundary intensity determined based on the condition described above (S1930). For example, when the edge intensity of the target edge is determined to be a component of the first value (eg, 0), the image encoding/decoding equipment may not apply the deblocking filter to the target edge. The filtering strength (eg, strong or weak) and/or filter length for the deblocking filter applied to the target boundary may be determined in various ways. Meanwhile, when the sample to be filtered is a luminance sample, filtering may be performed using a short filter or a long filter. Alternatively, when the sample to be filtered is a chroma sample, filtering may be performed using a chroma filter.

[263] Determining the Sample to be Filtered

[264] The reconstructed samples pi, k and qj, k to which the deblocking filter is applied may be indicated on the vertical boundary, as shown in Equation 3 below.

[265] Equation 3

qj, k=recPicture[xCb+xBl+j][yCb+yBl+k]

pi, k=recPicture[xCb+xBl-i-1][yCb+yBl+k]

[266] The recovered samples pi, k and qj, k to which the filtering is applied may be indicated on the horizontal boundary, as shown in Equation 4 below.

[267] Equation 4

pi, k=recPicture[xCb+xBl+k][yCb+yBl-i-1]

qj, k=recPicture[xCb+xBl+k][yCb+yBl+j]

[268] In Equations 3 and 4, i has a value equal to or greater than 0 and equal to or less than the maximum filter length (eg, maxFilterLengthP) for the P-block, j has a value equal to or greater than 0 and equal to or less than the maximum filter length ( for example, maxFilterLengthQ) for a Q-block, and k may have a value equal to or greater than 0 and equal to or less than 3. In addition, xCb and yCb may indicate the position (xCb, yCb) of the top left fetch of the current CU for which filtering to remove deblocking is performed, and xBl and yBl may indicate a boundary position (xBl, yBl) to indicate a P-box and a Q-box for which deblocking filtering is performed based on (xCb, yCb).

[269] FIG. 20 is a view illustrating a reconstructed sample to which a deblocking filter is applied.

[270] Referring to FIG. 20, the top left sample position of the current CU may be (xCb, yCb) (2010). When deblocking filtering is performed on the vertical boundary of the current CU, the P-box 2021 and Q-box 2022 on which the deblocking filtering is performed may be indicated using the first sample position (xCb+xBl1, yCb+yBl1) (2020). In addition, when deblocking filtering is performed on the horizontal boundary of the current CU, the P-box 2031 and Q-box 2031 on which deblocking filtering is performed may be indicated using the second sample position (xCb+xBl2, yCb+yBl2) ( 2030). In each case, P-boxes 2021 and 2031 may include a plurality of recovered samples pi, k. In addition, Q-blocks 2022 and 2032 may include a plurality of recovered samples qj, k.

[271] Filtering Strength

[272] In the case of HEVC, in contrast to controlling the filtering strength of the deblocking filter by the variables β and tC derived from the average quantization parameter qP _L , the filtering strength of the deblocking filter according to the embodiments of the present disclosure can be controlled by summing a predetermined offset according to the level the luminance signal of the reconstructed sample with an average quantization parameter qP _L. Here, the luminance signal level (LL) of the reconstructed sample may be extracted as shown in Equation 5 below.

[273] Equation 5

LL=((p _0.0 +p _0.3 +q _0.0 +q _0.3 )>>2)/(1<<bitDepth),

[274] where p ₀ , ₀ and p ₀ , ₃ may denote the reconstructed samples adjacent to the target boundary and most spaced apart in the P-box, and q ₀ , ₀ and q ₀ , ₃ may denote the reconstructed samples, adjacent to the target boundary and most spaced apart in the Q-block. In addition, bitDepth may indicate the bit depth of the reconstructed depth.

[275] A predetermined offset (eg, qpOffset) added to the average quantization parameter qP _L may be signaled via the SPS and depends on the transfer function.

[276] For the reconstructed luminance component sample, a bilinear filter (or a stronger deblocking filter) may be applied to samples located on any boundary relative to the large block. Here, a relatively large block may mean a block having a size of 32x32 or more. In contrast, for the reconstructed chrominance component sample, a strong deblocking filter is applied. Here, the strong deblocking filter may be set as shown in Equation 6 below.

[277] Equation 6

p ₂ '=(3*p ₃ +2*p ₂ +p ₁ +p ₀ +q ₀ +4)>>3

p ₁ '=(2*p ₃ +p ₂ +2*p ₁ +p ₀ +q ₀ +q ₁ +4)>>3

p ₀ '=(p ₃ +p ₂ +p ₁ +2*p ₀ +q ₀ +q ₁ +q ₂ +4)>>3,

[278] where p ₁ -p ₃ may denote recovered samples in a P-box, and q ₀ -q ₂ may denote recovered samples in a Q-box. The filtered retrieval samples p ₀ '-p ₂ ' in the P-box may be extracted using the retrieval samples in the P-box and the retrieval samples in the Q-box.

[279] In the present disclosure, a strong deblocking filter applied to the reconstructed chrominance component sample may be referred to as a "chrominance filter". The chrominance filter may be applied to an 8×8 chroma sample grid and may be used for horizontal and vertical boundaries. In the example, the chrominance filter may be selected only when the horizontal and vertical edges of the chrominance block are equal to or greater than 8 chroma sample units and all of the following first to third conditions are satisfied. Here, the first condition may mean that the edge intensity of the chrominance block is equal to the edge intensity with respect to the large block of luminance signals. For example, when the edge intensity of the chrominance block has a second value (eg, 1) or a third value (eg, 2), a chrominance filter may be selected. The second condition and the third condition may be equal to a strong filter determination condition for the reconstructed luma component sample, for example, an on-off determination condition and a strong filter determination condition.

[280] In addition, the boundary intensity for filtering the chrominance signal may be modified based on the attributes of an adjacent block adjacent to the boundary of the current block, as shown in Table 2.

[281] Tab. 2

Conditions Y (luminance signal) U (chroma signal) V (chrominance signal) At least one of the adjacent blocks is internal 2 2 2 At least one of the adjacent blocks has non-zero transform coefficients 1 1 1 The absolute difference between motion vectors that belong to adjacent blocks is greater than or equal to one integer luma sample 1 No data No data Motion prediction in adjacent blocks referring to vectors is different 1 No data No data Other 0 0 0

[282] Referring to Table 2, when at least one adjacent block adjacent to the current block is intra-predicted, the edge intensity of the current block may have a third value (eg, 2) for all color components (eg, Y, U, and V). In addition, when at least one adjacent block adjacent to the current block has non-zero transform coefficients, the edge intensity of the current block may have a second value (eg, 1) for all color components. In addition, when the absolute difference between the motion vectors of adjacent blocks adjacent to the current block is equal to or greater than one integer luminance sample, the edge intensity of the current block may only be present for the luminance component and may have a second value (e.g., 1). In addition, when motion prediction refers to different motion vectors in contiguous blocks adjacent to the current block, the edge intensity of the current block may only be present for the luma component and may have a second value (eg, 1). Meanwhile, when all of the above conditions are not satisfied, the edge intensity of the current block may have a first value (eg, 0) for all color components.

[283] In an example, a deblocking filter may be applied based on an 8×8 grid. For example, a deblocking filter may be applied to a CU boundary aligned in an 8×8 grid and coinciding with a sub-block boundary. Hereinafter, a subblock boundary may include a PU boundary in subblock-based TMVP mode and/or affine mode and a TU boundary in SBT and/or ISP mode. In SBT and/or ISP mode, the deblocking filtering process for a TU boundary may proceed with the same logic as in the case of HEVC. For example, when there are non-zero coefficients at a block edge in a sub-block, deblocking filtering for the TU boundary may be performed. In addition, in the subblock-based TMVP mode and/or the affine mode, the deblocking filtering process for the PU boundary can proceed with the same logic as in the case of HEVC. For example, based on a difference between motion vectors of an adjacent subblock and a difference between key frames, deblocking filtering for a PU boundary may be performed.

[284] Prior Art Problems

[285] When each subframe is treated as a frame, each subframe may be independently encoded/decoded. Here, independent encoding/decoding may mean that a block partition structure (e.g., single tree structure, dual tree structure, etc.) for a subframe, a prediction mode (e.g., intra prediction, inter prediction, etc.) etc.), encoding/decoding order, etc. are determined regardless of the encoding/decoding information of the various subframes. For example, when each of the first subframe and the second subframe is treated as a frame, the first subframe may be encoded/decoded based on intra prediction and the second subframe may be encoded/decoded based on inter prediction. Whether or not each subframe is treated as a frame may be signaled using the subpic_treated_as_pic_flag in a high-level syntax (eg, SPS syntax) as described above with reference to FIG. 15. For example, subpic_treated_as_pic_flag for a subframe that is not treated as a frame may have a first value (eg, 0). In contrast, subpic_treated_as_pic_flag for a subframe being treated as a frame may have a second value (eg, 1).

[286] Meanwhile, for the boundary of each subframe, in-loop filtering (eg, deblocking filtering) across the boundary may be performed. In the present disclosure, in-loop filtering across a subframe boundary may be referred to as "lateral in-loop filtering" or "transverse filtering". In addition, the filter for transverse filtration may be referred to as "transverse in-loop filter" or "transverse filter". Whether or not a cross filter is applicable to the boundary of each subframe may be signaled using the loop_fliter_across_subpic_enabled_flag in the SPS described above with reference to FIG. 15. For example, loop_fliter_across_subpic_enabled_flag for a subframe to which a cross filter is not applicable may have a first value (eg, 0). In contrast, loop_fliter_across_subpic_enabled_flag for a subframe to which a cross filter is applicable may have a second value (eg, 1).

[287] Whether or not a transverse filter is applicable to a subframe boundary can be independently determined regardless of whether or not a subframe is treated as a frame. In addition, whether or not a cross filter is applicable to a subframe boundary can be independently determined without regard to attributes of an adjacent subframe adjacent to the subframe boundary.

[288] Even if the lateral filter is not applicable to the boundary of the current subframe treated as a frame, since the lateral filtering is performed on the boundary of an adjacent subframe adjacent to the boundary, the area treated as a frame may decrease in the current subframe (or co-located subframe).

[289] FIG. 21 and 22 are views illustrating examples of reducing an area treated as a frame in a subframe.

[290] First, referring to FIG. 21, the first subframe 2110 may not be treated as a frame (eg, subpic_treated_as_pic_flag[0=0). In addition, a cross filter may be applied to the left vertical border of the first subframe 2110 (eg, loop_filter_across_subpic_enabled_flag[0=1).

[291] In contrast, the second subframe 2120 adjacent to the left vertical border of the first subframe 2110 may be treated as a frame (eg, subpic_treated_as_pic_flag[1]=1). In addition, the cross filter is not applicable to the right vertical border of the second subframe 2120 (eg, loop_filter_across_subpic_enabled_flag[1]=0).

[292] Even if the lateral filter is not applicable to the right vertical border of the second subframe 2120 treated as a frame, since the lateral filter is applied to the left vertical border of the adjacent first subframe 2110, the right vertical border of the second subframe 2120 may be filtered along with the left vertical border of the first subframe 2120.

[293] As a result, in the filtered region 2130, the sample values may be changed using the sample values of the first subframe 2110 and the sample values of the second subframe 2120. Therefore, in the second subframe 2120, the area treated as a frame may be reduced to an area except filtered right border.

[294] As another example, referring to FIG. 22, fourth subframe Sub-picture[4]), which is the current frame including the current block, may be treated as a frame (eg, subpic_treated_as_pic_flag=1). When the sub-block-based temporal motion vector prediction (TMVP) mode is applied to the current block, the fourth Sub-picture[4] subframe in the reference frame may be referred to as the co-located subframe for the current block. Thus, the fourth Sub-picture[4] in the reference frame may include a temporary reference block (collocated reference block) for the current block.

[295] In the fourth Sub-picture[4] in the reference frame, it is assumed that the lateral filter is applied to the left vertical border and the upper horizontal border, and the lateral filter is not applied to the right vertical border and the lower horizontal border.

[296] Because the lateral filter is not applied to the right vertical border of the fourth Sub-picture[4] subframe in the reference frame, but the left vertical border of the fifth Sub-picture[5] subframe adjacent to the right vertical border is lateral filtered, the right vertical border of the fourth sub-picture[4] in the reference frame may also be filtered. In addition, since the lateral filter is not applied to the lower horizontal boundary of the fourth Sub-picture[4] in the reference frame, but the upper horizontal boundary of the seventh Sub-picture[7] adjacent to the lower horizontal boundary is lateral filtered, the lower horizontal boundary of the fourth sub-picture[4] in the reference frame may also be filtered. As a result, the sample values of the filtered area in the fourth Sub-picture[4] subframe in the reference frame can be modified using the sample values of different Sub-picture[0]-Sub-picture[3] and Sub-picture[5]-Sub- picture[8]. Therefore, in the fourth Sub-picture[4] in the reference frame, the area in which the temporary reference block for the current block may be present may be reduced to the remaining area 2210 except for the filtered area.

[297] Meanwhile, when independently encoded subframes are decoded, even if the lateral filter is not applicable to the boundary of the current subframe, since the lateral filtering is performed at the boundary of an adjacent subframe adjacent to the boundary, such a problem may occur that the encoding result and the decoding result (t .e. decoding error) do not match.

[298] FIG. 23 and 24 are views illustrating examples in which a subframe decoding error occurs.

[299] First, referring to FIG. 23, each first subframe 2310 and second subframe 2320 may be treated as a frame and independently encoded. In an example, the first information indicating whether or not each subframe is treated as a frame may be signaled using the subpic_treated_as_pic_flag described above with reference to FIG. 15.

[300] In-loop filtering across the boundary, i. e. lateral filtering may be performed on a boundary of at least one of the first subframe 2310 and second subframe 2320. For example, lateral filtering may not be performed on a boundary of first subframe 2310, and lateral filtering may be performed on a boundary of second subframe 2320. information indicating whether or not cross filtering can be performed at the boundary of each subframe may be signaled using the loop_filter_across_subpic_enabled_flag described above with reference to FIG. 15.

[301] Whether or not lateral filtering can be performed at the boundary of each subframe may be determined regardless of whether or not each subframe is treated as a frame. Thus, in the above example, the value of loop_filter_across_subpic_enabled_flag can be determined regardless of the value of subpic_treated_as_pic_flag. As a result, even if a lateral filter is not applied to the boundary of the first subframe 2310, since the boundary of the second subframe 2320 is lateral filtered, the boundary of the first subframe 2310 may also be filtered.

[302] Information regarding each subframe may include first information and second information. In addition, information regarding each independently encoded subframe may be signaled via different bitstreams. For example, information 2341 regarding the first subframe 2310 may be signaled via the first stream 2340 bits, and information 2351 regarding the second subframe 2320 may be signaled via the second stream 2350 bits.

[303] Meanwhile, in the decoding step, the image decoding equipment may construct at least one frame including a plurality of subframes by combining the plurality of bit streams. For example, the image decoding equipment may construct one frame including the first subframe 2310 and the second subframe 2320 based on information 2341 regarding the first subframe 2310 obtained from the first bit stream 2340 and information 2351 regarding the second subframe 2320 obtained from the second stream 2350. bits. In this case, according to the decoding order, the second subframe may be present at a position near the right vertical border of the first subframe 2310.

[304] Then, referring to FIG. 24, when the sub-block-based temporal motion vector prediction (TMVP) mode is applied to the current block 2410 included in the first subframe 2310, at least a partial area in the collocation reference block 2420 for the current block 2410 may be included in the filtered area 2430. As as a result, in the encoding step, the temporal reference block 2420 for the current block 2410 does not include the filtered region. However, in the decoding step, the temporal reference block 2420 for the current block 2410 may include the filtered region. Therefore, in the partial area 2411 in the current block 2410 referring to the filtered area, a decoding error may occur in which the decoding result and the encoding result do not match.

[305] As described above, in order to solve the problems described above with reference to FIG. 21-24, according to an embodiment of the present disclosure, in-loop filtering across a current subframe boundary may be performed based on whether or not the current subframe is treated as a frame. In addition, according to an embodiment of the present disclosure, in-loop filtering across a current block boundary may be performed based on whether or not the current block boundary is equal to the current subframe boundary. In addition, according to an embodiment of the present disclosure, in-loop filtering across a boundary of the current subframe may be performed based on whether or not in-loop filtering may be performed through an adjacent subframe adjacent to the boundary.

[306] The following describes exemplary embodiments of the present disclosure in detail with reference to the accompanying drawings.

[307] Embodiment 1

[308] According to Embodiment 1, whether or not lateral filtering can be performed at the boundary of the current subframe may be determined based on whether or not the current subframe is treated as a frame.

[309] FIG. 25 is a view illustrating the SPS syntax for signaling a syntax element for subframes according to an embodiment of the present disclosure. The description of the SPS syntax of FIG. 25 overlaying the SPS syntax of FIG. 15 is omitted or simplified.

[310] Referring to FIG. 25, the subpic_treated_as_pic_flag[i] syntax element may indicate whether or not a subframe is treated as identical to a common frame during decoding. For example, the first value (eg, 0) of subpic_treated_as_pic_flag[i] may indicate that the i-th subframe included in a single CVS encoded frame is not treated as a frame. The second value (eg, 1) of subpic_treated_as_pic_flag[i] may indicate that the i-th subframe included in a single CVS encoded frame is treated as a frame. When the value of subpic_treated_as_pic_flag[i] is not obtained from the bit stream, the value of subpic_treated_as_pic_flag[i] may be inferred as the first value (eg, 0).

[311] The loop_filter_across_subpic_enabled_flag[i] syntax element may indicate whether or not in-loop filtering (eg, deblocking filtering) can be performed across the boundary of the i-th subframe included in a single encoded CVS frame. For example, the first value (eg, 0) of loop_filter_across_subpic_enabled_flag[i] may indicate that in-loop filtering cannot be performed across the boundary of the i-th subframe included in a single CVS encoded frame. The second value (eg, 1) of loop_filter_across_subpic_enabled_flag[i] may indicate that in-loop filtering can be performed across the boundary of the i-th subframe included in a single CVS encoded frame.

[312] loop_filter_across_subpic_enabled_flag[i] may be signaled only when the subframe is not treated as identical to the overall frame during the decoding process. For example, when the i-th subframe included in a single encoded CVS frame is not treated as a frame during decoding (eg, subpic_treated_as_pic_flag[i]=0), loop_filter_across_subpic_enabled_flag[i] may be signaled (2510).

[313] In an example where loop_filter_across_subpic_enabled_flag[i] is not signaled, the value of loop_filter_across_subpic_enabled_flag[i] may be determined to be the first value (eg, 0) or the second value (eg, 1) according to the following condition. When the i-th subframe included in a single encoded CVS frame is treated as a frame in the decoding process (eg, subpic_treated_as_pic_flag[i]=1), the value of loop_filter_across_subpic_enabled_flag[i] may be determined to be the first value (eg, 0). In other cases, the value of loop_filter_across_subpic_enabled_flag[i] may be defined as constituting a second value (eg, 1).

[314] In an example, to match the bitstream when the value of subpic_treated_as_pic_flag[i] is the second value (eg, 1), a restriction may be applied such that the value of loop_filter_across_subpic_enabled_flag[i] is the first value (eg, 0).

[315] FIG. 26 is a view illustrating an example of the luminance sample filtering process based on the SPS syntax of FIG. 25.

[316] Referring to FIG. 26, the luminance samples may be filtered using a long filter based on predetermined filtering constraints.

[317] The luma samples may include a left or top p ₀ P block sample and a right or bottom q ₀ Q block sample based on the target boundary on which the filtering is performed.

[318] In the case of p ₀ P-box samples, when the number nDp of p ₀ P-box samples to be filtered is greater than 0, and at least one of the following constraints is satisfied, the number nDp of p ₀ P-box samples, to be filtered may change to 0.

[319] Constraint 1: The quantization, transformation, and in-loop filtering process is bypassed (eg, cu_transquant_bypass_flag==1) for the current CU including a coding block having a sample of p ₀ P-blocks.

[320] Constraint 2: Palette mode is applied (eg, pred_mode_plt_flag==1) for the current CU including a coding block having a sample of p ₀ P-blocks.

[321] Constraint 3: The edge of the current CU including a coding block having a sample of p ₀ P-blocks is a subframe boundary, and the current CU is treated as a frame (eg, subpic_treated_as_pic_flag[SubPicIdx]==1).

[322] In the case of q ₀ Q-box sampling, when the number nDq of q ₀ Q-box samples to be filtered is greater than 0, and at least one of the following constraints 4 and 5 is satisfied, the number nDq of q ₀ Q samples -blocks to be filtered can change to 0.

[323] Constraint 4: The quantization, transformation, and in-loop filtering process is bypassed (eg, cu_transquant_bypass_flag==1) for the current CU including a coding block having a sample of q ₀ Q-blocks.

[324] Constraint 5: Palette mode is applied (eg, pred_mode_plt_flag==1) for the current CU including a coding block having a sample of q ₀ Q-blocks.

[325] FIG. 27 is a view illustrating another example of the luminance sample filtering process based on the SPS syntax of FIG. 25.

[326] Referring to FIG. 27, the luminance samples may be filtered using a long filter based on predetermined filtering constraints.

[327] The luminance samples may include a left or top p _i P block sample and a right or bottom q _j Q block sample based on the target boundary on which the filtering is performed. Here, i may have a value equal to or greater than 0 and equal to or less than the value obtained by subtracting 1 from the maximum filter length (eg, maxFilterLengthP) for the P-block. In addition, j may have a value equal to or greater than 0 and equal to or less than the value obtained by subtracting 1 from the maximum filter length (eg, maxFilterLengthQ) for the Q-block.

[328] In the case of a sample p _i P-boxes, when at least one of the following constraints is satisfied, the sample value of the filtered sample p _i ' P-boxes may be replaced by the sample value of the input sample p _i P-boxes.

[329] Constraint 1: The quantization, transformation, and in-loop filtering process is bypassed (eg, cu_transquant_bypass_flag==1) for the current CU including a coding block having a sample of p _i P-blocks.

[330] Constraint 2: Palette mode is applied (eg, pred_mode_plt_flag==1) for the current CU including a coding block having a sample of p _i P-blocks.

[331] Constraint 3: The edge of the current CU including a coding block having a sample of p _i P-blocks is a subframe boundary, and the current CU is treated as a frame (eg, subpic_treated_as_pic_flag[SubPicIdx]==1).

[332] According to the filtering constraints described above, when the sample value of the filtered sample p _i 'P-boxes is replaced by the sample value of the input sample p _i P-boxes, the sample p _i P-boxes can be treated as unfiltered.

[333] In the case of sampling q _j Q-blocks, when at least one of the following constraints 4 and 5 is satisfied, the sample value of the filtered sample q _j ' Q-boxes may be replaced by the sample value of the input sample q _j Q-blocks.

[334] Constraint 4: The quantization, transformation, and in-loop filtering process is bypassed (eg, cu_transquant_bypass_flag==1) for the current CU including a coding block having a sample of q _i Q blocks.

[335] Constraint 5: Palette mode is applied (eg, pred_mode_plt_flag==1) for the current CU including a coding block having a sample of q _i Q blocks.

[336] According to the filtering restrictions described above, when the sample value of the filtered sample q _j ' Q-boxes is replaced by the sample value of the input sample Q _j Q-boxes, the q _j Q-blocks sample can be treated as unfiltered.

[337] FIG. 28 is a view illustrating an example of a chrominance sample filtering process based on the SPS syntax of FIG. 25.

[338] Referring to FIG. 28, the chrominance samples may be filtered using a strong filter or a weak filter according to predetermined filtering constraints. In an example, chrominance samples can only be filtered when the corresponding color components (eg, Y, Cb, Cr) are not separately encoded/decoded. For example, chrominance samples can only be filtered when the ChromaArrayType value indicating the type of the chrominance sample array is not equal to 0.

[339] The chrominance samples may include a left and top p _i P block sample and a right or bottom q _i Q block sample based on the target boundary on which the filtering is performed. Here, i may have a value equal to or greater than 0 and equal to or less than the value obtained by subtracting 1 from the maximum filter length (eg, maxFilterLengthCbCr) for the chroma block. The sample p _i P-blocks may be indicated using the first sample position (xPi, yPi), and the sample q _i Q-blocks may be indicated using the second position (xQi, yQi).

[340] Using the sample value and sample position of the p _i P-boxes, the sample value and sample position of the q _i Q-blocks, and the variable t _c as input, the sample value of the filtered sample p _i ' P-boxes and the sample value of the filtered samples of q _i ' Q-blocks.

[341] In an example where the maximum filter length (e.g., maxFilterLengthCbCr) for a chrominance block is 3, strong filtering can be performed on the chrominance samples, as shown in Equation 7 below.

[342] Equation 7

p ₀ '=Clip3(p ₀ -t _C , p ₀ +t _C , (p ₃ +p ₂ +p ₁ +2*p ₀ +q ₀ +q ₁ +q ₂ +4)>>3)

p ₁ '=Clip3(p ₁ -t _C , p ₁ +t _C , (2*p ₃ +p ₂ +2*p ₁ +p ₀ +q ₀ +q ₁ +4)>>3)

p ₂ '=Clip3(p ₂ -t _C , p ₂ +t _C , (3*p ₃ +2*p ₂ +p ₁ +p ₀ +q ₀ +4)>>3)

q ₀ '=Clip3(q ₀ -t _C , q ₀ +t _C , (p ₂ +p ₁ +p ₀ +2*q ₀ +q ₁ +q ₂ +q ₃ +4)>>3)

q ₁ '=Clip3(q ₁ -t _C , q ₁ +t _C , (p ₁ +p ₀ +q ₀ +2*q ₁ +q ₂ +2*q ₃ +4)>>3)

q ₂ '=Clip3(q ₂ -t _C , q ₂ +t _C , (p ₀ +q ₀ +q ₁ +2*q ₂ +3*q ₃ +4)>>3),

[343] where Clip3(x, y, z) can mean a function for clipping the z parameter value for a value between the x parameter value and the y parameter value.

[344] In contrast, when the maximum filter length (e.g., maxFilterLengthCbCr) for a chrominance block is not 3, weak filtering can be performed on the chrominance samples, as shown in Equation 8 below.

[345] Equation 8

Δ=Clip3(-t _C , t _C , ((((q ₀ -p ₀ )<<2)+p ₁ -q ₁ +4)>>3))

p ₀ '=Clip1 _C (p ₀ +Δ)

q ₀ '=Clip1 _C (q ₀ -Δ),

[346] where Clip1C(x) may denote a function for clipping the value of parameter x based on the bit depth of the chroma block.

[347] Meanwhile, the sample value of the filtered sample p _i ' P-blocks may be replaced by the sample value of the input sample p _i P-blocks when at least one of the following constraints 1-3 is satisfied.

[348] Constraint 1: The quantization, transformation, and in-loop filtering process is bypassed (eg, cu_transquant_bypass_flag==1) for the current CU including a coding block having a sample of p _i P-blocks.

[349] Constraint 2: Palette mode is applied (eg, pred_mode_plt_flag==1) for the current CU including a coding block having a sample of p _i P-blocks.

[350] Constraint 3: The edge of the current CU including a coding block having a sample of p _i P-blocks is a subframe boundary, and the current CU is treated as a frame (eg, subpic_treated_as_pic_flag[SubPicIdx]==1).

[351] According to the filtering restrictions described above, when the sample value of the filtered sample p _i 'P-boxes is replaced by the sample value of the input sample p _i P-boxes, the sample p _i P-boxes can be treated as unfiltered.

[352] The sample value of the filtered sample q _j ' Q-blocks may be replaced by the sample value of the input sample q _j Q-blocks when at least one of the constraints 4 and 5 below is satisfied.

[353] Constraint 4: The quantization, transformation, and in-loop filtering process is bypassed (eg, cu_transquant_bypass_flag==1) for the current CU including a coding block having a sample of q _i Q blocks.

[354] Constraint 5: Palette mode is applied (eg, pred_mode_plt_flag==1) for the current CU including a coding block having a sample of q _i Q blocks.

[355] According to the filtering restrictions described above, when the sample value of the filtered sample q _j ' of Q-blocks is replaced by the sample value of the input sample Q _j of Q-blocks, the sample of q _j Q-blocks can be treated as unfiltered.

[356] Hereinafter, the filtering method according to Embodiment 1 of the present disclosure is described in detail.

[357] FIG. 29 is a flowchart illustrating a filtering method based on a subframe structure according to an embodiment of the present disclosure. The filtering method according to Fig. 29 may be performed by the image encoding equipment of FIG. 2. For example, steps S2910-S2940 may be performed by the filter 160 of the image encoding equipment 100 . Alternatively, the filtering method of FIG. 29 may be performed by the image decoding equipment of FIG. 3. For example, steps S2910-S2940 may be performed by the filter 240 of the image decoding equipment 200 .

[358] In the encoding step, the filtering method of FIG. 29 may be performed across the boundary of the current frame, which includes the reconstructed current block. Alternatively, in the decoding step, the filtering method of FIG. 29 may be performed across the boundary of the current frame including the decoded current block. Hereinafter, in-loop filtering across a subframe boundary is referred to as "cross-filtering".

[359] Referring to FIG. 29, the image encoding/decoding equipment can determine whether or not the current subframe including the current block can be treated as a frame (S2910).

[360] Treating the current subframe as a frame may mean that the current subframe is independently encoded/decoded without regard to encoding/decoding information of other subframes. For example, the current subframe may be encoded using an inter prediction mode, and an adjacent subframe adjacent to the current subframe may be encoded/decoded using an intra prediction mode.

[361] In the encoding step, information indicating whether or not the current subframe can be treated as a frame may be encoded using subpic_treated_as_pic_flag in a high-level syntax (eg, SPS (sequence parameter set) syntax). For example, when subpic_treated_as_pic_flag has a first value (eg, 0), the current subframe may not be treated as a frame. Conversely, when subpic_treated_as_pic_flag has a second value (eg, 1), the current subframe may be treated as a frame.

[362] In the decoding step, whether or not the current subframe can be treated as a frame may be determined using the subpic_treated_as_pic_flag value obtained from the high-level syntax. In an example where subpic_treated_as_pic_flag fails, subpic_treated_as_pic_flag may be inferred as having the first value (eg, 0).

[363] When the current subframe is not treated as a frame ("No" in S2910), it may be determined whether or not cross-sectional filtering (eg, deblocking filtering) can be performed at the boundary of the current subframe (S2920).

[364] In the encoding step, information indicating whether or not cross-sectional filtering can be performed at the boundary of the current subframe may be encoded using loop_filter_across_subpic_enabled_flag in a high-level syntax (for example, in SPS (sequence parameter sets) syntax). For example, when loop_filter_across_subpic_enabled_flag has a first value (eg, 0), cross filtering may not be performed at the boundary of the current subframe. In contrast, when loop_filter_across_subpic_enabled_flag has a second value (eg, 1), lateral filtering may be performed at the boundary of the current subframe.

[365] At the decoding stage, whether or not cross filtering can be performed at the boundary of the current subframe may be determined based on the loop_filter_across_subpic_enabled_flag value obtained from the high-level syntax. In an example, when loop_filter_across_subpic_enabled_flag fails, the value of loop_filter_across_subpic_enabled_flag may be determined based on whether or not the current subframe is treated as a frame. For example, when the current subframe is treated as a frame, loop_filter_across_subpic_enabled_flag may be inferred to have a first value (eg, 0). Conversely, when the current subframe is not treated as a frame, loop_filter_across_subpic_enabled_flag may be inferred as having a second value (eg, 1). In an example, to match the bitstream when subpic_treated_as_pic_flag has a second value (eg, 1), a restriction may be applied such that loop_filter_across_subpic_enabled_flag has a first value (eg, 0).

[366] When lateral filtering can be performed at the boundary of the current subframe ("Yes" in S2920), the image encoding/decoding equipment may determine the boundary intensity of the current subframe (S2930). In the present disclosure, the boundary intensity may be referred to as "edge filtering intensity" or "filtering intensity".

[367] The boundary intensity of the current subframe may be determined as being any of a first value (eg, 0) to a third value (eg, 2) according to a predetermined condition, and a specific determination method is described above with reference to FIG. 19.

[368] The image encoding/decoding equipment may perform lateral filtering at the border of the current subframe based on the boundary intensity of the current subframe (S2940). For example, lateral filtering may be performed at the border of the current subframe only when the edge intensity of the current subframe has a second value (eg, 1) or a third value (eg, 2).

[369] Meanwhile, when lateral filtering cannot be performed at the boundary of the current subframe ("No" in S2920), the filtering process of S2930 and S2940 for the boundary of the current subframe may be skipped.

[370] In addition, when the current subframe is treated as a frame ("Yes" in S2910), the filtering processes in S2930 and S2940 for the boundary of the current subframe may be skipped.

[371] As described above, according to Embodiment 1 of the present disclosure, in-loop filtering across the boundary of the current subframe can only be performed when the current subframe is not treated as a frame. Therefore, it is possible to prevent the reduction of the area in which the current subframe is treated as a frame.

[372] Embodiment 2

[373] According to Embodiment 2, in-loop filtering across the current block boundary can be performed only when a predetermined filtering constraint is not satisfied. For example, when the left or top boundary of the current block is the left or top boundary of the current subframe, or when in-loop filtering cannot be performed across the boundary of the current sub-frame, in-loop filtering through the boundary of the current block may not be performed. Additionally, since in-loop filtering across an adjacent subframe boundary is performed when a current subframe boundary is filtered together, according to a predetermined filtering constraint, the number of samples to be filtered at the current subframe boundary may change to 0, or sample values of the filtered samples may be replaced by input sample values. values.

[374] FIG. 30 is a view illustrating a cross-filtering process according to an embodiment of the present disclosure.

[375] Referring to FIG. 30, in-loop filtering across the boundary of the current block, such as one-way filtering to deblock, can be performed. In the example, whether or not filtering to deblock across the current block boundary may be performed may be signaled using filterEdgeFlag. For example, the first value (eg, 0) of filterEdgeFlag may indicate that deblocking filtering across the current block boundary cannot be performed. In contrast, a second value (eg, 1) of filterEdgeFlag may indicate that deblocking filtering across the current block boundary may be performed.

[376] The filterEdgeFlag value can be retrieved as follows.

[377] When the current block boundary is a vertical boundary (eg, edgeType==EDGE_VER), if at least one of the following constraints 1-5 is satisfied, filterEdgeFlag may have a first value (eg, 0).

[378] Constraint 1: The left border of the current coding block is the left border of the current frame.

[379] Constraint 2: The left boundary of the current coding block is the left boundary of the current subframe, and deblocking filtering across the boundary of the current subframe cannot be performed (eg, loop_filter_across_subpic_enabled_flag==0).

[380] Constraint 3: The left border of the current coding block is the left border of the current tile, and in-loop filtering cannot be performed across the border of the current tile (eg, loop_filter_across_tiles_enabled_flag==0).

[381] Constraint 4: The left edge of the current coding block is the left edge of the current slice, and in-loop filtering across the edge of the current slice cannot be performed (eg, loop_filter_across_slices_enabled_flag==0).

[382] Constraint 5: The left boundary of the current coding block is one of the vertical virtual boundaries of the current frame, and in-loop filtering cannot be performed across virtual boundaries (eg, VirtualBoundariesDisabledFlag==1).

[383] When the current block boundary is a horizontal boundary (eg, edgeType==EDGE_HOR), if at least one of constraints 6-7 is satisfied, filterEdgeFlag may have a first value (eg, 0).

[384] Constraint 6: The upper bound of the current coding block is the upper bound of the current frame.

[385] Constraint 7: The upper bound of the current coding block is the upper bound of the current subframe, and deblocking filtering across the boundary of the current block cannot be performed (eg, loop_filter_across_subpic_enabled_flag==0).

[386] Constraint 8: The upper bound of the current coding block is the upper bound of the current tile, and deblocking filtering across the boundary of the current tile cannot be performed (eg, loop_filter_across_tiles_enabled_flag==0).

[387] Constraint 9: The upper bound of the current coding block is the upper bound of the current slice, and deblocking filtering across the boundary of the current slice cannot be performed (eg, loop_filter_across_slices_enabled_flag=0).

[388] Constraint 10: The upper bound of the current coding block is one of the horizontal virtual bounds of the current frame, and in-loop filtering across the virtual bounds cannot be performed (eg, VirtualBoundariesDisabledFlag==1).

[389] When all of the above constraints 1-10 are not satisfied, filterEdgeFlag may have a second value (eg, 1). For example, when the left boundary of the current coding block is the left or right boundary of the current subframe, and filtering to deblock across the boundary of the current block cannot be performed (eg, loop_filter_across_subpic_enabled_flag[SubPicIdx]==0), filterEdgeFlag may have a second value (eg, 1 ). In addition, when the upper boundary of the current coding block is the upper or lower boundary of the current subframe, and deblocking filtering across the boundary of the current block cannot be performed, filterEdgeFlag may have a second value (eg, 1). The above examples may consider the case in which in-loop filtering across the boundary is performed for an adjacent subframe adjacent to the boundary of the current coding block.

[390] Meanwhile, the current coding block may include luma samples and chrominance samples. Each of the luma samples and the chrominance samples may include a left or top p ₀ P block sample and a right or bottom q ₀ Q block sample based on the target boundary on which the filtering is performed. Alternatively, each of the luma samples and the chrominance samples may include a left or top p _i P block sample and a right or bottom q _j Q block sample based on the target boundary on which the filtering is performed. Here, i may have a value equal to or greater than 0 and equal to or less than the value obtained by subtracting 1 from the maximum filter length (eg, maxFilterLengthP) for the P-block. In addition, j may have a value equal to or greater than 0 and equal to or less than the value obtained by subtracting 1 from the maximum filter length (eg, maxFilterLengthQ) for the Q-block. Hereinafter, a process for performing cross-sectional filtering on luma samples and chrominance samples is described in detail with reference to FIG. 31-33.

[391] FIG. 31 and 32 are views illustrating a cross-filtering process for luminance samples according to an embodiment of the present disclosure.

[392] First, referring to FIG. 31, the number of luminance samples to be filtered using the short filter may change to 0 according to predetermined filtering restrictions. In addition, luma samples filtered using a short filter may be treated as unfiltered according to predetermined filtering constraints.

[393] For example, when the number nDp of p ₀ P-block samples to be filtered is greater than 0, and the palette mode is applied to the current CU including a coding block having a p ₀ P-block sample (for example, pred_mode_plt_flag==1 ), the number nDp of samples p ₀ of P-boxes to be filtered may change to 0.

[394] In addition, when the number nDq of samples q ₀ of Q-blocks to be filtered is greater than 0, and the palette mode is applied to the current CU including a coding block having a sample of q ₀ Q-blocks, the number nDq of samples q ₀ Q-blocks to be filtered may change to 0.

[395] In addition, when in-loop filtering cannot be performed across the boundary of a subframe including the sample p_i P-blocks (e.g. loop_filter_across_subpic_enabled_flag[subPicIdxP]==0), sample sample value p_i' P-blocks filtered using sample q_j Q-blocks, can be replaced by the sample value of the input sample p_i P-blocks.

[396] Then, referring to FIG. 32, luma samples filtered using a long filter may be treated as unfiltered according to predetermined filtering constraints.

[397] For example, when palette mode is applied to a current CU including a coding block having a sample of p _i P-boxes (eg, pred_mode_plt_flag==1), the sample value of the filtered sample of p _i 'P-boxes may be replaced by the sample value of the input samples of p _i P-blocks.

[398] In addition, when the palette mode is applied to the current CU including a coding block having a sample of q _j Q-blocks (eg, pred_mode_plt_flag==1), the sample value of the filtered sample of q _j ' Q-blocks may be replaced by the sample value input sample of q _j Q-blocks.

[399] In addition, when in-loop filtering across the boundary of a subframe including sample p_i P-blocks cannot be executed (e.g. loop_filter_across_subpic_enabled_flag[subPicIdxP]==0), sample sample value p_i' P-blocks filtered using sample q_j Q-blocks, can be replaced by the sample value of the input sample p_i P-blocks.

[400] FIG. 33 is a view illustrating a cross-filtering process for chrominance samples according to an embodiment of the present disclosure.

[401] Referring to FIG. 33, filtered chrominance samples may be treated as unfiltered according to predetermined filtering constraints.

[402] For example, when the palette mode is applied to the current CU including a coding block having a sample of p _i P-boxes (eg, pred_mode_plt_flag==1), the sample value of the filtered sample of p _i 'P-boxes may be replaced by the sample value of the input samples of p _i P-blocks.

[403] In addition, when the palette mode is applied to the current CU including a coding block having a sample of q _j Q-boxes, the sample value of the filtered sample of q _j ' Q-boxes may be replaced by the sample value of the input sample of q _j Q-blocks.

[404] In addition, when in-loop filtering across the boundary of a subframe including sample p_i P-blocks cannot be executed (e.g. loop_filter_across_subpic_enabled_flag[subPicIdxP]==0), sample sample value p_i' P-blocks filtered using sample q_j Q-blocks, can be replaced by the sample value of the input sample p_i P-blocks.

[405] Hereinafter, this document describes in detail the filtering method according to embodiment 2 of the present disclosure.

[406] FIG. 34 is a flowchart illustrating a filtering method based on a subframe structure according to an embodiment of the present disclosure. The filtering method according to Fig. 34 may be performed by the image encoding equipment of FIG. 2. For example, steps S3410-S3440 may be performed by the filter 160 of the image encoding equipment 100 . Alternatively, the filtering method of FIG. 34 may be performed by the image decoding equipment of FIG. 3. For example, steps S3410-S3440 may be performed by the filter 240 of the image decoding equipment 200 .

[407] In the encoding step, the filtering method of FIG. 34 may be across a current frame boundary including the reconstructed current block. Alternatively, in the decoding step, the filtering method of FIG. 34 may be across the boundary of the current frame including the decoded current block. Hereinafter, in-loop filtering across a subframe boundary is referred to as "cross-filtering".

[408] Referring to FIG. 34, the image encoding/decoding equipment can determine whether or not a predetermined filtering constraint for the current block boundary is satisfied (S3410). Here, the restriction on filtering may mean that cross-sectional filtering cannot be performed on the boundary of the current block.

[409] A specific example of filtering restriction will be described with reference to FIG. 30-33. For example, the filtering constraint may include the case where the left boundary of the current block is the left boundary of the current subframe and cross-sectional filtering cannot be performed on the boundary of the current subframe (eg, loop_filter_across_subpic_enabled_flag[SubPicIdx]==0). In addition, the filtering constraint may include a case in which the upper boundary of the current block is the upper boundary of the current subframe, and cross-sectional filtering cannot be performed on the boundary of the current subframe. In addition, the filtering constraint may include a case in which the current block includes a sample of p _i P luminance component blocks located to the left or top based on the boundary of the current block, and cross-sectional filtering cannot be performed on the boundary of the current subframe. In addition, the filtering constraint may include a case in which the current block includes a left or top sample of q _i Q luminance component blocks based on the current block boundary, and cross-sectional filtering cannot be performed on the current block boundary. subframe.

[410] When the filtering constraint is not satisfied for the boundary of the current block ("No" in S3410), the image encoding/decoding equipment may determine the boundary intensity of the current block (S3420). In the present disclosure, the boundary intensity may be referred to as "edge filtering intensity" or "filtering intensity".

[411] The boundary intensity of the current subframe may be determined as a component of any of the first value (eg, 0) to the third value (eg, 2) according to a predetermined condition, and a specific determination method is described above with reference to FIG. 19.

[412] The image encoding/decoding equipment may perform lateral filtering at the border of the current subframe based on the edge intensity of the current subframe (S3440). For example, lateral filtering may be performed at the border of the current subframe only when the edge intensity of the current subframe has a second value (eg, 1) or a third value (eg, 2).

[413] Meanwhile, when the filtering constraint is satisfied for the boundary of the current subframe ("No" in S3420), the filtering process of S3420 and S3430 for the boundary of the current subframe may be skipped.

[414] As described above, according to Embodiment 2 of the present disclosure, in-loop filtering across the boundary of the current subframe can only be performed when a predetermined filtering constraint is not satisfied. Therefore, it is possible to prevent a decoding error such that the decoding result of the current block boundary does not match the encoding result.

[415] Embodiment 3

[416] According to Embodiment 3, in-loop filtering across the current block boundary can be performed only when a predetermined filtering constraint is not satisfied. For example, when in-loop filtering cannot be performed across a boundary of the current subframe including a luma sample or a chrominance sample located to the left or top of the current block, in-loop filtering across the current block boundary may not be performed. In addition, when in-loop filtering cannot be performed across a boundary of the current subframe including a luma sample or a chrominance sample located to the right or below the current block, in-loop filtering across the current block boundary may not be performed.

[417] FIG. 35 is a view illustrating a cross-filtering process according to an embodiment of the present disclosure.

[418] Referring to FIG. 35, in-loop filtering across the current block boundary, such as one-way deblocking filtering, can be performed. In an example, whether or not filtering to deblock across the current block boundary may be performed may be signaled using the filterEdgeFlag described above with reference to FIG. thirty.

[419] The filterEdgeFlag value can be retrieved as follows.

[420] When the current block boundary is a vertical boundary (eg, edgeType==EDGE_VER), if at least one of the following constraints 1-4 is satisfied, filterEdgeFlag may have a first value (eg, 0).

[421] Constraint 1: The left border of the current coding block is the left border of the current frame.

[422] Constraint 2: The left border of the current coding block is the left border of the current tile, and in-loop filtering across the border of the current tile cannot be performed (eg, loop_filter_across_tiles_enabled_flag==0).

[423] Constraint 3: The left edge of the current coding block is the left edge of the current slice, and in-loop filtering across the edge of the current slice cannot be performed (eg, loop_filter_across_slices_enabled_flag==0).

[424] Constraint 4: The left border of the current coding block is one of the vertical virtual borders of the current frame, and in-loop filtering across the virtual borders cannot be performed (eg, VirtualBoundariesDisabledFlag==1).

[425] When the current block boundary is a horizontal boundary (eg, edgeType==EDGE_HOR), if at least one of the constraints 5-8 below is satisfied, filterEdgeFlag may have a first value (eg, 0).

[426] Constraint 5: The upper bound of the current coding block is the upper bound of the current frame.

[427] Constraint 6: The upper bound of the current coding block is the upper bound of the current tile, and in-loop filtering across the boundary of the current tile cannot be performed (eg, loop_filter_across_tiles_enabled_flag==0).

[428] Constraint 7: The upper bound of the current coding block is the upper bound of the current slice, and in-loop filtering across the boundary of the current slice cannot be performed (eg, loop_filter_across_slices_enabled_flag==0).

[429] Constraint 8: The upper bound of the current coding block is one of the horizontal virtual bounds of the current frame, and in-loop filtering across the virtual bounds cannot be performed (eg, VirtualBoundariesDisabledFlag==1).

[430] When all of the above constraints 1-8 are not satisfied, filterEdgeFlag may have a second value (eg, 1).

[431] In the cross-filtering process of FIG. 35, in contrast to the cross-filtering process of FIG. 30, even if deblocking filtering across the current block boundary cannot be performed, in-loop filtering across the current block boundary can be performed. This may be done considering the case in which in-loop filtering is performed through an adjacent subframe boundary adjacent to the current subframe boundary.

[432] Meanwhile, the current coding block may include luma samples and chrominance samples. Each of the luma samples and the chrominance samples may include a left or top p ₀ P block sample and a right or bottom q ₀ Q block sample based on the target boundary on which the filtering is performed. Alternatively, each of the luma samples and the chrominance samples may include a left or top p _i P block sample and a right or bottom q _j Q block sample based on the target boundary on which the filtering is performed. Here, i may have a value equal to or greater than 0 and equal to or less than the value obtained by subtracting 1 from the maximum filter length (eg, maxFilterLengthP) for the P-block. In addition, j may have a value equal to or greater than 0 and equal to or less than the value obtained by subtracting 1 from the maximum filter length (eg, maxFilterLengthQ) for the Q-block. Hereinafter, a process for performing cross-sectional filtering on luma samples and chrominance samples is described in detail with reference to FIG. 34-36.

[433] FIG. 36 and 37 are views illustrating a cross-filtering process for luminance samples according to an embodiment of the present disclosure.

[434] First, referring to FIG. 36, the number of luminance samples to be filtered using the short filter may change to 0 according to predetermined filtering restrictions. In addition, luma samples filtered using a short filter may be treated as unfiltered according to predetermined filtering constraints.

[435] For example, when the number nDp of p ₀ P-block samples to be filtered is greater than 0, and the palette mode is applied to the current CU including a coding block having a p ₀ P-block sample (eg, pred_mode_plt_flag==1 ), the number nDp of samples p ₀ of P-boxes to be filtered may change to 0.

[436] In addition, when the number nDq of samples q ₀ of Q-blocks to be filtered is greater than 0, and the palette mode is applied to the current CU including a coding block having a sample of q ₀ Q-blocks, the number nDq of samples q ₀ Q-blocks to be filtered may change to 0.

[437] In addition, when in-loop filtering across the boundary of a subframe including sample p_i P-blocks cannot be executed (e.g. loop_filter_across_subpic_enabled_flag[subPicIdxP]==0), sample sample value p_i' P-blocks filtered using sample q_j Q-blocks, can be replaced by the sample value of the input sample p_i P-blocks.

[438] In addition, when in-loop filtering cannot be performed across the boundary of a subframe including sample q_j Q-blocks (e.g. loop_filter_across_subpic_enabled_flag[subPicIdxQ]==0), sample sample value q_j' Q-boxes filtered using sample p_i P-boxes, can be replaced by the sample value of the input sample q_j Q-blocks.

[439] Then, referring to FIG. 37, luma samples filtered using a long filter may be treated as unfiltered according to predetermined filtering constraints.

[440] For example, when the palette mode is applied to the current CU including a coding block having a sample of p _i P-boxes (eg, pred_mode_plt_flag==1), the sample value of the filtered sample of p _i 'P-boxes may be replaced by the sample value of the input samples of p _i P-blocks.

[441] In addition, when the palette mode is applied to the current CU including a coding block having a sample of q _j Q-boxes, the sample value of the filtered sample of q _j ' Q-boxes may be replaced by the sample value of the input sample of q _j Q-blocks.

[442] In addition, when in-loop filtering cannot be performed across the boundary of a subframe including sample p_i P-blocks (e.g. loop_filter_across_subpic_enabled_flag[subPicIdxP]==0), sample sample value p_i' P-blocks filtered using sample q_j Q-blocks, can be replaced by the sample value of the input sample p_i P-blocks.

[443] In addition, when in-loop filtering across the boundary of a subframe including sample q_j Q-blocks, cannot be executed (e.g. loop_filter_across_subpic_enabled_flag[subPicIdxQ]==0), sample sample value q_j' Q-boxes filtered using sample p_i P-boxes, can be replaced by the sample value of the input sample q_j Q-blocks.

[444] FIG. 38 is a view illustrating a cross-filtering process for chrominance samples according to an embodiment of the present disclosure.

[445] Referring to FIG. 38, filtered chrominance samples may be treated as unfiltered according to predetermined filtering constraints.

[446] For example, when palette mode is applied to a current CU including a coding block having a sample of p _i P-boxes (eg, pred_mode_plt_flag==1), the sample value of the filtered sample of p _i 'P-boxes may be replaced by the sample value of the input samples of p _i P-blocks.

[447] In addition, when the palette mode is applied to the current CU including a coding block having a sample of q _j Q-blocks (eg, pred_mode_plt_flag==1), the sample value of the filtered sample of q _j ' Q-blocks may be replaced by the sample value input sample of q _j Q-blocks.

[448] In addition, when in-loop filtering across the boundary of a subframe including sample p_i P-blocks cannot be executed (e.g. loop_filter_across_subpic_enabled_flag[subPicIdxP]==0), sample sample value p_i' P-blocks filtered using sample q_j Q-blocks, can be replaced by the sample value of the input sample p_i P-blocks.

[449] In addition, when in-loop filtering across the boundary of a subframe including sample q_j Q-blocks, cannot be executed (e.g. loop_filter_across_subpic_enabled_flag[subPicIdxQ]==0), sample sample value q_j' Q-boxes filtered using sample p_i P-boxes, can be replaced by the sample value of the input sample q_j Q-blocks.

[450] The filtering method according to Embodiment 3 of the present disclosure differs from the filtering method according to Embodiment 2 in restrictions on filtering, and the filtering method described above with reference to FIG. 34 is applicable without change. Accordingly, its description is omitted.

[451] As described above, according to Embodiment 3 of the present disclosure, in-loop filtering across the boundary of the current subframe can only be performed when a predetermined filtering constraint is not satisfied. Therefore, it is possible to prevent a decoding error such that the decoding result of the current block boundary does not match the encoding result.

[452] Embodiment 4

[453] According to Embodiment 4, in-loop filtering across a current block boundary can only be performed when in-loop filtering through each boundary can be performed both on a current frame boundary and on an adjacent frame boundary adjacent to the current block boundary.

[454] FIG. 39 is a view illustrating a cross-filtering process according to an embodiment of the present disclosure.

[455] Referring to FIG. 39, in-loop filtering across a current block boundary (eg, deblocking filtering) can be performed only when a predetermined filtering constraint is not satisfied. Here, the border of the current box may include all edges of the sub-box and all edges of the transform box.

[456] Constraint 1: The above-described edges are included in the frame boundary.

[457] Constraint 2: The above-described edges are included in the upper boundary of the current subframe, and in-loop filtering through the boundary of an adjacent subframe adjacent to the upper boundary cannot be performed (for example, loop_filter_across_subpic_enabled_flag[SubPicIdxAbove]==0), where the position of the upper right sample of the current block is (xCtb, yCtb), and the position of an adjacent subframe may be indicated using the first sample position (xCtb, yCtb-1).

[458] Constraint 3: The above-described edges are included in the left border of the current subframe, and in-loop filtering through the border of an adjacent subframe adjacent to the left border cannot be performed (for example, loop_filter_across_subpic_enabled_flag[SubPicIdxLeft]==0), where the top left sample position of the current block is (xCtb, yCtb), and the position of the adjacent subframe may be indicated using the second sample position (xCtb-1, yCtb).

[459] Constraint 4: The above-described edges are included in the left or top boundary of the current subframe, and filtering to deblock across the current block boundary cannot be performed (e.g., loop_filter_across_subpic_enabled_flag[SubPicIdx]==0), where the position of the current subframe can be indicated using position (xCtb, yCtb) upper left sample of the current block.

[460] Constraint 5: The above described edges are included in the virtual boundaries of the current frame, and in-loop filtering across the virtual boundaries cannot be performed (eg, VirtualBoundariesDisabledFlag==1).

[461] Constraint 6: The edges described above are included in the boundaries of the current tile, and in-loop filtering across the boundaries of the current tile cannot be performed (eg, loop_filter_across_tiles_enabled_flag==0).

[462] Constraint 7: The edges described above are included in the boundaries of the current slice, and in-loop filtering across the boundaries of the current slice cannot be performed (eg, loop_filter_across_slices_enabled_flag==0).

[463] Constraint 8: The edges described above are included in the top or bottom of the current slice, and deblocking filtering is disabled for the current slice (eg, slice_deblocking_filter_disabled_flag==1).

[464] Constraint 9: The edges described above are included in the current slice, and deblocking filtering is disabled for the current slice (eg, slice_deblocking_filter_disabled_flag==1).

[465] Constraint 10: The above-described edges do not correspond to the boundary of the 4×4 luma sample grid.

[466] Constraint 11: The above-described edges do not correspond to the boundary of the 8x8 chrominance sample grid.

[467] Constraint 12: As luma component edges, block-based delta pulse code modulation (BDPCM) is applied to both sides of each edge (e.g., intra_bdpcm_luma_flag==1), where BDPCM may mean a coding mode in which the quantized the residual sample value of the current TU is modified using the quantized residual sample value of the TU adjacent to the current TU.

[468] Constraint 13: As chrominance component edges, block-based delta pulse code modulation (BDPCM) is applied to both sides of each edge (eg, intra_bdpcm_chroma_flag==1).

[469] Constraint 14: The edges of the chrominance component subblocks are not equal to the edges of the TU associated with them.

[470] Meanwhile, the filtering process of luminance samples and chrominance samples according to Embodiment 2 described above with reference to FIG. 31-33 is applicable to Embodiment 4 of the present disclosure. In addition, the filtering process of luma samples and chrominance samples according to Embodiment 3 described above with reference to FIG. 36-38 is applicable to Embodiment 4 of the present disclosure. For example, when the prediction mode of the current block is palette mode, in-loop filtering (eg, deblocking filtering) across the boundary of the current block may not be performed.

[471] Hereinafter, the filtering method according to Embodiment 4 of the present disclosure is described in detail.

[472] FIG. 40 is a flowchart illustrating a filtering method based on a subframe structure according to an embodiment of the present disclosure. The filtering method according to Fig. 40 may be performed by the image encoding equipment of FIG. 2. For example, steps S4010-S4040 may be performed by the filter 160 of the image encoding equipment 100 . Alternatively, the filtering method of FIG. 40 may be performed by the image decoding equipment of FIG. 3. For example, steps S4010-S4040 may be performed by the filter 240 of the image decoding equipment 200 .

[473] In the encoding step, the filtering method of FIG. 40 may be across the boundary of the reconstructed current block. Alternatively, in the decoding step, the filtering method of FIG. 40 may be across a decoded current block boundary. Hereinafter in this document, in-loop filtering across the boundary of the current block (or current frame/subframe/slice/tile, etc.) is referred to as "transverse filtering".

[474] Referring to FIG. 40, the image encoding/decoding equipment can determine whether or not cross filtering can be performed at the boundary of the current subframe including the current block (S4010).

[475] In the encoding step, information indicating whether or not cross filtering can be performed at the boundary of the current subframe may be encoded using loop_filter_across_subpic_enabled_flag in a high-level syntax (eg, sequence parameter set (SPS) syntax). For example, when loop_filter_across_subpic_enabled_flag has a first value (eg, 0), cross filtering cannot be performed at the boundary of the current subframe. In contrast, when loop_filter_across_subpic_enabled_flag has a second value (eg, 1), lateral filtering may be performed at the boundary of the current subframe.

[476] In the decoding step, whether or not cross filtering can be performed at the boundary of the current subframe may be determined based on the loop_filter_across_subpic_enabled_flag value obtained from the high-level syntax. In an example, when loop_filter_across_subpic_enabled_flag fails, the value of loop_filter_across_subpic_enabled_flag may be determined based on whether or not the current subframe is treated as a frame. For example, when the current subframe is treated as a frame, loop_filter_across_subpic_enabled_flag may be inferred to have a first value (eg, 0). Conversely, when the current subframe is not treated as a frame, loop_filter_across_subpic_enabled_flag may be inferred as having a second value (eg, 1). In an example, to match the bitstream when subpic_treated_as_pic_flag has a second value (eg, 1), a constraint may be applied such that loop_filter_across_subpic_enabled_flag must have a first value (eg, 0).

[477] When lateral filtering can be performed on the boundary of the current subframe ("Yes" in S4010), the image encoding/decoding equipment can determine whether or not the lateral filtering can be performed on the boundary of an adjacent subframe adjacent to the boundary of the current block (S4020).

[478] Similar to the current subframe, information indicating whether or not cross filtering can be performed on the boundary of an adjacent subframe may be encoded/decoded using loop_filter_across_subpic_enabled_flag in a high-level syntax (e.g., Sequence Parameter Set (SPS) syntax).

[479] When lateral filtering can be performed at the boundary of an adjacent subframe ("Yes" in S4020), the image encoding/decoding equipment may determine that lateral filtering is performed at the boundary of the current block and determine the boundary intensity of the current block (S4030). In the present disclosure, the boundary intensity may be referred to as "edge filtering intensity" or "filtering intensity".

[480] The boundary intensity of the current block may be determined as a component of any of the first value (eg, 0) to the third value (eg, 2) according to a predetermined condition, and a specific determination method is described above with reference to FIG. 19.

[481] Meanwhile, in the example, even if lateral filtering may be performed on both the boundary of the current subframe and the boundary of the adjacent subframe ("Yes" in S4020), when the predetermined filtering constraint is satisfied, the lateral filtering may not be performed on the boundary current block. Thus, when the filtering constraint is satisfied, the filtering process of S4030 and S404 of the current block boundary may be skipped.

[482] The filtering constraint may include the constraints 1-13 described above with reference to FIG. 39. For example, a filtering constraint may include the case in which the edges (or boundaries) of the current block are included in a frame boundary. In addition, the filtering constraint may include a case in which the edges of the current block are included in the left or top boundary of the current subframe. In addition, the filtering constraint may include the case where the edges of the current block are included in the virtual boundaries of the current frame, and cross-sectional filtering cannot be performed on the virtual boundaries of the current frame (eg, VirtualBoundariesDisabledFlag==1). In addition, the filtering constraint may include the case where the edges of the current block are included in the boundaries of the current tile, and cross filtering cannot be performed on the boundaries of the current tile (eg, loop_filter_across_tiles_enabled_flag==0). In addition, the filtering constraint may include the case in which the edges of the current block are included in the boundaries of the current slice, and cross-sectional filtering cannot be performed on the boundaries of the current slice (eg, loop_filtering_across_slices_enabled_flag==0). In addition, a filtering constraint may include the case where the edges of the current block are included in the top or bottom of the current slice, and cross-sectional filtering (eg, filtering to deblock) is disabled for the top or bottom of the current slice (eg, slice_deblocking_filter_disabled_flag==1 ).

[483] The image encoding/decoding equipment may perform lateral filtering at the boundary of the current block, based on the boundary intensity of the current block determined in step S4030 (S4040). For example, lateral filtering of the current block boundary can only be performed when the current block boundary intensity has a second value (eg, 1) or a third value (eg, 2).

[484] On the contrary, when cross filtering cannot be performed on the current subframe boundary ("No" in S4010), the filtering processes on S4030 and S404 of the current block boundary may be skipped. In addition, when lateral filtering cannot be performed on a boundary of an adjacent subframe adjacent to a boundary of the current subframe (No in S4020), the filtering processes of S4030 and S4040 of the current block boundary may be skipped.

[485] Meanwhile, although in FIG. 40, step S4010 is shown as being performed before step S4020, this is an example, and the embodiments of the present disclosure are not limited to this. For example, step S4020 may be performed before step S4010, or may be performed simultaneously with step S4010.

[486] As described above, according to Embodiment 4 of the present disclosure, in-loop filtering across the boundary of the current subframe can only be performed when in-loop filtering through each boundary can be performed on both the boundary of the current frame and the boundary of an adjacent boundary. Therefore, it is possible to prevent a decoding error such that the decoding result of the current block boundary does not match the encoding result.

[487] Embodiment 5

[488] According to Embodiment 5, in-loop filtering across the current block boundary can be performed only when a predetermined filtering constraint is not satisfied. For example, when the prediction mode of the current block is the palette mode, and in-loop filtering cannot be performed through the boundary of the current subframe including the current subframe, in-loop filtering through the boundary of the current block may not be performed.

[489] Meanwhile, the current coding block may include luma samples and chrominance samples. Each of the luma samples and the chrominance samples may include a left or top p ₀ P block sample and a right or bottom q ₀ Q block sample based on the target boundary on which the filtering is performed. Alternatively, each of the luma samples and the chrominance samples may include a left or top p _i P block sample and a right or bottom q _j Q block sample based on the target boundary on which the filtering is performed. Here, i may have a value equal to or greater than 0 and equal to or less than the value obtained by subtracting 1 from the maximum filter length (eg, maxFilterLengthP) for the P-block. In addition, j may have a value equal to or greater than 0 and equal to or less than the value obtained by subtracting 1 from the maximum filter length (eg, maxFilterLengthQ) for the Q-block. Hereinafter, a process for performing cross-sectional filtering on luma samples and chrominance samples is described in detail with reference to FIG. 41-43.

[490] FIG. 41 and 42 are views illustrating a cross-filtering process for luminance samples according to an embodiment of the present disclosure.

[491] First, referring to FIG. 41, the number of luminance samples to be filtered using the short filter may change to 0 according to predetermined filtering restrictions.

[492] For example, when the number nDp of p ₀ P-block samples to be filtered is greater than 0, and the palette mode is applied to the current CU including a coding block having a p ₀ P-block sample (for example, pred_mode_plt_flag==1 ), the number nDp of samples p ₀ of P-boxes to be filtered may change to 0.

[493] In addition, when the number nDq of samples q ₀ of Q-blocks to be filtered is greater than 0, and the palette mode is applied to the current CU including a coding block having a sample of q ₀ Q-blocks, the number nDq of samples q ₀ Q-blocks to be filtered may change to 0.

[494] In addition, when the number nDp of p ₀ P-block samples to be filtered is greater than 0, p ₀ P-block samples and q ₀ Q-block samples are included in different subframes, and in-loop filtering across the border of a subframe including p ₀ P-box sampling itself cannot be performed (e.g. loop_filter_across_subpic_enabled_flag[subPicIdxP]==0), the number nDp of p ₀ P-box samples to be filtered may change to 0.

[495] In addition, when the number nDq of q ₀ Q block samples to be filtered is greater than 0, q ₀ Q block sample and p ₀ P block sample are included in different subframes, and in-loop filtering across the border of a subframe including sample q ₀ Q-boxes cannot be executed (eg loop_filter_across_subpic_enabled_flag[subPicIdxQ]==0), the number nDq of q ₀ Q-box samples to be filtered can change to 0.

[496] Then, referring to FIG. 42, luma samples filtered using a long filter may be treated as unfiltered according to predetermined filtering constraints.

[497] For example, when palette mode is applied to a current CU including a coding block having a sample of p _i P-boxes (eg, pred_mode_plt_flag==1), the sample value of the filtered sample of p _i ' P-boxes may be replaced by the sample value of the input samples of p _i P-blocks.

[498] In addition, when the palette mode is applied to the current CU including a coding block having a sample of q _j Q-boxes, the sample value of the filtered sample of q _j ' Q-boxes may be replaced by the sample value of the input sample of q _j Q-blocks.

[499] In addition, when p _i P block sampling and q _j Q block sampling are included in different subframes, and in-loop filtering across the boundary of a subframe including p _i P block sampling cannot be performed (for example, loop_filter_across_subpic_enabled_flag[ subPicIdxP]==0), the sample value of the filtered sample p _i ' P-boxes may be replaced by the sample value of the input sample p _i P-boxes.

[500] In addition, when fetching q _j Q-blocks and fetching p _i P-blocks are included in different subframes, and in-loop filtering across the boundary of a subframe including fetching _qj Q-blocks cannot be performed (for example, loop_filter_across_subpic_enabled_flag[ subPicIdxQ]==0), the sample value of the filtered sample q _j ' Q-boxes may be replaced by the sample value of the input sample q _j Q-boxes.

[501] FIG. 43 is a view illustrating a cross-filtering process for chrominance samples according to an embodiment of the present disclosure.

[502] Referring to FIG. 43, filtered chrominance samples may be treated as unfiltered according to predetermined filtering constraints.

[503] For example, when palette mode is applied to a current CU including a coding block having a sample of p _i P-boxes (eg, pred_mode_plt_flag==1), the sample value of the filtered sample of p _i ' P-boxes may be replaced by the sample value of the input samples of p _i P-blocks.

[504] In addition, when the palette mode is applied to the current CU including a coding block having a sample of q _j Q-blocks (eg, pred_mode_plt_flag==1), the sample value of the filtered sample of q _j ' Q-blocks may be replaced by the sample value input sample of q _j Q-blocks.

[505] In addition, when p _i P block sampling and q _j Q block sampling are included in different subframes, and in-loop filtering across the boundary including p _i P block sampling cannot be performed (for example, loop_filter_across_subpic_enabled_flag[subPicIdxP ]==0), the sample value of the filtered sample p _i ' P-boxes may be replaced by the sample value of the input sample p _i P-boxes.

[506] In addition, when fetching q _j Q-blocks and fetching p _i P-blocks are included in different subframes, and in-loop filtering across the boundary of a subframe including fetching _qj Q-blocks cannot be performed (for example, loop_filter_across_subpic_enabled_flag[ subPicIdxQ]==0), the sample value of the filtered sample q _j ' Q-boxes may be replaced by the sample value of the input sample q _j Q-boxes.

[507] Embodiment 5 of the present disclosure differs from Embodiment 2 and Embodiment 3 in restrictions on filtering, and the filtering method described above with reference to FIG. 34 is applicable without change. Additionally, the filtering limitations of Embodiment 5 of the present disclosure are equally applicable to Embodiment 4.

[508] As described above, according to Embodiment 5 of the present disclosure, in-loop filtering across the boundary of the current subframe can only be performed when a predetermined filtering constraint is not satisfied. Therefore, it is possible to prevent a decoding error such that the decoding result of the current block boundary does not match the encoding result.

[509] Although the exemplary methods of the present disclosure described above are presented as a sequence of operations for clarity of description, it is not intended to limit the order in which the steps are performed, and the steps may be performed simultaneously or in a different order as needed. In order to implement the method according to the present disclosure, the described steps may further include other steps, may include the remaining steps except for some steps, or may include other additional steps except for some steps.

[510] In the present disclosure, an image encoding equipment or an image decoding equipment that performs a predetermined operation (step) may perform an operation (step) of confirming a condition or situation of performing a corresponding operation (step). For example, if it is described that the predetermined operation is performed when the predetermined condition is satisfied, the image encoding equipment or the image decoding equipment may perform the predetermined operation after determining whether or not the predetermined condition is satisfied.

[511] The various embodiments of the present disclosure are not a list of all possible combinations and are intended to describe salient aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combination of two or more of them.

[512] Various embodiments of the present disclosure may be implemented in hardware, firmware, software, or a combination of the above. If the present disclosure is implemented in hardware, the present disclosure may be implemented using Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGA), general processors, controllers, microcontrollers, microprocessors, etc.

[513] In addition, image decoding equipment and image encoding equipment to which embodiments of the present disclosure apply may be included in a multimedia broadcast transceiver, a mobile communication terminal, a home theater system video device, a digital cinema system video device, a surveillance camera, video chatting device, real-time communication device, for example, for video communication, mobile streaming device, storage media, video recording camera, video-on-demand (VoD) service delivery device, OTT video device ( video over networks), an Internet streaming service provider, a 3D video device, a video telephony system video device, a medical video device, and the like. and can be used to process video signals or data signals. For example, OTT video devices may include a game console, a Blu-ray player, an Internet access TV, a home theater system, a smartphone, a tablet PC, a digital video recorder (DVR), and the like.

[514] FIG. 44 is a view showing a content streaming system to which an embodiment of the present disclosure is applicable.

[515] Referring to FIG. 44, the content streaming system to which the embodiment(s) of the present document is applied may mainly include an encoding server, a streaming server, a web server, a media storage, a user device, and a media input device.

[516] The encoding server compresses content input from media input devices such as a smartphone, camera, camcorder, etc. into digital data to form a bit stream, and transmits the bit stream to the stream server. As another example, when media input devices such as smartphones, cameras, camcorder recorders, etc. directly generate a bitstream, the encoding server may be omitted.

[517] The bitstream may be generated by an image encoding method or image encoding equipment to which an embodiment of the present disclosure applies, and the streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[518] The streaming server transmits media data to the user device based on the user's request through the web server, and the web server serves as a medium for informing the user about the service. When a user requests a desired service from a web server, the web server may deliver it to the streaming server, and the streaming server may transmit media data to the user. In this case, the content streaming system may include a separate control server. In this case, the control server serves to manage the command/response between devices in the content streaming system.

[519] The streaming server may receive content from a media store and/or an encoding server. For example, when content is received from an encoding server, the content may be received in real time. In this case, in order to provide a smooth streaming service, the streaming server may keep the bitstream for a predetermined time.

[520] Examples of the user device may include a mobile phone, smartphone, laptop computer, digital broadcast terminal, personal digital assistant (PDA), portable media player (PMP), navigation device, stylus tablet PC, tablet PCs, ultrabooks, wearable devices (e.g., smart watches, smart glasses, head-mounted displays), digital TVs, desktop computers, digital signage system, etc.

[521] Each server in the content streaming system may operate as a distributed server, in which case the data received from each server may be distributed.

[522] The scope of the disclosure includes software or computer-executable instructions (e.g., an operating system, an application, a firmware, a program, etc.) to enable operations to be performed according to the methods of various embodiments in equipment or a computer, a non-volatile computer-readable medium , which has such software or commands stored and executed in hardware or a computer.

Industrial Applicability

[523] Embodiments of the present disclosure may be used to encode or decode an image.

Claims (25)

1. An image decoding method performed by image decoding equipment, wherein the image decoding method comprises the steps of: decoding a first flag indicating whether or not in-loop filtering can be performed across a subframe boundary from the bitstream; And perform in-loop filtering on the boundary of the current block, based on the first flag, while based on the boundary of the current block, which coincides with the boundary of the current subframe that includes the current block, in-loop filtering on the boundary of the current block is performed only when in-loop filtering is available both for the boundary of the current subframe and for the boundary of an adjacent subframe adjacent to the current subframe . 2. The image decoding method according to claim 1, wherein, based on a current block boundary satisfying a predetermined filtering constraint, no in-loop filtering is performed on the current block boundary. 3. The image decoding method of claim 2, wherein the filtering constraint comprises a case where the current block boundary is a current frame boundary including the current block. 4. The image decoding method of claim 2, wherein the filtering constraint comprises the case where the current block boundary is virtual boundaries of the current frame including the current block. 5. The image decoding method according to claim 2, wherein the filtering constraint comprises a case where the current block boundary is a current tile boundary including the current block, and in-loop filtering cannot be performed across the current tile boundary. 6. The image decoding method according to claim 2, wherein the filtering constraint comprises a case where the current block boundary is a current slice boundary including the current block, and in-loop filtering cannot be performed through the current slice boundary. 7. The image decoding method of claim 2, wherein the filtering constraint comprises the case where the current block boundary is the left or upper boundary of the current slice including the current block, and in-loop filtering is deactivated for the current slice. 8. The image decoding method of claim 1, wherein the in-loop filtering comprises deblocking filtering. 9. The image decoding method according to claim 1, wherein, based on the current block prediction mode being the paletted mode, no in-loop filtering is performed at the boundary of the current block. 10. The image decoding method according to claim 1, wherein performing in-loop filtering comprises the steps of: determine the boundary intensity with respect to the current block boundary, and filter the border of the current block based on the border intensity. 11. The image decoding method of claim 10, wherein, based on the edge intensity having the first value, no in-loop filtering is performed at the boundary of the current block. 12. The image decoding method of claim 1, wherein, based on treating the current subframe as a frame, the first flag for the current subframe indicates that in-loop filtering cannot be performed across a boundary of the current subframe. 13. An image encoding method carried out by means of image encoding equipment, wherein the image encoding method comprises the steps of: determining whether or not in-loop filtering can be performed across a subframe boundary; And perform in-loop filtering on the boundary of the current block based on the result of the determination, wherein, based on the boundary of the current block coinciding with the boundary of the current subframe including the current block, in-loop filtering on the boundary of the current block is performed only when in-loop filtering is available for both the boundary of the current subframe and the boundary of an adjacent subframe adjacent to the current one. subframe. 14. A method for transmitting a bit stream to a receiver of image decoding equipment generated by an image encoding method, comprising the steps of: determining whether or not in-loop filtering can be performed across a subframe boundary; And perform in-loop filtering on the boundary of the current block based on the result of the determination, while based on the boundary of the current block, which coincides with the boundary of the current subframe that includes the current block, in-loop filtering on the boundary of the current block is performed only when in-loop filtering is available both for the boundary of the current subframe and for the boundary of an adjacent subframe adjacent to the current subframe .

Images