RU2826613C2 - Method of encoding/decoding images, equipment and method for transmitting bit stream using setting maximum conversion size of block of chroma signals - Google Patents

Abstract

FIELD: video and image encoding.

SUBSTANCE: invention relates to video or image encoding, in particular to setting the maximum size of a conversion block used for a block of chroma signals. Disclosed is a method and equipment for encoding/decoding images, where the method includes obtaining a current block by splitting an image, determining the prediction mode of the current block, forming an internal prediction block relative to the current block, based on the prediction mode of the current block, which is an internal prediction mode, forming a residual block relative to the current block and restoring the current block based on the prediction block and the residual block relative to the current block.

EFFECT: high efficiency of encoding/decoding.

12 cl, 31 dwg, 1 tbl

Description

Field of technology to which the invention relates

[1] The present disclosure relates to a method and equipment for encoding/decoding images, and more particularly, to a method and equipment for encoding/decoding an image by setting a maximum size of a transform block used for a color signal block, and to a method for transmitting a bit stream generated by the image encoding method/equipment of the present disclosure.

State of the art

[2] Recently, the demand for high-quality, high-resolution images, such as high-definition (HD) images and ultra-high-definition (UHD) images, has been increasing in various fields of technology. As the resolution and quality of image data increases, the amount of information or bits transmitted increases relatively compared with existing image data. The increase in the amount of information or bits transmitted leads to an increase in transmission costs and storage costs.

[3] Accordingly, there is a need for a highly efficient image compression technology to efficiently transmit, store and reproduce relatively high-quality, high-resolution image information.

The essence of the invention

Technical task

[4] The purpose of the present disclosure is to provide a method and equipment for encoding/decoding images with improved encoding/decoding efficiency.

[5] The purpose of the present disclosure is to provide a method and equipment for encoding/decoding images that can improve encoding/decoding efficiency by setting a maximum transform block size for a color signal block.

[6] Another object of the present disclosure is to provide a method for transmitting a bit stream generated by an image coding method or equipment according to the present disclosure.

[7] Another object of the present disclosure is to provide a recording medium storing a bit stream generated by an image coding method or equipment according to the present disclosure.

[8] Another object of the present disclosure is to provide a recording medium storing a bit stream received, decoded and used to reconstruct an image by an image decoding equipment according to the present disclosure.

[9] The technical problems solved by the present disclosure are not limited to the above technical problems, and other technical problems that are not described herein should become apparent to those skilled in the art from the following description.

Technical solution

[10] A method for decoding images performed by an image decoding equipment according to an aspect of the present disclosure may include obtaining a current block by splitting an image, determining a prediction mode of the current block, generating an intra-prediction block relative to the current block based on the prediction mode of the current block, which is an intra-prediction mode, generating a residual block relative to the current block; and restoring the current block based on the prediction block and the residual block relative to the current block. In this case, the intra-prediction block and the residual block may be generated based on a size of a transformation block of the current block, and the size of the transformation block may be determined based on a color component of the current block.

[11] An image decoding equipment according to an aspect of the present disclosure may include a memory device and at least one processor. The at least one processor may obtain a current block by splitting an image, determine a prediction mode of the current block; generate an intra-prediction block with respect to the current block, based on the prediction mode of the current block, which is an intra-prediction mode, generate a residual block with respect to the current block, and restore the current block based on the prediction block and the residual block with respect to the current block. In this case, the intra-prediction block and the residual block may be generated based on the size of the transformation block of the current block, and the size of the transformation block may be determined based on the color component of the current block.

[12] An image encoding method implemented by an image encoding equipment according to an aspect of the present disclosure may include determining a current block by splitting an image, generating an intra-prediction block relative to the current block, generating a residual block relative to the current block based on the intra-prediction block, and encoding intra-prediction mode information relative to the current block. In this case, the intra-residual block and the residual block may be encoded based on a size of a transformation block of the current block, and the size of a transformation block may be determined based on a color component of the current block.

[13] In addition, the transmission method according to another aspect of the present disclosure may transmit a bit stream generated by the image coding equipment or the image coding method of the present disclosure.

[14] In addition, a machine-readable recording medium according to another aspect of the present disclosure may store a bit stream generated by the image encoding equipment or image encoding method of the present disclosure.

[15] The features briefly summarized above with respect to the present disclosure are merely exemplary aspects of the following detailed description of the present disclosure and do not limit the scope of the present disclosure.

Advantages of the invention

[16] According to the present disclosure, it is possible to provide a method and equipment for encoding/decoding images with improved encoding/decoding efficiency.

[17] According to the present disclosure, it is possible to provide a method and equipment for encoding/decoding images capable of improving encoding/decoding efficiency by setting a maximum size of a transform block for a color signal block.

[18] In addition, according to the present disclosure, it is possible to provide a method for transmitting a bit stream generated by the image coding method or equipment according to the present disclosure.

[19] In addition, according to the present disclosure, it is possible to provide a recording medium storing a bit stream generated by the image encoding method or equipment according to the present disclosure.

[20] In addition, according to the present disclosure, it is possible to provide a recording medium storing a bit stream received, decoded, and used to reconstruct an image by the image decoding equipment according to the present disclosure.

[21] Those skilled in the art will appreciate that the advantages that can be achieved through the present disclosure are not limited to those specifically described above, and other advantages of the present disclosure will be more clearly understood from the detailed description.

Brief description of the drawings

[22] Fig. 1 is a view schematically showing a video coding system to which an embodiment of the present disclosure is applicable.

[23] Fig. 2 is a view schematically showing image encoding equipment to which an embodiment of the present disclosure is applicable.

[24] Fig. 3 is a view schematically showing image decoding equipment to which an embodiment of the present disclosure is applicable.

[25] Fig. 4 is a view showing an image segmentation structure according to an embodiment.

[26] Fig. 5 is a view showing an embodiment of a block segmentation type according to a multi-type tree structure.

[27] Fig. 6 is a view showing a mechanism for transmitting block partitioning information in service signals in a quad tree with a nested multi-type tree structure according to the present disclosure.

[28] Fig. 7 is a view showing an embodiment in which the CTU is segmented into multiple CUs.

[29] Fig. 8 is a view illustrating an embodiment of a redundant partitioning pattern.

[30] Fig. 9 is a flow chart illustrating a video/image coding method based on inter prediction.

[31] Fig. 10 is a view illustrating the configuration of the inter-prediction module 180 according to the present disclosure.

[32] Fig. 11 is a flow chart illustrating a method for decoding video/images based on inter prediction.

[33] Fig. 12 is a view illustrating the configuration of the inter-prediction module 260 according to the present disclosure.

[34] Fig. 13 is a view illustrating adjacent blocks available as a spatial possible combination option according to an embodiment.

[35] Fig. 14 is a view schematically illustrating a method for constructing lists of possible merge options according to an embodiment.

[36] Fig. 15 is a view schematically illustrating a method for constructing lists of possible motion vector predictor options according to an embodiment.

[37] Fig. 16 is a view illustrating a syntax structure for transmitting an MVD from an image encoding equipment to an image decoding equipment according to an embodiment.

[38] Fig. 17 is a flow chart illustrating a video/image coding method based on IBC according to an embodiment.

[39] Fig. 18 is a view illustrating the configuration of a prediction module for implementing the IBC-based video/image coding method according to the embodiment.

[40] Fig. 19 is a flow chart illustrating a video/image decoding method based on IBC according to an embodiment.

[41] Fig. 20 is a view illustrating the configuration of a prediction module for implementing a video/image decoding method based on IBC according to an embodiment.

[42] Fig. 21 is a view illustrating a syntax for transmitting color signal formats in service signals according to an embodiment.

[43] Fig. 22 is a view illustrating a color signal format classification table according to an embodiment.

[44] Fig. 23 is a view illustrating an example of a CU partitioning constraint for virtual pipeline processing.

[45] Figs. 24-26 are views illustrating an example of partitioning a CU and a TU according to an embodiment.

[46] Figs. 27 and 28 are flow charts illustrating IBC prediction and intra prediction to which the maximum transform size is applied according to an embodiment.

[47] Fig. 29 is a flow chart illustrating a method for encoding an image by encoding equipment according to an embodiment.

[48] Fig. 30 is a flow chart illustrating a method for decoding an image by decoding equipment according to an embodiment.

[49] Fig. 31 is a view showing a content streaming system to which an embodiment of the present disclosure is applicable.

Optimal mode of implementation of the invention

[50] Hereinafter, embodiments of the present disclosure are described in detail with reference to the accompanying drawings so that they can be easily implemented by those skilled in the art. However, the present disclosure can be implemented in various forms and is not limited to the embodiments described herein.

[51] In describing the present disclosure, if it is determined that a detailed description of a related known function or structure would unnecessarily obscure the scope of the present disclosure, the detailed description thereof is omitted. In the drawings, portions not related to the description of the present disclosure are omitted, and like reference numerals are attached to like portions.

[52] In the present disclosure, when a component is "connected," "coupled," or "linked" to another component, this may include not only a direct connection-based relationship, but also an indirect connection-based relationship in which an intermediate component is present. In addition, when a component "includes" or "has" other components, this means that the other components may be additionally included, rather than to the exclusion of the other components, unless otherwise specified.

[53] In the present disclosure, the terms "first," "second," etc. may be used only for the purpose of distinguishing one component from other components and do not limit the order or importance of the components unless otherwise specified. Accordingly, within the scope of the present disclosure, a first component in one embodiment may be referred to as a "second component" in another embodiment, and similarly, a second component in one embodiment may be referred to as a "first component" in another embodiment.

[54] In the present disclosure, components that are different from each other are intended to clearly describe each feature and do not mean that the components are necessarily separated. Thus, a plurality of components may be integrated and implemented in a single hardware or software module, or one component may be distributed and implemented in a plurality of hardware or software modules. Therefore, even if not otherwise indicated, such embodiments in which components are integrated or a component is distributed are also included in the scope of the present disclosure.

[55] In the present disclosure, the components described in various embodiments do not necessarily mean essential components, and some components may be optional components. Accordingly, an embodiment consisting of a subset of the components described in an embodiment is also included within the scope of the present disclosure. In addition, embodiments including other components in addition to the components described in the various embodiments are included within the scope of the present disclosure.

[56] The present disclosure relates to image encoding and decoding, and terms used in the present disclosure may have a general meaning commonly used in the technical field to which the present disclosure belongs, unless defined for the first time in the present disclosure.

[57] In the present disclosure, a "frame" generally means a unit representing one image in a specific period of time, and a slice/tile is a coding unit that constitutes a portion of a frame, and one frame may consist of one or more slices/tiles. In addition, a slice/tile may include one or more coding tree units (CTUs).

[58] In the present disclosure, "pixel" or "pel" may mean the smallest unit that makes up one frame (or image). In addition, "sample" may be used as a term corresponding to a pixel. A sample, in general, may represent a pixel or a pixel value, and may represent only a pixel/pixel value of a luminance signal component or only a pixel/pixel value of a color signal component.

[59] In the present disclosure, a "unit" may represent a basic unit of image processing. A unit may include at least one of a specific region of a frame and information associated with the region. A unit may be used interchangeably with terms such as an "array of samples," a "block," or a "zone" in some cases. In general, an MxN block may include samples (or arrays of samples) or a set (or array) of transform coefficients from M columns and N rows.

[60] In the present disclosure, the "current block" may mean one of the "current coding block", "current coding unit", "coding target block", "decoding target block", or "processing target block". When prediction is performed, the "current block" may mean the "current prediction block" or the "prediction target block". When transform (inverse transform)/quantization (dequantization) is performed, the "current block" may mean the "current transform block" or the "transform target block". When filtering is performed, the "current block" may mean the "filter target block".

[61] In addition, in the present disclosure, "current block" may mean "a luminance block of the current block" if not explicitly specified as a chrominance block. "A chrominance block of the current block" may be expressed by including an explicit description of the chrominance block, such as "chrominance block" or "current chrominance block."

[62] In the present disclosure, the terms "/" and "" are to be interpreted as indicating "and/or." For example, the expression "A/B" and "A, B" may mean "A and/or B." Additionally, "A/B/C" and "A/B/C" may mean "at least one of A, B and/or C."

[63] In this disclosure, the term "or" shall be interpreted to indicate "and/or." For example, the expression "A or B" may contain 1) only "A," 2) only "B," and/or 3) "both A and B." In other words, in this disclosure, the term "or" shall be interpreted to indicate "additionally or alternatively."

[64] General presentation of video coding system

[65] Fig. 1 is a view showing a video coding system according to the present disclosure.

[66] The video coding system according to the embodiment may include coding equipment 10 and decoding equipment 20. Coding equipment 10 may deliver coded information or video and/or image data to decoding equipment 20 in the form of a file or streaming via a digital storage medium or a network.

[67] The encoding equipment 10 according to the embodiment may include a video source generator 11, an encoding module 12, and a transmitter 13. The decoding equipment 20 according to the embodiment may include a receiver 21, a decoding module 22, and a rendering module 23. The encoding module 12 may be called a "video/image encoding module," and the decoding module 22 may be called a "video/image decoding module." The transmitter 13 may be included in the encoding module 12. The receiver 21 may be included in the decoding module 22. The rendering module 23 may include a display, and the display may be configured as a separate device or an external component.

[68] The video source generator 11 may obtain a video/image through a process of capturing, synthesizing or generating a video/image. The video source generator 11 may include a video/image capturing device and/or a video/image generating device. The video/image capturing device may include, for example, one or more cameras, video/image archives including previously captured videos/images, etc. The video/image generating device may include, for example, computers, tablet computers and smartphones and may (electronically) generate videos/images. For example, a virtual video/image may be generated via a computer etc. In this case, the process of capturing video/images may be replaced by a process of generating associated data.

[69] The encoding unit 12 may encode the input video/image. The encoding unit 12 may perform a series of procedures such as prediction, transformation, and quantization for compression and encoding efficiency. The encoding unit 12 may output the encoded data (encoded video/image information) in the form of a bit stream.

[70] The transmitter 13 may transmit encoded information or video/image data output in the form of a bit stream to the receiver 21 of the decoding equipment 20 via a digital storage medium or a network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-Ray, HDD, SSD, and the like. The transmitter 13 may include an element for generating a multimedia file via a predetermined file format and may include an element for transmitting via a broadcast network/communication network. The receiver 21 may extract/receive the bit stream from the storage medium or network and transmit the bit stream to the decoding module 22.

[71] The decoding unit 22 may decode the video/image by performing a series of procedures such as dequantization, inverse transform, and prediction corresponding to the operation of the encoding unit 12.

[72] The rendering module 23 may render a decoded video/image. The video/image prepared by rendering may be displayed via a display.

[73] General presentation of image coding equipment

[74] Fig. 2 is a view schematically showing image encoding equipment to which an embodiment of the present disclosure is applicable.

[75] As shown in Fig. 2, the image encoding equipment 100 may include an image segmentation module 110, a subtractor 115, a converter 120, a quantizer 130, a dequantizer 140, an inverse converter 150, an adder 155, a filter 160, a memory 170, an inter-prediction module 180, an intra-prediction module 185, and an entropy encoder 190. The inter-prediction module 180 and the intra-prediction module 185 may be collectively referred to as a "prediction module." The converter 120, the quantizer 130, the dequantizer 140, and the inverse converter 150 may be included in a residual processor. The residual processor may further include the subtractor 115.

[76] All or at least part of the plurality of components configuring the image encoding equipment 100 may be configured by means of a single hardware component (e.g., an encoder or a processor) in some embodiments. In addition, the memory device 170 may include a decoded frame buffer (DPB) and may be configured by means of a digital storage medium.

[77] The image segmentation module 110 may segment an input image (or a frame or a movie frame) input to the image encoding equipment 100 into one or more processing units. For example, a processing unit may be referred to as a "coding unit (CU)". The coding unit may be obtained by recursively segmenting a coding tree unit (CTU) or a longest coding unit (LCU) according to a quadtree, binary tree and ternary tree (QT/BT/TT) structure. For example, one coding unit may be segmented into a plurality of coding units of greater depth based on a quadtree structure, a binary tree structure and/or a ternary structure. To segment a coding unit, a quadtree structure may be applied first, and a binary tree structure and/or a ternary structure may be applied subsequently. The encoding procedure according to the present disclosure may be performed based on the final coding unit, which is no longer segmented. The largest coding unit may be used as the final coding unit, or a coding unit of greater depth obtained by segmenting the largest coding unit may be used as the final coding unit. Here, the coding procedure may include a prediction, transformation and restoration procedure, which is described below. As another example, the processing unit of the coding procedure may be a prediction unit (PU) or a transformation unit (TU). The prediction unit and the transformation unit may be split or segmented from the final coding unit. The prediction unit may be a sample prediction unit, and the transformation unit may be a unit for extracting a transformation coefficient and/or a unit for extracting a residual signal from a transformation coefficient.

[78] The prediction module (inter-prediction module 180 or intra-prediction module 185) may perform prediction with respect to a block to be processed (current block) and generate a predicted block including prediction samples for the current block. The prediction module may determine whether intra-prediction or inter-prediction is applied based on the current block or CU. The prediction module may generate various information related to the prediction of the current block and transmit the generated information to the entropy encoder 190. The information regarding the prediction may be encoded in the entropy encoder 190 and output in the form of a bit stream.

[79] The intra prediction module 185 may predict the current block by referring to samples in the current frame. The samples for reference may be located in the neighborhood of the current block or may be located with diversity according to the intra prediction mode and/or the intra prediction technology. The intra prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The non-directional mode may include, for example, a DC mode and a planar mode. The directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes according to the degree of granularity of the prediction direction. However, this is just an example, a larger or smaller number of directional prediction modes may be used depending on the setting. The intra prediction module 185 may determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring block.

[80] The inter-prediction unit 180 may extract a predicted block for the current block based on a reference block (an array of reference samples) indicated by a motion vector for the reference frame. In this case, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between a neighboring block and the current block. The motion information may include a motion vector and an index of the reference frame. The motion information may further include information of the direction of inter-prediction (L0 prediction, L1 prediction, bi-prediction, etc.). In the case of inter-prediction, the neighboring block may include a spatial neighboring block present in the current frame and a temporal neighboring block present in the reference frame. The reference frame including the reference block and the reference frame including the temporal neighboring block may be identical or different. A temporal neighboring block may be called a "collocated reference block", a "collocated CU (colCU)", and the like. A reference frame including the temporal neighboring block may be called a "collocated frame (colPic)". For example, the inter-prediction unit 180 may configure a list of candidate motion information based on neighboring blocks and generate information indicating which candidate is used to extract a motion vector and/or a reference frame index of the current block. Inter-prediction may be performed based on various prediction modes. For example, in the case of the skip mode and the combination mode, the inter-prediction unit 180 may use motion information of the neighboring block as motion information of the current block. In the case of the skip mode, unlike the combination mode, a residual signal may not be transmitted. In the case of the motion vector prediction (MVP) mode, the motion vector of the neighboring block can be used as a motion vector predictor, and the motion vector of the current block can be transmitted in the signaling by encoding the motion vector difference and an indicator for the motion vector predictor. The motion vector difference may mean the difference between the motion vector of the current block and the motion vector predictor.

[81] The prediction module may generate a prediction signal based on various prediction methods and prediction technologies described below. For example, the prediction module may not only apply intra prediction or inter prediction, but also simultaneously apply both intra prediction and inter prediction to predict the current block. The prediction method of simultaneously applying both intra prediction and inter prediction to predict the current block may be called "combined inter and intra prediction (CIIP)". In addition, the prediction module may perform intra-block copying (IBC) to predict the current block. Intra-block copying may be used for coding image/video game content and the like, such as on-screen content coding (SCC). IBC is a method of predicting the current frame using a previously reconstructed reference block in the current frame at a location spaced from the current block by a predetermined distance. When IBC is applied, the location of the reference block in the current frame may be encoded as a vector (block vector) corresponding to the predetermined distance. IBC essentially performs prediction in the current frame, but can be performed similarly to inter-prediction in that the reference block is retrieved within the current frame. Thus, IBC can use at least one of the inter-prediction technologies described in this disclosure.

[82] The prediction signal generated by the prediction module may be used to generate a reconstructed signal or to generate a residual signal. The subtractor 115 may generate a residual signal (a residual block or an array of residual samples) by subtracting the prediction signal (the predicted block or the array of predicted samples) output from the prediction module from the input image signal (the original block or the array of original samples). The generated residual signal may be transmitted to the converter 120.

[83] The converter 120 may generate the transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), a graph-based transform (GBT), or a conditionally nonlinear transform (CNT). Here, GBT means a transform obtained from a graph when the relationship information between pixels is represented by a graph. CNT means a transform obtained based on a prediction signal generated using all previously reconstructed pixels. In addition, the transform process may be applied to square pixel blocks having an identical size, or may be applied to blocks having a variable size rather than a square one.

[84] The quantizer 130 may quantize the transform coefficients and transmit them to the entropy encoder 190. The entropy encoder 190 may encode the quantized signal (information regarding the quantized transform coefficients) and output a bit stream. The information regarding the quantized transform coefficients may be referred to as "residual information." The quantizer 130 may rearrange the quantized transform coefficients in a block form into a one-dimensional vector form based on a scan order of the coefficients and generate information regarding the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form.

[85] The entropy encoder 190 may implement various encoding methods, such as, for example, exponential Golomb coding, context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), and the like. The entropy encoder 190 may encode information necessary for reconstructing video/images, other than quantized transform coefficients (e.g., values of syntax elements, etc.), together or separately. The encoded information (e.g., encoded video/image information) may be transmitted or stored in units of network abstraction layers (NAL) in the form of a bit stream. The video/image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general constraint information. The information transmitted in the service signals, the transmitted information and/or the syntax elements described in this disclosure may be encoded via the above-described encoding procedure and included in the bit stream.

[86] The bit stream may be transmitted over a network or may be stored on a digital storage medium. The network may include a broadcast network and/or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-Ray, HDD, SSD, etc. A transmitter (not shown) transmitting a signal output from the entropy encoder 190 and/or a storage module (not shown) storing the signal may be included as an internal/external element of the encoding equipment 100, and alternatively, the transmitter may be included in the entropy encoder 190.

[87] The quantized transform coefficients output from the quantizer 130 may be used to form a residual signal. For example, the residual signal (residual block or residual samples) may be reconstructed by applying dequantization and inverse transform to the quantized transform coefficients via the dequantizer 140 and the inverse transformator 150.

[88] The adder 155 adds the reconstructed residual signal to the predicted signal output from the inter-prediction unit 180 or the intra-prediction unit 185 to generate a reconstructed signal (a reconstructed frame, a reconstructed block, an array of reconstructed samples). If there is no residual for a block to be processed, for example in a case in which a skip mode is applied, the predicted block can be used as a reconstructed block. The adder 155 can be called a "reconstruction unit" or a "reconstructed block generator". The reconstructed signal generated can be used for intra-prediction of the next block to be processed in the current frame, and can be used for inter-prediction of the next frame by filtering as described below.

[89] The filter 160 may enhance the subjective/objective quality of images by applying filtering to the reconstructed signal. For example, the filter 160 may generate a modified reconstructed frame by applying various filtering methods to the reconstructed frame and store the modified reconstructed frame in the memory 170, namely, in the DPB of the memory 170. The various filtering methods may include, for example, deblocking filtering, a quantized adaptive shift, an adaptive loop filter, a bilateral filter, and the like. The filter 160 may generate various information associated with the filtering and transmit the generated information to the entropy encoder 190, as described below in the description of each filtering method. The information associated with the filtering may be encoded by the entropy encoder 190 and output in the form of a bit stream.

[90] The modified reconstructed frame transmitted to the memory device 170 can be used as a reference frame in the inter-prediction unit 180. When inter-prediction is applied through the image encoding equipment 100, the prediction mismatch between the image encoding equipment 100 and the image decoding equipment can be eliminated, and the encoding efficiency can be improved.

[91] The DPB of the memory 170 may store the modified reconstructed frame for use as a reference frame in the inter-prediction unit 180. The memory 170 may store motion information of the block from which the motion information in the current frame is extracted (or coded) and/or motion information of the blocks in the frame that have already been reconstructed. The stored motion information may be transmitted to the inter-prediction unit 180 and used as motion information of a spatial neighboring block or motion information of a temporal neighboring block. The memory 170 may store reconstructed samples of the reconstructed blocks in the current frame and may transmit the reconstructed samples to the intra-prediction unit 185.

[92] General introduction of image decoding equipment

[93] Fig. 3 is a view schematically showing image decoding equipment to which an embodiment of the present disclosure is applicable.

[94] As shown in Fig. 3, the image decoding equipment 200 may include an entropy decoder 210, a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter-prediction module 260, and an intra-prediction module 265. The inter-prediction module 260 and the intra-prediction module 265 may be collectively referred to as a "prediction module." The dequantizer 220 and the inverse transformer 230 may be included in a residual processor.

[95] All or at least part of the plurality of components configuring the image decoding equipment 200 may be configured by a hardware component (for example, a decoder or a processor) according to an embodiment. In addition, the memory device 250 may include a decoded frame buffer (DPB) or may be configured by a digital storage medium.

[96] The image decoding equipment 200, which receives a bit stream including video/image information, can restore an image by performing a process corresponding to the process performed by the image encoding equipment 100 in Fig. 2. For example, the image decoding equipment 200 can perform decoding using a processing unit applied in the image encoding equipment. That is, the decoding processing unit may be, for example, a coding unit. The coding unit may be obtained by segmenting a coding tree unit or the largest coding unit. The restored image signal decoded and output by the decoding equipment 200 can be reproduced by the playback equipment.

[97] The image decoding equipment 200 may receive a signal output from the image encoding equipment of Fig. 2 in the form of a bit stream. The received signal may be decoded via the entropy decoder 210. For example, the entropy decoder 210 may parse the bit stream to extract information (e.g., video/image information) necessary for reconstructing images (or reconstructing frames). The video/image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include common constraint information. The image decoding equipment may further decode the frame based on the information regarding the parameter set and/or the common constraint information. The signaling/receiving information and/or the syntax elements described in this disclosure may be decoded via the decoding procedure and obtained from the bit stream. For example, the entropy decoder 210 decodes information in a bit stream based on a coding method such as Exponential Golomb coding, CAVLC or CABAC coding, and output values of syntax elements required for image reconstruction and quantized values of transform coefficients for the residual. More specifically, the entropy CABAC decoding method can receive a resolution unit corresponding to each syntax element in the bit stream, determine a context model using information of the decoding target syntax element, decoding information of a neighboring block and the decoding target block or information of a symbol/resolution unit decoded in a previous stage, and perform arithmetic decoding for the resolution unit by predicting the probability of occurrence of the resolution unit according to the determined context model and generate a symbol corresponding to the value of each syntax element. In this case, the entropy CABAC decoding method can update the context model by using the information of the decoded symbol/bin for the context model of the next symbol/bin after determining the context model. The information related to the prediction from the information decoded by the entropy decoder 210 can be provided to the prediction unit (the inter prediction unit 260 and the intra prediction unit 265), and the residual value for which the entropy decoding is performed in the entropy decoder 210, i.e., quantized transform coefficients and related parameter information may be input to the dequantizer 220. In addition, information regarding filtering from the information decoded by the entropy decoder 210 may be provided to the filter 240. Meanwhile, a receiver (not shown) for receiving a signal output from the image encoding equipment may be further configured as an internal/external element of the image decoding equipment 200, or the receiver may be a component of the entropy decoder 210.

[98] Meanwhile, the image decoding equipment according to the present disclosure may be called "video/image/frame decoding equipment". The image decoding equipment may be classified into an information decoder (a video/image/frame information decoder) and a sample decoder (a video/image/frame sample decoder). The information decoder may include an entropy decoder 210. The sample decoder may include at least one of a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter-prediction unit 260, or an intra-prediction unit 265.

[99] The dequantizer 220 may dequantize the quantized transform coefficients and output the transform coefficients. The dequantizer 220 may rearrange the quantized transform coefficients in the form of a two-dimensional block. In this case, the rearrangement may be performed based on the scanning order of the coefficients performed in the image encoding equipment. The dequantizer 220 may perform dequantization for the quantized transform coefficients by using a quantization parameter (for example, quantization step size information) and obtain the transform coefficients.

[100] The inverse transformer 230 may inversely transform the transform coefficients to obtain a residual signal (residual block, residual sample array).

[101] The prediction module may perform prediction for the current block and may generate a predicted block including prediction samples for the current block. The prediction module may determine whether intra prediction or inter prediction is applied to the current block based on the prediction information output from the entropy decoder 210, and may determine a specific intra/inter prediction mode.

[102] Identical to what is described in the prediction module of the image encoding equipment 100, the prediction module can generate a prediction signal based on various prediction methods (technologies), which are described below.

[103] The intra-prediction module 265 may predict the current block by referring to samples in the current frame. The description of the intra-prediction module 185 applies equally to the intra-prediction module 265.

[104] The inter-prediction unit 260 may extract a predicted block for the current block based on a reference block (an array of reference samples) indicated by a motion vector for the reference frame. In this case, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between a neighboring block and the current block. The motion information may include a motion vector and an index of the reference frame. The motion information may further include information of the direction of inter-prediction (L0 prediction, L1 prediction, bi-prediction, etc.). In the case of inter-prediction, the neighboring block may include a spatial neighboring block present in the current frame and a temporal neighboring block present in the reference frame. For example, the inter-prediction module 260 may configure a list of candidate motion information based on neighboring blocks and extract a motion vector of the current block and/or a reference frame index based on the received candidate selection information. Inter-prediction may be performed based on various prediction modes, and the information regarding the prediction may include information indicating an inter-prediction mode for the current block.

[105] The adder 235 may generate a reconstructed signal (a reconstructed frame, a reconstructed block, an array of reconstructed samples) by adding the obtained residual signal to a predicted signal (a predicted block, an array of predicted samples) output from the prediction module (including the inter-prediction module 260 and/or the intra-prediction module 265). If there is no residual for a block to be processed, such as when a skip mode is applied, the predicted block may be used as the reconstructed block. The description of the adder 155 is equally applicable to the adder 235. The adder 235 may be referred to as a "reconstruction module" or a "reconstructed block generator". The reconstructed signal generated may be used for intra-prediction of the next block to be processed in the current frame and may be used for inter-prediction of the next frame by filtering as described below.

[106] The filter 240 may improve the subjective/objective quality of images by applying filtering to the reconstructed signal. For example, the filter 240 may generate a modified reconstructed frame by applying various filtering methods to the reconstructed frame and store the modified reconstructed frame in the memory 250, namely, in the DPB of the memory 250. The various filtering methods may include, for example, deblocking filtering, discretized adaptive shifting, adaptive loop filter, bilateral filter, and the like.

[107] The (modified) reconstructed frame stored in the DPB of the memory 250 may be used as a reference frame in the inter-prediction unit 260. The memory 250 may store motion information of the block from which the motion information in the current frame is extracted (or decoded), and/or motion information of the blocks in the frame that have already been reconstructed. The stored motion information may be transmitted to the inter-prediction unit 260, so that it is used as motion information of the spatial neighboring block or motion information of the temporal neighboring block. The memory 250 may store reconstructed samples of the reconstructed blocks in the current frame and transmit the reconstructed samples to the intra-prediction unit 265.

[108] In the present disclosure, the embodiments described in the filter 160, the inter-prediction module 180, and the intra-prediction module 185 of the image encoding equipment 100 may be equally or correspondingly applied to the filter 240, the inter-prediction module 260, and the intra-prediction module 265 of the image decoding equipment 200.

[109] General presentation of image segmentation

[110] The video/image encoding method according to the present disclosure may be implemented based on an image segmentation structure as follows. Specifically, the prediction, residual processing ((inverse) transform, (de)quantization, etc.), syntax element encoding, and filtering procedures, which are described below, may be performed based on a CTU, CU (and/or TU, PU) extracted based on the image segmentation structure. The image may be segmented in units of blocks, and the segmentation procedure into blocks may be performed in the image segmentation module 110 of the encoding equipment. The segmentation-related information may be encoded by the entropy encoder 190 and transmitted to the decoding equipment in the form of a bit stream. The entropy decoder 210 of the decoding equipment can extract a segmentation structure into blocks of the current frame based on segmentation-related information obtained from the bit stream, and based on this, can perform a series of procedures (e.g., prediction, residual processing, block/frame restoration, in-loop filtering, etc.) for decoding images.

[111] Frames may be segmented into a sequence of coding tree units (CTUs). Fig. 4 shows an example in which a frame is segmented into CTUs. A CTU may correspond to a coding tree block (CTB). Alternatively, a CTU may include a coding tree block of luminance samples and two coding tree blocks of corresponding chrominance samples. For example, for a frame that contains three arrays of samples, a CTU may include an NxN block of luminance samples and two corresponding blocks of chrominance samples.

[112] General presentation of CTU segmentation

[113] As described above, a coding unit may be obtained by recursively segmenting a coding tree unit (CTU) or a largest coding unit (LCU) according to a quadtree/binary tree/ternary tree (QT/BT/TT) structure. For example, a CTU may first be segmented into a quadtree structure. After that, the leaf nodes of the quadtree structure may be further segmented by a multi-type tree structure.

[114] Quadtree segmentation means that the current CU (or CTU) is segmented equally into four. By means of quadtree segmentation, the current CU can be segmented into four CUs having the same width and the same height. When the current CU is no longer segmented into the quadtree structure, the current CU corresponds to a leaf node of the quadtree structure. The CU corresponding to the leaf node of the quadtree structure can no longer be segmented and can be used as the above-described final coding unit. Alternatively, the CU corresponding to the leaf node of the quadtree structure can be further segmented by a multi-type tree structure.

[115] Fig. 5 is a view showing an embodiment of a block segmentation type according to a multi-type tree structure. The segmentation according to a multi-type tree structure may include two types of partitioning according to a binary tree structure and two types of partitioning according to a ternary tree structure.

[116] Two types of partitioning according to the binary tree structure may include vertical binary partitioning (SPLIT_BT_VER) and horizontal binary partitioning (SPLIT_BT_HOR). Vertical binary partitioning (SPLIT_BT_VER) means that the current CU is equally partitioned in half in the vertical direction. As shown in FIG. 4, through vertical binary partitioning, two CUs having a height identical to the height of the current CU and having a width that is half of the width of the current CU may be formed. Horizontal binary partitioning (SPLIT_BT_HOR) means that the current CU is equally partitioned in half in the horizontal direction. As shown in FIG. 5, through horizontal binary partitioning, two CUs having a height that is half of the height of the current CU and having a width that is identical to the width of the current CU may be formed.

[117] Two types of partitioning according to the ternary tree structure may include vertical ternary partitioning (SPLIT_TT_VER) and horizontal ternary partitioning (SPLIT_TT_HOR). In the vertical ternary partitioning (SPLIT_TT_VER), the current CU is partitioned in the vertical direction in a ratio of 1:2:1. As shown in FIG. 5, through the vertical ternary partitioning, two CUs having a height identical to the height of the current CU and having a width that is 1/4 of the width of the current CU and a CU having a height identical to the height of the current CU and having a width that is half of the width of the current CU may be formed. In the horizontal ternary partitioning (SPLIT_TT_HOR), the current CU is partitioned in the horizontal direction in a ratio of 1:2:1. As shown in FIG. 5, by means of horizontal ternary partitioning, two CUs can be formed, having a height that is 1/4 of the height of the current CU and having a width identical to the width of the current CU, and a CU having a height that is half of the height of the current CU and having a width identical to the width of the current CU.

[118] Fig. 6 is a view showing a mechanism for transmitting block partitioning information in service signals in a quad tree with a nested multi-type tree structure according to the present disclosure.

[119] Here, the CTU is treated as the root node of the quadtree and is segmented for the first time into a quadtree structure. Information (e.g., qt_split_flag) indicating whether or not quadtree splitting is performed for the current CU (CTU or quadtree node (QT_node)) is carried in signaling. For example, when qt_split_flag has the first value (e.g., "1"), the current CU can be segmented into a quadtree. In addition, when qt_split_flag has the second value (e.g., "0"), the current CU is not segmented into a quadtree but becomes a leaf node (QT_leaf_node) of the quadtree. Each leaf node of the quadtree can then be further segmented into multiple tree structures. In this way, a leaf node of a quadtree can become a node (MTT_node) of a multi-type tree. In a multi-type tree structure, the first flag (e.g., Mtt_split_cu_flag) is signaled to indicate whether the current node is further segmented or not. If the corresponding node is further segmented (e.g., if the first flag is 1), the second flag (e.g., Mtt_split_cu_vertical_flag) can be signaled to indicate the splitting direction. For example, the splitting direction can be the vertical direction if the second flag is 1, and can be the horizontal direction if the second flag is 0. Then, the third flag (e.g., Mtt_split_cu_binary_flag) can be signaled to indicate whether the splitting type is a binary splitting type or a ternary splitting type. For example, the partition type may be a binary partition-based type when the third flag is 1, and may be a ternary partition-based type when the third flag is 0. A node of a multi-type tree obtained by binary partitioning or ternary partitioning may be further segmented into multi-type tree structures. However, a node of a multi-type tree may not be segmented into quadtree structures. If the first flag is 0, the corresponding node of a multi-type tree is no longer partitioned, but becomes a leaf node (MTT_leaf_node) of a multi-type tree. A CU corresponding to a leaf node of a multi-type tree may be used as the above-described final coding unit.

[120] Based on the mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag, the multi-tree split mode (MttSplitMode) of the CU can be extracted as shown in Table 1 below. In the description below, the multi-tree split mode may be referred to as the "multi-tree split type" or "split type".

[121] Table 1

MttSplitMode mtt_split_cu_vertical_flag mtt_split_cu_binary_flag SPLIT_TT_HOR 0 0 SPLIT_BT_HOR 0 1 SPLIT_TT_VER 1 0 SPLIT_BT_VER 1 1

[122] FIG. 7 is a view showing an example in which a CTU is segmented into multiple CUs by applying a multi-type tree after applying a quad tree. In FIG. 7, the bold edges 710 of the blocks represent the segmentation based on the quad tree, and the remaining edges 720 represent the segmentation based on the multi-type tree. A CU may correspond to a coding block (CB). In an embodiment, the CU may include a coding block of luminance samples and two coding blocks of chrominance samples corresponding to the luminance samples. A CB or TB size of the chrominance components (samples) may be extracted based on the CB or TB size of the luminance components (samples) according to a component ratio according to a color format (chrominance format, such as 4:4:4, 4:2:2, 4:2:0, etc.) of a frame/picture. In the case of the 4:4:4 color format, the CB/TB size of the chrominance components can be set equal to the CB/TB size of the luminance components. In the case of the 4:2:2 color format, the width of the CB/TB of the chrominance components can be set equal to half the width of the CB/TB of the luminance components, and the height of the CB/TB of the chrominance components can be set equal to the height of the CB/TB of the luminance components. In the case of the 4:2:0 color format, the width of the CB/TB of the chrominance components can be set equal to half the width of the CB/TB of the luminance components, and the height of the CB/TB of the chrominance components can be set equal to half the height of the CB/TB of the luminance components.

[123] In an embodiment, when the CTU size is 128 based on the luminance signal sampling unit, the CU size may have a size from 128×128 to 4×4, which is the same size as the CTU. In one embodiment, in the case of a 4:2:0 color format (or chrominance signal format), the CB size of the chrominance signals may have a size from 64×64 to 2×2.

[124] Meanwhile, in the embodiment, the CU size and the TU size may be identical. Alternatively, a plurality of TUs may be provided in the CU region. The TU size generally represents the size of a transform block (TB) of component (samples) luminance signals.

[125] The TU size may be extracted based on the largest allowable TB size maxTbSize, which is a predetermined value. For example, when the CU size exceeds maxTbSize, a plurality of TUs (TB) having maxTbSize may be extracted from the CU, and the transform/inverse transform may be performed in units of TU (TB). For example, the largest allowable TB size of luminance signals may be 64×64, and the largest allowable TB size of chrominance signals may be 32×32. If the width or height of the CB segmented according to the tree structure is greater than the largest transform width or height, the CB may be automatically (or implicitly) segmented until the TB size limit in the horizontal and vertical directions is satisfied.

[126] In addition, for example, when intra prediction is applied, the intra prediction mode/type may be extracted in CU (or CB) units, and the procedure of extracting adjacent reference samples and generating prediction samples may be performed in TU (or TB) units. In this case, one or multiple TUs (or TBs) may be provided in one CU (or CB) region, and, in this case, multiple TUs or (TBs) may share the same intra prediction mode/type.

[127] Meanwhile, for the quadtree-based coding tree scheme with nested multitype tree, the following parameters can be transmitted in the service signals as syntactic SPS elements from the encoding equipment to the decoding equipment. For example, at least one of CTU-size, which is a parameter representing the size of the root nodes of the quad tree, MinQTSize, which is a parameter representing the minimum allowed size of the leaf nodes of the quad tree, MaxBtSize, which is a parameter representing the maximum allowed size of the root nodes of the binary tree, MaxTtSize, which is a parameter representing the maximum allowed size of the root nodes of the ternary tree, MaxMttDepth, which is a parameter representing the maximum allowed depth of the partition hierarchy based on a multi-type tree from a leaf node of the quad tree, MinBtSize, which is a parameter representing the minimum allowed size of the leaf nodes of the binary tree, or MinTtSize, which is a parameter representing the minimum allowed size of the leaf nodes of the ternary tree, is transmitted in service signals.

[128] As an embodiment of using the 4:2:0 chroma signal format, the CTU size may be set to 128×128 luminance signal blocks and two 64×64 chroma signal blocks corresponding to the luminance signal blocks. In this case, MinOTSize may be set to 16×16, MaxBtSize may be set to 128×128, MaxTtSzie may be set to 64×64, MinBtSize and MinTtSize may be set to 4×4, and MaxMttDepth may be set to 4. Quadtree-based segmentation may be applied to the CTU to form leaf nodes of the quadtree. A leaf node of the quadtree may be referred to as a "QT leaf node." The leaf nodes of a quadtree can have a size ranging from 16×16 (e.g. MinOTSize) to 128×128 (e.g. CTU size). If a QT leaf node is 128×128, it may not be further segmented into a binary tree/ternary tree. This is because in this case, even with segmentation, it exceeds MaxBtsize and MaxTtszie (e.g. 64×64). In other cases, QT leaf nodes may be further segmented into a multi-tree. Therefore, a QT leaf node is the root node for a multi-tree, and a QT leaf node may have a value of 0 for the multi-tree depth (mttDepth). If the multi-tree depth reaches MaxMttdepth (e.g. 4), further segmentation may not be considered. If the width of a multi-type tree node is MinBtSize and is less than or equal to 2xMinTtSize, additional horizontal segmentation may not be considered. If the height of a multi-type tree node is MinBtSize and is less than or equal to 2xMinTtSize, additional vertical segmentation may not be considered. When segmentation is not considered, the encoding equipment may skip signaling the segmentation information. In this case, the decoding equipment may extract the segmentation information with a predetermined value.

[129] Meanwhile, one CTU may include a luminance signal sample coding unit (hereinafter referred to as a "luminance signal unit") and two chrominance signal sample coding units corresponding thereto (hereinafter referred to as "chrominance signal units"). The above-described tree-based coding scheme may be applied equally or separately to the luminance signal unit and the chrominance signal unit of the current CU. Specifically, the luminance signal units and the chrominance signal units in one CTU may be segmented into an identical block tree structure, in which case, the tree structure is represented as a SINGLE_TREE. Alternatively, the luminance signal units and the chrominance signal units in one CTU may be segmented into separate block tree structures, in which case, the tree structure may be represented as a DUAL_TREE. Thus, when a CTU is segmented into dual trees, the block tree structure for the luminance block and the block tree structure for the chrominance block may be separately present. In this case, the block tree structure for the luminance block may be called "DUAL_TREE_LUMA", and the block tree structure for the chrominance component may be called "DUAL_TREE_CHROMA". For P- and B-slice/tile groups, the luminance and chrominance blocks in a single CTU may be restricted to having an identical coding tree structure. However, for I-slice/tile groups, the luminance and chrominance blocks may have a separate block tree structure with respect to each other. If a separate block tree structure is applied, the CTB of the luminance signals may be segmented into CUs based on a specific coding tree structure, and the CTB of the chrominance signals may be segmented into chrominance CUs based on another coding tree structure. Thus, it means that a CU in an I-slice/tile group to which a separate block tree structure is applied may include a luminance component coding unit or coding units of two chrominance components, and a CU of a P- or B-slice/tile group may include three color component units (a luminance component and two chrominance components).

[130] Although a structure in the form of a coding tree based on a quad tree with a nested multi-type tree is described, the structure in which a CU is segmented is not limited thereto. For example, a BT structure and a TT structure may be interpreted as a concept included in a structure in the form of a multi-type segmentation tree (MPT), and a CU may be interpreted as being segmented through a QT structure and an MPT structure. In an example in which a CU is segmented through a QT structure and an MPT structure, a syntax element (e.g., MPT_split_type) including information regarding how many blocks a leaf node of the QT structure is segmented into and a syntax element (e.g., MPT_split_mode) including information regarding in which of the vertical and horizontal directions a leaf node of the QT structure is segmented may be transmitted in overhead signals to determine the segmentation structure.

[131] In another example, a CU may be segmented in a manner different from a QT structure, a BT structure, or a TT structure. Thus, unlike a CU of shallower depth being segmented into 1/4 of a CU of deeper depth according to a QT structure, a CU of shallower depth being segmented into 1/2 of a CU of deeper depth according to a BT structure, or a CU of shallower depth being segmented into 1/4 or 1/2 of a CU of deeper depth according to a TT structure, a CU of shallower depth may be segmented into 1/5, 1/3, 3/8, 3/5, 2/3, or 5/8 of a CU of deeper depth in some cases, and the manner of segmenting a CU is not limited thereto.

[132] The quadtree coding block structure with a multi-type tree can provide a very flexible block segmentation structure. Due to the segmentation types supported in the multi-type tree, different segmentation patterns can potentially result in the same coding block structure in some cases. In the encoding equipment and decoding equipment, by limiting the occurrence of such redundant segmentation patterns, the data volume of segmentation information can be reduced.

[133] For example, FIG. 8 shows redundant partitioning patterns that may arise in binary tree partitioning and ternary tree partitioning. As shown in FIG. 8, the continuous binary partitioning 810 and 820 for one direction of the two-stage layers has a coding unit structure identical to the coding unit structure for the binary partitioning for the center segment after ternary partitioning. In this case, the binary tree partitioning for the center blocks 830 and 840 of the ternary tree partitioning may be prohibited. This prohibition is applicable to the CUs of all frames. When such a particular partitioning is prohibited, the signaling of the corresponding syntax elements may be modified to reflect this prohibited case, thereby reducing the number of bits signaled for the partitioning. For example, as shown in the example given in FIG. 8, when the binary tree splitting for the central block CU is prohibited, the syntax element mtt_split_cu_binary_flag indicating whether the splitting is a binary splitting or a ternary splitting is not transmitted in the signaling, and its value can be extracted as 0 by the decoding equipment.

[134] General presentation of mutual forecasting

[135] This document further describes mutual prediction according to the present disclosure.

[136] The prediction module of the image encoding equipment/image decoding equipment according to the present disclosure may perform inter-prediction in units of blocks to extract a prediction sample. The inter-prediction may represent a prediction extracted in a manner that depends on data elements (e.g., sample values, motion information, etc.) of a frame(s) other than the current frame. When inter-prediction is applied to the current block, the predicted block (prediction block or prediction sample array) for the current block may be extracted based on a reference block (reference sample array) indicated by a motion vector for the reference frame indicated by a reference frame index. In this case, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information of the current block may be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between an adjacent block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction type information (L0 prediction, L1 prediction, bi-prediction, etc.). In the case of inter-prediction, the neighboring block may include a spatial neighboring block present in the current frame and a temporal neighboring block present in the reference frame. The reference frame including the reference block and the reference frame including the temporal neighboring block may be identical or different. The temporal neighboring block may be referred to as a "co-located reference block", "co-located CU (colCU)", or "colBlock", and the reference frame including the temporal neighboring block may be referred to as a "co-located frame (colPic)" or "colPicture". For example, a list of possible motion information options may be constructed based on neighboring blocks relative to the current block, and flag or index information indicating which possible option is selected (used) may be transmitted in the signaling so as to extract the motion vector of the current block and/or the index of the reference frame.

[137] Inter-prediction can be performed based on various prediction modes. For example, in the case of the skip mode and the fusion mode, the motion information of the current block may be equal to the motion information of the selected neighboring block. In the case of the skip mode, the residual signal may not be transmitted, unlike the fusion mode. In the case of the motion information prediction (MVP) mode, the motion vector of the selected neighboring block can be used as a motion vector predictor, and the motion vector difference can be transmitted in the service signals. In this case, the motion vector of the current block can be extracted using the sum of the motion vector predictor and the motion vector difference. In the present disclosure, the MVP mode can have a semantic meaning identical to the semantic meaning of the advanced motion vector prediction (AMVP).

[138] The motion information may include L0 motion information and/or L1 motion information according to the type of mutual prediction (L0 prediction, L1 prediction, bi-prediction, etc.). The motion vector in the L0 direction may be called an "L0 motion vector" or "MVL0", and the motion vector in the L1 direction may be called an "L1 motion vector" or "MVL1". Prediction based on the L0 motion vector may be called "L0 prediction", prediction based on the L1 motion vector may be called "L1 prediction", and prediction based on both the L0 motion vector and the L1 motion vector may be called "bi-prediction". Here, the L0 motion vector may indicate a motion vector associated with a reference frame list L0 (L0), and the L1 motion vector may indicate a motion vector associated with a reference frame list L1 (L1). The reference frame list L0 may include frames before the current frame in the output order as reference frames, and the reference frame list L1 may include frames after the current frame in the output order. The previous frames may be called "forward (reference) frames", and the subsequent frames may be called "backward (reference) frames". The reference frame list L0 may further include frames after the current frame in the output order as reference frames. In this case, in the reference frame list L0, the previous frames may be indexed first, and the subsequent frames may be indexed then. The reference frame list L1 may further include frames before the current frame in the output order as reference frames. In this case, in the L1 reference frame list, subsequent frames can be indexed first, and previous frames can be indexed later. Here, the output order can correspond to the order of the frame order (POC) numbers.

[139] Fig. 9 is a flow chart illustrating a video/image coding method based on inter prediction.

[140] Fig. 10 is a view illustrating the configuration of the inter-prediction module 180 according to the present disclosure.

[141] The encoding method of FIG. 9 may be performed by the image encoding equipment of FIG. 2. Specifically, step S610 may be performed by the inter-prediction unit 180, and step S620 may be performed by the residual processor. Specifically, step S620 may be performed by the subtractor 115. Step S630 may be performed by the entropy encoder 190. The prediction information of step S630 may be extracted by the inter-prediction unit 180, and the residual information of step S630 may be extracted by the residual processor. The residual information is information regarding the residual samples. The residual information may include information regarding the quantized transform coefficients for the residual samples. As described above, the residual samples can be extracted as transform coefficients through the converter 120 of the image encoding equipment, and the transform coefficients can be extracted as quantized transform coefficients through the quantizer 130. Information about the quantized transform coefficients can be encoded by the entropy encoder 190 through a residual encoding procedure.

[142] The image coding equipment may perform inter-prediction for the current block (S610). The image coding equipment may extract the inter-prediction mode and the motion information of the current block and generate prediction samples of the current block. Here, the procedures of determining the inter-prediction mode, extracting the motion information and generating the prediction samples may be simultaneously performed, or any of them may be performed before the other procedures. For example, as shown in FIG. 10, the inter-prediction unit 180 of the image coding equipment 100 may include a prediction mode determination unit 181, a motion information extraction unit 182 and a prediction sample extraction unit 183. The prediction mode determination unit 181 may determine the prediction mode of the current block, the motion information extraction unit 182 may extract the motion information of the current block, and the prediction sample extraction unit 183 may extract the prediction samples of the current block. For example, the inter-prediction unit 180 of the image coding equipment may search for a block similar to the current block in a predetermined area (search area) of reference frames through motion estimation, and extract a reference block whose difference with respect to the current block is equal to or less than a predetermined criterion or minimum. Based on this, a reference frame index indicating the reference frame in which the reference block is located may be extracted, and a motion vector may be extracted based on the position difference between the reference block and the current block. The image coding equipment may determine a mode applied to the current block from among various prediction modes. The image coding equipment may compare the rate-dependent (RD) distortion cost functions for various prediction modes and determine an optimal prediction mode of the current block. However, the method for determining the prediction mode of the current block by the image coding equipment is not limited to the above example, and various methods may be used.

[143] For example, when the skip mode or the combining mode is applied to the current block, the image coding equipment may extract combining candidates from neighboring blocks relative to the current block and construct a combining candidate list using the extracted combining candidates. In addition, the image coding equipment may extract a reference block whose difference relative to the current block is equal to or less than a predetermined criterion or minimum from the reference blocks indicated by the combining candidates included in the combining candidate list. In this case, a combining candidate associated with the extracted reference block may be selected, and combining index information indicating the selected combining candidate may be generated and transmitted in service signals to the image decoding equipment. Motion information of the current block may be extracted using motion information of the selected combining candidate.

[144] As another example, when the MVP mode is applied to the current block, the image coding equipment may extract motion vector predictor (MVP) candidates from neighboring blocks relative to the current block and construct a list of MVP candidates using the extracted MVP candidates. In addition, the image coding equipment may use a motion vector of an MVP candidate selected from among the MVP candidates included in the MVP candidate list as the MVP of the current block. In this case, for example, a motion vector indicating a reference block extracted by the above-described motion estimation may be used as the motion vector of the current block, and the MVP candidate with a motion vector having the smallest difference with respect to the motion vector of the current block from the MVP candidates may be the selected MVP candidate. A motion vector difference (MVD), which is a difference obtained by subtracting the MVP from the motion vector of the current block, may be extracted. In this case, the index information indicating the selected possible MVP variant and the information regarding the MVD may be transmitted in the service signals to the image decoding equipment. In addition, when the MVP mode is applied, the index value of the reference frame may be constructed as the index information of the reference frame and separately transmitted in the service signals to the image decoding equipment.

[145] The image coding equipment may extract residual samples based on the predicted samples (S620). The image coding equipment may extract residual samples through a comparison between the original samples of the current block and the predicted samples. For example, the residual sample may be extracted by subtracting the corresponding predicted sample from the original sample.

[146] The image encoding equipment may encode image information including prediction information and residual information (S630). The image encoding equipment 100 may output the encoded image information in the form of a bit stream. The prediction information may include prediction mode information (for example, a skip flag, a combining flag, or a mode index, etc.) and motion information as information related to a prediction procedure. Of the prediction mode information, the skip flag indicates whether or not the skip mode is applied to the current block, and the combining flag indicates whether or not the combining mode is applied to the current block. Alternatively, the prediction mode information may indicate one of a plurality of prediction modes, for example, a mode index. When the skip flag and the combining flag are 0, it may be determined that the MVP mode is applied to the current block. The information regarding the motion information may include candidate selection information (for example, a combining index, an MVP flag, or an MVP index), which is information for extracting a motion vector. Of the candidate selection information, a combination index may be transmitted in signaling when the combination mode is applied to the current block, and may be information for selecting one of the possible combination options included in the possible combination option list. Of the candidate selection information, an MVP flag or an MVP index may be transmitted in signaling when the MVP mode is applied to the current block, and may be information for selecting one of the possible MVP options in the possible MVP option list. In addition, the information regarding the motion information may include information regarding the above-described MVD and/or reference frame index information. In addition, the information regarding the motion information may include information indicating whether L0 prediction, L1 prediction, or biprediction should be applied. The residual information is information regarding the residual samples. The residual information may include information regarding quantized transform coefficients for the residual samples.

[147] The output bit stream may be stored on a (digital) storage medium and transmitted to image decoding equipment, or may be transmitted to image decoding equipment via a network.

[148] As described above, the image encoding equipment may form a reconstructed frame (a frame including reconstructed samples and a reconstructed block) based on the reference samples and the residual samples. This serves to extract, by the image encoding equipment, a prediction result identical to the prediction performed by the image decoding equipment, thereby improving the encoding efficiency. Accordingly, the image encoding equipment may store the reconstructed frame (or reconstructed samples and a reconstructed block) in a memory device and use it as a reference frame for inter-prediction. As described above, the in-loop filtering procedure may further be applied to the reconstructed frame.

[149] Fig. 11 is a flow chart illustrating a method for decoding video/images based on inter prediction.

[150] Fig. 12 is a view illustrating the configuration of the inter-prediction module 260 according to the present disclosure.

[151] The image decoding equipment may perform an operation corresponding to the operation performed by the image encoding equipment. The image decoding equipment may perform prediction for the current block based on the received prediction information and extract prediction samples.

[152] The decoding method of Fig. 11 may be performed by the image decoding equipment of Fig. 3. Steps S810-S830 may be performed by the inter-prediction unit 260, and the prediction information of step S810 and the residual information of step S840 may be obtained from the bitstream by the entropy decoder 210. The residual processor of the image decoding equipment may extract residual samples for the current block based on the residual information (S840). Specifically, the dequantizer 220 of the residual processor may perform dequantization based on the dequantized transform coefficients extracted based on the residual information to extract the transform coefficients, and the inverse transformer 230 of the residual processor may perform inverse transformation for the transform coefficients to extract residual samples for the current block. Step S850 may be performed by the adder 235 or the restorer.

[153] In particular, the image decoding equipment may determine a prediction mode of the current block based on the received prediction information (S810). The image decoding equipment may determine which inter-prediction mode is applied to the current block based on the prediction mode information in the prediction information.

[154] For example, it may be determined whether the skip mode is applied to the current block based on the skip flag. In addition, it may be determined whether the merge mode or the MVP mode is applied to the current block based on the merge flag. Alternatively, one of various possible inter-prediction mode options may be selected based on the mode index. The possible inter-prediction mode options may include the skip mode, the merge mode, and/or the MVP mode, or may include various inter-prediction modes, which are described below.

[155] The image decoding equipment may extract motion information of the current block based on the determined inter-prediction mode (S820). For example, when the skip mode or the combining mode is applied to the current block, the image decoding equipment may construct a combining candidate list, which is described below, and select one of the combining candidate options included in the combining candidate list. The selection may be performed based on the above-described selecting information of the combining candidate (combination index). The motion information of the current block may be extracted using the motion information of the selected combining candidate option. For example, the motion information of the selected combining candidate option may be used as the motion information of the current block.

[156] As another example, when the MVP mode is applied to the current block, the image decoding equipment may construct a list of candidate MVPs and use a motion vector of a candidate MVP selected from among the candidate MVPs included in the list of candidate MVPs as the MVP of the current block. The selection may be performed based on the candidate selection information (MVP flag or MVP index) described above. In this case, the MVD of the current block may be extracted based on the information regarding the MVD, and the motion vector of the current block may be extracted based on the MVP and MVD of the current block. In addition, the index of the reference frame of the current block may be extracted based on the reference frame index information. The frame indicated by the reference frame index in the reference frame list of the current block may be extracted as the reference frame referenced for inter-prediction of the current block.

[157] The image decoding equipment may generate predicted samples of the current block based on the motion information of the current block (S830). In this case, a reference frame may be extracted based on the index of the reference frame of the current block, and the predicted samples of the current block may be extracted using samples of the reference block indicated by the motion vector of the current block on the reference frame. In some cases, the filtering procedure of the predicted samples may further be performed for all or some of the predicted samples of the current block.

[158] For example, as shown in Fig. 12, the inter-prediction unit 260 of the image decoding equipment may include a prediction mode determination unit 261, a motion information extraction unit 262, and a prediction sample extraction unit 263. In the inter-prediction unit 260 of the image decoding equipment, the prediction mode determination unit 261 may determine a prediction mode of the current block based on the received prediction mode information, the motion information extraction unit 262 may extract motion information (a motion vector and/or a reference frame index, etc.) of the current block based on the received motion information, and the prediction sample extraction unit 263 may extract prediction samples of the current block.

[159] The image decoding equipment may generate residual samples of the current block based on the received residual information (S840). The image decoding equipment may generate reconstructed samples of the current block based on the predicted samples and the residual samples and generate a reconstructed frame based on this (S850). After this, the in-loop filtering procedure may be applied to the reconstructed frame as described above.

[160] As described above, the inter-prediction procedure may include a step of determining an inter-prediction mode, a step of extracting motion information according to the determined prediction mode, and a step of performing prediction (generating prediction samples) based on the extracted motion information. The inter-prediction procedure may be performed by the image coding equipment and the image decoding equipment as described above.

[161] The step of extracting motion information according to the prediction mode is described in more detail later in this document.

[162] As described above, the inter-prediction can be performed using the motion information of the current block. The image coding equipment can extract the optimal motion information of the current block through the motion estimation procedure. For example, the image coding equipment can search for a similar reference block with a high correlation in a predetermined search range in a reference frame using a source block in a source frame for the current block in a sub-pixel unit and extract motion information using it. The block similarity can be calculated based on the sum of absolute differences (SAD) between the current block and the reference block. In this case, the motion information can be extracted based on the reference block with the smallest SAD in the search area. The extracted motion information can be transmitted in service signals to the image decoding equipment according to various methods based on the inter-prediction mode.

[163] When the combining mode is applied to the current block, the motion information of the current block is not directly transmitted, and the motion information of the current block is derived using the motion information of the neighboring block. Accordingly, the motion information of the current prediction block may be indicated by transmitting flag information indicating that the combining mode is used and candidate selection information (for example, a combining index) indicating which neighboring block is used as a candidate for combining. In the present disclosure, since the current block is a unit of prediction performance, the current block may be used with a meaning identical to a meaning of the current prediction block, and the neighboring block may be used with a meaning identical to a meaning of the neighboring prediction block.

[164] The image coding equipment may search for possible candidates of merging units used to extract motion information of the current block to perform the merging mode. For example, up to five possible candidates of merging units may be used without being limited thereto. The maximum number of possible candidates of merging units may be transmitted in a slice header or in a tile group header without being limited thereto. After finding the possible candidates of merging units, the image coding equipment may form a list of possible merging units and select the possible merging unit candidate with the lowest RD cost as the final possible merging unit candidate.

[165] The present disclosure provides various embodiments for possible merge unit variants configuring a list of possible merge variants. The list of possible merge variants may use, for example, five possible merge unit variants. For example, four spatial possible merge variants and one temporal possible merge variant may be used.

[166] Fig. 13 is a view illustrating adjacent blocks available as a spatial possible combination option.

[167] Fig. 14 is a view schematically illustrating a method for constructing lists of possible combination options according to an example of the present disclosure.

[168] The image encoding/decoding equipment may insert, into the combination candidate list, spatial combination candidates extracted by searching for spatial neighboring blocks relative to the current block (S1110). For example, as shown in FIG. 13 , the spatial neighboring blocks may include a left lower corner neighboring block A ₀ , a left neighboring block A ₁ , a right upper corner neighboring block B ₀ , an upper neighboring block B ₁ and a left upper corner neighboring block B ₂ relative to the current block. However, this is an example, and in addition to the above-described spatial neighboring blocks, additional neighboring blocks such as a right neighboring block, a lower neighboring block and a right lower neighboring block may further be used as spatial neighboring blocks. The image encoding/decoding equipment may detect available blocks by searching for spatial neighboring blocks based on priority and extract motion information of the detected blocks as spatial combination candidates. For example, the image encoding/decoding equipment may construct a list of possible combination options by searching the five blocks shown in Fig. 13 in the order A ₁ , B ₁ , B ₀ , A ₀ , and B ₂ and sequentially indexing the available possible options.

[169] The image encoding/decoding equipment may insert, into the combination candidate list, a temporal combination candidate extracted by searching for temporal neighboring blocks relative to the current block (S1120). The temporal neighboring blocks may be located in a reference frame that is different from the current frame in which the current block is located. The reference frame in which the temporal neighboring block is located may be called a "co-located frame" or a "co-located frame". It is possible to search for the temporal neighboring block in the order of a lower right corner neighboring block and a lower right center block relative to the co-located block for the current block in the co-located frame. Meanwhile, when applying motion data compression to reduce the load on a memory device, specific motion information may be stored as motion characteristic information for each predetermined storage unit for the co-located frame. In this case, the motion information of all the blocks in the predetermined storage unit does not necessarily need to be stored, thereby obtaining the effect of motion data compression. In this case, the predetermined storage unit may be predetermined as, for example, a 16×16 sampling unit or an 8×8 sampling unit, or the size information of the predetermined storage unit may be transmitted in service signals from the image encoding equipment to the image decoding equipment. When applying motion data compression, the motion information of the temporal neighboring block may be replaced by motion characteristic information of the predetermined storage unit in which the temporal neighboring block is located. Thus, in this case, from the implementation perspective, the temporal combination candidate may be extracted based on the motion information of the prediction block covering the arithmetic left-shifted position after the arithmetic right shift by a predetermined value based on the coordinates (upper left sampling position) of the temporal neighboring block, rather than the prediction block located at the coordinates of the temporal neighboring block. For example, when the predetermined storage unit is a 2nx2n sampling unit and the coordinates of the temporary neighboring block are (xTnb, yTnb), the motion information of the prediction block located at the modified position ((xTnb>>n)<<n), (yTnb>>n)<<n)) can be used for the temporary combination candidate. Specifically, for example, when the predetermined storage unit is a 16x16 sampling unit and the coordinates of the temporary neighboring block are (xTnb, yTnb), the motion information of the prediction block located at the modified position ((xTnb>>4)<<4), (yTnb>>4)<<4)) can be used for the temporary combination candidate. Alternatively, for example, when the predetermined storage unit is an 8×8 sampling unit, and the coordinates of the temporal neighboring block are (xTnb, yTnb), the motion information of the prediction block located at the modified position ((xTnb>>3)<<3), (yTnb>>3)<<3)) can be used for the temporal possible merging option.

[170] Referring again to Fig. 14, the image encoding/decoding equipment may check whether the number of current possible combination options is less than the maximum number of possible combination options (S1130). The maximum number of possible combination options may be preset or transmitted in signaling from the image encoding equipment to the image decoding equipment. For example, the image encoding equipment may generate and encode information regarding the maximum number of possible combination options and transmit the encoded information to the image decoding equipment in the form of a bit stream. When the maximum number of possible combination options is satisfied, the subsequent process S1140 of adding the possible options may not be performed.

[171] When the number of current possible combination options is less than the maximum number of possible combination options as a result of checking step S1130, the image encoding/decoding equipment may extract an additional possible combination option according to a predetermined method and then insert the additional possible combination option into the possible combination option list (S1140).

[172] When the number of current combination candidates is not less than the maximum number of combination candidates as a result of the check of step S1130, the image encoding/decoding equipment may complete the construction of the combination candidate list. In this case, the image encoding equipment may select an optimum combination candidate from among the combination candidates configuring the combination candidate list, and transmit, in signaling, the choice information of choice (for example, a combination index) indicating the selected combination candidate to the image decoding equipment. The image decoding equipment may select the optimum combination candidate based on the combination candidate list and the choice information.

[173] The motion information of the selected possible combination option may be used as the motion information of the current block, and the predicted samples of the current block may be extracted based on the motion information of the current block, as described above. The image encoding equipment may extract the residual samples of the current block based on the predicted samples and the residual information of the signal of the residual samples to the image decoding equipment. The image decoding equipment may generate reconstructed samples based on the residual samples extracted based on the residual information and the predicted samples, and generate a reconstructed frame based on them, as described above.

[174] When applying the skip mode to the current block, the motion information of the current block can be extracted using a method identical to that in the case of applying the merging mode. However, when applying the skip mode, the residual signal for the corresponding block is omitted, and therefore the predicted samples can be directly used as reconstructed samples.

[175] When applying the MVP mode to the current block, a list of candidate motion vector predictors (MVPs) may be formed using the motion vector of the reconstructed spatial neighboring blocks (e.g., the neighboring blocks shown in Fig. 13) and/or the motion vector corresponding to the temporal neighboring blocks (or co-located blocks). In this way, the motion vector of the reconstructed spatial neighboring blocks and the motion vector corresponding to the temporal neighboring blocks may be used as candidate motion vector predictors of the current block. When applying biprediction, a list of candidate MVPs for extracting L0 motion information and a list of candidate MVPs for extracting L1 motion information are separately formed and used. The prediction information (or information regarding the prediction) of the current block may include candidate selection information (for example, an MVP flag or an MVP index) indicating an optimal candidate motion vector predictor selected from among the candidate motion vector predictor candidates included in the MVP candidate list. In this case, the prediction module may select a motion vector predictor of the current block from among the candidate motion vector predictor candidates included in the MVP candidate list using the candidate selection information. The prediction module of the image coding equipment may obtain and encode a motion vector difference (MVD) between the motion vector of the current block and the motion vector predictor, and output the encoded MVD in the form of a bit stream. In this way, the MVD may be obtained by subtracting the motion vector predictor from the motion vector of the current block. The prediction module of the image decoding equipment may obtain the motion vector difference included in the prediction information and extract the motion vector of the current block by summing the motion vector difference and the motion vector predictor. The prediction module of the image coding equipment may obtain or extract a reference frame index indicating a reference frame from the prediction information.

[176] Fig. 15 is a view schematically illustrating a method for constructing lists of possible motion vector predictor options according to an example of the present disclosure.

[177] First, a spatial possible block variant may be searched for relative to the current block, and the available possible block variants may be inserted into the possible MVP variant list (S1210). After that, it is determined whether or not the number of possible MVP variants included in the possible MVP variant list is less than 2 (S1220), and when the number of possible MVP variants is two, the construction of the possible MVP variant list may be completed.

[178] In step S1220, when the number of available spatial block candidates is less than 2, a search for a temporary block candidate relative to the current block may be performed, and the available block candidates may be inserted into the MVP candidate list (S1230). When temporary block candidates are not available, a zero motion vector may be inserted into the MVP candidate list, thereby completing the construction of the MVP candidate list.

[179] Meanwhile, when the MVP mode is applied, the reference frame index can be explicitly transmitted in the overhead signals. In this case, the reference frame index refidxL0 for L0 prediction and the reference frame index refidxL1 for L1 prediction can be transmitted in the overhead signals in a distinct manner. For example, when the MVP mode is applied and biprediction is applied, both the information regarding refidxL0 and the information regarding refidxL1 can be transmitted in the overhead signals.

[180] As described above, when applying the MVP mode, information regarding the MVP extracted by the image encoding equipment may be transmitted in signaling to the image decoding equipment. The information regarding the MVD may include, for example, information indicating the X-axis and Y-axis components of the absolute value (absolute MVD value) and the sign of the MVD. In this case, when the absolute MVD value exceeds 0, whether or not the absolute MVD value exceeds 1 and information indicating the MVD remainder may be transmitted in signaling in steps. For example, information indicating whether or not the absolute MVD value exceeds 1 may be transmitted in signaling only when the value of the flag information indicating whether or not the absolute MVD value exceeds 0 is 1.

[181] Fig. 16 is a view illustrating a syntax structure for transmitting an MVD from an image encoding equipment to an image decoding equipment according to an embodiment of the present disclosure.

[182] In Fig. 16, abs_mvd_greater0_flag[0] indicates whether or not the absolute value of the x-axis component of the MVD exceeds 0, and abs_mvd_greater0_flag[1] indicates whether or not the absolute value of the y-axis component of the MVD exceeds 0. Similarly, abs_mvd_greater1_flag[0] indicates whether or not the absolute value of the x-axis component of the MVD exceeds 1, and abs_mvd_greater1_flag[1] indicates whether or not the absolute value of the y-axis component of the MVD exceeds 1. As shown in Fig. 16, abs_mvd_greater1_flag can be transmitted only when abs_mvd_greater0_flag is 1. In Fig. 16, abs_mvd_minus2 may indicate a value obtained by subtracting 2 from the absolute value of the MVD, and mvd_sign_flag indicates whether the sign of the MVD is positive or negative. Using the syntactic structure shown in Fig. 16, the MVD can be extracted as shown in Equation 1 below.

[183] equation 1

[184] MVD[compIdx]=abs_mvd_greater0_flag[compIdx]*(abs_mvd_minus2[compIdx]+2)*(1-2*mvd_sign_flag[compIdx])

[185] Meanwhile, the MVD (MVDL0) for L0 prediction and the MVD (MVDL1) for L1 prediction may be differentially transmitted in the signaling, and the information regarding the MVD may include the information regarding the MVDL0 and/or the information regarding the MVDL1. For example, when applying the MVP mode and applying bi-prediction to the current block, both the information regarding the MVDL0 and the information regarding the MVDL1 may be transmitted in the signaling.

[186] General view of intra-block copy (IBC) prediction

[187] This document further describes IBC forecasting in accordance with this disclosure.

[188] IBC prediction may be performed by a prediction module of image encoding/decoding equipment. IBC prediction may be referred to simply as "IBC". IBC may be used for content/moving image coding, such as screen content coding (SCC). IBC prediction may be performed essentially in the current frame, but may be performed similarly to inter prediction in that a reference block is extracted within the current frame. Thus, IBC may use at least one of the inter prediction technologies described in this disclosure. For example, IBC may use at least one of the above-described methods for extracting motion information (motion vectors). At least one of the inter prediction technologies may be partially modified and used taking into account IBC prediction. IBC may refer to the current frame and as such may be referred to as "current frame reference (CPR)".

[189] For IBC, the image encoding equipment 100 may perform block matching (BM) and extract an optimal block vector (or motion vector) for the current block (e.g., CU). The extracted block vector (or motion vector) may be signaled to the image decoding equipment through a bitstream using a method similar to signaling motion information (motion vector) in the above-described inter-prediction. The image decoding equipment may extract a reference block for the current block in the current frame through the signaled block vector (motion vector) and extract a prediction signal (predicted block or prediction samples) for the current block through it. Here, the block vector (or motion vector) may indicate an offset from the current block to a reference block located in an already reconstructed area in the current frame. Accordingly, the block vector (or motion vector) may be called an "offset vector". Hereinafter in this document, in IBC, the motion vector may correspond to a block vector or a displacement vector. The motion vector of the current block may include a motion vector for a luminance component (luminance motion vector) or a motion vector for a chrominance component (chrominance motion vector). For example, the luminance motion vector for an IBC-coded CU may be an integer sample unit (i.e., integer precision). The chrominance motion vector may be clipped in integer sample units. As described above, IBC may use at least one of the inter-prediction technologies, and, for example, the luminance motion vector may be encoded/decoded using the above-described fusion mode or MVP mode.

[190] When applying the combining mode to an IBC luminance signal block, a list of possible combining options for the IBC luminance signal block may be constructed similar to the list of possible combining options in the mutual mode described with reference to Fig. 14. However, in the case of an IBC luminance signal block, a temporal neighboring block may not be used as a possible combining option.

[191] When applying the MVP mode to an IBC luminance signal block, the list of possible MVP variants for the IBC luminance signal block may be constructed similarly to the list of possible MVP variants in the mutual mode described with reference to Fig. 15. However, in the case of an IBC luminance signal block, the time candidate variant of the block may not be used as a possible MVP variant.

[192] In IBC, the reference block is retrieved from an already reconstructed area in the current frame. In this case, in order to reduce the memory consumption and the complexity of the image decoding equipment, only a predetermined area can be referenced from the already reconstructed areas in the current frame. The predetermined area may include the current CTU in which the current block is included. By limiting the referenceable reconstructed area to the predetermined area, the IBC mode can be implemented in hardware using a local on-chip memory.

[193] The image coding equipment for performing IBC may search in a predetermined area to determine a reference block with the lowest RD cost and extract a motion vector (block vector) based on the positions of the reference block and the current block.

[194] Whether or not to apply IBC to the current block may be signaled as IBC performance information at the CU level. Information regarding the signaling method (IBC MVP mode or IBC skip/merge mode) of the motion vector of the current block may be signaled. The IBC performance information may be used to determine the prediction mode of the current block. Accordingly, the IBC performance information may be included in the information regarding the prediction mode of the current block.

[195] In the case of the IBC skip/combine mode, the combination candidate index may be signaled in the overhead to indicate the block vector that is to be used to predict the current luminance signal block from the block vectors included in the combination candidate list. In this case, the combination candidate list may include IBC-coded neighboring blocks. The combination candidate list may be configured to include spatial combination candidates and not include temporal combination candidates. In addition, the combination candidate list may further include history-based motion vector predictor (HMVP) candidates and/or pairwise candidates.

[196] In the case of the IBC MVP mode, the difference value of the block vectors may be encoded using a method identical to the method for the difference value of the motion vectors of the above-described mutual mode. The block vector prediction method may construct and use a list of candidate MVP options, including two candidate options, as predictors similar to the MVP mode for the mutual mode. One of the two candidate options may be extracted from the left neighboring block, and the other candidate option may be extracted from the upper neighboring block. In this case, only when the left or upper neighboring block is IBC-encoded, the candidate options may be extracted from the corresponding neighboring block. If the left or upper neighboring block is not available, for example, not IBC-encoded, the default block vector may be included in the list of candidate MVP options as a predictor. In addition, information (e.g., a flag) indicating one of the two block vector predictors is transmitted in the service signals and is used as the selection information of possible options similar to the MVP mode for the mutual mode. The list of possible MVP options may include a possible HMVP option and/or a zero motion vector as the default block vector.

[197] The HMVP candidate may be called a "history-based MVP candidate", and the MVP candidate used before encoding/decoding the current block, the merge candidate, or the block vector candidate may be stored in the HMVP list as HMVP candidate candidates. After that, when the merge candidate list of the current block or the MVP candidate list does not include the maximum number of candidate candidates, the candidate candidates stored in the HMVP list may be added to the merge candidate list or the MVP candidate list of the current block as HMVP candidate candidates.

[198] A pairwise candidate means a candidate that is extracted by selecting two candidate options according to a predetermined order from among the candidate options already included in the list of candidate options for combining the current block and averaging the selected two candidate options.

[199] Fig. 17 is a flow chart illustrating a video/image coding method based on IBC.

[200] Fig. 18 is a view illustrating the configuration of a prediction module for implementing the IBC-based video/image coding method according to the present disclosure.

[201] The encoding method of Fig. 17 may be performed by the image encoding equipment of Fig. 2. Specifically, step S1410 may be performed by a prediction module, and step S1420 may be performed by a residual processor. Specifically, step S1420 may be performed by a subtractor 115. Step S1430 may be performed by an entropy encoder 190. The prediction information of step S1430 may be extracted by the prediction module, and the residual information of step S1430 may be extracted by the residual processor. The residual information is information regarding residual samples. The residual information may include information regarding quantized transform coefficients for the residual samples. As described above, the residual samples can be extracted by means of the transform coefficients via the converter 120 of the image encoding equipment, and the transform coefficients can be extracted by means of the transform coefficients quantized via the quantizer 130. The information regarding the quantized transform coefficients can be encoded by means of the entropy encoder 190 via the residual encoding procedure.

[202] The image coding equipment may perform IBC prediction (IBC-based prediction) for the current block (S1410). The image coding equipment may extract a prediction mode and a motion vector (block vector) of the current block and generate prediction samples of the current block. The prediction mode may include at least one of the above-described inter-prediction modes. Here, the procedures of determining the prediction mode, extracting the motion vectors and generating the prediction samples may be performed simultaneously, or any procedure may be performed before the other procedures. For example, as shown in FIG. 18, a prediction module of the image coding equipment for implementing the IBC-based video/image coding method may include a prediction mode determining module, a motion vector extracting module and a prediction sample extracting module. The prediction mode determining module may determine the prediction mode of the current block, the motion vector extracting module may extract the motion vector of the current block, and the prediction sample extracting module may extract the prediction samples of the current block. For example, the prediction module of the image coding equipment may search for a block similar to the current block in the reconstructed area (or in a certain area (search area) of the reconstructed area) of the current frame and extract a reference block whose difference with respect to the current block is equal to or less than a certain criterion or minimum. The image coding equipment may extract a motion vector based on the offset difference between the reference block and the current block. The image coding equipment may determine a mode applied to the current block from among various prediction modes. The image coding equipment may compare RD costs for various prediction modes and determine an optimal prediction mode for the current block. However, the method for determining the prediction mode for the current block by the image coding equipment is not limited to the above example, and various methods may be used.

[203] For example, when applying the skip mode or the combining mode to the current block, the image coding equipment may extract combining candidates from neighboring blocks relative to the current block and construct a combining candidate list using the extracted combining candidates. In addition, the image coding equipment may extract a reference block whose difference relative to the current block is equal to or less than a certain criterion or minimum from the reference blocks indicated by the combining candidates included in the combining candidate list. In this case, a combining candidate associated with the extracted reference block may be selected, and combining index information indicating the selected combining candidate may be generated and transmitted in service signals to the image decoding equipment. Using the motion vector of the selected combining candidate, the motion vector of the current block may be extracted.

[204] As another example, when applying the MVP mode to the current block, the image coding equipment may extract motion vector predictor candidates (MVPs) from neighboring blocks relative to the current block and construct a list of possible MVPs using the extracted possible MVPs. In addition, the image coding equipment may use a motion vector of a possible MVP selected from among the possible MVPs included in the list of possible MVPs as the MVP of the current block. In this case, for example, a motion vector indicating a reference block extracted by the above-described motion estimation may be used as a motion vector of the current block, and a possible MVP having the smallest difference with respect to the motion vector of the current block from the possible MVPs may become the selected possible MVP. A motion vector difference (MVD) that is obtained by subtracting the MVP from the motion vector of the current block may be extracted. In this case, index information indicating the selected possible MVP variant and information regarding the MVD may be transmitted in service signals to the image decoding equipment.

[205] The image encoding equipment may extract residual samples based on the predicted samples (S1420). The encoding equipment may extract residual samples through a comparison between the original samples of the current block and the predicted samples. For example, the residual sample may be extracted by subtracting the corresponding predicted sample from the original sample.

[206] The image encoding equipment may encode image information including prediction information and residual information (S1430). The image encoding equipment 100 may output the encoded image information in the form of a bit stream. The prediction information may include information regarding prediction mode information (for example, a skip flag, a combining flag, or a mode index) and motion information as information related to a prediction procedure. Of the prediction mode information, the skip flag indicates whether or not to apply the skip mode to the current block, and the combining flag indicates whether or not to apply the combining mode to the current block. Alternatively, the prediction mode information may indicate one of a plurality of prediction modes, for example, a mode index. When the skip flag and the combining flag are 0, it may be determined that the MVP mode is applied to the current block. The information regarding the motion vector may include candidate selection information (for example, a combining index, an MVP flag, or an MVP index), which is information for extracting a motion vector. Of the candidate selection information, the combination index may be transmitted in signaling when applying the combination mode to the current block, and may be information for selecting one of the possible combination options included in the list of possible combination options. Of the candidate selection information, the MVP flag or the MVP index may be transmitted in signaling when applying the MVP mode to the current block, and may be information for selecting one of the possible MVP options included in the list of possible MVP options. In addition, the information regarding the motion vector may include information regarding the above-described MVD. In addition, the information regarding the motion vector may include information indicating whether L0 prediction, L1 prediction, or biprediction should be applied. The residual information is information regarding the residual samples. The residual information may include information regarding quantized transform coefficients for the residual samples.

[207] The output bit stream may be stored on a (digital) storage medium and transmitted to image decoding equipment, or may be transmitted to image decoding equipment via a network.

[208] Meanwhile, as described above, the image encoding equipment may form a reconstructed frame (a frame including reconstructed samples and a reconstructed block) based on the reference samples and the residual samples. This serves to extract, by the image encoding equipment, a prediction result identical to the prediction performed by the image decoding equipment, thereby improving the encoding efficiency. Accordingly, the image encoding equipment may store the reconstructed frame (or reconstructed samples and a reconstructed block) in a memory device and use it as a reference frame for inter-prediction. An in-loop filtering procedure may further be applied to the reconstructed frame as described above.

[209] Fig. 19 is a flow chart illustrating a video/image decoding method based on IBC.

[210] Fig. 20 is a view illustrating the configuration of a prediction module for implementing the IBC-based video/image decoding method according to the present disclosure.

[211] The image decoding equipment may perform an operation corresponding to the operation performed by the image encoding equipment. The image decoding equipment may perform IBC prediction for the current block based on the received prediction information to extract prediction samples.

[212] The decoding method of Fig. 19 may be performed by the image decoding equipment of Fig. 3. Steps S1610 to S1630 may be performed by the prediction module and the prediction information of step S1610, and the residual information of step S1640 may be obtained from the bit stream by the entropy decoder 210. The residual processor of the image decoding equipment may extract residual samples for the current block based on the residual information (S1640). Specifically, the dequantizer 220 of the residual processor may perform dequantization based on the dequantized transform coefficients extracted based on the residual information to extract the transform coefficients, and the inverse transformer 230 of the residual processor may perform inverse transformation for the transform coefficients to extract residual samples for the current block. Step S1650 may be performed by the adder 235 or the restorer.

[213] In particular, the image decoding equipment may determine a prediction mode of the current block based on the received prediction information (S1610). The image decoding equipment may determine which prediction mode is applied to the current block based on the prediction mode information in the prediction information.

[214] For example, it may be determined whether or not to apply the skip mode to the current block based on the skip flag. In addition, it may be determined whether to apply the merge mode or the MVP mode to the current block based on the merge flag. Alternatively, one of the various possible prediction mode options may be selected based on the mode index. The possible inter-prediction mode options may include the skip mode, the merge mode, and/or the MVP mode, or may include the various inter-prediction modes described above.

[215] The image encoding equipment may extract a motion vector of the current block based on a determined prediction mode (S1620). For example, when applying a skip mode or a combination mode to the current block, the image decoding equipment may construct the above-described combination candidate list and select one of the combination modes included in the combination candidate list. The selection may be performed based on the above-described choice information (combination index). The motion vector of the current block may be extracted using the motion vector of the selected combination candidate. For example, the motion vector of the selected combination candidate may be used as the motion vector of the current block.

[216] As another example, when applying the MVP mode to the current block, the image decoding equipment may construct a list of possible MVPs and use a motion vector of a possible MVP selected from among the possible MVPs included in the list of possible MVPs as the MVP of the current block. The selection may be performed based on the above-described selection information of the possible variants (MVP flag or MVP index). In this case, the MVD of the current block may be extracted based on the information regarding the MVD, and the motion vector of the current block may be extracted based on the MVP and MVD of the current block.

[217] The image decoding equipment may generate predicted samples of the current block based on the motion vector of the current block (S1630). The predicted samples of the current block may be extracted using samples of a reference block indicated by the motion vector of the current block in the current frame. In this case, a filtering procedure of the predicted samples for all or some of the predicted samples of the current block may be further performed.

[218] For example, as shown in Fig. 20, a prediction module of the image decoding equipment for implementing the video/image decoding method based on IBC may include a prediction mode determination module, a motion vector extraction module, and a prediction sample extraction module. The prediction module of the image decoding equipment may determine a prediction mode for the current block based on the received prediction mode information in the prediction mode determination module, extract a motion vector of the current block based on the received information regarding the motion vector in the motion vector extraction module, and extract prediction samples of the current block in the prediction sample extraction module.

[219] The image decoding equipment may generate residual samples of the current block based on the received residual information (S1640). The image decoding equipment may generate reconstructed samples for the current block based on the predicted samples and the residual samples and generate a reconstructed frame based on them (S1650). After that, the in-loop filtering procedure may further be applied to the reconstructed frame as described above.

[220] As described above, one unit (for example, a coding unit (CU)) may include a luminance signal block (luminance signal coding block (CB)) and a chrominance signal block (chrominance signal CB). In this case, the luminance signal block and the chrominance signal block corresponding to it may have identical motion information (for example, a motion vector) or different motion information. For example, the motion information of the chrominance signal block may be extracted based on the motion information of the luminance signal block, so that the luminance signal block and the chrominance signal block corresponding to it have identical motion information.

[221] General presentation of the color signal format

[222] Hereinafter, this document describes a chrominance signal format. An image may be encoded into coded data including an array of luminance signal components (e.g., Y) and two arrays of chrominance signal components (e.g., Cb and Cr). For example, one pixel of an encoded image may include a luminance sample and a chrominance sample. The chrominance signal format may be used to represent a configuration format of a luminance sample and a chrominance sample, and the chrominance signal format may be referred to as a "color format."

[223] In an embodiment, the image may be encoded in various chroma signal formats, such as monochrome, 4:2:0, 4:2:2, or 4:4:4. With monochrome sampling, one array of samples may be provided, and the array of samples may be an array of luminance signals. With 4:2:0 sampling, one array of luminance signal samples and two arrays of chroma signal samples may be provided, each of the two chroma signal arrays may have a height equal to half the height of the array of luminance signals and a width equal to half the width of the array of luminance signals. With 4:2:2 sampling, one array of luminance signal samples and two arrays of chroma signal samples may be provided, each of the two chroma signal arrays may have a height equal to the height of the array of luminance signals and a width equal to half the width of the array of luminance signals. In 4:4:4 sampling, there may be one array of luminance samples and two arrays of chrominance samples, and each of the two chrominance arrays may have a height and width equal to the height and width of the luminance array.

[224] For example, in 4:2:0 sampling, a chroma sample may be located below a luminance sample corresponding to it. In 4:2:2 sampling, a chroma sample may be positioned to overlap a luminance sample corresponding to it. In 4:4:4 sampling, both a luminance sample and a chroma sample may be positioned in an overlapping position.

[225] The format of the chroma signals used in the encoding equipment and the decoding equipment may be predetermined. Alternatively, the format of the chroma signals may be signaled from the encoding equipment to the decoding equipment with the ability to be adaptively used in the encoding equipment and the decoding equipment. In an embodiment, the format of the chroma signals may be signaled based on at least one of chroma_format_idc or separate_colour_plane_flag. At least one of chroma_format_idc or separate_colour_plane_flag may be signaled through a high-level syntax such as DPS, VPS, SPS or PPS. For example, chroma_format_idc and separate_colour_plane_flag may be included in the SPS syntax shown in FIG. 21.

[226] Meanwhile, Fig. 22 shows an embodiment of classifying chroma signal formats using transmission in the chroma_format_idc and separate_colour_plane_flag service signals; chroma_format_idc may be information indicating a chroma signal format applied to an encoded image; separate_colour_plane_flag may indicate whether a color array is processed separately in a specific chroma signal format. For example, the first value (e.g., 0) of chroma_format_idc may indicate monochrome sampling. The second value (e.g., 1) of chroma_format_idc may indicate 4:2:0 sampling. The third value (e.g., 2) of chroma_format_idc may indicate 4:2:2 sampling. The fourth value (e.g., 3) of chroma_format_idc may indicate 4:4:4 sampling.

[227] For 4:4:4, the following may apply based on the value of separate_colour_plane_flag. If the value of separate_colour_plane_flag is the first value (e.g., 0), each of the two chroma sample arrays may have a height and width identical to the height and width of the luma sample array. In this case, the value of ChromaArrayType, which specifies the type of chroma sample array, may be set to chroma_format_idc. If the value of separate_colour_plane_flag is the second value (e.g., 1), the luma sample arrays, Cb samples, and Cr samples may be processed separately and processed along with the monochrome sampled frames. In this case, ChromaArrayType may be set to 0.

[228] Intra prediction for the chrominance block

[229] When intra prediction is performed for the current block, prediction with respect to a block of luminance signal components (luminance signal block) of the current block and prediction with respect to a block of chrominance signal components (chrominance signal block) may be performed. In this case, the intra prediction mode for the chrominance signal block may be set separately from the intra prediction mode for the luminance signal block.

[230] For example, an intra-prediction mode for a chroma block may be indicated based on the intra-prediction mode information of the chroma, and the intra-prediction mode information of the chroma may be transmitted in the overhead signaling in the form of an intra_chroma_pred_mode syntax element. For example, the intra-prediction mode information of the chroma may represent one of a planar mode, a DC mode, a vertical mode, a horizontal mode, a derived mode (DM), and a cross-component linear model (CCLM)-based mode. Here, the planar mode may indicate the intra-prediction mode #0, the DC mode may indicate the intra-prediction mode #1, the vertical mode may indicate the intra-prediction mode #26, and the horizontal mode may indicate the intra-prediction mode #10. DM may also be referred to as a "direct mode." CCLM may also be referred to as a "linear model (LM)."

[231] Meanwhile, DM and CCLM are dependent intra-prediction modes for predicting a chrominance signal block using information about a luminance signal block. DM may represent a mode in which an intra-prediction mode identical to the intra-prediction mode for the luminance signal component is applied as an intra-prediction mode for the chrominance signal component. In addition, CCLM may represent an intra-prediction mode using, as prediction samples of the chrominance signal block, samples extracted by subsampling the reconstructed samples of the luminance signal block and then applying α and β, which are CCLM parameters, to the subsampled samples in the process of generating a prediction block for the chrominance signal block.

[232] equation 2

[233]

[234] where predc(i, j) may denote a predicted sample of the coordinates (i, j) of the current chroma block in the current CU; rec _L '(i, j) may denote a reconstructed sample of the coordinates (i, j) of the current luma block in the CU. For example, rec _L '(i, j) may denote a downsampled reconstructed sample of the current luma block. The coefficients α and β of the linear model may be carried in the signaling or extracted from neighboring samples.

[235] Virtual Pipeline Data Unit

[236] Virtual Pipeline Data Units (VPDUs) may be defined for pipeline processing within a frame. VPDUs may be defined as non-overlapping units within a single frame. In decoding hardware, consecutive VPDUs may be processed simultaneously by multiple pipeline stages. In most pipeline stages, the VPDU size may be roughly proportional to the buffer size. Accordingly, keeping the VPDU size small is important when considering the buffer size from a hardware perspective. In most decoding hardware, the VPDU size may be set equal to the maximum transform block (TB) size. For example, the VPDU size may be 64×64 (64×64 luminance samples). Alternatively, in VVC, the VPDU size may vary (increase or decrease) based on the ternary tree (TT) and/or binary tree (BT) segmentation described above.

[237] Meanwhile, in order to maintain the VPDU size when sampling 64×64 luminance signal samples, the CU partitioning shown in Fig. 23 may be limited. More specifically, at least one of the following restrictions may be applied.

[238] Constraint 1: Ternary tree (TT) partitioning is not allowed for a CU that has a width or height of 128, or a width and height of 128.

[239] Constraint 2: Horizontal binary tree (BT) partitioning is not allowed for a CU that has 128xN (where N is an integer equal to or less than 64 and greater than 0) (e.g., horizontal binary tree partitioning is not allowed for a CU that has a width of 128 and a height less than 128).

[240] Constraint 3: Vertical binary tree (BT) partitioning is not allowed for a CU that has Nx128 (where N is an integer equal to or less than 64 and greater than 0) (e.g., vertical binary tree partitioning is not allowed for a CU that has a height of 128 and a width less than 128).

[241] The problem of limiting the maximum block size of color signals for pipeline processing

[242] As described above with respect to the segmentation structure and the transform process, a CU may be partitioned to form a plurality of TUs. When the size of a CU exceeds a maximum TU size, the CU may be partitioned into a plurality of TUs. Therefore, a transform and/or an inverse transform may be performed for each TU. In general, the maximum TU size for a luminance signal block may be set equal to the maximum available transform size that can be performed by the encoding equipment and/or the decoding equipment. Examples of partitioning a CU and a TU according to an embodiment are shown in FIGS. 24-26.

[243] Fig. 24 shows an example of a TU formed by splitting a luminance signal CU and a color signal CU according to an embodiment. In the embodiment, the maximum size of a luminance signal CU may be 64×64, the maximum available transform size may be 32×32, and a non-square TU may not be allowed. Therefore, the maximum size of a luminance signal component transform block may be 32×32. In this embodiment, the maximum TU size may be set as shown in the following equation.

[244] equation 3

[245] maxTbSize=(cIdx==0)?MaxTbSizeY:MaxTbSizeY/max(SubWidthC, SubHeightC)

[246] In the above equation, maxTbSize may represent the maximum size of a transform block (TB), and cIdx may represent a color component of the corresponding block; cIdx 0 may denote a luminance component, 1 may denote a Cb chrominance component, and 2 may denote a Cr chrominance component. MaxTbSizeY may denote the maximum size of a transform block of luminance components, SubWidthC may denote the ratio of the width of a luminance block to the width of a chrominance block, SubHeightC may denote the ratio of the height of a luminance block to the height of a chrominance block, and max(A, B) may denote a function for returning a larger value of A and B as a result value.

[247] According to the above equation, in the above embodiment, in the case of a luminance signal block, the maximum size of the transformation block may be set as the maximum size of the transformation block of the luminance signal components. Here, the maximum size of the transformation block of the luminance signal components is a value set during encoding, and may be transmitted in service signals from the encoding equipment to the decoding equipment through a bit stream.

[248] In addition, in the above embodiment, the maximum size of the transformation block of the chroma signal block may be set to a value obtained by dividing the maximum size of the transformation block of the luminance signal components by a larger value of SubWidthC and SubHeightC. Here, SubWidthC and SubHeightC may be determined based on chroma_format_idc and separate_colour_plane_flag transmitted in the service signals from the encoding equipment to the decoding equipment through a bit stream, as shown in FIG. 23.

[249] According to the above equation, in the above embodiment, the maximum size of the transformation unit may be determined as any one of the minimum width and the minimum height of the transformation unit. Therefore, the TU division of the luminance signal unit and the chrominance signal unit in the above embodiment may be performed as shown in Fig. 24. For example, as shown in Fig. 24, in the case of a chrominance signal unit having a 4:2:2 format, since the maximum size of the transformation unit is determined to be 16, the chrominance CU may be divided into a plurality of transformation units, in a form different from the form of division of the luminance CU into transformation units.

[250] FIG. 25 shows an example of a TU formed by splitting a luminance CU and a chrominance CU according to another embodiment. In the embodiment, the maximum size of a luminance CU may be 128×128, the maximum available transform size may be 64×64, and a non-square TU may not be allowed. Therefore, the maximum size of a transform block of luminance components may be 64×64. In this embodiment, the maximum size of the transform block may be set as shown in the following equation. In the following equation, min(A, B) may be a function for returning the smaller value of A and B.

[251] equation 4

[252] maxTbSize=(cIdx==0)?MaxTbSizeY:MaxTbSizeY/min(SubWidthC, SubHeightC)

[253] Meanwhile, according to the above equation, since the larger value of the width and height of the corresponding block is applied as the maximum size of the transformation block, the CU of luminance signals and the CU of color signals can be divided into a plurality of TUs similar to the example in Fig. 25.

[254] In the examples of Figs. 24 and 25, when a CU of chrominance signals having a 4:2:2 format is divided into TUs, it is divided in a form different from the form of dividing the corresponding CU of luminance signals into TUs. However, when encoding/decoding of a chrominance signal block is performed with reference to a luminance signal block, similar to the DM mode or CCLM mode for predicting a chrominance signal block described above, performing encoding (or decoding) of the corresponding luminance signal block immediately after encoding (or decoding) of the luminance signal block corresponding to the chrominance signal block is effective in reducing a delay in pipeline processing and saving memory.

[255] However, in the example of Fig. 24, encoding of two chroma signal blocks 2421 and 2423 must be performed after encoding of one luminance signal converting block 2411, which requires separate processing in connection with different color formats (4:4:4 or 4:2:0). In addition, in the example of Fig. 25, encoding of one chroma signal converting block 2521 must be performed after encoding of two luminance signal converting blocks 2511 and 2512. Thus, in the above-described TU partitioning method, when the 4:2:2 format is used, since the luminance signal block and the chroma signal block corresponding to it do not match, a separate process may be added to perform pipelining, or the pipelining may not be performed.

[256] Limiting the Maximum Chroma Conversion Block Size for Pipeline Processing

[257] This document further describes a method for specifying the maximum transform block size for a CU of color signals so as to satisfy the condition for executing the above-described VPDU.

[258] Fig. 26 shows an example of a TU formed by partitioning a luminance signal CU and a color signal CU according to another embodiment. In the embodiment, the maximum size of the luminance signal CU may be 128×128, the maximum available transform size may be 64×64, and partitioning into a non-square TU may be allowed. Therefore, the maximum size of the transform block of the luminance signal components may be 64×64.

[259] As shown in Fig. 26, for partitioning into non-square TUs, the maximum size of the transformation block may be specified for the width and height. For example, the maximum width (maxTbWidth) of the transformation block and the maximum height (maxTbHeight) of the transformation block may be specified as shown in the following equations, thereby specifying the maximum size of the transformation block.

[260] equation 5

[261] maxTbWidth=(cIdx==0)?MaxTbSizeY:MaxTbSizeY/SubWidthC

[262] equation 6

[263] maxTbHeight=(cIdx==0)?MaxTbSizeY:MaxTbSizeY/SubHeightC

[264] Similar to the above-described embodiment, by setting the maximum size of the transform block as the width and the height as shown in the example of FIG. 26, even in the case of a chrominance signal CU having a 4:2:2 format, similar to the form of dividing the corresponding luminance signal CU into TUs, the chrominance signal CU can be divided into TUs. Therefore, by dividing the chrominance signal CU into TUs to correspond to the TU of the luminance signal CU, immediately after encoding (or decoding) the luminance signal block, encoding (or decoding) the chrominance signal block corresponding to it can be performed, thereby reducing the delay in pipeline processing.

[265] Limiting the maximum size of the color signal conversion block in intra prediction mode

[266] This document further describes the execution of an intra prediction mode to which the limitation of the maximum size of the chrominance conversion block is applied for the above-described chrominance pipeline processing. The encoding equipment and the decoding equipment can perform intra prediction by limiting the maximum size of the chrominance conversion block as described below, and their operations can correspond to each other. Therefore, this document further describes the operation of the decoding equipment.

[267] The decoding equipment according to the embodiment may generate a reconstructed frame by performing intra-prediction. In-loop filtering may be performed on the reconstructed frame. The decoding equipment according to the embodiment may directly obtain the next information from the bit stream or extract the next information from other information obtained from the bit stream to perform intra-prediction.

[268] - Sample position (xTb0, yTb0) indicating the position of the top-left sample of the current transform block relative to the position of the top-left sample of the current frame

[269] - nTbW parameter specifies the width of the current transformation block

[270] - nTbH parameter, which specifies the height of the current transformation block

[271] - predModeIntra parameter, indicating the internal prediction mode of the current CU

[272] - cIdx parameter indicating the color component of the current CU

[273] The decoding equipment may extract the maximum width maxTbWidth of the transform block and the maximum height maxTbHeight of the transform block from the received information as follows.

[274] equation 7

[275] maxTbWidth=(cIdx==0)?MaxTbSizeY:MaxTbSizeY/SubWidthC

[276] equation 8

[277] maxTbHeight=(cIdx==0)?MaxTbSizeY:MaxTbSizeY/SubHeightC

[278] In addition, the decoding equipment may extract the upper-left sample position (xTbY, yTbY) of the current transform block based on whether the current CU represents a luminance signal component or a chrominance signal component, as follows.

[279] equation 9

[280] (xTbY, yTbY)=(cIdx==0)?(xTb0, yTb0):(xTb0*SubWidthC, yTb0*SubHeightC)

[281] Hereinafter in this document, the decoding equipment may perform intra prediction by performing the following procedure. First, the decoding equipment may determine whether the current transform block is split or not (S2710). For example, the decoding equipment may determine whether the current transform block is split or not based on whether the width and the height of the current transform block exceed the width and the height of the maximum transform block. In addition, the decoding equipment may determine whether splitting is performed or not by further considering whether an intra sub-segment (ISP) is applied to the current CU. For example, when nTbW exceeds maxTbWidth, or nTbH exceeds maxTbHeight, the decoding equipment may determine that intra prediction is performed by splitting the current transform block. In addition, even in this case, the decoding equipment may determine that intra prediction is performed by splitting the current transform block only when the ISP does not apply to the current CU (e.g., the IntraSubpartitonSplitType value is NO_ISP_SPLIT, i.e., the ISP does not apply to the current CU).

[282] When the current transform block is divided into transform blocks of the lower layer, the decoding equipment may extract the width newTbW of the transform block of the lower layer and the height newTbH of the transform block of the lower layer, as shown in the following equation (S2720).

[283] equation 10

[284] newTbW=(nTbW>maxTbWidth)?(nTbW/2):nTbW

[285] equation 11

[286] newTbH=(nTbH>maxTbHeight)?(nTbH/2):nTbH

[287] This is described with reference to Fig. 26. In an embodiment, the width nTbW of the current transform block may be the width of the CU of color signals, and the height nTbH of the current transform block may be the height of the CU of color signals. In this embodiment, the width newTbW of the lower layer transform block and the height newTbH of the lower layer transform block may be determined as the width and the height of the transform block 2621 divided from the CU of color signals. Thus, in this embodiment, the current transform block may be a transform block having the width and the height of the CU of color signals having a 4:2:2 format, and the lower layer transform blocks may be the first lower layer transform block 2621 to the fourth lower layer transform block 2624 obtained by dividing the current transform block into four in a non-square form.

[288] Then, the decoding equipment may perform intra prediction using the lower layer transformation blocks divided from the current transformation block (S2730). First, the decoding equipment may perform intra prediction for the first lower layer transformation block. Referring to FIG. 26, the first lower layer transformation block 2621 may be specified by a sample position (xTb0, yTb0), a width newTbW of the lower layer transformation block, and a height newTbH of the lower layer transformation block. The decoding equipment may perform intra prediction of the first lower layer transformation block 2621 using the predModeIntra intra prediction mode of the current CU and the color component cIdx of the CU. Therefore, a modified reconstructed frame of the first lower layer transformation block 2621 may be generated.

[289] For example, the decoding equipment may form a matrix predSamples of prediction samples having a size of (newTbW)x(newTbH) by performing an intra-sample prediction process. For example, the decoding equipment may perform an intra-sample prediction process using a sample position (xTb0, yTb0), an intra-prediction mode predModeIntra, a width (nTbW) newTbW of a transform block, a height (nTbH) newTbH of a transform block, a width (nCbW) nTbW of a coding block, and a height (nCbH) nTbH of a coding block, and a value of the parameter cIdx.

[290] In addition, the decoding equipment may form a matrix reSamples of residual samples having a size of (newTbW)x(newTbH) by performing a scaling and transforming process. For example, the decoding equipment may perform a scaling and transforming process based on a sample position (xTb0, yTb0), a value of the parameter cIdx, a width (nTbW) newTbW of a transform block, and a height (nTbH) newTbH of a transform block.

[291] In addition, the decoding equipment may generate a reconstructed frame by performing a frame reconstruction process for the color component. For example, the decoding equipment may set the position of the transformation block to (xTb0, yTb0), the width (nTbW) of the transformation block to newTbW, set the height (nTbH) of the transformation block to newTbH, use the value of the cIdx parameter, use the matrix predSamples of the prediction samples having a size of (newTbW)x(newTbH), and use the matrix reSamples of the residual samples having a size of (newTbW)x(newTbH), thereby performing a frame reconstruction process for the color component.

[292] Then, when nTbW exceeds maxTbWidth, the decoding equipment may perform intra-prediction for the second transformation block of the lower layer. The second transformation block 2622 of the lower layer may be indicated by a sample position (xTb0+newTbW, yTb0), a width newTbW of the transformation block of the lower layer, and a height newTbH of the transformation block of the lower layer. The decoding equipment may perform intra-prediction of the second transformation block 2622 of the lower layer using the predModeIntra intra-prediction mode of the current CU and the color component cIdx of the current CU. The intra-prediction of the second transformation block 2622 of the lower layer may be performed for its sample position similar to the intra-prediction of the first transformation block 2621 of the lower layer. Therefore, a modified reconstructed frame of the second transformation block 2622 of the lower layer may be generated.

[293] Then, when nTbH exceeds maxTbHeight, the decoding equipment may perform intra prediction for the third transformation block of the lower layer. The third transformation block 2623 of the lower layer may be indicated by a sample position (xTb0, yTb0+newTbH), a width newTbW of the transformation block of the lower layer, and a height newTbH of the transformation block of the lower layer. Similar to the above description, the decoding equipment may perform intra prediction using the predModeIntra intra prediction mode of the current CU and the color component cIdx of the current CU.

[294] Then, when nTbW exceeds maxTbWidth, and nTbH exceeds maxTbHeight, the decoding equipment may perform intra prediction for the fourth transformation block of the lower layer. The fourth transformation block 2624 of the lower layer may be indicated by a sample position (xTb0+newTbW, yTb0+newTbH), a width newTbW of the transformation block of the lower layer, and a height newTbH of the transformation block of the lower layer. Similar to the above description, the decoding equipment may perform intra prediction using the predModeIntra intra prediction mode of the current CU and the color component cIdx of the current CU.

[295] Meanwhile, the decoding equipment may perform intra prediction as follows when the current transform block is not split. For example, when nTbW is less than maxTbWidth, and nTbH is less than maxTbHeight, or ISP is applied to the current CU (for example, the IntraSubpartitonSplitType value is not NO_ISP_SPLIT), the current transform block may not be split.

[296] First, the decoding hardware can extract the parameters nW, nH, numPartsX, and numPartsY, as shown in the following equations.

[297] equation 12

[298] nW=IntraSubPartitionsSplitType==ISP_VER_SPLIT? nTbW/NumIntraSubPartitions:nTbW

[299] nH=IntraSubPartitionsSplitType==ISP_HOR_SPLIT? nTbH/NumIntraSubPartitions:nTbH

[300] numPartsX=IntraSubPartitionsSplitType==ISP_VER_SPLIT? NumIntraSubPartitions:1

[301] numPartsY=IntraSubPartitionsSplitType==ISP_HOR_SPLIT? NumIntraSubPartitions:1

[302] In the above equation, IntraSubPartitionsSplitType indicates the ISP partition type of the current CU, and ISP_VER_SPLIT indicates the vertical ISP partition, and ISP_HOR_SPLIT indicates the horizontal ISP partition. NumIntraSubPartitions indicates the number of ISP sub-segments.

[303] Then, the decoding equipment may form a matrix predSamples of prediction samples having a size of (nTbW)×(nTbH) by performing an intra-sample prediction process. For example, the decoding equipment may perform an intra-sample prediction process using a sample position of (xTb0+nW*xPartIdx, yTb0+nH*yPartIdx), an intra-prediction mode predModeIntra, a width (nTbW) nW of a transformation block, a height (nTbH) nH of a transformation block, a width (nCbW) nTbW of a coding block, and a height (nCbH) nTbH of a coding block, and a parameter value of cIdx. Here, the value of the segment index xPartIdx may have a value from 0 to numPartX-1, and yPartIdx may have a value from 0 to numPartsY-1.

[304] Then, the decoding equipment may form a matrix reSamples of residual samples having a size of (nTbW)x(nTbH) by performing a scaling and transforming process. For example, the decoding equipment may perform a scaling and transforming process based on a sample position (xTbY+nW*xPartIdx, yTbY+nH*yPartIdx), a value of the parameter cIdx, a width (nTbW) nW of a transform block, and a height (nTbH) nH of a transform block.

[305] Then, the decoding equipment may generate a reconstructed frame by performing a frame reconstruction process for the color component. For example, the decoding equipment may set the position of the transformation block to (xTb0+nW*xPartIdx, yTb0+nH*yPartIdx), set the width (nTbW) of the transformation block to nW, set the height (nTbH) of the transformation block to nH, use the value of the preset cIdx, and use a matrix predSamples of prediction samples having a size of (nTbW)x(nTbH) and a matrix reSamples of residual samples having a size of (nTbW)x(nTbH), thereby performing a frame reconstruction process for the color component.

[306] Limiting the Maximum Size of the Chroma Signal Conversion Block in Inter-Prediction Mode and IBC Prediction Mode

[307] Hereinafter, this document describes the execution of an inter-prediction mode and an IBC prediction mode to which the limitation of the maximum size of the chrominance conversion unit is applied for the above-described chrominance pipeline processing. The encoding equipment and the decoding equipment can perform inter-prediction and IBC prediction by limiting the maximum size of the chrominance conversion unit according to the description below, and their operations can correspond to each other. In addition, the description below of the inter-prediction can be applied to the IBC prediction mode with modification. Therefore, hereinafter, this document describes the inter-prediction operation of the decoding equipment according to the embodiment.

[308] The decoding equipment according to the embodiment may generate a luminance signal prediction block predSamplesL having a size of (cbWidth)x(cbHeight), and chrominance signal prediction blocks predSamplesCb and predSamplesCr having a size of (cbWidth/SubWidthC)x(cbHeight/SubHeightC), by performing inter prediction. Here, cbWidth may be the width of the current CU, and cbHeight may be the height of the current CU.

[309] In addition, the decoding equipment may form a residual block resSamplesL of luminance signals having a size of (cbWidth) x (cbHeight), and residual blocks resSamplesCr and resSamplesCb of chrominance signals having a size of (cbWidth/SubWidthC) x (cbHeight/SubHeightC). Finally, the decoding equipment may form a reconstructed block using the prediction blocks and the residual blocks.

[310] Hereinafter, this document describes a method for limiting the maximum size of a color signal transformation block to generate a residual block of a CU coded in an inter-prediction mode by means of a decoding equipment according to an embodiment. The decoding equipment can generate a reconstructed block using the residual block generated in this step.

[311] The decoding equipment according to the embodiment may directly obtain the next information from the bit stream or extract the next information from other information obtained from the bit stream to form a residual block having a size of (nTbW)x(nTbH) of the CU coded in the inter-prediction mode. Here, nTbW and nTbH may be set equal to the width cbWidth of the current CU and the height cbHeight of the current CU.

[312] - Sample position (xTb0, yTb0) indicating the position of the top-left sample of the current transform block relative to the position of the top-left sample of the current frame

[313] - nTbW parameter specifies the width of the current transformation block

[314] - nTbH parameter specifies the height of the current transformation block

[315] - cIdx parameter indicating the color component of the current CU

[316] The decoding equipment may extract the maximum width maxTbWidth of the transform block and the maximum height maxTbHeight of the transform block from the received information as follows.

[317] equation 13

[318] maxTbWidth=(cIdx==0)?MaxTbSizeY:MaxTbSizeY/SubWidthC

[319] equation 14

[320] maxTbHeight=(cIdx==0)?MaxTbSizeY:MaxTbSizeY/SubHeightC

[321] In addition, the decoding equipment may extract the upper-left sample position (xTbY, yTbY) of the current transform block based on whether the current CU represents a luminance signal component or a chrominance signal component, as follows.

[322] equation 15

[323] (xTbY, yTbY)=(cIdx==0)?(xTb0, yTb0):(xTb0*SubWidthC, yTb0*SubHeightC)

[324] As indicated in the above equation, when the current transform block is a chrominance signal block, in order to reflect the size of the chrominance signal block determined according to the chrominance signal format of the current transform block, the maximum width and height of the transform block and the upper-left sampling position of the current transform block may be determined based on the chrominance signal format.

[325] Hereinafter in this document, the decoding equipment may form a residual block by performing the following procedure. First, the decoding equipment may determine whether the current transformation block is split or not (S2810). For example, the decoding equipment may determine whether the current transformation block is split or not based on whether the width and the height of the current transformation block exceed the width and the height of the maximum transformation block. For example, when nTbW exceeds maxTbWidth, or nTbH exceeds maxTbHeight, the decoding equipment may determine that the transformation blocks of the lower layer are formed by splitting the current transformation block.

[326] When the current transform block is divided into transform blocks of the lower layer as indicated in the above example of intra prediction, the decoding equipment may extract the width newTbW of the transform block of the lower layer and the height newTbH of the transform block of the lower layer as shown in the following equation (S2820).

[327] equation 16

[328] newTbW=(nTbW>maxTbWidth)?(nTbW/2):nTbW

[329] equation 17

[330] newTbH=(nTbH>maxTbHeight)?(nTbH/2):nTbH

[331] Then, the decoding equipment may form a residual block using the lower layer transform blocks divided from the current transform block (S2830). In the embodiment, as shown in Fig. 26, the current transform block may be a transform block having a width and a height of CU of color signals having a 4:2:2 format, and the lower layer transform blocks may be the first lower layer transform block 2621 to the fourth lower layer transform block 2624 obtained by dividing the current transform block into four in a non-square form.

[332] First, the decoding equipment may generate a residual block for the first transformation block of the lower layer. Referring to Fig. 26, the first transformation block 2621 of the lower layer may be indicated by a sample position (xTb0, yTb0), a width newTbW of the transformation block of the lower layer, and a height newTbH of the transformation block of the lower layer. The decoding equipment may generate the residual block of the first transformation block 2621 of the lower layer using the color component cIdx of the current CU. Based on this, the decoding equipment may generate a modified reconstructed frame. After this, in-loop filtering may be performed on the modified reconstructed frame.

[333] Then, when nTbW exceeds maxTbWidth, the decoding equipment may generate a residual block for the second lower layer transformation block. The second lower layer transformation block 2622 may be indicated by a sample position (xTb0+newTbW, yTb0), a width newTbW of the lower layer transformation block, and a height newTbH of the lower layer transformation block. The decoding equipment may generate the residual block of the second lower layer transformation block 2622 using the color component cIdx of the current CU. Based on this, the decoding equipment may generate a modified reconstructed frame. After this, in-loop filtering may be performed on the modified reconstructed frame.

[334] Then, when nTbH exceeds maxTbHeight, the decoding equipment may generate a residual block for the third transformation block of the lower layer. The third transformation block 2623 of the lower layer may be indicated by a sample position (xTb0, yTb0+newTbH), a width newTbW of the transformation block of the lower layer, and a height newTbH of the transformation block of the lower layer. Similar to the above description, the decoding equipment may generate a residual block using the color component cIdx of the current CU.

[335] Then, when nTbW exceeds maxTbWidth, and nTbH exceeds maxTbHeight, the decoding equipment may generate a residual block for the fourth transformation block of the lower layer. The fourth transformation block 2624 of the lower layer may be indicated by a sample position (xTb0+newTbW, yTb0+newTbH), a width newTbW of the transformation block of the lower layer, and a height newTbH of the transformation block of the lower layer. Similar to the above description, the decoding equipment may generate a residual block using the color component cIdx of the current CU.

[336] Meanwhile, when the current transform block is not split, the decoding equipment may perform inter prediction as follows. For example, when nTbW is less than maxTbWidth and nTbH is less than maxTbHeight, the current transform block may not be split. In this case, the decoding equipment may form a residual block for the inter prediction mode by performing a scaling and transforming process using the sample position (xTbY, xTbY), the color component cIdx of the current CU, the width nTbW of the transform block, and the height nTbH of the transform block as input. Based on this, the decoding equipment may form a modified reconstructed frame. After that, in-loop filtering may be performed on the modified reconstructed frame.

[337] Coding method

[338] Hereinafter, this document describes a method for performing encoding by an encoding equipment according to an embodiment using the above-described method with reference to Fig. 29. The encoding equipment according to the embodiment may include a memory device and at least one processor, and the at least one processor may perform the following encoding method.

[339] First, the encoding equipment may determine the current block by splitting the image (S2910). Then, the encoding equipment may generate an intra prediction block with respect to the current block (S2920). Then, the encoding equipment may generate a residual block with respect to the current block based on the intra prediction block (S2930). Then, the encoding equipment may encode intra prediction mode information with respect to the current block (S2940).

[340] In this case, the intra prediction block and the residual block may be coded based on the transform block size of the current block, and the transform block size may be determined based on the color component of the current block.

[341] The color component may be any of a luminance component and a chrominance component, and when the color component of the current block is a chrominance component, the size of the transform block may be determined based on the color format.

[342] To specify the size of the transformation block, the width of the transformation block may be determined based on the maximum width of the transformation block, and the maximum width of the transformation block may be determined based on the maximum size and color format of the transformation block of the luminance signal block corresponding to the current block.

[343] In addition, to specify the size of the transformation block, the height of the transformation block may be determined based on the maximum height of the transformation block, and the maximum height of the transformation block may be determined based on the maximum size and color format of the transformation block of the luminance signal block corresponding to the current block.

[344] In addition, the position of the upper left sample of the transform block may be determined based on the position and color format of the upper left sample of the luminance signal block corresponding to the current block.

[345] More specifically, the upper left position of the transformation block may be determined based on the maximum width of the transformation block and the maximum height of the transformation block, the maximum width of the transformation block may be determined based on the maximum size and color format of the transformation block of the luminance signal block corresponding to the current block, and the maximum height of the transformation block may be determined based on the maximum size and color format of the transformation block of the luminance signal block corresponding to the current block.

[346] In addition, the current block is a color signal block, and when the width of the transformation block exceeds the maximum width of the transformation block, a plurality of transformation blocks of the lower layer may be formed by vertically splitting the current block. In this case, the intra prediction block and the residual block may be formed based on the plurality of transformation blocks of the lower layer.

[347] More specifically, the plurality of lower layer transformation blocks may include a first lower layer transformation block and a second lower layer transformation block, and the width of the first lower layer transformation block is determined as a maximum width of the transformation block, and the upper left coordinate of the second lower layer transformation block may be determined as a value shifted from the upper left coordinate of the first transformation block to the right by the maximum width of the transformation block.

[348] In addition, the current block is a color signal block, and when the height of the transformation block exceeds the maximum height of the transformation block, a plurality of transformation blocks of the lower layer may be formed by horizontally splitting the current block. In this case, the intra prediction block and the residual block may be formed based on the plurality of transformation blocks of the lower layer.

[349] More specifically, the plurality of lower layer transformation blocks may include a third lower layer transformation block and a fourth lower layer transformation block. The height of the third lower layer transformation block may be determined as a maximum height of the transformation block, and the upper left coordinate of the fourth lower layer transformation block may be determined as a value shifted downward from the upper left coordinate of the first transformation block by the maximum height of the transformation block.

[350] As an implementation of the above description, the maximum size of the transformation block may be 64×64 when the color format of the current block is a 4:4:4 format, the maximum size of the transformation block may be 32×64 when the color format of the current block is a 4:2:2 format, and the maximum size of the transformation block may be 32×32 when the color format of the current block is a 4:2:0 format.

[351] Decoding method

[352] Hereinafter, this document describes a method for performing decoding by a decoding equipment according to an embodiment using the above-described method with reference to Fig. 30. The decoding equipment according to the embodiment may include a memory device and at least one processor, and the at least one processor may perform the following decoding method.

[353] First, the decoding equipment may obtain the current block by splitting the image (S3010). Then, the decoding equipment may determine the prediction mode of the current block (S3020). Then, when the prediction mode of the current block is an intra prediction mode, the decoding equipment may generate an intra prediction block relative to the current block (S3030). Then, the decoding equipment may generate a residual block relative to the current block (S3040). Then, the decoding equipment may restore the current block based on the prediction block and the residual block relative to the current block (S3050).

[354] In this case, the intra prediction block and the residual block may be formed based on the transformation block size of the current block, and the transformation block size may be determined based on the color component of the current block.

[355] The color component may be any of a luminance component and a chrominance component, and when the color component of the current block is a chrominance component, the size of the transform block may be determined based on the color format.

[356] To specify the size of the transformation block, the width of the transformation block may be determined based on the maximum width of the transformation block, and the maximum width of the transformation block may be determined based on the maximum size and color format of the transformation block of the luminance signal block corresponding to the current block.

[357] In addition, to specify the size of the transformation block, the height of the transformation block may be determined based on the maximum height of the transformation block, and the maximum height of the transformation block may be determined based on the maximum size and color format of the transformation block of the luminance signal block corresponding to the current block.

[358] In addition, the position of the upper left sample of the transform block may be determined based on the position and color format of the upper left sample of the luminance signal block corresponding to the current block.

[359] More specifically, the upper left position of the transformation units of the lower layer may be determined based on the maximum width of the transformation unit and the maximum height of the transformation unit, the maximum width of the transformation unit may be determined based on the maximum size and color format of the transformation unit of the luminance signal unit corresponding to the current block, and the maximum height of the transformation unit may be determined based on the maximum size and color format of the transformation unit of the luminance signal unit corresponding to the current block.

[360] In addition, the current block is a color signal block, and when the width of the transformation block exceeds the maximum width of the transformation block, a plurality of transformation blocks of the lower layer may be formed by vertically splitting the current block. In this case, the intra prediction block and the residual block may be formed based on the plurality of transformation blocks of the lower layer.

[361] More specifically, the plurality of lower layer transformation blocks may include a first lower layer transformation block and a second lower layer transformation block, and the width of the first lower layer transformation block is determined as a maximum width of the transformation block, and the upper left coordinate of the second lower layer transformation block may be determined as a value shifted from the upper left coordinate of the first transformation block to the right by the maximum width of the transformation block.

[362] In addition, the current block is a color signal block, and when the height of the transformation block exceeds the maximum height of the transformation block, a plurality of transformation blocks of the lower layer may be formed by horizontally splitting the current block. In this case, the intra prediction block and the residual block may be formed based on the plurality of transformation blocks of the lower layer.

[363] More specifically, the plurality of lower layer transformation blocks may include a third lower layer transformation block and a fourth lower layer transformation block. The height of the third lower layer transformation block may be determined as a maximum height of the transformation block, and the upper left coordinate of the fourth lower layer transformation block may be determined as a value shifted downward from the upper left coordinate of the first transformation block by the maximum height of the transformation block.

[364] As an implementation of the above description, the maximum size of the transformation block may be 64×64 when the color format of the current block is a 4:4:4 format, the maximum size of the transformation block may be 32×64 when the color format of the current block is a 4:2:2 format, and the maximum size of the transformation block may be 32×32 when the color format of the current block is a 4:2:0 format.

[365] Application option

[366] Although the exemplary methods of the present disclosure described above are presented as a sequence of operations for clarity of description, this is not intended to limit the order in which the steps are performed, and the steps may be performed simultaneously or in another order as desired. To implement the method according to the present disclosure, the steps described may additionally include other steps, may include the remaining steps excluding some steps, or may include other additional steps excluding some steps.

[367] In the present disclosure, the image encoding equipment or the image decoding equipment that performs a predetermined operation (step) may perform an operation (step) of confirming a condition or situation of executing the corresponding operation (step). For example, if it is described that the predetermined operation is executed when the predetermined condition is satisfied, the image encoding equipment or the image decoding equipment may perform the predetermined operation after determining whether the predetermined condition is satisfied or not.

[368] The various embodiments of the present disclosure do not represent a list of all possible combinations and are intended to describe representative aspects of the present disclosure, and the issues described in the various embodiments may be applied independently or in combination of two or more of them.

[369] Various embodiments of the present disclosure may be implemented in hardware, firmware, software, or a combination thereof. When implemented in hardware, the present disclosure may be implemented using application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general processors, controllers, microcontrollers, microprocessors, etc.

[370] In addition, the image decoding equipment and the image encoding equipment to which the embodiments of the present disclosure are applied may be included in a multimedia broadcast transceiver, a mobile communication terminal, a video device of a home theater system, a video device of a digital theater system, a surveillance camera, a video chat device, a real-time communication device such as a video communication, a mobile streaming device, a storage medium, a recording video camera, a video on demand (VoD) service providing device, an OTT (video over the air) video device, an Internet streaming service providing device, a 3D video device, a video device of a video telephony system, a medical video device, and the like, and may be used to process video signals or data signals. For example, OTT video devices may include a gaming console, Blu-Ray player, Internet-enabled TV, home theater system, smartphone, tablet PC, digital video recorder (DVR), etc.

[371] Fig. 31 is a view showing a content streaming system to which an embodiment of the present disclosure is applicable.

[372] Referring to Fig. 31, a content streaming system to which the embodiment(s) of the present document are applied may mainly include an encoding server, a streaming server, a web server, a media storage, a user device, and a media input device.

[373] The encoding server compresses content input from media input devices such as a smartphone, camera, video recording camera, etc. into digital data to form a bit stream, and transmits the bit stream to the streaming server. As another example, when media input devices such as smartphones, cameras, video recording cameras, etc. directly form a bit stream, the encoding server may be omitted.

[374] The bit stream may be generated by an image coding method or image coding equipment to which an embodiment of the present disclosure is applied, and the streaming server may temporarily store the bit stream in the process of transmitting or receiving the bit stream.

[375] The streaming server transmits media data to the user device based on the user's request through the web server, and the web server serves as a medium for informing the user about the service. When the user requests the desired service from the web server, the web server can deliver it to the streaming server, and the streaming server can transmit the media data to the user. In this case, the content streaming system may include a separate control server. In this case, the control server serves to control the command/response between the devices in the content streaming system.

[376] The streaming server may receive content from a media storage and/or an encoding server. For example, when content is received from an encoding server, the content may be received in real time. In this case, in order to provide a smooth streaming service, the streaming server may store the bit stream for a predetermined time.

[377] Examples of a user device may include a mobile phone, a smart phone, a laptop computer, a digital broadcast terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, a slate tablet PC, tablet PCs, ultrabooks, wearable devices (e.g., smart watches, smart glasses, head-mounted displays), digital televisions, desktop computers, a digital signage system, and the like.

[378] Each server in a content streaming system may operate as a distributed server, in which case the data received from each server may be distributed.

[379] The scope of the disclosure includes software or computer-executable instructions (e.g., an operating system, an application, firmware, a program, etc.) for enabling the execution of operations according to the methods of various embodiments in equipment or a computer, a non-volatile computer-readable medium having such software or instructions stored and executable in the equipment or computer.

Industrial applicability

[380] Embodiments of the present disclosure may be used to encode or decode an image.

Claims (52)

1. A method for decoding images, carried out by means of image decoding equipment, wherein the method for decoding images comprises the steps of: determine the forecasting mode of the current block; form an internal prediction block relative to the current block, based on the prediction mode of the current block, which is the internal prediction mode; and reconstruct the current block based on the internal prediction block, in this case, the internal prediction block is formed based on the size of the transformation block of the current block, in this case, the maximum size of the transformation block is determined separately for each width and height of the transformation block, wherein, based on the color component of the current block, which is the color signal component, the maximum size of the conversion block is determined based on the color format, and wherein the size of the transformation block is determined based on the result of the comparison between the size of the transformation block and the maximum size of the transformation block. 2. The method for decoding images according to item 1, wherein the width of the transform block is determined based on the maximum width of the transform block, and wherein the maximum width of the transformation block is determined based on the maximum size and color format of the transformation block of the luminance signal block corresponding to the current block. 3. The method for decoding images according to item 1, wherein the height of the transformation block is determined based on the maximum height of the transformation block, and wherein the maximum height of the transformation block is determined based on the maximum size and color format of the transformation block of the luminance signal block corresponding to the current block. 4. The method for decoding images according to claim 1, wherein the position of the upper left sample of the transform block is determined based on the position and color format of the upper left sample of the luminance signal block corresponding to the current block. 5. The method for decoding images according to item 1, wherein the upper left position of the transformation block is determined based on the maximum width of the transformation block and the maximum height of the transformation block, wherein the maximum width of the transformation block is determined based on the maximum size and color format of the transformation block of the luminance signal block corresponding to the current block, and wherein the maximum height of the transformation block is determined based on the maximum size and color format of the transformation block of the luminance signal block corresponding to the current block. 6. The method for decoding images according to item 1, in which the current block is a block of color signals, wherein, based on the width of the transformation block exceeding the maximum width of the transformation block, a plurality of transformation blocks of the lower layer are formed by vertically splitting the current block, and wherein the internal prediction block and the residual block are formed on the basis of a plurality of transformation blocks of the lower layer. 7. The method for decoding images according to item 6, wherein the plurality of lower layer transformation units include a first lower layer transformation unit and a second lower layer transformation unit, wherein the width of the first transformation block of the lower layer is determined as half the width of the transformation block, and wherein the upper left coordinate of the second transformation block of the lower layer is defined as a value shifted from the upper left coordinate of the transformation block to the right by half the width of the transformation block. 8. The method for decoding images according to item 1, in which the current block is a block of color signals, wherein, based on the height of the transformation block exceeding the maximum height of the transformation block, a plurality of transformation blocks of the lower layer are formed by horizontally splitting the current block, and wherein the internal prediction block and the residual block are formed on the basis of a plurality of transformation blocks of the lower layer. 9. The method for decoding images according to item 8, wherein the plurality of lower layer transformation units include a third lower layer transformation unit and a fourth lower layer transformation unit, wherein the height of the third transformation block of the lower layer is determined as half the height of the transformation block, and wherein the upper left coordinate of the fourth transformation block of the lower layer is defined as a value shifted downwards from the upper left coordinate of the first transformation block by half the height of the transformation block. 10. The method for decoding images according to claim 1, wherein the maximum size of the transformation block is 64×64 based on the color format of the current block, which is a 4:4:4 format, the maximum size of the transformation block is 32×64 based on the color format of the current block, which is a 4:2:2 format, and the maximum size of the transformation block is 32×32 based on the color format of the current block, which is a 4:2:0 format. 11. A method for encoding images, carried out by means of image encoding equipment, wherein the method for encoding images comprises the steps of: determine the forecasting mode of the current block; form an internal prediction block relative to the current block, based on the prediction mode of the current block, which is the internal prediction mode; and encode the current block based on the intra-prediction block, in this case, the internal prediction block is formed based on the size of the transformation block of the current block, in this case, the maximum size of the transformation block is determined separately for each width and height of the transformation block, wherein, based on the color component of the current block, which is the color signal component, the maximum size of the conversion block is determined based on the color format, and wherein the size of the transformation block is determined based on the result of the comparison between the size of the transformation block and the maximum size of the transformation block. 12. A method for transmitting a bit stream formed by means of an image coding method, wherein the image coding method comprises the steps of: determine the forecasting mode of the current block; form an internal prediction block relative to the current block, based on the prediction mode of the current block, which is the internal prediction mode; and encode the current block into a bit stream based on the intra prediction block, in this case, the internal prediction block is formed based on the size of the transformation block of the current block, in this case, the maximum size of the transformation block is determined separately for each width and height of the transformation block, wherein, based on the color component of the current block, which is the color signal component, the maximum size of the conversion block is determined based on the color format, and wherein the size of the transformation block is determined based on the result of the comparison between the size of the transformation block and the maximum size of the transformation block.