RU2785731C1 - Method and equipment for encoding/decoding images and method for bitstream transmission - Google Patents

Abstract

FIELD: image data processing.

SUBSTANCE: invention relates to a method and equipment for encoding/decoding images. The result is achieved by the method for decoding images possibly including the stage of determining whether or not the prediction mode of the current block is an internal prediction mode, based on information related to the prediction mode of the current block; determining whether or not the mode is available based on intra-subpartitions (ISP) for the current block, when the prediction mode of the current block constitutes an internal prediction mode; decoding the ISP application indicator indicating whether or not the ISP should be applied to the current block when the ISP is available for the current block; and generating a prediction block for the current block by applying the ISP to the current block, if it was determined that the ISP should be applied to the current block, based on the ISP application indicator.

EFFECT: increase in the efficiency of encoding/decoding images.

13 cl, 23 dwg, 5 tbl

Description

The technical field to which the invention belongs

[1] The present disclosure relates to a method and equipment for encoding/decoding images and a method for transmitting a bit stream, and more specifically, to a method and equipment for encoding/decoding an image by applying an intra subblock segment (ISP) based mode to images having various chrominance signal formats, and to a method for transmitting a bit stream generated by the image encoding method/equipment of the present disclosure.

State of the art

[2] Recently, the demand for high-quality high definition images, such as high definition (HD) images and ultra high definition (UHD) images, has been growing in various fields of technology. As the resolution and quality of image data increases, the amount of information or bits to be transmitted increases relatively compared to existing image data. An increase in the amount of information or bits to be transmitted results in an increase in transmission costs and storage costs.

[3] Accordingly, there is a need for a high-performance image compression technology for efficiently transmitting, storing, and reproducing information regarding high-quality, high-resolution images.

The essence of the invention

Technical task

[4] An object of the present disclosure is to provide an image encoding/decoding method and equipment with improved encoding/decoding efficiency.

[5] An object of the present disclosure is to provide a method and equipment for encoding/decoding an image by applying ISP to images having various chrominance signal formats.

[6] Another object of the present disclosure is to provide a method for transmitting a bit stream generated by an image encoding method or equipment according to the present disclosure.

[7] Another object of the present disclosure is to provide a recording medium storing a bit stream generated by an image encoding method or equipment according to the present disclosure.

[8] Another object of the present disclosure is to provide a recording medium storing a bit stream received, decoded, and used in order to reconstruct an image by image decoding equipment according to the present disclosure.

[9] The technical problems solved by the present disclosure are not limited to the above technical problems, and other technical problems that are not described herein should become apparent to those skilled in the art from the description below.

Technical solution

[10] The image decoding method performed by the image decoding equipment according to an aspect of the present disclosure may include determining whether or not the prediction mode of the current block is an intra prediction mode, based on information regarding the prediction mode of the current block, determining whether or not the prediction mode of the current block is available. or no intra-sub-segment (ISP) based mode for the current block when the current block prediction mode is an intra prediction mode, decoding an ISP application indicator indicating whether or not ISP should be applied to the current block when ISP is available for the current block, and generating a prediction block for the current block by applying the ISP to the current block, while determining that the ISP is applied to the current block, based on the ISP application indicator.

[11] In the image decoding method of the present disclosure, determining whether or not an ISP is available for a current block may be performed based on a comparison between the block size of the luma components of the current block and a predetermined threshold value.

[12] In the image decoding method of the present disclosure, the predetermined threshold may be a maximum transform size.

[13] In the image decoding method of the present disclosure, when the width and height of the luma component block of the current block are equal to or less than a predetermined threshold value, it can be determined that the ISP is available for the current block.

[14] In the image decoding method of the present disclosure, generating a prediction block for the current block by applying an ISP to the current block may include generating a prediction block for the luma component block by applying the ISP to the luminance signal component block of the current block, determining whether to whether or not to apply ISP to the chrominance signal component block of the current block, and generating a prediction block for the chrominance signal component block by applying ISP to the chrominance signal component block when determining that ISP is applied to the chrominance signal component block of the current block.

[15] In the image decoding method of the present disclosure, determining whether or not ISP should be applied to the chrominance component block of the current block may be performed based on at least one of the chrominance component block size or the chrominance signal format of the current block.

[16] In the image decoding method of the present disclosure, when the chrominance signal format of the current block is 4:2:2 or 4:4:4, and at least one of the width or height of the chrominance signal component block exceeds the maximum transform size , it may be determined that the ISP is applied to the chroma component block of the current block.

[17] In the image decoding method of the present disclosure, when ISP is applied to a chroma component block, a split direction and a number of sub-segments for the chroma component block may be determined based on a split direction and a number of sub-segments for the luma component block.

[18] In the image decoding method of the present disclosure, the split direction and the number of sub-segments for the chrominance component block may be equal to the split direction and the number of sub-segments for the luma component block, respectively.

[19] In the image decoding method of the present disclosure, generating a prediction block for the current block by applying an ISP to the current block may include generating a prediction block for the luma component block by applying ISP to the luminance signal component block of the current block, and generating a prediction block for the chroma component block by applying the ISP to the chrominance signal component block of the current block.

[20] In the image decoding method of the present disclosure, the split direction and the number of sub-segments for the chroma component block may be determined based on the split direction and the number of sub-segments for the luma component block.

[21] In the image decoding method of the present disclosure, the split direction and the number of sub-segments for the chrominance component block may be equal to the split direction and the (number of sub-segments/2) of the luma component block, respectively.

[22] Image decoding equipment according to another aspect of the present disclosure may include a memory and at least one processor. The at least one processor can determine whether or not the prediction mode of the current block is an intra prediction mode, based on information regarding the prediction mode of the current block, determine whether or not an intra-sub-segment (ISP) based mode is available for the current block, when the current block prediction mode is an intra prediction mode, decode an ISP application indicator indicating whether or not ISP should be applied to the current block when ISP is available for the current block, and generate a prediction block for the current block by applying ISP to the current block when determining that the ISP is applied to the current block based on the ISP applied indicator.

[23] A method for encoding pictures according to another aspect of the present disclosure may include determining whether or not the prediction mode of the current block is an intra prediction mode, determining whether or not an intra-sub-segment (ISP) based mode is available for the current block, when the current block prediction mode is an intra prediction mode, determining whether or not ISP should be applied to the current block when ISP is available for the current block, generating a prediction block for the current block by applying ISP to the current block when ISP is applied to the current block, and encoding information regarding the prediction mode of the current block and information regarding whether or not to apply ISP to the current block.

[24] The transmission method according to another aspect of the present disclosure may transmit a bitstream generated by the image encoding equipment or the image encoding method of the present disclosure.

[25] A computer-readable recording medium according to another aspect of the present disclosure may store a bit stream generated by the image encoding equipment or the image encoding method of the present disclosure.

[26] A computer-readable recording medium according to another aspect of the present disclosure may store a bitstream received and decoded by image decoding equipment and used to reconstruct an image. The bitstream may include information on the prediction mode of the current block and an ISP application indicator indicating whether or not ISP should be applied to the current block, information on the prediction mode of the current block may be used to determine whether or not the prediction mode is of the current block is the intra prediction mode, the ISP application indicator may be used to determine whether or not ISP should be applied to the current block when the current block's prediction mode is the intra prediction mode, and it is determined that the ISP is available for the current block , and the prediction block of the current block may be generated by performing the ISP with respect to the current block while determining that the ISP is applied to the current block.

[27] The features briefly summarized above with respect to the present disclosure are merely exemplary aspects of the following detailed description of the present disclosure and do not limit the scope of the present disclosure.

Benefits of the Invention

[28] According to the present disclosure, it is possible to provide an image encoding/decoding method and equipment with improved encoding/decoding efficiency.

[29] In addition, according to the present disclosure, it is possible to provide a method and equipment for encoding/decoding an image by applying ISP to images having various chrominance signal formats.

[30] In addition, according to the present disclosure, it is possible to provide a method for transmitting a bit stream generated by the image encoding method or equipment according to the present disclosure.

[31] In addition, according to the present disclosure, it is possible to provide a recording medium storing a bit stream generated by the image encoding method or equipment according to the present disclosure.

[32] In addition, according to the present disclosure, it is possible to provide a recording medium storing a bitstream received, decoded, and used to reconstruct an image by the image decoding equipment according to the present disclosure.

[33] Those skilled in the art should appreciate that the benefits that can be achieved through the present disclosure are not limited to those specifically described above, and other benefits of the present disclosure should be more clearly understood from the detailed description.

Brief description of the drawings

[34] FIG. 1 is a view schematically showing a video coding system to which an embodiment of the present disclosure is applicable.

[35] FIG. 2 is a view schematically showing image encoding equipment to which an embodiment of the present disclosure is applicable.

[36] FIG. 3 is a view schematically showing image decoding equipment to which an embodiment of the present disclosure is applicable.

[37] FIG. 4 is a flowchart illustrating a video/image coding method based on intra prediction.

[38] FIG. 5 is a view illustrating the configuration of the intra prediction unit 185 according to the present disclosure.

[39] FIG. 6 is a flowchart illustrating a video/image decoding method based on intra prediction.

[40] FIG. 7 is a view illustrating the configuration of the intra prediction unit 265 according to the present disclosure.

[41] FIG. 8a and 8b are views illustrating the direction of intra prediction according to an embodiment of the present disclosure.

[42] FIG. 9 is a view illustrating a conventional method for performing encoding/decoding of a current block according to the ISP mode.

[43] FIG. 10 is a view illustrating an example of partitioning the ISP for current blocks having different sizes.

[44] FIG. 11 is a view illustrating the relationship between the luminance signal component block (luminance signal component array) and the chrominance signal component block (chrominance signal component array) according to the chrominance signal format.

[45] FIG. 12 is a view illustrating the size of the chrominance signal component block according to the chrominance signal format when the luminance signal component block is a 64×128 block.

[46] FIG. 13 is a flowchart illustrating an ISP application method according to an embodiment of the present disclosure.

[47] FIG. 14 is a view illustrating an example of splitting a block of chrominance signal components according to the ISP application method of FIG. 13.

[48] FIG. 15 is a flowchart illustrating an ISP application method according to another embodiment of the present disclosure.

[49] FIG. 16 is a view illustrating an example of splitting a block of chrominance signal components according to the ISP application method of FIG. fifteen.

[50] FIG. 17 is a view illustrating an example of a bitstream structure in which a modified ISP availability condition is reflected.

[51] FIG. 18 is a view illustrating another example of a bitstream structure in which a modified ISP availability condition is reflected.

[52] FIG. 19 is a view showing a content streaming system to which an embodiment of the present disclosure is applicable.

Optimal Mode for Carrying Out the Invention

[53] Hereinafter, embodiments of the present disclosure are described in detail with reference to the accompanying drawings so that they can be easily implemented by those skilled in the art. However, the present disclosure may be implemented in any number of forms and is not limited to the embodiments described herein.

[54] In describing the present disclosure, if it is determined that a detailed description of an associated known function or construct would unnecessarily ambiguize the scope of the present disclosure, the detailed description thereof is omitted. In the drawings, parts not related to the description of the present disclosure are omitted and like reference numbers are attached to like parts.

[55] In the present disclosure, when a component is "connected", "coupled", or "linked" to another component, this may include not only a direct connection-based relationship, but also an indirect connection-based relationship in which an intermediate component is present. In addition, when a component "includes" or "has" other components, it means that other components may be included in addition to, rather than excluding, other components, unless otherwise noted.

[56] In the present disclosure, the terms "first", "second", etc. may be used only for the purpose of distinguishing one component from other components and do not limit the order or importance of components unless otherwise noted. Accordingly, within the scope of the present disclosure, the first component in one embodiment may be referred to as the "second component" in another embodiment, and similarly, the second component in one embodiment may be referred to as the "first component" in another embodiment.

[57] In the present disclosure, components that are different from each other are intended to clearly describe each feature and do not mean that the components are necessarily separated. Thus, multiple components may be integrated and implemented in a single hardware or software module, or a single component may be distributed and implemented in multiple hardware or software modules. Therefore, even if not stated otherwise, such embodiments in which components are integrated or a component is distributed are also included within the scope of this disclosure.

[58] In the present disclosure, the components described in various embodiments do not necessarily mean essential components, and some components may be optional components. Accordingly, an embodiment consisting of a subset of the components described in the embodiment is also included within the scope of the present disclosure. In addition, embodiments including other components, in addition to the components described in the various embodiments, are included within the scope of the present disclosure.

[59] The present disclosure relates to image encoding and decoding, and the terms used in the present disclosure may have a general meaning widely used in the technical field to which the present disclosure belongs, unless first defined in the present disclosure.

[60] In the present disclosure, "frame" generally means a unit representing one image in a particular time period, and a slice/tile is a coding unit constituting a frame, and one frame may be composed of one or more slices/ tiles. In addition, a slice/tile may include one or more coding tree units (CTUs).

[61] In the present disclosure, "pixel" or "pel" can mean the smallest unit constituting one frame (or image). In addition, "sample" can be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, and may represent only a pixel/pixel value of a luma component, or only a pixel/pixel value of a chroma component.

[62] In the present disclosure, a "unit" may represent a basic unit of image processing. The unit may include at least one of a specific area of the frame and information associated with the area. The unit can be used interchangeably with terms such as "array of samples", "block" or "zone" in some cases. In general, an MxN block may include samples (or arrays of samples) or a set (or array) of transform coefficients of M columns and N rows.

[63] In the present disclosure, "current block" may mean one of "current coding block", "current coding unit", "target coding block", "target decoding block", or "target processing block". When prediction is performed, "current block" may mean "current prediction block" or "target prediction block". When transformation (inverse transformation)/quantization (dequantization) is performed, "current block" may mean "current transformation block" or "transformation target block". When filtering is performed, "current block" may mean "filtering target block".

[64] In the present disclosure, the terms "/" and "" are to be interpreted as indicating "and/or". For example, the expression "A/B" and "A, B" can mean "A and/or B". Additionally, "A/B/C" and "A/B/C" can mean "at least one of A, B and/or C".

[65] In the present disclosure, the term "or" is to be interpreted as indicating "and/or". For example, the expression "A or B" may contain 1) only "A", 2) only "B", and/or 3) "both A and B". In other words, in the present disclosure, the term "or" is to be interpreted as indicating "in addition or alternatively".

[66] Overview of the Video Coding System

[67] FIG. 1 is a view showing a video coding system according to the present disclosure.

[68] The video encoding system according to the embodiment may include encoding equipment 10 and decoding equipment 20. The encoding equipment 10 may deliver the encoded information or video and/or image data to the decoding equipment 20 in the form of a file or streaming via a digital storage medium or a network.

[69] The encoding equipment 10 according to the embodiment may include a video source generator 11, an encoding unit 12, and a transmitter 13. The decoding equipment 20 according to the embodiment may include a receiver 21, a decoding unit 22, and a rendering unit 23. The encoding module 12 may be referred to as a "video/image coding module", and the decoding module 22 may be referred to as a "video/image decoding module". The transmitter 13 may be included in the encoding module 12. The receiver 21 may be included in the module 22 decoding. The rendering module 23 may include a display, and the display may be configured as a separate device or an external component.

[70] The video source generator 11 can acquire video/image through a process of capturing, synthesizing, or generating video/image. The video source generator 11 may include a video/image capture device and/or a video/image generator. The video/image capture device may include, for example, one or more cameras, video/image archives including previously captured videos/images, and the like. The video/image generation apparatus may include, for example, computers, tablet computers, and smartphones, and may (electronically) generate video/images. For example, the virtual video/image may be generated via a computer or the like. In this case, the video/image capturing process may be replaced by a linked data generation process.

[71] The encoding unit 12 may encode the input video/image. Encoding unit 12 may perform a series of procedures such as prediction, transformation, and quantization for compression and encoding efficiency. The encoding unit 12 may output encoded data (encoded video/image information) in the form of a bit stream.

[72] The transmitter 13 may transmit the encoded information or video/image data output in the form of a bit stream to the receiver 21 of the decoding equipment 20 via a digital storage medium or a network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, and the like. The transmitter 13 may include an element for generating a media file via a predetermined file format and may include an element for transmission via a broadcast/communications network. The receiver 21 can extract/receive a bit stream from a storage medium or a network and transmit the bit stream to the decoding module 22 .

[73] The decoding module 22 may decode the video/image by performing a sequence of procedures such as dequantization, inverse transformation, and prediction corresponding to the operation of the encoding module 12.

[74] The rendering unit 23 may prepare a decoded video/image by rendering. The rendered video/image can be displayed through the display.

[75] Overview of Image Coding Equipment

[76] FIG. 2 is a view schematically showing image encoding equipment to which an embodiment of the present disclosure is applicable.

[77] As shown in FIG. 2, the image encoding equipment 100 may include an image segmentation unit 110, a subtractor 115, a transform 120, a quantizer 130, a dequantizer 140, an inverse transform 150, an adder 155, a filter 160, a memory 170, an inter prediction unit 180, an intra prediction unit 185. and entropy encoder 190. Inter prediction module 180 and intra prediction module 185 may be collectively referred to as a "prediction module". Transformer 120, quantizer 130, dequantizer 140, and inverse transform 150 may be included in the residual processor. The residual processor may further include a subtractor 115.

[78] All or at least a portion of the plurality of components configuring the image encoding hardware 100 may be configured by a single hardware component (eg, encoder or processor) in some embodiments. In addition, the storage device 170 may include a decoded frame buffer (DPB) and may be configured by a digital storage medium.

[79] The image segmentation unit 110 may segment an input image (either a frame or a movie frame) input to the image encoding equipment 100 into more than one processing unit. For example, a processing unit may be referred to as a "coding unit (CU)". A coding unit may be obtained by recursively segmenting a coding tree unit (CTU) or a largest coding unit (LCU) according to a quadtree, binary tree, and ternary tree (QT/BT/TT) structure. For example, one coding unit may be segmented into multiple deeper coding units based on a quadtree structure, a binary tree structure, and/or a ternary structure. To segment a coding unit, a quadtree structure may be applied first, and subsequently a binary tree structure and/or a ternary structure may be applied. The encoding procedure according to the present disclosure may be performed based on a final encoding unit that is no longer segmented. The largest coding unit may be used as the final coding unit, or the larger depth coding unit obtained by segmenting the largest coding unit may be used as the final coding unit. Here, the encoding procedure may include a prediction, transformation, and recovery procedure, which is described below. As another example, the encoding procedure processing unit may be a prediction unit (PU) or a transformation unit (TU). The prediction unit and the transformation unit may be split or segmented from the final coding unit. The prediction unit may be a selective prediction unit, and the transform unit may be a unit for extracting a transform coefficient and/or a unit for extracting a residual signal from a transform coefficient.

[80] The prediction unit (inter prediction unit 180 or intra prediction unit 185) can perform prediction on a block to be processed (current block) and generate a predicted block including prediction samples for the current block. The prediction module may determine whether intra prediction or inter prediction is applied based on the current block or CU. The prediction module may generate various information related to the prediction of the current block and pass the generated information to the entropy encoder 190. The prediction information may be encoded in the entropy encoder 190 and output in the form of a bit stream.

[81] Intra prediction module 185 may predict the current block by referring to samples in the current frame. The samples for a reference may be located in the surroundings of the current block, or may be spaced apart according to an intra prediction mode and/or an intra prediction technique. The intra prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The non-directional mode may include, for example, a DC mode and a planar mode. The directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes according to the granularity of the prediction direction. However, this is just an example, more or less directional prediction modes may be used depending on the setting. The intra prediction unit 185 may determine the prediction mode applied to the current block by using the prediction mode applied to the adjacent block.

[82] Inter prediction unit 180 may derive a prediction block for the current block based on a reference block (reference sample array) indicated by a motion vector for the reference frame. In this case, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between an adjacent block and a current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction direction information (L0 prediction, L1 prediction, or bi-prediction, etc.). In the case of inter prediction, an adjacent block may include a spatial adjacent block present in the current frame and a temporal adjacent block present in the reference frame. The reference frame including the reference block and the reference frame including the temporal neighbor block may be the same or different. The temporary neighbor block may be referred to as a "collocated reference block", "collocated CU (colCU)", or the like. A reference frame including a temporal adjacent block may be referred to as a "collocated frame (colPic)". For example, inter prediction module 180 may configure a list of motion information candidates based on neighboring blocks and generate information indicating which candidate is used to derive the motion vector and/or reference frame index of the current block. Mutual prediction may be performed based on various prediction modes. For example, in the case of the skip mode and the merge mode, the inter prediction unit 180 may use the neighboring block motion information as the current block motion information. In the case of the skip mode, unlike the combine mode, the residual signal may not be transmitted. In the case of a motion vector prediction (MVP) mode, the motion vector of an adjacent block may be used as a motion vector predictor, and the motion vector of the current block may be signaled by encoding the motion vector difference and an indicator for the motion vector predictor. The motion vector difference may mean the difference between the motion vector of the current block and the motion vector predictor.

[83] The prediction module may generate a prediction signal based on various prediction methods and prediction technologies described below. For example, the prediction unit may not only apply intra prediction or inter prediction, but also apply both intra prediction and inter prediction at the same time to predict the current block. A prediction method that simultaneously applies both intra prediction and inter prediction to predict the current block may be referred to as "Combined Inter-Intra Prediction (CIIP)". In addition, the prediction module may perform intra-block copy (IBC) to predict the current block. In-block copying can be used for image/video encoding of game content and the like, such as on-screen content coding (SCC). IBC is a current frame prediction method using a previously reconstructed reference block in the current frame at a location separated from the current block by a predetermined distance. When IBC is applied, the location of the reference block in the current frame may be encoded as a vector (block vector) corresponding to a predetermined distance.

[84] The prediction signal generated by the prediction module may be used to generate a recovered signal or generate a residual signal. The subtractor 115 may generate a residual signal (residual block or residual sample array) by subtracting the prediction signal (prediction block or prediction sample array) output from the prediction module from the input image signal (source block or source sample array). The generated residual signal may be transmitted to transducer 120.

[85] Converter 120 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technology may include at least one of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen-Loeve Transform (KLT), Graph Based Transform (GBT), or Conditionally Nonlinear Transform (CNT). ). Here, GBT means a transformation obtained from a graph when relationship information between pixels is represented by a graph. CNT means a transformation obtained based on a prediction signal generated using all previously reconstructed pixels. In addition, the transformation process may be applied to square pixel blocks having the same size, or may be applied to blocks having a variable size rather than square ones.

[86] The quantizer 130 may quantize the transform coefficients and pass them to the entropy encoder 190. The entropy encoder 190 may encode the quantized signal (information regarding the quantized transform coefficients) and output a bit stream. The information regarding the quantized transform coefficients may be referred to as "residual information". The quantizer 130 may repackage the quantized transform coefficients in block form into one-dimensional vector form based on the scan order of the coefficients, and generate information about the quantized transform coefficients based on the quantized transform coefficients in one-dimensional vector form.

[87] The entropy encoder 190 may implement various coding techniques such as exponential Golomb coding, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), and the like, for example. Entropy encoder 190 may encode information needed for video/image reconstruction other than quantized transform coefficients (eg, syntax element values, etc.) together or separately. Encoded information (eg, encoded video/image information) may be transmitted or stored in units of network abstraction layers (NALs) in the form of a bitstream. The video/image information may further include information regarding various parameter sets such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. The signaling information, the signaling information and/or the syntax elements described in the present disclosure may be encoded via the encoding procedure described above and included in the bitstream.

[88] The bit stream may be transmitted over a network or may be stored on a digital storage medium. The network may include a broadcast network and/or a communications network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, and the like. A transmitter (not shown) transmitting a signal output from the entropy encoder 190 and/or a storage module (not shown) storing the signal may be included as an internal/external member of the encoding equipment 100, and alternatively, a transmitter may be included in the entropy encoder 190 .

[89] The quantized transform coefficients output from quantizer 130 may be used to generate a residual signal. For example, the residual signal (residual block or residual samples) may be recovered by applying dequantization and inverse transform to the quantized transform coefficients via dequantizer 140 and inverse transform 150.

[90] The adder 155 adds the recovered residual signal with the prediction signal output from the inter prediction unit 180 or the intra prediction unit 185 so as to generate a recovered signal (recovered frame, recovered block, recovered sample array). If there is no remainder for a block to be processed, such as in a case in which the skip mode is applied, the prediction block may be used as the reconstructed block. The adder 155 may be referred to as a "recovery module" or "recovered block generator". The generated recovered signal may be used for intra prediction of the next block to be processed in the current frame, and may be used for inter prediction of the next frame through filtering as described below.

[91] Meanwhile, as described below, chrominance scaling (MCS) luminance signal transformation is applicable in the frame encoding process.

[92] The filter 160 can improve subjective/objective image quality by applying filtering to the reconstructed signal. For example, filter 160 may generate a modified reconstructed frame by applying various filtering techniques to the reconstructed frame, and store the modified reconstructed frame in memory 170, namely the DPB of memory 170. Various filtering techniques may include, for example, filtering to deblock , sampled adaptive bias, adaptive loop filter, bilateral filter, etc. The filter 160 may generate various filtering-related information and pass the generated information to the entropy encoder 190 as described below in the description of each filtering method. Filtering related information may be encoded by entropy encoder 190 and output in the form of a bitstream.

[93] The modified reconstructed frame transmitted to the memory 170 may be used as a reference frame in the inter prediction module 180. When inter prediction is applied via the image encoding equipment 100, prediction mismatch between the image encoding equipment 100 and the image decoding equipment can be eliminated, and encoding efficiency can be improved.

[94] The DPB of the memory 170 may store the modified reconstructed frame for use as a reference frame in the inter prediction module 180. The memory 170 may store block motion information from which motion information in the current frame is extracted (or encoded) and/or motion information of blocks in the frame that have already been recovered. The stored motion information may be transmitted to the inter prediction unit 180 and used as spatial neighbor block motion information or temporal neighbor block motion information. The memory 170 may store the recovered samples of the recovered blocks in the current frame and may transmit the recovered samples to the intra prediction module 185 .

[95] Overview of image decoding equipment

[96] FIG. 3 is a view schematically showing image decoding equipment to which an embodiment of the present disclosure is applicable.

[97] As shown in FIG. 3, the image decoding equipment 200 may include an entropy decoder 210, a dequantizer 220, an inverse transform 230, an adder 235, a filter 240, a memory 250, an inter prediction unit 260, and an intra prediction unit 265. Inter prediction module 260 and intra prediction module 265 may be collectively referred to as a "prediction module". Dequantizer 220 and inverse converter 230 may be included in the residual processor.

[98] All or at least a portion of the plurality of components configuring the image decoding equipment 200 may be configured by a hardware component (eg, decoder or processor) according to the embodiment. In addition, the storage device 250 may include a decoded frame buffer (DPB) or may be configured by a digital storage medium.

[99] The image decoding equipment 200, which receives a bit stream including video/image information, can restore an image by performing a process corresponding to the process performed by the image encoding equipment 100 of FIG. 2. For example, the image decoding equipment 200 may perform decoding using the processing unit used in the image encoding equipment. Thus, the decoding processing unit, for example, may be an encoding unit. The coding unit may be obtained by segmenting the coding tree unit or the largest coding unit. The reconstructed image signal decoded and output by the decoding equipment 200 can be reproduced by the playback equipment.

[100] The image decoding equipment 200 may receive a signal output from the image encoding equipment of FIG. 2 in the form of a bitstream. The received signal may be decoded via entropy decoder 210. For example, entropy decoder 210 may parse the bit stream to extract information (eg, video/image information) needed for image recovery (or frame recovery). The video/image information may further include information regarding various parameter sets such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. The image decoding equipment may further decode the frame based on the parameter set information and/or general constraint information. Signaling/receiving information and/or syntax elements described in the present disclosure may be decoded by a decoding procedure and obtained from a bitstream. For example, entropy decoder 210 decodes information in the bitstream based on an encoding method such as Exponential Golomb, CAVLC, or CABAC encoding, and output syntax element values required for image reconstruction, and quantized transform coefficient values for the remainder. More specifically, the CABAC entropy decoding method can receive a bin corresponding to each syntax element in the bit stream, determine a context model using decoding target syntax element information, neighboring block decoding information, and decoding target block, or symbol/bin information decoded at the previous step, and perform arithmetic decoding for the bin by predicting the occurrence probability of the bin according to the determined context model, and generating a symbol corresponding to the value of each syntax element. In this case, the CABAC entropy decoding method can update the context model by using the decoded symbol/bin information for the context model of the next symbol/bin after the context model is determined. Prediction related information from the information decoded by the entropy decoder 210 may be provided to a prediction unit (inter prediction unit 260 and intra prediction unit 265), and a residual value for which entropy decoding is performed in the entropy decoder 210, i. quantized transform coefficients and associated parameter information may be input to the dequantizer 220. In addition, information regarding filtering from the information decoded by the entropy decoder 210 may be provided to the filter 240. Meanwhile, a receiver (not shown) for receiving a signal output from the equipment image encoding equipment may be further configured as an internal/external element of the image decoding equipment 200, or the receiver may be a component of the entropy decoder 210.

[101] Meanwhile, the image decoding equipment according to the present disclosure may be referred to as "video/image/frame decoding equipment". The image decoding equipment can be classified into an information decoder (video/image/frame information decoder) and a sampling decoder (video/image/frame samples decoder). The information decoder may include an entropy decoder 210. The selective decoder may include at least one of a dequantizer 220, an inverse transform 230, an adder 235, a filter 240, a memory 250, an inter prediction module 160, or an intra prediction module 265.

[102] The dequantizer 220 may dequantize the quantized transform coefficients and output the transform coefficients. The dequantizer 220 may recompose the quantized transform coefficients in the form of a two-dimensional block. In this case, the re-arranging may be performed based on the order of coefficient scanning performed in the image encoding equipment. The dequantizer 220 may perform dequantization on the quantized transform coefficients by using a quantization parameter (eg, quantization step size information) and obtain the transform coefficients.

[103] The inverse transform 230 may inversely transform the transform coefficients to obtain a residual signal (residual block, array of residual samples).

[104] Prediction module 230 may perform prediction for the current block and may generate a prediction block including prediction samples for the current block. The prediction unit may determine whether intra prediction or inter prediction is applied to the current block based on the prediction information output from the entropy decoder 210, and may determine a specific intra/inter prediction mode (prediction technique).

[105] Identical to that described in the prediction module of the image encoding equipment 100, the prediction module can generate a prediction signal based on various prediction methods (technologies) which are described below.

[106] Intra prediction module 265 may predict the current block by referring to samples in the current frame. The description of the intra prediction module 185 applies equally to the intra prediction module 265 .

[107] Inter prediction unit 260 may derive a prediction block for the current block based on a reference block (array of reference samples) indicated by a motion vector for the reference frame. In this case, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between an adjacent block and a current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction direction information (L0 prediction, L1 prediction, or bi-prediction, etc.). In the case of inter prediction, an adjacent block may include a spatial adjacent block present in the current frame and a temporal adjacent block present in the reference frame. For example, inter prediction module 260 may configure a motion information candidate list based on neighboring blocks and derive the current block's motion vector and/or reference frame index based on the received candidate selection information. Inter prediction may be performed based on various prediction modes, and the prediction information may include information indicative of the inter prediction mode for the current block.

[108] The adder 235 may generate a reconstructed signal (recovered frame, reconstructed block, reconstructed sample array) by summing the obtained residual signal with a prediction signal (prediction block, prediction sample array) output from the prediction module (including the inter prediction module 260 and /or the intra prediction module 265). The description of the adder 155 is equally applicable to the adder 235.

[109] Meanwhile, as described below, chrominance scaling (MCS) luminance signal transformation is applicable in the frame decoding process.

[110] The filter 240 can improve subjective/objective image quality by applying filtering to the reconstructed signal. For example, filter 240 may generate a modified reconstructed frame by applying various filtering methods to the reconstructed frame and store the modified reconstructed frame in memory 250, namely the DPB of memory 250. Various filtering methods may include, for example, filtering to deblock , sampled adaptive bias, adaptive loop filter, bilateral filter, etc.

[111] The (modified) reconstructed frame stored in the DPB of the memory 250 may be used as a reference frame in the inter prediction module 260. The memory 250 may store block motion information from which motion information in the current frame is extracted (or decoded) and/or motion information of blocks in the frame that have already been recovered. The stored motion information may be transmitted to the inter prediction unit 260 so that it is used as the spatial neighbor block motion information or the temporal neighbor block motion information. The memory 250 may store the recovered samples of the recovered blocks in the current frame and pass the recovered samples to the intra prediction module 265 .

[112] In the present disclosure, the embodiments described in the filter 160, the inter prediction module 180, and the intra prediction module 185 of the image encoding equipment 100 can be equally or appropriately applied to the filter 240, the inter prediction module 260, and the intra prediction module 265 image decoding equipment 200 .

[113] General representation of intra prediction

[114] Hereinafter, this document describes intra prediction according to an embodiment.

[115] Intra prediction may indicate a prediction that generates prediction samples for the current block based on reference samples in the frame to which the current block belongs (hereinafter referred to as the "current frame"). When intra prediction is applied to the current block, adjacent reference samples to be used for intra prediction of the current block may be extracted. Adjacent reference samples of the current block may include the sample adjacent to the left edge of the current block having size nWxnH and the sum of the 2xnH samples adjacent to the bottom left, the samples adjacent to the top edge of the current block, and the sum of the 2xnW samples adjacent to the right top, and one selection adjacent to the left top with respect to the current block. Alternatively, the adjacent reference samples of the current block may include a plurality of columns of top neighbor samples and a plurality of rows of left neighbor samples. In addition, adjacent reference samples of the current block may include the sum of nH samples adjacent to the right edge of the current block having size nWxnH, the sum of nW samples adjacent to the bottom edge of the current block, and one sample adjacent to the bottom right of the current block.

[116] Some adjacent reference samples of the current block have not yet been decoded or may not be available. In this case, the decoder may construct adjacent reference samples to be used for prediction by substituting the available samples for the unavailable samples. Alternatively, adjacent reference samples to be used for prediction may be constructed using an interpolation of the available samples.

[117] When adjacent reference samples are extracted, (i) a prediction sample may be derived based on an average or an interpolation of adjacent reference samples of the current block, and (ii) a prediction sample may be derived based on a reference sample present in a particular direction (prediction) relative to the prediction sample from neighboring reference samples of the current block. Case (i) may be referred to as "non-directional mode" or "non-angular mode", and case (ii) may be referred to as "directional mode" or "angular mode".

[118] In addition, a prediction sample may be generated through interpolation with a first neighbor sample located in the prediction direction of the intra prediction mode of the current block and a second neighbor sample located in the opposite direction based on the prediction target sample of the current block from neighboring reference samples. The above case may be referred to as "linear interpolation intra prediction (LIP)".

[119] In addition, chroma prediction samples may be generated based on luma samples using a linear model. This case may be referred to as "linear model (LM) mode".

[120] In addition, the prediction temporal sample of the current block may be derived based on the filtered adjacent reference samples, and the prediction sample of the current block may be derived by weighted addition of the prediction temporal sample and at least one reference sample derived according to the intra prediction mode from existing adjacent reference samples, i.e. unfiltered neighboring reference samples. This case may be referred to as "position dependent intra prediction (PDPC)".

[121] In addition, the reference sample line with the highest prediction accuracy may be selected from multiple adjacent reference sample lines of the current block to extract a prediction sample using a reference sample located in the prediction direction in the corresponding line, and, at that time, information (for example, , intra_luma_ref_idx) relative to the reference sample line used, may be encoded and signaled in the bitstream. This case may be referred to as "multiple reference line (MRL) intra prediction" or "MRL intra prediction".

[122] In addition, the current block may be partitioned into vertical or horizontal subsegments to perform intra prediction on each subsegment based on the same intra prediction mode. At this time, adjacent intra prediction reference samples may be extracted in units of sub-segments. Thus, the reconstructed sample of the previous sub-segment in the encoding/decoding order can be used as the adjacent reference sample of the current sub-segment. In this case, the intra prediction mode for the current block is identically applied to sub-segments, and adjacent reference samples are extracted and used in units of sub-segments, thereby improving intra prediction performance. Such a prediction method may be referred to as "intra-sub-segment prediction (ISP) or ISP-based".

[123] The intra prediction technology may be referred to by various terms such as "type of intra prediction" or "additional intra prediction mode", so as to be distinguished from the directional or non-directional intra prediction mode. For example, an intra prediction technology (type of intra prediction or additional intra prediction mode) may include at least one of LIP, LM, PDPC, MRL, ISP, or MIP. Meanwhile, if necessary, post-filtering may be performed on the extracted prediction sample.

[124] Specifically, the intra prediction procedure may include an intra prediction mode/type determination step, an adjacent reference sample extraction step, and a prediction sample extraction step based on the intra prediction mode/type. In addition, if necessary, post-filtering may be performed on the extracted prediction sample.

[125] FIG. 4 is a flowchart illustrating a video/image coding method based on intra prediction.

[126] The encoding method of FIG. 4 may be performed by the image encoding equipment of FIG. 2. In particular, step S410 may be performed by the intra prediction unit 185, and step S420 may be performed by the residual processor. Specifically, step S420 may be performed by subtractor 115. Step S430 may be performed by entropy encoder 190. Prediction information of step S430 may be extracted by intra prediction unit 185, and residual information of step S430 may be extracted by residual processor. The residual information is information about the residual samples. The residual information may include information regarding the quantized transform coefficient for the residual samples. As described above, the residual samples may be extracted as a transform coefficient via the image encoding equipment transform 120, and the transform coefficient may be extracted as transform coefficients quantized via the quantizer 130. Information regarding the quantized transform coefficients may be encoded by the entropy encoder 190 via a residual encoding procedure.

[127] The image encoding equipment may perform intra prediction with respect to the current block (S410). The image coding equipment may determine the intra prediction mode/type for the current block, extract adjacent reference samples of the current block, and generate prediction samples in the current block based on the intra prediction mode/type and adjacent reference samples. Here, the procedures for determining the mode/type of intra prediction, extracting adjacent reference samples, and generating prediction samples may be simultaneously performed, or any procedure may be performed before other procedures.

[128] FIG. 5 is a view illustrating the configuration of the intra prediction unit 185 according to the present disclosure.

[129] As shown in FIG. 5, the intra prediction module 185 of the image coding equipment may include an intra prediction mode/type determination module 186, a reference sample extractor 187, and/or a prediction sample extractor 188. The intra prediction mode/type determination unit 186 may determine the intra prediction mode/type for the current block. Reference sample extractor 187 may extract adjacent reference samples of the current block. Prediction sample extractor 188 may extract prediction samples of the current block. Meanwhile, although not shown, when the prediction sample filtering procedure described below is performed, the intra prediction unit 185 may further include a prediction sample filter (not shown).

[130] The image encoding equipment may determine the mode/type applied to the current block from a plurality of intra prediction modes/types. The image coding equipment can compare the rate-versus-rate (RD) distortion cost function for intra prediction modes/types and determine the optimal intra prediction mode/type for the current block.

[131] Meanwhile, the image encoding equipment may perform a prediction sample filtering procedure. The filtering of prediction samples may be referred to as "post-filtering". Through the prediction sample filtering procedure, some or all of the prediction samples may be filtered. In some cases, the prediction sample filtering procedure may be omitted.

[132] Referring to FIG. 4 again, the image coding equipment may generate residual samples for the current block based on prediction samples or filtered prediction samples (S420). The image coding equipment may extract residual samples by subtracting prediction samples from the original samples of the current block. Thus, the image encoding equipment can extract the residual sample values by subtracting the corresponding prediction sample value from the original sample value.

[133] The image encoding equipment may encode image information including information regarding intra prediction (prediction information) and residual information of residual samples. The prediction information may include intra prediction mode information and/or intra prediction technology information. The image encoding equipment 100 may output encoded image information in the form of a bit stream. The output bit stream may be transmitted to the image decoding equipment via a storage medium or a network.

[134] The residual information may include a residual encoding syntax, which is described below. The image encoding equipment may transform/quantize the residual samples and extract the quantized transform coefficients. The residual information may include information regarding the quantized transform coefficients.

[135] Meanwhile, as described above, the image encoding equipment may generate a reconstructed frame (including reconstructed samples and a reconstructed block). To this end, the image coding equipment may perform dequantization/inverse transform with respect to the quantized transform coefficients and extract (modified) residual samples. The reason for converting/quantizing the residual samples and then performing the dequantization/inverse transform is to extract residual samples identical to the residual samples extracted by the image decoding equipment. The image encoding equipment may generate a reconstructed block including reconstructed samples for the current block based on the prediction samples and the (modified) residual samples. Based on the reconstructed block, a reconstructed frame for the current frame may be generated. As described above, the in-loop filtering procedure may additionally be applied to the reconstructed frame.

[136] FIG. 6 is a flowchart illustrating a video/image decoding method based on intra prediction.

[137] The image decoding equipment can perform an operation corresponding to the operation performed by the image encoding equipment.

[138] The decoding method of FIG. 6 may be performed by the image decoding equipment of FIG. 3. Steps S610-S630 may be performed by the intra prediction unit 265, and the prediction information of step S610 and the residual information of step S640 may be obtained from the bitstream by the entropy decoder 210. The residual processor of the image decoding equipment may derive residual samples for the current block based on the residual information (S640). In particular, the residual processor dequantizer 220 may perform dequantization based on the dequantized transform coefficients derived from the residual information to extract the transform coefficients, and the residual processor inverse transform 230 may perform an inverse transform on the transform coefficients to extract the residual samples for the current block. Step S650 may be performed by an adder 235 or a recovery module.

[139] In particular, the image decoding equipment can derive the intra prediction mode/type for the current block based on the received prediction information (intra prediction mode/type information) (S610). The image decoding equipment may extract adjacent reference samples of the current block (S620). The image decoding equipment may generate prediction samples in the current block based on the intra prediction mode/type and neighboring reference samples (S630). In this case, the image decoding equipment may perform a prediction sample filtering procedure. The filtering of prediction samples may be referred to as "post-filtering". Through the prediction sample filtering procedure, some or all of the prediction samples may be filtered. In some cases, the prediction sample filtering procedure may be omitted.

[140] The image decoding equipment may generate residual samples for the current block based on the received residual information (S640). The image decoding equipment may generate reconstructed samples for the current block based on the prediction samples and the residual samples, and extract the reconstructed block including the reconstructed samples (S650). Based on the reconstructed block, a reconstructed frame for the current frame may be generated. The in-loop filtering procedure may additionally be applied to the reconstructed frame as described above.

[141] FIG. 7 is a view illustrating the configuration of the intra prediction unit 265 according to the present disclosure.

[142] As shown in FIG. 7, the intra prediction unit 265 of the image decoding equipment may include an intra prediction mode/type determination unit 266, a reference sample extractor 267, and a prediction sample extractor 268. The intra prediction mode/type determination unit 266 may determine the intra prediction mode/type for the current block based on the intra prediction mode/type information generated and signaled by the intra prediction mode/type determination unit 186 of the image coding equipment, and the extraction unit 267 reference samples may extract adjacent reference samples of the current block from the reconstructed reference area in the current frame. Prediction sample extractor 268 may extract prediction samples of the current block. Meanwhile, although not shown, when the above-described prediction sample filtering procedure is executed, the intra prediction unit 265 may further include a prediction sample filter (not shown).

[143] The intra prediction mode information may include, for example, flag information (eg, intra_luma_mpm_flag) indicating whether or not the most likely mode (MPM) or remaining mode should be applied to the current block, and when the MPM is applied to the current block, the intra prediction mode information may further include index information (eg, intra_luma_mpm_idx) indicating one of the candidate intra prediction modes (MPM candidates). The intra prediction mode candidates (MPM candidates) may consist of a list of MPM candidates or an MPM list. In addition, when MPM is not applied to the current block, the intra prediction mode information may further include remaining mode information (e.g., intra_luma_mpm_remainder) indicating one of the remaining intra prediction modes except for possible intra prediction modes (MPM candidates) . The image decoding equipment may determine the intra prediction mode of the current block based on the intra prediction mode information. The MPM mode options may include intra block neighboring (eg, left neighbor and top neighbor) modes relative to the current block, and additional mode options.

[144] FIG. 8a shows the direction of intra prediction according to an embodiment of the present disclosure. To capture any edge direction represented in natural video, as shown in FIG. 8a, the intra prediction mode may include two non-directional intra prediction modes and 65 directional intra prediction modes. The non-directional intra prediction modes may include a planar intra prediction mode (planar mode) and a DC intra prediction mode (DC mode), and the directional intra prediction modes may include the second to 66th intra prediction modes.

[145] Meanwhile, the intra prediction mode may further include a cross-component linear model (CCLM) based mode for chroma samples in addition to the intra prediction modes described above. The CCLM mode may be partitioned into L_CCLM, T_CCLM, LT_CCLM according to whether or not left samples, top samples, or both are considered to extract the LM parameter, and can only be applied to the chrominance signal component.

[146] For example, the intra prediction mode, for example, may be indexed as shown in the table below.

[147] Tab. one

Intra prediction mode Associated name 0 INTRA_PLANAR one INTRA_DC 2...66 INTRA_ANGULAR2...INTRA_ANGULAR66 81...83 INTRA_LT_CCLM, INTRA_L_CCLM, INTRA_T_CCLM

[148] FIG. 8b shows the direction of intra prediction according to another embodiment of the present disclosure. In FIG. 8b, the dotted line direction shows the wide-angle mode applied to the non-square block only. As shown in FIG. 8b, in order to capture any edge direction represented in natural video, an intra prediction mode according to an embodiment may include two non-directional intra prediction modes and 93 directional intra prediction modes. Non-directional intra prediction modes may include planar mode and DC mode, and directional intra prediction modes may include second-80th and -1st--14th intra prediction modes, as indicated by the arrow in FIG. 8b. The planar mode may be referred to as INTRA_PLANAR and the DC mode may be referred to as INTRA_DC. In addition, the directional intra prediction mode may be denoted by INTRA_ANGULAR-14 through INTRA_ANGULAR-1 and INTRA_ANGULAR2 through INTRA_ANGULAR80. In addition, the intra prediction technology information may be implemented in various forms. For example, the intra prediction technology information may include intra prediction type index information indicating one of the intra prediction technologies. As another example, the intra prediction technology information may include at least one of reference sample line information (e.g., intra_luma_ref_idx) indicating whether or not the MRL should be applied to the current block, and if so, which line reference samples are used, ISP flag information (eg, intra_subpartitions_mode_flag) indicating whether or not ISP should be applied to the current block, ISP type information (eg, intra_subpartitions_split_flag)) indicating the type of sub-segment splitting when applying ISP, flag information indicating whether whether PDPC should be applied or not, or flag information indicating whether or not LIP should be applied. In the present disclosure, the ISP flag information may be referred to as "ISP application indicator".

[149] The intra prediction mode information and/or the intra prediction technology information may be encoded/decoded via the coding method described in the present disclosure. For example, intra prediction mode information and/or intra prediction technology information may be encoded/decoded by truncated binary code (Rice) entropy coding (eg, CABAC, CAVLC).

[150] Before determining whether or not to apply a predetermined intra prediction technology to the current block, it may first be determined whether or not the corresponding intra prediction technology is available for the current block. For example, whether or not a corresponding intra prediction technology is available for the current block may be determined based on the coding parameters of the current block. In this case, the encoding parameters may include the size (width and/or height) of the current block, the position of the current block, the color component of the current block, or whether or not another intra prediction technique should be applied.

[151] In addition, a determination as to whether or not a corresponding intra prediction technology is available for the current block may be made based on information signaled in the upper layer of the current block, such as sequence, frame, slice, and CTU. For example, when information transmitted at the sequence level indicates that a predetermined intra prediction technology is not available, it may be determined that the corresponding intra prediction technology is not available for blocks belonging to the corresponding sequence.

[152] When determining that a predetermined intra prediction technology is available for the current block, the image coding equipment may determine whether or not to apply the corresponding intra prediction technology to the current block using various methods. For example, the image encoding equipment may determine whether or not to apply the corresponding intra prediction technology based on RDO.

[153] The image encoding equipment may encode and signal intra prediction technology information in a bit stream. The intra prediction technology information may be encoded according to the corresponding intra prediction technology in various forms. For example, when the intra prediction technology is MRL, the intra prediction technology information may be an index (eg, intra_luma_ref_idx) indicating a reference line used to predict the current block from multiple reference lines. When the intra prediction technology is ISP, the intra prediction technology information may be flag information (eg, intra_subpartitions_mode_flag) indicating whether or not ISP should be applied to the current block. In addition, when using ISP, the intra prediction technology information may further include information (eg, intra_subpartitions_split_flag) regarding the split direction. In addition, the intra prediction technology information may include flag information indicating whether or not to apply PDPC, flag information indicating whether or not to apply LIP, or flag information indicating whether or not to apply LM mode.

[154] In case it is determined that a predetermined intra prediction technology is available for the current block, the image decoding equipment may determine whether or not to apply the corresponding intra prediction technology to the current block based on signaling the intra prediction technology information.

[155] As another example, whether or not a predetermined intra prediction technique should be applied to the current block may be implicitly derived by the image encoding equipment and the image decoding equipment based on the encoding parameters for the current block, in addition to being explicitly signaled information technology of internal prediction. In this case, the encoding parameter may include the size (width and/or height) of the current block, the position of the current block, the color component of the current block, or whether or not another intra prediction technique should be applied.

[156] Hereinafter, this document describes in detail the ISP mode according to the present disclosure.

[157] The current block may include a block (array) of luma components and a block (array) of chroma components corresponding thereto. In the present disclosure, "current block" or "luminance signal component block" may mean "luminance signal component block of the current block", and "chrominance signal component block" or "corresponding chrominance signal component block" may mean "current chrominance signal component block". block". In addition, "luma component block" may be referred to as "luminance signal block", "luminance component block" or "luminance block", and "chroma signal component block" may be referred to as "chroma signal block", " chrominance component block" or "chroma block".

[158] In conventional intra prediction, the current encoding/decoding target block (current block) is considered as one unit, and therefore encoding/decoding is performed without splitting. However, when applying the ISP mode, the current block is split in the horizontal or vertical direction to perform intra predictive encoding/decoding. In this case, encoding/decoding is performed in units of split sub-segments to form a reconstructed sub-segment, and the reconstructed sub-segment is used as a reference block of the next split sub-segment.

[159] Whether or not ISP is available for the current block may be determined based on the following conditions of Table 2. The following conditions may be determined based on the luma component block of the current block. Thus, under the following conditions, the width, height, and position of the current block may mean the width, height, and position of the luma component block of the current block, respectively.

[160] Tab. 2

ISP Availability Conditions
intra_luma_ref_idx[x0][y0]==0
cbWidth<=MaxTbSizeY || cbHeight<=MaxTbSizeY
cbWidth*cbHeight>MinTbSizeY*MinTbSizeY

[161] For example, when all of the above conditions for the current block are satisfied, it may be determined that the ISP is available for the current block. After determining that the ISP is available for the current block, it may be determined whether or not to apply the ISP to the current block. Under ISP availability, (x0, y0) are coordinates indicating the location of the top left sample of the current block. In addition, intra_luma_ref_idx[x0][y0] is information indicating a reference line used to predict the current block. According to ISP availability conditions, when intra_luma_ref_idx is 0, i.e. when the reference line used to predict the current block is the line is line 0 (the reference line immediately adjacent to the current block), it can be determined that the ISP is available for the current block. When the reference line used to predict the current block is a line other than line 0, it may be determined that the ISP is not available for the current block.

[162] In terms of ISP availability, cbWidth and cbHeight correspond to the width and height of the current block, respectively. In addition, MaxTbSizeY and MinTbSizeY can specify the maximum transform size and the minimum transform size, respectively. As described above, the residual processing may include a transform or an inverse transform. In this case, the size of the transform block in which the transform or inverse transform is available may be predetermined or signaled via a bit stream. Thus, the maximum transform size means the maximum size of a transform block at which a transform or inverse transform can be performed. In addition, the minimum transform size means the minimum size of a transform block at which a transform or inverse transform can be performed. For example, when the size of the current block exceeds the maximum transform size, the current block may be split into two or more transform blocks. In addition, the current block may not be split into transform blocks having a size smaller than the minimum transform size. The maximum transform size and/or the minimum transform size may be predetermined in the image coding equipment and the image decoding equipment, or may be derived based on information signaled at the upper block level.

[163] According to the conditions of ISP availability, only when at least one of cbWidth or cbHeight is equal to or less than MaxTbSizeY, it can be determined that the ISP is available for the current block. Thus, when cbWidth and cbHeight exceed MaxTbSizeY, it may be determined that the ISP is not available for the current block. When cbWidth is greater than MaxTbSizeY, cbHeight is equal to or less than MaxTbSizeY, and ISP is applied to the current block, the ISP split direction may be determined as the vertical direction, as described below. When cbHeight is greater than MaxTbSizeY, cbWidth is equal to or less than MaxTbSizeY, and ISP is applied to the current block, the ISP split direction may be determined as a horizontal direction, as described below.

[164] According to the ISP availability conditions, when cbWidth*cbHeight exceeds MinTbSizeY*MinTbSizeY, it may be determined that the ISP is available for the current block. cbWidth*cbHeight can mean the area of the current block or the number of samples included in the current block. For example, if MinTbSizeY is 4, it may be determined that the ISP is available for the current block only when the number of samples included in the current block exceeds 16 (4*4).

[165] FIG. 9 is a view illustrating a conventional method for performing encoding/decoding of a current block according to the ISP mode.

[166] When the current block to be encoded/decoded is input (S910), it may be determined whether or not to apply ISP to the current block (S920). Determining step S920 may include determining whether or not an ISP is available for the current block and/or determining whether or not the ISP should be applied to the current block. The determination as to whether or not an ISP is available for the current block may be made based on the above-described ISP availability conditions. When the ISP is available, the image encoding equipment may determine whether or not the ISP should be applied to the current block based on various methods as described above, and the result of the determination may be encoded in the bitstream as intra prediction technology information. When the ISP is available, the image decoding equipment can determine whether or not the ISP should be applied to the current block based on the intra prediction technology signaling information.

[167] When the ISP is not applied to the current block, TU tiling may be performed relative to the current block (S930). TU tiling means a process of dividing a current block into a plurality of transform blocks such that the width and height of the current block are equal to or smaller than the maximum transform size, which is a transform size. As described above, the residual signal processing includes a transformation, and the maximum transformation size means the maximum transformation block size at which the transformation process can be performed. Accordingly, when the width or height of the current block exceeds the maximum transformation size by splitting the current block through TU tiling, the width and height of the partitioning block may be equal to or less than the maximum transformation size. For example, when the maximum transform size is 64 samples and the current block has a size of 128×128, the current block may be divided into four 64×64 blocks. Alternatively, when the maximum transform size is 64 samples and the current block has a size of 64x128 or 128x64, the current block may be split into two 64x64 blocks. For example, the maximum transform size is 64 samples, and the current block has a size of 64x64 or less, TU tiling may not be performed.

[168] Thereafter, encoding/decoding may be performed on the current block or each of the blocks tiled through the TU tiling step (S960). The encoding of step S960 may include intra prediction, residual processing, and/or encoding of prediction information and residual information. The encoding of step S960 may include intra prediction, extraction of residual samples, and/or generation of reconstructed blocks.

[169] In step S920, when applying the ISP to the current block, the split direction and the number of splits (number of sub-segments) may be determined (S940).

[170] In step S920, the split direction may be derived based on information (eg, intra_subpartitions_split_flag) signaled via the bitstream. Alternatively, the split direction may be implicitly derived based on the size of the current block. For example, as described above, when the width of the current block exceeds the maximum transform size, the split direction of the ISP may be extracted in the vertical direction. In addition, when the height of the current block exceeds the maximum transform size, the split direction of the ISP can be extracted in the horizontal direction. In addition, when the width and height of the current block exceed the maximum transform size, a constraint may be imposed such that the ISP is not available for the current block. For example, when the maximum transform size is a size of 64 samples, and the current block to which the ISP is applied is a 128×64 block, the split direction of the ISP is extracted in the vertical direction, and the width (128/4) and height (64) of the sub-segment defined as the maximum transform size (64) or less. Similarly, when the maximum transform size is a size of 64 samples, and the current block to which the ISP is applied is a 64×128 block, the split direction of the ISP is extracted in the horizontal direction, and the width (64) and height (128/4) of the sub-segment defined as the maximum transform size (64) or less. As described above, when ISP is applied to the current block, the current block is split such that the width and height of the sub-segments are equal to or less than the maximum transform size. For this reason, the TU tiling of step S930 need not necessarily be performed relative to the current block to which the ISP is applied.

[171] In step S940, the number of splits may be implicitly derived based on the size of the current block. In particular, the current block may be split according to the size of the current block, as shown in Table 3.

[172] Tab. 3

Block size Number of splits 4×4 No split 4×8, 8×4 2 All other cases 4

[173] FIG. 10a is a view illustrating an example of ISP partitioning for a 4×8 block or an 8×4 block. As shown in FIG. 10a, a 4x8 block or 8x4 block may be split into two sub-segments. When the current block is a 4×8 block and is split in the horizontal direction, the current block may be split into two 4×4 sub-segments. When the current block is a 4×8 block and is split in the vertical direction, the current block may be split into two 2×8 sub-segments. When the current block is an 8x4 block and is split in the horizontal direction, the current block may be split into two 8x2 sub-segments. When the current block is an 8x4 block and is split in the vertical direction, the current block may be split into two 4x4 sub-segments.

[174] FIG. 10b is a view illustrating an example of splitting an ISP for a block having a size of 8×8 or larger.

[175] As shown in FIG. 10b, a block having a size of 8×8 or more may be divided into four sub-segments. When the current block is a WxH block, W and H are 8 or more, and the current block is partitioned in the horizontal direction, the current block may be partitioned into four Wx(H/4) sub-segments. When the current block is a WxH block, W and H are 8 or more, and the current block is partitioned in the vertical direction, the current block may be partitioned into four (W/4)xH sub-segments.

[176] Referring to FIG. 9 again, when the split direction and the number of splits are determined in step S940, the block of luminance signal components of the current block may be split based on the indicated (S950).

[177] Thereafter, encoding/decoding may be performed on each broken sub-segment (S960). As described above, the encoding of step S960 may include intra prediction, residual processing, and/or encoding of prediction information and residual information. In addition, the decoding of step S960 may include intra prediction, extraction of residual samples, and/or generation of reconstructed blocks. In particular, when applying ISP, the intra prediction mode for the current block is identically applied to sub-segments, and the adjacent reference sample is extracted and used in units of sub-segments, thereby improving intra prediction performance. Thus, when using ISP, the residual sample processing procedure is performed in units of sub-segments. In other words, the intra prediction samples are extracted for each sub-segment, and the residual signal(s) for the corresponding sub-segment is summed with it, thereby obtaining reconstructed samples. The residual signal (residual samples) may be extracted through a dequantization/inverse transform procedure based on residual information (quantized transform coefficient information or residual coding syntax) in the above-described bitstream. Thus, extraction of prediction samples and extraction of residual samples for the first sub-segment can be performed, and reconstructed samples for the first sub-segment can be extracted based on that. In this case, when extracting prediction samples for the second sub-segment, some reconstructed samples in the first sub-segment (eg, reconstructed samples adjacent to the left or top side of the second sub-segment) may be used as adjacent reference samples for the second sub-segment. Similarly, extraction of prediction samples and extraction of residual samples for the second sub-segment can be extracted, and reconstructed samples for the second sub-segment can be extracted based on that. In this case, when the prediction samples for the third sub-segment are extracted, some of the reconstructed samples in the second sub-segment (eg, the reconstructed samples adjacent to the left or top side of the third sub-segment) may be used as neighboring reference samples for the third sub-segment. Likewise, some recovered samples in the third sub-segment may be used as adjacent reference samples for the fourth sub-segment.

[178] The encoding/decoding order of the plurality of sub-segments is from top to bottom when the splitting direction is the horizontal direction, and is from left to right when the splitting direction is the vertical direction. For example, in FIG. 10b, when the splitting direction is the horizontal direction, the sub-segments may be sequentially encoded/decoded from the uppermost sub-segment to the lowermost sub-segment. In addition, when the split direction is the vertical direction, the sub-segments may be sequentially encoded/decoded from the leftmost sub-segment to the right-most sub-segment.

[179] When applying ISP to the current block in order to reduce the coding complexity, an MPM list is generated according to each partitioning method (horizontal partitioning and vertical partitioning), and a suitable prediction mode from the prediction modes in the generated MPM list is compared in terms of distortion optimization in baud rate (RDO) to generate the optimal mode. In addition, when multiple reference line (MRL) intra prediction is used, a restriction may be imposed such that the above-described ISP is not used. Thus, when a null reference line is used (eg, intra_luma_ref_idx==0), ISP is applicable. In addition, when using the ISP, the limitation may be imposed in such a way that the above-described PDPC is not used. Thus, when using ISP, PDPC may not be used.

[180] When using ISP as the intra prediction technology information, information (intra_subpartitions_mode_flag) indicating whether or not ISP should be applied may be transmitted in units of current blocks, and if the current block uses ISP (for example, when intra_subpartitions_mode_flag is 1), information (intra_subpartitions_split_flag) regarding the partitioning method (horizontal splitting or vertical splitting) may be transmitted.

[181] According to the conventional method described with reference to FIG. 9, in a single tree structure in which the luma component block of the current block and the chrominance signal component block of the current block are partitioned into an identical tree structure, when ISP is applied to the current block, the luma component block is partitioned into a plurality of sub-segments, but ISP is not applied. to the chrominance signal component block. In this case, the width or height of the chrominance signal component block becomes larger than the maximum transformation size according to the chrominance signal format, and therefore the transformation or inverse transformation of the chrominance signal component block may become impossible.

[182] Hereinafter, this document describes the relationship between the luma component block size and the chrominance signal component block size according to the chrominance signal format.

[183] FIG. 11 is a view illustrating the relationship between the luminance signal component block (luminance signal component array) and the chrominance signal component block (chrominance signal component array) according to the chrominance signal format.

[184] The original or encoded frame/image may include a luma component (Y) block and two chrominance component (Cb, Cr) blocks. Thus, one frame/image pixel may include a luminance sample and two chrominance samples (Cb, Cr). The color format may represent a configuration format of luma samples and chrominance samples (Cb, Cr), and may be referred to as a "chroma format". The chrominance signal format may be predetermined or may be adaptively signaled. For example, the chroma format may be signaled based on at least one of chroma_format_idc or separate_colour_plane_flag, as shown in Table 4.

[185] Tab. 4

chroma_format_idc separate_colour_plane_flag ChromaArrayType Chroma Format SubWidthC SubHeightC 0 0 0 Monochrome one one one 0 one 4:2:0 2 2 2 0 2 4:2:2 2 one 3 0 3 4:4:4 one one 3 one 0 4:4:4 one one

[186] In the above table 4, chroma_format_idc is information indicating the sampling format of the luminance signal and the sampling of the chrominance signal corresponding to it, and separate_colour_plane_flag is information indicating that the three color components Y, Cb and Cr are separately encoded in the chrominance signal format 4:4:4. In the above Table 4, when chroma_format_idc is 0, the chrominance signal format is monochrome, and the current block does not include a chrominance signal component block and only includes a luminance signal component block.

[187] In the above Table 4, when chroma_format_idc is 1, the chroma format corresponds to the 4:2:0 chroma format, and the width and height of the chroma component block appropriately correspond to half the width and half the height of the luma component block. Fig. 11a shows the location relationship between luma sampling and chrominance sampling in a 4:2:0 chrominance format.

[188] In the above Table 4, when chroma_format_idc is 2, the chrominance signal format corresponds to the 4:2:2 chrominance signal format, the width of the chroma component block appropriately corresponds to half the width of the luma component block, and the height of the chroma component block is equal to the block height components of the luminance signal. Fig. 11b shows the location relationship between luma sampling and chrominance sampling in a 4:2:2 chrominance format.

[189] In Table 4 above, when chroma_format_idc is 3, the chroma format corresponds to the 4:4:4 chroma format, and the width and height of the chroma component block appropriately correspond to the width and height of the luma component block. Fig. 11c shows the relationship of locations between luma sampling and chrominance sampling in a 4:4:4 chrominance format.

[190] In Table 4 above, SubWidthC and SubHeightC represent the ratio of luminance sampling and chrominance sampling. For example, when the width and height of the luma component block are respectively CbWidth and CbHeight, the width and height of the chroma component block corresponding to it can be extracted as (CbWidth/SubwidthC) and (CbHeight/SubHeightC), respectively.

[191] As described with reference to FIG. 11, the size of the chrominance signal component block corresponding to the luminance signal component block of the current block may vary according to the format of the chrominance signal.

[192] FIG. 12 is a view illustrating the size of the chrominance signal component block according to the chrominance signal format when the luminance signal component block is a 64×128 block. As shown in FIG. 12, the chrominance component block corresponding to the 64×128 luminance signal component block may be a 32×64 block in a 4:2:0 chroma format, a 32×128 block in a 4:2:2 chrominance format, and a 64× 128 in 4:4:4 chroma format.

[193] As described above, according to the conventional method described with reference to FIG. 9, in a single tree structure in which the luma component block and the chroma component block are partitioned into an identical tree structure, when ISP is applied to the current block, the luma component block is partitioned into a plurality of sub-segments, but the chroma component block is not partitioned. For example, when the luma component block of the current block to which ISP is applied is a 64x128 block, since the splitting direction is determined as the horizontal direction, the luminance component block is divided into four 64x32 sub-segments, and the width (64) and height (32) of each sub-segment are equal to or less than the maximum transform size (64). Accordingly, transformation or inverse transformation of each of the sub-segments for the luma component block can be performed. However, as shown in FIG. 12, the chrominance component block is a 32x64 block in 4:2:0 chroma format, a 32x128 block in 4:2:2 chroma format, or a 64x128 block in 4:4:4 chroma format, and, for example, in 4:2:2 chrominance format and 4:4:4 chrominance format, since the height (128) of the chrominance component block exceeds the maximum transformation size (64), the transformation or inverse transformation of the chroma component block is not possible .

[194] Hereinafter, various embodiments of the present disclosure are described in detail in order to resolve the above problems.

[195] Embodiment #1

[196] In Embodiment #1 of the present disclosure, in order to solve conventional problems when applying ISP to a current block, ISP is adaptively applied to a chroma component block based on the chrominance signal format and/or chrominance component block size. According to Embodiment #1 of the present disclosure, for example, when applying ISP to a chrominance component block, the split direction and the number of splits defined with respect to the luma component block are equally applicable to the chrominance component block.

[197] FIG. 13 is a flowchart illustrating an ISP application method according to an embodiment of the present disclosure.

[198] FIG. 14 is a view illustrating an example of splitting a block of chrominance signal components according to the ISP application method of FIG. 13.

[199] When the current block to be encoded/decoded is input (S1310), it may be determined whether or not to apply ISP to the current block (S1320). Determining step S1320 may include determining whether or not an ISP is available for the current block, and/or whether or not the ISP should be applied to the current block. The determination as to whether or not an ISP is available for the current block may be made based on the above-described ISP availability conditions. When the ISP is available, the image encoding equipment may determine whether or not the ISP should be applied to the current block based on various methods as described above, and the result of the determination may be encoded in the bitstream as intra prediction technology information. When the ISP is available, the image decoding equipment can determine whether or not the ISP should be applied to the current block based on the intra prediction technology signaling information.

[200] When the ISP is not applied to the current block, TU tiling may be performed relative to the current block (S1330). Thereafter, encoding/decoding may be performed on the current block or each of the blocks tiled through the TU tiling step (S1390). Steps S1330 and S1390 are identical to steps S930 and S960 in FIG. 9, and therefore their detailed description is omitted.

[201] In step S1320, when applying the ISP to the current block, the split direction and the number of splits (number of sub-segments) may be determined (S1340). The description of determining the split direction and the number of splits is identical to that of FIG. 9 and is therefore omitted.

[202] When the split direction and the number of splits are determined in step S1340, the block of luminance signal components of the current block may be split based on the indicated (S1350).

[203] Thereafter, it may be determined whether or not to apply the ISP to the chrominance signal component block corresponding to the luminance signal component block (S1360). The determination of step S1360 may be performed by the chrominance signal format and/or by comparing the width and height of the chroma component block and the maximum transform size.

[204] In the case of a 4:2:0 chrominance format as described below, the width and height of the chrominance component block is equal to or less than the maximum transform size. Accordingly, in the case of a chrominance signal format of 4:2:0, without comparing the width and height of the chrominance signal component block with the maximum transform size, it can be determined that ISP is not applied to the chrominance signal component block.

[205] In the case of a 4:2:2 chrominance signal format or a 4:4:4 chrominance signal format as described below, the width or height of the chrominance component block may exceed the maximum transform size. Accordingly, in the case of a 4:2:2 chrominance format or a 4:4:4 chrominance format, the width and height of the chroma component block can be compared with the maximum transform size, and when the width or height of the chrominance component block exceeds the maximum transform size, it may be determined that the ISP is applied to the chrominance component block.

[206] As shown in FIG. 14, when the luma component block is a 64×128 block, the chrominance signal component block corresponding to it is a 32×64 block in 4:2:0 chroma format, a 32×128 block in 4:2 chroma format: 2 or 64x128 block in 4:4:4 chroma format.

[207] In the example shown in FIG. 14, it can be seen that the width (32) and height (64) of the 4:2:0 chrominance format chrominance signal component block are equal to or smaller than the maximum transform size (64). Accordingly, in the case of a chrominance signal format of 4:2:0, it can be determined that the ISP is not applied to the 32×64 chrominance signal component block without additional size comparison.

[208] In contrast, in the case of a 4:2:2 chrominance format or a 4:4:4 chrominance format, further, the width and height of the chrominance component block may be compared with the maximum transform size. In the example shown in FIG. 14, since the width (128) of the 32x128 4:2:2 chroma format chroma component box and the 64x128 4:4:4 chroma format chroma component box width (128) exceed the maximum transform size (64), it may be determined that the ISP is applied to the chrominance component block.

[209] As a modification of Embodiment #1, regardless of the format of the chrominance signal, based on whether the width and height of the chrominance signal component block of the maximum transform size are equal or less than or not, it can be determined whether or not ISP should be applied to the signal component block chromaticity. For example, when the width and height of the chroma component block are equal to or less than the maximum transform size, it may be determined that the ISP is not applied to the chroma component block. In addition, when the width or height of the chrominance component block exceeds the maximum transform size, it may be determined that ISP is applied to the chrominance component block.

[210] When determining that the ISP is applied to the chrominance signal component block, the chrominance signal component block may be split (S1370). Splitting of the chrominance signal component block may be performed based on the splitting direction and the number of splits determined in step S1340. In particular, the split direction and the number of splits for the chroma component block may be determined identically to the split direction and the number of splits for the luma component block. According to Embodiment #1 of the present disclosure, in the example shown in FIG. 14, the luma component block is divided into four sub-segments in the horizontal direction. Accordingly, the 32×128 chroma component block of the 4:2:2 chroma format and the 64×128 chrominance signal component block of the 4:4:4 chroma format can be divided into four sub-segments in the horizontal direction in the same manner as the luma component block.

[211] Thereafter, each of the broken sub-segments may be encoded/decoded (S1390). In this case, encoding/decoding of step S1390 may be performed on each sub-segment of the luma component block and each sub-segment of the chrominance component block. Step S1390 is the same as Step S960, and therefore its detailed description will be omitted.

[212] In step S1360, when determining that ISP is not applied to the chrominance signal component, the chrominance component block is not decomposed (S1380), and each sub-segment of the luma component block and the undecomposed chrominance component block may be encoded/decoded (S1390) . Step S1390 is the same as Step S960, and therefore its detailed description will be omitted. According to Embodiment #1 of the present disclosure, in the example shown in FIG. 14, a chrominance signal component block of 32×64 of 4:2:0 chrominance format may not be split.

[213] According to Embodiment #1 of the present disclosure, when the width or height of the chroma component block of the current block to which the ISP is applied exceeds the maximum transform size, it may be possible to resolve such a problem that the transformation or inverse transformation of the chroma component block is impossible by applying ISP to the chrominance component block. In addition, according to Embodiment #1 of the present disclosure, when the chrominance signal format of the current block to which ISP is applied is 4:2:0, or the width and height of the chroma component block of the current block to which ISP is applied are equal to or less than the maximum transform size, ISP is not applied to the chrominance component block. Therefore, since the case in which ISP is unnecessarily applied to a block of chrominance signal components can be prevented, the amount of encoding/decoding calculation can be reduced. In addition, according to Embodiment #1 of the present disclosure, since the split direction and the number of ISPs for the chrominance component block are determined to be identical to the split direction and number of ISPs for the luminance component block, the split direction and number of ISPs for the chrominance component block are not must be separately signaled or retrieved.

[214] Embodiment #2

[215] In Embodiment #2 of the present disclosure, in order to solve the conventional problem, when ISP is applied to the current block, ISP is also applied to the chroma component block. According to Embodiment #2 of the present disclosure, a split direction and a number of splits for a chroma component block may be determined based on a split direction and a number of splits determined for the luma component block.

[216] FIG. 15 is a flowchart illustrating an ISP application method according to another embodiment of the present disclosure.

[217] FIG. 16 is a view illustrating an example of splitting a block of chrominance signal components according to the ISP application method of FIG. fifteen.

[218] When the current block to be encoded/decoded is input (S1510), it may be determined whether or not to apply ISP to the current block (S1520). Determining step S1520 may include determining whether or not an ISP is available for the current block, and/or whether or not the ISP should be applied to the current block. The determination as to whether or not an ISP is available for the current block may be made based on the above-described ISP availability conditions. When the ISP is available, the image encoding equipment may determine whether or not the ISP should be applied to the current block based on various methods as described above, and the result of the determination may be encoded in the bitstream as intra prediction technology information. When the ISP is available, the image decoding equipment can determine whether or not the ISP should be applied to the current block based on the intra prediction technology signaling information.

[219] When ISP is not applied to the current block, TU tiling may be performed relative to the current block (S1530). Thereafter, encoding/decoding may be performed on the current block or each of the blocks tiled through the TU tiling step (S1570). Steps S1530 and S1570 are identical to steps S930 and S960 in FIG. 9, and their detailed description is omitted.

[220] In step S1520, when applying the ISP to the current block, the split direction and the number of splits (number of sub-segments) may be determined (S1540). The description of determining the split direction and the number of splits is identical to that of FIG. 9 and is therefore omitted.

[221] The luma component block of the current block may be partitioned based on the partition direction and the number of partitions determined in step S1540 (S1550).

[222] Thereafter, the chrominance signal component block of the current block may be partitioned based on the partition direction and the number of partitions determined in step S1540 (S1560). In particular, the splitting direction for the chroma component block may be determined identically to the splitting direction for the luma component block. In addition, the number of tilings for the chroma component block may be determined based on the number of tilings for the luma component block. For example, when the number of tilings for the luma component block is N, the number of tilings for the chrominance component block may be determined as N/n. In Embodiment #2 of the present disclosure, for example, n may be but is not limited to 2, and n may be any integer. According to Embodiment #2, in the example shown at the top of FIG. 16, the 16×16 luma component block may be divided into four sub-segments in the horizontal direction. In this case, the splitting direction of the corresponding chrominance component block is the horizontal direction, and the number of splits may be defined as 2. Thus, the corresponding chrominance component block can be split into two sub-segments in the horizontal direction regardless of the chrominance signal format and/or size. chrominance signal component block. In addition, in the example shown at the bottom of FIG. 16, a 4×8 luma component block is split into two sub-segments in the vertical direction. In this case, the splitting direction of the corresponding chrominance component block is the vertical direction, and the number of splits may be determined as 1. In this case, since the chrominance signal component block is almost not partitioned, it can be seen that ISP is not applied to the chrominance component block. .

[223] Thereafter, each sub-segment or unsplit block of chrominance components may be encoded/decoded (S1570). In this case, encoding/decoding of step S1570 may be performed on each sub-segment of the luma component block and each sub-segment of the chrominance component block or the chroma component block. Step S1570 is the same as Step S960, and therefore its detailed description will be omitted.

[224] In Embodiment #2 of the present disclosure, when the size of the maximum coding unit is 128×128, the maximum transform size is 64, which is half the width and height of the maximum coding unit, and the width and height of the current block exceed the maximum transform size, ISP does not apply to the current block. According to Embodiment #2 of the present disclosure, since the width and height of the chrominance component block of the current block to which ISP is applied or a sub-segment of the chroma component block is always equal to or less than the maximum transform size, the problem that the transform or inverse transform of the block color signal components is not possible. In addition, according to Embodiment #2 of the present disclosure, since a determination as to whether or not ISP should be applied to a block of chrominance signal components of the current block (for example, determination of S1360) need not necessarily be performed, the amount of encoding/decoding calculation may be reduced. In addition, according to Embodiment #2 of the present disclosure, since the number of sub-segments for a chroma component block is determined to be 2 or 1, the encoding/decoding process of the chroma component block can be simplified. In addition, according to Embodiment #2 of the present disclosure, since the split direction and the number of ISPs for the chroma component block are determined to be identical to the split direction and (number of splits/2) ISP for the luma component block, the split direction and the number of ISPs for the block chrominance signal components do not need to be separately signaled or extracted.

[225] Embodiment #3

[226] In Embodiment #3 of the present disclosure, in order to solve the conventional problem, the width and height of the chrominance component block is equal to or less than the maximum transform size by changing the ISP availability conditions.

[227] According to the conventional ISP method described with reference to FIG. 9, when the width and height of the current block exceed the maximum transform size, it is determined that the ISP is not applied to the current block. Thus, when one of the width or height of the current block is greater than the maximum transform size and the other is equal to or less than the maximum transform size, it is determined that the ISP is applied to the current block.

[228] In Embodiment #3 of the present disclosure, the above-described ISP availability conditions are modified as the following Table 5.

[229] Tab. five

Modified ISP availability conditions
intra_luma_ref_idx[x0][y0]==0
cbWidth<=MaxTbSizeY andand cbHeight<=MaxTbSizeY
cbWidth*cbHeight>MinTbSizeY*MinTbSizeY

[230] Of the modified ISP availability conditions, the condition in which intra_luma_ref_idx[x0][y0] is 0 and the condition in which cbWidth*cbHeight is greater than MinTbSizeY*MinTbSizeY are equal to the existing ISP availability conditions. According to the modified ISP availability conditions, only when the cbWidth and cbHeight of the current block are equal to or less than MaxTbSizeY can it be determined that the ISP is available for the current block.

[231] According to embodiment #3 of the present disclosure, in which modified ISP accessibility conditions are applied, only when the width and height of the current block are equal to or less than the maximum transformation size, it is determined that ISP is applied to the current block, and intra_subpartitions_mode_flag, indicating whether or not to apply the ISP may be transmitted.

[232] According to Embodiment #3 of the present disclosure, since the width and height of the current block is equal to or less than the maximum transform size, the width and/or height of the chroma sample block corresponding to the luma sample block of the current block is always equal to or less than the maximum transform size. , regardless of the color signal format. Accordingly, it is possible to solve the problem of the conventional ISP method described with reference to FIG. 9 by applying modified ISP accessibility conditions.

[233] The method according to Embodiment #3 of the present disclosure may be implemented by modifying the ISP availability conditions of the conventional method. Accordingly, the method according to Embodiment #3 of the present disclosure may be performed identically to the flowchart shown in FIG. 9, except that only the condition for determining whether or not an ISP is available in step S920 is different.

[234] FIG. 17 is a view illustrating an example of a bitstream structure in which a modified ISP availability condition is reflected.

[235] The rectangular field of FIG. 17 is related to the signaling of information regarding the ISP of the intra prediction technology information of the current coding unit. As shown in FIG. 17, when the above-described modified ISP availability conditions are satisfied, an intra_subpartitions_mode_flag regarding whether or not ISP should be applied may be transmitted; intra_subpartitions_mode_flag is information indicating whether or not ISP should be applied to the current coding unit. When the ISP availability conditions are not satisfied, the intra_subpartitions_mode_flag is not transmitted, and it may be determined that the ISP is not applied to the current coding unit.

[236] The modified ISP accessibility conditions may include one or more conditions, and the one or more conditions included in the modified ISP accessibility conditions are not limited to the above examples. Thus, some terms may be omitted, or other terms may additionally be included within the scope of the technical idea according to the present disclosure.

[237] In addition, as shown in FIG. 17 when ISP is applied to the current block, i. e. when intra_subpartitions_mode_flag is equal to 1, information (intra_subpartitions_split_flag) indicating a split direction may be transmitted.

[238] In the example shown in FIG. 17, only when the condition cbWidth<=MaxTbSizeY andand cbHeight<=MaxTbSizeY is satisfied, intra_subpartitions_mode_flag may be transmitted. Thus, when the transmitted intra_subpartitions_mode_flag is equal to 1, it can be seen that the condition cbWidth<=MaxTbSizeY andand cbHeight<=MaxTbSizeY I is already satisfied. Accordingly, the condition of FIG. 17 related to whether intra_sinpartitions_split_flag should be transmitted or not may vary as shown in FIG. eighteen.

[239] FIG. 18 is a view illustrating another example of a bitstream structure in which a modified ISP availability condition is reflected.

[240] The rectangular field of FIG. 18 is related to the signaling of information regarding the ISP of the intra prediction technology information of the current coding unit. As shown in FIG. 18, intra_subpartitions_mode_flag may be signaled when the above-described modified ISP availability conditions are satisfied, and intra_sinpartitions_split_flag may be signaled when a condition in which intra_subpartitions_mode_flag is 1 is satisfied.

[241] According to embodiment #3 of the present disclosure, the width and height of the current block to which the ISP is applied is equal to or less than the maximum transform size. Accordingly, since the width and height of the chrominance component block of the current block are equal to or smaller than the maximum transformation size regardless of the chrominance signal format, the problem can be solved that the transformation or inverse transformation of the chroma component block is impossible. In addition, according to Embodiment #3 of the present disclosure, since the determination as to whether or not ISP should be applied to the chrominance signal component block of the current block (for example, determination S1360) and ISP for the chrominance signal component block need not be made, the scope encoding/decoding calculations may decrease. In addition, according to Embodiment #3 of the present disclosure, since the conventional encoding/decoding process is not substantially changed, the conventional problem can be resolved without significant encoding/decoding complexity.

[242] Although the exemplary methods of the present disclosure described above are presented as a sequence of operations for clarity of description, it is not intended to limit the order in which the steps are performed, and the steps may be performed simultaneously or in a different order as needed. In order to implement the method according to the present disclosure, the described steps may further include other steps, may include the remaining steps except for some steps, or may include other additional steps except for some steps.

[243] In the present disclosure, an image encoding equipment or an image decoding equipment that performs a predetermined operation (step) may perform an operation (step) of confirming a condition or situation of performing a corresponding operation (step). For example, if it is described that the predetermined operation is performed when the predetermined condition is satisfied, the image encoding equipment or the image decoding equipment may perform the predetermined operation after determining whether or not the predetermined condition is satisfied.

[244] The various embodiments of the present disclosure are not a list of all possible combinations and are intended to describe salient aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combination of two or more of them.

[245] Various embodiments of the present disclosure may be implemented in hardware, firmware, software, or a combination of the above. If the present disclosure is implemented in hardware, the present disclosure may be implemented using Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGA), general processors, controllers, microcontrollers, microprocessors, etc.

[246] In addition, image decoding equipment and image encoding equipment to which embodiments of the present disclosure apply may be included in a multimedia broadcast transceiver, a mobile communication terminal, a home theater system video device, a digital cinema system video device, a surveillance camera, video chatting device, real-time communication device, for example, for video communication, mobile streaming device, storage media, video recording camera, video-on-demand (VoD) service delivery device, OTT video device ( video over networks), an Internet streaming service provider, a 3D video device, a video telephony system video device, a medical video device, and the like. and can be used to process video signals or data signals. For example, OTT video devices may include a game console, a Blu-ray player, an Internet access TV, a home theater system, a smartphone, a tablet PC, a digital video recorder (DVR), and the like.

[247] FIG. 19 is a view showing a content streaming system to which an embodiment of the present disclosure is applicable.

[248] Referring to FIG. 19, the content streaming system to which the embodiment(s) of the present document is applied may mainly include an encoding server, a streaming server, a web server, a media storage, a user device, and a media input device.

[249] The encoding server compresses content input from media input devices such as a smartphone, camera, camcorder, etc. into digital data to form a bit stream, and transmits the bit stream to the stream server. As another example, when media input devices such as smartphones, cameras, camcorder recorders, etc. directly generate a bitstream, the encoding server may be omitted.

[250] The bitstream may be generated by an image encoding method or image encoding equipment to which an embodiment of the present disclosure applies, and the streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[251] The streaming server transmits media data to the user device based on the user's request through the web server, and the web server serves as a medium for informing the user about the service. When a user requests a desired service from a web server, the web server may deliver it to the streaming server, and the streaming server may transmit media data to the user. In this case, the content streaming system may include a separate control server. In this case, the control server serves to manage the command/response between devices in the content streaming system.

[252] The streaming server may receive content from a media store and/or an encoding server. For example, when content is received from an encoding server, the content may be received in real time. In this case, in order to provide a smooth streaming service, the streaming server may keep the bitstream for a predetermined time.

[253] Examples of a user device may include a mobile phone, smartphone, laptop computer, digital broadcast terminal, personal digital assistant (PDA), portable media player (PMP), navigation device, stylus tablet PC, tablet PCs, ultrabooks, wearable devices (e.g., smart watches, smart glasses, head-mounted displays), digital TVs, desktop computers, digital signage system, etc.

[254] Each server in the content streaming system can operate as a distributed server, in which case the data received from each server can be distributed.

[255] The scope of the disclosure includes software or computer-executable instructions (e.g., an operating system, application, firmware, program, etc.) for enabling operations according to the methods of various embodiments in equipment or a computer, a non-volatile computer a readable medium having such software or instructions stored and executable in a hardware or computer.

[256] Industrial Applicability

[257] Embodiments of the present disclosure may be used to encode or decode an image.

Claims (43)

1. An image decoding method performed by image decoding equipment, wherein the image decoding method comprises the steps of: determining a prediction mode of the current block based on the prediction mode information of the current block; determining whether an intra sub-segment (ISP) mode is available for the current block based on information of the prediction mode of the current block, which is an intra prediction mode; decoding the ISP mode information for the current block based on the ISP that is available for the current block; and generating a prediction sample for the current block based on the ISP mode information, wherein, based on both a width and a height of a block of luma components of the current block equal to or less than a specific threshold value, the ISP is determined to be available to the current block. 2. The image decoding method of claim 1, wherein the specific threshold is a maximum transform size. 3. The image decoding method according to claim 1, wherein prediction sampling for the current block comprises the steps of: generating a prediction sample for the block of luminance signal components by applying ISP to the block of luminance signal components of the current block; determining whether to apply the ISP to the chrominance signal component block of the current block; and generating a prediction sample for the chrominance signal component block by applying the ISP to the chrominance signal component block, while determining that the ISP is applied to the chrominance signal component block of the current block. 4. The image decoding method of claim 3, wherein determining whether or not ISP should be applied to the chrominance component block of the current block is performed based on at least one of the chrominance component block size or the chrominance signal format of the current block. 5. The image decoding method according to claim 4, wherein, based on the chrominance signal format of the current block, which is 4:2:2 or 4:4:4, and at least one of the width or height of the chrominance signal component block that exceeds the maximum the size of the transform, ISP is defined as being applied to the block of chroma components of the current block. 6. The image decoding method according to claim 3, wherein, based on the ISP applied to the chrominance signal component block, the split direction and the number of sub-segments for the chroma component block are determined based on the split direction and the number of sub-segments for the luma component block. 7. The image decoding method according to claim 6, wherein the split direction and the number of sub-segments for the chroma component block are equal to the split direction and the number of sub-segments for the luma component block, respectively. 8. The image decoding method according to claim 1, wherein prediction sampling for the current block comprises the steps of: generating a prediction sample for the block of luminance signal components by applying ISP to the block of luminance signal components of the current block; and generating a prediction sample for the chrominance signal component block by applying the ISP to the chrominance signal component block of the current block. 9. The image decoding method according to claim 8, wherein the split direction and the number of sub-segments for the chroma component block are determined based on the split direction and the number of sub-segments for the luma component block. 10. The image decoding method according to claim 9, wherein the split direction and the number of sub-segments for the chrominance component block are equal to the split direction and (number of sub-segments/2) of the luma component block, respectively. 11. Image decoding equipment, comprising: Memory device; and at least one processor wherein at least one processor is configured to: determine the prediction mode of the current block based on the prediction mode information of the current block; determine whether an intra sub-segment (ISP) mode is available for the current block based on the information of the prediction mode of the current block, which is the intra prediction mode; decode the ISP mode information for the current block, based on the ISP that is available for the current block; and generate a prediction sample for the current block based on the ISP mode information, wherein, based on both a width and a height of a block of luma components of the current block equal to or less than a specific threshold value, the ISP is determined to be available to the current block. 12. An image encoding method carried out by means of image encoding equipment, wherein the image encoding method comprises the steps of: determining a prediction mode of the current block; determining whether an intra sub-segment (ISP) mode is available for the current block based on information of the prediction mode of the current block, which is an intra prediction mode; determining whether to apply the ISP to the current block based on the ISP available to the current block; generating a prediction sample for the current block by applying the ISP to the current block, based on the ISP determined to be applied to the current block; and encoding prediction mode information for the current block and ISP mode information for the current block, wherein, based on both a width and a height of a block of luma components of the current block equal to or less than a specific threshold value, the ISP is determined to be available to the current block. 13. A non-volatile computer-readable medium storing a bitstream generated by an image encoding method, wherein the image encoding method comprises the steps of: determining a prediction mode of the current block; determining whether an intra sub-segment (ISP) mode is available for the current block based on information of the prediction mode of the current block, which is an intra prediction mode; determining whether to apply the ISP to the current block based on the ISP available to the current block; generating a prediction sample for the current block by applying the ISP to the current block, based on the ISP determined to be applied to the current block; and encoding the prediction mode information for the current block and the ISP mode information for the current block into a bit stream, wherein, based on both a width and a height of a block of luma components of the current block equal to or less than a specific threshold value, the ISP is determined to be available to the current block.

Images