RU2811456C2 - Method and equipment for image encoding/decoding and method for bit stream transfer - Google Patents

Abstract

FIELD: image encoding/decoding.

SUBSTANCE: invention consists of applying an intra-subblock segment (ISP) mode to images having different chrominance signal formats. The image decoding method includes determining whether or not the prediction mode of the current block is an intra prediction mode based on information regarding the prediction mode of the current block, determining whether or not the mode is available based on intra sub segments (ISP) for of the current block when the prediction mode of the current block is an intra prediction mode, decoding an ISP application indicator indicating whether or not ISP should be applied to the current block when ISP is available for the current block, and generating a prediction block for the current block by applying the ISP to the current block, when determining that the ISP is applied to the current block based on the ISP applied indicator.

EFFECT: increase of efficiency of image compression technology.

11 cl, 23 dwg, 5 tbl

Description

Field of technology to which the invention relates

[1] The present disclosure relates to a method and equipment for encoding/decoding images and a method for transmitting a bit stream, and more specifically, to a method and equipment for encoding/decoding an image by applying an intra-subblock segment (ISP)-based mode to images having various chrominance signal formats, and to a method of transmitting a bit stream generated by the image encoding method/equipment of the present disclosure.

State of the art

[2] Recently, the demand for high-quality, high-definition images, such as high-definition (HD) images and ultra-high-definition (UHD) images, has been increasing in various fields of technology. As the resolution and quality of image data increases, the amount of transmitted information or bits increases relatively compared to existing image data. Increasing the amount of information or bits transmitted results in increased transmission costs and storage costs.

[3] Accordingly, there is a need for high-performance image compression technology for efficiently transmitting, storing and reproducing information regarding high-quality, high-resolution images.

The essence of the invention

Technical problem

[4] It is an object of the present disclosure to provide an image encoding/decoding method and equipment with improved encoding/decoding efficiency.

[5] It is an object of the present disclosure to provide a method and equipment for encoding/decoding an image by applying ISP to images having different chrominance signal formats.

[6] Another object of the present disclosure is to provide a method for transmitting a bitstream generated by an image encoding method or equipment according to the present disclosure.

[7] Another object of the present disclosure is to provide a recording medium storing a bitstream generated by an image encoding method or equipment according to the present disclosure.

[8] Another object of the present disclosure is to provide a recording medium storing a bit stream received, decoded and used to reconstruct an image by the image decoding equipment according to the present disclosure.

[9] The technical problems addressed by the present disclosure are not limited to the above technical problems, and other technical problems not described herein should become apparent to those skilled in the art from the following description.

Technical solution

[10] An image decoding method performed by the image decoding equipment according to an aspect of the present disclosure may include determining whether or not a prediction mode of the current block is an intra prediction mode, based on information regarding the prediction mode of the current block, determining whether or not a prediction mode is available. or not an internal subsegment (ISP) mode for the current block, when the prediction mode of the current block is an intra prediction mode, decoding an ISP application indicator indicating whether or not an ISP should be applied to the current block when an ISP is available for the current block, and generating a prediction block for the current block by applying the ISP to the current block, determining that the ISP is applied to the current block based on the ISP-apply indicator.

[11] In the image decoding method of the present disclosure, determining whether or not an ISP is available for the current block can be performed based on a comparison between the block size of the luminance signal components of the current block and a predetermined threshold value.

[12] In the image decoding method of the present disclosure, a predetermined threshold value may represent a maximum transform size.

[13] In the image decoding method of the present disclosure, when the block width and height of luminance signal components of the current block are equal to or less than a predetermined threshold value, it can be determined that an ISP is available for the current block.

[14] In the image decoding method of the present disclosure, generating a prediction block for a current block by applying an ISP to the current block may include generating a prediction block for a block of luma signal components by applying ISP to a block of luma signal components of the current block, determining whether or not whether or not to apply the ISP to the chrominance component block of the current block, and generating a prediction block for the chrominance component block by applying the ISP to the chrominance component block, while determining that the ISP is applied to the chrominance component block of the current block.

[15] In the image decoding method of the present disclosure, determining whether or not an ISP should be applied to a chrominance component block of the current block can be performed based on at least one of the chrominance component block size or the chrominance signal format of the current block.

[16] In the image decoding method of the present disclosure, when the chrominance signal format of the current block is 4:2:2 or 4:4:4, and at least one of the block width or height of the chrominance signal components exceeds the maximum conversion size , it may be determined that the ISP is applied to the chrominance component block of the current block.

[17] In the image decoding method of the present disclosure, when ISP is applied to a chrominance component block, the partitioning direction and the number of sub-segments for the chrominance signal component block can be determined based on the partitioning direction and the number of sub-segments for the luma component block.

[18] In the image decoding method of the present disclosure, the partitioning direction and the number of sub-segments for a chrominance signal component block may be equal to the partitioning direction and the number of sub-segments for a luminance signal component block, respectively.

[19] In the image decoding method of the present disclosure, generating a prediction block for a current block by applying an ISP to the current block may include generating a prediction block for a block of luminance signal components by applying ISP to a block of luminance signal components of the current block, and generating a prediction block for a chrominance component block by applying the ISP to the chrominance component block of the current block.

[20] In the image decoding method of the present disclosure, the partitioning direction and the number of sub-segments for a chroma component block may be determined based on the partitioning direction and the number of sub-segments for a luminance signal component block.

[21] In the image decoding method of the present disclosure, the partitioning direction and the number of sub-segments for a chroma component block may be equal to the partitioning direction and (number of sub-segments/2) of the luma component block, respectively.

[22] Image decoding equipment according to another aspect of the present disclosure may include a storage device and at least one processor. The at least one processor may determine whether or not the prediction mode of the current block is an intra prediction mode based on information regarding the prediction mode of the current block, determine whether or not an intra subsegment (ISP) based mode is available for the current block when the current block prediction mode is an intra prediction mode, decode an ISP application indicator indicating whether or not ISP should be applied to the current block when the ISP is available for the current block, and generate a prediction block for the current block by applying the ISP to the current block when determining that the ISP is applied to the current block based on the ISP applied indicator.

[23] A method for encoding images according to another aspect of the present disclosure may include determining whether or not a prediction mode of the current block is an intra prediction mode, determining whether or not an intra sub-segment (ISP) based mode is available for the current block, when the current block prediction mode is an intra prediction mode, determining whether or not an ISP should be applied to the current block when the ISP is available for the current block, generating a prediction block for the current block by applying the ISP to the current block when the ISP is applied to the current block, and encoding information regarding the prediction mode of the current block and information regarding whether or not ISP should be applied to the current block.

[24] A transmission method according to another aspect of the present disclosure may transmit a bitstream generated by an image encoding equipment or an image encoding method of the present disclosure.

[25] A computer-readable recording medium according to another aspect of the present disclosure can store a bitstream generated by an image encoding equipment or an image encoding method of the present disclosure.

[26] A computer-readable recording medium according to another aspect of the present disclosure may store a bit stream received and decoded by image decoding equipment and used to reconstruct an image. The bitstream may include information regarding the prediction mode of the current block and an ISP-apply indicator indicating whether or not ISP should be applied to the current block, information regarding the prediction mode of the current block may be used to determine whether or not the prediction mode is present. of the current block is an intra prediction mode, the ISP application indicator can be used to determine whether or not ISP should be applied to the current block, when the prediction mode of the current block is an intra prediction mode, and it is determined that the ISP is available for the current block , and a current block prediction block may be generated by executing an ISP on the current block upon determining that the ISP is applied to the current block.

[27] The features briefly summarized above with respect to the present disclosure are merely exemplary aspects of the following detailed description of the present disclosure and do not limit the scope of the present disclosure.

Advantages of the invention

[28] According to the present disclosure, it is possible to provide an image encoding/decoding method and equipment with improved encoding/decoding efficiency.

[29] In addition, according to the present disclosure, it is possible to provide a method and equipment for encoding/decoding an image by applying an ISP to images having different chrominance signal formats.

[30] In addition, according to the present disclosure, it is possible to provide a method for transmitting a bit stream generated by an image encoding method or equipment according to the present disclosure.

[31] In addition, according to the present disclosure, it is possible to provide a recording medium storing a bit stream generated by the image encoding method or equipment according to the present disclosure.

[32] In addition, according to the present disclosure, it is possible to provide a recording medium storing a bitstream received, decoded, and used to reconstruct an image by the image decoding equipment according to the present disclosure.

[33] Those skilled in the art will appreciate that the advantages that may be achieved through the present disclosure are not limited to those specifically described above, and other advantages of the present disclosure will be more clearly understood from the detailed description.

Brief description of drawings

[34] FIG. 1 is a view schematically showing a video encoding system to which an embodiment of the present disclosure is applicable.

[35] FIG. 2 is a view schematically showing image encoding equipment to which an embodiment of the present disclosure is applicable.

[36] FIG. 3 is a view schematically showing image decoding equipment to which an embodiment of the present disclosure is applicable.

[37] FIG. 4 is a flowchart illustrating a video/image encoding method based on intra prediction.

[38] FIG. 5 is a view illustrating the configuration of the intra prediction unit 185 according to the present disclosure.

[39] FIG. 6 is a flowchart illustrating a video/image decoding method based on intra prediction.

[40] FIG. 7 is a view illustrating the configuration of the intra prediction module 265 according to the present disclosure.

[41] FIG. 8a and 8b are views illustrating an intra prediction direction according to an embodiment of the present disclosure.

[42] FIG. 9 is a view illustrating a conventional method of performing encoding/decoding of a current block according to an ISP mode.

[43] FIG. 10 is a view illustrating an example of ISP partitioning for current blocks having different sizes.

[44] FIG. 11 is a view illustrating the relationship between a luma component block (luminance signal component array) and a chrominance signal component block (chrominance signal component array) according to a chrominance signal format.

[45] FIG. 12 is a view illustrating the size of a chrominance component block according to a chrominance signal format when the luma component block is a 64×128 block.

[46] FIG. 13 is a flowchart illustrating an ISP application method according to an embodiment of the present disclosure.

[47] FIG. 14 is a view illustrating an example of dividing a block of chrominance signal components according to the ISP application method of FIG. 13.

[48] FIG. 15 is a flowchart illustrating an ISP application method according to another embodiment of the present disclosure.

[49] FIG. 16 is a view illustrating an example of dividing a block of chrominance signal components according to the ISP application method of FIG. 15.

[50] FIG. 17 is a view illustrating an example of a bitstream structure in which the modified ISP availability condition is reflected.

[51] FIG. 18 is a view illustrating another example of a bitstream structure in which a modified ISP availability condition is reflected.

[52] FIG. 19 is a view showing a content streaming system to which an embodiment of the present disclosure is applicable.

Optimal mode for carrying out the invention

[53] Embodiments of the present disclosure are described in detail hereinafter with reference to the accompanying drawings so that they can be easily implemented by those skilled in the art. However, the present disclosure may be implemented in a variety of forms and is not limited to the embodiments described herein.

[54] In describing the present disclosure, if it is determined that a detailed description of a related known function or construct would cause unnecessary ambiguity to the scope of the present disclosure, the detailed description thereof is omitted. In the drawings, parts not related to the description of the present disclosure are omitted, and like reference numbers are added to like parts.

[55] In the present disclosure, when a component is “connected,” “coupled,” or “linked” to another component, this may include not only a direct connection-based relationship, but also an indirect connection-based relationship in which an intermediate component is present. In addition, when a component “includes” or “has” other components, it means that other components may be additionally included, not the exclusion of other components, unless otherwise noted.

[56] In the present disclosure, the terms “first”, “second”, etc. may be used only for the purpose of distinguishing one component from other components and does not limit the order or importance of the components unless otherwise noted. Accordingly, within the scope of the present disclosure, a first component in one embodiment may be referred to as a “second component” in another embodiment, and likewise, a second component in one embodiment may be referred to as a “first component” in another embodiment.

[57] In the present disclosure, components that are different from each other are intended to clearly describe each feature and do not mean that the components are necessarily separate. Thus, multiple components may be integrated and implemented in a single hardware or software module, or a single component may be distributed and implemented in multiple hardware or software modules. Therefore, even if not stated otherwise, such embodiments in which components are integrated or a component is distributed are also included within the scope of the present disclosure.

[58] In the present disclosure, the components described in the various embodiments do not necessarily mean essential components, and some components may be optional components. Accordingly, an embodiment consisting of a subset of the components described in the embodiment is also included within the scope of the present disclosure. In addition, embodiments including other components in addition to the components described in the various embodiments are included within the scope of the present disclosure.

[59] The present disclosure relates to image encoding and decoding, and the terms used in the present disclosure may have a general meaning commonly used in the art to which the present disclosure pertains, unless defined for the first time in the present disclosure.

[60] In the present disclosure, "frame" generally means a unit representing one image at a particular time period, and a slice/tile is an encoding unit constituting part of a frame, and one frame may consist of one or more slices/ tiles In addition, a slice/tile may include one or more coding tree units (CTUs).

[61] In the present disclosure, "pixel" or "pixel" can mean the smallest unit constituting one frame (or image). In addition, "sample" can be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, and may represent only a pixel/pixel value of a luma component or only a pixel/pixel value of a chrominance signal component.

[62] In the present disclosure, a "unit" may represent a basic unit of image processing. The unit may include at least one of a specific frame area and information associated with the area. The unit may be used interchangeably with terms such as "sample array", "block" or "zone" in some cases. In general, an MxN block may include samples (or arrays of samples) or a set (or array) of transform coefficients of M columns and N rows.

[63] In the present disclosure, "current block" may mean one of "current encoding block", "current encoding unit", "target encoding block", "target decoding block" or "target processing block". When prediction is performed, "current block" may mean "current prediction block" or "target prediction block". When transform (inverse transform)/quantization (dequantization) is performed, "current block" can mean "current transform block" or "transform target block". When filtering is performed, "current block" can mean "target filtering block".

[64] In the present disclosure, the terms "/" and "" are to be interpreted to indicate "and/or". For example, the expression "A/B" and "A, B" can mean "A and/or B". Additionally, "A/B/C" and "A/B/C" may mean "at least one of A, B and/or C".

[65] In the present disclosure, the term “or” should be interpreted to indicate “and/or.” For example, the expression "A or B" may contain 1) only "A", 2) only "B", and/or 3) "both A and B". In other words, in the present disclosure, the term “or” is to be interpreted as indicating “in addition or alternatively.”

[66] Video Coding System Overview

[67] FIG. 1 is a view showing a video encoding system according to the present disclosure.

[68] The video encoding system according to an embodiment may include encoding equipment 10 and decoding equipment 20. Encoding equipment 10 may deliver encoded information or video and/or image data to decoding equipment 20 in the form of a file or streaming via a digital storage medium or network.

[69] The encoding equipment 10 according to an embodiment may include a video source generator 11, an encoding unit 12, and a transmitter 13. The decoding equipment 20 according to an embodiment may include a receiver 21, a decoding unit 22, and a rendering unit 23. The encoding unit 12 may be referred to as a “video/image encoding unit,” and the decoding unit 22 may be referred to as a “video/image decoding unit.” The transmitter 13 may be included in the encoding module 12. The receiver 21 may be included in the decoding module 22. The rendering module 23 may include a display, and the display may be configured as a separate device or an external component.

[70] The video source generator 11 may obtain video/image through a video/image capturing, synthesizing, or shaping process. The video source generator 11 may include a video/image capture device and/or a video/image generation device. The video/image capturing device may include, for example, one or more cameras, video/image archives including previously captured video/images, and the like. The video/image generating apparatus may include, for example, computers, tablet computers and smartphones and may (electronically) generate video/images. For example, a virtual video/image may be generated via a computer or the like. In this case, the video/image capturing process can be replaced by the associated data generation process.

[71] The encoding unit 12 may encode the input video/image. The encoding unit 12 may perform a series of procedures such as prediction, transform and quantization for compression and encoding efficiency. The encoding unit 12 can output encoded data (encoded video/image information) in the form of a bit stream.

[72] The transmitter 13 may transmit encoded information or video/image data output in the form of a bitstream to the receiver 21 of the decoding equipment 20 via a digital storage medium or a network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-Ray, HDD, SSD, and the like. The transmitter 13 may include an element for generating a multimedia file through a predefined file format and may include an element for transmission through a broadcast/communications network. The receiver 21 may retrieve/receive a bit stream from the storage medium or network and transmit the bit stream to the decoding unit 22.

[73] The decoding unit 22 can decode a video/image by performing a series of procedures such as dequantization, deconversion and prediction corresponding to the operation of the encoding unit 12.

[74] The rendering unit 23 may prepare a decoded video/image by rendering. The video/image prepared by rendering can be displayed through the display.

[75] General introduction to image encoding equipment

[76] FIG. 2 is a view schematically showing image encoding equipment to which an embodiment of the present disclosure is applicable.

[77] As shown in FIG. 2, the image encoding equipment 100 may include an image segmentation unit 110, a subtractor 115, a converter 120, a quantizer 130, a dequantizer 140, an inverse converter 150, an adder 155, a filter 160, a memory 170, an inter prediction unit 180, an intra prediction unit 185. and an entropy encoder 190. The inter prediction unit 180 and the intra prediction unit 185 may be collectively referred to as a “prediction unit.” Converter 120, quantizer 130, dequantizer 140, and inverse converter 150 may be included in the residual processor. The residual processor may further include a subtractor 115.

[78] All or at least a portion of the plurality of components configuring image encoding equipment 100 may be configured by a single hardware component (eg, an encoder or processor) in some embodiments. In addition, storage device 170 may include a decoded frame buffer (DPB) and may be configured with a digital storage medium.

[79] The image segmentation module 110 may segment an input image (either a frame or a movie frame) input to the image encoding equipment 100 into one more processing unit. For example, the processing unit may be called a "coding unit (CU)". A coding unit may be obtained by recursively segmenting a coding tree unit (CTU) or largest coding unit (LCU) according to a quadtree, binary tree, and ternary tree (QT/BT/TT) structure. For example, one coding unit may be segmented into multiple coding units of greater depth based on a quadtree structure, a binary tree structure, and/or a ternary structure. To segment a coding unit, a quadtree structure may first be applied, and subsequently a binary tree structure and/or a ternary structure may be applied. The encoding procedure of the present disclosure may be performed based on a final encoding unit that is no longer segmented. The largest coding unit may be used as the final coding unit, or a coding unit of greater depth obtained by segmenting the largest coding unit may be used as the final coding unit. Here, the encoding procedure may include a prediction, transformation and reconstruction procedure, which is described below. As another example, the processing unit of the encoding procedure may be a prediction unit (PU) or a transformation unit (TU). The prediction unit and the transformation unit may be split or segmented from the final encoding unit. The prediction unit may be a sample prediction unit, and the transform unit may be a unit for extracting a transform coefficient and/or a unit for extracting a residual signal from the transform coefficient.

[80] A prediction module (inter prediction module 180 or intra prediction module 185) may perform a prediction regarding a block to be processed (the current block) and generate a predicted block including prediction samples for the current block. The prediction module may determine whether intra prediction or inter prediction is applied based on the current block or CU. The prediction module may generate various information related to the prediction of the current block and transmit the generated information to the entropy encoder 190. Information regarding the prediction may be encoded in the entropy encoder 190 and output in the form of a bit stream.

[81] Intra prediction module 185 may predict the current block by reference to samples in the current frame. The reference samples may be located in the surroundings of the current block or may be spaced apart according to an intra prediction mode and/or intra prediction technology. The intra prediction modes may include a plurality of non-directional modes and a plurality of directed modes. The omnidirectional mode may include, for example, DC mode and planar mode. The directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes according to the granularity of the prediction direction. However, this is merely an example; more or less directional prediction modes may be used depending on the setting. The intra prediction unit 185 may determine the prediction mode applied to the current block by using the prediction mode applied to the adjacent block.

[82] The inter prediction unit 180 may extract a prediction block for the current block based on a reference block (an array of reference samples) indicated by a motion vector for the reference frame. In this case, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information may be predicted in units of blocks, sub-blocks or samples based on the correlation of motion information between a neighboring block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction direction information (L0 prediction, L1 prediction or bi-prediction, etc.). In the case of mutual prediction, the neighbor block may include a spatial neighbor block present in the current frame and a temporal neighbor block present in the reference frame. The reference frame including the reference block and the reference frame including the temporary adjacent block may be identical or different. A temporary neighbor block may be referred to as a "co-located reference block", "co-located CU (colCU)", or the like. A reference frame including a temporary adjacent block may be referred to as a "co-located frame (colPic)". For example, the inter-prediction module 180 may configure a list of motion information candidates based on neighboring blocks and generate information indicating which candidate is used to extract the motion vector and/or reference frame index of the current block. Mutual prediction can be performed based on different prediction modes. For example, in the case of the skip mode and the combine mode, the mutual prediction unit 180 may use the motion information of the adjacent block as the motion information of the current block. In the case of the skip mode, unlike the combining mode, the residual signal may not be transmitted. In the case of the motion vector prediction (MVP) mode, the motion vector of an adjacent block can be used as a motion vector predictor, and the motion vector of the current block can be signaled by encoding the motion vector difference and an indicator for the motion vector predictor. The motion vector difference may mean the difference between the motion vector of the current block and the motion vector predictor.

[83] The prediction module can generate a prediction signal based on various prediction methods and prediction technologies described below. For example, the prediction module may not only apply intra prediction or inter prediction, but also simultaneously apply both intra prediction and inter prediction to predict the current block. A prediction method that simultaneously uses both intra prediction and inter prediction to predict the current block may be called "combined inter and intra prediction (CIIP)". In addition, the prediction module can perform intra-block copy (IBC) to predict the current block. In-block copying can be used to encode game image/video content and the like, such as screen content encoding (SCC). IBC is a method of predicting the current frame using a previously reconstructed reference block in the current frame at a location spaced from the current block by a predetermined distance. When IBC is applied, the location of the reference block in the current frame can be encoded as a vector (block vector) corresponding to a predetermined distance.

[84] The prediction signal generated by the prediction unit can be used to generate a reconstructed signal or generate a residual signal. The subtractor 115 can generate a residual signal (residual block or residual sample array) by subtracting a prediction signal (prediction block or prediction sample array) output from the prediction module from an input image signal (original block or source sample array). The generated residual signal can be transmitted to the converter 120.

[85] Converter 120 can generate conversion coefficients by applying conversion technology to the residual signal. For example, the transform technology may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), a graph-based transform (GBT), or a conditionally nonlinear transform (CNT). ). Here, GBT means a graph-derived transformation when the relationship information between pixels is represented by the graph. CNT means a transform obtained from a prediction signal generated using all previously reconstructed pixels. In addition, the conversion process may be applied to square pixel blocks that are identical in size, or may be applied to blocks that are of variable size rather than square ones.

[86] Quantizer 130 may quantize the transform coefficients and pass them to entropy encoder 190. Entropy encoder 190 may encode the quantized signal (information regarding the quantized transform coefficients) and output a bit stream. Information regarding the quantized transform coefficients may be referred to as "residual information". Quantizer 130 may re-arrange the quantized transform coefficients in block form into one-dimensional vector form based on the scan order of the coefficients, and generate information regarding the quantized transform coefficients based on the quantized transform coefficients in one-dimensional vector form.

[87] Entropy encoder 190 may implement various encoding techniques, such as, for example, exponential Golomb coding, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), and the like. Entropy encoder 190 may encode information needed to reconstruct video/images other than quantized transform coefficients (eg, syntax element values, etc.), together or separately. Encoded information (eg, encoded video/image information) may be transmitted or stored in network abstraction layer (NAL) units in the form of a bit stream. The video/image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. The signaling information, transmission information, and/or syntax elements described in the present disclosure may be encoded through the encoding procedure described above and included in the bitstream.

[88] The bitstream may be transmitted over a network or may be stored on a digital storage medium. The network may include a broadcast network and/or a communications network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-Ray, HDD, SSD, and the like. A transmitter (not shown) transmitting the signal output from the entropy encoder 190 and/or a storage module (not shown) storing the signal may be included as an internal/external element of the encoding equipment 100, and alternatively, the transmitter may be included in the entropy encoder 190 .

[89] The quantized transform coefficients output from quantizer 130 can be used to generate a residual signal. For example, a residual signal (residual block or residual samples) may be recovered by applying dequantization and inverse transform to the quantized transform coefficients via dequantizer 140 and inverse transformer 150.

[90] An adder 155 adds the reconstructed residual signal with the prediction signal output from the inter prediction unit 180 or the intra prediction unit 185 so as to generate a reconstructed signal (reconstructed frame, reconstructed block, reconstructed sample array). If there is no remainder for a block to be processed, for example in a case in which the skip mode is applied, the prediction block can be used as the reconstructed block. The adder 155 may be referred to as a "recovery module" or a "recovered block generator." The generated reconstructed signal can be used for intra-prediction of the next block to be processed in the current frame, and can be used for inter-prediction of the next frame through filtering, as described below.

[91] Meanwhile, as described below, luma signal conversion with chrominance signal scaling (MCS) is applicable in the frame encoding process.

[92] Filter 160 may enhance the subjective/objective quality of images by applying filtering to the reconstructed signal. For example, filter 160 may generate a modified reconstructed frame by applying various filtering techniques to the reconstructed frame and store the modified reconstructed frame in storage device 170, namely, a DPB of storage device 170. Various filtering techniques may include, for example, deblocking filtering. , sampled adaptive bias, adaptive loop filter, bilateral filter, etc. The filter 160 may generate various filtering-related information and transmit the generated information to the entropy encoder 190, as described below in the description of each filtering method. Information associated with filtering may be encoded by entropy encoder 190 and output in the form of a bit stream.

[93] The modified reconstructed frame transferred to the storage device 170 can be used as a reference frame in the inter prediction unit 180. When mutual prediction is applied through the image encoding equipment 100, prediction mismatch between the image encoding equipment 100 and the image decoding equipment can be eliminated, and encoding efficiency can be improved.

[94] The DPB of the storage device 170 may store the modified reconstructed frame for use as a reference frame in the inter prediction module 180. The storage device 170 may store block motion information from which motion information in the current frame is retrieved (or encoded) and/or motion information of blocks in a frame that have already been reconstructed. The stored motion information may be transmitted to the inter-prediction unit 180 and used as spatial neighbor motion information or temporal neighbor motion information. Memory 170 may store reconstructed samples of reconstructed blocks in the current frame and may transmit reconstructed samples to intra prediction module 185.

[95] General introduction of image decoding equipment

[96] FIG. 3 is a view schematically showing image decoding equipment to which an embodiment of the present disclosure is applicable.

[97] As shown in FIG. 3, the image decoding equipment 200 may include an entropy decoder 210, a dequantizer 220, an inverse converter 230, an adder 235, a filter 240, a memory 250, an inter prediction unit 260, and an intra prediction unit 265. The inter prediction module 260 and the intra prediction module 265 may be collectively referred to as a “prediction module”. A dequantizer 220 and an inverse converter 230 may be included in the residual processor.

[98] All or at least a portion of the plurality of components configuring the image decoding equipment 200 may be configured by a hardware component (eg, a decoder or processor) according to an embodiment. In addition, storage device 250 may include a decoded frame buffer (DPB) or may be configured with a digital storage medium.

[99] The image decoding equipment 200, which receives the bit stream including video/image information, can reconstruct the image by performing a process corresponding to the process performed by the image encoding equipment 100 in FIG. 2. For example, the image decoding equipment 200 may perform decoding using a processing unit used in the image encoding equipment. Thus, a decoding processing unit, for example, may be an encoding unit. A coding unit may be obtained by segmenting a coding tree unit or a largest coding unit. The reconstructed image signal decoded and output by the decoding equipment 200 can be reproduced by the playback equipment.

[100] The image decoding equipment 200 may receive a signal output from the image encoding equipment of FIG. 2 in the form of a bitstream. The received signal may be decoded through entropy decoder 210. For example, entropy decoder 210 may parse the bitstream to extract information (eg, video/image information) needed to reconstruct images (or reconstruct frames). The video/image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. The image decoding equipment may further decode the frame based on information regarding the parameter set and/or general constraint information. The signaling/receiving information and/or syntax elements described in the present disclosure may be decoded by a decoding procedure and obtained from the bit stream. For example, entropy decoder 210 decodes information in a bitstream based on an encoding method, such as exponential Golomb, CAVLC, or CABAC encoding, and the output values of the syntax elements required for image reconstruction and the quantized values of the transform coefficients for the remainder. More specifically, the entropy CABAC decoding method may take a resolution element corresponding to each syntax element in the bitstream, determine a context model using information of a target decoding syntax element, decoding information of an adjacent block and a target decoding block, or information of a symbol/resolution element decoded on a previous one. stage, and perform arithmetic decoding for the resolution element by predicting the probability of occurrence of the permission element according to the determined context model and generating a symbol corresponding to the meaning of each syntactic element. In this case, the entropy CABAC decoding method can update the context model by using the decoded symbol/grant element information for the context model of the next symbol/grant element after determining the context model. The prediction related information from the information decoded by the entropy decoder 210 may be provided to the prediction module (inter prediction module 260 and intra prediction module 265), and the residual value for which entropy decoding is performed in the entropy decoder 210, i.e. quantized conversion coefficients and related parameter information may be input to the dequantizer 220. In addition, information regarding filtering from information decoded by the entropy decoder 210 may be provided to the filter 240. Meanwhile, a receiver (not shown) for receiving a signal output from the equipment image encoding device may be further configured as an internal/external element of image decoding equipment 200, or the receiver may be a component of entropy decoder 210.

[101] Meanwhile, the image decoding equipment according to the present disclosure may be called "video/image/frame decoding equipment". The image decoding equipment can be classified into an information decoder (video/image/frame information decoder) and a sample decoder (video/image/frame sample decoder). The information decoder may include an entropy decoder 210. The sample decoder may include at least one of a dequantizer 220, an inverse converter 230, an adder 235, a filter 240, a memory 250, an inter-prediction module 160, or an intra-prediction module 265.

[102] Dequantizer 220 may dequantize the quantized transform coefficients and output the transform coefficients. Dequantizer 220 may re-arrange the quantized transform coefficients in the form of a two-dimensional block. In this case, rearrangement may be performed based on the order of coefficient scanning performed in the image encoding equipment. Dequantizer 220 may perform dequantization on the quantized transform coefficients by using a quantization parameter (eg, quantization step size information) and obtain the transform coefficients.

[103] The inverse converter 230 may inversely convert the transform coefficients to obtain a residual signal (residual block, residual sample array).

[104] Prediction module 230 may perform prediction for the current block and may generate a prediction block including prediction samples for the current block. The prediction module can determine whether intra prediction or inter prediction is applied to the current block based on the prediction information output from the entropy decoder 210, and can determine a particular intra/inter prediction mode (prediction technology).

[105] Identical to that described in the prediction module of the image encoding equipment 100, the prediction module can generate a prediction signal based on various prediction methods (techniques), which are described below.

[106] Intra prediction module 265 may predict the current block by reference to samples in the current frame. The description of the intra prediction module 185 applies equally to the intra prediction module 265.

[107] The inter prediction unit 260 may extract a prediction block for the current block based on a reference block (an array of reference samples) indicated by a motion vector for the reference frame. In this case, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information may be predicted in units of blocks, sub-blocks or samples based on the correlation of the motion information between a neighboring block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction direction information (L0 prediction, L1 prediction or bi-prediction, etc.). In the case of mutual prediction, the neighbor block may include a spatial neighbor block present in the current frame and a temporal neighbor block present in the reference frame. For example, the inter-prediction module 260 may configure a motion information candidate list based on neighboring blocks and extract the current block's motion vector and/or reference frame index based on the received candidate selection information. Inter prediction may be performed based on various prediction modes, and information regarding the prediction may include information indicating the inter prediction mode for the current block.

[108] The adder 235 can generate a reconstructed signal (recovered frame, reconstructed block, reconstructed sample array) by adding the resulting residual signal with a prediction signal (prediction block, prediction sample array) output from a prediction module (including inter-prediction module 260 and /or internal prediction module 265). The description of adder 155 is equally applicable to adder 235.

[109] Meanwhile, as described below, luma signal conversion with chrominance signal scaling (MCS) is applicable in the frame decoding process.

[110] Filter 240 may enhance the subjective/objective quality of images by applying filtering to the reconstructed signal. For example, filter 240 may generate a modified reconstructed frame by applying various filtering techniques to the reconstructed frame and store the modified reconstructed frame in storage device 250, namely, a DPB of storage device 250. Various filtering techniques may include, for example, deblocking filtering. , sampled adaptive bias, adaptive loop filter, bilateral filter, etc.

[111] The (modified) reconstructed frame stored in the DPB of the storage device 250 can be used as a reference frame in the inter prediction unit 260. Memory 250 may store block motion information from which motion information in the current frame is retrieved (or decoded) and/or motion information of blocks in a frame that have already been reconstructed. The stored motion information may be transmitted to the inter-prediction unit 260 so that it is used as spatial neighbor block motion information or temporal neighbor block motion information. Memory 250 may store reconstructed samples of reconstructed blocks in the current frame and transmit the reconstructed samples to intra prediction module 265.

[112] In the present disclosure, the embodiments described in the filter 160, the inter prediction unit 180, and the intra prediction unit 185 of the image encoding equipment 100 may be equally or correspondingly applied to the filter 240, the inter prediction unit 260, and the intra prediction unit 265 image decoding equipment 200.

[113] General presentation of intra prediction

[114] Hereinafter, intra prediction according to an embodiment is described.

[115] The intra prediction may indicate a prediction that generates prediction samples for the current block based on reference samples in the frame to which the current block belongs (hereinafter referred to as the “current frame”). When intra prediction is applied to the current block, neighboring reference samples that should be used for intra prediction of the current block can be retrieved. The current block's adjacent reference samples may include a sample adjacent to the left boundary of the current block having size nWxnH, and the sum of 2xnH samples adjacent to the bottom left, a sample adjacent to the top boundary of the current block, and a sum of 2xnW samples adjacent to the right the top part, and one selection adjacent to the top left part relative to the current block. Alternatively, the current block's neighboring reference samples may include a plurality of columns of top neighboring samples and a plurality of rows of left neighboring samples. In addition, the adjacent reference samples of the current block may include a sum of nH samples adjacent to the right boundary of the current block having size nWxnH, a sum of nW samples adjacent to the bottom boundary of the current block, and one sample adjacent to the bottom right of the current block.

[116] Some adjacent reference samples of the current block have not yet been decoded or may not be available. In this case, the decoder can construct neighboring reference samples to be used for prediction by substituting available samples for unavailable samples. Alternatively, neighboring reference samples to be used for prediction can be constructed using interpolation of the available samples.

[117] When neighboring reference samples are retrieved, (i) the prediction sample may be retrieved based on the average or interpolation of the current block's adjacent reference samples, and (ii) the prediction sample may be retrieved based on the reference sample present in a particular direction (prediction) relative to the prediction sample from neighboring reference samples of the current block. Case (i) may be referred to as "non-directional mode" or "non-angular mode", and case (ii) may be referred to as "directional mode" or "angular mode".

[118] In addition, the prediction sample may be generated through interpolation with a first adjacent sample located in the intra prediction mode prediction direction of the current block and a second adjacent sample located in the opposite direction based on the current block's prediction target sample from the adjacent reference samples. The above case may be called "linear interpolation intra prediction (LIP)".

[119] In addition, chrominance signal prediction samples can be generated from the luminance signal samples using a linear model. This case may be called "linear model (LM) mode".

[120] In addition, the prediction time sample of the current block may be extracted based on the filtered neighboring reference samples, and the prediction sample of the current block may be extracted by weighting summation of the prediction time sample and at least one reference sample extracted according to an intra prediction mode from existing neighboring reference samples, i.e. unfiltered neighboring reference samples. This case may be called "position-dependent intraprediction (PDPC)".

[121] In addition, the reference sample line with the highest prediction accuracy may be selected from several adjacent reference sample lines of the current block to extract a prediction sample using a reference sample located in the prediction direction in the corresponding line, and, at this time, information (eg , intra_luma_ref_idx) relative to the reference sample line used may be encoded and signaled in the bit stream. This case may be called "multiple reference line (MRL) intra-prediction" or "MRL-based intra-prediction".

[122] In addition, the current block may be divided into vertical or horizontal sub-segments to perform intra prediction on each sub-segment based on an identical intra prediction mode. At this time, neighboring intra prediction reference samples can be retrieved in units of sub-segments. Thus, the reconstructed sample of the previous subsegment in encoding/decoding order can be used as an adjacent reference sample of the current subsegment. In this case, the intra prediction mode for the current block is identically applied to the sub-segments, and adjacent reference samples are retrieved and used in the units of the sub-segments, thereby improving the intra prediction performance. Such a prediction method may be called "intra subsegment-based prediction (ISP) or ISP-based".

[123] The intra prediction technology may be referred to by various terms such as "intra prediction type" or "additional intra prediction mode" so as to distinguish it from a directed or undirected intra prediction mode. For example, an intra prediction technology (intra prediction type or extra intra prediction mode) may include at least one of LIP, LM, PDPC, MRL, ISP, or MIP. Meanwhile, if necessary, post-filtering can be performed on the extracted prediction sample.

[124] Specifically, the intra prediction procedure may include a step of determining an intra prediction mode/type, a step of extracting a neighboring reference sample, and a step of extracting prediction samples based on the intra prediction mode/type. In addition, if necessary, post-filtering can be performed on the extracted prediction sample.

[125] FIG. 4 is a flowchart illustrating a video/image encoding method based on intra prediction.

[126] The encoding method of FIG. 4 may be implemented by the image encoding equipment of FIG. 2. Specifically, step S410 may be performed by the intra prediction unit 185, and step S420 may be performed by the residual processor. Specifically, step S420 may be performed by the subtractor 115. Step S430 may be performed by the entropy encoder 190. The prediction information of step S430 may be extracted by the intra prediction unit 185, and the residual information of step S430 may be extracted by the residual processor. Residual information is information regarding the residual samples. The residual information may include information regarding a quantized transform coefficient for the residual samples. As described above, the residual samples can be extracted as a transform coefficient through the image encoding equipment converter 120, and the transform coefficient can be extracted as transform coefficients quantized through the quantizer 130. Information regarding the quantized transform coefficients can be encoded by the entropy encoder 190 through a residual encoding procedure.

[127] The image encoding equipment may perform intra prediction regarding the current block (S410). The image encoding equipment may determine the intra prediction mode/type for the current block, extract adjacent reference samples of the current block, and generate prediction samples in the current block based on the intra prediction mode/type and the adjacent reference samples. Here, the procedures for determining the intra prediction mode/type, extracting neighboring reference samples, and generating prediction samples may be simultaneously performed, or any procedure may be performed before other procedures.

[128] FIG. 5 is a view illustrating the configuration of the intra prediction unit 185 according to the present disclosure.

[129] As shown in FIG. 5, the intra prediction unit 185 of the image encoding equipment may include an intra prediction mode/type determining unit 186, a reference sample extracting unit 187, and/or a prediction sample extracting unit 188. The intra prediction mode/type determining unit 186 may determine the intra prediction mode/type for the current block. The reference sample extractor 187 may retrieve adjacent reference samples of the current block. The prediction sample extracting unit 188 may extract prediction samples of the current block. Meanwhile, although not shown, when the prediction sample filtering procedure described below is performed, the intra prediction unit 185 may further include a prediction sample filter (not shown).

[130] The image encoding equipment may determine a mode/type applied to the current block from a plurality of intra prediction modes/types. The image encoding equipment may compare the distortion cost function as a function of the bit rate (RD) for the intra prediction modes/types and determine the optimal intra prediction mode/type for the current block.

[131] Meanwhile, the image encoding equipment may perform a filtering procedure on the prediction samples. Filtering the prediction samples may be called "post-filtering". Through the prediction sample filtering procedure, some or all of the prediction samples may be filtered. In some cases, the procedure for filtering prediction samples may be omitted.

[132] Referring to FIG. 4 again, the image encoding equipment may generate residual samples for the current block based on the prediction samples or filtered prediction samples (S420). The image encoding equipment may extract residual samples by subtracting prediction samples from the original samples of the current block. Thus, the image encoding equipment can extract residual sample values by subtracting the corresponding prediction sample value from the original sample value.

[133] Image encoding equipment can encode image information including information regarding intra prediction (prediction information) and residual information of residual samples. The prediction information may include intra prediction mode information and/or intra prediction technology information. The image encoding equipment 100 may output encoded image information in the form of a bitstream. The output bit stream may be transmitted to image decoding equipment via a storage medium or a network.

[134] The residual information may include residual encoding syntax, which is described below. Image encoding hardware can transform/quantize the residual samples and extract the quantized transform coefficients. The residual information may include information regarding the quantized transform coefficients.

[135] Meanwhile, as described above, the image encoding equipment can generate a reconstructed frame (including reconstructed samples and a reconstructed block). To this end, image encoding equipment may dequantize/deconvert the quantized transform coefficients and extract the (modified) residual samples. The reason for transforming/quantizing the residual samples and then performing dequantization/de-transforming is to extract residual samples identical to the residual samples extracted by the image decoding equipment. The image encoding equipment may generate a reconstructed block including reconstructed samples for the current block based on the prediction samples and the (modified) residual samples. Based on the reconstructed block, a reconstructed frame for the current frame can be generated. As described above, an in-loop filtering procedure may further be applied to the reconstructed frame.

[136] FIG. 6 is a flowchart illustrating a video/image decoding method based on intra prediction.

[137] The image decoding equipment may perform an operation corresponding to the operation performed by the image encoding equipment.

[138] The decoding method of FIG. 6 may be performed by the image decoding equipment of FIG. 3. Steps S610 to S630 may be performed by the intra prediction unit 265, and the prediction information of step S610 and the residual information of step S640 may be obtained from the bitstream by the entropy decoder 210. A residual processor of the image decoding equipment may extract residual samples for the current block based on the residual information. (S640). In particular, the residual processor dequantizer 220 may perform dequantization based on the dequantized transform coefficients extracted based on the residual information to extract transform coefficients, and the residual processor inverter 230 may perform an inverse transform on the transform coefficients to extract residual samples for the current block. Step S650 may be performed by the adder 235 or the recovery module.

[139] Specifically, the image decoding equipment can extract the intra prediction mode/type for the current block based on the received prediction information (intra prediction mode/type information) (S610). The image decoding equipment may extract adjacent reference samples of the current block (S620). The image decoding equipment may generate prediction samples in the current block based on the intra prediction mode/type and adjacent reference samples (S630). In this case, the image decoding equipment may perform a filtering procedure on the prediction samples. Filtering the prediction samples may be called "post-filtering". Through the prediction sample filtering procedure, some or all of the prediction samples may be filtered. In some cases, the procedure for filtering prediction samples may be omitted.

[140] The image decoding equipment may generate residual samples for the current block based on the received residual information (S640). The image decoding equipment can generate reconstructed samples for the current block based on the prediction samples and residual samples, and extract the reconstructed block including the reconstructed samples (S650). Based on the reconstructed block, a reconstructed frame for the current frame can be generated. An in-loop filtering procedure may additionally be applied to the reconstructed frame as described above.

[141] FIG. 7 is a view illustrating the configuration of the intra prediction module 265 according to the present disclosure.

[142] As shown in FIG. 7, the intra prediction unit 265 of the image decoding equipment may include an intra prediction mode/type determining unit 266, a reference sample extracting unit 267, and a prediction sample extracting unit 268. The intra prediction mode/type determining unit 266 can determine the intra prediction mode/type for the current block based on the intra prediction mode/type information generated and signaled by the intra prediction mode/type determining unit 186 of the image encoding equipment and the extraction unit 267 reference samples can retrieve adjacent reference samples of the current block from the reconstructed reference region in the current frame. The prediction sample extractor 268 may retrieve prediction samples of the current block. Meanwhile, although not shown, when the above-described prediction sample filtering procedure is performed, the intra prediction unit 265 may further include a prediction sample filter (not shown).

[143] The intra prediction mode information may include, for example, flag information (e.g., intra_luma_mpm_flag) indicating whether or not the most probable mode (MPM) or remaining mode should be applied to the current block, and when the MPM is applied to the current block. the intra prediction mode information may further include index information (eg, intra_luma_mpm_idx) indicating one of the intra prediction mode options (MPM options). Possible options for intra prediction modes (possible MPM options) may consist of a list of possible MPM options or an MPM list. In addition, when MPM is not applied to the current block, the intra prediction mode information may further include remaining mode information (e.g., intra_luma_mpm_remainder) indicating one of the remaining intra prediction modes excluding the intra prediction mode candidates (MPM candidates) . The image decoding equipment may determine the intra prediction mode of the current block based on the intra prediction mode information. Possible MPM mode options may include intra-prediction modes of neighbor blocks (eg, left neighbor block and top neighbor block) relative to the current block and additional possible mode options.

[144] FIG. 8a shows an intra prediction direction according to an embodiment of the present disclosure. To capture any edge direction presented in a natural video, as shown in FIG. 8a, the intra prediction mode may include two undirected intra prediction modes and 65 directed intra prediction modes. The undirected intra prediction modes may include a planar intra prediction mode (planar mode) and an intra DC mode (DC mode), and the directed intra prediction modes may include the second to 66th intra prediction modes.

[145] Meanwhile, the intra prediction mode may further include a cross-component linear model (CCLM) based mode for chrominance signal samples in addition to the above-described intra prediction modes. The CCLM mode can be split into L_CCLM, T_CCLM, LT_CCLM according to whether left samples, top samples or both are considered or not to extract the LM parameter, and can only be applied to the chrominance signal component.

[146] For example, the intra prediction mode, for example, may be indexed as shown in the table below.

[147] Tab. 1

Intra prediction mode Associated title 0 INTRA_PLANAR 1 INTRA_DC 2...66 INTRA_ANGULAR2...INTRA_ANGULAR66 81...83 INTRA_LT_CCLM, INTRA_L_CCLM, INTRA_T_CCLM

[148] FIG. 8b shows an intra prediction direction according to another embodiment of the present disclosure. In fig. 8b, the direction along the dotted line shows the wide-angle mode applied only to the non-square block. As shown in FIG. 8b, in order to capture any edge direction present in natural video, the intra prediction mode according to the embodiment may include two non-directional intra prediction modes and 93 directed intra prediction modes. The undirected intra prediction modes may include a planar mode and a DC mode, and the directed intra prediction modes may include the second to 80th and -1st to -14th intra prediction modes, as indicated by an arrow in FIG. 8b. Planar mode may be designated by INTRA_PLANAR, and DC mode may be designated by INTRA_DC. In addition, the directed intra prediction mode may be designated by INTRA_ANGULAR-14 - INTRA_ANGULAR-1 and INTRA_ANGULAR2 - INTRA_ANGULAR80. In addition, intra prediction technology information can be implemented in various forms. For example, the intra prediction technology information may include intra prediction type index information indicating one of the intra prediction technologies. As another example, the intra prediction technology information may include at least one of reference sample line information (eg, intra_luma_ref_idx) indicating whether or not the MRL should be applied to the current block, and if so, which line reference samples are used, ISP flag information (eg, intra_subpartitions_mode_flag) indicating whether or not ISP should be applied to the current block, ISP type information (eg, intra_subpartitions_split_flag) indicating the type of subsegment split when applying ISP, flag information indicating whether whether PDPC should be applied or not, or flag information indicating whether LIP should be applied or not. In the present disclosure, the ISP flag information may be referred to as an “ISP application indicator”.

[149] The intra prediction mode information and/or the intra prediction technology information may be encoded/decoded through the encoding method described in the present disclosure. For example, intra prediction mode information and/or intra prediction technology information may be encoded/decoded by entropy coding (eg, CABAC, CAVLC) based on a truncated binary code (Rice).

[150] Before determining whether or not a predetermined intra prediction technology should be applied to the current block, it may first be determined whether or not a corresponding intra prediction technology is available for the current block. For example, whether or not a corresponding intra prediction technology is available for the current block may be determined based on the encoding parameters of the current block. In this case, the encoding parameters may include the size (width and/or height) of the current block, the position of the current block, the color component of the current block, or whether or not another intra prediction technology should be used.

[151] In addition, determination as to whether or not a corresponding intra prediction technology is available for the current block can be made based on information signaled in the upper layer of the current block, such as sequence, frame, slice, and CTU. For example, when information transmitted at the sequence level indicates that a predetermined intra prediction technology is not available, it may be determined that the corresponding intra prediction technology is not available for blocks belonging to the corresponding sequence.

[152] When determining that a predetermined intra prediction technology is available for the current block, the image encoding equipment may determine whether or not to apply the corresponding intra prediction technology to the current block using various methods. For example, image encoding equipment may determine whether or not to apply a corresponding intra prediction technology based on RDO.

[153] Image encoding equipment may encode and signal intra prediction technology information in a bit stream. The intra prediction technology information may be encoded according to the corresponding intra prediction technology in various forms. For example, when the intra prediction technology is MRL, the intra prediction technology information may be an index (eg, intra_luma_ref_idx) indicating a reference line used to predict the current block among multiple reference lines. When the intra prediction technology is an ISP, the intra prediction technology information may be flag information (eg, intra_subpartitions_mode_flag) indicating whether or not the ISP should be applied to the current block. In addition, when using an ISP, the intra prediction technology information may further include information (eg, intra_subpartitions_split_flag) regarding the split direction. In addition, the intra prediction technology information may include flag information indicating whether or not PDPC should be applied, flag information indicating whether or not LIP should be applied, or flag information indicating whether or not LM mode should be applied.

[154] In case it is determined that a predetermined intra prediction technology is available for the current block, the image decoding equipment can determine whether or not to apply the corresponding intra prediction technology to the current block based on the intra prediction technology information signaled.

[155] As another example, whether or not a predetermined intra prediction technology should be applied to the current block may be implicitly retrieved by the image encoding equipment and image decoding equipment based on the encoding parameters for the current block, in addition to that explicitly signaled information of internal prediction technology. In this case, the encoding parameter may include the size (width and/or height) of the current block, the position of the current block, the color component of the current block, or whether or not another intra prediction technology should be used.

[156] The ISP mode according to the present disclosure is described in detail later in this document.

[157] The current block may include a luma component block (array) and a chrominance component block (array) corresponding to it. In the present disclosure, "current block" or "luminance component block" may mean "luminance component block of the current block", and "chrominance component block" or "corresponding chrominance component block" may mean "current chrominance component block". block". In addition, a "luminance signal component block" may be referred to by the term "luminance signal block", "luminance component block" or "luminance block", and a "chrominance signal component block" may be referred to by the term "chrominance signal block", " chrominance component block" or "chrominance block".

[158] In traditional intra prediction, the current target encoding/decoding block (current block) is considered as one unit, and therefore encoding/decoding is performed without partitioning. However, when ISP mode is applied, the current block is split in the horizontal or vertical direction to perform internal predictive encoding/decoding. In this case, encoding/decoding is performed in units of broken sub-segments to form a reconstructed sub-segment, and the reconstructed sub-segment is used as a reference block of the next broken sub-segment.

[159] Whether or not an ISP is available for the current block may be determined based on the following conditions of Table 2. The following conditions may be determined based on the luminance signal component block of the current block. Thus, in the following conditions, the width, height and position of the current block may mean the width, height and position of the block of luminance signal components of the current block, respectively.

[160] Tab. 2

ISP Availability Terms
intra_luma_ref_idx[x0][y0]==0
cbWidth<=MaxTbSizeY || cbHeight<=MaxTbSizeY
cbWidth*cbHeight>MinTbSizeY*MinTbSizeY

[161] For example, when all of the above conditions for the current block are satisfied, it may be determined that the ISP is available for the current block. After determining that the ISP is available for the current block, it can be determined whether or not the ISP should be applied to the current block. Under ISP accessibility, (x0, y0) are coordinates indicating the location of the top-left sample of the current block. In addition, intra_luma_ref_idx[x0][y0] is information indicating a reference line used to predict the current block. According to ISP availability conditions, when intra_luma_ref_idx is equal to 0, i.e. when the reference line used to predict the current block is a line is line 0 (a reference line immediately adjacent to the current block), it can be determined that the ISP is available for the current block. When the reference line used to predict the current block is a line other than line 0, it may be determined that the ISP is not available for the current block.

[162] Under ISP accessibility, cbWidth and cbHeight correspond to the width and height of the current block, respectively. In addition, MaxTbSizeY and MinTbSizeY may indicate the maximum transform size and minimum transform size, respectively. As described above, the residual processing may include transforming or de-transforming. In this case, the size of the transform block in which conversion or demapping is available may be predefined or signaled via a bitstream. Thus, the maximum transform size means the maximum transform block size at which a transform or reverse transform can be performed. In addition, the minimum transform size means the minimum transform block size at which a transform or reverse transform can be performed. For example, when the current block size exceeds the maximum transform size, the current block may be split into two or more transform blocks. In addition, the current block may not be split into transform blocks that are smaller than the minimum transform size. The maximum transform size and/or the minimum transform size may be preset in the image encoding equipment and the image decoding equipment, or may be extracted based on information signaled in the upper layer of the block.

[163] According to the ISP availability conditions, only when at least one of cbWidth or cbHeight is equal to or less than MaxTbSizeY can it be determined that the ISP is available for the current block. Thus, when cbWidth and cbHeight exceed MaxTbSizeY, it may be determined that the ISP is not available for the current block. When cbWidth is greater than MaxTbSizeY, cbHeight is equal to or less than MaxTbSizeY, and the ISP is applied to the current block, the ISP split direction may be determined as the vertical direction, as described below. When cbHeight is greater than MaxTbSizeY, cbWidth is equal to or less than MaxTbSizeY, and the ISP is applied to the current block, the ISP split direction may be determined as the horizontal direction, as described below.

[164] According to the ISP availability conditions, when cbWidth*cbHeight exceeds MinTbSizeY*MinTbSizeY, it may be determined that the ISP is available for the current block. cbWidth*cbHeight can mean the area of the current block or the number of samples included in the current block. For example, if MinTbSizeY is 4, it may be determined that the ISP is available for the current block only when the number of samples included in the current block exceeds 16 (4*4).

[165] FIG. 9 is a view illustrating a conventional method of performing encoding/decoding of a current block according to an ISP mode.

[166] When the current block to be encoded/decoded is input (S910), it can be determined whether or not ISP should be applied to the current block (S920). The determination of step S920 may include determining whether or not an ISP is available for the current block, and/or determining whether or not the ISP should be applied to the current block. Determination as to whether or not an ISP is available for the current block can be made based on the ISP availability conditions described above. When the ISP is available, the image encoding equipment can determine whether or not the ISP should be applied to the current block based on various methods as described above, and the result of the determination can be encoded in the bit stream as intra prediction technology information. When the ISP is available, the image decoding equipment can determine whether or not the ISP should be applied to the current block based on the intra prediction technology signaled information.

[167] When ISP is not applied to the current block, TU tiling can be performed relative to the current block (S930). TU tiling refers to the process of dividing the current block into multiple transform blocks such that the width and height of the current block are equal to or less than the maximum transform size, which is the transform size. As described above, residual signal processing includes transformation, and the maximum transformation size means the maximum size of a transformation block at which the transformation process can be performed. Accordingly, when the width or height of the current block exceeds the maximum transform size by splitting the current block through the TU tile arrangement, the width and height of the split block may be equal to or less than the maximum transform size. For example, when the maximum transform size is a size of 64 samples, and the current block has a size of 128×128, the current block may be divided into four 64×64 blocks. Alternatively, when the maximum transform size is a size of 64 samples, and the current block is 64x128 or 128x64, the current block may be split into two 64x64 blocks. For example, the maximum transform size is 64 samples, and the current block is 64x64 or smaller, TU tiling may not be performed.

[168] Thereafter, encoding/decoding may be performed with respect to the current block or each of the blocks divided through the TU tiling step (S960). The encoding of step S960 may include intra prediction, residual processing, and/or encoding of prediction information and residual information. The encoding of step S960 may include intra prediction, extracting residual samples, and/or generating reconstructed blocks.

[169] In step S920, when applying the ISP to the current block, the split direction and the number of splits (number of sub-segments) can be determined (S940).

[170] At step S920, the split direction may be extracted based on information (eg, intra_subpartitions_split_flag) signaled through the bit stream. Alternatively, the splitting direction may be implicitly derived based on the size of the current block. For example, as described above, when the width of the current block exceeds the maximum transform size, the ISP split direction may be extracted in the vertical direction. In addition, when the height of the current block exceeds the maximum transform size, the ISP split direction can be extracted in the horizontal direction. In addition, when the width and height of the current block exceed the maximum transform size, a limit may be imposed such that the ISP is not available for the current block. For example, when the maximum transform size is a size of 64 samples, and the current block to which the ISP is applied is a 128x64 block, the ISP split direction is extracted in the vertical direction, and the width (128/4) and height (64) of the sub-segment are defined as the maximum transform size (64) or less. Similarly, when the maximum transform size is a size of 64 samples, and the current block to which the ISP is applied is a 64x128 block, the ISP partitioning direction is extracted in the horizontal direction, and the width (64) and height (128/4) of the sub-segment are defined as the maximum transform size (64) or less. As described above, when applying ISP to the current block, the current block is split such that the width and height of the sub-segments are equal to or less than the maximum transform size. For this reason, the TU tile arrangement of step S930 need not necessarily be relative to the current block to which the ISP is applied.

[171] In step S940, the number of splits may be implicitly retrieved based on the size of the current block. In particular, the current block can be divided according to the size of the current block, as shown in Table 3.

[172] Tab. 3

Block size Number of splits 4x4 Without splitting 4x8, 8x4 2 All other cases 4

[173] FIG. 10a is a view illustrating an example of ISP partitioning for a 4x8 block or an 8x4 block. As shown in FIG. 10a, a 4x8 block or an 8x4 block may be divided into two sub-segments. When the current block is a 4x8 block and is split in the horizontal direction, the current block can be split into two 4x4 sub-segments. When the current block is a 4x8 block and is split in the vertical direction, the current block can be split into two 2x8 sub-segments. When the current block is an 8x4 block and is split in the horizontal direction, the current block can be split into two 8x2 sub-segments. When the current block is an 8x4 block and is split in the vertical direction, the current block can be split into two 4x4 sub-segments.

[174] FIG. 10b is a view illustrating an example of ISP partitioning for a block having a size of 8x8 or larger.

[175] As shown in FIG. 10b, a block having a size of 8x8 or larger may be divided into four sub-segments. When the current block is a WxH block, W and H are 8 or more, and the current block is split in the horizontal direction, the current block can be split into four Wx(H/4) sub-segments. When the current block is a WxH block, W and H are 8 or more, and the current block is split in the vertical direction, the current block can be split into four (W/4)xH sub-segments.

[176] Referring to FIG. 9 again, when the dividing direction and the number of divisions are determined in step S940, the luminance signal component block of the current block can be divided based on the specified (S950).

[177] Thereafter, encoding/decoding can be performed on each split sub-segment (S960). As described above, the encoding of step S960 may include intra prediction, residual processing, and/or encoding of prediction information and residual information. In addition, the decoding of step S960 may include intra prediction, extracting residual samples, and/or generating reconstructed blocks. In particular, when ISP is applied, the intra prediction mode for the current block is identically applied to the sub-segments, and the adjacent reference sample is retrieved and used in the units of the sub-segments, thereby improving the intra prediction performance. Thus, when using ISP, the procedure for processing residual samples is performed in units of subsegments. In other words, the internal prediction samples are extracted for each sub-segment, and the residual signal(s) for the corresponding sub-segment are added to it, thereby obtaining reconstructed samples. The residual signal (residual samples) may be extracted through a dequantization/deconversion procedure based on the residual information (quantized transform coefficient information or residual encoding syntax) in the above-described bit stream. Thus, extraction of prediction samples and extraction of residual samples for the first sub-segment can be performed, and reconstructed samples for the first sub-segment can be extracted based on the above. In this case, when retrieving prediction samples for the second sub-segment, some reconstructed samples in the first sub-segment (eg, reconstructed samples adjacent to the left or top side of the second sub-segment) may be used as adjacent reference samples for the second sub-segment. Likewise, extraction of prediction samples and extraction of residual samples for the second sub-segment may be extracted, and reconstructed samples for the second sub-segment may be extracted based on the above. In this case, when the prediction samples for the third sub-segment are retrieved, some reconstructed samples in the second sub-segment (eg, reconstructed samples adjacent to the left or top side of the third sub-segment) can be used as adjacent reference samples for the third sub-segment. Likewise, some of the reconstructed samples in the third subsegment can be used as adjacent reference samples for the fourth subsegment.

[178] The encoding/decoding order of a plurality of sub-segments is from top to bottom when the partitioning direction is a horizontal direction, and is from left to right when the partitioning direction is a vertical direction. For example, in FIG. 10b, when the partitioning direction is a horizontal direction, sub-segments can be sequentially encoded/decoded from the uppermost sub-segment to the lowermost sub-segment. In addition, when the partitioning direction is a vertical direction, sub-segments can be sequentially encoded/decoded from the leftmost sub-segment to the rightmost sub-segment.

[179] When applying ISP to the current block, in order to reduce coding complexity, an MPM list is generated according to each partitioning method (horizontal partitioning and vertical partitioning), and a suitable prediction mode from the prediction modes in the generated MPM list is compared in terms of distortion optimization in depending on the transmission speed (RDO) to form the optimal mode. In addition, when multiple reference line (MRL) based intra prediction is used, a limitation may be imposed such that the above-described ISP is not used. Thus, when the zero reference line is used (eg intra_luma_ref_idx==0), the ISP is applicable. In addition, when using ISP, a restriction may be imposed such that the above-described PDPC is not used. Thus, when using ISP, PDPC may not be used.

[180] When using ISP, as intra prediction technology information, information (intra_subpartitions_mode_flag) indicating whether or not ISP should be applied may be transmitted in units of current blocks, and if the current block uses ISP (for example, when intra_subpartitions_mode_flag is 1), information may be passed (intra_subpartitions_split_flag) regarding the splitting method (horizontal split or vertical split).

[181] According to the conventional method described with reference to FIG. 9, in a single tree structure in which the luma component block of the current block and the chrominance component block of the current block are split into an identical tree structure, when ISP is applied to the current block, the luma component block is split into a plurality of sub-segments, but ISP is not applied. to the chrominance signal component block. In this case, the width or height of the chrominance component block becomes larger than the maximum conversion size according to the chrominance signal format, and therefore conversion or deconversion of the chrominance component block may become impossible.

[182] Hereinafter, this document describes the relationship between the luma component block size and the chrominance component block size according to the chrominance signal format.

[183] FIG. 11 is a view illustrating the relationship between a luma component block (luminance signal component array) and a chrominance signal component block (chrominance signal component array) according to a chrominance signal format.

[184] The original or encoded frame/image may include a block of luma components (Y) and two blocks of chrominance components (Cb, Cr). Thus, one pixel of a frame/image may include a luma sample and two chrominance samples (Cb, Cr). The color format may represent a configuration format of the luma signal sample and the chrominance signal samples (Cb, Cr), and may be referred to as a “chrominance signal format.” The chrominance signal format may be predefined or may be adaptively signaled. For example, the chroma signal format may be signaled based on at least one of chroma_format_idc or separate_colour_plane_flag, as shown in Table 4.

[185] Tab. 4

chroma_format_idc separate_colour_plane_flag ChromaArrayType Color signal format SubWidthC SubHeightC 0 0 0 Monochrome 1 1 1 0 1 4:2:0 2 2 2 0 2 4:2:2 2 1 3 0 3 4:4:4 1 1 3 1 0 4:4:4 1 1

[186] In the above Table 4, chroma_format_idc is information indicating the format of a luminance signal sample and a chrominance signal sample corresponding thereto, and separate_colour_plane_flag is information indicating that the three color components Y, Cb and Cr are separately encoded in the chrominance signal format 4:4:4. In the above Table 4, when chroma_format_idc is 0, the chroma signal format is monochrome, and the current block does not include a chroma component block and only includes a luma component block.

[187] In the above Table 4, when chroma_format_idc is equal to 1, the chroma signal format corresponds to the 4:2:0 chroma signal format, and the width and height of the chroma component block appropriately correspond to half the width and half the height of the luma component block. Fig. 11a shows the location relationship between a luma signal sample and a chrominance signal sample in a 4:2:0 chrominance signal format.

[188] In the above Table 4, when chroma_format_idc is equal to 2, the chroma signal format corresponds to the 4:2:2 chroma signal format, the width of the chroma component block is suitably equal to half the width of the luma component block, and the height of the chroma component block is equal to the block height brightness signal components. Fig. 11b shows the location relationship between a luma signal sample and a chrominance signal sample in a 4:2:2 chrominance signal format.

[189] In the above Table 4, when chroma_format_idc is 3, the chroma signal format corresponds to the 4:4:4 chroma signal format, and the width and height of the chroma component block suitably correspond to the width and height of the luma component block. Fig. 11c shows the location relationship between a luma signal sample and a chrominance signal sample in a 4:4:4 chrominance signal format.

[190] In the above Table 4, SubWidthC and SubHeightC represent the ratio of the luma signal sample and the chrominance signal sample. For example, when the width and height of a luma component block are, respectively, CbWidth and CbHeight, the width and height of a chroma component block corresponding to it may be extracted as (CbWidth/SubwidthC) and (CbHeight/SubHeightC), respectively.

[191] As described with reference to FIG. 11, the size of a chrominance component block corresponding to a luma component block of the current block may vary according to the chrominance signal format.

[192] FIG. 12 is a view illustrating the size of a chrominance component block according to a chrominance signal format when the luma component block is a 64×128 block. As shown in FIG. 12, a chrominance component block corresponding to a 64×128 luma component block may be a 32×64 block in 4:2:0 chrominance signal format, a 32×128 block in 4:2:2 chrominance signal format, and a 64× block 128 in 4:4:4 chrominance signal format.

[193] As described above, according to the conventional method described with reference to FIG. 9, in a single tree structure in which a luma component block and a chrominance component block are split into an identical tree structure, when ISP is applied to the current block, the luma component block is split into a plurality of sub-segments, but the chroma component block is not split. For example, when the luminance component block of the current block to which the ISP is applied is a 64×128 block, since the split direction is defined as the horizontal direction, the luminance component block is divided into four 64×32 sub-segments, and width (64) and height (32) of each subsegment are equal to or less than the maximum transform size (64). Accordingly, conversion or deconversion of each of the subsegments for a block of luminance signal components may be performed. However, as shown in FIG. 12, the chroma component block is a 32x64 block in 4:2:0 chroma format, a 32x128 block in 4:2:2 chroma format, or a 64x128 block in 4:4:4 chroma format. and, for example, in a 4:2:2 chrominance signal format and a 4:4:4 chrominance signal format, since the chroma component block height (128) exceeds the maximum transform size (64), conversion or deconversion of the chrominance component block is not possible .

[194] Various embodiments of the present disclosure to address the above problems are described in detail later in this document.

[195] Embodiment #1

[196] In Embodiment #1 of the present disclosure, in order to solve traditional problems in applying an ISP to a current block, an ISP is adaptively applied to a chrominance component block based on the chrominance signal format and/or the size of the chrominance component block. According to Embodiment #1 of the present disclosure, for example, when applying an ISP to a block of chrominance signal components, the dividing direction and number of divisions determined with respect to the luma component block are equally applicable to the chrominance component block.

[197] FIG. 13 is a flowchart illustrating an ISP application method according to an embodiment of the present disclosure.

[198] FIG. 14 is a view illustrating an example of dividing a block of chrominance signal components according to the ISP application method of FIG. 13.

[199] When the current block to be encoded/decoded is input (S1310), it can be determined whether or not ISP should be applied to the current block (S1320). The determination of step S1320 may include determining whether or not an ISP is available for the current block, and/or whether or not the ISP should be applied to the current block. Determination as to whether or not an ISP is available for the current block can be made based on the ISP availability conditions described above. When the ISP is available, the image encoding equipment can determine whether or not the ISP should be applied to the current block based on various methods as described above, and the result of the determination can be encoded in the bit stream as intra prediction technology information. When the ISP is available, the image decoding equipment can determine whether or not the ISP should be applied to the current block based on the intra prediction technology signaled information.

[200] When ISP is not applied to the current block, TU tiling can be performed relative to the current block (S1330). Thereafter, encoding/decoding may be performed with respect to the current block or each of the blocks divided through the TU tile arrangement step (S1390). Steps S1330 and S1390 are identical to steps S930 and S960 in FIG. 9, and therefore detailed description thereof will be omitted.

[201] In step S1320, when applying the ISP to the current block, the split direction and the number of splits (number of sub-segments) can be determined (S1340). The description of determining the dividing direction and the number of divisions is identical to that in FIG. 9 and is therefore omitted.

[202] When the dividing direction and the number of divisions are determined in step S1340, the luminance signal component block of the current block can be divided based on the specified (S1350).

[203] Thereafter, it can be determined whether or not ISP should be applied to the chroma component block corresponding to the luma component block (S1360). The determination of step S1360 may be performed by the chrominance signal format and/or by a comparison between the width and height of the chrominance signal component block and the maximum transform size.

[204] In the case of a 4:2:0 chrominance signal format, as described below, the width and height of the chroma component block is equal to or less than the maximum transform size. Accordingly, in the case of a 4:2:0 chrominance signal format, without comparing the width and height of the chrominance component block with the maximum transform size, it may be determined that the ISP is not applied to the chrominance component block.

[205] In the case of a 4:2:2 chroma signal format or a 4:4:4 chroma signal format as described below, the width or height of a chroma component block may exceed the maximum transform size. Accordingly, in the case of a 4:2:2 chrominance signal format or a 4:4:4 chrominance signal format, the width and height of a chroma component block may be compared with the maximum transform size, and when the width or height of a chrominance component block exceeds the maximum transform size, it may be determined that the ISP is applied to a block of chrominance signal components.

[206] As shown in FIG. 14, when the luma component block is a 64×128 block, the chrominance component block corresponding to it is a 32×64 block in the 4:2:0 chrominance signal format, a 32×128 block in the 4:2 chrominance signal format: 2 or 64x128 block in 4:4:4 chrominance signal format.

[207] In the example shown in FIG. 14, it can be seen that the width (32) and height (64) of the chrominance component block of the 4:2:0 chrominance signal format are equal to or less than the maximum transform size (64). Accordingly, in the case of a 4:2:0 chrominance signal format, it can be determined that the ISP is not applied to a 32x64 chroma component block without further size comparison.

[208] In contrast, in the case of a 4:2:2 chrominance signal format or a 4:4:4 chrominance signal format, further, the width and height of a block of chrominance signal components may be compared with the maximum transform size. In the example shown in FIG. 14, since the width (128) of a 32×128 chrominance component block of a 4:2:2 chrominance signal format and the width (128) of a 64×128 chrominance component block of a 4:4:4 chrominance signal format exceed the maximum transform size (64), it may be determined that the ISP is applied to a block of chrominance signal components.

[209] As a modification of Embodiment #1, regardless of the chrominance signal format, based on whether the width and height of the chrominance component block are equal to or less than or not the maximum transform size, it can be determined whether or not the ISP should be applied to the signal component block chromaticity. For example, when the width and height of the chrominance component block are equal to or less than the maximum transform size, it may be determined that the ISP is not applied to the chrominance component block. In addition, when the width or height of the chrominance component block exceeds the maximum transform size, it may be determined that ISP is applied to the chrominance component block.

[210] When determining that the ISP is applied to the chrominance component block, the chrominance component block may be split (S1370). The splitting of the chroma component block may be performed based on the splitting direction and the number of splits determined in step S1340. In particular, the partitioning direction and the number of partitions for a block of chrominance signal components may be determined identically to the partitioning direction and the number of partitions for a block of luma signal components. According to Embodiment #1 of the present disclosure, in the example shown in FIG. 14, the luminance signal component block is divided into four sub-segments in the horizontal direction. Accordingly, a 32×128 chroma component block of a 4:2:2 chrominance signal format and a 64×128 chroma component block of a 4:4:4 chrominance signal format may be divided into four sub-segments in the horizontal direction identical to the luma component block.

[211] Thereafter, each of the divided sub-segments can be encoded/decoded (S1390). In this case, encoding/decoding of step S1390 may be performed on each sub-segment of the luma component block and each sub-segment of the chrominance component block. Step S1390 is identical to step S960, and therefore a detailed description thereof will be omitted.

[212] In step S1360, when determining that ISP is not applied to the chrominance component, the chrominance component block is not split (S1380), and each sub-segment of the luma component block and the unsplit chrominance component block can be encoded/decoded (S1390) . Step S1390 is identical to step S960, and therefore a detailed description thereof will be omitted. According to Embodiment #1 of the present disclosure, in the example shown in FIG. 14, a 32×64 chroma component block of a 4:2:0 chrominance signal format may not be split.

[213] According to Embodiment #1 of the present disclosure, when the width or height of the chroma component block of the current block to which the ISP is applied exceeds the maximum transform size, it may be possible to resolve such a problem that the transform or inverse transform of the chrominance component block is impossible by applying ISP to a block of chroma components. In addition, according to Embodiment #1 of the present disclosure, when the chrominance signal format of the current block to which the ISP is applied is 4:2:0, or the block width and height of the chrominance signal components of the current block to which the ISP is applied are equal to or less than the maximum transform size, ISP is not applied to the chrominance component block. Therefore, since a case in which ISP is unnecessarily applied to a chroma component block can be avoided, the amount of encoding/decoding calculations can be reduced. In addition, according to Embodiment #1 of the present disclosure, since the partitioning direction and number of ISP partitions for a chrominance signal component block are determined identically to the partitioning direction and number of ISP partitions for a luma signal component block, the partitioning direction and number of ISP partitions for a chrominance signal component block are not must be separately signaled or retrieved.

[214] Embodiment #2

[215] In Embodiment #2 of the present disclosure, in order to solve the traditional problem where the ISP is applied to the current block, the ISP is also applied to the chroma component block. According to Embodiment #2 of the present disclosure, the partitioning direction and the number of partitions for a block of chrominance signal components can be determined based on the partitioning direction and the number of partitions determined for the luma component block.

[216] FIG. 15 is a flowchart illustrating an ISP application method according to another embodiment of the present disclosure.

[217] FIG. 16 is a view illustrating an example of dividing a block of chrominance signal components according to the ISP application method of FIG. 15.

[218] When the current block to be encoded/decoded is input (S1510), it can be determined whether or not ISP should be applied to the current block (S1520). The determination of step S1520 may include determining whether or not an ISP is available for the current block, and/or whether or not the ISP should be applied to the current block. Determination as to whether or not an ISP is available for the current block can be made based on the ISP availability conditions described above. When the ISP is available, the image encoding equipment can determine whether or not the ISP should be applied to the current block based on various methods as described above, and the result of the determination can be encoded in the bit stream as intra prediction technology information. When the ISP is available, the image decoding equipment can determine whether or not the ISP should be applied to the current block based on the intra prediction technology signaled information.

[219] When ISP is not applied to the current block, TU tiling can be performed relative to the current block (S1530). Thereafter, encoding/decoding may be performed with respect to the current block or each of the blocks divided through the TU tile arrangement step (S1570). Steps S1530 and S1570 are identical to steps S930 and S960 in FIG. 9, and their detailed description will be omitted.

[220] In step S1520, when applying the ISP to the current block, the split direction and the number of splits (number of sub-segments) can be determined (S1540). The description of determining the dividing direction and the number of divisions is identical to that in FIG. 9 and is therefore omitted.

[221] The luminance signal component block of the current block may be divided based on the direction of division and the number of divisions determined in step S1540 (S1550).

[222] Thereafter, the chroma component block of the current block may be split based on the split direction and the number of splits determined in step S1540 (S1560). In particular, the partitioning direction for a block of chrominance signal components may be determined identically to the partitioning direction for a block of luma signal components. In addition, the number of splits for the chroma component block may be determined based on the number of splits for the luma component block. For example, when the number of splits for a luma component block is N, the number of splits for a chrominance component block may be determined to be N/n. In Embodiment #2 of the present disclosure, for example, n may be equal to, but is not limited to, 2, and n may be any integer. According to embodiment #2, in the example shown at the top of FIG. 16, a 16×16 luminance signal component block may be divided into four sub-segments in the horizontal direction. In this case, the partitioning direction of the corresponding chrominance component block is the horizontal direction, and the number of partitions may be determined as 2. Thus, the corresponding chrominance component block can be divided into two sub-segments in the horizontal direction regardless of the chrominance signal format and/or size block of chrominance signal components. In addition, in the example shown at the bottom of FIG. 16, a 4x8 luminance signal component block is divided into two sub-segments in the vertical direction. In this case, the partitioning direction of the corresponding chrominance component block is a vertical direction, and the number of partitions can be determined as 1. In this case, since the chrominance component block is hardly divided, it can be seen that ISP is not applied to the chrominance component block .

[223] Thereafter, each sub-segment or undivided block of chrominance signal components can be encoded/decoded (S1570). In this case, encoding/decoding of step S1570 may be performed on each sub-segment of a luma component block and each sub-segment of a chrominance component block or a chrominance component block. Step S1570 is identical to step S960, and therefore a detailed description thereof will be omitted.

[224] In Embodiment #2 of the present disclosure, when the maximum encoding unit size is 128×128, the maximum transform size is 64, which is half the width and height of the maximum encoding unit, and the width and height of the current block exceeds the maximum transform size, ISP is not applied to the current block. According to Embodiment #2 of the present disclosure, since the width and height of a chrominance component block of the current block to which the ISP is applied or a sub-segment of a chrominance component block are always equal to or less than the maximum transform size, the problem that transforming or inversely transforming the block can be resolved color signal components is not possible. In addition, according to Embodiment #2 of the present disclosure, since determination as to whether or not ISP should be applied to the chrominance component block of the current block (eg, determination S1360) need not be made, the amount of encoding/decoding calculations can be reduced. In addition, according to Embodiment #2 of the present disclosure, since the number of subsegments for a chrominance component block is determined to be 2 or 1, the encoding/decoding process of the chrominance component block can be simplified. In addition, according to Embodiment #2 of the present disclosure, since the partitioning direction and the number of ISP partitions for a chrominance signal component block are determined identically to the partitioning direction and (number of partitions/2) ISP for a luminance signal component block, the partitioning direction and the number of ISP partitions for the block The chrominance signal components shall not be separately signaled or extracted.

[225] Embodiment #3

[226] In Embodiment #3 of the present disclosure, to solve the traditional problem, the width and height of the chroma component block are equal to or less than the maximum transform size by changing the ISP availability conditions.

[227] According to the conventional ISP method described with reference to FIG. 9, when the width and height of the current block exceed the maximum transform size, it is determined that the ISP is not applied to the current block. Thus, when one of the width or height of the current block is greater than the maximum transform size, and the other is equal to or less than the maximum transform size, it is determined that the ISP is applied to the current block.

[228] In Embodiment #3 of the present disclosure, the above-described ISP availability conditions are modified as the following Table 5.

[229] Tab. 5

Modified ISP Availability Terms
intra_luma_ref_idx[x0][y0]==0
cbWidth<=MaxTbSizeY andand cbHeight<=MaxTbSizeY
cbWidth*cbHeight>MinTbSizeY*MinTbSizeY

[230] Of the modified ISP availability conditions, the condition in which intra_luma_ref_idx[x0][y0] is equal to 0, and the condition in which cbWidth*cbHeight is greater than MinTbSizeY*MinTbSizeY are equal to the existing ISP availability conditions. According to the modified ISP availability conditions, only when the cbWidth and cbHeight of the current block are equal to or less than MaxTbSizeY can it be determined that the ISP is available for the current block.

[231] According to embodiment #3 of the present disclosure, in which the modified ISP availability conditions are applied, only when the width and height of the current block are equal to or less than the maximum transform size, it is determined that the ISP is applied to the current block, and the intra_subpartitions_mode_flag. indicating whether or not to use the ISP may be transmitted.

[232] According to Embodiment #3 of the present disclosure, since the width and height of the current block are equal to or less than the maximum transform size, the width and/or height of the block of chroma samples corresponding to the block of luma samples of the current block is always equal to or less than the maximum transform size , regardless of the chrominance signal format. Accordingly, the problem of the conventional ISP method described with reference to FIG. 9, through the application of modified ISP availability conditions.

[233] The method according to Embodiment #3 of the present disclosure can be implemented by modifying the ISP availability conditions of the conventional method. Accordingly, the method according to Embodiment #3 of the present disclosure may be carried out identically to the flowchart shown in FIG. 9, except that only the condition for determining whether or not the ISP is available in step S920 is different.

[234] FIG. 17 is a view illustrating an example of a bitstream structure in which the modified ISP availability condition is reflected.

[235] The rectangular field of FIG. 17 is associated with signaling information regarding the ISP intra prediction technology information of the current coding unit. As shown in FIG. 17, when the above-described modified ISP availability conditions are satisfied, an intra_subpartitions_mode_flag regarding whether or not ISP should be applied may be transmitted; intra_subpartitions_mode_flag is information indicating whether or not the ISP should be applied to the current encoding unit. When the ISP availability conditions are not satisfied, the intra_subpartitions_mode_flag is not sent and it may be determined that the ISP does not apply to the current encoding unit.

[236] The modified ISP accessibility terms may include one or more terms, and the one or more terms included in the modified ISP accessibility terms are not limited to the above examples. Thus, some terms may be omitted or other terms may be additionally included within the scope of the technical concept according to the present disclosure.

[237] In addition, as shown in FIG. 17 when ISP is applied to the current block, i.e. when intra_subpartitions_mode_flag is 1, information (intra_subpartitions_split_flag) indicating the direction of splitting may be transmitted.

[238] In the example shown in FIG. 17, only when the condition cbWidth<=MaxTbSizeY andand cbHeight<=MaxTbSizeY is satisfied, intra_subpartitions_mode_flag can be transmitted. Thus, when the passed intra_subpartitions_mode_flag is 1, you can see that the condition cbWidth<=MaxTbSizeY andand cbHeight<=MaxTbSizeY I is already satisfied. Accordingly, the condition of FIG. 17 related to whether or not the intra_sinpartitions_split_flag should be transmitted may change as shown in FIG. 18.

[239] FIG. 18 is a view illustrating another example of a bitstream structure in which a modified ISP availability condition is reflected.

[240] The rectangular field of FIG. 18 is associated with signaling information regarding ISP intra prediction technology information of the current coding unit. As shown in FIG. 18, intra_subpartitions_mode_flag may be signaled when the above-described modified ISP availability conditions are satisfied, and intra_sinpartitions_split_flag may be signaled when the condition in which intra_subpartitions_mode_flag is 1 is satisfied.

[241] According to embodiment #3 of the present disclosure, the width and height of the current block to which the ISP is applied are equal to or less than the maximum transform size. Accordingly, since the width and height of the chrominance component block of the current block are equal to or smaller than the maximum conversion size regardless of the chrominance signal format, the problem that conversion or inverse conversion of the chrominance component block is not possible can be resolved. In addition, according to Embodiment #3 of the present disclosure, since the determination as to whether or not the ISP should be applied to the chrominance component block of the current block (eg, the determination of S1360), and the ISP for the chrominance component block need not be made, the volume encoding/decoding calculations may decrease. In addition, according to Embodiment #3 of the present disclosure, since the traditional encoding/decoding process is not significantly changed, the traditional problem can be resolved without significant encoding/decoding complexity.

[242] Although the exemplary methods of the present disclosure described above are presented as a sequence of operations for clarity of description, it is not intended to limit the order in which the steps are performed, and the steps may be performed simultaneously or in a different order as necessary. To implement the method of the present disclosure, the described steps may further include other steps, may include the remaining steps except some steps, or may include other additional steps except some steps.

[243] In the present disclosure, image encoding equipment or image decoding equipment that performs a predetermined operation (step) may perform an operation (step) of confirming a condition or situation for executing the corresponding operation (step). For example, if it is described that a predetermined operation is performed when a predetermined condition is satisfied, the image encoding equipment or the image decoding equipment may perform the predetermined operation after determining whether or not the predetermined condition is satisfied.

[244] The various embodiments of the present disclosure are not intended to be a list of all possible combinations and are intended to describe characteristic aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combination of two or more of them.

[245] Various embodiments of the present disclosure may be implemented in hardware, firmware, software, or a combination of the above. If the present disclosure is implemented in hardware, the present disclosure may be implemented in application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays. (FPGA), general processors, controllers, microcontrollers, microprocessors, etc.

[246] In addition, the image decoding equipment and image encoding equipment to which embodiments of the present disclosure apply may be included in a multimedia broadcast transceiver device, a mobile communication terminal, a home theater system video device, a digital cinema system video device, a surveillance camera, video chat device, real-time communication device such as video communication, mobile streaming device, storage medium, video recording camera, video-on-demand (VoD) service device, OTT video device ( video over networks), Internet streaming service device, 3D video device, video telephony system video device, medical video device, etc. and can be used to process video or data signals. For example, OTT video devices may include a gaming console, Blu-ray player, Internet-enabled television, home theater system, smartphone, tablet PC, digital video recorder (DVR), etc.

[247] FIG. 19 is a view showing a content streaming system to which an embodiment of the present disclosure is applicable.

[248] Referring to FIG. 19, the content streaming system to which the embodiment(s) of this document applies may mainly include an encoding server, a streaming server, a web server, a media storage, a user device, and a media input device.

[249] The encoding server compresses content input from media input devices such as a smartphone, camera, video recorder, etc. into digital data to form a bitstream, and transmits the bitstream to the streaming server. As another example, when media input devices such as smartphones, cameras, video recorders, etc. directly generate a bit stream, the encoding server may be omitted.

[250] The bitstream may be generated by an image encoding method or image encoding equipment to which an embodiment of the present disclosure is applied, and the streaming server may temporarily store the bitstream during the process of transmitting or receiving the bitstream.

[251] The streaming server transmits multimedia data to the user device based on the user's request through the web server, and the web server serves as a medium for informing the user regarding the service. When a user requests a desired service from a web server, the web server may deliver it to the streaming server, and the streaming server may stream the media data to the user. In this case, the content streaming system may include a separate control server. In this case, the control server serves to manage the command/response between devices in the content streaming system.

[252] The streaming server may receive content from a media storage and/or encoding server. For example, when content is received from an encoding server, the content may be received in real time. In this case, in order to provide a smooth streaming service, the streaming server may maintain the bit stream for a predetermined time.

[253] Examples of a user device may include a mobile phone, a smartphone, a laptop computer, a digital broadcast terminal, a personal digital assistant (PDA), a portable media player (PMP), a navigation device, a tablet PC, tablet PCs, ultrabooks, wearable devices (such as smart watches, smart glasses, head-mounted displays), digital TVs, desktop computers, digital signage system, etc.

[254] Each server in a content streaming system may operate as a distributed server, in which case data received from each server may be distributed.

[255] The scope of the disclosure includes software or computer executable instructions (e.g., operating system, application, firmware, program, etc.) to enable operations according to the methods of various embodiments in hardware or computer, non-volatile computer-based a readable medium having such software or instructions stored and executed in hardware or a computer.

[256] Industrial applicability

[257] Embodiments of the present disclosure may be used to encode or decode an image.

Claims (29)

1. An image decoding method carried out by image decoding equipment, wherein the image decoding method comprises the steps of: determining a prediction mode of the current block based on the prediction mode information of the current block; determining whether an intra-sub-segment (ISP) mode is available for the current block based on the prediction mode of the current block being the intra-prediction mode; decoding ISP mode information for the current block based on the ISP available for the current block; And generating a prediction sample for the current block based on the ISP mode information, wherein the ISP mode information includes ISP split direction information, wherein based on both the width and the block height of the luminance signal components of the current block equal to or less than a particular threshold value, the ISP is determined to be available for the current block. 2. The image decoding method of claim 1, wherein the predetermined threshold value is a maximum transform size. 3. The image decoding method according to claim 1, wherein generating a prediction sample for the current block based on ISP mode information comprises the steps of: generating a prediction sample for a block of luminance signal components by applying an ISP to the block of luminance signal components of the current block; determining whether the ISP should be applied to the chrominance component block of the current block; And generating a prediction sample for the chrominance component block by applying the ISP to the chrominance component block, upon determining that the ISP is applied to the chrominance component block of the current block. 4. The image decoding method of claim 3, wherein determining whether or not ISP should be applied to a chrominance component block of the current block is made based on at least one of a chrominance component block size or a chrominance signal format of the current block. 5. The image decoding method of claim 4, wherein when the chrominance signal format of the current block is 4:2:2 or 4:4:4 and at least one of the block width or height of the chrominance signal components exceeds the maximum transform size, it is determined that the ISP is applied to the chrominance component block of the current block. 6. The image decoding method of claim 3, wherein when ISP is applied to a chrominance component block, the partitioning direction and the number of sub-segments for the chrominance signal component block are determined based on the partitioning direction and the number of sub-segments for the luma component block. 7. The image decoding method according to claim 6, wherein the partitioning direction and the number of subsegments for the chrominance signal component block are equal to the partitioning direction and the number of subsegments for the luminance signal component block, respectively. 8. The image decoding method according to claim 1, wherein generating a prediction sample for the current block based on ISP mode information comprises the steps of: generating a prediction sample for a block of luminance signal components by applying an ISP to the block of luminance signal components of the current block; And generating a prediction sample for the chrominance component block by applying the ISP to the chrominance component block of the current block. 9. The image decoding method of claim 8, wherein the partitioning direction and the number of sub-segments for the chroma component block are determined based on the partitioning direction and the number of sub-segments for the luminance signal component block. 10. The image decoding method according to claim 9, wherein the partitioning direction and the number of subsegments for the chrominance signal component block are equal to the partitioning direction and (number of subsegments/2) of the luminance signal component block, respectively. 11. An image encoding method carried out by image encoding equipment, wherein the image encoding method comprises the steps of: determine the prediction mode of the current block; determining whether an intra-sub-segment (ISP) mode is available for the current block based on the prediction mode of the current block being the intra-prediction mode; determining whether the ISP should be applied to the current block based on the ISP available for the current block; generating a prediction sample for the current block by applying an ISP to the current block based on the ISP determined to be applied to the current block; And encode prediction mode information of the current block and ISP mode information, wherein the ISP mode information includes ISP split direction information, wherein based on both the width and the block height of the luminance signal components of the current block equal to or less than a particular threshold value, the ISP is determined to be available for the current block.