RU2808004C2 - Method and device for internal prediction based on internal subsegments in image coding system - Google Patents

Abstract

FIELD: image coding technology.

SUBSTANCE: invention relates to a method and equipment for intra-prediction based on internal sub-segments in an image coding system. Increased coding efficiency is achieved by the method of image decoding by a decoding device that comprises the steps of: receiving prediction mode information regarding the current block; receiving internal subsegment mode-related information regarding the current block based on the size and maximum size of the transform block of the current block; extracting the internal prediction mode of the current block based on the prediction mode information; and generating a prediction sample of the current block based on the intra prediction mode and the ISP-related information.

EFFECT: increasing the coding efficiency.

3 cl, 12 dwg, 6 tbl

Description

State of the art

Field of technology to which the invention relates

[1] This disclosure relates to image coding technology, and more particularly to a method and equipment for intra-prediction based on internal sub-segments in an image coding system.

Description of the prior art

[2] The demand for high-quality, high-definition images, such as high-definition (HD) and ultra-high-definition (UHD) images, is growing in various fields of technology. Since the image data is high resolution and high quality, the amount of information or bits that must be transmitted increases relative to the legacy image data. Therefore, when image data is transmitted using a medium such as a traditional wired/wireless broadband line, or image data is stored using an existing storage medium, transmission costs and storage costs for them increase.

[3] Accordingly, there is a need for high-performance image compression technology to efficiently transmit, store and reproduce high-quality, high-resolution image information.

The essence of the invention

[4] This document shall provide a method and equipment to improve the efficiency of image encoding.

[5] This document should also provide an effective internal forecasting method and equipment.

[6] This document shall also provide an intra-prediction method and hardware for applying intra-subsegment-based (ISP) mode to a block less than or equal to the size of a virtual pipeline data unit (VPDU).

[7] This document shall also provide a method and hardware for using the ISP, taking into account the decoding pipeline for hardware implementation.

[8] According to an embodiment of this document, an image decoding method performed by decoding equipment is provided. The method includes receiving prediction mode information for a current block, receiving intra-subsegment-based (ISP) mode-related information for the current block based on the size of the current block and a maximum transform block size, retrieving the intra-prediction mode of the current block based on the prediction mode information, and generating a predictive sample of the current block based on the internal prediction mode and ISP-related information.

[9] According to another embodiment of this disclosure, decoding equipment for performing image decoding is provided. The decoding equipment includes an entropy decoder configured to receive prediction mode information for a current block, receive intra-subsegment-based (ISP) mode-related information for the current block based on the current block size and a maximum transform block size, a prediction module configured with the ability to extract the intra prediction mode of the current block based on the prediction mode information and generate a prediction sample of the current block based on the intra prediction mode and ISP related information.

[10] According to yet another embodiment of this disclosure, an image encoding method carried out by encoding equipment is provided. The method includes retrieving the intra-prediction mode of the current block, retrieving the intra-subsegment-based (ISP) mode-related information for the current block based on the size of the current block and the maximum size of the transform block, generating a predictive sample of the current block based on the intra-prediction mode and associated ISP information, generating a residual sample based on the prediction sample, and encoding image information including prediction mode information regarding the intra prediction mode, ISP-related information, and information regarding the residual sample.

[11] According to yet another embodiment of this disclosure, encoding equipment for performing image encoding is provided. The encoding equipment includes a prediction module configured to extract an intra-prediction mode of a current block, extract intra-subsegment-based (ISP) mode-related information for the current block based on the size of the current block and the maximum size of the transform block, generate a predictive sample of the current block on based on the intra prediction mode and ISP related information, a subtractor configured to generate a residual sample based on the prediction sample, and an entropy encoder configured to encode image information including prediction mode information regarding the intra prediction mode, ISP related information, and information regarding the residual sample.

[12] According to yet another embodiment of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores the bit stream generated by the encoding method.

[13] According to this document, the efficiency of image compression can be improved.

[14] According to this document, efficient internal forecasting can be provided.

[15] According to this document, the problem caused by applying an internal subsegment-based (ISP) mode to a block larger than a virtual pipeline data unit (VPDU) size can be resolved.

[16] According to this document, it is possible to prevent misaligned VPDU processing caused by applying an ISP to a block larger than the VPDU size.

[17] According to this document, it is possible to provide ISPs with a decoding pipeline in mind for hardware implementation.

Brief description of drawings

[18] FIG. 1 schematically illustrates a video/image coding system to which this document may be applied.

[19] FIG. 2 is a circuit diagram illustrating the configuration of video/image encoding equipment to which this document may be applied.

[20] FIG. 3 is a circuit diagram illustrating the configuration of video/image decoding equipment to which this document can be applied.

[21] FIG. 4 illustrates an example of 67 intra prediction modes.

[22] FIG. 5 shows an example of block segmentation according to ISP in intra-prediction.

[23] FIG. 6 shows another example of block segmentation according to ISP in intra-prediction.

[24] FIG. 7 shows an example of TT and BT segmentation that is not resolved based on the VPDU size.

[25] FIG. 8 shows an example of an HW processing pipeline.

[26] FIG. 9 shows an example of a pipeline when the ISP is applied to a block larger than the VPDU size.

[27] FIG. 10 schematically illustrates a method for encoding images by encoding equipment according to the present document.

[28] FIG. 11 schematically illustrates a method for decoding images by decoding equipment according to the present document.

[29] FIG. 12 is a diagram illustrating the structure of a content streaming system.

Detailed Description of Embodiments

[30] This document may be modified in various forms, and specific embodiments thereof are described and shown in the drawings. However, the embodiments are not intended to limit this document. The terms used in the following description are used to simply describe specific embodiments, but are not intended to limit this document. The singular expression includes the plural expression as long as they are clearly interpreted differently. Terms such as “include” and “have” are intended to indicate that there are features, numbers, steps, operations, elements, components, or combinations thereof used in the following description, and as such are to be understood that the possibility of the presence or addition of one or more other features, numbers, steps, operations, elements, components or combinations of the above is not excluded.

[31] Meanwhile, each of the components in the drawings described in this document is shown independently for convenience of description regarding various characteristic functions and does not imply that the components are implemented in separate hardware or software. For example, two or more of each configuration may be combined to form one configuration, or one configuration may be divided into multiple configurations. Embodiments in which each configuration is integrated and/or separated are also included within the scope of this document without departing from the spirit of this document.

[32] Exemplary embodiments of this document are described in detail below with reference to the accompanying drawings. Hereinafter in this document, identical reference numbers are used for identical components in the drawings, and redundant descriptions of identical components may be omitted.

[33] FIG. 1 schematically illustrates a video/image coding system to which this document may be applied.

[34] Referring to FIG. 1, a video/image encoding system may include a first equipment (source device) and a second equipment (receiver device). A source device may deliver encoded information or video/image data in file or streaming form to a receiving device via a digital storage medium or network.

[35] The source device may include a video source, encoding equipment, and a transmitter. The receiving device may include a receiver, decoding equipment, and a rendering module. The encoding equipment may be referred to as "video/image encoding equipment" and the decoding equipment may be referred to as "video/image decoding equipment". The transmitter may be included in the encoding equipment. The receiver may be included in the decoding equipment. The rendering module may include a display, and the display may be configured as a separate device or an external component.

[36] A video source may obtain video/image through a video/image capture, synthesis, or shaping process. The video source may include a video/image capture device and/or a video/image generation device. The video/image capturing device may include, for example, one or more cameras, video/image archives including previously captured video/images, and the like. The video/image generating apparatus may include, for example, computers, tablet computers and smartphones and may (electronically) generate video/images. For example, a virtual video/image may be generated via a computer or the like. In this case, the video/image capturing process can be replaced by the associated data generation process.

[37] Encoding equipment can encode the input video/image. Encoding hardware may perform a sequence of procedures, such as prediction, transform, and quantization, for compression and encoding efficiency. Encoded data (encoded video/image information) may be output in the form of a bitstream.

[38] A transmitter may transmit information or encoded image/image data output in the form of a bitstream to a receiver of a receiving device via a digital storage medium or network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-Ray, HDD, SSD, and the like. The transmitter may include an element for generating a multimedia file through a predefined file format and may include an element for transmission through a broadcast/communication network. The receiver may receive/extract the bit stream and transmit the received bit stream to decoding equipment.

[39] The decoding equipment may decode a video/image by performing a sequence of procedures such as dequantization, deconversion, and prediction corresponding to the operation of the encoding equipment.

[40] The rendering module may prepare a decoded video/image by rendering. The video/image prepared by rendering can be displayed through the display.

[41] This document relates to video/image coding. For example, the methods/embodiments disclosed in this document may be applied to the method disclosed in the Versatile Video Coding (VVC) standard, the EVC (Fundamental Video Coding) standard, the AOMedia Video 1 (AV1) standard, the second generation audio/video coding standard (AVS2) or next generation video/image coding standard (eg H.267 or H.268, etc.).

[42] This document presents various video/image encoding embodiments, and the embodiments may be performed in combination with each other unless otherwise noted.

[43] In this document, video may refer to a sequence of images over time. A frame generally means a unit representing one image in a particular time zone, and a slice/tile is a unit that constitutes part of a frame when encoded. A slice/tile may include one or more coding tree units (CTUs). One frame can consist of one or more slices/tiles. One frame can consist of one or more groups of tiles. One tile group may include one or more tiles. A brick can represent a rectangular area of CTU rows in a tile in a frame. A tile can be segmented into multiple bricks, each of which consists of one or more CTU rows in the tile. A tile that is not segmented into multiple bricks may also be called a "brick". A brick scan is a specific sequential ordering of CTUs segmenting a frame, where CTUs are ordered sequentially in a raster CTU scan within a brick, bricks within a tile are ordered sequentially in a raster scan of tile bricks, and tiles within a frame are ordered sequentially in a raster scan of frame tiles. A tile is a rectangular CTU area in a particular column of tiles and a particular row of tiles in a frame. A column of tiles is a rectangular area of CTUs having a height equal to the height of the frame and a width specified by syntax elements in the frame parameter set. A row of tiles is a rectangular area of CTUs having a height specified by syntax elements in the frame parameter set and a width equal to the width of the frame. A tile scan is a specific sequential ordering of the CTUs segmenting a frame, in which CTUs are ordered sequentially in a raster scan of the CTUs in a tile, whereas tiles in a frame are ordered sequentially in a raster scan of the frame's tiles. A slice includes an integer number of frame bricks that can be contained exclusively in a single NAL unit. A slice can consist of either a specific number of full tiles, or only a rigid sequence of full bricks of a single tile. Tile groups and slices can be used interchangeably in this document. For example, in this document, a group of tiles/header of a group of tiles may be referred to as a "slice/slice header".

[44] Pixel or pel can mean the smallest unit that makes up one frame (or image). Additionally, "sample" can be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, and may represent only a pixel/pixel value of a luma component or only a pixel/pixel value of a chroma component.

[45] The unit may represent a basic image processing unit. The unit may include at least one of a specific frame area and information associated with the area. One unit may include one block of luma signals and two blocks of chrominance signals (eg, Cb, Cr). Unit may be used interchangeably with terms such as block or zone in some cases. In general, an MxN block may include samples (or arrays of samples) or a set (or array) of transform coefficients of M columns and N rows.

[46] In this document, the terms "/" and "," shall be interpreted to indicate "and/or". For example, the expression "A/B" could mean "A and/or B". Additionally, "A, B" may mean "A and/or B". Additionally, "A/B/C" may mean "at least one of A, B and/or C". In addition, "A/B/C" may mean "at least one of A, B and/or C".

[47] Additionally, as used herein, the term “or” should be interpreted to indicate “and/or.” For example, the expression "A or B" may contain 1) only A, 2) only B, and/or 3) both A and B. In other words, the term "or" in this document should be interpreted to indicate "in addition or alternatively" .

[48] FIG. 2 is a circuit diagram illustrating the configuration of video/image encoding equipment to which this document may be applied. Hereinafter herein, video encoding equipment may include image encoding equipment.

[49] Referring to FIG. 2, the encoding equipment 200 includes an image segmentation module 210, a prediction module 220, a residual processor 230 and an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The prediction module 220 may include an inter-prediction module 221 and an internal prediction module 222. forecasting. Residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. Residual processor 230 may further include a subtractor 231. The adder 250 may be referred to as a “recovery module” or a “recovered block generator.” The image segmentation module 210, prediction module 220, residual processor 230, entropy encoder 240, adder 250, and filter 260 may be configured by at least one hardware component (eg, a chipset or encoder processor) according to an embodiment. In addition, storage device 270 may include a decoded frame buffer (DPB) or may be configured with a digital storage medium. The hardware component may further include a storage device 270 as an internal/external component.

[50] The image segmentation module 210 may segment an input image (either a frame or a movie frame) input to the encoding equipment 200 into one more processing units. For example, a processing unit may be called a "coding unit (CU)". In this case, the coding unit may be recursively segmented according to a quadtree, binary tree, and ternary tree (QTBTTT) structure from a coding tree unit (CTU) or largest coding unit (LCU). For example, one coding unit may be segmented into multiple coding units of greater depth based on a quadtree structure, a binary tree structure, and/or a ternary structure. In this case, for example, a quadtree structure may be applied first, and a binary tree structure and a ternary structure may be subsequently applied. Alternatively, a binary tree structure may be used first. The encoding procedure according to embodiments may be performed based on a final encoding unit that is no longer segmented. In this case, the largest coding unit can be used as the final coding unit based on the coding efficiency according to the characteristics of the images, or if necessary, the coding unit can be recursively segmented into coding units of greater depth, and the coding unit having the optimal size can be used as the final coding units. Here, the encoding procedure may include a prediction, transformation and reconstruction procedure, which is described below. As another example, the processing unit may further include a prediction unit (PU) or a transformation unit (TU). In this case, the prediction unit and the transformation unit may be split or segmented from the above final encoding unit. The prediction unit may be a sample prediction unit, and the transformation unit may be a unit for extracting a transformation coefficient and/or a unit for extracting a residual signal from the transformation coefficient.

[51] Unit may be used interchangeably with terms such as block or zone in some cases. In general, an MxN block can represent a set of samples or transform coefficients consisting of M columns and N rows. A sample may generally represent a pixel or a pixel value, may represent only a pixel/pixel value of a luma component, or may represent only a pixel/pixel value of a chrominance signal component. Sampling can be used as a term corresponding to one frame (or image) for a pixel or pel.

[52] In the encoding equipment 200, the prediction signal (predicted block, prediction sample array) output from the inter prediction unit 221 or the intra prediction unit 222 is subtracted from the input image signal (original block, original sample array) to generate a residual signal ( residual block, residual sample array), and the generated residual signal is transmitted to converter 232. In this case, as shown, a module for subtracting the prediction signal (predicted block, prediction sample array) from the input image signal (source block, source sample array) into encoder 200 may be referred to as "subtractor 231". The prediction module may perform prediction on a block to be processed (hereinafter referred to as a "current block") and generate a prediction block including prediction samples for the current block. The prediction module may determine whether intra prediction or inter prediction is applied based on the current block or CU. As described below in the description of each prediction mode, the prediction module may generate various kinds of prediction-related information, for example, prediction mode information, and transmit the generated information to the entropy encoder 240. Information regarding the prediction may be encoded in the entropy encoder 240 and output in the form bit stream.

[53] The intra prediction module 222 may predict the current block by reference to the samples in the current frame. Samples for reference may be located in the surroundings of the current block or may be spaced according to the prediction mode. In intraprediction, prediction modes may include a plurality of non-directional modes and a plurality of directed modes. The omnidirectional mode may include, for example, DC mode and planar mode. The directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes according to the granularity of the prediction direction. However, this is merely an example; more or less directional prediction modes may be used depending on the setting. The intra prediction module 222 may determine the prediction mode applied to the current block by using the prediction mode applied to the adjacent block.

[54] The inter-prediction unit 221 can extract a predicted block for the current block based on a reference block (an array of reference samples) indicated by a motion vector for the reference frame. Here, in order to reduce the amount of motion information transmitted in the inter-prediction mode, motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between a neighboring block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction direction information (L0-prediction, L1-prediction, bi-prediction, etc.). In the case of inter-prediction, the neighbor block may include a spatial neighbor block present in the current frame and a temporal neighbor block present in the reference frame. The reference frame including the reference block and the reference frame including the temporary adjacent block may be identical or different. A temporary neighbor block may be called a “co-located reference block”, a “col-located CU (colCU)”, etc., and a reference frame including a temporary neighboring block may be called a “col-located frame (colPic)”. For example, the inter-prediction module 221 may configure a list of motion information candidates based on neighboring blocks and generate information indicating which candidate is used to retrieve the motion vector and/or reference frame index of the current block. Mutual prediction can be performed based on different prediction modes. For example, in the case of the skip mode and the merge mode, the inter-prediction unit 221 may use the motion information of the adjacent block as the motion information of the current block. In the skip mode, unlike the combine mode, the residual signal may not be transmitted. In the case of the motion vector prediction (MVP) mode, the motion vector of an adjacent block may be used as a motion vector predictor, and the motion vector of the current block may be indicated by signaling a motion vector difference.

[55] The prediction module 220 may generate a prediction signal based on various prediction methods described below. For example, the prediction module may not only apply intra-prediction or inter-prediction to predict one block, but also simultaneously apply both intra-prediction and inter-prediction. This may be referred to as "combined inter-intraprediction (CIIP)". In addition, the prediction module can be based on intra-block copy (IBC) prediction mode or palette mode for block prediction. The IBC prediction mode or palette mode can be used for encoding image/video content of a game and the like, for example, screen content encoding (SCC). IBC essentially performs prediction in the current frame, but can be performed similarly to inter-prediction in which the reference block is retrieved in the current frame. Thus, the IBC may use at least one of the inter-prediction technologies described in this document. Palette mode can be considered an example of intra-coding or intra-prediction. When palette mode is applied, a sampled value in a frame may be signaled based on information regarding the palette table and palette index.

[56] The prediction signal generated by the prediction module (including inter prediction module 221 and/or intra prediction module 222) can be used to generate a reconstructed signal or generate a residual signal. Converter 232 may generate conversion coefficients by applying conversion technology to the residual signal. For example, the transform technology may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), a graph-based transform (GBT), or a conditionally nonlinear transform (CNT). ). Here, GBT means a graph-derived transformation when the relationship information between pixels is represented by the graph. CNT means a transform generated based on a prediction signal generated using all previously reconstructed pixels. In addition, the conversion process can be applied to square pixel blocks that are identical in size, or can be applied to blocks that are variable in size rather than square.

[57] Quantizer 233 may quantize the transform coefficients and transmit them to entropy encoder 240, and entropy encoder 240 may encode the quantized signal (information regarding the quantized transform coefficients) and output a bit stream. Information regarding the quantized transform coefficients may be referred to as "residual information". Quantizer 233 may re-arrange the block quantized transform coefficients into one-dimensional vector form based on the scanning order of the coefficients, and generate information regarding the quantized transform coefficients based on the quantized transform coefficients in one-dimensional vector form. Information regarding conversion coefficients can be generated. The entropy encoder 240 may implement various encoding techniques, such as, for example, exponential Golomb coding, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), and the like. Entropy encoder 240 may encode information needed to reconstruct video/images other than quantized transform coefficients (eg, syntax element values, etc.), together or separately. Encoded information (eg, encoded video/image information) may be transmitted or stored in NAL (Network Abstraction Layer) units in the form of a bit stream. The video/image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. In this document, information and/or syntax elements transmitted/signaled from encoding equipment to decoding equipment may be included in video/frame information. The video/image information may be encoded through the encoding procedure described above and included in the bit stream. The bit stream may be transmitted over a network or may be stored on a digital storage medium. The network may include a broadcast network and/or a communications network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-Ray, HDD, SSD, and the like. A transmitter (not shown) transmitting the signal output from the entropy encoder 240 and/or a storage module (not shown) storing the signal may be included as an internal/external element of the encoding equipment 200, and alternatively, the transmitter may be included in the entropy encoder 240 .

[58] The quantized transform coefficients output from quantizer 233 can be used to generate a predictive signal. For example, a residual signal (residual block or residual samples) may be reconstructed by applying dequantization and inverse transform to the quantized transform coefficients through dequantizer 234 and inverse transformer 235. Adder 250 adds the reconstructed residual signal with the prediction signal output from interprediction module 221 or module 222 internal prediction to generate a reconstructed signal (reconstructed frame, reconstructed block, array of reconstructed samples). If there is no remainder for a block to be processed, for example in a case in which the skip mode is applied, the predicted block can be used as a reconstructed block. The adder 250 may be referred to as a "recovery module" or a "recovered block generator." The generated reconstructed signal can be used to internally predict the next block to be processed in the current frame, and can be used to inter-predict the next frame through filtering, as described below.

[59] Meanwhile, luma conversion with chrominance scaling (LMCS) can be applied during encoding and/or reconstruction of frames.

[60] Filter 260 may enhance the subjective/objective quality of images by applying filtering to the reconstructed signal. For example, filter 260 may generate a modified reconstructed frame by applying various filtering techniques to the reconstructed frame and storing the modified reconstructed frame in storage device 270, namely, a DPB of storage device 270. Various filtering techniques may include, for example, deblocking filtering. , sampled adaptive bias, adaptive contour filter, bilateral filter, etc. The filter 260 may generate various types of filtering-related information and transmit the generated information to the entropy encoder 240, as described below in the description of each filtering method. Information associated with filtering may be encoded by entropy encoder 240 and output in the form of a bitstream.

[61] The modified reconstructed frame transferred to the storage device 270 can be used as a reference frame in the inter-prediction unit 221. When inter-prediction is applied by the encoding equipment, prediction mismatch between the encoding equipment 200 and the decoding equipment can be eliminated, and encoding efficiency can be improved.

[62] The DPB of the storage device 270 may store the modified reconstructed frame for use as a reference frame in the inter-prediction module 221. Memory 270 may store block motion information from which motion information in the current frame is retrieved (or encoded) and/or motion information of blocks in a frame that have already been reconstructed. The stored motion information may be transmitted to the inter-prediction unit 221 and used as spatial neighbor block motion information or temporal neighbor block motion information. Memory 270 may store reconstructed samples of reconstructed blocks in the current frame and may transmit reconstructed samples to intra prediction module 222.

[63] FIG. 3 is a circuit diagram illustrating the configuration of video/image decoding equipment to which this document can be applied.

[64] Referring to FIG. 3, the decoding equipment 300 may include an entropy decoder 310, a residual processor 320, a prediction module 330, an adder 340, a filter 350, and a memory 360. The prediction module 330 may include an inter-prediction module 331 and an intra-prediction module 332. Residual processor 320 may include a dequantizer 321 and an inverse converter 321. Entropy decoder 310, residual processor 320, prediction module 330, adder 340, and filter 350 may be configured by a hardware component (eg, a chipset or decoder processor) according to an embodiment. In addition, storage device 360 may include a decoded frame buffer (DPB) or may be configured with a digital storage medium. The hardware component may further include storage device 360 as an internal/external component.

[65] When a bitstream including video/image information is input, the decoding equipment 300 can reconstruct an image corresponding to the process in which the video/image information is processed in the encoding equipment of FIG. 2. For example, decoding equipment 300 may extract units/blocks based on block segmentation-related information obtained from the bitstream. The decoding equipment 300 may perform decoding using a processing unit used in the encoding equipment. Thus, the decoding processing unit, for example, may be a coding unit, and the coding unit may be segmented according to a quadtree structure, a binary tree structure, and/or a ternary tree structure from a coding tree unit or a largest coding unit. One or more transformation units may be derived from a coding unit. The reconstructed image signal decoded and output by the decoding equipment 300 can be reproduced by the playback equipment.

[66] The decoding equipment 300 may receive a signal output from the encoding equipment of FIG. 2 in the form of a bit stream, and the received signal may be decoded through entropy decoder 310. For example, entropy decoder 310 may parse the bit stream to extract information (eg, video/image information) needed for image reconstruction (or frame reconstruction). The video/image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. The decoding equipment may further decode the frame based on information regarding the parameter set and/or general constraint information. The signaling/receiving information and/or syntax elements described later in this document may be decoded, may be decoded by a decoding procedure, and obtained from the bit stream. For example, entropy decoder 310 decodes information in a bitstream based on an encoding method, such as exponential Golomb, CAVLC, or CABAC encoding, and the output syntax elements required for image reconstruction and the quantized values of the transform coefficients for the remainder. More specifically, the entropy CABAC decoding method may receive a resolution element corresponding to each syntax element in the bitstream, determine a context model using information of a target decoding syntax element, decoding information of a target decoding block, or information of a symbol/resolution element decoded in a previous stage, and perform arithmetic decoding for the permission element by predicting the probability of occurrence of the permission element according to the determined context model and generating a symbol corresponding to the meaning of each syntactic element. In this case, the entropy CABAC decoding method can update the context model by using the decoded symbol/grant element information for the context model of the next symbol/grant element after determining the context model. The prediction-related information from the information decoded by the entropy decoder 310 may be provided to the prediction module (inter-prediction module 332 and intra-prediction module 331), and the residual value for which entropy decoding is performed in the entropy decoder 310, i.e. the quantized transform coefficients and associated parameter information may be input to residual processor 320. Residual processor 320 may extract a residual signal (residual block, residual samples, array of residual samples). In addition, information regarding filtering from information decoded by the entropy decoder 310 may be provided to the filter 350. Meanwhile, a receiver (not shown) for receiving a signal output from the encoding equipment may be further configured as an internal/external element of the equipment 300 decoding, or the receiver may be a component of the entropy decoder 310. Meanwhile, the decoding equipment according to this document may be called "video/image/frame decoding equipment", and the decoding equipment may be classified into information decoder (video/image/frame information decoder) and selective decoder (video/image/frame sampling decoder). The information decoder may include an entropy decoder 310, and the sample decoder may include at least one of a dequantizer 321, an inverter 322, an adder 340, a filter 350, a memory 360, an inter-prediction module 332, and an intra-prediction module 331. .

[67] The dequantizer 321 may dequantize the quantized transform coefficients and output the transform coefficients. The dequantizer 321 may re-arrange the quantized transform coefficients into a two-dimensional block form. In this case, re-arrangement may be performed based on the order of coefficient scanning performed in the encoding equipment. The dequantizer 321 may perform dequantization on the quantized transform coefficients by using a quantization parameter (eg, quantization step size information) and obtain the transform coefficients.

[68] An inverter 322 inverts the transform coefficients to obtain a residual signal (residual block, residual sample array).

[69] Prediction module 230 may perform prediction for the current block and may generate a predicted block including prediction samples for the current block. The prediction module may determine whether intra prediction or inter prediction is applied to the current block based on the prediction information output from the entropy decoder 310, and may determine a particular intra/inter prediction mode.

[70] The prediction module 320 may generate a prediction signal based on various prediction methods described below. For example, the prediction module may not only apply intra-prediction or inter-prediction to predict one block, but also simultaneously apply intra-prediction and inter-prediction. This may be referred to as "combined inter-intraprediction (CIIP)". In addition, the prediction module can be based on intra-block copy (IBC) prediction mode or palette mode for block prediction. The IBC prediction mode or palette mode can be used for encoding image/video content of a game and the like, for example, screen content encoding (SCC). IBC essentially performs prediction in the current frame, but can be performed similarly to inter-prediction in which the reference block is retrieved in the current frame. Thus, the IBC may use at least one of the inter-prediction technologies described in this document. Palette mode can be considered an example of intra-coding or intra-prediction. When palette mode is applied, a sampled value in a frame may be signaled based on information regarding the palette table and palette index.

[71] The intra prediction module 331 may predict the current block by reference to the samples in the current frame. Samples for reference may be located in the surroundings of the current block or may be spaced according to the prediction mode. In intraprediction, prediction modes may include a plurality of non-directional modes and a plurality of directed modes. The intra prediction unit 331 may determine the prediction mode applied to the current block by using the prediction mode applied to the adjacent block. The intra prediction module 331 may predict the current block by reference to the samples in the current frame. Samples for reference may be located in the surroundings of the current block or may be spaced according to the prediction mode. In intraprediction, prediction modes may include a plurality of non-directional modes and a plurality of directed modes. The intra prediction unit 331 may determine the prediction mode applied to the current block by using the prediction mode applied to the adjacent block.

[72] The inter-prediction unit 332 may extract a predicted block for the current block based on a reference block (an array of reference samples) indicated by a motion vector for the reference frame. In this case, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between a neighboring block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction direction information (L0-prediction, L1-prediction, bi-prediction, etc.). In the case of inter-prediction, the neighbor block may include a spatial neighbor block present in the current frame and a temporal neighbor block present in the reference frame. For example, the inter-prediction module 332 may configure a motion information candidate list based on neighboring blocks and extract the current block's motion vector and/or reference frame index based on the received candidate selection information. The inter-prediction may be performed based on various prediction modes, and the information regarding the prediction may include information indicating the inter-prediction mode for the current block.

[73] The adder 340 may generate a reconstructed signal (recovered frame, reconstructed block, reconstructed sample array) by adding the resulting residual signal with a prediction signal (predicted block, predicted sample array) output from a prediction module (including inter-prediction module 332 and /or internal prediction module 331). If there is no remainder for a block to be processed, for example when a skip mode is applied, the predicted block can be used as the reconstructed block.

[74] The adder 340 may be referred to as a “recovery module” or a “recovered block generator.” The generated reconstructed signal may be used to internally predict the next block to be processed in the current frame, may be output by filtering as described below, or may be used to inter-predict the next frame.

[75] Meanwhile, luma conversion with chrominance scaling (LMCS) can be applied in the frame decoding process.

[76] Filter 350 may enhance the subjective/objective quality of images by applying filtering to the reconstructed signal. For example, filter 350 may generate a modified reconstructed frame by applying various filtering techniques to the reconstructed frame and store the modified reconstructed frame in storage device 360, namely, a DPB of storage device 360. Various filtering techniques may include, for example, deblocking filtering. , sampled adaptive bias, adaptive contour filter, bilateral filter, etc.

[77] The (modified) reconstructed frame stored in the DPB of the storage device 360 can be used as a reference frame in the inter-prediction unit 332. Memory 360 may store block motion information from which motion information in the current frame is retrieved (or decoded) and/or motion information of blocks in a frame that have already been reconstructed. The stored motion information may be transmitted to the inter-prediction unit 260 so that it is used as spatial neighbor block motion information or temporal neighbor block motion information. Storage device 360 may store reconstructed samples of reconstructed blocks in the current frame and transmit the reconstructed samples to intra prediction module 331.

[78] In the present disclosure, the embodiments described in the filter 260, the inter-prediction module 221, and the intra-prediction module 222 of the encoding equipment 200 may be identical or appropriately applied so that they correspond to the filter 350, the inter-prediction module 332, and the 331 internal prediction equipment 300 decoding. The same may also apply to module 332 and internal prediction module 331.

[79] As described above, when performing video encoding, prediction is performed in order to improve compression efficiency. A predicted block including predicted samples for the current block, i.e. target encoding block, can be generated through prediction. In this case, the predicted block includes predicted samples into a spatial domain (or pixel domain). The predicted block is identically retrieved in the encoding equipment and decoding equipment. The encoding equipment can improve the efficiency of image encoding by signaling, to the decoding equipment, information regarding the remainder (residual information) between the original block, rather than the original sample value of the original block itself, and the predicted block. The decoding equipment can extract a residual block including residual samples based on the residual information, can generate a reconstructed block including the reconstructed samples by adding the residual block with a predicted block, and can generate a reconstructed frame including the reconstructed blocks.

[80] Residual information can be generated through a transformation and quantization procedure. For example, the encoding equipment may extract a residual block between an original block and a predicted block, may extract transform coefficients by performing a transform procedure on residual samples (an array of residual samples) included in the residual block, may extract quantized transform coefficients by performing a quantization procedure on the transform coefficients, and may signal associated residual information to decoding equipment (via a bit stream). In this case, the residual information may include information such as value information, location information, a transform circuit, a transform kernel, and a quantization parameter for the quantized transform coefficients. The decoding equipment may perform a dequantization/deconversion procedure based on the residual information and may extract the residual samples (or residual block). The decoding equipment may generate a reconstructed frame based on the predicted block and the residual block. In addition, the encoding equipment may extract a residual block by dequantizing/de-converting the quantized transform coefficients to reference the inter-prediction of a subsequent frame, and may generate a reconstructed frame.

[81] Intra prediction may represent prediction that generates prediction samples for the current block based on reference samples in the frame (hereinafter referred to as the "current frame") to which the current block belongs. When intra prediction is applied to the current block, neighboring reference samples that should be used for intra prediction of the current block can be retrieved. The adjacent reference samples of the current block may include a sample adjacent to the left boundary of the current block having size nWxnH, and the sum of 2xnH samples bordering the bottom left, a sample adjacent to the top boundary of the current block, and the sum of 2xnW samples bordering the right the top part, and one selection bordering the top left part relative to the current block. Additionally, the adjacent reference samples of the current block may also include top adjacent samples of a plurality of columns and left adjacent samples of a plurality of rows. Additionally, the adjacent reference samples of the current block may also include a sum of nH samples adjacent to the right boundary of the current block having size nWxnH, a sum of nW samples adjacent to the bottom boundary of the current block, and one sample adjacent to the bottom right of the current block. .

[82] However, some adjacent reference samples of the current block may not yet be decoded or may not be available. In this case, the decoder can configure adjacent reference samples to be used for prediction by substituting available samples instead of unavailable samples. Additionally, the decoder can configure adjacent reference samples to be used for prediction through interpolation of the available samples.

[83] When neighboring reference samples are retrieved, (i) the prediction sample can be inferred based on the average or interpolation of the current block's neighboring reference samples, and (ii) the prediction sample can be inferred based on the reference sample existing in a particular direction (prediction) with respect to predictive sample from neighboring reference samples of the current block. Case (i) may be referred to as "non-directional mode" or "non-angular mode", and case (ii) may be referred to as "directional mode" or "angular mode".

[84] Additionally, the prediction sample may be generated through interpolation between a first adjacent sample located in the prediction direction of the intra prediction mode of the current block and a second adjacent sample located in the opposite direction to the prediction direction from adjacent reference samples. The above case may be called "linear interpolation intraprediction (LIP)". Additionally, chrominance signal prediction samples can be generated based on the luminance signal samples using a linear model. This case may be called "LM mode".

[85] Additionally, the temporary prediction sample of the current block may be extracted based on the filtered neighboring reference samples, and the prediction sample of the current block may be extracted by weighting summation of at least one reference sample extracted according to the intra prediction mode from the existing neighboring reference samples , i.e. unfiltered neighboring reference samples, and temporary predictive samples. The above case may be called "position-dependent intraprediction (PDPC)".

[86] Additionally, a reference sample line having the highest prediction accuracy among several adjacent reference sample lines of the current block can be selected, and a prediction sample can be extracted from the corresponding line using a reference sample located in the prediction direction. In this case, intra prediction coding may be performed as a method for indicating (signalling) a reference sample line to be used to the decoding equipment. The above-described case may be called "multiple reference line (MRL) intra-prediction" or "MRL-based intra-prediction".

[87] Additionally, intra prediction can be performed based on the same intra prediction mode by dividing the current block into vertical or horizontal sub-segments, and adjacent reference samples can be retrieved and used in the sub-segment unit. Thus, in this case, the intra-prediction mode for the current block is equally applied to the sub-segments, and the adjacent reference sample is retrieved and used in the sub-segment unit, thereby improving the intra-prediction performance as needed. This forecasting method may be referred to as "internal subsegment-based (ISP) or ISP-based forecasting."

[88] The above-described intra prediction methods may be called "intra prediction type" to be distinguished from the intra prediction mode. The intra-prediction type may be referred to by various terms such as "intra-prediction technology" or "additional intra-prediction mode". For example, the intra prediction type (or additional intra prediction mode, etc.) may include at least one of the above LIP, PDPC, MRL and ISP. The general intra prediction method, excluding a specific intra prediction type such as LIP, PDPC, MRL and ISP, may be referred to as "normal intra prediction type". The normal intra-prediction type can generally be applied when the above specific intra-prediction type is not applied, and prediction can be performed based on the above-described intra-prediction mode. Information regarding the type of intra prediction may be encoded by the encoding equipment and included in the bit stream and may be signaled to the decoding equipment. Information regarding the type of intra prediction may be implemented in various forms, for example, as flag information indicating whether or not each type of intra prediction is applied, or index information indicating one of several types of intra prediction. Meanwhile, post-processing filtering can be performed on the extracted prediction sample if necessary.

[89] In particular, the intra prediction procedure may include a step of determining an intra prediction mode/type, a step of retrieving neighboring reference samples, and a step of retrieving prediction samples based on the intra prediction mode/type. In addition, a post-filtering step can be performed on the extracted predictive sample as needed.

[90] The Most Probable Mode (MPM) list for extracting the above-described intra prediction mode may be configured differently according to the intra prediction type. Alternatively, the MPM list can be configured the same regardless of the internal prediction type.

[91] In case that intra prediction is applied, the intra prediction mode that is applied to the current block can be determined using the intra prediction mode of the neighboring block. For example, the decoding equipment may select one of the MPM candidates in an MPM list that is retrieved based on the intra-prediction mode of neighboring blocks (e.g., left and/or top neighboring blocks) relative to the current block and additional mode candidates based on the received MPM -index, or may select one of the remaining intra-prediction modes that are not included in the MPM options (and the planar mode) based on information from the remaining intra-prediction mode. The MPM list may be configured to include or not include planar mode as an option. For example, if an MPM list includes planar mode as a possible option, the MPM list may have 6 possible options, whereas if the MPM list does not include planar mode as an option, the MPM list may have 5 possible options. options. If the MPM list does not include planar mode as an option, a non-planar flag (eg, intra_luma_not_planar_flag) indicating whether the current block's intraprediction mode is not planar mode may be signaled. For example, the MPM flag may be signaled first, and the MPM index and the non-planar flag may be signaled in case the value of the MPM flag is 1. Additionally, the MPM index may be signaled in case the value non-planar flag is 1. Here, the reason why the MPM list is configured to not include the planar mode as an option is to check the planar mode first by signaling the flag (non-planar flag) first, since planar mode is always considered MPM, not that planar mode is not MPM.

[92] For example, whether the intra prediction mode applied to the current block is in the possible MPM options (and planar mode) or the remaining mode can be indicated based on the MPM flag (eg, intra_luma_mpm_flag). An MPM flag value of 1 may represent that the intra-prediction mode for the current block is in the possible MPM options (and in planar mode), and an MPM flag value of 0 may represent that the intra-prediction mode for the current block is not is in possible MPM variants (and in planar mode). a non-planar flag value (eg, intra_luma_not_planar_flag) of 0 may represent that the intra prediction mode for the current block is a planar mode, and a non-planar flag value of 1 may represent that the intra prediction mode for the current block is not a planar mode. The MPM index may be signaled in the form of an mpm_idx or intra_luma_mpm_idx syntax element, and the remaining intra prediction mode information may be signaled in the form of a rem_intra_luma_pred_mode or intra_luma_mpm_remainder syntax element. For example, the remaining intra-prediction mode information may indicate one of the remaining intra-prediction modes that are not included in the MPM candidates (and the planar mode) from all intra-prediction modes by indexing the remaining intra-prediction modes in order of prediction mode numbers. The intra prediction mode may be an intra prediction mode for the luminance signal component (sample). Hereinafter herein, the intra prediction mode information may include at least one of an MPM flag (eg, intra_luma_mpm_flag), a non-planar flag (eg, intra_luma_not_planar_flag), an MPM index (eg, mpm_idx or intra_luma_mpm_idx), or information remaining internal prediction mode (rem_intra_luma_pred_mode or intra_luma_mpm_remainder). In a document, the MPM list may be referred to using various terms, such as "list of possible MPM options", "candModeList", etc. If a MIP is applied to the current block, a separate MPM flag (eg, intra_mip_mpm_flag), MPM index (eg, intra_mip_mpm_idx), and intra prediction mode remaining information (eg, intra_mip_mpm_remainder) may be signaled for the MIP, and the non-planar flag is not sent to service signals.

[93] In other words, if block segmentation for an image is generally performed, the current block to be encoded and the adjacent block have similar image characteristics. Accordingly, there is a high probability that the current block and the neighboring block have the same or similar intra prediction mode. Accordingly, the encoder may use the intra-prediction mode of an adjacent block to encode the intra-prediction mode of the current block.

[94] For example, the encoder/decoder may configure a list of most probable modes (MPM) for the current block. An MPM list can be represented as a list of possible MPM options. Here, the term "MPM" may mean a mode that is used to improve coding efficiency by taking into account the similarity between the current block and a neighboring block during intra prediction mode coding. As described above, the MPM list may be configured to include a planar mode or may be configured to exclude a planar mode. For example, if the MPM list includes a planar mode, the number of possible MPM list options may be 6. Additionally, if the MPM list does not include a planar mode, the number of possible MPM list options may be 5.

[95] The encoder/decoder can configure an MPM list that includes 6 MPMs.

[96] To configure the MPM list, three kinds of modes from default internal modes, adjacent internal modes and extracted internal modes can be considered.

[97] For adjacent internal modes, two adjacent blocks can be considered, i.e. left adjacent block and top adjacent block.

[98] As described above, if the MPM list is configured not to include a planar mode, the planar mode is excluded from the list, and the number of possible MPM list options may be set to 5.

[99] FIG. 4 illustrates an example of 67 intra prediction modes.

[100] Referring to FIG. 4, among the intra prediction modes, the directional mode or the angular mode can distinguish the intra prediction mode having a horizontal directionality and the intra prediction mode having a vertical directionality from each other with respect to the intra prediction mode number 34 having a left upper diagonal prediction direction. In fig. 4, H and V indicate horizontal directivity and vertical directivity, respectively, and numbers -32 to 32 represent offsets in units of 1/32 for a position on the sample grid. Intra prediction modes number 2-33 are horizontally directed, and intra prediction modes numbered 34-66 are vertically directed. The intra prediction mode number 18 and the intra prediction mode number 50 represent the horizontal intra prediction mode and the vertical intra prediction mode, respectively. The intra-prediction mode number 2 may be called the “left lower diagonal intra-prediction mode”, the intra-prediction mode number 34 may be called the “left-upper diagonal intra-prediction mode”, and the intra-prediction mode number 66 may be called the “right upper diagonal intra-prediction mode”.

[101] Additionally, the non-directional mode or non-angular mode of the intra prediction modes may include a DC mode based on the average of neighboring reference samples of the current block or a planar mode based on interpolation.

[102] In traditional intra prediction, the block that is currently to be encoded is considered as one encoding unit, and encoding is performed without segmentation. However, in the intra-sub-segment-based (ISP) encoding mode, intra-predictive encoding can be performed by segmenting the block that is currently to be encoded in the horizontal direction or in the vertical direction. In this case, the reconstructed block can be generated by performing encoding/decoding in units of segmented blocks, and the reconstructed block can be used as a reference block of the next segmented block. For example, the ISP may segment into blocks as shown in Table 1, according to the block size.

[103] Tab. 1

Unit size (CU) Number of segments 4x4 Not segmented 4x8, 8x4 2 All other cases 4

[104] Referring to Table 1, block segmentation according to the ISP may be performed differently depending on the size or shape of the block, for example, block segmentation may be performed as shown in FIG. 5 and 6. In Table 1, the number of segments may indicate the number of sub-segments, and the sub-segments may indicate a block formed by segmentation, a segmented block, or a segmented area.

[105] FIG. 5 shows an example of block segmentation according to ISP in intra-prediction.

[106] When the block size is 4x8 or 8x4, the block may be segmented into two blocks in the ISP. Here, the segmented block may be called a "sub-segment" or "sub-block".

[107] Referring to FIG. 5, horizontal segmentation or vertical segmentation can be performed for a block of size WxH. For example, with horizontal segmentation, a block of size WxH can be segmented into two sub-segments of size WxH/2. With vertical segmentation, a block of size WxH can be segmented into two sub-segments of size W/2xH. Here, W may represent the width of the block, and H may represent the height of the block. Additionally, WxH can be 4x8 or 8x4.

[108] For example, an 8x4 block may be segmented into two sub-segments having a size of 8x2 when segmented horizontally, or into two sub-segments having a size of 4x4 when segmented vertically. Alternatively, a 4x8 block may be segmented into two 4x4 subsegments when partitioned horizontally, or into two 2x8 subsegments when partitioned vertically.

[109] FIG. 6 shows another example of block segmentation according to ISP in intra-prediction.

[110] When the block size is not 4x4, 4x8 or 8x4, the block may be segmented into 4 blocks in the ISP. Thus, in all cases in which the block size is not 4x4, 4x8 or 8x4, the block can be segmented into 4 blocks in the ISP. Here, the segmented block may be called a "sub-segment" or "sub-block".

[111] Referring to FIG. 6, horizontal segmentation or vertical segmentation can be performed for a WxH size block. For example, with horizontal segmentation, a block of size WxH can be segmented into 4 sub-segments of size WxH/4, and with vertical segmentation, a block of size WxH can be segmented into 4 sub-segments of size W/4xH. Here, W may represent the width of the block, and H may represent the height of the block. Additionally, WxH may not be 4x4, 4x8, or 8x4.

[112] In intra-prediction, for ISP, an MPM list can be generated according to each segmentation method (horizontal segmentation and vertical segmentation) to reduce coding complexity, and an optimal mode can be generated by comparing a suitable prediction mode from the prediction modes in the generated MPM list in terms of rate dependent distortion (RDO). Thus, the best candidate or optimal intra-prediction mode can be retrieved from among the candidates or intra-prediction modes included in the MPM list, and intra-forecasting can be performed based on it. Here, the best candidate or the best intra prediction mode may be extracted using distortion depending on the bit rate. Alternatively, the best candidate may be extracted by comparing the distortion as a function of the bit rate for each candidate in the MPM list.

[113] In addition, when the above-described MRL is used in intra prediction, the ISP may not be used. Thus, when intraprediction is performed using multiple reference lines, ISP cannot be used in conjunction with it. Alternatively, when internal prediction is performed using a zero reference line, ISP can be used. Here, the case in which intra-forecasting is performed using the zero reference line may indicate a case in which the MRL is not used, or a case in which the zero reference line is used, since the value of the intra_luma_ref_idx field is 0 even if the MRL is used. Additionally, when ISP is used in intraprediction, PDPC may not be used.

[114] In an embodiment, ISP-related information for using the ISP may include information regarding whether or not the ISP is being used. Here, ISP-related information may be referred to as “internal sub-segment information” or “internal sub-segment information”. The information regarding whether or not the ISP is applied may indicate information regarding whether or not the ISP should be applied to the current block and may include an intra_subpartitions_mode_flag. Information regarding whether or not an ISP is used may be expressed as an ISP flag. The ISP-related information may further include information regarding the splitting direction when the ISP is applied to the current block by information regarding whether or not the ISP is applied. The split direction information may indicate information regarding whether the segment is horizontal or vertical and may include an intra_subpartitions_split_flag. An embodiment may encode/decode ISP-related information.

[115] ISP-related information may be signaled in block units. Alternatively, it may be signaled via coding unit (CU) syntax. Alternatively, the encoding unit syntax may include information regarding whether or not an ISP is used. Alternatively, the encoding unit syntax may include information regarding whether or not an ISP is applied and information regarding the direction of partitioning. Alternatively, the coding unit syntax may be generated in the encoding equipment and signaled to the decoding equipment. Alternatively, the decoding equipment may perform intra prediction or decoding based on the signaled coding unit syntax. For example, the encoding unit syntax may include Table 2 below. Alternatively, the encoding unit syntax may include ISP-related information as shown in Table 2 below.

[116] Tab. 2

[117] In Table 2, information regarding whether or not an ISP is applied may include an intra_subpartitions_mode_flag field, and information regarding a split direction may include an intra_subpartitions_split_flag field.

[118] For example, when ISP is applied, the intra-prediction mode for the current block is equally applied to the sub-segments, and the intra-prediction performance can be improved by retrieving and using a neighboring reference sample in the units of the sub-segments. Thus, when ISP is applied, the residual sample processing procedure can be performed in sub-segment units.

[119] In other words, internal prediction samples can be retrieved for each sub-segment, and reconstructed samples can be obtained by summing the residual signals (residual samples) for the corresponding sub-segment. Here, the residual signal (residual samples) can be extracted through an inverse transform/inverse quantization procedure or the like. based on residual information (quantized transform coefficient information or residual encoding syntax) in the above-described bit stream.

[120] Thus, the prediction samples for the first sub-segment can be retrieved, the residual samples can be retrieved, and based on them, the reconstructed samples for the first sub-segment can be retrieved. In this case, when the prediction samples for the second subsegment are retrieved, some reconstructed samples in the first subsegment (eg, left or top adjacent reference samples of the second subsegment) can be used as adjacent reference samples for the second subsegment. Likewise, retrieval of predicted samples for the second sub-segment may be performed, and retrieval of residual samples may be performed, and based thereon, reconstructed samples for the second sub-segment may be retrieved. In this case, when the prediction samples for the third subsegment are retrieved, some reconstructed samples in the second subsegment (eg, left or top adjacent reference samples of the third subsegment) can be used as adjacent reference samples for the third subsegment. Likewise, reconstructed samples can be retrieved by performing an identical or similar process for other subsegments.

[121] FIG. 7 shows an example of TT and BT segmentation that is not resolved based on the VPDU size.

[122] Virtual Pipeline Data Units (VPDUs) may be specified for pipelining within a frame. VPDUs may be specified as non-overlapping units in the same frame. In a hardware decoder, successive VPDUs can be processed simultaneously through multiple pipeline stages. The VPDU size can be approximately proportional to the buffer size in most pipeline stages. Therefore, keeping the VPDU size small can be important when considering buffer size from a hardware perspective. In most hardware decoders, the VPDU size can be set to the maximum TB (translation block) size. For example, the VPDU size may be 64×64 (64×64 luminance samples). However, as an example, the VPDU size may vary (increase or decrease) based on the segment according to a ternary tree (TT) and/or a binary tree (BT).

[123] Meanwhile, referring to FIG. 7, in order to maintain the VPDU size as a 64x64 luminance signal sample size, at least one of the restrictions shown in Table 3 may be applied.

[124] Tab. 3

- TT splitting is not allowed for CUs with width or height or both width and height equal to 128.

- For CU 128xN with N≤64 (i.e. with a width equal to 128 and a height less than 128), horizontal BT is not permitted.

- For CU Nx128 with N≤64 (i.e. height equal to 128 and width less than 128), vertical BT is not allowed.

[125] In other words, when using an ISP, the intra-luminance signal prediction block may be divided into two or four sub-segments vertically or horizontally according to the block size. For example, the minimum block size for an ISP may be 4x8 (or 8x4), and when the block size exceeds 4x8 (or 8x4), the corresponding block may be divided into four sub-segments.

[126] In addition, when ISP is used for a block size of Mx128 (M is a natural number less than or equal to 64) or 128xN (N is a natural number less than or equal to 64), a potential problem may arise regarding the virtual data pipeline unit (VDPU) size 64x64. For example, an Mx128 size coding unit (CU) having a single tree may include a Mx128 size luma transform unit (TB) and two M/2x64 TBs of chrominance signals. Here, a single tree may represent a partition by an identical tree between a luma CU and a corresponding chrominance CU. Here, when the CU uses ISP, a TB of luma signals can be segmented into 4 Mx32 sized TBs (only horizontal segment is possible here), each of which can be smaller than a 64x64 block. However, in the current ISP design, the color signal blocks, i.e. TBs may not be segmented. Thus, all chrominance signal components can be larger than a 32x32 block. Likewise, a similar situation may occur when a 128xN size CU uses an ISP, and the above two cases may pose a problem in a 64x64 decoder pipeline.

[127] Accordingly, an embodiment may provide a method taking into account the size of the VPDU in the process of applying the ISP to the CU. Thus, an embodiment may consider a Virtual Pipeline Data Unit (VPDU) during an ISP application. Alternatively, an embodiment may provide a VPDU-aware decoding pipeline when executing the ISP. Alternatively, an embodiment may provide a decoding pipeline-aware ISP method for a VPDU-aware hardware (HW) implementation. Alternatively, in one embodiment, a hardware-aware ISP method may be provided, i.e. transmission method in ISP signaling taking into account VPDUs.

[128] FIG. 8 shows an example of an HW processing pipeline.

[129] In hardware aware (HW) processing, the pipeline may be as shown in FIG. 8. Alternatively, decoding may be performed in VPDU units, as shown in FIG. 8. Alternatively, according to an embodiment, each block may be processed sequentially according to the arrow. Alternatively, according to an embodiment, some blocks may be processed in parallel according to the order of the arrows.

[130] For example, it is assumed that the size of the luma signal CU is 128×128, the size of the corresponding chrominance signal CU is 64×64, and it can be divided into four blocks according to the quadtree. In this case, the luminance signal CU may be divided into 4 blocks of size 64×64, and the chrominance signal CU may be divided into 4 blocks of size 32×32. Here, four blocks of size 64x64 divided from CU luminance signals are represented as a zero block, a first block, a second block and a third block in the order of top left, top right, bottom left and bottom right, and 4 blocks of size 32x32. divided from the chroma CU, the blocks may be represented as a fourth block, a fifth block, a sixth block and a seventh block in the order of upper left, upper right, lower left and lower right.

[131] In this case, the processing of each of the blocks may have a zero block order, i.e. as a component of the luminance signal, the fourth block, i.e. as a component of the chrominance signal corresponding to the zero block of the first block, i.e. as a component of the luminance signal, the fifth block, i.e. as a component of the chrominance signal corresponding to the first block of the second block, i.e. as a component of the luminance signal, the sixth block, i.e. as a component of the chrominance signal corresponding to the second block of the third block, i.e. as a component of the luminance signal, and the seventh block, i.e. as a component of the chrominance signal corresponding to the third block. Alternatively, VPDU processing may be performed as described above. Here, the VPDU size may be 64x64. Alternatively, the VPDU size for the luma signal component may be 64×64, and the VPDU size for the chrominance signal component may be 32×32. Alternatively, the VPDU may be equal to the maximum transform unit size (per component).

[132] However, as described above, when ISP is applied to a CU, a problem may arise. Thus, when ISP is applied to a block having a size larger than the VPDU size, a problem may appear from a hardware point of view, which is described in detail with reference to FIG. 9.

[133] FIG. 9 shows an example of a pipeline when the ISP is applied to a block larger than the VPDU size.

[134] For example, it is assumed that the size of the luma CU is 128×128, the size of the corresponding chrominance CU is 64×64, it has a single tree structure, and is divided into two blocks according to a horizontal binary tree. In this case, the luminance signal CU may be divided into two blocks of size 128×64, and the chrominance signal CU may be divided into two blocks of size 64×32. Here, when considering the case of applying the ISP to the current CU, the ISP may be applied to an upper block of two blocks divided from the luminance signal CU, and the upper block may be divided into four sub-segments of size 32×62 according to the application of the ISP. However, the ISP may only be applied to the luma signal component according to the ISP design decision, and in this case, the ISP may not be applied to the chroma CU or to two blocks divided from the chrominance CU. Here, the four sub-segments may be referred to as "first sub-segment", "second sub-segment", "third sub-segment" and "fourth sub-segment" from left to right.

[135] In this case, the first sub-segment and the second sub-segment, which are luminance signal components, may be processed first in consideration of the 64×64 VPDU. Here, since the chrominance signal CU is not divided or indicated only to/as regions or samples corresponding to the first and second sub-segments, the chrominance signal components corresponding to the first and second sub-segments may not be processed immediately. That is, after the third sub-segment and the fourth sub-segment, which are luma signal components, are processed, the chrominance signal component corresponding to the first to fourth sub-segments (the upper block of the two blocks separated from the chrominance signal CU) can be processed. However, the VPDU may have a size of 32×32 for the chrominance signal component, in this case, since the size of the upper block of the two blocks divided from the chrominance signal CU is 64×32, it may not be performed in VPDU pipeline units. Alternatively, coding efficiency may decrease.

[136] For example, in the above description, since the VPDU size may be defined as the maximum transform block size, referring to Table 4, the maximum transform block size may have a preset maximum transform block size for the luminance signal component and may be half the size of the preset a specified maximum transform block size for a chrominance component. Here, the preset maximum transform block size may be 64×64, but is not limited to this.

[137] Tab. 4

8.4.4. Decoding process for indoor units

8.4.4.1. General decoding process for indoor units

The inputs to this process are as follows:

- sample location (xTb0, yTb0) indicating the top left sample of the current transform block relative to the top left sample of the current frame,

- nTbW variable indicating the width of the current transformation block,

- nTbH variable indicating the height of the current transformation block,

- predModeIntra variable indicating the internal forecasting mode,

- cIdx variable indicating the color component of the current block.

The output of this process is the modified reconstructed frame before in-loop filtering. The maximum size of the transform block maxTbSize is retrieved as follows:

- maxTbSize=(cIdx==0)? MaxTbSizeY:MaxTbSizeY/2

[138] In Table 4, the cIdx field value may include a color component index, or information indicating the color component may indicate a luminance signal component (Y) if the cIdx field value is 0, and may indicate a chrominance signal component (Cb). , if the cIdx field value is 1, and may indicate the chrominance component (Cr) if the cIdx field value is 2.

[139] In other words, the maximum size (maxTbSize) of the transform block may be defined as a preset MaxTbSizeY in the case of a luminance signal component (cIdx==0) and may be defined as a preset MaxTbSizeY/2 if it is not a luminance signal component (cIdx!==0).

[140] An embodiment according to this document may provide a method for disabling ISP coding for a block having a size of 128xN and/or Nx128 to resolve the above complexity or problem. Alternatively, according to an embodiment, the block size to which ISP cannot be applied can be specified. Alternatively, according to an embodiment, a condition under which ISP cannot be applied based on the block size (luminance signal components) can be set.

[141] For example, an ISP flag (eg, intra_subpartitions_mode_flag) indicating whether or not ISP is applied to the current block (eg, CU/CB) may be signaled and/or parsed when both conditions i) the width of the current block is less than or equal to MaxTbSizeY (ie, the width/height of the maximum transform block for the luminance signal component), and ii) the height of the current block is less than or equal to MaxTbSizeY. However, if the above condition is not satisfied, the ISP flag may not be signaled and/or parsed and its value may be evaluated to 0.

[142] Thus, if both conditions are satisfied: i) the current block's width is less than or equal to MaxTbSizeY, and ii) the current block's height is less than or equal to MaxTbSizeY, the ISP is activated. Otherwise, the ISP may not be available (deactivated). Here, it is described that the width and height of the maximum transform block (TB) are set to be identical, but the present disclosure is not limited to this, and the maximum width and height of the TB may be set to be different.

[143] In other words, only when the width and height of the current block are less than or equal to the maximum transform block size, ISP can be applied to the current block. Alternatively, ISP-related information for the current block may be signaled. Alternatively, the ISP may be applied to a block of the current luma signal component only when the width and height of the current luma signal component block is less than or equal to the maximum transform block size. Alternatively, ISP-related information for the current luminance signal component block may be signaled.

[144] For example, ISP-related information may be signaled in block units based on the conditions described above. Alternatively, when the above conditions are satisfied, the information may be signaled through coding unit (CU) syntax. Alternatively, the encoding unit syntax may include information regarding whether or not the ISP is applied when the above conditions are met. Alternatively, when the above-described conditions are satisfied, the encoding unit syntax may include information regarding whether or not an ISP is applied and information regarding the partitioning direction. Alternatively, the coding unit syntax may be generated in the encoding equipment and signaled to the decoding equipment. Alternatively, the decoding equipment may perform intra prediction or decoding based on the signaled coding unit syntax.

[145] For example, the encoding unit syntax, taking into account the above-described conditions, may be represented as shown in Table 5 below. In addition, it may include Table 5. Alternatively, the encoding unit syntax may include ISP-related information, as shown in table 5 below.

[146] Tab. 5

[147] Referring to Table 5, in the embodiment, when (intra_luma_ref_idx[x0][y0]==0 andand (cbWidth<=MaxTbSizeY andand cbHeight<=MaxTbSizeY) andand (cbWidth*cbHeight>MinTbSizeY*MinTbSizeY))) is satisfied, it may include information regarding whether or not the ISP is being used, including the intra_subpartitions_mode_flag field. Alternatively, if(cbWidth<=MaxTbSizeY andand cbHeight<=MaxTbSizeY) is satisfied, information regarding whether or not the ISP is applied, including the intra_subpartitions_mode_flag field, may be included. Here, cbWidth may indicate the width of the current block, chHeight may indicate the height of the current block, and MaxTbSizeY may indicate the maximum size of the transform block.

[148] In addition, the embodiment includes a field, when the intra_subpartitions_mode_flag field has a value of 1, or when (intra_subpartitions_mode_flag[x0][y0]==1 andand cbWidth<=MaxTbSizeY andand cbHeight<=MaxTbSizeY) is satisfied, information regarding the direction a split that includes the intra_subpartitions_split_flag field may be included.

[149] For example, the semantics of intra_subpartitions_mode_flag and intra_subpartitions_split_flag could be represented as shown in Table 6.

[150] Tab. 6

- intra_subpartitions_mode_flag[x0][y0], equal to 1, indicates that the current internal coding unit is segmented into subsegments of the rectangular transform block NumIntraSubPartitions[x0][y0]; intra_subpartitions_mode_flag[x0][y0] equal to 0 indicates that the current intracoding unit is not segmented into rectangular transform block subsegments.

- intra_subpartitions_split_flag[x0][y0] indicates whether the split type of internal subsegments is horizontal or vertical. When intra_subpartitions_split_flag[x0][y0] is not present, it may be logically inferred to be 0.

The IntraSubPartitionsSplitType variable specifies the type of partition used for the current luma coding block, as illustrated in the following table. IntraSubPartitionsSplitType is retrieved as follows:

- If intra_subpartitions_mode_flag[x0][y0] is 0, IntraSubPartitionsSplitType is set to 0.

- Otherwise, IntraSubPartitionsSplitType is set to 1+intra_subpartitions_split_flag[x0][y0].

Associating titles with IntraSubPartitionsSplitType IntraSubPartitionsSplitType Name IntraSubPartitionsSplitType 0 ISP_NO_SPLIT 1 ISP_HOR_SPLIT 2 ISP_VER_SPLIT

[151] FIG. 10 schematically illustrates a method for encoding images by encoding equipment according to the present document.

[152] The method disclosed in FIG. 10 may be performed by the encoding equipment disclosed in FIG. 2. In particular, for example, steps S1000-S1020 in FIG. 10 may be performed by the encoding equipment prediction unit, step S1030 may be performed by the encoding equipment subtractor, and step S1040 may be performed by the entropy encoder of the encoding equipment. In addition, although not shown, the process of generating residual information for the current block based on the residual sample may be performed by the encoding equipment converter.

[153] The encoding equipment extracts the intra prediction mode of the current block (S1000). For example, the encoding equipment may derive the intra-prediction mode of the current block based on neighboring samples of the current block. Alternatively, the encoding equipment may derive reference samples of the current block based on neighboring samples of the current block and may determine an intra prediction mode accordingly. Alternatively, the encoding equipment may retrieve an MPM list to determine the intra prediction mode of the current block and may generate MPM list related information to determine the intra prediction mode from the MPM list. For example, the information associated with the MPM lists may include MPM index information, and an intra prediction mode may be determined from the MPM list based on the MPM index information.

[154] The encoding equipment extracts ISP related information for the current block based on the size of the current block and the maximum size of the transform block (S1010). For example, encoding equipment may extract ISP-related information by comparing the width and/or height of the current block with the maximum transform block size. Alternatively, when the width of the current block is less than or equal to the maximum transform block size, and the height of the current block is less than or equal to the maximum transform block size, the encoding equipment may extract ISP-related information for the current block. Alternatively, the encoding equipment may extract ISP-related information regarding a current block having a width less than or equal to the maximum transform block size and a height less than or equal to the maximum transform block size. Here, the maximum size of the transform block may be expressed as maxTbsizeY, and the width and height may be identical. Alternatively, the maximum transform block size may have different widths and heights. For example, when the width of the current block is less than or equal to the width of the maximum transform block, and the height of the current block is less than or equal to the height of the maximum transform block, the encoding equipment may extract ISP-related information for the current block.

[155] For example, the ISP-related information may include information regarding whether or not an ISP is applied to the current block. Thus, the encoding equipment can extract information regarding whether or not an ISP is applied to the current block by comparing the width and/or height of the current block with the maximum transform block size. Alternatively, when the width of the current block is less than or equal to the maximum transform block size, and the height of the current block is less than or equal to the maximum transform block size, the encoder may extract information regarding whether or not the ISP is applied to the current block. Alternatively, the encoder may extract ISP-related information for a current block having a width less than or equal to the maximum transform block size and a height less than or equal to the maximum transform block size, wherein the ISP-related information includes information regarding whether or no ISP to the current block. Here, information regarding whether or not an ISP is applied to the current block may include an intra_subpartitions_mode_flag field. Here, the intra_subpartitions_mode_flag field may be represented as an ISP flag. For example, the intra_subpartitions_mode_flag field may have a value of 0 or 1. Additionally, information regarding whether or not an ISP is applied to the current block may be included in the encoding unit syntax.

[156] For example, information regarding whether or not the ISP is applied may include information regarding whether the ISP is applied to the current block or information regarding whether the ISP is not applied to the current block. For example, when information regarding whether or not an ISP is applied includes an intra_subpartitions_mode_flag field, information regarding the application of an ISP to the current block may indicate a case in which the value of the intra_subpartitions_mode_flag field is 1, and information regarding not applying an ISP to the current block may indicate a case in which in which the value of the intra_subpartitions_mode_flag field is 0. Alternatively, information for applying an ISP in the current block may indicate a case in which the value of the intra_subpartitions_mode_flag field is 0, and information regarding not applying an ISP to the current block may indicate a case in which the value of the intra_subpartitions_mode_flag field is 1. In other words , information regarding the application of an ISP to the current block or information regarding not applying an ISP to the current block may be indicated by the value of the intra_subpartitions_mode_flag field.

[157] For example, the ISP-related information may include information regarding whether or not an ISP is applied to the current block and information regarding the splitting direction. Alternatively, when information regarding whether or not an ISP is applied includes information for applying an ISP in the current block, the ISP-related information may include information regarding a partitioning direction. Alternatively, when the ISP-related information includes information for applying the ISP in the current block, information regarding the partitioning direction may also be included. Alternatively, the ISP-related information may further include information regarding the splitting direction based on information regarding whether or not the ISP is applied, indicating that the ISP is applied to the current block. For example, information regarding the direction of splitting may include an intra_subpartitions_split_flag field. Here, the intra_subpartitions_split_flag field may be represented as a split direction flag. For example, the intra_subpartitions_split_flag field may have a value of 0 or 1. For example, when information regarding the split direction includes an intra_subpartitions_split_flag field, information in which the split direction is horizontal may indicate the case in which the value of the intra_subpartitions_split_flag field is 0, and information in which the split direction is vertical may indicate when the intra_subpartitions_split_flag field value is 1. Alternatively, information in which the split direction is horizontal may indicate the case in which the intra_subpartitions_split_flag field value is 1, and information in which the split direction is vertical may indicate the case in which the value of the intra_subpartitions_split_flag field is 0.

[158] Alternatively, when the width or height of the current block exceeds the maximum transform block size, the ISP-related information may include information regarding not to apply the ISP to the current block. For example, when the width or height of the current block exceeds the maximum transform block size, information regarding whether or not an ISP is applied to the current block may not be included. Alternatively, it may not be retrieved. Here, when information regarding whether or not the ISP is applied is not retrieved, it may be implied that information regarding whether the ISP is not applied to the current block is retrieved.

[159] The encoding equipment predictively samples the current block based on the intra prediction mode and ISP related information (S1020). For example, when applying the ISP to the current block based on the ISP-related information, the encoding equipment may determine the intra-prediction mode in the same manner for the divided sub-segments. Alternatively, the internal forecasting regime may apply equally to sub-segments. For example, when the ISP is applied to the current block, the encoding equipment may retrieve the MPM list according to the splitting direction. Alternatively, the encoding device may further obtain MPM list-related information and may retrieve another MPM list according to the partitioning direction. For example, different MPM lists may be retrieved when partitioning in the horizontal direction and when partitioning in the vertical direction. The encoding equipment may extract the MPM list differently according to the partitioning direction, determine an inter-prediction mode based on the extracted MPM list, and apply the determined intra-prediction mode to the sub-segments equally.

[160] For example, the prediction sample may be generated based on the intra prediction mode and subsegments of the current block. In addition, subsegments of the current block can be retrieved based on information regarding the direction of the split. Thus, subsegments of the current block may be retrieved by dividing in the vertical direction or may be retrieved by dividing in the horizontal direction. For example, the MPM list may be retrieved differently according to the split direction, as described above.

[161] For example, the subsegments of the current block may include a first subsegment and a second subsegment. Alternatively, the subsegments of the current block may include a first subsegment, a second subsegment, a third subsegment, and a fourth subsegment. Here, the number of sub-segments may be determined based on the size of the current block.

[162] For example, when the current block is divided into a first sub-segment and a second sub-segment, a prediction sample of the first sub-segment may be generated based on the intra-prediction mode and neighboring samples of the first sub-segment, a prediction sample of the second sub-segment may be generated based on the intra-prediction mode and the prediction sample of the first sub-segment. Thus, the prediction sample of the first sub-segment may be generated based on the left neighbor sample and/or the top neighbor sample of the first sub-segment according to the intra prediction mode. Alternatively, the reference sample of the first sub-segment may be extracted based on the left neighbor sample and/or the top neighbor sample of the first sub-segment according to the intra prediction mode, and the prediction sample of the first sub-segment may be generated based on the extracted reference sample. A reference sample of the second sub-segment may be extracted based on a left adjacent sample and/or an upper adjacent sample of the second sub-segment according to an intra prediction mode, and a prediction sample of the second sub-segment may be generated based on the extracted reference sample. Here, the prediction sample of the first sub-segment may be used as an adjacent sample of the second sub-segment. Alternatively, it may be retrieved as a reference sample of the second sub-segment. Here, the intra prediction mode of the second sub-segment may be identical to the intra prediction mode of the first sub-segment.

[163] Alternatively, the encoding equipment may directly use the prediction sample as a reconstructed sample according to the prediction mode. In addition, the encoding equipment may generate a predicted block of the current block based on the predicted sample. In addition, the encoding equipment may generate a residual sample based on the original sample and the generated predictive sample. The encoding equipment may generate residual information based on the residual sample. The residual information may include conversion factors for the residual sample. The encoding equipment can extract the reconstructed sample based on the predictive sample and the residual sample. Thus, the encoding equipment can extract the reconstructed sample by summing the prediction sample and the residual sample. Here, the encoding equipment can generate a residual block based on the original block and the predicted block, and can also generate information regarding the remainder based on the residual block.

[164] The encoding equipment extracts the residual sample based on the predictive sample (S1030). For example, the encoding equipment may derive a residual sample for the current block based on the original sample and the predictive sample for the current block. However, the encoding equipment may use the predictive sample as the reconstructed sample, in which case, the encoding equipment may omit the process of extracting the residual sample based on the predictive sample. Alternatively, although not shown, the encoding equipment may generate a reconstructed sample by adding the residual sample with the predictive sample.

[165] The encoding equipment encodes image information including prediction mode information regarding the intra prediction mode, ISP related information, and information regarding the residual sample (S1040). For example, the encoding equipment may determine a prediction mode of the current block and may generate information indicating the prediction mode. Alternatively, the encoding equipment may determine the intra prediction mode of the current block and may generate prediction mode information of the current block including information regarding the intra prediction mode. The prediction mode information may include information regarding prediction of the current block or various information for prediction. The prediction mode information may include MPM list-related information. Alternatively, the MPM list-related information may include MPM index information for an intra prediction mode used for intra prediction of the current block in the MPM list.

[166] In addition, the encoder can generate ISP-related information. The ISP-related information may include information regarding whether or not an ISP is applied to the current block. Alternatively, the ISP-related information may include information regarding whether or not an ISP is applied to the current block and information regarding the splitting direction. Alternatively, the ISP-related information may include information indicating that the ISP is not applied to the current block. In addition, the encoding equipment can generate information regarding the remainder. The residual information may include information regarding a residual sample (or an array of residual samples). Alternatively, the residual information may include conversion factors for the residual sample.

[167] The encoding equipment may generate a bit stream by encoding image information including all or part of the above-described information. Alternatively, it may be output in the form of a bitstream. In addition, the bit stream may be transmitted to decoding equipment via a network or storage medium. Alternatively, the bitstream may be stored on a computer-readable storage medium.

[168] FIG. 11 schematically illustrates a method for decoding images by decoding equipment according to the present document.

[169] The method disclosed in FIG. 11 may be performed by the decoding equipment illustrated in FIG. 3. In particular, for example, S1100 according to FIG. 11 may be performed by the entropy decoder of the decoding equipment, and S1110 to S1130 may be performed by the prediction unit of the decoding equipment. In addition, although not shown, the process of obtaining image information including information regarding the prediction of the current block and information regarding the remainder through the bit stream may be performed by the entropy decoder of the decoding equipment, and the process of extracting a residual sample for the current block based on the information regarding the remainder may be performed by an inverse converter of the decoding equipment, and the process of generating a reconstructed frame based on the prediction sample and the residual sample may be performed by an adder of the decoding equipment.

[170] The decoding equipment receives prediction mode information of the current block (S1100). For example, the prediction mode information may include information regarding the intra prediction mode of the current block. Alternatively, information regarding prediction of the current block or various information for prediction may be included. For example, decoding equipment may obtain prediction mode information from the bit stream. Alternatively, the decoding equipment may entropy decode the bit stream to obtain prediction mode information. Alternatively, the decoding equipment may obtain prediction mode information by parsing the bit stream.

[171] The decoding equipment receives ISP related information regarding the current block based on the size of the current block and the maximum size of the transform block (S1110). For example, decoding equipment may receive or retrieve ISP-related information by comparing the width and/or height of the current block with the maximum transform block size. Alternatively, when the width of the current block is less than or equal to the maximum transform block size, and the height of the current block is less than or equal to the maximum transform block size, the decoding equipment may receive ISP-related information regarding the current block. Alternatively, the decoding equipment may receive ISP-related information regarding a current block having a width less than or equal to the maximum transform block size and a height less than or equal to the maximum transform block size. Here, the maximum size of the transform block may be expressed as maxTbsizeY, and the width and height may be identical. Alternatively, the maximum transform block size may have different widths and heights. For example, when the width of the current block is less than or equal to the width of the maximum transform block, and the height of the current block is less than or equal to the height of the maximum transform block, the decoding equipment may extract ISP-related information for the current block.

[172] For example, the ISP-related information may include information regarding whether or not an ISP is applied to the current block. Thus, the decoding equipment may compare the width and/or height of the current block with the maximum transform block size to receive information regarding whether or not the ISP is applied to the current block. Alternatively, when the width of the current block is less than or equal to the maximum transform block size, and the height of the current block is less than or equal to the maximum transform block size, the decoding equipment may receive information regarding whether or not an ISP is applied to the current block. Alternatively, the decoding equipment may receive ISP-related information for a current block having a width less than or equal to the maximum transform block size and a height less than or equal to the maximum transform block size, and the ISP-related information may include information regarding whether or no ISP to the current block. Here, information regarding whether or not an ISP is applied to the current block may include an intra_subpartitions_mode_flag field. Here, the intra_subpartitions_mode_flag field may be represented as an ISP flag. For example, the intra_subpartitions_mode_flag field may have a value of 0 or 1. Additionally, information regarding whether or not an ISP applies to the current block may be included in the encoding unit syntax.

[173] For example, information regarding whether or not the ISP is applied may include information regarding whether the ISP is applied to the current block or information regarding whether the ISP is not applied to the current block. For example, when information regarding whether or not an ISP is applied includes an intra_subpartitions_mode_flag field, information regarding the application of an ISP to the current block may indicate a case in which the value of the intra_subpartitions_mode_flag field is 1, and information regarding not applying an ISP to the current block may indicate a case in which in which the value of the intra_subpartitions_mode_flag field is 0. Alternatively, information regarding the application of an ISP to the current block may indicate a case in which the value of the intra_subpartitions_mode_flag field is 0, and information regarding not applying an ISP to the current block may indicate a case in which the value of the intra_subpartitions_mode_flag field is 1. In other words , information regarding the application of an ISP to the current block or information regarding not applying an ISP to the current block may be indicated by the value of the intra_subpartitions_mode_flag field.

[174] For example, the ISP-related information may include information regarding whether or not an ISP is applied to the current block and information regarding the splitting direction. Alternatively, when information regarding whether or not an ISP is applied includes information for applying an ISP in the current block, the ISP-related information may include information regarding a partitioning direction. Alternatively, when the ISP-related information includes information for applying the ISP in the current block, information regarding the partitioning direction may also be included. Alternatively, the ISP-related information may further include information regarding the splitting direction based on information regarding whether or not the ISP is applied, indicating that the ISP is applied to the current block. For example, information regarding the direction of splitting may include an intra_subpartitions_split_flag field. Here, the intra_subpartitions_split_flag field may be represented as a split direction flag. For example, the intra_subpartitions_split_flag field may have a value of 0 or 1. For example, when information regarding the split direction includes an intra_subpartitions_split_flag field, information in which the split direction is horizontal may indicate the case in which the value of the intra_subpartitions_split_flag field is 0, and information in which the split direction is vertical may indicate when the intra_subpartitions_split_flag field value is 1. Alternatively, information in which the split direction is horizontal may indicate the case in which the intra_subpartitions_split_flag field value is 1, and information in which the split direction is vertical may indicate the case in which the value of the intra_subpartitions_split_flag field is 0.

[175] Alternatively, when the width or height of the current block exceeds the maximum transform block size, the ISP-related information may include information regarding not to apply the ISP to the current block. For example, when the width or height of the current block exceeds the maximum transform block size, information regarding whether or not an ISP is applied to the current block may not be included. Alternatively, it may not be retrieved. Here, when information regarding whether or not the ISP is applied is not retrieved, it may be implied that information regarding whether the ISP is not applied to the current block is retrieved.

[176] The decoding equipment extracts the intra prediction mode of the current block based on the prediction mode information (S1120). For example, the prediction mode information may include information regarding the intra prediction mode of the current block, and the decoding equipment may determine the intra prediction mode of the current block based on the information regarding the intra prediction mode of the current block. Alternatively, the decoding equipment may further obtain MPM list-related information and may determine an intra prediction mode based on the MPM list-related information. Alternatively, the information associated with the MPM lists may include MPM index information, and the decoding equipment may determine the intra prediction mode of the current block in the MPM list based on the MPM index information.

[177] The decoding equipment predictively samples the current block based on the intra prediction mode and ISP related information (S1130). For example, when applying the ISP to the current block based on the ISP-related information, the decoding equipment may determine the intra prediction mode in the same way for the divided sub-segments. Alternatively, the internal forecasting regime may apply equally to sub-segments. For example, when applying the ISP to the current block, the decoding equipment may extract the MPM list according to the splitting direction. Alternatively, the decoding equipment may further obtain MPM list-related information and may retrieve another MPM list according to the partitioning direction. For example, different MPM lists may be retrieved when it is split in the horizontal direction and when it is split in the vertical direction. The decoding equipment may extract the MPM list differently according to the partitioning direction, determine the intra prediction mode based on the extracted MPM list, and apply the determined intra prediction mode to the sub-segments equally.

[178] For example, the prediction sample may be generated based on the intra-prediction mode and sub-segments of the current block. In addition, subsegments of the current block can be retrieved based on information regarding the direction of the split. Thus, subsegments of the current block may be retrieved by dividing in the vertical direction or may be retrieved by dividing in the horizontal direction. For example, the MPM list may be retrieved differently according to the split direction, as described above.

[179] For example, the subsegments of the current block may include a first subsegment and a second subsegment. Alternatively, the subsegments of the current block may include a first subsegment, a second subsegment, a third subsegment, and a fourth subsegment. Here, the number of sub-segments may be determined based on the size of the current block.

[180] For example, when the current block is divided into a first sub-segment and a second sub-segment, a prediction sample of the first sub-segment may be generated based on the intra prediction mode and neighboring samples of the first sub-segment, a prediction sample of the second sub-segment may be generated based on the intra prediction mode and the prediction sample of the first sub-segment. Thus, the prediction sample of the first sub-segment may be generated based on the left neighbor sample and/or the top neighbor sample of the first sub-segment according to the intra prediction mode. Alternatively, the reference sample of the first sub-segment may be extracted based on the left neighbor sample and/or the top neighbor sample of the first sub-segment according to the intra prediction mode, and the prediction sample of the first sub-segment may be generated based on the extracted reference sample. A reference sample of the second sub-segment may be extracted based on a left adjacent sample and/or an upper adjacent sample of the second sub-segment according to an intra prediction mode, and a prediction sample of the second sub-segment may be generated based on the extracted reference sample. Here, the prediction sample of the first sub-segment may be used as an adjacent sample of the second sub-segment. Alternatively, it may be retrieved as a reference sample of the second sub-segment. Here, the intra prediction mode of the second sub-segment may be identical to the intra prediction mode of the first sub-segment.

[181] Alternatively, the decoding equipment may directly use the prediction sample as a reconstructed sample according to the prediction mode. In addition, the decoding equipment may generate a predicted block of the current block based on the predicted sample. In addition, the decoding equipment may generate a reconstructed sample by adding the residual sample with the predictive sample. When there is a residual sample for the current block, the decoding equipment can obtain information regarding the residual for the current block from the bit stream. The residual information may include conversion factors for the residual sample. The decoding equipment may extract a residual sample (or an array of residual samples) for the current block based on the residual information. The decoding equipment can generate a reconstructed sample based on the prediction sample and the residual sample, and can extract a reconstructed block or a reconstructed frame based on the reconstructed sample. Thereafter, as described above, the decoding equipment may apply an in-loop filtering procedure, eg, a deblocking and/or SAO filtering procedure, to the reconstructed frame to improve the subjective/objective quality of the frames, if necessary.

[182] In the above embodiment, the methods are described on the basis of a method flowchart as a sequence of steps or blocks, but this document is not limited to the order of the steps, and some steps may occur in a different order or simultaneously with other steps as described above . In addition, those skilled in the art will understand that the steps shown in the flowchart are not the only ones possible, and that other steps may be included, or that one or more steps of the flowchart may be deleted without impact on the scope of this document.

[183] The method according to this document described above may be implemented in the form of software and encoding equipment, and/or decoding equipment according to this document may be included in, for example, a television, computer, smartphone, set-top box, display device that performs the processing images.

[184] When embodiments of the present document are implemented in software, the above-described method may be implemented through modules (processes, functions, etc.) that perform the functions described above. Such modules may be stored in a memory device and executed by a processor. The storage device may be located inside or outside the processor, and the storage device may be coupled to the processor using various known means. The processor may include an application specific integrated circuit (ASIC), other chipsets, logic circuitry, and/or data processing apparatus. The storage device may include read-only memory (ROM), random access memory (RAM), flash memory, memory card, storage medium, and/or other storage devices.

[185] FIG. 12 is a diagram illustrating the structure of a content streaming system.

[186] Thus, the exemplary embodiments described herein may be performed through implementation on a processor, microprocessor, controller, or chip. For example, the functional modules illustrated in each drawing may be implemented by implementation on a computer, processor, microprocessor, controller, or chip.

[187] Additionally, the decoding equipment and encoding equipment to which this document applies may be included in a broadcast multimedia transceiver device, a mobile communication terminal, a home theater system video device, a digital cinema system video device, a surveillance camera, a video chat device, a communication device real-time e.g. video communication, mobile streaming device, storage medium, video recording camera, video-on-demand (VoD) service provider, over-the-top (OTT) video device, internet streaming service provider, 3D video device, video call system video device, medical video device, etc. and be used to process video or data signals. For example, an over-the-top (OTT) video device may include a game console, Blu-ray player, Internet-enabled television, home theater system, smartphone, tablet PC, digital video recorder (DVR), etc.

[188] Additionally, the processing method to which this document applies may be formed in the form of a program executed by a computer and stored on a computer-readable recording medium. Multimedia data having a data structure according to this document may also be stored on a computer-readable recording medium. The computer-readable recording medium includes all kinds of storage devices and distributed storage devices on which computer-readable data is stored. The computer-readable recording medium may include, for example, a Blu-ray disc (BD), a universal serial bus (USB), a ROM, a PROM, an EPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical storage device. . Additionally, the computer-readable recording medium includes media implemented in the form of a carrier (eg, when transmitted over the Internet). Additionally, the bit stream generated by the encoding method may be stored in a computer-readable recording medium or transmitted through a wired/wireless communication network. Additionally, an exemplary embodiment of the present document may be implemented by a computer program product through program codes, and the program codes may be executed by a computer according to an exemplary embodiment of the present document. The program codes may be stored on a machine-readable medium.

[189] Additionally, the content streaming system to which the present document applies may mainly include an encoding server, a streaming server, a web server, a media storage, a user device, and a media input device.

[190] The encoding server serves to compress content input from media input devices such as a smartphone, a camera, and a video recorder into digital data to form a bit stream and transmit the bit stream to a streaming server. As another example, if media input devices such as a smartphone, camera, and video recorder directly generate a bit stream, the encoding server may be omitted. The bitstream may be generated by an encoding method to which this document applies or a bitstream generation method, and the streaming server may temporarily store the bitstream while transmitting or receiving the bitstream.

[191] The streaming server serves to transmit multimedia data to a user device based on a user request through a web server, and the web server serves as a medium that informs the user as to what services are available. When a user requests a required service from a web server, the web server delivers the user's request to the streaming server, and the streaming server transmits the media data to the user. At this time, the content streaming system may include a separate control server, in which case the control server serves to manage commands/responses between devices in the content streaming system.

[192] The streaming server may receive content from a media storage and/or encoding server. For example, when receiving content from an encoding server, the streaming server may receive the content in real time. In this case, in order to provide a smooth streaming service, the streaming server can store the bit stream for a predetermined time.

[193] As an example of a user device, a portable telephone, a smartphone, a laptop computer, a digital broadcast terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, a tablet PC, a tablet PC, an ultrabook, may be provided. wearable device (e.g., smart watch, smart glasses, head-mounted display (HMD)), digital TV, desktop computer, digital signage system, etc. The respective servers in the content streaming system may be operated by a distribution server, in which case, data received through each server may be distributed and processed.

Claims (31)

1. Decoding equipment for decoding images, containing: Memory device; And at least one processor coupled to the storage device, wherein the at least one processor is configured to: receive image information including prediction mode information for the current block and intra-subsegment-based (ISP) mode-related information for the current block via a bitstream, wherein the ISP-related information is received based on the size of the current block and the maximum size of the transform block ; And decode the current block based on prediction mode information and ISP related information, in this case, the current block is also decoded based on: extracting the internal prediction mode of the current block based on the prediction mode information; And generating a predictive sample of the current block based on the internal prediction mode and ISP-related information, wherein the ISP-related information includes information about whether ISP is applied to the current block and information regarding the splitting direction, wherein the ISP related information is received based on the following condition related to the size of the current block being executed that is satisfied: cbWidth <= MaxTbSizeY && cbHeight <= MaxTbSizeY, where cbWidth represents the width of the current block, cbHeight represents the height of the current block, and MaxTbSizeY represents the maximum size of the transform block. 2. Encoding equipment for encoding images, containing: Memory device; And at least one processor coupled to the storage device, wherein the at least one processor is configured to: retrieving the internal prediction mode of the current block; retrieving internal subsegment (ISP) mode-related information for the current block based on the size of the current block and the maximum size of the transform block; generating a predictive sample of the current block based on the internal prediction mode and ISP-related information; forming a residual sample based on a predictive sample; And encoding image information including prediction mode information regarding the intra prediction mode, ISP related information and information regarding the residual sample, wherein the ISP-related information includes information about whether ISP is applied to the current block and information regarding the splitting direction, wherein the ISP related information is received based on the following condition related to the size of the current block being executed that is satisfied: cbWidth <= MaxTbSizeY && cbHeight <= MaxTbSizeY, where cbWidth represents the width of the current block, cbHeight represents the height of the current block, and MaxTbSizeY represents the maximum size of the transform block. 3. Image data transmission equipment containing: at least one processor configured to obtain a bit stream for an image, wherein the bit stream is generated based on extracting an intra prediction mode of the current block, extracting intra sub segment (ISP) mode related information for the current block based on the size of the current block, and maximum transform block size, generating a prediction sample of the current block based on the intra prediction mode and ISP related information, generating a residual sample based on the prediction sample, and encoding image information including prediction mode information regarding the intra prediction mode, ISP related information and information relative to the residual sample; And a transmitter configured to transmit data comprising a stream of bits, wherein the ISP-related information includes information about whether ISP is applied to the current block and information regarding the splitting direction, wherein the ISP related information is received based on the following condition related to the size of the current block being executed that is satisfied: cbWidth <= MaxTbSizeY && cbHeight <= MaxTbSizeY, where cbWidth represents the width of the current block, cbHeight represents the height of the current block, and MaxTbSizeY represents the maximum size of the transform block.

Images