WO2021118310A1

WO2021118310A1 - Video signal processing method and device

Info

Publication number: WO2021118310A1
Application number: PCT/KR2020/018201
Authority: WO
Inventors: 임성원
Original assignee: 주식회사 케이티
Priority date: 2019-12-13
Filing date: 2020-12-11
Publication date: 2021-06-17

Abstract

An image decoding method according to the present disclosure may comprise the steps of: determining whether lossless encoding is applied to a current image; decoding residual coefficients for a current block in the current image; and deriving a residual sample on the basis of the residual coefficients.

Description

Video signal processing method and apparatus

The present disclosure relates to a video signal processing method and apparatus.

Recently, demand for high-resolution and high-quality images such as HD (High Definition) images and UHD (Ultra High Definition) images is increasing in various application fields. As the image data becomes higher resolution and higher quality, the amount of data increases relative to the existing image data. Therefore, when transmitting image data using a medium such as an existing wired or wireless broadband line or storing it using an existing storage medium, the transmission cost and storage costs will increase. High-efficiency image compression techniques can be used to solve these problems that occur as image data becomes high-resolution and high-quality.

Interprediction technology that predicts pixel values included in the current picture from pictures before or after the current picture as an image compression technology, intra prediction technology that predicts pixel values included in the current picture using pixel information in the current picture, emergence Various technologies exist, such as entropy encoding technology in which a short code is assigned to a value with a high frequency and a long code is allocated to a value with a low frequency of occurrence, and the image data can be effectively compressed and transmitted or stored using these image compression techniques.

Meanwhile, as the demand for high-resolution images increases, the demand for stereoscopic image content as a new image service is also increasing. A video compression technique for effectively providing high-resolution and ultra-high-resolution stereoscopic image content is being discussed.

An object of the present disclosure is to provide a method and apparatus for efficiently encoding/decoding a residual coefficient in encoding/decoding a video signal.

An object of the present disclosure is to provide a method and apparatus for adaptively determining an encoding method of a residual coefficient according to whether lossless encoding is applied in encoding/decoding a video signal.

An object of the present disclosure is to provide an encoding/decoding method and apparatus using a palette table in encoding/decoding a video signal.

The technical problems to be achieved in the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those of ordinary skill in the art to which the present disclosure belongs from the description below. will be able

An image decoding method according to the present disclosure includes the steps of determining whether lossless encoding is applied to a current image, decoding a residual coefficient for a current block in the current image, and deriving a residual sample based on the residual coefficient may include the step of In this case, when decoding the residual coefficient, one of a first method using a maximum of m comparison flags and a second method using a maximum of n comparison flags is selected based on whether lossless encoding is applied to the current image, and , the comparison flag may indicate whether an absolute value of the residual coefficient exceeds a predetermined value.

In the image decoding method according to the present disclosure, it may be determined whether at least one comparison flag is decoded with respect to the residual coefficient by comparing the number of decoded bins and a threshold value using context information.

In the image decoding method according to the present disclosure, when the number of bins decoded using context information is equal to or greater than the threshold value, a syntax indicating the absolute value of the residual coefficient as it is instead of the comparison flag may be decoded.

In the image decoding method according to the present disclosure, when at least one of the comparison flag or a parity flag indicating whether the absolute value of the residual coefficient is an even number is decoded, the number of bins decoded using the context information increases. can

In the image decoding method according to the present disclosure, when the first syntax is decoded and the first syntax indicates that the residual coefficient has a value other than 0, whether the absolute value of the residual coefficient has a value greater than 1 gt_1_flag indicating whether or not can be further decoded.

In the video decoding method according to the present disclosure, when the gt_1_flag indicates that the absolute value has a value greater than 1, a parity flag indicating whether the absolute value is an even number and gt_2_flag indicating whether the absolute value is greater than 3 may be further decoded.

In the image decoding method according to the present disclosure, the threshold value may be determined based on the size of the current block.

An image encoding method according to the present disclosure includes determining whether lossless encoding is applied to a current image, deriving a residual coefficient based on a residual sample of a current block, and encoding the residual coefficient of the current block may include steps. In this case, when encoding the residual coefficient, one of a first method using a maximum of m comparison flags and a second method using a maximum of n comparison flags is selected based on whether lossless encoding is applied to the current image, , the comparison flag may indicate whether an absolute value of the residual coefficient exceeds a predetermined value.

According to the present disclosure, encoding/decoding efficiency can be improved by differently setting the encoding method of a residual coefficient according to the number of bins to be encoded using context information.

According to the present disclosure, coding/decoding efficiency can be improved by adaptively determining a coding method for residual coefficients according to whether lossless coding is applied.

According to the present disclosure, encoding/decoding efficiency can be improved by using the palette table.

Effects obtainable in the present disclosure are not limited to the above-mentioned effects, and other effects not mentioned may be clearly understood by those of ordinary skill in the art to which the present disclosure belongs from the description below. will be.

1 is a block diagram illustrating an image encoding apparatus according to an embodiment of the present disclosure.

2 is a block diagram illustrating an image decoding apparatus according to an embodiment of the present disclosure.

3 and 4 illustrate a lossless encoding method according to the present disclosure.

5 illustrates an intra prediction method according to the present disclosure.

6 shows a pre-defined intra prediction mode.

7 illustrates a method of encoding an intra prediction mode according to the present disclosure.

8 relates to a method of constructing an MPM candidate according to the present disclosure.

9 illustrates an intra prediction method based on a planar mode according to the present disclosure.

10 illustrates an intra prediction method based on a DC mode according to the present disclosure.

11 to 13 illustrate an improved intra prediction method according to the present disclosure.

14 is an exemplary diagram for explaining an application aspect of DPCM.

15 illustrates a search order when a zigzag scan is applied, according to an embodiment of the present disclosure.

16 is a flowchart illustrating a process of encoding residual coefficients in an encoder.

17 is a flowchart illustrating a process of encoding size information of residual coefficients.

18 is a flowchart illustrating a process of decoding a residual coefficient in a decoder.

19 is a diagram illustrating a decoding process of size information of residual coefficients.

20 to 24 are diagrams illustrating examples of counting the number of bins using context information.

25 and 26 show the surrounding restoration area referenced to determine context information.

27 to 29 are diagrams for explaining the concept of the palette mode (palette mode) according to the present disclosure.

30 illustrates a method of performing intra prediction based on a palette mode according to the present disclosure.

31 and 32 show a method of configuring a pallet table according to the present disclosure.

33 is a diagram illustrating an example in which palette entries are added to the palette entry candidate list.

34 shows an example in which a palette table predefined in an encoder and a decoder is used.

35 is a diagram illustrating a method of signaling a palette prediction flag in the form of a binary vector based on run length encoding as an embodiment to which the present disclosure is applied.

36 illustrates a method of encoding/decoding a palette index according to a scan order according to the present disclosure.

37 is an exemplary diagram for describing a pixel adjacent to a current pixel.

38 shows an example of encoding a run merge flag using context information.

39 is an example showing the range of the context information index.

40 shows an example in which index-related information is encoded in units of a region having a preset size.

41 shows an example in which index-related information is encoded using inter-block dependency.

42 is an exemplary diagram for explaining an encoding aspect of an escape value.

43 shows an example of deriving a difference value with respect to an escape value based on the intra prediction mode.

44 shows an example of deriving a difference value with respect to an escape value using a block vector.

45 is an example for describing a neighboring block of the current block.

Since the present disclosure can make various changes and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present disclosure to specific embodiments, and it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present disclosure. In describing each figure, like reference numerals have been used for like elements.

Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present disclosure, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

When a component is referred to as being “connected” or “connected” to another component, it is understood that the other component may be directly connected or connected to the other component, but other components may exist in between. it should be On the other hand, when it is mentioned that a certain element is "directly connected" or "directly connected" to another element, it should be understood that no other element is present in the middle.

The terms used in the present application are used only to describe specific embodiments, and are not intended to limit the present disclosure. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Hereinafter, preferred embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. Hereinafter, the same reference numerals are used for the same components in the drawings, and repeated descriptions of the same components are omitted.

Referring to FIG. 1 , the image encoding apparatus 100 includes a picture division unit 110 ,

prediction units

120 and 125 , a transform unit 130 , a quantization unit 135 , a rearrangement unit 160 , and an entropy encoding unit ( 165 ), an inverse quantization unit 140 , an inverse transform unit 145 , a filter unit 150 , and a memory 155 .

Each of the constituent units shown in FIG. 1 is independently illustrated to represent different characteristic functions in the image encoding apparatus, and does not mean that each constituent unit is composed of separate hardware or one software constituent unit. That is, each component is listed as each component for convenience of description, and at least two components of each component are combined to form one component, or one component can be divided into a plurality of components to perform a function, and each of these components Integrated embodiments and separate embodiments of the components are also included in the scope of the present disclosure without departing from the essence of the present disclosure.

In addition, some components are not essential components to perform an essential function in the present disclosure, but may be optional components for merely improving performance. The present disclosure may be implemented by including only essential components to implement the essence of the present disclosure, except for components used for performance improvement, and a structure including only essential components excluding optional components used for performance improvement Also included in the scope of the present disclosure.

The picture divider 110 may divide the input picture into at least one processing unit. In this case, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The picture splitter 110 divides one picture into a combination of a plurality of coding units, prediction units, and transformation units, and combines one coding unit, prediction unit, and transformation unit based on a predetermined criterion (eg, a cost function). can be selected to encode the picture.

For example, one picture may be divided into a plurality of coding units. In order to split a coding unit in a picture, a recursive tree structure such as a quad tree structure can be used. A coding in which one image or a largest coding unit is used as a root and is divided into other coding units. A unit may be divided having as many child nodes as the number of divided coding units. A coding unit that is no longer split according to certain restrictions becomes a leaf node. That is, when it is assumed that only square splitting is possible for one coding unit, one coding unit may be split into up to four different coding units.

Hereinafter, in an embodiment of the present disclosure, a coding unit may be used as a unit for performing encoding or may be used as a meaning for a unit for performing decoding.

A prediction unit may be split in the form of at least one square or rectangle of the same size within one coding unit, and one prediction unit among the split prediction units within one coding unit is a prediction of another. It may be divided to have a shape and/or size different from that of the unit.

When a prediction unit for performing intra prediction based on a coding unit is generated, if it is not the smallest coding unit, intra prediction may be performed without dividing the prediction unit into a plurality of prediction units NxN.

The

prediction units

120 and 125 may include an inter prediction unit 120 performing inter prediction and an intra prediction unit 125 performing intra prediction. Whether to use inter prediction or to perform intra prediction for a prediction unit may be determined, and specific information (eg, intra prediction mode, motion vector, reference picture, etc.) according to each prediction method may be determined. In this case, a processing unit in which prediction is performed and a processing unit in which a prediction method and specific content are determined may be different. For example, a prediction method and a prediction mode may be determined in a prediction unit, and prediction may be performed in a transformation unit. A residual value (residual block) between the generated prediction block and the original block may be input to the transform unit 130 . Also, prediction mode information, motion vector information, etc. used for prediction may be encoded by the entropy encoder 165 together with the residual value and transmitted to the decoding apparatus. When a specific encoding mode is used, it is also possible to encode the original block as it is without generating the prediction block through the

prediction units

120 and 125 and transmit it to the decoder.

The inter prediction unit 120 may predict a prediction unit based on information on at least one of a picture before or after a picture of the current picture, and in some cases, prediction based on information of a partial region in the current picture that has been encoded Units can also be predicted. The inter prediction unit 120 may include a reference picture interpolator, a motion prediction unit, and a motion compensator.

The reference picture interpolator may receive reference picture information from the memory 155 and generate pixel information of integer pixels or less in the reference picture. In the case of luminance pixels, a DCT-based 8-tap interpolation filter (DCT-based Interpolation Filter) with different filter coefficients may be used to generate pixel information of integer pixels or less in units of 1/4 pixels. In the case of the color difference signal, a DCT-based 4-tap interpolation filter in which filter coefficients are different to generate pixel information of integer pixels or less in units of 1/8 pixels may be used.

The motion prediction unit may perform motion prediction based on the reference picture interpolated by the reference picture interpolator. As a method for calculating the motion vector, various methods such as Full search-based Block Matching Algorithm (FBMA), Three Step Search (TSS), and New Three-Step Search Algorithm (NTS) may be used. The motion vector may have a motion vector value of 1/2 or 1/4 pixel unit based on the interpolated pixel. The motion prediction unit may predict the current prediction unit by using a different motion prediction method. Various methods, such as a skip method, a merge method, an AMVP (Advanced Motion Vector Prediction) method, an intra block copy method, etc., may be used as the motion prediction method.

The intra prediction unit 125 may generate a prediction unit based on reference pixel information around the current block, which is pixel information in the current picture. When a neighboring block of the current prediction unit is a block on which inter prediction is performed, and thus a reference pixel is a pixel on which inter prediction is performed, a reference pixel included in the block on which inter prediction is performed is a reference pixel of the block on which intra prediction has been performed. information can be used instead. That is, when the reference pixel is not available, the unavailable reference pixel information may be replaced with at least one reference pixel among the available reference pixels.

In intra prediction, the prediction mode may have a directional prediction mode that uses reference pixel information according to a prediction direction and a non-directional mode that does not use directional information when prediction is performed. A mode for predicting luminance information and a mode for predicting chrominance information may be different, and intra prediction mode information used for predicting luminance information or predicted luminance signal information may be utilized to predict chrominance information.

When intra prediction is performed, if the size of the prediction unit and the size of the transformation unit are the same, intra prediction for the prediction unit based on the pixel present on the left side, the pixel present on the upper left side, and the pixel present on the upper side of the prediction unit can be performed. However, when the size of the prediction unit is different from the size of the transformation unit when performing intra prediction, intra prediction may be performed using a reference pixel based on the transformation unit. In addition, intra prediction using NxN splitting may be used only for the smallest coding unit.

The intra prediction method may generate a prediction block after applying an adaptive intra smoothing (AIS) filter to a reference pixel according to a prediction mode. The type of AIS filter applied to the reference pixel may be different. In order to perform the intra prediction method, the intra prediction mode of the current prediction unit may be predicted from the intra prediction mode of the prediction unit existing around the current prediction unit. When the prediction mode of the current prediction unit is predicted using the mode information predicted from the neighboring prediction unit, if the intra prediction mode of the current prediction unit and the neighboring prediction unit are the same, the current prediction unit and the neighboring prediction unit are used using predetermined flag information It is possible to transmit information that the prediction modes of . , and if the prediction modes of the current prediction unit and the neighboring prediction units are different from each other, entropy encoding may be performed to encode prediction mode information of the current block.

In addition, a residual block including residual information, which is a difference value between a prediction unit and an original block of the prediction unit, in which prediction is performed based on the prediction unit generated by the

prediction units

120 and 125 may be generated. The generated residual block may be input to the transform unit 130 .

The transform unit 130 converts the original block and the residual block including residual information of the prediction unit generated by the

prediction units

120 and 125 to DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), KLT and It can be converted using the same conversion method. Whether to apply DCT, DST, or KLT to transform the residual block may be determined based on intra prediction mode information of a prediction unit used to generate the residual block.

The quantizer 135 may quantize the values transformed by the transform unit 130 into the frequency domain. The quantization coefficient may change according to blocks or the importance of an image. The value calculated by the quantization unit 135 may be provided to the inverse quantization unit 140 and the rearrangement unit 160 .

The rearrangement unit 160 may rearrange the coefficient values on the quantized residual values.

The reordering unit 160 may change the two-dimensional block form coefficient into a one-dimensional vector form through a coefficient scanning method. For example, the rearranging unit 160 may use a Zig-Zag Scan method to scan from DC coefficients to coefficients in a high-frequency region and change them into a one-dimensional vector form. A vertical scan that scans a two-dimensional block shape coefficient in a column direction and a horizontal scan that scans a two-dimensional block shape coefficient in a row direction may be used instead of a zig-zag scan according to a size of a transform unit and an intra prediction mode. That is, it may be determined whether any of the zig-zag scan, the vertical scan, and the horizontal scan is used according to the size of the transform unit and the intra prediction mode.

The entropy encoding unit 165 may perform entropy encoding based on the values calculated by the reordering unit 160 . For entropy encoding, various encoding methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) may be used.

The entropy encoding unit 165 receives the residual value coefficient information and block type information, prediction mode information, division unit information, prediction unit information and transmission unit information, motion of the coding unit from the reordering unit 160 and the

prediction units

120 and 125 . Various information such as vector information, reference frame information, interpolation information of a block, and filtering information may be encoded.

The entropy encoder 165 may entropy-encode the coefficient values of the coding units input from the reordering unit 160 .

The inverse quantizer 140 and the inverse transform unit 145 inversely quantize the values quantized by the quantizer 135 and inversely transform the values transformed by the transform unit 130 . The residual values generated by the inverse quantizer 140 and the inverse transform unit 145 are combined with the prediction units predicted through the motion estimation unit, the motion compensator, and the intra prediction unit included in the

prediction units

120 and 125 and restored. You can create a Reconstructed Block.

The filter unit 150 may include at least one of a deblocking filter, an offset correcting unit, and an adaptive loop filter (ALF).

The deblocking filter may remove block distortion caused by the boundary between blocks in the reconstructed picture. In order to determine whether to perform deblocking, it may be determined whether to apply the deblocking filter to the current block based on pixels included in several columns or rows included in the block. When a deblocking filter is applied to a block, a strong filter or a weak filter can be applied according to the required deblocking filtering strength. In addition, in applying the deblocking filter, horizontal filtering and vertical filtering may be concurrently processed when performing vertical filtering and horizontal filtering.

The offset corrector may correct the offset of the deblocked image with respect to the original image in units of pixels. In order to perform offset correction on a specific picture, a method of dividing pixels included in an image into a certain number of regions, determining the region to be offset and applying the offset to the region, or taking edge information of each pixel into consideration can be used to apply

Adaptive loop filtering (ALF) may be performed based on a value obtained by comparing the filtered reconstructed image and the original image. After dividing the pixels included in the image into a predetermined group, one filter to be applied to the corresponding group is determined, and filtering can be performed differentially for each group. As for information on whether to apply ALF, the luminance signal may be transmitted for each coding unit (CU), and the shape and filter coefficients of the ALF filter to be applied may vary according to each block. In addition, the ALF filter of the same type (fixed type) may be applied regardless of the characteristics of the target block.

The memory 155 may store the reconstructed block or picture calculated through the filter unit 150 , and the stored reconstructed block or picture may be provided to the

predictors

120 and 125 when inter prediction is performed.

Referring to FIG. 2 , the image decoding apparatus 200 includes an entropy decoding unit 210, a reordering unit 215, an inverse quantization unit 220, an inverse transform unit 225,

prediction units

230 and 235, and a filter unit ( 240) and a memory 245 may be included.

When an image bitstream is input by the image encoding apparatus, the input bitstream may be decoded by a procedure opposite to that of the image encoding apparatus.

The entropy decoding unit 210 may perform entropy decoding in a procedure opposite to that performed by the entropy encoding unit of the image encoding apparatus. For example, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) may be applied corresponding to the method performed by the image encoding apparatus.

The entropy decoding unit 210 may decode information related to intra prediction and inter prediction performed by the encoding apparatus.

The reordering unit 215 may perform reordering based on a method of rearranging the entropy-decoded bitstream by the entropy decoding unit 210 by the encoder. Coefficients expressed in the form of a one-dimensional vector may be restored and rearranged as coefficients in the form of a two-dimensional block. The reordering unit 215 may receive information related to coefficient scanning performed by the encoder and perform the reordering by performing a reverse scanning method based on the scanning order performed by the corresponding encoder.

The inverse quantization unit 220 may perform inverse quantization based on the quantization parameter provided by the encoding apparatus and the reordered coefficient values of the blocks.

The inverse transform unit 225 may perform inverse transforms, ie, inverse DCT, inverse DST, and inverse KLT, on the transforms performed by the transform unit, ie, DCT, DST, and KLT, on the quantization result performed by the image encoding apparatus. Inverse transform may be performed based on a transmission unit determined by the image encoding apparatus. The inverse transform unit 225 of the image decoding apparatus may selectively perform a transformation technique (eg, DCT, DST, KLT) according to a plurality of pieces of information such as a prediction method, a size of a current block, and a prediction direction.

The

prediction units

230 and 235 may generate a prediction block based on the prediction block generation related information provided from the entropy decoding unit 210 and previously decoded block or picture information provided from the memory 245 .

As described above, when intra prediction is performed in the same manner as in the operation in the image encoding apparatus, when the size of the prediction unit and the size of the transformation unit are the same, the pixel present on the left side of the prediction unit, the pixel present on the upper left side, and the upper Intra prediction is performed on the prediction unit based on the existing pixel, but when the size of the prediction unit and the size of the transformation unit are different when performing intra prediction, intra prediction is performed using the reference pixel based on the transformation unit can do. Also, intra prediction using NxN splitting may be used only for the smallest coding unit.

The

prediction units

230 and 235 may include a prediction unit determiner, an inter prediction unit, and an intra prediction unit. The prediction unit determining unit receives various information such as prediction unit information input from the entropy decoder 210, prediction mode information of the intra prediction method, and motion prediction related information of the inter prediction method, and divides the prediction unit from the current coding unit, and predicts It may be determined whether the unit performs inter prediction or intra prediction. The inter prediction unit 230 uses information required for inter prediction of the current prediction unit provided from the image encoding apparatus based on information included in at least one of a picture before or after the current picture including the current prediction unit. Inter prediction may be performed on the prediction unit. Alternatively, inter prediction may be performed based on information of a pre-restored partial region in the current picture including the current prediction unit.

In order to perform inter prediction, a motion prediction method of a prediction unit included in a corresponding coding unit based on a coding unit is selected from among skip mode, merge mode, AMVP mode, and intra block copy mode. You can decide which way to go.

The intra prediction unit 235 may generate a prediction block based on pixel information in the current picture. When the prediction unit is a prediction unit on which intra prediction is performed, intra prediction may be performed based on intra prediction mode information of the prediction unit provided by the image encoding apparatus. The intra prediction unit 235 may include an Adaptive Intra Smoothing (AIS) filter, a reference pixel interpolator, and a DC filter. The AIS filter is a part that performs filtering on the reference pixel of the current block, and may be applied by determining whether to apply the filter according to the prediction mode of the current prediction unit. AIS filtering may be performed on the reference pixel of the current block by using the prediction mode and AIS filter information of the prediction unit provided by the image encoding apparatus. When the prediction mode of the current block is a mode in which AIS filtering is not performed, the AIS filter may not be applied.

When the prediction mode of the prediction unit is a prediction unit in which intra prediction is performed based on a pixel value obtained by interpolating the reference pixel, the reference pixel interpolator may interpolate the reference pixel to generate a reference pixel of a pixel unit having an integer value or less. When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. The DC filter may generate the prediction block through filtering when the prediction mode of the current block is the DC mode.

The reconstructed block or picture may be provided to the filter unit 240 . The filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

Information on whether a deblocking filter is applied to a corresponding block or picture and information on whether a strong filter or a weak filter is applied when the deblocking filter is applied may be provided from the image encoding apparatus. The deblocking filter of the image decoding apparatus may receive deblocking filter-related information provided from the image encoding apparatus, and the image decoding apparatus may perform deblocking filtering on the corresponding block.

The offset correction unit may perform offset correction on the reconstructed image based on the type of offset correction applied to the image during encoding, information on the offset value, and the like.

ALF may be applied to a coding unit based on information on whether ALF is applied, ALF coefficient information, etc. provided from the encoding apparatus. Such ALF information may be provided by being included in a specific parameter set.

The memory 245 may store the reconstructed picture or block to be used as a reference picture or reference block, and may also provide the reconstructed picture to an output unit.

As described above, hereinafter, in the embodiments of the present disclosure, a coding unit is used as a term for a coding unit for convenience of description, but it may also be a unit for performing decoding as well as coding.

In addition, the current block denotes an encoding/decoding target block, and depending on the encoding/decoding step, a coding tree block (or coding tree unit), a coding block (or a coding unit), a transform block (or a transform unit), or a prediction block (or prediction unit) and the like. In this specification, a 'unit' may indicate a basic unit for performing a specific encoding/decoding process, and a 'block' may indicate a pixel array of a predetermined size. Unless otherwise specified, 'block' and 'unit' may be used interchangeably. For example, in the embodiments to be described below, it may be understood that the coding block (coding block) and the coding unit (coding unit) are mutually equivalent.

Image compression can be broadly classified into lossy coding and lossless coding. The biggest difference between the two encodings is the presence or absence of a quantization process. In lossy coding, greater compression efficiency than lossless coding can be obtained through a quantization process, but data loss may occur. In lossless encoding, the original data can be maintained as it is, but compression efficiency is lower than in lossy encoding.

Among the encoding processes illustrated in FIG. 1 , by using the quantization and in-loop filtering processes, reconstructed data different from the original data may be generated (ie, loss occurs). Accordingly, in lossless coding, a quantization process and an in-loop filtering process may be skipped. In this case, if the quantization process is omitted, the transform process for transforming the residual data into frequency domain components becomes meaningless, so that the transform process can be further omitted when lossless encoding is applied.

Since the processes of lossless encoding and lossless encoding are different, information indicating whether lossless encoding is applied must be transmitted to the decoder. For example, information indicating whether lossless encoding is performed may be encoded through an upper header (eg, SPS, PPS, or slice header). If the upper header indicates that lossless encoding has been performed using 1-bit information (flag), the decoder determines that lossless encoding has been performed using the 1-bit information. When it is determined that lossless encoding has been performed, the decoder may omit the quantization and in-loop filtering processes and decode the image.

As an example, a flag for determining a variable lossless_coding for determining whether to use lossless encoding may be signaled through a bitstream. A value of the variable lossless_coding being true indicates that lossless encoding is applied, and a value of the variable lossless_coding being false indicates that lossless encoding is not applied.

Variables indicating whether transform, quantization, deblocking filter, SAO, or ALF are used may be defined as t_skip, q_skip, d_skip, s_skip, and a_skip. A value of true of the variables indicates that a corresponding coding process is omitted, and a value of false indicates that a corresponding coding process is not omitted.

According to the value of the variable lossless_coding, it may be determined whether a flag for determining the values of the variables is signaled through the bitstream. For example, when the value of the variable lossless_coding is true, signaling of a flag for determining the variables t_skip, q_skip, d_skip, s_skip, and a_skip may be omitted. Variables whose encoding is omitted may be estimated as pre-defined values. A pre-defined value may be true. When the value of lossless_coding is true, application of the transform, quantization deblocking filter, SAO, and ALF may be skipped regardless of the values of the variables t_skip, q_skip, d_skip, s_skip, and a_skip.

When the value of the variable lossless_coding is false, flags for determining variables t_skip, q_skip, d_skip, s_skip, and a_skip may be decoded. Based on whether the value of each variable is true or false, it is possible to determine whether to skip the corresponding coding process.

Referring to FIG. 3 , when the value of lossless_coding is false, at least one of t_skip, q_skip, d_skip, s_skip, and a_skip may be set to true. Accordingly, when values of variables other than the first variable are false, signaling of a flag for determining the value of the first variable may be omitted and the value of the first variable may be set to true. For example, when values of t_skip, q_skip, d_skip, and s_skip are false, encoding of a flag for determining a_skip may be omitted and the variable a_skip may be set to true.

Alternatively, instead of signaling a flag for determining the value of the variable lossless_coding, lossless_coding may be defined as an internal variable. The value of the internal variable lossless_coding may be determined based on the variables t_skip, q_skip, d_skip, s_skip, and a_skip.

Referring to FIG. 4 , a flag for determining a value of each of variables t_skip, q_skip, d_skip, s_skip, and a_skip may be signaled through a bitstream. In this case, when the values of the variables, t_skip, q_skip, d_skip, s_skip, and a_skip are all true, the variable lossless_coding may be set to true. On the other hand, when at least one of t_skip, q_skip, d_skip, s_skip, and a_skip is false, the variable lossless_coding may be determined to be false.

Then, it is possible to define internal variables related to lossless coding in units of blocks. When this variable is defined as blk_lossless_coding, the value of blk_lossless_coding may be set to the value of the variable lossless_coding set in FIG. 3 or FIG. 4 .

Alternatively, when lossless_coding is set to true, blk_lossless_coding may be encoded for each block to determine whether lossless encoding is performed on a block-by-block basis.

In the above example, for convenience of explanation, only transform, quantization, deblocking filter, SAO, and ALF are given as examples, but for all techniques that make lossless coding impossible, such as the joint_CbCr coding method and LMCS (luma mapping with chroma scaling), The above-described method may be applied in the same/similar manner.

5 illustrates an intra prediction method according to the present disclosure.

Referring to FIG. 5 , a reference pixel line for intra prediction of a currently encoded/decoded block (hereinafter, referred to as a current block) may be determined ( S500 ).

The reference pixel line according to the present disclosure may belong to a neighboring block adjacent to the current block. The neighboring block may include at least one of a left block, an upper block, an upper left block, a lower left block, an upper right block, a right block, a lower right block, and a lower block. Also, the reference pixel line may belong to a spatial block belonging to the same picture as the current block but not adjacent to the current block, or may belong to a temporal block belonging to a picture different from the current block. Here, the temporal block may be a block in the same position as a neighboring block of the aforementioned current block.

The reference pixel line may be limited to belong only to a block pre-encoded/decoded before the current block, or may belong to a block encoded/decoded after the current block. In this case, a separate padding process may be involved, which will be described later.

The reference pixel line may be determined as any one of a plurality of reference pixel line candidates. The plurality of reference pixel line candidates include a first reference pixel line adjacent to the current block, a second reference pixel line adjacent to the first reference pixel line, . , at least one of an n-th reference pixel line adjacent to the (n-1)-th reference pixel line. Here, n may be an integer of 2, 3, 4, or more.

For example, the plurality of reference pixel line candidates may include first to fourth reference pixel lines, only first to third reference pixel lines, or only first and third reference pixel lines. .

Any one of the above-described first to nth reference pixel lines may be selectively used, and for this purpose, information for specifying a position of the reference pixel line may be used. The information may be signaled by the encoding apparatus or may be derived from the decoding apparatus based on a predetermined encoding parameter. The encoding parameter may include at least one of a block size, a shape, a position, a component type (Y/Cb/Cr), a prediction mode (intra/inter), an intra prediction mode, or a non-directional mode. Here, a block may mean a current block and/or a neighboring block, or a coding block, a prediction block, and/or a transform block. Also, the number of reference pixel line candidates available for the current block may be variably determined based on the above-described encoding parameter, and information for specifying the number may be separately signaled.

Referring to FIG. 5 , an intra prediction mode of the current block may be determined ( S510 ).

The intra prediction mode of the current block may be determined as any one of intra prediction modes pre-defined in the encoding/decoding apparatus. The pre-defined intra prediction mode may be composed of two non-directional modes and 65 directional modes, as shown in FIG. 6 .

The directional mode includes at least one of a planar mode or a DC mode, and the non-directional mode includes a mode (eg, vertical mode, horizontal mode, diagonal mode, etc.) having a predetermined angle/direction. can

A method of determining the intra prediction mode will be described in detail with reference to FIGS. 7 and 8 .

Referring to FIG. 5 , intra prediction of the current block may be performed based on the determined reference pixel line and the intra prediction mode ( S520 ).

That is, a pixel corresponding to the determined intra prediction mode among pixels of the reference pixel line may be specified as a reference pixel, and intra prediction may be performed using the specified reference pixel.

First, when the prediction mode of the current block is the intra mode, the reference pixel line index for intra prediction of the current block may be encoded. Here, the reference pixel line index may be information for specifying the position of the aforementioned reference pixel line, and may be encoded in units of blocks.

When the first reference pixel line is selected by the reference pixel line index, the MPM flag may be encoded. Here, the MPM flag may indicate whether the intra prediction mode of the current block is determined based on the MPM.

If the MPM flag is true, a plane mode flag indicating whether the plane mode is used may be coded.

When the planar mode is used as the MPM, the encoding process of the intra prediction mode is terminated. If the planar mode is not used, an index specifying the same MPM candidate as the intra prediction mode used for the current block may be encoded.

If a non-adjacent reference pixel line (ie, a second reference pixel line, a third reference pixel line, etc.) rather than an adjacent reference pixel line (ie, the first reference pixel line) is selected by the reference pixel line index, the MPM flag may be omitted and the value of the MPM flag may be set to true. That is, it is possible to encode the MPM index without encoding the MPM flag.

Meanwhile, if the MPM flag is false, one of the remaining modes except for the MPM candidate among intra prediction modes pre-defined in the encoding apparatus may be encoded.

From a decoding point of view, the reference pixel line index for intra prediction of the current block may be decoded. When the first reference pixel line is selected by the reference pixel line index, the MPM flag may be decoded.

If the MPM flag is true, the plane mode flag indicating whether the plane mode is used may be decoded. When the plane mode flag is true, the intra prediction mode of the current block may be set to the planar mode. When the plane mode flag is false, the MPM index specifying the same MPM candidate as the intra prediction mode used for the current block is decoded. can do. The intra prediction mode of the current block may be determined based on the pre-configured MPM candidate and the decoded MPM index. A method of configuring the MPM candidate will be described in detail with reference to FIG. 8 .

If a non-adjacent reference pixel line (ie, a second reference pixel line, a third reference pixel line, etc.) is selected by the reference pixel line index, decoding of the MPM flag may be omitted and the value of the MPM flag may be set to true. have. That is, the MPM index can be decoded without decoding the MPM flag. The intra prediction mode of the current block may be determined based on the pre-configured MPM candidate and the decoded MPM index.

On the other hand, if the MPM flag is false, the remaining mode information specifying one of the remaining modes except for the planar mode and the pre-configured MPM candidate among intra prediction modes pre-defined in the decoding apparatus may be decoded. The mode specified by the decoded remaining mode information may be set as the intra prediction mode of the current block.

In order to encode/decode the intra prediction mode, an MPM candidate may be constructed using an intra prediction mode used in a neighboring block of the current block. If the intra prediction mode of the current block is the same as the MPM candidate included in the MPM candidate list, the same MPM candidate as the intra prediction mode of the current block may be specified using the MPM index. If the same MPM candidate as the intra prediction mode of the current block does not exist, the intra prediction mode of the current block may be encoded/decoded after reassigning indexes of the remaining intra prediction modes except for the MPM candidates. For convenience of description, it is assumed that the number of MPM candidates included in the MPM candidate list is 5.

First, the intra prediction mode used in the neighboring block of the current block is brought. Here, the neighboring block may include at least one of an upper neighboring block, a left neighboring block, an upper left neighboring block, an upper right neighboring block, or a lower left neighboring block.

8 is an example showing the location of a neighboring block. Referring to FIG. 8 , LB denotes a pixel located at the lower left of the block, and RT denotes a pixel located at the upper right of the block. L denotes a position of a restored pixel that exists immediately to the left of LB, and A denotes a position of a restored pixel that exists immediately above RT. In order to construct an MPM candidate of the current block, an intra prediction mode used in a reconstructed block existing to the left of the current block and an intra prediction mode used in a reconstructed block existing above the current block may be used. In this case, the reconstructed block existing on the left is a block including L, and the reconstructed block existing above is a block including A. Alternatively, an MPM candidate may be derived from a reconstructed block including a reconstructed pixel (g or h) positioned in the middle left or a block including a reconstructed pixel (c or d) positioned in the upper middle.

The intra prediction mode used in the left reconstruction block is called Left, and the intra prediction mode used in the upper reconstruction block is defined as Above.

If the left reconstructed block is unavailable, the left reconstructed block is not encoded by intra prediction, or when matrix-based prediction (MIP) is applied to the left reconstructed block, Left may be set to the planar mode. And, if the upper reconstructed block is unavailable, if the upper reconstructed block is not encoded by intra prediction, if matrix-based prediction (MIP) is used for the upper reconstructed block, or if the upper reconstructed block is the CTU ( coding tree unit), Above can be set to flat mode. If it does not correspond to the above-mentioned case, Left and Above may be respectively set to the intra prediction mode used by the left reconstructed block and the intra prediction mode used by the upper reconstructed block. When BDPCM is applied to the left or upper restoration block, Left or Above may be set to horizontal mode or vertical mode depending on the BDPCM application direction.

If both Left and Above modes are the same and are directional mode, MPM candidates can be composed of [Left, Left-1, Left+1, Left-2, Left+2].

Rather, if the two modes of Left and Above are different, and Left or Above is a directional mode, the candidate composition method will be different depending on whether both Left and Above are directional modes or only one of them is a directional mode. First, the mode with the largest value among the two modes is defined as Max, and the mode with the smallest value is defined as Min.

When both Left and Above are directional modes, MPM candidates can be configured in the following way. If the value of (Max - Min) is 1, the MPM candidate may be composed of [Left, Above, Min-1, Max +1, Min-2]. Rather, if the value of (Max - Min) is 62 or higher, the MPM candidate can be composed of [Left, Above, Min+1, Max -1, Min+2]. Rather, if the value of (Max - Min) is 2, the MPM candidate can be composed of [Left, Above, Min+1, Min -1, Max+1]. Otherwise, the MPM candidate can consist of [Left, Above, Min-1, Min +1, Max-1].

If only one of Left and Above is a directional mode, MPM candidates can be composed of [Max, Max-1, Max+1, Max -2, Max+2].

Otherwise (ie, both Left and Above are not in directional mode), MPM candidates can consist of [DC mode, 50 (vertical mode), 18 (horizontal mode), vertical-4, vertical+4].

A prediction pixel may be generated as shown in FIG. 9 under the planar mode (planar mode) allocated to mode 0.

Referring to FIG. 9 , T and L are examples of neighboring reconstructed pixels used when generating a prediction block in a planar mode. T denotes a reconstructed pixel located in the upper right corner. L denotes a reconstructed pixel located in the lower left corner. These restored pixels are set as reference pixels and used for prediction. Here, P1 is a prediction pixel for the horizontal direction. P1 may be generated by linearly interpolating the reconstructed pixel and T located at the same position along the Y axis as P1. P2 is a prediction pixel for the vertical direction. P2 may be generated by linearly interpolating the reconstructed pixel and L located at the same position along the X axis as P2. Thereafter, as in Equation 1 below, a final prediction pixel is generated by weighting P1 and P2.

In this case, in Equation 1, the weights α and β may have the same value. Alternatively, the weights α and β may be determined in consideration of the shape, width, height, or aspect ratio of the block.

Under the DC mode allocated to the first mode, the prediction pixel may be generated as shown in FIG. 10 . In FIG. 10, it is assumed that the size of a block is 4x4.

Referring to FIG. 10 , reconstructed pixels existing around the block are set as reference pixels, and after calculating the average value of the reference pixels, the calculated values are set as all prediction pixels in the prediction block. The reference pixels used to calculate the average value may include at least one of upper reconstructed pixels B to E, left reconstructed pixels F to I, or upper left reconstructed pixel A. Alternatively, the average value may be calculated using only the upper restored pixels and the left restored pixels.

Alternatively, the average value may be calculated by selectively using only one of the upper restored pixels or the left restored pixels according to the shape of the block.

Alternatively, it is also possible to apply a greater weight to either the upper restored pixels or the left restored pixels according to the shape of the block. For example, after calculating the upper average of the upper reconstructed pixels and the left average of the left reconstructed pixels, the weighted average value derived by performing a weighted sum operation on the upper average and the left mean is finally predicted It can be set in pixels.

When lossless encoding is performed, pixels in a block may be sequentially reconstructed. For example, the upper left pixel is restored first and the lower right pixel is restored last, but based on a raster scan, a horizontal scan, a vertical scan, a diagonal scan, an inverse raster scan, an inverse horizontal scan, an inverse vertical scan, or an inverse diagonal scan. to determine the restoration order. Alternatively, various restoration orders can be defined, such as restoring the lower left pixel first and restoring the upper right pixel last.

The reconstruction order may be determined based on the size, shape, or intra prediction mode of the current block. For example, when the value of the intra prediction mode is smaller than the value of the horizontal mode, the lower left pixel is initially reconstructed, whereas when the value of the intra prediction mode is greater than or equal to the value of the horizontal mode, the upper left pixel may be initially reconstructed .

Alternatively, the restoration order may be determined for each predetermined unit. The predetermined unit may be a row, a column, or a sub-block. The restoration order for each predetermined unit and the above-described restoration order for each pixel may be set to be the same as each other. For example, pixels may be sequentially restored from an upper row to a lower row, or pixels may be sequentially restored from a left column to a right column. In the drawings to be described later, a number indicated on each pixel indicates a restoration order.

Improved intra prediction, using prediction/reconstruction pixels in a block as reference pixels, may be applied. The improved intra prediction may be applied only when lossless coding is applied. The improved intra prediction according to the present disclosure will be described in detail below with reference to FIGS. 11 to 13 .

Referring to FIG. 11 , in the improved DC mode, the prediction pixel of the current block may be derived based on a previously reconstructed pixel located in the vicinity of the corresponding prediction pixel. Here, the previously reconstructed pixel may include at least one of an upper reconstructed pixel, a left reconstructed pixel, and an upper left reconstructed pixel. In this case, the prediction pixel adjacent to the boundary of the current block may be derived using the reconstructed pixel included in the neighboring block adjacent to the current block. As an example, the prediction pixel for a0 may be derived based on the average value of A, B, and F adjacent to the current block. On the other hand, a prediction pixel that is not adjacent to the boundary of the current block may be derived using a pixel belonging to the current block and reconstructed before the corresponding prediction pixel. As an example, the prediction pixel for a6 may be derived based on the average value of a1, a2, and a5 reconstructed before a6.

Assuming that the prediction pixels corresponding to the pixels a0 to a15 in the current block are p0 to p15, the prediction pixels may be derived based on Equation 2 below.

In Equation 2, cL _n , cU _n , and cUL _n mean pixels that exist immediately to the left, immediately above, and immediately above p _n , respectively, and wL, wU and wUL are applied to _{cL n} , cU _n , and cUL _{n , respectively.} means the weight to be For example, assuming that wL, wU, and wUL are set to 1, 1, and 0, respectively, p0 becomes the average value of F and B, and in the case of p6, it becomes the average value of a2 and a5. Other prediction pixels may be generated in the same manner.

A weight applied to each pixel may be determined based on at least one of a location of a prediction pixel, a size of a current block, a shape of a current block, or whether a neighboring reconstruction pixel is included in the current block.

Referring to FIG. 12 , in directional modes except for mode 0 (planar mode) and mode 1 (DC mode), restored pixels A to Q are set as reference pixels and are positioned in a direction corresponding to the directional mode A prediction pixel is generated using at least one reference pixel. 12 is an example of prediction pixels generated by a directional mode. Assuming that mode 66 is used, each prediction pixel may be set as a reference pixel located in a diagonal direction from the upper right. For example, p0 is C, p1 and p4 are D, ... , p15 is set to I.

As another example, if the 18th (horizontal) mode is used instead of the 66th mode, each prediction pixel may be set as a reference pixel located in the left direction. For example, p0 to p3 are J, ... , p12 to p15 are set to M.

In lossless coding, intra prediction based on an improved directional mode may be applied. In intra prediction based on the enhanced directional mode, a pixel in the current block may be used as a reference pixel. That is, a pixel included in the upper row of the prediction pixel or the left column of the prediction pixel may be used as the reference pixel. Here, the pixel in the current block may mean a prediction pixel or a reconstructed pixel.

For example, if it is assumed that intra prediction based on the 18th mode is applied to the current block shown in FIG. 12 , each prediction pixel may be set as a reconstructed pixel located to the left of the prediction pixel. For example, p0 to p3 are set to J, a0, a1, and a2, respectively. p4 to p7 are set to K, a4, a5, a6, respectively, p8 to p11 are set to L, a8, a9, a10, respectively, and p12 to p15 are set to M, a12, a13, and a14, respectively.

When a prediction/reconstruction pixel included in an upper row or a left column of a prediction pixel is to be used as a reference pixel, additional padding may be required depending on the number of the intra prediction mode of the current block. For example, since the reconstructed pixels included in the right neighboring block or the lower neighboring block of the current block are unavailable, padding for the reconstructed pixels included in the right neighboring block or the lower neighboring block of the current block needs to be performed.

Referring to FIG. 13 , it is assumed that the intra prediction mode used in FIG. 13 is mode 67 and will be described.

Prediction pixels p0 to p3 adjacent to the upper boundary of the current block are generated by using reconstructed pixels on the current block as reference pixels, respectively. Prediction pixels p4 to p7 not adjacent to the upper boundary of the current block may set prediction/reconstruction pixels included in the upper row as reference pixels. In this case, in the case of p6 and p7, there is no pixel on the right side of the block in the coding order. In this way, padding may be performed using the previously restored pixel for a position where the restored pixel does not exist. In this case, the reference pixel included in the right neighboring block of the current block may be generated by padding the pixel located at the right boundary of the current block. For example, for prediction of p4 to p7, padding is performed using a3. Alternatively, padding may be performed using pixels in the current block (hereinafter, referred to as a first pixel) adjacent to the right boundary of the current block and reconstructed pixels existing in the upper right corner of the current block. As an example, the padding pixel may be generated based on a restored pixel having the same x-axis coordinate as the padding pixel and a first pixel having the same y-axis coordinate as the padding pixel. As an example, the first padded pixel may be generated by weighting a3 and F according to the distance. Similarly, it is also possible to create a second padded pixel using a3 and G. Alternatively, the padding pixel may be generated using the first pixel and the padding pixel of the current block, the restored pixel located at the upper right (eg, the restored pixel located on the angular line in the directional prediction mode).

In this case, the number of pixels to be padded may be determined according to the size of the block and the number of the intra prediction mode. For example, if the number of the intra prediction mode is 66, only one pixel per line is padded to the right of the block because the upper right angle is 45 degrees. Alternatively, in the case of mode 50, padding may not be performed because padding is not required.

Alternatively, a method of matching the same number of reference pixels as the number of reference pixels set using reconstructed pixels to the same when padding using original pixels in a block is also possible. For example, in the upper part of the current block of FIG. 13 , if B to I are set as reference pixels using reconstructed pixels, 4 pixels may be padded in the lower row as well.

In this example, for convenience of explanation, the intra prediction mode in the upper right direction is described as a reference, but a similar concept may be applied to the prediction mode in the lower left direction. For example, reference pixels included in the lower neighboring block of the current block may be generated through padding using pixels adjacent to the lower boundary of the current block and belonging to the current block.

Whether or not intra prediction using pixels in the block described above is applied can be always performed when lossless coding is performed. Alternatively, when lossless encoding is performed, it is also possible to encode information indicating whether the corresponding technology is applied in block units and transmit the encoded information to the decoder. Alternatively, whether the corresponding technology is supported or not may be encoded through the upper header. In this way, it is possible to control whether the corresponding technology is used in lossless encoding through a higher header.

A residual pixel may be derived by differentiating the prediction pixel from the original pixel, and transform and/or quantization may be applied to the residual pixel. In this case, instead of encoding the residual data as it is, a difference value between the residual data may be generated and then the generated difference value may be encoded. As described above, a method of encoding/decoding a difference value between residual data instead of encoding the residual data as it is may be referred to as DPCM (Delta Pulse Coded Modulation).

Here, the residual data may represent a residual pixel or a residual coefficient. The residual coefficient may include at least one of a transform coefficient obtained by transforming the residual pixel, a quantized transform coefficient obtained by quantizing the transform coefficient, or a quantized coefficient obtained by quantizing the residual pixel. Depending on whether transform and/or quantization is applied to the current block, the residual coefficient may represent at least one of a transform coefficient, a quantized transform coefficient, or a quantized coefficient. The decoder may derive a transform coefficient by applying an inverse quant to the quantized transform coefficient, and may derive a residual pixel by applying an inverse transform to the transform coefficient.

14 is an exemplary diagram for explaining an application aspect of DPCM.

14 , rX (X is an integer including 0) represents a residual pixel that is a difference between an original pixel and a predicted pixel. Quantization may be applied to the residual pixels to generate quantized coefficients. As an example, the residual pixels (ie, r0 to r15) may be quantized to generate quantized coefficients (ie, Q(r0) to Q(r15)). Here, Q represents a quantized coefficient.

Thereafter, by applying DPCM, a difference value (ie, s0 to s15) for the quantized coefficients may be generated. A difference value sX with respect to the quantized coefficient Q(rX) may be derived by differentiating the adjacent quantized coefficient Q(rY) from Q(rX). Here, the position of the adjacent quantized coefficient Q(rY) may be determined based on the DPCM direction.

The DPCM direction may indicate at least one of vertical, horizontal, diagonal, or zigzag.

The horizontal DPCM direction indicates that adjacent quantized coefficients are selected according to the order in which the quantized coefficients are one-dimensionally arranged by applying a horizontal scan. Specifically, when the DPCM direction is horizontal, the adjacent quantized coefficient Q(rY) may be located before the quantized coefficient Q(rX) when the quantized coefficients are arranged in one dimension along the horizontal direction. have. For the first quantized coefficient Q(r0), since there is no previously positioned quantized coefficient, the quantized coefficient is set as a difference value as it is. For example, when the DPCM is in the horizontal direction, the difference values s0 to s4 with respect to the quantized coefficients Q(r0) to Q(r4) may be derived as follows.

s0 = Q(r0), s1 = Q(r1)-Q(r0), s2 = Q(r2)-Q(r1), s3 = Q(r3)-Q(r2), s4 = Q(r4)- Q(r3)

That is, the difference value sX with respect to the quantized coefficient Q(rX) except for the first quantized transform coefficient Q(r0) is the quantized coefficient Q(rX) and the adjacent quantized coefficient Q(r(X-1)) can be derived from the difference between

The vertical DPCM direction indicates that adjacent quantized coefficients are selected according to the order in which the quantized coefficients are one-dimensionally arranged by applying a vertical scan. Specifically, when the DPCM direction is vertical, the adjacent quantized coefficient Q(rY) may be located before the quantized coefficient Q(rX) when the quantized coefficients are arranged in one dimension along the vertical direction. have. For the first quantized coefficient Q(r0), since there is no previously positioned quantized coefficient, the quantized coefficient is set as a difference value as it is. For example, when the DPCM is in the vertical direction, the difference values s0, s4, s8, and s12 for the quantized coefficients Q(r0), Q(r4), Q(r8), and Q(r12) can be derived as follows. have.

s0 = Q(r0), s4 = Q(r4)-Q(r0), s10 = Q(r8)-Q(r4), s12 = Q(r12)-Q(r8)

That is, the difference value sX with respect to the quantized coefficient Q(rX) except for the first quantized transform coefficient Q(r0) is the quantized coefficient Q(rX) and the adjacent quantized coefficient Q(r(X-(block_width)) )) can be derived from the difference between Here, block_width indicates the width of the current block.

The fact that the DPCM direction is diagonal indicates that adjacent quantized coefficients are selected according to the order in which the quantized coefficients are one-dimensionally arranged by applying a diagonal scan. Specifically, when the DPCM direction is diagonal, the adjacent quantized coefficient Q(rY) can be located before the quantized coefficient Q(rX) when the quantized coefficients are arranged in one dimension along the diagonal direction. have. In this case, the diagonal direction may be any one of an upper left diagonal direction, an upper right diagonal direction, a lower left diagonal direction, or a lower right diagonal direction.

When the DPCM direction is the upper-left diagonal direction, a difference value sX with respect to the quantized coefficient Q(rX) may be derived based on a difference value from the previous quantized coefficient Q(rY) located in the upper-left diagonal direction. At this time, with respect to the first quantized coefficient Q(r0), since there is no quantized coefficient located in the upper left diagonal direction, the quantized coefficient is set as a difference value as it is. For example, when the DPCM is in the upper left diagonal direction, the difference values s0, s5, s10, and s15 for the quantized coefficients Q(r0), Q(r5), Q(r10), and Q(r15) can be derived as follows. can

s0 = Q(r0), s5 = Q(r5)-Q(r0), s10 = Q(r10)-Q(r5), s15 = Q(r15)-Q(r10)

The fact that the DPCM direction is zigzag indicates that adjacent quantized coefficients are selected according to the order in which the quantized coefficients are one-dimensionally arranged by applying a zigzag scan.

When the quantized coefficients are arranged in a primary column according to the zigzag scan order, the difference value with respect to the quantized coefficient Q(rX) using the quantized coefficient Q(rY) located before the quantized coefficient Q(rX) sX can be induced. As an example, assuming that 'zigzag 3' shown in FIG. 15 is applied, the difference values for the quantized coefficients Q(r0), Q(r1), Q(r8), and Q(15) are as follows. can be induced.

s0 = Q(r0), s4 = Q(r4)-Q(r0), s1 = Q(r1)-Q(r4), s15 = Q(r8)-Q(r1)

After deriving a difference value for each quantized coefficient, the difference value may be encoded instead of the quantized coefficient to be signaled.

In the above-described embodiment, it has been described that the difference value sX is derived by using the quantized coefficient Q(rY) located before the quantized coefficient Q(rX). Unlike the described example, a difference value sX may be derived using a quantized coefficient located next to the quantized coefficient Q(rX).

In addition, in the above-described embodiment, it has been explained that the difference value sX is derived by differentiating the adjacent quantized coefficient Q(rY) from the quantized coefficient Q(rX). Unlike the described example, a difference value sX may be derived by differentiating the quantized coefficient Q(rX) from the adjacent quantized coefficient Q(rY).

The DPCM direction may be predefined in the encoder and the decoder. For example, one of horizontal, vertical, diagonal, and zigzag may be fixedly used. Alternatively, information (eg, index) specifying one of a plurality of DPCM directions may be encoded and signaled. The index may be signaled at a block level, a slice level, or a picture level.

Alternatively, one of a plurality of DPCM directions may be selected based on at least one of the size, shape, and intra prediction mode of the current block. For example, when a horizontal intra prediction mode (eg, Mode_18 of FIG. 5 ) is applied to the current block, the DPCM direction may be determined horizontally. On the other hand, when a vertical intra prediction mode (eg, Mode_50 of FIG. 5 ) is applied to the current block, the DPCM direction may be determined vertically.

Alternatively, the number or type of DPCM direction candidates may be set differently based on at least one of the size, shape, intra prediction mode, and encoding mode of the current block. For example, when inter prediction is applied to the current block, the DPCM modes in the horizontal direction and the vertical direction may be set to be unavailable. In this case, the DPCM mode in the diagonal direction or the zigzag direction may be applied, or an index specifying one of a plurality of zigzag direction candidates may be encoded and signaled.

Conversely, after determining the DPCM direction, the intra prediction mode of the current block may be determined based on the DPCM direction. For example, the DPCM modes in the vertical direction, the horizontal direction, and the diagonal direction (or zigzag) may correspond to the intra prediction modes in the vertical direction, the horizontal direction, and the diagonal direction, respectively. That is, a vertical DPCM direction for the current block indicates that the vertical intra prediction mode is applied to the current block, and a horizontal DPCM direction for the current block indicates that the horizontal intra prediction mode is applied to the current block. and that the DPCM direction for the current block is diagonal (or zigzag) indicates that the intra prediction mode in the diagonal direction is applied to the current block.

Alternatively, each of the DPCM modes in the vertical direction and the horizontal direction corresponds to the intra prediction mode in the vertical direction and the horizontal direction, and the DPCM mode in the diagonal direction (or zigzag) corresponds to the non-directional intra prediction mode (eg, DC or planar). may be That is, a vertical DPCM direction for the current block indicates that the vertical intra prediction mode is applied to the current block, and a horizontal DPCM direction for the current block indicates that the horizontal intra prediction mode is applied to the current block. and that the DPCM direction for the current block is diagonal (or zigzag) indicates that DC or planar mode is applied to the current block.

When the DPCM direction is determined to be zigzag, the encoder and the decoder may generate a difference value using a predefined search order. Alternatively, information (eg, an index) specifying at least one of a plurality of zigzag scan orders may be encoded and signaled. For example, the index may specify one of the zigzag scan orders shown in FIG. 15 . Alternatively, one of a plurality of zigzag scan orders may be specified based on at least one of the size, shape, and intra prediction mode of the current block.

The intra prediction mode of the current block may be determined based on the MPM candidate. In this case, when DPCM is applied to a neighboring block, an MPM candidate of the current block may be derived in consideration of the DPCM direction applied to the neighboring block. Specifically, an MPM candidate may be derived by estimating that the intra prediction mode of the neighboring block is the intra prediction mode corresponding to the DPCM direction. For example, when DPCM in the horizontal direction is applied to a neighboring block, an MPM candidate may be derived by estimating that the intra prediction mode of the neighboring block is in the horizontal direction. Alternatively, when vertical DPCM is applied to a neighboring block, an MPM candidate may be derived by estimating that the intra prediction mode of the neighboring block is vertical. Alternatively, when diagonal (or zigzag) DPCM is applied to a neighboring block, an MPM candidate may be derived by estimating that the intra prediction mode of the neighboring block is a diagonal or non-directional intra prediction mode (eg, DC or planar). .

When encoding the DPCM direction of the current block, context information of a neighboring block or a previous block may be referred to. Here, the neighboring block or previous block, which is the reference target of the context information, may include at least one of a left neighboring block or an upper neighboring block of the current block. For example, when the DPCM directions of the left neighboring block and the upper neighboring block are the same, when the DPCM directions of the left neighboring block and the upper neighboring block are different, when DPCM is applied to both the left neighboring block and the upper neighboring block, the left neighboring block and the upper In consideration of at least one of a case in which DPCM is applied to only one of the neighboring blocks or a case in which DPCM is not applied to both the left neighboring block and the upper neighboring block, the context information accumulation (or update) method of the current block may be different. .

In the above-described embodiment, a DPCM method based on quantized coefficients has been described. In the described example, instead of quantized coefficients, DPCM based on residual pixels, transform coefficients, or quantized transform coefficients may be applied. For example, when DPCM based on the residual pixel is applied, a difference value sX with respect to the residual pixel rX may be derived by differentiating the previous residual pixel rY from the residual pixel rX.

When lossless coding is applied, DPCM may be applied to residual data in which quantization is omitted. That is, when lossless coding is applied, DPCM based on residual pixels or transform coefficients may be applied.

When lossless encoding is not applied, information (eg, a flag) indicating whether quantization is applied to the current block may be encoded and signaled. Based on the information, it may be determined whether DPCM is performed on the quantized transform coefficients. For example, when the flag indicates that quantization is applied, it may be determined that DPCM is performed on a quantized coefficient or a quantized transform coefficient. On the other hand, when the flag indicates that quantization is not applied, it may be determined that DPCM is performed on the residual pixel or transform coefficient.

As another example, DPCM may be applied to the reconstructed pixel.

When DPCM is applied, the decoder may decode the difference value and reconstruct the residual data using the decoded difference value. As an example, the residual data may be reconstructed by adding adjacent residual data and difference values. A residual pixel may be obtained by applying at least one of inverse quantization or inverse transform to the reconstructed residual data according to the type of residual data.

Next, a method of encoding/decoding residual data will be described in detail. In an embodiment to be described later, it is assumed that residual data are residual coefficients. When transform and quantization are not applied to the current block, a residual coefficient encoding/decoding method described later may be used for encoding/decoding the residual pixel. In addition, when DPCM is applied to the current block, a residual data encoding/decoding method described later may be used for encoding/decoding a difference value.

A flag indicating whether a non-zero residual coefficient exists in the current block may be encoded and signaled. When a non-zero residual coefficient exists in the current block, the position of the last non-zero residual coefficient in the scan order may be encoded. As an example, the syntax last_x indicating the x-coordinate of the last non-zero residual coefficient and the syntax last_y indicating the y-coordinate may be encoded and signaled.

In addition, a sub-block flag indicating whether a non-zero residual coefficient exists in a sub-block may be encoded in units of sub-blocks in the current block. For example, after encoding information indicating the position of the last non-zero residual coefficient, the syntax sub_blk_flag indicating whether a non-zero residual coefficient exists for each sub-block may be encoded and signaled.

When a non-zero residual coefficient exists in a subblock, information on each residual coefficient may be additionally encoded according to a scan order.

In this case, for a subblock having a scan order earlier than that of the subblock including the last non-zero residual coefficient, encoding of the subblock flag may be omitted. Since non-zero residual coefficients are not included in the sub-block, the value of the sub-block flag may be regarded as zero.

Also, for a subblock including the last non-zero residual coefficient, encoding of the subblock flag may be omitted. Since a non-zero residual coefficient is necessarily included in the sub-block, the value of the sub-block flag may be regarded as 1.

As another example, encoding of location information of the last non-zero residual coefficient may be omitted. When the encoding of the position information of the last non-zero residual coefficient is omitted, sub-block flags may be encoded for all sub-blocks in the current block.

In this case, when it is determined that non-zero residual coefficients are not included in the remaining sub-blocks except for the last sub-block in the scan order, it may be understood that the non-zero residual coefficients are necessarily included in the last sub-block. Accordingly, for the last sub-block, encoding of the sub-block flag may be omitted and the value may be regarded as 1.

Information indicating whether position information of the last non-zero coefficient is encoded may be additionally encoded. When the location information of the last non-zero coefficient is encoded, the value of the information may be set to 1. In this case, the sub-block flag may be coded from the sub-block in which the last non-zero coefficient exists. On the other hand, when the location information of the last non-zero coefficient is not encoded, the value of the information may be set to 0. In this case, the sub-block flag may be coded from the sub-block having the first scan order.

When lossless encoding is applied, encoding of information indicating the position of the last residual coefficient (eg, syntax last_x and last_y) may be omitted.

Alternatively, when lossless encoding is applied, information indicating the position of the last residual coefficient may be encoded, but encoding of the sub-block flag may be omitted. In this case, transform coefficients may be encoded starting from the subblock including the last residual coefficient.

Alternatively, when lossless encoding is applied, both information indicating the position of the last residual coefficient and the sub-block flag may not be encoded. In this case, it can be estimated that all sub-blocks include non-zero residual coefficients. Accordingly, residual coefficients may be sequentially encoded from the first sub-block to the last sub-block in the scan order.

When a non-zero residual coefficient exists in the current block, it may be assumed that a non-zero residual coefficient is necessarily included in the first sub-block in the current block. Accordingly, encoding of a sub-block flag indicating whether a non-zero residual coefficient exists in the first sub-block may be omitted.

The information on each residual coefficient may include at least one of a flag indicating whether the residual coefficient has a non-zero value, information indicating a size of the residual coefficient, and information indicating a sign of the residual coefficient.

Residual coefficients may be encoded according to a predetermined scan order. In this case, the encoding order of the residual coefficients may be different based on whether the transform is skipped in the current block. For example, when a transform is not skipped in the current block, a residual coefficient located at the lower right of the subblock may be encoded first, and the residual coefficient located at the upper left of the subblock may be encoded last. That is, a scan order between residual coefficients may be determined according to an inverse-diagonal scan, an inverse-horizontal scan, or an inverse-vertical scan. On the other hand, when the transform is skipped in the current block, the residual coefficient located at the upper left of the subblock may be encoded first, and the residual coefficient located at the lower right of the subblock may be encoded last. That is, a scan order between residual coefficients may be determined along a diagonal scan, a horizontal scan, or a vertical scan.

Alternatively, even when a transform is skipped in the current block, a scan order between residual coefficients may be determined according to an inverse-diagonal scan, an inverse-horizontal scan, or an inverse-vertical scan.

The scan order of the residual coefficients may be predefined in the encoder and the decoder. Alternatively, information indicating a scan order of residual coefficients may be encoded and signaled. Alternatively, the scan order may be determined based on at least one of the size, shape, intra prediction mode, whether the transform is skipped, whether the secondary transform is performed, or the DPCM direction of the current block.

First, a flag indicating whether the residual coefficient has a non-zero value, sig_flag, may be coded ( S1610 ). When the value of the residual coefficient is 0, encoding may be performed by setting the value of the flag sig_flag to 0. On the other hand, when the value of the residual coefficient is not 0, encoding may be performed by setting the value of the flag sig_flag to 1. When the value of the residual coefficient is not 0, information on the magnitude of the residual coefficient may be further encoded (S1620).

An absolute value of the residual coefficient may be encoded using at least one or more gt_N_flags. Here, N may be a natural number of 1 or more. The flag gt_N_flag may indicate whether the absolute value of the residual coefficient has a value greater than 2 (N-1). The number of gt_N_flags used to encode the absolute value of the residual coefficient may be determined based on whether a transform is skipped in the current block. For example, when a transform is not skipped in the current block, two gt_N_flags (N is 1 to 2) may be used. On the other hand, when the transform is skipped in the current block, three or more gt_N_flags (eg, three, four, or five) may be used. As an example, the maximum number of gt_N_flags of 2 indicates that the syntax rem_level indicating the residual size from the residual coefficient whose absolute value exceeds 3 is encoded. On the other hand, the fact that the maximum number of gt_N_flags is 5 indicates that the syntax rem_level indicating the residual size from the residual coefficient whose absolute value exceeds 9 is encoded. In the embodiment described below, it is assumed that two gt_N_flags are used.

A flag gt1_flag indicating whether the absolute value of the residual coefficient is greater than 1 may be encoded ( S1710 ). When the absolute value of the residual coefficient is 1, encoding can be performed by setting the value of the flag gt1_flag to 0. On the other hand, when the absolute value of the residual coefficient is greater than 1, encoding can be performed by setting the value of the flag gt1_flag to 1.

When the absolute value of the residual coefficient is greater than 1, a flag par_flag indicating whether the absolute value of the residual coefficient is an even number or an odd number may be encoded (S1720). When the absolute value of the residual coefficient is an even number, it can be encoded by setting the flag par_flag to 0. On the other hand, when the absolute value of the residual coefficient is an odd number, encoding may be performed by setting the flag par_flag to 1. Alternatively, conversely, when the absolute value of the residual coefficient is an even number, the flag par_flag may be set to 1, and when the absolute value of the residual coefficient is an odd number, the flag par_flag may be set to 0.

Next, a flag gt_2_flag indicating whether the absolute value of the residual coefficient is greater than 3 may be encoded ( S1730 ). When the absolute value of the residual coefficient is 3 or less, the value of the flag gt_2_flag may be set to 0. On the other hand, when the absolute value of the residual coefficient is greater than 3, the value of the flag gt_2_flag may be set to 1.

When the absolute value of the residual coefficient is greater than 3, rem_level indicating the residual size may be coded (S1740). The syntax rem_level may be derived by shifting a value derived by subtracting 4 from the absolute value of the residual coefficient to the right by 1.

In addition to the gt_1_flag and gt_2_flag flags shown in FIG. 17, gt_N_flag, such as gt_3_flag, gt_4_flag or gt_5_flag, may be further encoded. In this case, when the value of gt_(N-1)_flag is 1, gt_N_flag may be additionally encoded.

gt_N_flag may indicate whether the absolute value of the residual coefficient has a value greater than (2N-1). When gt_N_flag is additionally used, rem_level may be derived by shifting a value derived by subtracting 2N from the absolute value of the residual coefficient to the right by 1.

As in the described example, when sig_flag is 1, sign_flag and gt_1_flag may be encoded and signaled. Also, when gt_1_flag is 1, parity_flag and gt_2_flag may be additionally encoded and signaled.

When the last gt_N_flag is true, rem_level indicating the residual coefficient may be encoded and signaled. The syntax rem_level may indicate a difference from the absolute value of the residual coefficient with a value obtained by adding 1 to the maximum value indicated by gt_N_flag (ie, 2N).

In the above example, it has been exemplified that the absolute value of the residual coefficient is encoded using sig_flag, gt_1_flag, par_flag, gt_2_flag, and rem_level. As another example, the absolute value of the residual coefficient may be encoded as it is. As an example, the syntax abs_level indicating the absolute value of the residual coefficient may be encoded. A method of selecting the encoding method of the absolute value of the residual coefficient will be described later.

After encoding the size information of the residual coefficient, a flag sign_flag indicating the sign of the residual coefficient may be encoded (S1630). A value of the flag sign_flag of 0 indicates that the residual coefficient is positive. On the other hand, the value of the flag sign_flag being 1 indicates that the residual coefficient is negative.

Table 1 shows the values assigned to each syntax when the value of the residual coefficient is -21 and two gt_N_flags are used.

구분division	수식Equation	값value
잔차 계수(Coeff)Residual Coefficient (Coeff)	coeffcoeff	-21-21
sig_flagsig_flag	coeff != 0coeff != 0	1One
gt_1_flaggt_1_flag	!! (\|coeff\|-1)!! (\|coeff\|-1)	1One
par_flag par_flag	(\|coeff\|-2) & 1(\|coeff\|-2) & 1	1One
gt_2_flaggt_2_flag	(\|coeff\|-2) >> 1(\|coeff\|-2) >> 1	1One
rem_levelrem_level	(\|coeff\|-4) >> 1(\|coeff\|-4) >> 1	88
sign_flagsign_flag		1One

In Table 1, coeff represents the value of the residual coefficient, and the 'Formula' item represents the formula used to derive the value of each syntax. Table 2 shows values assigned to each syntax when the value of the residual coefficient is -21 and five gt_N_flags are used.

구분division	수식Equation	값value
잔차 계수(Coeff)Residual Coefficient (Coeff)	coeffcoeff	-21-21
sig_flagsig_flag	coeff != 0coeff != 0	1One
gt_1_flaggt_1_flag	!! (\|coeff\|-1)!! (\|coeff\|-1)	1One
par_flag par_flag	(\|coeff\|-2) & 1(\|coeff\|-2) & 1	1One
gt_2_flaggt_2_flag	\|coeff\| >= 4\|coeff\| >= 4	1One
gt_3_flaggt_3_flag	\|coeff\| >= 6\|coeff\| >= 6	1One
gt_4_flaggt_4_flag	\|coeff\| >= 8\|coeff\| >= 8	1One
gt_5_flaggt_5_flag	\|coeff\| >= 10\|coeff\| >= 10	1One
rem_levelrem_level	(\|coeff\|-10) >> 1(\|coeff\|-10) >> 1	55
sign_flagsign_flag		1One

18 is a flowchart illustrating a process of decoding a residual coefficient in a decoder. When it is determined that non-zero residual coefficients are included in the subblock, the residual coefficients may be reconstructed based on a predetermined scan order.

First, a flag indicating whether the residual coefficient has a non-zero value, sig_flag, may be decoded ( S1810 ). A value of the flag sig_flag of 0 indicates that the value of the residual coefficient is 0. On the other hand, the value of the flag sig_flag being 1 indicates that the value of the residual coefficient is not 0. When the value of the flag sig_flag is 1, information on the magnitude of the residual coefficient may be further decoded (S1820).

For convenience of description, it is assumed that the residual coefficients are coded using a maximum of two gt_N_flags.

A flag gt1_flag indicating whether the absolute value of the residual coefficient is greater than 1 may be decoded ( S1910 ). A value of the flag gt_1_flag of 0 indicates that the absolute value of the residual coefficient is 1. On the other hand, the value of the flag gt_1_flag being 1 indicates that the absolute value of the residual coefficient is greater than 1.

When the value of the flag gt_1_flag is 1, the flag par_flag indicating whether the absolute value of the residual coefficient is an even number or an odd number may be decoded ( S1920 ). A value of the flag par_flag of 0 indicates that the absolute value of the residual coefficient is an even number, and a value of the flag par_flag of 1 indicates that the absolute value of the residual coefficient is an odd number.

Next, a flag gt_2_flag indicating whether the absolute value of the residual coefficient is greater than 3 may be decoded ( S1930 ). A value of the flag gt_2_flag equal to 0 indicates that the absolute value of the residual coefficient is less than 3. When the value of the flag gt_2_flag is 0, the absolute value of the residual coefficient may be determined to be 2 or 3 according to the value of the flag par_flag.

A value of the flag gt_2_flag of 1 indicates that the absolute value of the residual coefficient is greater than 3.

When the value of the flag gt_2_flag is 1, rem_level indicating the residual size may be decoded (S1940). An absolute value of the residual coefficient may be derived by adding 3 or 4 to a value derived by shifting the value representing the syntax rem_level to the left by 1.

In addition to the gt_1_flag and gt_2_flag flags shown in FIG. 19, gt_N_flag, such as gt_3_flag, gt_4_flag, or gt_5_flag, may be further decoded. In this case, when the value of gt_(N-1)_flag is 1, gt_N_flag may be further decoded.

gt_N_flag may indicate whether the absolute value of the residual coefficient has a value greater than (2N-1). When gt_N_flag is additionally used, rem_level may be set to a value derived by shifting a value derived by differentiating 2N from the absolute value of the residual coefficient by 1 to the right.

In the above-described example, it is exemplified that the absolute value of the residual coefficient is decoded using sig_flag, gt_1_flag, par_flag, gt_2_flag, and rem_level. As another example, the absolute value of the residual coefficient may be decoded as it is. As an example, the syntax abs_level indicating the absolute value of the residual coefficient may be decoded. A method of selecting a decoding method for the absolute value of the residual coefficient will be described later.

After decoding the size information of the residual coefficient, a flag sign_flag indicating the sign of the residual coefficient may be decoded (S1830). A value of the flag sign_flag of 0 indicates that the residual coefficient is positive. On the other hand, the value of the flag sign_flag being 1 indicates that the residual coefficient is negative.

Table 3 shows an example of decoding a residual coefficient with a value of -21 using two gt_N_flags.

구분division	값value	수식Equation
sig_flagsig_flag	1One	--
gt_1_flaggt_1_flag	1One	--
par_flag par_flag	1One	--
gt_2_flaggt_2_flag	1One	--
tmp_coeff tmp_coeff	55	1+gt_1_flag+par_flag+(gt_2_flag <<1)1+gt_1_flag+par_flag+(gt_2_flag <<1)
rem_levelrem_level	88
sign_flagsign_flag	1One	sign = (sign_flag == 1 ? -1 : 1)sign = (sign_flag == 1 ? -1 : 1)
잔차계수(coeff)residual coefficient (coeff)	-21-21	tmp_coeff + (rem_level<<1) * signtmp_coeff + (rem_level<<1) * sign

In Table 3, the variable tmp_coeff represents a temporary restoration coefficient. When the value of gt_2_flag is 0, the temporary restoration coefficient tmp_coeff may be set as an absolute value of the residual coefficient. On the other hand, when the value of gt_2_flag is 1, the absolute value of the residual coefficient can be derived by updating the temporary restoration coefficient tmp_coeff based on the syntax rem_level. Table 4 shows an example of decoding a residual coefficient having a value of -21 using five gt_N_flags.

구분division	값value	수식Equation
sig_flagsig_flag	1One	--
gt_1_flaggt_1_flag	1One	--
par_flag par_flag	1One	--
tmp_coeff tmp_coeff	55	1+gt_1_flag+par_flag1+gt_1_flag+par_flag
gt_2_flaggt_2_flag	1One	tmp_coeff += (sig_2_flag <<1)tmp_coeff += (sig_2_flag <<1)
gt_3_flaggt_3_flag	1One	tmp_coeff += (sig_3_flag <<1)tmp_coeff += (sig_3_flag <<1)
gt_4_flaggt_4_flag	1One	tmp_coeff += (sig_4_flag <<1)tmp_coeff += (sig_4_flag <<1)
gt_5_flaggt_5_flag	1One	tmp_coeff += (sig_5_flag <<1)tmp_coeff += (sig_5_flag <<1)
rem_levelrem_level	55	tmp_coeff += (rem_level << 1)tmp_coeff += (rem_level << 1)
sign_flagsign_flag	1One	sign = (sign_flag == 1 ? -1 : 1)sign = (sign_flag == 1 ? -1 : 1)
잔차계수(coeff)residual coefficient (coeff)	-21-21	tmp_coeff + (rem_level<<1) * signtmp_coeff + (rem_level<<1) * sign

In Table 3, the variable tmp_coeff represents a temporary restoration coefficient. When gt_N_flag is 0, the temporary restoration coefficient tmp_coeff may be set as an absolute value of the residual coefficient. On the other hand, when gt_N_flag is 1, the temporary restoration coefficient may be updated (eg, tmp_coeff += sig_N_flag << 1), and the following syntax may be parsed. As described, the residual coefficient may be encoded by at least one syntax. A residual coefficient may be changed into a plurality of bins through a binarization process of the syntax(s), and the changed bins may be encoded through entropy encoding.

Entropy encoding may be divided into encoding using context information and encoding not using context information. The context indicates the probability that the value of the bin is 0 or 1.

In order to limit the number of bins to be encoded using context information, a threshold may be set. The threshold value may be individually set for each block. Among the generated bins, a bin having a count value smaller than a threshold value is encoded using context information. When the count value is greater than or equal to the threshold, encoding using context information may no longer be used.

The threshold value may be determined based on the number of non-zero residual coefficients in the current block. For example, a value obtained by multiplying the number of non-zero residual coefficients in the current block by a real number or by adding or subtracting an offset may be set as the threshold value.

Alternatively, the threshold value may be determined based on the number of pixels included in the current block. For example, a value obtained by multiplying the number of pixels in the current block by a real number or by adding or subtracting an offset may be set as the threshold value.

Alternatively, information indicating a threshold value may be signaled through a bitstream. The information may be encoded through an upper header such as a sequence, picture header, or slice header.

Alternatively, the threshold value may be determined based on at least one of the size and shape of the current block. For example, after a threshold value is previously defined for each size or shape of a block, a threshold value corresponding to the size or shape of the current block may be called and used. Alternatively, information specifying one of a plurality of threshold value candidates corresponding to the size or shape of the current block may be encoded and signaled.

Alternatively, the threshold value may be determined based on at least one of whether transform skip is applied, a transform kernel applied to the current block, or a quantization parameter.

When counting the number of bins to be encoded using context information, when information indicating the position of the last non-zero residual coefficient is encoded, the counter may be set not to operate. That is, the information may be excluded from counting.

Alternatively, when encoding a flag indicating whether a non-zero residual coefficient exists for each subblock in the current block, the counter may be set not to operate. That is, the flag may be excluded from counting.

According to an embodiment of the present disclosure, in order to limit the number of bins encoded using context information, when the number of bins encoded using context information is greater than or equal to a threshold value, gt_N_flag or the like is used to limit the residual coefficient. Instead of stepwise encoding , the absolute value of the residual coefficient may be encoded as it is. For example, when the number of bins encoded using context information is smaller than a threshold value, at least one of sig_flag, sign_flag, gt_1_flag, par_flag, gt_2_flag, gt_3_flag, gt_4_flag, gt_5_flag, or rem_level illustrated in Tables 1 to 4 can be used to encode the absolute value of the residual coefficient. On the other hand, when the number of bins encoded using context information is equal to or greater than a threshold value, the syntax abs_level indicating the absolute value of the residual coefficient and sign_flag indicating the sign of the residual coefficient may be encoded.

Even in the decoder, a counter may be operated whenever an encoded bin is decoded using context information. When the value of the counter is smaller than the threshold value, the absolute value of the residual coefficient may be reconstructed using at least one of sig_flag, sign_flag, gt_1_flag, par_flag, gt_2_flag, gt_3_flag, gt_4_flag, gt_5_flag, or rem_level. On the other hand, when the value of the counter is equal to or greater than the threshold, the residual coefficient may be restored using the syntax abs_level and sign_flag.

20 is a diagram illustrating an example of counting the number of bins using context information.

For convenience of explanation, it is assumed that there are 16 residual coefficients in the sub-block, and it is assumed that each of the coefficients is C0 to C15. Here, C15 denotes a residual coefficient located in the lower right corner in the subblock, and C0 denotes a residual coefficient located in the upper left corner in the subblock.

In FIG. 20 , one pass indicates syntaxes encoded using context information. Except for pass 1, passes 2 and 3 indicate syntaxes that are encoded without using context information.

A path indicates an encoding order and a decoding order. For example, when the residual coefficient cannot be completely encoded using only syntaxes belonging to N-paths, syntaxes belonging to (N+1) may be encoded. The decoder may decode as many syntaxes as coded using context information in one pass, and then decode syntaxes belonging to the second pass. Also, after all syntaxes belonging to the 2nd pass are decoded, the syntaxes belonging to the 3-1 path may be decoded.

In the illustrated example, a 3-2 pass represents a path for a residual coefficient in which syntaxes belonging to pass 1 are not coded. That is, when a coefficient of a bin encoded using context information is smaller than a threshold value, the residual coefficient may be encoded through at least one of one pass, two passes, and three-1 passes. On the other hand, when a coefficient of a bin encoded using context information is equal to or greater than a threshold value, a residual coefficient may be encoded through 3-2 passes.

The encoding/decoding order of syntax may follow an arrow of FIG. 20 . First, when the number of bins encoded using context information is less than a threshold value, syntaxes belonging to one pass for a corresponding residual coefficient may be encoded. As an example, in FIG. 20 , it is exemplified that syntaxes belonging to one pass are encoded for C0 to C7. When at least one of C0 to C7 is not completely encoded only with syntaxes belonging to one pass, the syntaxes belonging to the second pass may be encoded. Similarly, when the residual coefficients are not completely coded only with syntaxes belonging to the 2nd pass, the syntaxes belonging to the 3-1 path may be coded.

When the number of bins encoded using context information reaches a threshold, syntax belonging to the 3-2 pass may be encoded from residual coefficients thereafter. For example, for the residual coefficients C8 to C15, the syntax abs_level and sign_flag belonging to the 3-2 pass may be encoded.

In this case, even when the number of bins encoded using the context information is smaller than the threshold value, when the difference between the number and the threshold value is not sufficient to encode syntaxes belonging to one pass with respect to the residual coefficient, the context It can be set so that abs_level is encoded without using information. For example, when sig_flag, gt_1_flag, par_flag, and gt_2_flag are set to be encoded using context information, the syntaxes may be encoded only when the difference between the number and the threshold is greater than 4. Alternatively, when the absolute value of the residual coefficient is 1, since the residual coefficient can be encoded only with sig_flag, gt_1_flag, and par_flag excluding gt_2_flag, the syntaxes can be encoded only when the difference between the number and the threshold is greater than 3. can Alternatively, when the absolute value of the residual coefficient is 0, since the residual coefficient can be encoded using only sig_flag, the syntax can be encoded only when the difference between the number and the threshold value is greater than 1.

In the illustrated example, it is shown that only four syntaxes belonging to one pass are encoded using context information. Unlike the described example, at least one of the syntax rem_level belonging to the 2-1 path and the syntax sign_flag belonging to the 3rd path may be encoded using context information. For example, when rem_level is encoded using context information, the counter may increase by the number of bins allocated to the syntax rem_level.

In the example shown in FIG. 20 , the flag par_flag may be set not to be coded using context information. 21 shows an example of this.

In FIG. 21 , one pass indicates syntaxes encoded using context information. Except for pass 1, passes 2, 3, and 4 indicate syntaxes that are encoded without using context information.

As in the illustrated example, the syntax par_flag may be classified into two passes different from one pass to which the encoded syntaxes belong by using context information. Accordingly, based on the syntax par_flag, the number of coded bins is not counted using context information.

When a coefficient of a bin encoded using context information is smaller than a threshold value, the absolute value of the residual coefficient may be encoded through one pass, two passes, three passes, or 4-1 passes. On the other hand, when the coefficient of the bin encoded using the context information is equal to or greater than the threshold value, the absolute value of the residual coefficient may be encoded through 4-2 passes.

When the flag par_flag is set to be encoded without using context information, the counter may be set not to increase with respect to the number of bins (ie, one) allocated to the flag par_flag. Accordingly, for each residual coefficient, the counter is incremented only for the bins assigned to the three syntaxes, sig_flag, gt_1_flag and gt_2_flag.

When the coefficient of a bin encoded using context information reaches a threshold, syntaxes belonging to the 4-2 pass may be encoded from the next residual coefficient. For example, with respect to C8 to C15, the absolute value of the residual coefficient may be encoded using the syntax abs_level belonging to the 4-2 path, and the sign of the residual coefficient may be encoded using the syntax sign_flag.

In the examples of FIGS. 20 and 21 , it is illustrated that a maximum of 5 gt_N_flags are available. Residual coefficients may be encoded using a smaller number or a larger number of gt_N_flags than the illustrated example. 22 is an example of this.

22 is a diagram illustrating an encoding mode using two gt_N_flags.

In the example shown in FIG. 22 , it is exemplified that sig_flag, gt_1_flag, par_flag, and gt_2_flag are included in one pass. By comparing the number of encoded bins and a threshold value using context information, it is possible to determine whether to encode the above four syntaxes for each residual coefficient.

In the example shown in Fig. 22, it is assumed that residual coefficients are encoded in the order of C15 to C0. In the example shown in FIG. 22 , it is exemplified that syntaxes belonging to one pass are encoded for C15 to C8. In this case, when the residual coefficient cannot be completely encoded by the syntaxes belonging to the first pass, the syntaxes belonging to the 2-1 pass and the third pass may be encoded.

When the number of bins encoded using context information is equal to or greater than a threshold value, syntaxes belonging to the 2-1 pass and the 3rd pass may be encoded. For example, for the residual coefficients C7 to C0, a syntax abs_level indicating an absolute value of the residual coefficient and a syntax sign_flag indicating a sign of the residual coefficient may be encoded.

In the example shown in FIG. 22 , the flag par_flag may be set not to be coded using context information. 23 shows an example of this.

In FIG. 23 , one pass represents syntaxes encoded using context information. Except for pass 1, passes 2, 3-1, 3-2, and 4 indicate syntaxes that are encoded without using context information. When the coefficient of a bin encoded using context information is smaller than a threshold value, the absolute value of the residual coefficient may be encoded through one pass, two passes, and a 3-1 pass. On the other hand, when the coefficient of the bin encoded using the context information is equal to or greater than the threshold value, the absolute value of the residual coefficient may be encoded through 3-2 passes.

When the flag par_flag is set to be encoded without using context information, the counter may be set not to increase with respect to the number of bins (ie, one) allocated to the flag par_flag.

Accordingly, for each residual coefficient, the counter is incremented only for the bins assigned to the three syntaxes, sig_flag, gt_1_flag and gt_2_flag.

In FIG. 23, it is assumed that the threshold value is set to 24.

Assuming that for each of the residual coefficients C15 to C8, the syntax sig_flag, gt_1_flag and gt_2_flag are encoded, the residual coefficient is set to 24 equal to the threshold value after encoding the syntaxes for C8 in 1 pass.

Accordingly, when encoding the residual coefficient C7, the absolute value of the residual coefficient C7 can be encoded as it is through the syntax abs_level included in the 3-2 path. That is, for the residual coefficients C7 to C0, the absolute values of the residual coefficients may be encoded using the syntax abs_level belonging to the 3-2 pass instead of the syntaxes belonging to the 1st pass and the 2-1 pass.

It is also possible to simplify the three passes illustrated in FIG. 22 into two passes. 24 is an example of this.

As in the example shown in FIG. 24 , sig_flag, gt_1_flag, par_flag, and gt_2_flag are configured in one pass (ie, one pass), and rem_level indicating the absolute difference and sign_flag indicating the sign are configured in one path (ie, 2). -1 pass). Also, for residual coefficients in which syntaxes belonging to one pass are not encoded, abs_level indicating the absolute value of the residual coefficient and sign_flag indicating a sign may be configured as one pass (ie, 2-2 passes).

It is also possible to set the number of paths and the configuration of each path to be different from the illustrated example. For example, instead of configuring all gt_N_flags as one pass, some gt_N_flags may be configured as one pass, and the remaining gt_N_flags may be configured as two passes.

As in the above example, residual coefficient encoding may be divided into a method using a maximum of m gt_N_flags and a method using a maximum of n gt_N_flags. m and n are different natural numbers, m may have a value greater than n, m may be a natural number such as 3, 4, 5 or 6, and n may be a natural number such as 2, 3 or 4. As an example, the first method may be a residual coefficient encoding method using a maximum of five gt_N_flags, and the second method may be a residual coefficient encoding method using a maximum of two gt_N_flags.

Information indicating whether to apply the first scheme using up to m gt_N_flags or the second scheme using up to n gt_N_flags may be signaled through the bitstream. As an example, whether the first scheme or the second scheme is applied may be determined based on transform_skip_flag indicating whether transform skip is applied. For example, when a transform is skipped in the current block (eg, when transform_skip_flag is 1), residual coefficients are encoded using the first method, whereas when a transform is not skipped in the current block (eg, when transform_skip_flag is 0) case), the residual coefficients may be encoded using the second method.

Alternatively, at a block or sub-block level, information for specifying an encoding method of a residual coefficient may be encoded and signaled.

Alternatively, information for specifying an encoding method of a residual coefficient may be encoded and signaled at a slice, picture, or sequence level.

As another example, the encoding method of the residual coefficient may be determined based on at least one of the size, shape, encoding mode, and intra prediction mode of the current block. As an example, when the current block is encoded by intra prediction, residual coefficients may be encoded based on the first method using a maximum of m gt_N_flags. On the other hand, when the current block is encoded by inter prediction, residual coefficients may be encoded based on the second method using a maximum of n gt_N_flags.

As another example, the encoding method of the residual coefficient may be determined according to whether lossless encoding is applied. As an example, when lossless encoding is applied, residual coefficients may be encoded based on the second method using a maximum of n gt_N_flags. On the other hand, when lossless coding is not applied, residual coefficients may be coded based on the first scheme using a maximum of m gt_N_flags. Alternatively, when lossless coding is applied, the second scheme is fixedly applied, and when lossless coding is not applied, either the first scheme or the second scheme can be adaptively selected.

As another example, the encoding method of the residual coefficient may be determined based on the bit-depth of the current image. For example, when the bit depth of the current image is 10 bits or less, the residual coefficients may be encoded based on the first method using a maximum of m gt_N_flags. On the other hand, when the bit depth of the current image is 10 bits, the residual coefficients may be encoded based on the first method using a maximum of m gt_N_flags. On the other hand, when the bit depth of the current image is greater than 10 bits, the residual coefficients may be encoded based on the second method using a maximum of n gt_N_flags.

After flipping the residual coefficients, the flipped residual coefficients may be encoded. Flipping may mean changing the positions of the residual coefficients to symmetric positions based on at least one of a horizontal direction or a vertical direction. The flipping mode may be determined as one of a horizontal mode, a vertical mode, or a mode in which vertical and horizontal are mixed.

Information for specifying whether flipping is applied or not or a flipping mode may be signaled through a bitstream. For example, a flag indicating whether flipping is applied may be encoded and signaled. When the flag indicates that flipping is applied, an index specifying one of a plurality of flipping modes may be encoded and signaled.

Alternatively, at least one of whether flipping is allowed to be applied, whether flipping is applied, or a flipping mode may be determined based on at least one of the size, shape, encoding mode, and intra prediction mode of the current block. As an example, when intra prediction is applied to the current block, it may be allowed to apply flipping to the current block. When it is allowed to apply flipping to the current block, a flag indicating whether flipping is applied to the current block may be encoded and signaled. On the other hand, when inter prediction is applied to the current block, it may not be allowed to apply flipping to the current block. When it is not permitted to apply flipping to the current block, encoding of a flag indicating whether flipping is applied to the current block may be omitted.

Alternatively, at least one of whether flipping is allowed to be applied, whether flipping is applied, or a flipping mode may be determined based on whether lossless encoding is applied. For example, when lossless coding is applied, it may be allowed to apply flipping to the current block. When it is allowed to apply flipping to the current block, a flag indicating whether flipping is applied to the current block may be encoded and signaled. On the other hand, when lossless coding is not applied, it may not be allowed to apply flipping to the current block. When it is not permitted to apply flipping to the current block, encoding of a flag indicating whether flipping is applied to the current block may be omitted.

After flipping is performed on a block-by-block or sub-block basis, the flipped residual coefficients may be encoded. Specifically, the flipped residual coefficients may be encoded based on at least one of a first scheme using at most m gt_N_flags or a second scheme using at most n gt_N_flags. As another example, when the residual coefficients are flipped, the first method may be fixedly applied or the second method may be fixedly applied. When at least one of the first scheme or the second scheme is fixedly applied, encoding of information indicating whether an encoding scheme is applied may be omitted.

The scan order may be different when the first scheme using up to m gt_N_flags is applied and when the second scheme using up to n gt_N_flags is applied. For example, when the first method is applied, the scan order of the residual coefficients and when the second method is applied, the scan order of the residual coefficients may be reversed.

For example, when the first method is applied, a first scan direction in which residual coefficients positioned at the lower right end of the block are first scanned and the residual coefficients positioned at the upper left end of the block are last scanned may be applied. On the other hand, when the second method is applied, a second scan direction in which the residual coefficients located at the upper left end of the block are first scanned and the residual coefficients located at the lower right end of the block are last scanned may be applied.

As in the above example, the absolute value of the residual coefficient may be encoded using at least one of sig_flag, par_flag, gt_N_flag, and rem_level. In this case, as described above, the remaining syntaxes except for rem_level may be coded with reference to various context information according to the properties of the surrounding coefficients. In this case, the number of referenceable context information may be determined according to the position of the pixel.

25 is an example of a case in which residual coefficients are encoded according to a scan order from the lower right residual coefficient to the upper left residual coefficient. As an example, FIG. 25 may be applied to the second scheme using a maximum of n gt_N_flags.

26 is an example of a case in which residual coefficients are encoded according to a scan order from the upper left residual coefficient to the lower right residual coefficient. As an example, FIG. 26 may be applied to the first scheme using a maximum of m gt_N_flags.

25 and 26 , a maximum of two or a maximum of five restoration coefficients may be referred to. For example, when the position of the residual coefficient is (x, y), a region including restoration coefficients having an absolute value of 1 or less of the sum of the x-coordinate difference with the residual coefficient and the y-coordinate difference or the restoration coefficient having an absolute value of 2 or less A region including the ? may be set as a peripheral restoration region.

Alternatively, if the reconstruction coefficients in the reconstruction area with the residual coefficients are out of a block boundary or there is a reconstruction coefficient that has not yet been restored in the scan order, the unavailable reconstruction coefficients may be excluded from the reference target.

When syntax is encoded using context information, values of syntaxes for reconstruction coefficients around residual coefficients or temporary reconstruction coefficients may be referred to. For example, during sig_flag encoding/decoding, values of sig_flag of reconstructed coefficients included in a reconstructed area around the residual coefficient may be summed.

Alternatively, an absolute value of a reconstructed coefficient or a partially reconstructed coefficient included in a reconstructed area around the residual coefficient may be calculated. Here, the absolute value of the partially reconstructed coefficient may mean a temporary reconstructed coefficient derived based on syntaxes included in one pass, for example, (sig_flag + gt_1_flag + par_flag + (gt_2_flag<<1)).

One of a plurality of context information may be specified using the derived value.

As another example, whether to use context information of a neighboring reconstruction region may be determined according to a residual coefficient encoding method. As an example, in at least one of the first method using up to n gt_N_flags and the second method using up to m gt_N_flags, context information of the surrounding reconstruction area is used, whereas in the other method, context information of the surrounding reconstruction area is not used. may not be For example, under an encoding method using a maximum of two gt_N_flags, syntaxes are encoded with reference to a neighboring reconstruction region, whereas under an encoding method using a maximum of five gt_N_flags, syntaxes may be encoded without referring to a neighboring reconstruction region. In addition to the sig_flag described above, other syntaxes may be encoded/decoded with reference to a neighboring reconstruction block. For example, when encoding rem_level or abs_level, the encoding may be performed with reference to a neighboring reconstruction block. When the above syntaxes are binarized by the exponential Gollum method or the truncated Rice method, the surrounding reconstruction block may be referred to when determining a binarization parameter (eg, a Rice parameter).

Alternatively, as described above, it may be determined whether to refer to the neighboring reconstruction block when determining the binarization parameter according to the encoding method of the residual coefficient. As an example, in one of the first scheme using up to n gt_N_flags and the second scheme using up to m gt_N_flags, a binarization parameter is determined by referring to a neighboring reconstruction block, while in the other, binarization is performed without referring to the neighboring reconstruction block. You can set parameters. When not referring to the neighboring block, the binarization parameter may be set to a value predefined in the encoder and the decoder. For example, the predefined value may be an integer such as 0, 1, or 2. Alternatively, information for determining a binarization parameter may be signaled through an upper header. For example, the information may be encoded and signaled at a slice, picture, or sequence level.

In the palette mode, pixels occurring a lot in a block to be encoded (hereinafter referred to as a current block) are displayed with a specific index, and then the specific index is encoded instead of the pixel and transmitted to the decoding device. A flag indicating whether the palette mode is permitted may be encoded and transmitted to the decoding apparatus. In this case, the flag may be coded only when the size of the current block is less than or equal to a preset size. The preset size may be determined based on the slice type of the slice to which the current block belongs, the encoding mode or the prediction mode of the current block. For example, when the current block belongs to the I slice, the palette mode may be used only when the size of the current block is 4x4. When the current block belongs to the B or P slice, the palette mode can be used only when the size of the current block is larger than 4x4 and smaller than 64x64. FIG. 27 is an example of a process of generating a palette table. For convenience of explanation, it is assumed that the size of the current block is 4x4. First, a histogram of 16 pixels existing in the current block is shown in FIG. 27 . In FIG. 27 , the horizontal axis indicates a pixel value (for example, a value from 0 to 255 in the case of a pixel quantized to 8 bits), and the vertical axis indicates the frequency of the pixel value. Thereafter, a quantization zone is set based on pixels that occur frequently. Pixels existing in the quantization zone are replaced with the pixel with the highest frequency, and one index is assigned to the pixel with the highest frequency. Information indicating the size of the quantization zone may be encoded and transmitted to a decoding apparatus. Alternatively, the size of the quantization zone may be determined based on at least one of the size, shape, and bit depth of the current block.

In FIG. 27 , a part represented by a thick solid line in the quantization zone means pixels a3, a8, a10, a11 having the highest frequency, and a part represented by a thin solid line means other pixels. In addition, pixels not included in the quantization zone (parts indicated by thick solid lines outside the quantization zone) are expressed as escape values, and in this case, they are quantized and encoded in addition to encoding by index.

Figure 28 shows an example for the palette table set in Figure 27.

In Fig. 28, each row of the palette table is expressed as a palette entry, and a different index is assigned to each entry. That is, the size of the palette table may mean the number of entries.

An entry is formed using pixels a3, a8, a10, and a11 having the highest frequency in each quantization zone, and an index is assigned to each entry. If an escape value exists, an escape can be placed in the last entry and an index can be assigned. That is, the last index in the palette can mean an escape value.

29 is an example of a process in which pixels in a block are allocated as indexes using a set palette table. In Figure 29, the allocated indexes are expressed as palette indexes.

Pixels existing in the block are replaced with indexes according to the set palette table, and the indexes are encoded and transmitted to the decoding device. In addition, when indicated as escape values (a5, a15 in FIG. 29), quantized a5' and a15' in addition to the index are encoded. In addition, the used palette table is also encoded and transmitted to the decoding device.

The palette mode may be applied on a block-by-block basis (eg, a coding unit, a prediction unit), and for this, flag information (pred_mode_plt_flag) indicating whether to use the palette mode may be signaled on a block-by-block basis. That is, when the value of the flag is 1, the palette mode is applied to the current block, and when the value of the flag is 0, the palette mode is not applied to the current block.

The flag may be adaptively encoded/decoded based on at least one of a prediction mode of the current block and a size of the current block. For example, the flag may be encoded/decoded only when the prediction mode of the current block is the intra mode. The flag may be encoded/decoded only when the prediction mode of the current block is not a skip mode. The flag may be encoded/decoded only when at least one of a width or a height of the current block is less than or equal to a predetermined first threshold size. Here, the first threshold size is a pre-defined value in the encoding/decoding apparatus, and may be any one of 16, 32, or 64. The flag may be encoded/decoded only when the product of the width and height of the current block is greater than a predetermined second threshold size. Here, the second threshold size is a value pre-defined in the encoding/decoding apparatus, and may be any one of 16, 32, or 64. However, the first threshold size and the second threshold size may be different values. If any one of the above conditions is not satisfied, the flag is not encoded/decoded, and in this case, the value of the flag may be set to 0.

Referring to Figure 30, it is possible to configure a palette table (palette table) for the palette mode of the current block (S3000).

The palette table may consist of at least one palette entry and a palette index identifying each palette entry. The palette table of the current block may be determined using the palette table of the previous block (hereinafter referred to as the previous palette table). Here, the previous block may mean a block coded or decoded before the current block.

Specifically, the palette entry of the current block may include at least one of a predicted palette entry or a signaled palette entry. The current block may use all or part of the palette entries used by the previous block, and among the palette entries used in the previous block, the palette entry reused in the current block is called a predicted palette entry.

The current block can use all of the palette entries in the previous palette table. Alternatively, the current block may use some of the palette entries of the previous palette table, and for this, a flag (PalettePredictorEntryReuseFlag, hereinafter referred to as a palette prediction flag) for specifying whether to reuse the palette entry may be used. The value of the palette prediction flag is assigned to each palette entry of the previous palette table, and the palette prediction flag (PalettePredictorEntryReuseFlag[i]) determines whether the palette entry corresponding to the palette index i in the previous palette table is reused in the palette table of the current block. can indicate whether For example, when the value of the palette prediction flag is 1, the palette entry corresponding to the palette index i in the previous palette table is reused in the palette table of the current block, and when the value of the palette prediction flag is 0, it is not reused No. A palette table of the current block may be constructed by extracting a palette entry having a value of the palette prediction flag of 1 from the previous palette table, and arranging them sequentially.

Meanwhile, the palette table of the current block may be initialized in units of a predetermined area. Here, the predetermined region may mean a parallel processing region or a CTU row of the current picture. If the current block belongs to the first CTU of the CTU row, the palette table of the current block may be initialized with the palette table of the neighboring CTU of the CTU to which the current block belongs. Here, the neighboring CTU may mean a CTU located above the CTU to which the current block belongs. That is, the palette table for the first CTU of the N-th CTU row may be initialized based on the palette table for the first CTU of the (N-1)-th CTU row. The initialized palette table may be updated based on the palette table of the previous block belonging to the same CTU row. The above-described embodiment is merely an example, and a method of configuring the palette table of the current block will be described in detail with reference to FIGS. 31 to 32 .

Meanwhile, the palette prediction flag may be signaled in the form of an encoded/decoded flag for each palette entry. Alternatively, the palette prediction flag may be encoded/decoded in the form of a binary vector based on run length encoding. That is, in the palette prediction flag array that specifies whether to reuse the previous palette entry, a syntax palette_predictor_run that specifies the number of palette prediction flags that is 0 between non-zero palette prediction flags may be encoded/decoded. This will be described later.

Alternatively, instead of encoding the run length, the palette prediction flag values may be directly encoded. This will be described later.

In addition, the palette table of the current block may further include a palette entry signaled through a bitstream, wherein the signaled palette entry is a palette entry that is not included in the previous palette table among the palette entries used by the current block can mean The signaled palette entry may be added after the predicted palette entry of the palette table.

Referring to FIG. 30 , a palette index may be determined in units of pixels of the current block ( S3010 ).

The current block may determine the palette index using at least one of an index mode (INDEX MODE) or a copy mode (COPY MODE).

Here, the index mode (INDEX MODE) may mean a method of encoding the palette index information (palette_idx_idc) in the encoding apparatus to specify the palette index used in the current block. The decoding apparatus may derive the palette index of the current pixel based on the encoded palette index information. The palette index information has a value between 0 and (MaxPaletteIndex-1), where MaxPaletteIndex may mean the size of the palette table of the current block or the number of palette entries constituting the palette table. In the index mode, the value of the palette index information signaled through the bitstream may be allocated as the palette index of the current pixel.

The copy mode (COPY MODE) may refer to a method of determining the palette index of the current pixel by using the palette index of the neighboring pixel according to a predetermined scan order. Here, as the scan order according to the present disclosure, a horizontal scan, a vertical scan, a diagonal scan, etc. may be used, and any one of them may be selectively used. To this end, a predetermined flag or index may be encoded/decoded. For example, the encoding apparatus encodes the flag as 0 when horizontal scan is applied as the scan order of the current block, and codes the flag as 1 when vertical scan is applied as the scan order of the current block. can do. The decoding apparatus may adaptively determine the scan order of the current block according to the coded flag. However, the present invention is not limited thereto, and a method of encoding/decoding the palette index according to the scan order will be described later.

In the copy mode, the palette index of the current pixel may be predicted based on the palette index of the neighboring pixel, or the palette index of the neighboring pixel may be copied and set as the palette index of the current pixel as it is. Here, the neighboring pixel may mean a pixel adjacent to the top, bottom, left, or right of the current pixel. In particular, the neighboring pixel may be located on the same horizontal line or the same vertical line as the current pixel.

For example, the copy mode is the first copy mode in which the palette index used by the pixel adjacent to the top or bottom of the current pixel is identically used as the palette index of the current pixel, the palette index used by the pixel adjacent to the left or right of the current pixel At least one of a second copy mode that uses the same as the palette index of the current pixel, or a third copy mode that uses the palette index used by diagonally adjacent pixels of the current pixel equally as the palette index of the current pixel can

Meanwhile, any one of the above-described first to third copy modes may be selectively used according to the scan order of the current block. For example, the first copy mode may be applied when the scan order of the current block is a vertical scan, and the second copy mode may be applied when the scan order of the current block is a horizontal scan.

In addition, the scan start position of the current block is not limited to the upper-left pixel of the current block, and other corner pixels of the current block (eg, lower-left pixel, upper-right pixel, and lower-right pixel) may be used as the scan start position. . Accordingly, according to the scan order and scan start position of the current block, the same palette index as the pixel adjacent to the top or left may be used as described above, or the same palette index as the pixel adjacent to the bottom or right may be used.

Any one of the above-described index mode and copy mode may be selectively used. For example, the encoding apparatus may encode a flag (run_copy_flag) indicating whether the copy mode is used. Here, if the copy mode is used, the encoding apparatus may encode the flag as 1, otherwise (ie, if the index mode is used), the encoding apparatus may encode the flag as 0.

Referring to FIG. 30, based on the palette table and the palette index, it is possible to predict the pixel of the current block (S3020).

Specifically, it is possible to extract a palette entry having a palette index of the same value as the palette index from the palette table of the current block, and predict/restore the pixel of the current block using this. For example, the value of the palette entry extracted from the palette table may be set as the predicted value or the restored value of the pixel of the current block.

However, when the palette index indicates the last palette entry among the palette entries in the palette table of the current block, the pixel may be inferred as being encoded in the escape mode (ESCAPE MODE). Here, the escape mode does not use the palette entry of the pre-configured palette table, but instead predicts / restores the pixel based on the additionally signaled palette escape value It can mean a method. . Accordingly, a pixel having a palette index equal to (the number of palette entries - 1) may be predicted/restored using the additionally signaled palette escape value.

The above-described embodiment is merely an example, and various methods of configuring the pallet table will be described in detail with reference to the following drawings.

When the current block is encoded in the palette mode, the same palette table used in the encoding apparatus must also exist in the decoding apparatus. Therefore, it is necessary to encode the palette table in the encoding device. Therefore, it is possible to encode the number of palette entries existing in the palette table, and to encode the pixel value assigned to each entry. However, in this method, as the size of the block increases and the number of entries increases, the amount of encoded bits rapidly increases. Therefore, if the palette mode was used in the previous block, the amount of bits required to encode the palette table can be greatly reduced by generating the palette table of the current block based on the palette table used in the previous block. Here, the previous block means a block that has been encoded/decoded before the current block. Specifically, at least one of a flag indicating whether to configure the palette table of the current block based on the previous palette table or a palette prediction flag indicating whether to add an entry included in the palette table of the previous block to the palette table of the current block is available.

31 is a method of reducing the bit amount of a palette table to be currently encoded by using a palette prediction flag.

In FIG. 31 , the palette table A may mean a palette table existing in a block encoded using the palette mode before the current block. Here, the previous block may be a neighboring block adjacent to the current block. The neighboring block may include at least one of an upper neighboring block, a left neighboring block, an upper left neighboring block, an upper right neighboring block, or a lower left neighboring block.

In the palette table A, it is possible to specify whether or not to be used as it is in the current palette table by using a palette prediction flag for each entry. For example, if the palette prediction flag is 1, it may mean that the corresponding entry is used as it is in the current palette table, and if 0, it may mean that the corresponding entry is not used in the current palette table. The index allocated to the entries predicted from the palette table A may be set to be the same as the index allocated to the palette table A. Alternatively, the index of each entry in the ascending/descending order of the indexes allocated to each entry in the palette table A may be reassigned.

In the example of Figure 31, the first entry, the third entry, and the fifth entry are used in the current palette table, so you can put them in order from the first entry to the third entry of the current palette table, and configure a new entry only from the fourth entry to the fifth entry. have. In this case, the palette prediction flag is encoded first, and the number of remaining entries (two in the example of FIG. 31: the fourth entry and the fifth entry of the current palette table) can be encoded. After that, the remaining entries may be encoded as much as the number of remaining entries. By transmitting the information to the decoding device, the decoding device can also generate the same palette table as the coding device and predict/restore the current block.

In order to configure the same palette table as the encoding apparatus in the decoding apparatus, information related to the palette table may be encoded and signaled.

For example, through a bitstream, at least one of information indicating the number of entries included in the current palette table or information indicating a difference value with the size of the previous palette table may be encoded and transmitted to the decoding apparatus. Here, the size of the palette table may indicate the maximum number of palette entries that the palette table can contain.

Alternatively, the size of the palette table may have a fixed value in the encoder and decoder. Alternatively, the bit depth of the current image, the color component of the current block (eg, whether it is a luminance component or a chrominance component), the size of the current block, or Based on at least one of the shapes, the size of the pallet table may be determined.

The palette table of the current block may consist of palette entries reused in the previous palette table and palette entries newly added. Information about the reused palette entries constituting the palette table and the newly added palette entries may be encoded and signaled.

As an example, a palette prediction flag indicating whether a palette entry included in a previous palette table is reused may be encoded.

When the number of palette entries reused in the previous palette table is smaller than the maximum number of palette entries that the palette table can contain, information on the new palette entry may be additionally encoded. In this case, the number of palette entries to be newly added to the palette table may be a value obtained by subtracting the number of palette entries reused from the size of the palette table. For example, if the maximum number of palette entries that the palette table can contain is 40, the sum of the number of reused palette entries and the number of new palette entries cannot exceed 40.

On the other hand, when the number of reused palette entries is the same as the maximum number of palette entries that the palette table can contain, no new palette entries are added to the palette table anymore. In this case, encoding of information for a new palette entry may be omitted.

It can be set so that the number of times of determining whether to reuse the palette entry does not exceed a threshold. That is, the number of palette prediction flags may be set so as not to exceed a threshold value. For example, if the threshold value is 10, for up to 10 of the palette entries included in the previous palette table, it can be determined whether to reuse. Accordingly, a maximum of 10 palette prediction flags may be generated.

As another example, it may be set so that the number of reused palette entries does not exceed a threshold value. That is, the number of palette prediction flags having a value of True may be set so as not to exceed a threshold value.

The threshold value may be encoded through an upper header such as a slice, a picture, or a sequence. Alternatively, information for specifying a threshold value for each block may be encoded and signaled.

As another example, an encoder and a decoder may use a fixed value threshold.

Alternatively, the threshold value may be determined based on at least one of a size, a shape, a color component (eg, a luminance component or a chrominance component) of the block, or a bit depth.

Alternatively, it is possible to limit the number of entries (hereinafter, referred to as the maximum number of predictions) that can be brought by using the palette prediction flag. As an example, information on the maximum number of predictions may be signaled through a bitstream. Alternatively, the maximum number of predictions may be determined based on at least one of the size of the palette table, the size/shape of the current block, the size/shape of the previous block, or the size of the previous palette table.

As an example, only a certain percentage of the size of the current palette table may be obtained by using the palette prediction flag to bring an entry from the previous palette table, and the remaining ratio may be unconditionally generated from the current palette table. For example, if the size of the current palette table is 6 and the ratio is set to 50%, up to 3 entries from the previous palette table are fetched using the palette prediction flag, and the remaining 3 entries can be created unconditionally from the current palette table. have. Accordingly, when the number of entries having the value of the palette prediction flag of 1 reaches three, encoding of the palette prediction flag may be omitted for subsequent entries.

Alternatively, when the size of the previous block is smaller than a preset threshold, it may be set so that the palette entries included in the palette table of the previous block are not added to the palette table of the current block. That is, when the size of the previous block is smaller than a preset threshold, encoding of the palette entry prediction flag for the palette entries of the previous block may be omitted, and the value may be regarded as 0.

For example, if the threshold value is 16 and the number of samples including the previous block is less than 16, the palette entry of the previous block may not be added to the palette table of the current block.

The threshold value may be encoded in an upper header and transmitted to a decoder. Alternatively, the encoder and the decoder may use a fixed threshold.

Alternatively, according to the size of the previous block, the number of palette entries that can be added to the palette table of the current block from the palette table of the previous block may be determined.

Alternatively, it is possible to predict entries to be included in the current palette table from a plurality of previous palette tables. For example, if the number of palette prediction flags having a value of 1 is smaller than the size of the current palette table, the entry is brought to the current palette table using a prediction flag for each of the entries included in the first previous palette table. A method of continuously allocating palette prediction flags using a second previous palette table that is earlier than the first previous palette table is also possible.

Alternatively, the encoding order of the palette prediction flag may be determined by considering the indexes of the entries included in the first previous palette table and the entries included in the second previous palette table. As an example, after encoding the palette prediction flag for the entry with index 0 included in the first previous palette table, the palette prediction flag may be encoded for the entry with index 0 included in the second previous palette table. Then, after encoding the palette prediction flag for the entry with index 1 included in the first previous palette table, it is possible to encode the palette prediction flag for the entry with index 1 included in the second previous palette table.

Alternatively, the palette table candidate list may be configured, and at least one of a plurality of previous palette table candidates included in the palette table candidate list may be used when encoding the current palette table.

32 is a method of reducing the bit amount of a palette table to be currently encoded by using a palette prediction flag. In FIG. 32 , RT denotes a pixel located at the upper right of the block, and LB denotes a pixel located at the bottom left of the block. For example, reference may be made to at least one of five neighboring blocks, that is, blocks including pixels A to E in FIG. 32 . Thereafter, the referenced block may be encoded as an index and transmitted to a decoding apparatus. Alternatively, only blocks at positions pre-defined in the encoding/decoding apparatus may be referred to among the blocks each including the pixels A to E described above. Here, the pre-defined position may be the upper block (B) or the left block (A). In this case, encoding of an index specifying the referenced block may be omitted.

It is possible to initialize / configure the palette table for the current block using only the palette entry of the block corresponding to the index.

Alternatively, if the palette table of the current block cannot be filled beyond the standard value using only the palette table of the referenced block, the palette table to be currently encoded can be filled by additionally designating a block based on an additional index. In this case, the encoding/decoding apparatus may refer to a pre-promised fixed number of blocks, and information specifying the number of referenced blocks may be transmitted through an upper header. Alternatively, a method in which the encoding/decoding apparatus refers to the same fixed number of neighboring blocks according to the size/form of the block and the size of the palette table is also possible. Alternatively, in addition to the location of FIG. 32, a method of fetching the palette table from the block by designating M blocks encoded in the palette mode before the current block in the encoding order as indexes is also possible. Alternatively, a method of fetching the palette table from the block by designating the block included in the collocated picture as an index is also possible.

Alternatively, a method of constructing a palette table candidate list is also possible. All used palette tables are stored in the candidate list starting from the block existing in the first position of the image until just before the current block. Alternatively, after setting the number N of tables to be stored in the candidate list, the N palette tables are stored in the candidate list. That is, when the encoding of the block is completed, the palette table of the encoded block may be stored in the candidate list. In this case, if the same palette table candidate as the palette table to be added to the candidate list exists, the palette table may not be added to the candidate list. Alternatively, the palette table may be added to the candidate list, and the same palette table candidate as the palette table may be deleted from the candidate list.

In this case, the method in which the palette table candidates in the candidate list are stored has a higher priority as it is closer to the current block, and may have a lower priority as it is further away from the current block. Alternatively, the priority may be set according to the size or reference frequency of the palette table. According to this priority, when the number of stored tables exceeds N, it can be deleted from the candidate list starting from the palette table with a lower priority.

Alternatively, in a parallel processing structure, a method of configuring a palette table list separately for each area to be processed in parallel is also possible. Alternatively, it is also possible to separately configure the palette table list for each CTU row of the area. In this case, if the palette table list is separately for each area in which parallel processing is performed, the number of palette tables stored in the palette table list may be very small in the initial part of the area. Therefore, it is also possible to fill a preset initial palette table without filling the palette table from the beginning for each area in which parallel processing is performed. For example, as shown in FIG. 30, the initial palette table may be the palette table of the first CTU of the previous CTU row. Alternatively, the preset initial palette table may be a palette table derived from the entire image, not a palette table derived for each block. In this case, each entry value of the palette table derived from the entire image may be encoded through a higher header along with the number of entries. Alternatively, when constructing the initial palette table, it is also possible to set a quantized value as an entry value according to the representation bit of a pixel. For example, when an 8-bit pixel is quantized into 5 (5 entries), 0 to 255 can be divided into 5 zones and set as entries and encoded using the representative values of each zone. Alternatively, if 0 to 255 are equally quantized, only information indicating that quantization was performed equally and information indicating how many quantizations were performed may be encoded through the upper header.

Alternatively, a method of configuring the entries included in the palette table as a palette entry candidate list is also possible. Entries included in the palette table of the encoded block can be added to the entry candidate list. In this case, among the entries included in the palette table, only entries having an index smaller than the threshold may be included in the entry candidate list. When the number of entries included in the palette table of the current block is less than the maximum number, the palette table may be configured with reference to the candidate entries included in the palette entry candidate list.

Palette entries included in the palette table of the encoded/decoded block may be added to the palette entry candidate list. When new palette entries are added to the palette entry candidate list, the smallest index may be assigned to the newly added palette entries. And, by adding the number of palette entries that are newly added to the indexes of the palette entries existing in the palette entry candidate list, it is possible to update the indexes of the existing palette entries.

As new palette indices are added, when the number of palette entries included in the palette entry candidate list exceeds the maximum value, the existing palette entries may be removed from the palette entry candidate list in the order of the highest index.

After configuring a palette table based on the palette prediction flag, blocks may be encoded/decoded using the configured palette table. When the encoding/decoding of the block is completed, the palette entry candidate list may add palette entries included in the palette table.

For example, when the palette table includes a0, a2, a4, a5, a7, the above palette entries may be added to the palette entry candidate list.

If the same palette entry as the palette entry to be added to the palette entry candidate list is already stored in the palette entry candidate list, the duplicate palette entry may not be added to the palette entry candidate list.

Alternatively, if the same palette entry as the palette entry to be added to the palette entry candidate list is already stored in the palette entry candidate list, the previously stored palette entry is removed from the palette entry candidate list, and the duplicate palette entry is removed from the palette entry candidate list can be added to

In the above-described example, it has been described that all palette entries included in the palette table of the encoded/decoded block are added to the palette entry candidate list.

In order to reduce the complexity of constructing the palette entry candidate list, only those whose index is less than or equal to a threshold value among palette entries may be added to the palette entry candidate list.

Alternatively, when the size of the block is smaller than a preset threshold, the palette entries included in the palette table may not be added to the palette entry candidate list. For example, when the number of pixels included in the block is less than or equal to the threshold, the palette entries included in the palette table of the block may not be added to the palette entry candidate list. Accordingly, the palette entry included in the palette table of the block in which the number of pixels included in the block is less than or equal to the threshold value cannot be utilized as a prediction palette entry in constructing the palette table of the next block.

On the other hand, when the size of the block is greater than or equal to a preset threshold, the palette entries included in the palette table may be added to the palette entry candidate list.

The threshold value may be encoded in an upper header and transmitted to a decoder. Alternatively, the encoder and the decoder may use a fixed threshold. For example, the threshold may be a natural number such as 4, 8, 16, or 32.

Alternatively, based on the size of the block or the size of the next block, the number of palette entries that can be added to the palette entry candidate list may be determined. For example, if the size of the block is less than or equal to the threshold, a maximum of n palette entries may be added to the palette entry candidate list, whereas if the size of the block is greater than the threshold, a maximum of m palette entries are palette entry candidates can be added to the list. Here, n may be a natural number smaller than m.

A palette table predefined in the encoder and the decoder may be used.

When the encoder and decoder use a predefined palette table, it is not necessary to encode the palette table for each block.

The predefined palette table means that the size of the palette table and/or pixel values allocated to palette entries are predefined in the encoder and the decoder.

After storing a plurality of predefined palette tables, an index specifying one of the plurality of palette tables may be encoded and transmitted to the decoder.

Alternatively, after defining only pixel values allocated to each palette entry, only information indicating an index allocation order between palette entries may be encoded.

For example, when the minimum value of the residual value in the block is -3, index 0 is assigned to a palette entry having a pixel value of -3, index 1 is assigned to a palette entry having a pixel value of +4, and a pixel value of -4 Index 2 can be assigned to the in-palette entry.

Alternatively, the minimum value m in the block may be encoded and transmitted to the decoding apparatus, and an index of each of the palette entries may be determined based on the minimum value m. As an example, index 0 may be allocated to the palette entry equal to the minimum value m, and indexes may be allocated in an order similar to the minimum value m. For example, an index assigned to a palette entry having a small difference from the minimum value m may have a smaller value than an index assigned to a palette entry having a large difference from the minimum value m.

Whether to use the predefined palette table may be determined based on whether lossless encoding is applied. For example, when lossless encoding is applied, a predefined palette table is used, and when lossless encoding is not applied, the decoder may configure and use the palette table in the same way as the encoder.

Even when the residual value is encoded using the palette table, the method of configuring the palette table may be set differently depending on whether lossless encoding is applied.

The above-described palette table may be used to derive a predicted value, a reconstructed value, or a residual value of a sample.

In the case of the palette prediction flag, a run length encoding method may be used. A continuous sequence of identical data is called a run, and the continuous length is expressed as a run length. For example, if there is a string aaaaaabbccccccc, 6 a, 2 B, and 7 c, so it can be expressed as 6a2b7c. Such an encoding method is called a run-length encoding method. When encoding the palette prediction flags using run-length encoding, the number of 0's, the number of 1's, etc. can be expressed. Alternatively, run-length encoding may be performed only on 0, and conversely, run-length encoding may be performed on only 1 as well.

In this embodiment, it is assumed that the palette table of the previous block uses 8 palette entries with palette indexes of 0 to 7.

The image encoding apparatus determines whether the corresponding palette entry is reused as the palette entry of the current block for each of the palette entries 0 to 7 of the previous block, and if the palette entry is reused as the palette entry of the current block, the corresponding palette The value of the palette prediction flag for the entry can be set to 1, otherwise 0, respectively. For example, as shown in FIG. 35, among the palette entries of the previous block, the

palette entries

0, 1, 3, and 7 are reused as the palette entries of the current block, and the remaining palette entries are not reused, A binary vector represented by 11010001 may be generated.

Then, encoding at least one of the number of 1s in the binary vector (that is, the number of palette entries reused as the palette entry of the current block among the palette entries of the previous block) or the number of 0s preceding 1 in the binary vector Signaling may be performed by a decoding device. For example, since the number of 1s in the binary vector is 4, 4 can be encoded as the number of palette entries of the previous block that is reused as the palette entry of the current block. In addition, the number of 0s preceding 1 in the binary vector, that is, 0, 0, 1, 3 may be sequentially encoded.

The decoding apparatus receives at least one of information (palette_entry_run) about the number of 0's preceding 1 in the binary vector or information about the number of palette entries of the previous block reused as the palette entry of the current block from the encoding device, and It can be used to compose the palette table of the current block.

For example, the decoding apparatus sequentially extracts information (palette_entry_run), that is, 0, 0, 1, 3 about the number of 0s before 1 in the binary vector, from the bitstream, and using this, the palette entry of the previous block A binary vector indicating whether to reuse or not, that is, 11010001 may be restored. If a value of 1 occurs in the process of restoring a binary vector, the palette entry of the previous block corresponding to the value of 1 may be inserted into the palette table of the current block. Through this process, the palette table of the current block can be configured by selectively reusing some palette entries from the palette table of the previous block.

After encoding the palette table, the palette index assigned to each pixel of the current block must also be encoded. 36 is an example of a scan order performed in a current block.

The main purpose of the scan sequence shown in FIG. 36 is to perform scanning in consideration of directionality. As shown in FIG. 36(a) , if characteristics of pixels existing in the current block have similar values in the horizontal or vertical direction, the possibility of clustering among the same indexes increases if scanning is performed as shown in FIG. 36(a). Alternatively, if characteristics of pixels existing in a block have similar values in the z-direction or diagonal direction as shown in FIG. 36(b), the possibility of clustering among the same indices increases when scanning is performed as shown in FIG. 36(b).

The encoding apparatus may indicate which scan method is used as an index, encode it, and transmit it to the decoding apparatus. Alternatively, the scan order may be determined according to the size and shape of the current block.

After selecting a scan method in which indices having the same value are highly likely to be clustered among a plurality of scan methods, the index for each pixel may be encoded. In this case, the index of the current pixel may be derived using run merge encoding. The run merge encoding method represents a method of deriving an index of a current pixel from an index of an adjacent pixel.

When run merge encoding is applied, it may be determined whether to encode the index of the current pixel as it is, depending on whether the index of the current pixel and the adjacent pixel are the same. Here, the adjacent pixel may include at least one of left and right adjacent pixels, an upper adjacent pixel, a lower adjacent pixel, or diagonally adjacent pixels of the current pixel, and the location of the adjacent pixel is determined by the scan applied to the current block. It may be adaptively determined according to the direction.

37 is an exemplary diagram for describing a pixel adjacent to a current pixel.

For convenience of description, it is assumed that a horizontal scan is applied to the current block.

In FIG. 37 , A indicates a current pixel to be encoded, and B indicates a pixel immediately preceding the current pixel in a scan order. C indicates a neighboring pixel adjacent to the current pixel among pixels included in an adjacent line (row/column). The adjacent line may be one of a top row, a bottom row, a left column, or a right column, depending on the scan direction.

Based on whether the run type or index of the current pixel A is the same as the run type or index of the previous pixel B, the run type flag may be set. Here, the run type is 'ABOVE' indicating that the index of the current pixel A is the same as the index of the neighboring pixel (ie, the neighboring pixel C) included in the adjacent line, or 'INDEX' indicating that the index of the current pixel A is encoded as it is. can point to one of them.

The value of the run merge flag may be set based on the run type of the previous pixel B and whether the index of the current pixel A is the same as that of the adjacent pixel B or C.

For example, when the run type of the previous pixel B is 'ABOVE', when the indexes of the current pixel A and the neighboring pixel C are the same, the value of the run merge flag may be set to 1 (true). On the other hand, when the run type of the previous pixel B is 'INDEX' and the indexes of the current pixel A and the previous pixel B are the same, the value of the run merge flag may be set to true.

When the value of the run merge flag is true, the decoder may derive the index of the current pixel with reference to the run type of the previous pixel B. For example, when the run type of the previous pixel B is 'ABOVE', the run type 'ABOVE' may be applied to the current pixel A to derive the index of the neighboring pixel C as the index of the current pixel A. On the other hand, when the run type of the previous pixel B is 'INDEX', the index of the previous pixel B may be derived as the index of the current pixel A.

When the run types of the current pixel A and the previous pixel B are different or the indexes of the current pixel A and the previous pixel B are different, the value of the run merge flag may be set to 0 (false).

When the value of the run merge flag is false, run type information indicating the run type of the current pixel A may be encoded/decoded. The run type may indicate either 'ABOVE' indicating that the index of the current pixel A is the same as the index of the neighboring pixel C or 'INDEX' in which the index of the current pixel A is encoded as it is.

The run type information may be a 1-bit flag or an index specifying one of a plurality of run types. For example, when the syntax run_type indicating the run type indicates 'ABOVE', it indicates that the index of the current pixel A is the same as the index of the neighboring pixel C. In this case, the index of the neighboring pixel C may be derived as the index of the current pixel A.

On the other hand, when the syntax run_type indicating the run type indicates 'INDEX', it indicates that index information indicating the index of the current pixel A is encoded/decoded. For example, when the run type indicates 'INDEX', palette_idx specifying one of a plurality of palette entries included in the palette table may be encoded/decoded.

In this case, when the index indicates an escape value, information for deriving an escape value may be additionally encoded.

A pixel located at the top row in the current block does not have a pixel adjacent to the top. Accordingly, with respect to the pixel located in the uppermost row, encoding of run type information may be omitted and it may be estimated that the run type is 'INDEX'.

When the value of the run merge flag is false, the decoder may derive the index of the current pixel A using the run type information. For example, when the run type information indicates 'ABOVE', the index of the neighboring pixel C may be derived as the index of the current pixel A. On the other hand, when the run type information indicates 'INDEX', index information may be additionally parsed, and the index information of the current pixel A may be derived based on the parsed index information.

For the first pixel in the current block, encoding/decoding of the run merge flag may be omitted. This is because, for the first pixel, there is no previous pixel whose index is coded. When the encoding of the run merge flag is omitted, it can be estimated that the value is 0 (false).

As in the above example, when the run merge candidate method is applied, the index of the current pixel may be derived based on at least one of the syntax run_merge_flag, run_type, and palette_idx. In this case, the above syntaxes may be encoded without using context information. A coding method that does not use context information may be defined as bypass coding.

Alternatively, at least one of the above syntaxes may be set to be encoded using context information. For example, when the syntax run_merge_flag is encoded, context information may be referred to. When context information is used, the probability that the value of the run merge flag is 1 or the probability that the value of the run merge flag is 0 may be determined based on the value of the previous run merge flag.

38 shows an example of encoding a run merge flag using context information.

In encoding the run merge flag, a variable PREV_POS indicating the scan order of a pixel having the highest scan order among pixels in which the value of the run merge flag is set to 0 may be used. Specifically, in the scan order of the current pixel, a context information index value may be derived by differentiating variables PREV_POS and 1, and a run merge flag may be encoded using the derived context information index value.

In this case, when the first run merge flag is encoded, since the previously encoded run merge flag does not exist, the value of the variable PREV_POS may be set to an initial value (eg, 0). Accordingly, for the first palette prediction flag, the context information index value may be set to -1.

Whenever a run merge flag with a value of 0 is encoded, the variable PREV_POS may be updated. On the other hand, when the run merge flag having a value of 1 is encoded, the variable PREV_POS may be maintained as it is.

In the example shown in FIG. 38 , for a pixel whose scan order number is 6, the variable PREV_POS is exemplified as having a value of 3. Accordingly, a context information index for a pixel having a scan order of 6 may be set to 2. When encoding the run merge flag of the pixel having the scan order number 6, the probability of the run merge flag may be determined according to the value of the context information index, and the palette prediction flag may be encoded based on the determined probability.

In FIG. 38 , the variable PREV_POS has been described as indicating the position of a pixel having the run merge flag having a value of 0, but the variable PREV_POS may be set to indicate the position of a pixel having the run merge flag having a value of 1.

39 is an example showing the range of the context information index.

The maximum value of the context information index may be set not to exceed a predefined threshold. When a value derived by differentiating the variables PREV_POS and 1 at the position of the current pixel exceeds a threshold value, the value of the context information index may be set to the maximum value. In FIG. 39 , it is illustrated that the maximum value is 4.

It may be set so that the minimum value of the context information index does not become less than a predefined threshold. When a value derived by differentiating the variables PREV_POS and 1 at the position of the current pixel is less than the threshold, the value of the context information index may be set to the minimum value. In FIG. 39 , it is illustrated that the minimum value is zero. Accordingly, the context information index for the first pixel may be changed from -1 to 0.

The maximum and/or minimum values of the context information index may be defined in the encoder and the decoder. Alternatively, information indicating the maximum and/or minimum values of the context information index may be signaled through the bitstream. For example, the information may be encoded and signaled at a higher level such as a slice, a picture, or a sequence.

After the block is divided into a plurality of regions, index-related information may be encoded for each region.

The example of FIG. 40(a) shows a case where the block size is 16x4, and the example of FIG. 40(b) shows a case where the block size is 8x8. For convenience of description, it is assumed that a horizontal scan is applied to a block.

A block may be divided into regions of a predefined size. For example, when the predefined size is 16, the block may be divided into a plurality of regions in units of 16 pixels. For example, in the example of FIG. 40A , the block is divided into 16x1 sized regions, and in the example of FIG. 40B , the block is divided into 8x2 sized regions.

Index-related information may be encoded for each region. For example, after encoding of the index-related information in the N-th region (eg, at least one of run_merge_flag, run_type, or palette_idx) is completed, the index-related information in the N+1 th region may be encoded. Alternatively, the encoding of index-related information may be parallel-processed between regions.

Index-related information may be coded independently between regions. For example, when encoding index-related information within a predetermined region, other regions may not be referred. For example, when the first pixel of the region 2 is encoded, information on the last pixel of the region 1 may not be referenced. Accordingly, encoding of the run merge flag may be omitted for the first pixel in each region. Since index-related information is coded independently between regions, a scan order may also be independently assigned between regions. For example, by assigning scan order numbers 0 to 15 for each area, pixels may be distinguished.

Also, encoding of run type information may be omitted for pixels located in the uppermost row in each region.

As another example, index-related information may be encoded by providing inter-region dependency. For example, index-related information of pixels positioned in the uppermost row of region 2 may be encoded with reference to index-related information of pixels positioned in the lowermost row of region 1. When the inter-region dependency is given, the pixels may be distinguished by assigning a continuous scan order between the regions. For example, scan orders 0 to 15 may be allocated to pixels belonging to region 1, while scan orders 16 to 31 may be allocated to pixels belonging to region 2 .

Also, the value of the run merge flag may be coded for the first pixel in the remaining area except for the first area. As an example, the run merge flag of the first pixel of region 2 may be encoded with reference to the last pixel of region 1 (eg, a pixel having a scan order of 15).

Information indicating whether inter-region index-related information is independently encoded or dependently encoded may be encoded and signaled. Alternatively, based on at least one of the size or shape of the current block, it may be determined whether the inter-region index-related information is independently coded or dependently coded.

In the above-described example, it has been exemplified that index-related information can be encoded by dividing one block into a plurality of regions and then providing dependencies between regions. By extending this, inter-block dependency may be given to encode index-related information.

When a neighboring block that has been encoded/decoded exists on the top or left side of the current block, index-related information may be coded with reference to the neighboring block. As an example, the run merge flag for the first pixel in the current block may be encoded with reference to the neighboring block. Specifically, the run merge flag for the first pixel of the current block is set by referring to the pixel having the last scan order in the upper neighboring block or the left neighboring block, or the pixel adjacent to the first pixel of the current block in the upper neighboring block or the left neighboring block. can be encoded.

For example, in the example shown in FIG. 41 , the run merge flag for the first pixel C in the current block may be encoded with reference to the pixel A neighboring to the top or the pixel B neighboring to the left.

In addition, run type information may be coded with reference to a neighboring block even for pixels located in the uppermost row in the current block.

It can be set to preferentially refer to either the upper neighboring block or the left neighboring block of the current block. As an example, it may be set to first refer to the left neighboring block among the left neighboring block and the upper neighboring block. The upper neighboring block may be referred to when the left neighboring block is unavailable. That is, when the run-type flag encoding for the first pixel C in the current block is performed, the pixel B is preferentially referenced, but when the pixel B is unavailable, the pixel A may be referenced.

If the current pixel index points to an escape value, the escape value must be additionally encoded. You can quantize escape values and encode quantized escape values. In this case, quantization may be performed using an initial quantization parameter (QP) defined at the SLAS level. Through the SLAS header, an initial quantization parameter may be signaled.

Alternatively, an offset value indicating a difference between a quantization parameter defined at the slice level and a quantization parameter applied to the current block may be encoded. The offset may be coded in units of blocks.

For example, when the value of the initial quantization parameter defined at the slice level is 23 and the value of the quantization parameter for the current block is 27, an offset having a value of 4 may be encoded and signaled.

Depending on whether an escape value exists in the current palette table, the size of the palette table (ie, the maximum number of palette entries) and/or the binarization method of the index may be different. Accordingly, information indicating whether an escape value exists in the current palette table can be encoded and signaled. The information may be a 1-bit flag.

When the flag indicates that an escape value exists, the size of the current palette table may be increased by one. On the other hand, when the flag indicates that the escape value does not exist, it is possible to maintain the size of the current palette table.

Instead of encoding the escape value as it is, the difference between the escape value and the predicted value derived through intra prediction may be encoded.

Figure 42 (a) shows the current block and reference pixels adjacent to the current block, Figure 42 (b) shows the palette entry for the current block.

In (a) of FIG. 42 , the pixel assigned with the index 4 is encoded as an escape value. In this case, instead of encoding the value A of the pixel as it is, a difference value derived by differentiating a prediction value obtained through intra prediction from the value A of the pixel may be encoded.

For example, when the intra prediction mode in the vertical direction is applied, a difference value (ie, A-R2) derived by differentiating the upper reference pixel R2 from the pixel value A may be encoded. Alternatively, after quantizing the derived difference value, the quantized difference value may be encoded.

The intra prediction mode used to derive the difference value may be at least one of planar, DC, vertical direction, horizontal direction, upper right diagonal direction, upper left diagonal direction, or lower left diagonal direction.

An escape value can be derived by using a fixed intra-picture prediction mode in the encoder and decoder.

Alternatively, information specifying an intra prediction mode for deriving an escape value among a plurality of intra prediction modes may be encoded and signaled. In this case, when there are two available intra prediction modes, the information may be a 1-bit flag. For example, when only the intra prediction mode in the vertical direction or the horizontal direction is available, the intra prediction mode used to derive the escape value may be determined based on the flag.

Alternatively, the intra prediction mode used to derive the escape value may be determined based on at least one of the size and shape of the current block and the intra prediction mode of the neighboring block.

Instead of deriving a difference value using a reference pixel outside the current block, a difference value may be derived using an adjacent pixel in the current block. Here, the adjacent pixel indicates a pixel whose index is determined before the current pixel in the scan order.

43 (a) is an example of deriving a difference value with respect to an escape value using a reference pixel, and FIG. 43 (b) is an example of deriving a difference value with respect to an escape value using an adjacent pixel. For convenience of description, it is assumed that the scan order is in the horizontal direction.

When a reference pixel outside the current block is used, a difference value between the value of the current pixel and the reference pixel located in the vertical or horizontal direction of the current pixel may be derived. As an example, in the example illustrated in FIG. 43A , a difference value may be derived by differentiating the reference pixel R2 or the reference pixel R6 from the value A of the pixel having the index 4 . In this case, when the vertical intra prediction mode is applied, a difference value is derived using the reference pixel in the vertical direction, and when the horizontal intra prediction mode is applied, the difference value can be derived using the reference pixel in the horizontal direction. have.

When a pixel in the current block is used, a difference value between the value of the current pixel and a neighboring pixel adjacent to the current pixel may be derived. As an example, in the example shown in FIG. 43B , a difference value may be derived by differentiating the value of the right neighboring pixel or the value of the upper neighboring pixel from the value A of the pixel having the index 4 . In this case, when the vertical intra prediction mode is applied, a difference value is derived using the neighboring pixels in the vertical direction, and when the horizontal intra prediction mode is applied, the difference value can be derived using the neighboring pixels in the horizontal direction. have.

Based on whether or not lossless encoding is applied, a method of deriving a difference value may be determined. For example, when lossless encoding is not applied, as in the example shown in FIG. 43A , a difference value may be derived using a reference pixel outside the current block. On the other hand, when lossless coding is applied, as in the example shown in (b) of FIG. 43 , a difference value can be derived using neighboring pixels in the current block.

The decoder may derive an escape value for the current pixel by summing the difference value and the predicted value. For example, when a difference value is derived using a reference pixel outside the current block, an escape value may be derived by adding the difference value decoded from the bitstream and the upper reference pixel or the left reference pixel. On the other hand, when the difference value is encoded using the reference pixel in the current block, the escape value is derived by adding the difference value decoded from the bitstream and the value of the restored neighboring pixel (eg, the upper neighboring pixel or the right neighboring pixel). can do.

As another example, a prediction value for an escape value may be generated using a block vector.

After deriving the block vector of the current block, the reference block specified through the block vector may be specified. And, in the reference block, the value of the pixel at the same position as the current pixel to which the escape value is assigned may be set as the predicted value.

For example, in the example shown in FIG. 44 , a difference value may be derived by differentiating A5, which is a pixel at the same position as the pixel in the reference block, from the value A of the pixel to which the escape value is assigned.

When the difference value is derived, the quantized difference value may be encoded by encoding the difference value or quantizing the difference value.

Whether to generate a prediction value for an escape value using a block vector may be determined based on whether lossless coding is applied. As an example, only when lossless encoding is applied to the current image, a prediction value for an escape value may be generated using a block vector.

Alternatively, information specifying a method of generating a predicted value for an escape value may be signaled through a bitstream. As an example, the information may specify at least one of an intra prediction mode and a block vector use mode.

The intra prediction mode or block vector of the current block may be derived with reference to a neighboring block adjacent to the current block. Here, the neighboring block may include at least one of an upper neighboring block, a left neighboring block, an upper left neighboring block, an upper right neighboring block, or a lower left neighboring block of the current block.

45 is an example for describing a neighboring block of the current block.

In FIG. 45 , RT indicates a pixel located at the upper right of the block, and LB indicates a pixel located at the bottom left of the block.

Assuming that a block vector is derived with reference to five neighboring blocks around the current block, a block vector candidate list may be constructed by searching for neighboring blocks according to a predefined priority. For example, an upper neighboring block including a pixel at position A, a left neighboring block including a pixel at position L, an upper right neighboring block including a pixel at position e, a lower left neighboring block including a pixel at position i, and position a The neighboring blocks may be searched in the order of the upper left neighboring blocks including the pixel of .

When a block encoded using a block vector is searched for, a block vector candidate list may be constructed based on the block vector of the searched block. When a plurality of block vector candidates are included in the block vector candidate list, index information specifying one of the plurality of block vector candidates may be encoded and signaled.

Alternatively, when neighboring blocks are searched according to the search order, a block vector of a first found block (ie, a first block encoded as a block vector) may be set as a block vector of the current block.

The names of syntaxes used in the above-described embodiments are merely named for convenience of description.

Applying the decoding process or the embodiments described based on the encoding process to the encoding process or the decoding process is included in the scope of the present disclosure. It is also within the scope of the present disclosure to change the embodiments described in a certain order in an order different from that described.

Although the above-described embodiment has been described based on a series of steps or a flowchart, this does not limit the time-series order of the invention, and may be performed simultaneously or in a different order, if necessary. In addition, each of the components (eg, unit, module, etc.) constituting the block diagram in the above-described embodiment may be implemented as a hardware device or software, or a plurality of components may be combined to form one hardware device or software. may be implemented. The above-described embodiment may be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. Examples of the computer-readable recording medium include a hard disk, a magnetic medium such as a floppy disk and a magnetic tape, an optical recording medium such as a CD-ROM, a DVD, and a magneto-optical medium such as a floppy disk. media), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. The hardware device may be configured to operate as one or more software modules to perform processing according to the present disclosure, and vice versa.

The present disclosure may be used to encode/decode a video signal.

Claims

determining whether lossless encoding is applied to the current image;

decoding a residual coefficient for a current block in the current image; and

based on the residual coefficients, deriving a residual sample;

When decoding the residual coefficient, one of a first method using a maximum of m comparison flags and a second method using a maximum of n comparison flags is selected based on whether lossless encoding is applied to the current image,

The comparison flag indicates whether an absolute value of the residual coefficient exceeds a predetermined value.
According to claim 1,

An image decoding method, characterized in that it is determined whether at least one comparison flag is decoded with respect to the residual coefficient by comparing the number of decoded bins and a threshold value using context information.
3. The method of claim 2,

When the number of bins decoded using context information is equal to or greater than the threshold value, a syntax representing the absolute value of the residual coefficient as it is instead of the comparison flag is decoded.
3. The method of claim 2,

When at least one of the comparison flag or a parity flag indicating whether the absolute value of the residual coefficient is an even number is decoded, the number of bins decoded using the context information is increased.
According to claim 1,

When the first syntax is decoded and the first syntax indicates that the residual coefficient has a non-zero value,

and gt_1_flag indicating whether the absolute value of the residual coefficient has a value greater than 1 is further decoded.
6. The method of claim 5,

When the gt_1_flag indicates that the absolute value has a value greater than 1, a parity flag indicating whether the absolute value is an even number and a gt_2_flag indicating whether the absolute value is greater than 3 are further decoded. Decryption method.
3. The method of claim 2,

The image decoding method, characterized in that the threshold value is determined based on the size of the current block.
determining whether lossless encoding is applied to the current image;

deriving a residual coefficient based on the residual sample of the current block; and

encoding the residual coefficients of the current block;

When encoding the residual coefficient, one of a first method using a maximum of m comparison flags and a second method using a maximum of n comparison flags is selected based on whether lossless encoding is applied to the current image,

and the comparison flag indicates whether an absolute value of the residual coefficient exceeds a predetermined value.
9. The method of claim 8,

An image encoding method, characterized in that it is determined whether at least one comparison flag is encoded with respect to the residual coefficient by comparing the number of encoded bins and a threshold value using context information.
10. The method of claim 9,

When the number of bins encoded using context information is equal to or greater than the threshold value, a syntax indicating the absolute value of the residual coefficient as it is instead of the comparison flag is encoded.
10. The method of claim 9,

When at least one of the comparison flag or the parity flag indicating whether the absolute value of the residual coefficient is an even number is encoded, the number of bins encoded using the context information is increased.
10. The method of claim 9,

The threshold value is an image encoding method, characterized in that determined based on the size of the current block.
A computer-readable recording medium storing a bitstream encoded by a video encoding method, comprising:

The video encoding method comprises:

determining whether lossless encoding is applied to the current image;

deriving a residual coefficient based on the residual sample of the current block; and

encoding the residual coefficients of the current block;

When encoding the residual coefficient, one of a first method using a maximum of m comparison flags and a second method using a maximum of n comparison flags is selected based on whether lossless encoding is applied to the current image,

and the comparison flag indicates whether an absolute value of the residual coefficient exceeds a predetermined value.