CN116648905A

CN116648905A - Image Coding and Decoding Method Using Adaptive Reference Pixel Selection

Info

Publication number: CN116648905A
Application number: CN202180085397.7A
Authority: CN
Inventors: 全炳宇; 朴智允; 金范允; 朴胜煜
Original assignee: Sungkyunkwan University School Industry Cooperation; Hyundai Motor Co; Kia Corp
Current assignee: Sungkyunkwan University School Industry Cooperation; Hyundai Motor Co; Kia Corp
Priority date: 2020-11-23
Filing date: 2021-11-23
Publication date: 2023-08-25
Also published as: KR20220071129A

Abstract

According to the present embodiment, there is provided a picture decoding method for decoding a current block by using intra prediction, the picture decoding method including the steps of: decoding, from the bitstream, related information including an intra prediction mode of the current block; determining an execution direction of intra prediction in consideration of a prediction direction (hereinafter, second prediction direction) different from the first prediction direction according to the presence of a reference sample at a position to be referred to in the prediction direction (hereinafter, first prediction direction) indicated by the intra prediction mode; generating a prediction block of the current block by performing intra prediction according to an execution direction; reconstructing a residual block of the current block from the bitstream; and reconstructing the current block by adding the prediction block and the residual block.

Description

Image coding and decoding method using adaptive reference pixel selection

Technical Field

The present invention relates to video encoding and decoding with adaptive reference pixel selection.

Background

Since the amount of video data is larger than the amount of voice data or still image data, a large amount of hardware resources (including memory) are required to store or transmit video data without performing compression processing.

Accordingly, when storing or transmitting video data, an encoder is generally used to compress the video data for storage or transmission. Then, the decoder receives the compressed video data, and the decoder decompresses and reproduces the video data. Compression techniques for such video include h.264/AVC, high efficiency video coding (High Efficiency Video Coding, HEVC), and multi-function video coding (Versatile Video Coding, VCC), with VCC being improved by about 30% or more over HEVC coding efficiency.

We have found that as the size, resolution and frame rate of the images constituting a video are gradually increased, and thus, the amount of data to be encoded is also increasing, there is a need to develop a new compression technique having better encoding efficiency and greatly improving image quality. In particular, there is a need for a compression technique that can more efficiently encode images of complex textures, such as images containing edges (boundaries between objects) that vary in direction due to the presence of various objects.

Disclosure of Invention

Technical problem

The present invention relates to a method of selecting reference samples when decoding in an intra prediction mode, and its main aspect is to propose an encoding and decoding method for adaptively resetting reference samples according to a direction of a mode when an angular mode, which is one of the intra prediction modes, is applied.

Solution method

One aspect of the present invention provides a video decoding method for decoding a current block by using intra prediction. The method includes decoding, from a bitstream, related information including an intra prediction mode of a current block. The method further comprises the steps of: the direction of intra prediction is determined in consideration of a second prediction direction different from the first prediction direction according to the presence or absence of a reference sample at a position to be referred to by the first prediction direction indicated by the intra prediction mode. The method further comprises the steps of: performing intra prediction according to a direction of intra prediction to generate a prediction block of the current block, reconstructing a residual block of the current block from the bitstream, and adding the prediction block and the residual block to reconstruct the current block.

One aspect of the present invention provides a video encoding method for encoding a current block by using intra prediction. The method comprises the following steps: determining an intra prediction mode of the current block; the direction of intra prediction is determined in consideration of a second prediction direction different from the first prediction direction according to the presence or absence of a reference sample at a position to be referred to by the first prediction direction indicated by the intra prediction mode. The method further comprises the steps of: performing intra prediction according to the determined direction of intra prediction to generate a prediction block of the current block; generating a residual block by subtracting the prediction block from the current block; and encoding information related to the intra prediction mode and the residual block.

An aspect of the present invention provides a recording medium readable by a decoder, storing a bitstream generated by a video encoding method. The video encoding method includes: determining an intra prediction mode of the current block; and determining a direction of intra prediction taking into account a second prediction direction different from the first prediction direction according to the presence or absence of a reference sample at a position to be referred to by the first prediction direction indicated by the intra prediction mode. The video encoding method further includes: performing intra prediction according to a direction of the intra prediction to generate a prediction block of the current block; generating a residual block by subtracting the prediction block from the current block; and encoding information related to the intra prediction mode and the residual block.

Effects of the invention

The present invention can enhance subjective image quality and coding efficiency by performing intra prediction encoding and decoding in consideration of conditions such as a location of a block, presence or absence of a reference sample, and the like.

Subjective image quality and coding efficiency can be enhanced by adaptively applying a prediction direction to reduce a residual signal.

Intra prediction may be performed by using a prediction sample that minimizes a residual signal according to a prediction direction and the presence or absence of a reference sample to enhance subjective image quality and coding efficiency.

Drawings

Fig. 1 is an exemplary block diagram of a video encoding apparatus according to an embodiment of the present invention.

Fig. 2 is a schematic diagram for explaining a method of dividing a block by using the QTBTTT structure.

Fig. 3 is a schematic diagram illustrating a plurality of intra prediction modes.

Fig. 4 is a schematic diagram showing a method of generating a reference sample at the lowermost position.

Fig. 5 is a schematic diagram showing a method of generating a reference sample at a position other than the lowermost position.

Fig. 6 is a schematic diagram illustrating a plurality of intra prediction modes.

Fig. 7 (a) is a schematic diagram showing reference samples of the current block grouped into a top reference line and a left reference line, and fig. 7 (b) is a schematic diagram showing the use of reference sample groups according to intra prediction directions.

Fig. 8 is a schematic diagram showing a case where a part of the first reference sample group cannot be used for reference and the entire second reference sample group can be used for reference.

Fig. 9 is a schematic diagram showing a case where a part of the first reference sample group and a part of the second reference sample group cannot be used for reference.

Fig. 10 is a schematic diagram showing a case where a current block is divided into regions and a reference sample is selected for each region.

Fig. 11 is an exemplary block diagram of a video decoding apparatus according to an embodiment of the present invention.

Fig. 12 is a flowchart for describing a video encoding method according to an embodiment of the present invention.

Fig. 13 is a flowchart for describing a video decoding method according to an embodiment of the present invention.

Detailed Description

Hereinafter, some embodiments of the present invention will be described in detail with reference to the accompanying drawings. It should be noted that when reference numerals are assigned to constituent elements in the respective drawings, the same reference numerals denote the same elements although the elements are shown in different drawings. Furthermore, in the following description of the present invention, a detailed description of known functions and configurations incorporated herein is omitted to avoid obscuring the subject matter of the present invention.

Fig. 1 is an exemplary block diagram of a video encoding apparatus according to an embodiment of the present invention. Hereinafter, the video encoding device 100 and its elements will be described with reference to fig. 1.

The video encoding apparatus 100 may include: block divider 110, predictor 120, subtractor 130, transformer 140, reordering unit 150, entropy encoder 155, inverse quantizer 160, inverse transformer 165, adder 170, loop filtering unit 180, and memory 190.

Each element of the video encoding apparatus may be implemented in hardware or software, or a combination of hardware and software. The functions of the respective elements may be implemented as software, and the microprocessor may be implemented to perform the software functions corresponding to the respective elements.

A video includes one or more sequences comprising a plurality of images. Each image is divided into a plurality of regions, and encoding is performed on each region. For example, an image is segmented into one or more tiles (tiles) and/or slices (slices). One or more tiles may be defined as a tile set. Each tile or slice is partitioned into one or more Coding Tree Units (CTUs). Each CTU is partitioned into one or more Coding Units (CUs) by a tree structure. The information applied to each CU is encoded as a syntax of the CU, and commonly applied to the syntax of the CTU in which the information included in one CTU is encoded.

In addition, information commonly applied to all blocks in one slice is encoded as syntax of a slice header, and information applied to all blocks constituting one or more pictures is encoded in a picture parameter set (Picture Parameter Set, PPS) or a picture header. Furthermore, information commonly referred to by a sequence of multiple images is encoded in a sequence parameter set (Sequence Parameter Set, SPS).

Information commonly applied to one tile or group of tiles may be encoded as syntax of a tile header or a tile group header. The syntax included in the SPS, PPS, slice header, or tile header or tile group header may be referred to as a high level syntax.

Further, the bitstream may include one or more adaptive parameter sets (adaptation parameter set, APS) that include parameters referenced by an image or a group of pixels (e.g., a slice) smaller than the image. The image header or slice header includes an ID identifying the APS for the corresponding image or slice. Pictures referring to different PPS or slices referring to different picture heads may share the same parameters by the same APS ID.

Each of the plurality of images may be divided into a plurality of sub-images that can be independently encoded/decoded and/or independently displayed. When sub-image segmentation is applied, information about the layout of the sub-images within the image is signaled.

The block divider 110 determines the size of a Coding Tree Unit (CTU). Information about the size of the CTU (CTU size) is encoded as a syntax of the SPS or PPS and transmitted to the video decoding apparatus.

The block divider 110 divides each image constituting a video into a plurality of CTUs having a predetermined size, and then recursively divides the CTUs using a tree structure. In the tree structure, leaf nodes are used as Coding Units (CUs), which are the basic units of coding.

The tree structure may be a QuadTree (QT), a Binary Tree (BT), a trigeminal tree (TernaryTree, TT), or a structure formed by a combination of two or more QT structures, BT structures, and TT structures, the QuadTree (QT), i.e., a node (or parent node), being divided into four slave nodes (or child nodes) of the same size, the Binary Tree (BT), i.e., a node, being divided into two slave nodes, and the Trigeminal Tree (TT), i.e., a node, being divided into three slave nodes at a ratio of 1:2:1. For example, a quad-plus-binary tree (quad tree plus BinaryTree, QTBT) structure may be utilized, or a quad-plus-binary tree (QuadTree plus BinaryTree TernaryTree, QTBTTT) structure may be utilized. BTTT may be collectively referred to as multiple-type tree (MTT).

As shown in fig. 2, the CTU may be first partitioned into QT structures. QT segmentation may be repeated until the size of the segment blocks reaches the minimum block size MinQTSize of leaf nodes allowed in QT. A first flag (qt_split_flag) indicating whether each node of the QT structure is partitioned into four lower-layer nodes is encoded by the entropy encoder 155 and signaled to the video decoding apparatus. When the leaf node of QT is not greater than the maximum block size (MaxBTSize) of the root node allowed in BT, it may be further partitioned into one or more BT structures or TT structures. The BT structure and/or the TT structure may have a plurality of division directions. For example, there may be two directions, i.e., a direction of dividing the blocks of the nodes horizontally and a direction of dividing the blocks vertically. As shown in fig. 2, when MTT segmentation starts, a second flag (MTT _split_flag) indicating whether a node is segmented, a flag indicating a segmentation direction (vertical or horizontal) in the case of segmentation, and/or a flag indicating a segmentation type (binary or trigeminal) are encoded by the entropy encoder 155 and signaled to the video decoding apparatus. Alternatively, a CU partition flag (split_cu_flag) indicating whether a node is partitioned may be encoded before encoding a first flag (qt_split_flag) indicating whether each node is partitioned into 4 nodes of a lower layer. When the value of the CU partition flag (split_cu_flag) indicates that partition is not performed, a block of nodes becomes a leaf node in the partition tree structure and serves as an encoding unit (CU), which is a basic unit of encoding. When the value of the CU partition flag (split_cu_flag) indicates that partition is performed, the video encoding apparatus starts encoding from the first flag in the above-described manner.

When QTBT is used as another example of the tree structure, there may be two types of division, i.e., a type of dividing a block horizontally into two blocks of the same size (i.e., symmetric horizontal division) and a type of dividing a block vertically into two blocks of the same size (i.e., symmetric vertical division). A partition flag (split_flag) indicating whether each node of the BT structure is partitioned into lower-layer blocks and partition type information indicating a partition type are encoded by the entropy encoder 155 and transmitted to the video decoding apparatus. There may be additional types of partitioning a block of nodes into two asymmetric blocks. The asymmetric division type may include a type of dividing a block into two rectangular blocks at a size ratio of 1:3, or a type of dividing a block of a node diagonally.

A CU may have various sizes according to QTBT or QTBTTT partitions of the CTU. Hereinafter, a block corresponding to a CU to be encoded or decoded (i.e., a leaf node of QTBTTT) is referred to as a "current block". When QTBTTT segmentation is employed, the shape of the current block may be square or rectangular.

The predictor 120 predicts a current block to generate a predicted block. Predictor 120 includes an intra predictor 122 and an inter predictor 124.

The intra predictor 122 constructs reference samples from reconstructed samples located around the current block in the current image including the current block, and predicts samples in the current block using the reference pixels.

The intra predictor 122 checks whether or not there are usable samples for each position to be referred to, to obtain a reference sample to be used for prediction. If it is determined that there are no usable samples at the position to be referred to, the intra predictor 122 generates a reference sample by inserting sample values at the position according to a predetermined method agreed between the video encoding apparatus and the video decoding apparatus and applying a filter.

Here, a reference sample refufilt [ x ] [ y ] before the filter is applied may be generated as in table 1. Here, refIdx represents an index of a reference line, and refW and refH represent a width and a height of a reference area, respectively.

TABLE 1

Fig. 3 is a schematic diagram showing a search sequence of reference samples.

As shown in process 1) of table 1 and fig. 3, the intra predictor 122 performs a search in a clockwise direction from a left lowermost reference sample of the current block to a top rightmost reference sample of the current block to determine whether the reference sample for prediction can be used.

If a reference sample is not available for each reference sample position, the intra predictor 122 rounds off half of the maximum allowed sample value and inserts it as a reference sample value into the unavailable position. That is, for example, the reference sample value 128 is inserted for a depth of 8 bits, and the reference sample value 512 is inserted for a depth of 10 bits.

Fig. 4 is a schematic diagram showing a method of generating a reference sample at the lowermost position. Fig. 5 is a schematic diagram showing a method of generating a reference sample at a position other than the lowermost position.

As shown in fig. 4 and 5, if reference samples are not available at some reference sample positions, the intra predictor 122 generates reference samples for each of the unavailable positions in the methods (a) to (c) in table 1.

As shown in fig. 4, if the reference sample is not available at the left lowermost position, the intra predictor 122 copies the available reference sample first encountered in the search sequence of fig. 3 and inserts it as a reference sample value at the left lowermost position.

If the reference sample at a position other than the left lowermost position is not available, the intra predictor 122 copies the previously available reference sample value in the search direction and inserts it into the unavailable position as the reference sample value, as shown in fig. 5.

After generating the reference samples at the top and left of the current block in this manner, the intra predictor 122 may generate final reference samples p [ x ] [ y ] by applying a filter.

The method of generating the final reference sample p [ x ] [ y ] is shown in Table 2.

TABLE 2

As shown in fig. 6, there are a plurality of intra prediction modes according to the prediction direction. For example, the plurality of intra prediction modes may include two non-directional modes including a Planar (Planar) mode and a Direct Current (DC) mode, and 65 directional modes. The reference samples and equations to be utilized are defined differently for each prediction mode.

The intra predictor 122 generates reference samples and performs intra prediction on intra prediction using one of the intra prediction modes in the above-described manner.

Since prediction is performed on a per block basis, a certain sample corresponding to a reference sample of a current block to be predicted may not have been reconstructed. Furthermore, for the current block on the CTU boundary or on the image boundary, there may be no reference samples for various reasons including lack of reference samples. In this case, for the reference sample that does not exist, the value of the adjacent reference sample may be used as the value of the reference sample that does not exist.

The intra predictor 122 determines an intra prediction mode to be used when encoding the current block. Intra predictor 122 may encode the current block using several intra prediction modes and select an appropriate intra prediction mode to be used by the current block from among the several modes tested. For example, the intra predictor 122 may calculate a rate distortion value using rate-distortion (rate-distortion) analysis of several tested intra prediction modes, and may select an intra prediction mode having the best rate distortion characteristics among the tested modes.

The intra predictor 122 selects one intra prediction mode from among a plurality of intra prediction modes, and predicts the current block using a reference sample and an equation determined according to the selected intra prediction mode. Information about the selected intra prediction mode is encoded by the entropy encoder 155 and transmitted to a video decoding device.

In fig. 7, (a) is a schematic diagram showing reference samples of the current block grouped into a top reference line and a left reference line, and (b) is a schematic diagram showing the use of reference sample groups according to intra prediction directions.

An embodiment of the present invention discloses a technique in which, when samples at a position to be referred to are divided into two or more groups and intra prediction is performed by using one group or two or more groups, as shown in fig. 7, if intra prediction using a specific reference sample group is not possible, intra prediction is performed using one or more reference samples of another reference sample group.

The intra predictor 122 determines a direction of intra prediction in consideration of a prediction direction (hereinafter, "second prediction direction") different from the first prediction direction according to the presence of a reference sample that can be used at a sample position to be referred to by a specific prediction direction (hereinafter, "first prediction direction") indicated by the intra prediction mode.

Here, a description will be given based on the following assumption: among a reference sample group (hereinafter, "first reference sample group") at the top of the current block and a reference sample group (hereinafter, "second reference sample group") at the left side of the current block, the first reference sample group is referenced for the first prediction direction. However, the description also applies when the second reference sample set is referenced for the first prediction direction, and when both the first reference sample set and the second reference sample set are referenced for the first prediction direction.

Further, the second prediction direction may be the opposite direction of the first prediction direction, but is not necessarily limited thereto. The second prediction direction may be preset or calculated in a preset manner for each first direction corresponding to each intra prediction mode.

If there is a reference sample at each position where the first prediction direction is to be referred to, the second prediction direction of the corresponding intra prediction mode is not considered. If there is no usable sample at least one of the positions where the first prediction direction is to be referred to, the second prediction direction of the corresponding intra prediction mode is considered, and one of the first prediction direction and the second prediction direction is determined as the direction of intra prediction.

When the second set of reference samples corresponds to any one of the following four cases, a second prediction direction is selected:

(1) Groups with samples present at all positions to be referenced

(2) Group with the largest number of reference samples

(3) Group with highest proportion of reference samples

(4) Groups not including image boundaries or slice boundaries

If there are no reference samples at least some positions to be referred to in the first prediction direction, intra prediction may be performed on the current block in the second direction by using the second reference sample group.

Further, if the number of reference samples that can be used at the position to be referred to in the first prediction direction is greater than the number of reference samples that can be used at the position to be referred to in the second prediction direction, intra prediction on the current block may be performed in the second direction by using the second reference sample group.

Further, if the proportion of reference samples that can be used for a position to be referred to in the first prediction direction in the first reference sample group is higher than the proportion of reference samples that can be used for a position to be referred to in the second prediction direction in the second reference sample group, intra prediction is performed on the current block in the second prediction direction by using the second reference sample group.

Fig. 8 shows a case where a part of the first reference sample group cannot be used for reference and the entire second reference sample group can be used for reference.

In fig. 8, if the first prediction direction corresponds to the number 66 mode and the second prediction direction corresponds to the number 2 mode, about half samples of the first reference sample group cannot be used for reference of intra prediction of the first prediction direction of the current block, whereas all samples of the second reference sample group can be used for reference of intra prediction of the second prediction direction of the current block. Accordingly, in this case, the intra predictor 122 performs intra prediction of the current block by determining the second prediction direction as a direction of intra prediction of the current block.

Fig. 9 shows a case where some samples of the first reference sample group cannot be used for reference and the same samples of the second reference sample group cannot be used for reference.

As shown in fig. 9, if the first prediction direction corresponds to the number 66 mode and the second prediction direction corresponds to the number 2 mode, the proportion of reference samples of the second reference sample group that can be used for reference for intra prediction in the second prediction direction is higher than the proportion of reference samples of the first reference sample group that can be used for reference for intra prediction in the first prediction direction. Accordingly, intra prediction is performed on the current block in the second prediction direction. For reference, the proportion of reference samples in the first reference sample group that can be used for reference for intra prediction in the first prediction direction is 1/2, and the proportion of reference samples in the second reference sample group that can be used for reference for intra prediction in the second prediction direction is about 14/18.

Further, since the number of reference samples of the first reference sample group that can be used for the reference for intra prediction in the second prediction direction is greater than the number of reference samples of the first reference sample group that can be used for the reference for intra prediction in the first prediction direction, intra prediction is performed on the current block in the second prediction direction. For reference, the number of reference samples in the first reference sample group that can be used for reference for intra prediction in the first prediction direction is 9, and the number of reference samples in the second reference sample group that can be used for reference for intra prediction in the second prediction direction is 14.

In fig. 9, the values of the reference samples of the second reference sample group that cannot be used for the reference of the intra prediction in the second prediction direction are determined by using the values of the other reference samples, as explained with reference to fig. 4 and 5.

If the current intra prediction mode is the number 66 mode, intra prediction is performed using only the first reference sample group at the top. However, if the current block is located at the top of the CTU, the first reference sample group cannot be used. Thus, if a second set of reference samples is available, prediction is performed in a second prediction direction.

In addition, if the current block is located at a CTU boundary, an image boundary, or a slice boundary, the intra predictor 122 determines a direction of intra prediction in consideration of predictions in the first prediction direction and the second prediction direction.

Fig. 10 shows a case where a current block is divided into regions and a reference sample is selected for each region.

As shown in fig. 10, if a portion of the first reference sample group cannot be used for reference of intra prediction in the first prediction direction, the current block is divided into a region corresponding to a sample that can be used in the first reference sample group (hereinafter, "first region") and a region corresponding to a sample that cannot be used in the first reference sample group (hereinafter, "second region"). Also, the second region is subdivided into a third region having samples of the second reference sample group that can be used for intra-prediction in the second prediction direction and a fourth region having no samples of the second reference sample group that can be used for intra-prediction in the second prediction direction.

In this case, intra prediction of the first region and the fourth region is performed in the first prediction direction, and intra prediction of the third region is performed in the second prediction direction. On the other hand, before intra prediction of the fourth region is performed in the first prediction direction, values of reference samples corresponding to intra prediction of the fourth region in the first prediction direction in the first reference sample group are determined using values of other reference samples.

Boundaries of the plurality of regions are formed parallel to the first prediction direction.

A specific example will be given in connection with fig. 10. If the intra prediction mode of the current block is number 66, the current block may be partitioned into several regions in the number 66 prediction mode due to the unavailability of the reference samples when a portion of the first reference sample set and a portion of the second reference sample set are not present.

In fig. 10, reference samples that can be used for prediction, which are related to pixels existing in the region "a", do not exist in the first reference sample group, but reference samples that can be used for prediction exist in the second reference sample group. Accordingly, the intra predictor 122 performs prediction of pixels existing in the region "a" in the second prediction direction by using the second reference sample group.

The reference samples that can be used for prediction, which are associated with the pixels present in the region "b", are not present in the respective positions of both the first reference sample group and the second reference sample group. Accordingly, the intra predictor 122 inserts reference sample values determined using neighboring reference samples into corresponding positions in the first reference sample group, as shown in fig. 4 and 5, and performs prediction on pixels existing in the region "b" in the first prediction direction.

With respect to pixels present in the region "c", reference may be made from the first reference sample group in the first prediction direction. Accordingly, the intra predictor 122 performs prediction of pixels existing in the region "c" in the first prediction direction by using the reference sample values of the first reference sample group.

The reference samples that can be used for prediction, which are related to the pixels existing in the region "d", do not exist in the first reference sample group, but the reference samples that can be used for prediction exist in the second reference sample group. Accordingly, the intra predictor 122 performs prediction in the second prediction direction by using the second reference sample group.

The intra predictor 122 may generate an ARS flag indicating whether or not the second prediction direction is considered based on the presence of the reference samples that can be used for the first prediction direction indicated by the intra prediction mode as intra prediction related information, and transmit the ARS flag to the video decoding apparatus.

For example, if the ARS flag is 0, intra prediction is performed in the first prediction direction indicated by the intra prediction mode without considering the second prediction direction. If the ARS flag is 1, intra prediction is performed on the current block in the first prediction direction or the second prediction direction, taking the second prediction direction into consideration.

If the ARS flag is 1, additional information may be included in the intra prediction related information when intra prediction is performed on the current block in consideration of the second prediction direction. In other words, the additional information indicates whether to perform prediction in the first prediction direction or the second prediction direction by dividing the current block into a plurality of regions according to the availability of the first reference sample group and the second reference sample group.

The inter predictor 124 generates a prediction block of the current block through motion compensation. Since details about the operation of the inter predictor 124 are beyond the scope of the present embodiment, a detailed description thereof will be omitted.

The subtractor 130 subtracts the prediction block generated by the intra predictor 122 or the inter predictor 124 from the current block to generate a residual block.

The transformer 140 may transform the residual signal in the residual block. A residual block of a two-dimensional size may be used as a transform unit (hereinafter, a "TU (transform unit)"), i.e., a block size in which a transform is performed. Alternatively, the residual block may be partitioned into a plurality of sub-blocks, and each sub-block may be used as a TU to transform the residual signal within the corresponding sub-block.

The transformer 140 may divide the residual block into one or more sub-blocks and apply the transform to the one or more sub-blocks. Accordingly, the residual values of the transform block may be transformed from the pixel domain to the frequency domain. In the frequency domain, a transform block is referred to as a coefficient block or transform block that contains one or more transform coefficient values. Two-dimensional transform kernels may be used for the transform, while one-dimensional transform kernels may be used for the horizontal transform and the vertical transform, respectively. The transform kernel may be based on a discrete cosine transform (discrete cosine transform, DCT), a discrete sine transform (discrete sine transform, DST), or the like.

The transformer 140 may separately transform the residual block in the horizontal direction and the vertical direction. For the transformation, various types of transformation functions or transformation matrices may be utilized. For example, a pair of transformation functions for transforming in the horizontal and vertical directions may be defined as a transformation set (multiple transform set, MTS). The transformer 140 may select one transform function pair having the best transform efficiency in the MTS and transform the residual block in the horizontal direction and the vertical direction, respectively. Information (mts_idx) about the transform function pair selected from the MTS is encoded by the entropy encoder 155 and signaled to the video decoding means.

The quantizer 145 quantizes the transform coefficient output from the transformer 140 using a quantization parameter, and outputs the quantized transform coefficient to the entropy encoder 155. For some blocks or frames, the quantizer 145 may quantize the relevant residual block directly without transformation. The quantizer 145 may apply different quantization coefficients (scaling values) according to the positions of the transform coefficients in the transform block. A quantization coefficient matrix applied to quantized transform coefficients arranged in two dimensions may be encoded and signaled to a video decoding apparatus.

The reordering unit 150 may reorder coefficient values of the quantized residual values. The rearrangement unit 150 may change the 2-dimensional coefficient array to a 1-dimensional coefficient sequence by coefficient scanning (coefficient scanning). For example, the rearrangement unit 150 may scan the coefficients from the DC coefficients to the coefficients in the high frequency region using a zig-zag scan (zig-zag scan) or a diagonal scan (diagonal scan) to output a 1-dimensional coefficient sequence. Depending on the size of the transform unit and the intra prediction mode, a vertical scan, i.e. scanning the coefficients of the two-dimensional array in the column direction, or a horizontal scan, i.e. scanning the coefficients of the two-dimensional block shape in the row direction, may be used instead of a zig-zag scan. In other words, the scan pattern to be utilized may be determined in zigzag scan, diagonal scan, vertical scan, and horizontal scan according to the size of the transform unit and the intra prediction mode.

The entropy encoder 155 encodes the one-dimensional quantized transform coefficients output from the rearrangement unit 150 using various encoding techniques such as Context-based adaptive binary arithmetic coding (Context-based Adaptive Binary Arithmetic Code, CABAC) and exponential golomb (exponential Golomb) to generate a bitstream.

The entropy encoder 155 encodes information related to block division (e.g., CTU size, CU division flag, QT division flag, MTT division type, and MTT division direction) so that a video decoding apparatus can divide blocks in the same manner as a video encoding apparatus. In addition, the entropy encoder 155 encodes information on a prediction type indicating whether the current block is encoded by intra prediction or inter prediction, and encodes intra prediction information (i.e., information on an intra prediction mode) or inter prediction information (i.e., information on a reference picture index and a motion vector) according to the prediction type. Further, the entropy encoder 155 encodes information related to quantization (i.e., information about quantization parameters and information about quantization matrices).

The inverse quantizer 160 inversely quantizes the quantized transform coefficient output from the quantizer 145 to generate a transform coefficient. The inverse transformer 165 transforms the transform coefficients output from the inverse quantizer 160 from the frequency domain to the spatial domain, and reconstructs a residual block.

The adder 170 adds the reconstructed residual block and the prediction block generated by the predictor 120 to reconstruct the current block. The pixels in the reconstructed current block are used as reference pixels in performing intra prediction of the subsequent block.

The loop filtering unit 180 filters the reconstructed pixels to reduce block artifacts (blocking artifacts), ringing artifacts (ringing artifacts), and blurring artifacts (blurring artifacts) due to block-based prediction and transform/quantization. Loop filtering unit 180 may include one or more of deblocking filter 182, sample adaptive offset (sample adaptive offset, SAO) filter 184, or adaptive loop filter (adaptive loop filter, ALF) 186.

Deblocking filter 182 filters boundaries between reconstructed blocks to remove block artifacts caused by block-wise encoding/decoding, and SAO filter 184 performs additional filtering on the deblock filtered video. The SAO filter 184 is a filter for compensating for differences between reconstructed pixels and original pixels caused by lossy coding (lossy coding), and performing filtering in such a manner that the corresponding offsets and each reconstructed pixel are added. The ALF 186 performs filtering on a target pixel to be filtered by applying filter coefficients to the target pixel and adjacent pixels of the target pixel. The ALF 186 may divide pixels included in an image into predetermined groups and then determine one filter to be applied to the corresponding group to differentially perform filtering on each group. Information about filter coefficients to be used for ALF may be encoded and signaled to the video decoding apparatus.

The reconstructed block filtered by the loop filtering unit 180 is stored in the memory 190. Once all blocks in one image are reconstructed, the reconstructed image can be used as a reference image for inter prediction of blocks in a subsequent image to be encoded.

Fig. 11 is a block diagram of a video decoding apparatus according to an aspect of the present invention. Hereinafter, a video decoding apparatus and elements of the apparatus are described with reference to fig. 11.

The video decoding apparatus (1100) may include: entropy decoder 1110, reordering unit 1115, inverse quantizer 1120, inverse transformer 1130, predictor 1140, adder 1150, loop filtering unit 1160, and memory 1170.

Each element of the video decoding apparatus may be implemented in hardware, software, or a combination of hardware and software. Further, the function of each element may be implemented as software, and the microprocessor may be implemented to execute the software function corresponding to each element.

The entropy decoder 1110 determines a current block to be decoded by decoding a bitstream generated by a video encoding apparatus and extracting information related to block segmentation. The entropy decoder 1110 also extracts prediction information required to reconstruct the current block, information about a residual signal, and the like.

The entropy decoder 1110 extracts information about the size of CTUs from a Sequence Parameter Set (SPS) or a Picture Parameter Set (PPS), determines the size of CTUs, and partitions a picture into CTUs of the determined size. Then, the decoder determines the CTU as the highest layer (i.e., root node) of the tree structure, and extracts partition information about the CTU to partition the CTU using the tree structure.

For example, when dividing the CTU using the QTBTTT structure, a first flag (qt_split_flag) related to the division of QT is extracted to divide each node into four nodes of a sub-layer. For a node corresponding to a leaf node of QT, a second flag (mtt_split_flag) related to the division of MTT and information on a division direction (vertical/horizontal) and/or a division type (binary/trigeminal) are extracted to divide the corresponding leaf node in an MTT structure. Thereby, each node below the leaf node of QT is recursively partitioned in the BT or TT structure.

As another example, when the CTU is divided using the QTBTTT structure, a CU division flag (split_cu_flag) indicating whether to divide the CU may be extracted. When the corresponding block is divided, a first flag (qt_split_flag) may be extracted. In a partitioning operation, zero or more recursive MTT partitions may occur per node after zero or more recursive QT partitions. For example, CTUs may undergo MTT segmentation directly without QT segmentation, or only QT segmentation multiple times.

As another example, when a CTU is segmented using a QTBT structure, a first flag (qt_split_flag) related to QT segmentation is extracted, and each node is segmented into four nodes of the lower layer. Then, a split flag (split_flag) indicating whether to further split a node corresponding to a leaf node of QT with BT and split direction information are extracted.

Once the current block to be decoded is determined through tree structure segmentation, the entropy decoder 1110 extracts information on a prediction type indicating whether the current block is intra-predicted or inter-predicted. When the prediction type information indicates intra prediction, the entropy decoder 1110 extracts syntax elements of intra prediction information (intra prediction mode) of the current block. When the prediction type information indicates inter prediction, the entropy decoder 1110 extracts syntax elements for the inter prediction information, i.e., information indicating a motion vector and a reference picture referred to by the motion vector.

The entropy decoder 1110 also extracts information on the transform coefficient of the quantized current block as quantization-related information and information on the residual signal.

The reordering unit 1115 may change the sequence of one-dimensional quantized transform coefficients entropy decoded by the entropy decoder 1110 into a 2-dimensional coefficient array (i.e., block) in the reverse order of coefficient scanning performed by the video encoding device.

The inverse quantizer 1120 inversely quantizes the quantized transform coefficients using quantization parameters. The inverse quantizer 1120 may apply different quantization coefficients (scaling values) to the quantized transform coefficients arranged in two dimensions. The inverse quantizer 1120 may perform inverse quantization by applying a matrix of quantized coefficients (scaled values) from the video encoding device to a two-dimensional array of quantized varying coefficients.

The inverse transformer 1130 inversely transforms the inversely quantized transform coefficients from the frequency domain to the spatial domain to reconstruct the residual signal. Thereby generating a reconstructed residual block for the current block. In addition, when applying MTS, the inverse transformer 1130 determines transform functions or transform matrices to be applied in horizontal and vertical directions, respectively, using MTS information (mts_idx) signaled from the video encoding apparatus. The inverse transformer 1130 also inversely transforms transform coefficients in the transform block in the horizontal direction and the vertical direction using the determined transform function.

Predictor 1140 may include an intra predictor 1142 and an inter predictor 1144.

The intra predictor 1142 is activated when the prediction type of the current block is intra prediction, and the inter predictor 1144 is activated when the prediction type of the current block is inter prediction.

The intra predictor 1142 determines an intra prediction mode of the current block among a plurality of intra prediction modes based on syntax elements of the intra prediction mode extracted from the entropy decoder 1110, and predicts the current block using reference pixels around the current block according to the intra prediction mode.

The intra predictor 1142 checks the presence of usable samples for each position to be referred to obtain a reference sample for prediction. If it is determined that there are no usable samples at the position to be referred to, the intra predictor 1142 generates a reference sample by inserting a sample value at the position according to a predetermined method agreed between the video encoding apparatus and the video decoding apparatus and applying a filter.

Here, the reference sample refufnfilt [ x ] [ y ] before the filter is applied may be generated as in the procedure of table 1. Here, refIdx represents an index of a reference line, and refW and refH represent a width and a height of a reference area, respectively.

As shown in step 1) of table 1 and fig. 3, the intra predictor 1142 performs a search in a clockwise direction from a left lowermost reference sample of the current block to a top rightmost reference sample of the current block to determine whether a reference sample for prediction is available.

If a reference sample is not available for each reference sample position, the intra predictor 1142 rounds off half of the maximum allowed sample value and inserts it as a reference sample value into the unavailable position. That is, for example, the intra predictor 1142 inserts the reference sample value 128 for a depth of 8 bits and inserts the reference sample value 512 for a depth of 10 bits.

As shown in fig. 4 and 5, if reference samples are not available at some reference sample positions, the intra predictor 1142 generates reference samples for each of the unavailable positions in the methods (a) to (c) in table 1.

As shown in fig. 4, if the reference sample is not available at the left lowermost position, the intra predictor 1142 copies the available reference sample first encountered in the search sequence of fig. 3 and inserts it as a reference sample value at the left lowermost position.

If the reference sample of the position other than the left lowermost position is not usable, the intra predictor 1142 copies the previously usable reference sample value in the search direction and inserts it into the unusable position as the reference sample value, as shown in fig. 5.

After generating the reference samples at the top and left of the current block in this way, the intra predictor 1142 may generate final reference samples p [ x ] [ y ] by applying a filter.

The intra predictor 1142 generates reference samples for intra prediction using one of the intra prediction modes in the above-described manner, and performs intra prediction.

The intra predictor 1142 determines or selects one of a plurality of intra prediction modes based on intra prediction related information decoded from the bitstream, and predicts the current block by using a reference sample and equation determined according to the selected intra prediction mode.

As shown in fig. 7, the intra predictor 1142 divides samples at positions to be referred to into two or more groups and performs intra prediction by using one group or two or more groups. In this case, if intra prediction using a specific reference sample group is impossible, the intra predictor 1142 performs intra prediction using one or more reference samples of another reference sample group.

The intra predictor 1142 determines a direction of intra prediction in consideration of a prediction direction (hereinafter, "second prediction direction") different from the first prediction direction according to the presence of a reference sample that can be used at a sample position to be referred to in a specific prediction direction (hereinafter, "first prediction direction") indicated by the intra prediction mode.

If there is a reference sample at each position where the first prediction direction is to be referred to, the second prediction direction of the corresponding intra prediction mode is not considered. If at least one of the positions to be referred to in the first prediction direction has no usable samples, the second prediction direction of the corresponding intra prediction mode is considered, and one of the first prediction direction and the second prediction direction is determined as the direction of intra prediction.

(1) Groups with samples present at all positions to be referenced

(2) Group with the largest number of reference samples

(3) Group with highest proportion of reference samples

(4) Groups not including image or slice boundaries

If there are no reference samples at least some positions to be referred to in the first prediction direction, intra prediction of the current block may be performed in the second direction by using the second reference sample group.

In fig. 8, if the first prediction direction corresponds to the number 66 mode and the second prediction direction corresponds to the number 2 mode, about half samples of the first reference sample group cannot be used for reference of intra prediction of the first prediction direction of the current block, whereas all samples of the second reference sample group can be used for reference of intra prediction of the second prediction direction of the current block. Accordingly, in this case, the intra predictor 1142 performs intra prediction of the current block by determining the second prediction direction as a direction of intra prediction of the current block.

As shown in fig. 9, if the first prediction direction corresponds to the number 66 mode and the second prediction direction corresponds to the number 2 mode, the proportion of reference samples of the second reference sample group that can be used for reference for intra prediction in the second prediction direction is higher than the proportion of reference samples of the first reference sample group that can be used for reference for intra prediction in the first prediction direction. Accordingly, the intra predictor 1142 performs intra prediction on the current block in the second prediction direction. For reference, the proportion of reference samples in the first reference sample group that can be used for reference for intra prediction in the first prediction direction is 1/2, and the proportion of reference samples in the second reference sample group that can be used for reference for intra prediction in the second prediction direction is about 14/18.

Further, since the number of reference samples of the first reference sample group that can be used for the reference for intra prediction in the second prediction direction is greater than the number of reference samples of the first reference sample group that can be used for the reference for intra prediction in the first prediction direction, the intra predictor 1142 performs intra prediction on the current block in the second prediction direction. For reference, the number of reference samples in the first reference sample group that can be used for reference for intra prediction in the first prediction direction is 9, and the number of reference samples in the second reference sample group that can be used for reference for intra prediction in the second prediction direction is 14.

If the current intra prediction mode is the number 66 mode, intra prediction is performed using only the first reference sample group at the top. However, if the current block is located at the top of the CTU, the first reference sample group cannot be used. Thus, if the second set of reference samples is available, prediction is performed in the second prediction direction.

In addition, if the current block is located at a CTU boundary, an image boundary, or a slice boundary, the intra predictor 1142 determines a direction for intra prediction in consideration of predictions in the first prediction direction and the second prediction direction. In other words, if the current block is located on a CTU boundary, a picture boundary, or a slice boundary, intra prediction may not be performed on the current block by using a reference sample group of a neighboring CTU, a neighboring picture, or a neighboring slice.

In this case, the intra predictor 1142 performs intra prediction of the first region and the fourth region in the first prediction direction, and performs intra prediction of the third region in the second prediction direction. On the other hand, before intra prediction of the fourth region is performed in the first prediction direction, values of reference samples corresponding to intra prediction of the fourth region in the first prediction direction in the first reference sample group are determined using values of other reference samples.

In fig. 10, reference samples that can be used for prediction, which are related to pixels existing in the region "a", do not exist in the first reference sample group, but reference samples that can be used for prediction exist in the second reference sample group. Accordingly, the intra predictor 1142 performs prediction of pixels existing in the region "a" in the second prediction direction by using the second reference sample group.

The reference samples that can be used for prediction, which are associated with the pixels present in the region b, are not present in the respective positions of both the first reference sample group and the second reference sample group. Accordingly, the intra predictor 1142 inserts reference sample values determined using neighboring reference samples into corresponding positions in the first reference sample group, as shown in fig. 4 and 5, and performs prediction on pixels existing in the region "b" in the first prediction direction.

With respect to pixels present in the region "c", reference may be made from the first reference sample group in the first prediction direction. Accordingly, the intra predictor 1142 performs prediction of pixels existing in the region "c" in the first prediction direction by using the reference sample values of the first reference sample group.

The reference samples that can be used for prediction, which are related to the pixels existing in the region "d", do not exist in the first reference sample group, but the reference samples that can be used for prediction exist in the second reference sample group. Thus, the intra predictor 1142 performs prediction in the second prediction direction by using the second reference sample group.

The intra predictor 1142 may extract an ARS flag indicating whether or not the second prediction direction is considered based on the presence of reference samples that can be used for the first prediction direction indicated by the intra prediction mode, according to intra prediction related information decoded from the bitstream.

For example, if the ARS flag is 0, the intra predictor 1142 performs intra prediction in the first prediction direction indicated by the intra prediction mode without considering the second prediction direction. If the ARS flag is 1, the intra predictor 1142 performs intra prediction on the current block in the first prediction direction or the second prediction direction, taking the second prediction direction into consideration.

The inter predictor 1144 determines a motion vector of the current block and a reference picture to be referred to by the motion vector based on syntax elements of the inter prediction mode extracted from the entropy decoder 1110, and predicts the current block based on the motion vector and the reference picture.

The inter predictor 1144 may perform interpolation filtering according to a value of a motion vector of the current block. That is, if the fractional part of the motion vector is not zero, the inter predictor 1144 generates a sub-sample indicated by the fractional part through interpolation. If the fractional part of the horizontal component (x component) of the motion vector is not zero, interpolation is performed horizontally, and if the fractional part of the vertical component (y component) of the motion vector is not zero, interpolation is performed vertically.

The adder 1150 reconstructs the current block by adding the residual block output from the inverse transformer 1130 to the prediction block output from the inter predictor 1144 or the intra predictor 1142. Samples in the reconstructed current block are used as reference samples when intra prediction is performed on a block to be decoded later.

Loop filtering unit 1160 may include at least one of deblocking filter 1162, SAO filter 1164, and ALF 1166.

Deblocking filter 1162 performs deblocking filtering of boundaries between reconstructed blocks to remove block artifacts caused by block-by-block decoding.

The SAO filter 1164 performs filtering after deblocking filtering in such a way that the corresponding offset is added to the reconstructed block to compensate for the difference between the reconstructed samples and the original samples caused by lossy encoding.

ALF 1166 performs filtering on target samples to be filtered by applying filter coefficients to the target samples and adjacent samples to the target samples. The ALF 1166 may divide samples included in the image into predetermined groups and then determine one filter to be applied to the corresponding group to differentially perform filtering on each group. The filter coefficients of ALF 1166 are determined by using information about the filter coefficients encoded from the bitstream.

The reconstructed block filtered by the loop filter unit 1160 is stored in a memory 1170. Once all blocks in one image are reconstructed, the reconstructed image can be used as a reference image for inter prediction of blocks in a subsequent image to be encoded.

Fig. 12 is an exemplary flowchart of a video encoding method according to an embodiment of the present invention.

The intra predictor 122 of the video encoding device 100 determines an intra prediction mode of the current block (S1210).

The intra predictor 122 determines a direction of intra prediction considering a prediction direction (hereinafter, "second prediction direction") different from the first prediction direction according to the presence or absence of a reference sample at each position to be referred to by the prediction direction indicated by the intra prediction mode (hereinafter, "first prediction direction") (S1220).

The intra predictor 122 generates a prediction block of the current block by performing intra prediction according to a direction of intra prediction (S1230).

The subtractor 130 generates a residual block by subtracting the prediction block from the current block (S1240).

The transformer 140 transforms the residual block, and the quantizer 145 quantizes the transformed residual block (S1250).

The reordering unit 150 reorders coefficient values of the quantized residual values, and the entropy encoder 155 performs entropy encoding on the quantized residual values and intra prediction related information including the selected intra prediction mode (S1260).

In the description of the video encoding apparatus 100, steps S1210 to S1250 have been fully explained, and thus a further detailed description will be omitted.

Fig. 13 is an exemplary flowchart of a video decoding method according to an embodiment of the present invention.

The entropy decoder 1110 of the video decoding apparatus 1100 decodes intra prediction related information including an intra prediction mode of the current block from the bitstream (S1310).

The entropy decoder 1110 decodes transform information from the bitstream, determines a rectangular or square transform unit based on the transform information, and decodes transform coefficients corresponding to the transform unit to generate a transform coefficient block (S1320).

The inverse quantizer 1120 inversely quantizes the transform coefficient block, and the inverse transformer 1130 inversely transforms the inversely quantized transform coefficient block to generate a residual block (S1330).

The intra predictor 1142 determines a direction of intra prediction considering a second prediction direction different from the first prediction direction according to the presence of a reference sample at a position to be referred to by the first prediction direction indicated by the intra prediction mode, and generates a prediction block of the current block by performing intra prediction according to the determined direction of intra prediction (S1340).

The adder 1150 adds the prediction block and the residual block (S1350) to reconstruct the current block.

In the description of the video decoding apparatus 1100, steps S1310 to S1350 have been fully explained, and thus further detailed description will be omitted.

In another aspect, various functions or methods of the present invention may be implemented as instructions stored on a non-volatile recording medium, which may be read and executed by one or more processors. Non-volatile recording media include, for example, all types of recording devices in which data is stored in a form readable by a computer system. For example, nonvolatile recording media include storage media such as erasable programmable read-only memory (EPROM), flash drives, optical drives, magnetic hard drives, and Solid State Drives (SSD).

In the foregoing description, it should be understood that the exemplary embodiments may be implemented in various other ways. The functions or methods described in one or more examples may be implemented as hardware, software, firmware, or any combination thereof. It should be appreciated that the functional components described in this disclosure have been labeled by adding a "unit" to the component name to explicitly emphasize their implementation independence.

The above description is merely illustrative of the technical idea of the present application, and various modifications and changes can be made by those skilled in the art to which the present application pertains without departing from the essential characteristics of the present application. Accordingly, the embodiments disclosed in the present application are not intended to limit the technical concept of the present application, but rather describe the present application, and the scope of the technical concept of the present application is not limited thereto. The scope of the present application should be construed by the appended claims, and all technical ideas within the equivalent scope should be construed to be included in the scope of the present application.

Cross Reference to Related Applications

The present application claims priority from korean patent application No.10-2020-0157795, filed 11/23 in 2020, and korean patent application No.10-2021-0161975, filed 11/23 in 2021, the entire contents of which are incorporated herein by reference.

[ description of reference numerals ]

100: video encoding device 110: block divider

120: predictor 122: intra-frame predictor

124: inter predictor 130: subtracter

140: inverter 145: quantizer

150: a rearrangement unit 155: entropy encoder

160: inverse quantizer 165: inverse transformer

170: adder 180: loop filter unit

182: deblocking filter 184: SAO (SAO)

186: ALF 190: memory device

1100: video decoding apparatus 1110: entropy decoder

1115: a rearrangement unit 1120: inverse quantizer

1130: inverse transformer 140: predictor(s)

1142: intra predictor 1144: inter-frame predictor

1150: adder 1160: loop filter unit

1162: deblocking filter 1164: SAO (SAO)

1166: ALF 1170: a memory.

Claims

1. A video decoding method of decoding a current block by using intra prediction, the method comprising:

decoding, from the bitstream, related information including an intra prediction mode of the current block;

determining a direction of intra prediction in consideration of a second prediction direction different from the first prediction direction according to the presence or absence of a reference sample at a position to be referred to by the first prediction direction indicated by the intra prediction mode;

performing intra prediction according to a direction of the intra prediction to generate a prediction block of the current block;

reconstructing a residual block of the current block from the bitstream; and

the prediction block and the residual block are added to reconstruct the current block.

2. The video decoding method of claim 1, wherein whether there are reference samples at a position to be referred to in the first prediction direction is determined according to a position of the current block.

3. The video decoding method of claim 1, wherein,

when the first prediction direction is related to a prediction direction using a first reference sample group at the top of the current block or a second reference sample group at the left side of the current block and at least a part of one of the reference sample groups cannot be used for reference, intra prediction is performed by using the other reference sample group as the first reference sample group or the second reference sample group.

4. The video decoding method of claim 1, wherein, regarding any one of a first reference sample group at the top of the current block and a second reference sample group at the left side of the current block, intra prediction is performed in the second prediction direction according to a comparison result between a proportion of reference samples in one reference sample group that can be used for intra prediction in the first prediction direction and a proportion of reference samples in another reference sample group that can be used for intra prediction in the second prediction direction.

5. The video decoding method of claim 4, wherein the values of reference samples of the other reference sample group that cannot be used for intra-prediction in the second prediction direction are determined by using the values of the other reference samples.

6. The video decoding method of claim 1, wherein, regarding any one of a first reference sample group at the top of the current block and a second reference sample group at the left side of the current block, intra prediction is performed in the second prediction direction according to a comparison result between the number of reference samples in one reference sample group that can be used for intra prediction in the first prediction direction and the number of reference samples in another reference sample group that can be used for intra prediction in the second prediction direction.

7. The video decoding method of claim 1, wherein, regarding any one of a first reference sample group at the top of the current block and a second reference sample group at the left side of the current block, the current block is divided into a plurality of regions according to whether a portion of one of the reference sample groups is not available for reference for intra prediction in the first prediction direction, and a direction of intra prediction of each region is selected between the first prediction direction and the second prediction direction.

8. The video decoding method of claim 7, wherein boundaries of the plurality of regions are parallel to the first prediction direction.

9. The video decoding method of claim 7, wherein intra prediction is performed in the first prediction direction for a region where there are neither reference samples that can be used in the first prediction direction nor reference samples that can be used in the second prediction direction.

10. The video decoding method of claim 1, wherein the related information includes an ARS flag, and the ARS flag is information indicating that the operation of determining the second prediction direction is performed based on the presence of a usable reference sample.

11. The video decoding method of claim 1, wherein the direction of intra prediction is determined in consideration of predictions in the first prediction direction and the second prediction direction when the current block is located on at least one of a CTU boundary, a picture boundary, and a slice boundary.

12. A video encoding method of encoding a current block by using intra prediction, the method comprising:

determining an intra prediction mode of the current block;

performing intra prediction according to the determined direction of intra prediction to generate a prediction block of the current block;

generating a residual block by subtracting the prediction block from the current block; and

information related to the intra prediction mode and the residual block are encoded.

13. The video encoding method of claim 12, wherein when the first prediction direction is related to a prediction direction using a first reference sample group at the top of the current block or a second reference sample group at the left side of the current block, and at least a portion of one of the reference sample groups cannot be used for reference, intra prediction is performed by using the other reference sample group that is the first reference sample group or the second reference sample group.

14. The video encoding method of claim 12, wherein, regarding any one of a first reference sample group at the top of the current block and a second reference sample group at the left side of the current block, intra prediction is performed in the second prediction direction according to a comparison result between a proportion of reference samples in one reference sample group that can be used for intra prediction in the first prediction direction and a proportion of reference samples in another reference sample group that can be used for intra prediction in the second prediction direction.

15. The video encoding method of claim 12, wherein, regarding any one of a first reference sample group at the top of the current block and a second reference sample group at the left side of the current block, intra prediction is performed in the second prediction direction according to a comparison result between the number of reference samples in one reference sample group that can be used for intra prediction in the first prediction direction and the number of reference samples in another reference sample group that can be used for intra prediction in the second prediction direction.

16. The video encoding method of claim 12, wherein, regarding any one of a first reference sample group at the top of the current block and a second reference sample group at the left side of the current block, the current block is divided into a plurality of regions according to whether a portion of one of the reference sample groups is not available for reference for intra prediction in the first prediction direction, and a direction of intra prediction of each region is selected between the first prediction direction and the second prediction direction.

17. The video coding method of claim 16, wherein boundaries of the plurality of regions are parallel to the first prediction direction.

18. The video encoding method of claim 16, wherein intra prediction is performed in the first prediction direction for regions where there are neither reference samples that can be used in the first prediction direction nor reference samples that can be used in the second prediction direction.

19. A recording medium readable by a decoder, storing a bitstream generated by a video encoding method, the video encoding method comprising:

determining an intra prediction mode of the current block;

determining a direction of intra prediction in consideration of a second prediction direction different from the first prediction direction according to the presence or absence of a reference sample at a position to be referred to as the first prediction direction indicated by the intra prediction mode;