WO2019078693A1 - Apparatus and method for image encoding or decoding - Google Patents

Abstract

The present invention relates to image encoding for efficient encoding of an image or to image decoding. More specifically, the present invention relates to scanning techniques performed on (quantized) transform coefficients. According to scanning techniques of the present disclosure, a scan pattern used for scanning quantized transform coefficients associated with a current block no longer has a dependency on a previously decoded data block (i.e., a neighboring block of a current block). Therefore, the present invention solves the problem of parsing delay of syntax elements relating to coefficients, which is generated by associating a scan pattern with an intra prediction mode.

Description

Apparatus and method for image encoding or decoding

The present invention relates to image encoding or decoding for efficiently encoding an image. More particularly, to scanning techniques performed on transform coefficients.

The contents described in this section merely provide background information on the present embodiment and do not constitute the prior art.

The HEVC standard supports three types of scan patterns that are used to encode blocks of transform coefficients. 1 is a diagram showing exemplary scan patterns used for encoding a square (8x8) coefficient block. 1 (a) shows an up-right diagonal scan pattern, (b) a horizontal scan pattern, and (c) shows a vertical scan pattern. As shown in FIG. 1, the scan order for the sub-blocks in the coefficient block is the same as the scan order for the coefficients in each sub-block. For example, in the case of a horizontal scan pattern, the scan sequence of the sub-blocks is also the horizontal direction, and the scan sequence of the coefficients in each sub-block is also the horizontal direction. However, the sequence in the actual bit stream is the reverse of the scan pattern.

In the HEVC standard, several syntax elements are used to efficiently signal the location of non-zero coefficients. For example, a sub-block significant flag is a flag expressed for 4x4 subblocks preceding the last valid subblock where the last non-zero coefficient on the scan pattern for the coefficient block (i.e., the last significant coefficient) Quot; 0 " if all coefficients in the subblock are zero, and " 1 " if there is more than one nonzero coefficient. Sub-block significant flags are not signaled for left-ordered subblocks containing DC components.

Consider the case where there are three non-zero coefficients in the 16x16 coefficient block as in Fig. In FIG. 2, the gray marked pixels represent non-zero coefficients (i.e., effective coefficients), the black marked pixels represent the last non-zero coefficients on the scan pattern (i.e., the last effective coefficients) Pixels are all coefficients having a zero value. In the case of a diagonal scan pattern, a sub-block significant flag must be signaled for eight subblocks (subblocks marked in FIG. 3 (a)), thus 8 bits are required. In the case of a horizontal scan pattern, a sub-block significant flag should be signaled for two subblocks (subblocks highlighted in FIG. 3 (b)), thus requiring two bits. Thus, for a given coefficient block, since the number of subblocks to which a sub-block significant flag is to be signaled depends on the scan pattern, it is not possible to parse the related syntax elements of the coefficients before knowing the scan pattern of the coefficient block.

In HEVC, when a given block is coded in an inter prediction scheme, the block of transform coefficients associated with the block is scanned in a diagonal scan pattern.

Some intra coding modes often generate distributions of significant coefficients biased toward the left vertical edge of the block or towards the top edge of the block. Therefore, in many cases, it is advantageous that the coefficients of the block predicted in the horizontal axis mode are scanned in the vertical scan pattern, and the coefficients of the block predicted in the vertical axis mode are scanned in the horizontal scan pattern. In consideration of this, in the HEVC, a vertical scan pattern is used for some intra prediction modes (mode numbers 6 to 14), horizontal scan patterns are used for some other intra prediction modes (mode numbers 22 to 30) In the modes, diagonal scan patterns are used. That is, when a given block is coded in the intra prediction mode, one of the three scan patterns is selected according to the intra prediction mode, and the coefficients of the corresponding block are scanned.

As described above, in the HEVC, when a given block is coded in the intra prediction mode, the scan pattern of the coefficient block related to the block is determined according to the intra prediction mode. Furthermore, in signaling which intra prediction mode is used, the intra prediction mode number is signaled in the form of an MPM index value or a non-MPM index value without signaling the number (or index) of the intra prediction mode used. Therefore, it is necessary to know the MPM and non-MPM configuration information of the corresponding block to know the actual intra prediction mode of the corresponding block. In order to know the MPM and non-MPM information of the corresponding block, In order to know the intra prediction mode of the block, the neighboring block information of the neighboring block must be known.

As described above, before the scan pattern of a given block is known, it is impossible to parse the syntax elements related to the transform coefficients related to the block. When a given block is coded by the intra prediction method, The syntax elements can not be parsed until the intra prediction mode is determined. That is, the parsing of the syntax elements related to the transform coefficients is delayed until the intra-prediction mode of the neighboring block of the given block or the neighboring block of the neighboring block is determined.

It is an object of the present invention to provide scan pattern determination techniques capable of eliminating the disadvantages described above.

According to an aspect of the present invention, there is provided a method of determining a scan pattern to be applied to transform coefficients of a current block in an image decoding process, the method comprising: determining whether a current block is coded in an intra- ; Comparing a width and a height of the current block with each other when the current block is coded in an intra prediction mode; And if the current block is a vertically long rectangle, a vertical scan pattern is determined as a scan pattern to be applied to the transform coefficients, and if the current block is a horizontally long rectangle, a horizontal scan pattern is used as a scan pattern to be applied to the transform coefficients And if the block of transform coefficients is square, determining a diagonal scan pattern as a scan pattern to be applied to the transform coefficients.

According to another aspect of the present invention there is provided an apparatus for determining a scan pattern to be applied to transform coefficients of a current block in an image decoding process, the apparatus comprising a memory and one or more processors configured to perform the following steps. Wherein the one or more processors determine whether a current block is coded in an intra prediction or an inter prediction; Comparing a width and a height of the current block with each other when the current block is coded in an intra prediction mode; And if the current block is a vertically long rectangle, a vertical scan pattern is determined as a scan pattern to be applied to the transform coefficients, and if the current block is a horizontally long rectangle, a horizontal scan pattern is used as a scan pattern to be applied to the transform coefficients And if the block of transform coefficients is square, determining a diagonal scan pattern as a scan pattern to be applied to the transform coefficients.

According to another aspect of the present invention, there is provided a method of determining a scan pattern to be applied to transform coefficients of a current block in an image decoding process, the method comprising parsing a syntax element indicating a position of a last valid coefficient in a block of the transform coefficients ; Determining whether a position of the last significant coefficient is greater than a preset threshold value; Parsing a syntax element indicating a scan pattern for a one-dimensional array of the transform coefficients from a plurality of available scan patterns if the position of the last effective coefficient is greater than a predetermined threshold; And determining a scan pattern set as a default among the plurality of available scan patterns as a scan pattern for the one-dimensional array of the transform coefficients if the position of the last effective coefficient is not greater than a preset threshold value do.

According to another aspect of the present invention there is provided an apparatus for determining a scan pattern to be applied to transform coefficients of a current block in an image decoding process, the apparatus comprising a memory and one or more processors configured to perform the following steps . The one or more processors parsing a syntax element indicating a location of a last significant coefficient within a block of transform coefficients; Determining whether a position of the last significant coefficient is greater than a preset threshold value; Parsing a syntax element indicating a scan pattern for a one-dimensional array of the transform coefficients from a plurality of available scan patterns if the position of the last effective coefficient is greater than a predetermined threshold; And determining a scan pattern set as a default among the plurality of available scan patterns as a scan pattern for the one-dimensional array of the transform coefficients if the position of the last effective coefficient is not greater than a predetermined threshold value.

1 is a diagram showing exemplary scan patterns used for encoding a square (8x8) coefficient block.

FIG. 2 shows an example of a 16-by-16 coefficient block in which there are three non-zero coefficients.

FIG. 3 is a view for explaining the number of subblocks to be signaled whether the subblock is a valid subblock for the 16 × 16 coefficient block of FIG.

4 is a block diagram of an image encoding apparatus capable of implementing the techniques of the present disclosure.

5 is an illustration of block partitioning using a QTBT structure.

6 shows an example of a plurality of intra prediction modes.

7 is an exemplary view of neighboring blocks of the current block.

Fig. 8 is a diagram showing exemplary scan patterns used for encoding non-square (8 x 16 or 16 x 8) coefficient blocks.

9 is a block diagram showing an example of an entropy encoding unit used in the image encoding apparatus of FIG.

10 illustrates an image decoding apparatus capable of implementing the techniques of the present disclosure.

11 is a block diagram showing an example of an entropy decoding unit used in the video decoding apparatus of FIG.

12 is a flowchart illustrating a method of determining a scan pattern by the image decoding apparatus according to an embodiment of the present invention.

13 is a flowchart illustrating a method of determining a scan pattern by an image decoding apparatus according to another embodiment of the present invention.

Hereinafter, some embodiments of the present invention will be described in detail with reference to exemplary drawings. It should be noted that, in adding the identification codes to the constituent elements of the respective drawings, the same constituent elements are denoted by the same reference symbols as possible even if they are displayed on different drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

In some scanning techniques described below, a horizontally long rectangular block has a strong tendency to be encoded in the intra-prediction mode in the vertical axis direction (since the intra-prediction is more accurate from the reference pixel), and the vertically long rectangular block has a strong It is noted that there is a strong tendency to be encoded in the intra prediction mode. Note also that the intra-prediction mode in the vertical axis generates distributions of the effective coefficients, which are often biased towards the upper horizontal edge, while the intra-prediction mode in the horizontal direction generates distributions of the effective coefficients biased towards the left vertical edge . Accordingly, it is advantageous that the block of transversal rectangular rectangular transform coefficients is advantageously scanned in the horizontal scan pattern, and the block of vertically long rectangle transform coefficients is scanned in the vertical scan pattern. Accordingly, in some scanning techniques, in determining a scan pattern for a block of transform coefficients of an intra-predicted block, the scan pattern may be in the form of a block (e.g., a horizontally long rectangle, a vertically long rectangle, . That is, the scan pattern is determined according to the type of the current block.

Some other scanning techniques described below are to explicitly signal the actually applied scan pattern among a plurality of available scan patterns (e.g., diagonal, horizontal, and vertical scan patterns). For example, by comparing the results of applying the plurality of available scan patterns in terms of compression and / or image quality, a desired scan pattern can be determined in the image encoding apparatus. The scan pattern selected by the image encoding apparatus may be transmitted to the image decoding apparatus as an index in the bitstream (for example, as a block level syntax).

According to the scanning techniques of the present disclosure, the scan pattern used to scan the quantized transform coefficients associated with the current block is no longer dependent on the intra prediction mode applied to the current block (and thus the previously decoded data block) Do not have. Thus, the problem of delaying the parsing of the syntax elements with respect to the coefficients arising by associating the scan pattern with the intra-prediction mode is solved. In addition, the decoding apparatus can perform a process of determining the intra prediction mode and a process of reconstructing a block of two-dimensional transform coefficients according to a scan pattern in parallel.

4 is a block diagram of an image encoding apparatus capable of implementing the techniques of the present disclosure.

The image encoding apparatus includes a block division unit 410, a prediction unit 420, a subtractor 430, a transform unit 440, a quantization unit 445, an entropy encoding unit 450, an inverse quantization unit 460, An adder 470, a filter unit 480 and a memory 490. [ Each component of the image encoding apparatus may be embodied as a hardware chip, or may be embodied as software, and the microprocessor may be implemented to execute the function of the software corresponding to each component.

One video (video) is composed of a plurality of pictures. Each picture is divided into a plurality of areas and coding is performed for each area. For example, one picture is divided into one or more slices or / and tiles, and each slice or tile is divided into one or more CTU (Coding Tree Unit). Each CTU is divided into one or more CUs (Coding Units) by a tree structure. The information applied to each CU is encoded as a syntax of the CU, and the information commonly applied to the CUs included in one CTU is encoded as the syntax of the CTU. In addition, information that is commonly applied to all blocks in one slice is encoded as a syntax of a slice, and information applied to all blocks constituting one picture is encoded into a picture parameter set (PPS) . Further, information that is commonly referred to by a plurality of pictures is encoded into a sequence parameter set (SPS). The information that is commonly referred to by one or more SPSs is encoded into a Video Parameter Set (VPS).

The block dividing unit 410 determines the size of the CTU (Coding Tree Unit). The information on the size of the CTU (CTU size) is encoded as the syntax of the SPS or PPS and transmitted to the image decoding apparatus. The block dividing unit 410 divides each picture constituting an image into a plurality of CTUs (Coding Tree Units) having a determined size, and thereafter recursively reuses the CTUs using a tree structure. . A leaf node in a tree structure becomes a coding unit (CU) which is a basic unit of coding. In the tree structure, a quad tree (QuadTree, QT) in which an upper node (or a parent node) is divided into four sub nodes (or child nodes) of the same size, or a binary tree in which an upper node is divided into two lower nodes , BT), or a ternary tree (TT) in which an ancestor node is divided into three subnodes at a ratio of 1: 2: 1, or a structure combining one or more of these QT structures, BT structures and TT structures have. For example, a QuadTree plus BinaryTree (QTBT) structure can be used, or a QuadTree plus BinaryTreeTernaryTree (QTBTTT) structure can be used.

5 is an illustration of block partitioning using a QTBT structure. FIG. 5A is an example in which a block is divided by a QTBT structure, and FIG. 5B is a tree structure thereof. In FIG. 5, the solid line represents the division by the QT structure, and the dotted line represents the division by the BT structure. 5 (b), the absence of parentheses indicates a layer of QT, and the presence of parentheses indicates a layer of BT. In the BT structure represented by the dotted line, the numbers represent the division type information.

As shown in FIG. 5, the CTU can be first divided into a QT structure. The quadtree partitioning can be repeated until the size of the splitting block reaches the minimum block size (MinQTSize) of the leaf node allowed in QT. The first flag (QT_split_flag) indicating whether each node of the QT structure is divided into four nodes of the lower layer is encoded by the entropy encoding unit 450 and signaled to the video decoding apparatus.

If the leaf node of the QT is not larger than the maximum block size (MaxBTSize) of the root node allowed in BT, it can be further partitioned into a BT structure. In BT, there may be a plurality of split types. For example, in some examples, there may be two types: a symmetric horizontal splitting and a vertically splitting type (i.e., symmetric vertical splitting) that divides a block of the node into two blocks of equal size horizontally. The second flag (BT_split_flag) indicating whether each node of the BT structure is divided into blocks of the lower layer and the division type information indicating the type to be divided are encoded by the entropy encoding unit 450 and transmitted to the image decoding apparatus . On the other hand, there may be additional types of dividing the blocks of the node into two asymmetric blocks. In the asymmetric form, the block of the node may be divided into two rectangular blocks having a size ratio of 1: 3, or the block of the corresponding node may be divided into diagonal directions.

When QTBTTT is used as another example of the tree structure, after the CTU is first divided into the QT structure, the leaf nodes of the QT may be divided into one or more of the BT structure or the TT structure. The TT structure may also have a plurality of split types. For example, when divided, there are two types: a symmetric horizontal splitting (i.e., symmetric vertical splitting) in which a block of the node is horizontally divided into three blocks at a 1: 2: 1 ratio Can exist. In the case of QTBTTT, a flag indicating whether each node is divided into blocks of a lower layer and whether the BT structure or the TT structure is used in addition to the division type information (or dividing direction information) indicating the divided type Additional information for discrimination can be signaled to the video decoding apparatus.

The CU may have various sizes depending on the QTBT or QTBTTT segment from the CTU. Hereinafter, a block corresponding to a CU to be encoded or decoded (i.e., a leaf node of QTBTTT) is referred to as a 'current block'.

The prediction unit 420 generates a prediction block by predicting the current block. The prediction unit 420 includes an intra prediction unit 422 and an inter prediction unit 424.

The intraprediction unit 422 predicts pixels in the current block using pixels (reference pixels) located around the current block in the current picture including the current block. There are a plurality of intra prediction modes according to the prediction direction, and the neighboring pixels to be used and the calculation formula are defined differently according to each prediction mode. In particular, the intra prediction unit 422 can determine an intra prediction mode to be used for coding the current block. In some examples, the intra-prediction unit 422 may encode the current block using several intra-prediction modes and select an appropriate intra-prediction mode to use from the tested modes. For example, the intra-prediction unit 422 may calculate rate-distortion values using a rate-distortion analysis for various tested intra-prediction modes, and may employ rate- The intra prediction mode may be selected.

6 shows an example of a plurality of intra prediction modes.

As shown in FIG. 6, the plurality of intraprediction modes may include two non-directional modes (planar mode and DC mode) and 65 directional modes.

The intra-prediction unit 422 selects one intra-prediction mode from a plurality of intra-prediction modes, and predicts the current block using neighboring pixels (reference pixels) determined by the selected intra-prediction mode and an equation. The information on the selected intra prediction mode is encoded by the entropy encoding unit 450 and transmitted to the image decoding apparatus.

In order to efficiently encode the intra prediction mode information indicating which of the plurality of intra prediction modes is used as the intra prediction mode of the current block, the intra prediction unit 422 performs intra prediction The most probable mode (MPM) can be determined as a part of the intra prediction mode. The mode information indicating whether the intra prediction mode of the current block is selected in the MPM is generated and transmitted to the entropy encoding unit 450. In general, when the intra prediction mode of the current block is selected from the MPM, the first intra identification information for indicating which mode of the MPM is selected as the intra prediction mode of the current block is transmitted to the entropy encoding unit 450. If the intraprediction mode of the current block is not selected among the MPMs, the entropy encoding unit 450 outputs second intra identification information for indicating which of the modes other than the MPM is selected as the intra prediction mode of the current block, .

Hereinafter, a method of constructing the MPM list will be described. Here, the MPM list is constructed by six MPMs, but the present invention is not limited thereto. The number of MPMs included in the MPM list can be selected within a range of three to ten.

First, an MPM list is constructed using an intra prediction mode of neighboring blocks of the current block. The neighboring blocks are all or part of the left block L, the upper block A, the lower left block BL, the upper right block AR and the upper left block AL of the current block, for example, as shown in Fig. . ≪ / RTI >

The intra prediction modes of these neighboring blocks are included in the MPM list. Here, the intra prediction modes of the valid blocks in the order of the left block L, the upper block A, the lower left block BL, the upper right block AR and the upper left block AL are included in the MPM list, The candidates are constructed by adding the planar mode and the DC mode to the intra prediction modes of the blocks. Alternatively, a mode effective in the order of the left block L, the upper block A, the planar mode, the DC mode, the lower left block BL, the upper right block AR, and the upper left block AL may be added to the MPM list have. Alternatively, a mode effective in order of the left block L, the upper block A, the planar mode, the lower left block BL, the upper right block AR, the upper left block AL, and the DC mode may be added to the MPM list have.

The MPM list includes only different intra prediction modes. That is, when there is a duplicated mode, only one mode is included in the MPM list.

If the number of MPMs in the list is less than the predetermined number (e.g., 6), the MPM may be derived by adding -1 or +1 to the directional modes in the list. If the number of MPMs in the list is smaller than the predetermined number, a number of modes are added to the MPM list in the order of vertical mode, horizontal mode, and diagonal mode. You may.

The inter-prediction unit 424 searches for a block most similar to the current block in the reference picture coded and decoded earlier than the current picture, and generates a prediction block for the current block using the searched block. Then, a motion vector corresponding to the displacement between the current block in the current picture and the prediction block in the reference picture is generated. Motion information including information on a reference picture used for predicting a current block and motion vector information is encoded by an entropy encoding unit 450 and transmitted to an image decoding apparatus.

The subtractor 430 subtracts the prediction block generated by the intra prediction block 422 or the inter prediction block 424 from the current block to generate a residual block.

The transforming unit 440 transforms the residual signal in the residual block having pixel values in the spatial domain into transform coefficients in the frequency domain. The transforming unit 440 may transform the residual signals in the residual block by using the size of the current block as a transform unit or divide the residual block into a plurality of smaller subblocks and transform residual signals into subblock- Conversion. There are various ways of dividing the residual block into smaller sub-blocks. For example, it may be divided into sub blocks of the same size that have been set, or a QT (quadtree) type partition using a residual block as a root node.

The quantization unit 445 quantizes the transform coefficients output from the transform unit 440 and outputs the quantized transform coefficients to the entropy encoding unit 450.

The inverse quantization unit 460 dequantizes the quantized transform coefficients output from the quantization unit 445 to generate transform coefficients. The inverse transform unit 465 transforms the transform coefficients output from the inverse quantization unit 460 from the frequency domain to the spatial domain and restores the residual block.

The adder 470 adds the reconstructed residual block and the prediction block generated by the predictor 420 to reconstruct the current block. The pixels in the reconstructed current block are used as reference pixels when intra prediction of the next-order block is performed.

The filter unit 480 deblock-filters the boundaries between the restored blocks and stores them in the memory 490 in order to remove blocking artifacts caused by coding / decoding on a block-by-block basis. When all the blocks in one picture are reconstructed, the reconstructed picture is used as a reference picture for inter prediction of a block in a picture to be coded later.

The entropy encoding unit 450 encodes the quantized transform coefficients using an entropy encoding scheme such as CABAC to generate a bitstream. In addition, the entropy encoding unit 450 may entropy-encode various syntax elements that are useful in decoding motion vector information and image data in the image decoding apparatus. The syntax elements may include, for example, block partition information, prediction type information, intra prediction information, inter prediction information, and the like. The syntax elements may also include an effective coefficient flag indicating whether certain coefficients are valid (e.g., non-zero) and an efficacy map having a last significant coefficient flag indicating whether the particular coefficient is the last significant coefficient have.

In order to entropy code a block of quantized transform coefficients (i.e., coefficient block), a scanning process is generally performed such that a two-dimensional array of coefficients in the coefficient block is transformed into an ordered one-dimensional array of transform coefficients . This scanning process is performed on the coefficient block using one of a plurality of available scan patterns. If the transform coefficients have been scanned, the entropy encoding unit 450 applies the entropy encoding to the one-dimensional array of coefficients. Scanning may be applied to encode the levels of valid (i.e., non-zero) coefficients, and to encode the signatures of significant coefficients, to encode positions of valid (i.e., non-zero) coefficients.

Note that although the present disclosure describes the entropy encoding unit 450 performing scanning, it is noted that other processing units (e.g., the quantization unit 460) may perform the scanning techniques of the present disclosure in other examples .

Fig. 8 is a diagram showing exemplary scan patterns used for encoding non-square (8 x 16 or 16 x 8) coefficient blocks. These scan patterns include a diagonal pattern (FIG. 8A), a vertical scan pattern (FIG. 8B), and a horizontal scan pattern (FIG. 8C). The illustrated scan patterns start at the left index coefficient of each coefficient block and end at the right edge coefficient. Also, in the illustrated scan patterns, the 4x4 sub-blocks in the coefficient block and the intra-sub-block coefficients have the same scan form.

9 is a block diagram showing an example of an entropy encoding unit used in the image encoding apparatus of FIG. 9, the entropy encoding unit may include a scan pattern selection unit 910, a scan unit 920, an entropy encoding engine 930, and a scan pattern memory 940. Note that while the components 910 to 940 are shown in Fig. 9 in terms of the functional aspects of the entropy encoding unit, these components 910 to 940 may be highly integrated with each other.

The scan pattern selection unit 910 selects a scan pattern to be used by the scan unit 920 for the validity map scan and the coefficient level scan among the available scan patterns (e.g., diagonal, horizontal, and vertical scan patterns) . In some examples, the scan pattern selector 910 can determine a desired scan pattern by comparing the results of applying the available scan patterns in terms of compression and / or image quality. In some examples, the scan pattern selector 910 may select one scan pattern from a plurality of available scan patterns, based on prediction type, block size, conversion, or other characteristics of the encoded image . The scan memory 940 may store instructions, tables, and / or data that define which scan patterns to use for particular situations. For example, when the current block to be encoded is inter-predicted, the scan pattern may be a predetermined scan pattern (e.g., diagonal scanning). In some embodiments, a default scan pattern (e.g., diagonal scanning) may be selected if the position of the last significant coefficient in the coefficient block is not greater than a predetermined threshold position (e.g., half the size of the coefficient block) . In some embodiments, when the current block to be coded is intra-predicted, a plurality of blocks (for example, a horizontally long rectangle, a vertically long rectangle, a square, or a block is larger than a predetermined threshold) One scan pattern among the available scan patterns can be selected.

The scan unit 920 receives a scan pattern from the scan pattern selection unit 910 and applies the received scan pattern to a block of transform coefficients to generate a one-dimensional array of coefficients from the block of transform coefficients.

The entropy encoding engine 930 may apply an entropy encoding process to the scanned coefficients to generate a bitstream. The entropy encoding process may be applied after the coefficients have been scanned into the one-dimensional array. In other examples, the coefficients may be processed directly in a two-dimensional array (i.e., coefficient block) using a scan pattern. In addition to the transform coefficient data, motion vector data and various syntax elements useful for decoding the encoded image in the bitstream and the various syntax elements may be encoded within the bitstream.

Additionally, the entropy encoding unit may provide signaling in the bitstream indicating the scan pattern used in the entropy encoding process. For example, the scan pattern may be passed to the image decoding device as an index in the bitstream (e.g., as block level syntax). If it is predetermined which scan patterns to use for certain situations, there may not be a need to provide signaling in the bitstream.

Hereinafter, a video decoding apparatus will be described.

10 illustrates an image decoding apparatus capable of implementing the techniques of the present disclosure.

The image decoding apparatus includes an entropy decoding unit 1010, an inverse quantization unit 1020, an inverse transformation unit 1030, a prediction unit 1040, an adder 1050, a filter unit 1060, and a memory 1070. 4, each component may be embodied as a hardware chip, or may be embodied as software, and a microprocessor may be implemented to execute the functions of software corresponding to each component.

The entropy decoding unit 1010 can reconstruct a one-dimensional array of quantized transform coefficients of the current block by applying an entropy decoding process such as CABAC to the bitstream generated by the image encoding apparatus. In addition, the entropy decoding unit 1010 may regenerate motion vector data and various syntax elements, which are useful for decoding an image encoded in a bitstream by applying an entropy decoding process.

The entropy decoding unit 1010 extracts information on a CTU size from a high level syntax such as a sequence parameter set (SPS) or a picture parameter set (PPS) to determine the size of the CTU, Of CTU. Then, the CTU is divided into a tree structure (e.g., a QTBT structure or a QTBTTT structure) by determining the CTU as a top layer of the tree structure, i.e., a root node, and extracting partition information for the CTU. As an example, when the CTU is divided using the QTBT structure, the first flag (QT_split_flag) related to the division of the QT is first extracted and each node is divided into four nodes of the lower layer. For the node corresponding to the leaf node of the QT, the second flag (BT_split_flag) and the split type (split direction) information related to the BT split are extracted and the corresponding leaf node is divided into the BT structure. As another example, when the CTU is divided using the QTBTTT structure, the first flag (QT_split_flag) related to the division of the QT is extracted and each node is divided into four nodes of the lower layer. A split flag (split_flag) and a split type (or dividing direction) information indicating whether the node corresponding to the leaf node of the QT is further divided into BT or TT and additional information for distinguishing whether the BT structure or the TT structure . In this way, each node below the leaf nodes of the QT is recursively divided into BT or TT structures.

When the current block (current block) to be decoded is determined through the division of the tree structure, the entropy decoding unit 1010 extracts information on a prediction type indicating whether the current block is intra-predicted or inter-predicted.

When the prediction type information indicates intra prediction, the entropy decoding unit 1010 extracts a syntax element for intra prediction information (intra prediction mode) of the current block. First, the entropy decoding unit 1010 extracts mode information (i.e., MPM flag) indicating whether the intra prediction mode of the current block is selected in the MPM. In general, when the intra-mode encoding information indicates that the intra-prediction mode of the current block is selected from the MPM, the first intra-identification information for indicating which mode of the MPM is selected as the intra-prediction mode of the current block is extracted And the intra-mode encoding information indicates that the intra-prediction mode of the current block is not selected among the MPMs, the second intra-prediction mode indicating which of the modes other than the MPM is selected as the intra- And extracts the identification information.

The entropy decoding unit 1010 converts the one-dimensional array of the quantized transform coefficients of the current block into a scan pattern (i.e., a reverse scan pattern) for mirroring the scan pattern used by the entropy encoding unit 460 of the image encoding apparatus, And then re-transformed into blocks of quantized transform coefficients (i.e., coefficient blocks) of the two-dimensional block format.

The entropy decoding unit 1010 may determine the scan pattern based on the signaling in the encoded bitstream. For example, the entropy decoding unit 1010 may receive a syntax element that explicitly signals a scan pattern. Further, in some embodiments, the entropy decoding unit 1010 may be able to infer the scan pattern without signaling. The inference of the scan pattern may be based on other characteristics of the encoded image, such as prediction type, block size, and whether or not to convert. In some examples, when the position of the last significant coefficient in the coefficient block signaled from the image coding apparatus is larger than a predetermined threshold value (for example, half of the size of the coefficient block), the entropy decoding unit 1010 sets the default scan pattern (E.g., diagonal scanning) may be used. In some examples, if the current block to be decoded is inter-predicted, the scan pattern may be a predetermined scan pattern (e.g., diagonal scan). In some examples, if the current block to be decoded is intra-predicted, the scan pattern may be determined according to the shape of the block (e.g., a horizontally long rectangle, a vertically long rectangle, a square, or a block greater than a predetermined threshold) It can be deduced from a plurality of usable scan patterns.

The entropy decoding unit 1010 generates syntax elements (e.g., valid (i. E., Non-processed) coefficients related to the transform coefficients based on the determined scan pattern to reconstruct a block of two- Syntax elements for the positions of the coefficients, syntax elements for the positions of the coefficients, syntax elements for the levels of the significant coefficients, and / or syntax elements for the signs of the significant coefficients).

The inverse quantization unit 1020 dequantizes the quantized transform coefficients and the inverse transform unit 1030 inversely transforms the inversely quantized transform coefficients from the frequency domain to the spatial domain to generate residual blocks for the current block by restoring the residual signals.

The prediction unit 1040 includes an intra prediction unit 1042 and an inter prediction unit 1044. The intra prediction unit 1042 is activated when the intra prediction is the prediction type of the current block, and the inter prediction unit 1044 is activated when the intra prediction is the prediction type of the current block.

The intra prediction unit 1042 determines an intra prediction mode of the current block among the plurality of intra prediction modes from the syntax element for the intra prediction mode extracted from the entropy decoding unit 1010, The pixels are used to predict the current block.

To determine the intra prediction mode of the current block, the intra prediction unit 1042 constructs an MPM list including a predetermined number of MPMs from neighboring blocks of the current block. The method of constructing the MPM list is the same as that of the intra prediction unit 422 of FIG.

In general, when the intra prediction mode information (i.e., the MPM flag) indicates that the intra prediction mode of the current block is selected from the MPM, the intra prediction unit 1042 sets the intra prediction mode And selects the MPM indicated by the intra prediction mode of the current block. On the other hand, when the mode information indicates that the intra prediction mode of the current block is not selected from the MPM, the intra prediction mode of the current block is determined among the intra prediction modes other than the MPMs in the MPM list using the second intra identification information do.

The inter-prediction unit 1044 determines the motion information of the current block using the syntax element extracted from the entropy decoding unit 1010, and predicts the current block using the determined motion information.

The adder 1050 adds the residual block output from the inverse transform unit and the prediction block output from the inter prediction unit or the intra prediction unit to reconstruct the current block. The pixels in the reconstructed current block are utilized as reference pixels for intra prediction of a block to be decoded later.

The filter unit 1060 deblock-filters the boundaries between the restored blocks and stores them in the memory 1070 in order to remove blocking artifacts caused by decoding on a block-by-block basis. When all blocks in one picture are reconstructed, the reconstructed picture is used as a reference picture for inter prediction of a block in a picture to be decoded later.

11 is a block diagram showing an example of an entropy decoding unit 1010 used in the video decoding apparatus of FIG. 11, the entropy decoding unit may include a scan pattern selection unit 1110, a scan unit 1120, an entropy decoding engine 1130, and a scan pattern memory 1140. It should be noted that although the components 1110 to 1140 are shown in Fig. 11 in terms of the functional aspects of the entropy decoding unit, these components 1110 to 1140 may be highly integrated with each other.

Entropy decoding engine 1130 may apply an entropy decoding process, e.g., a CABAC process, to the bitstream to reconstruct a one-dimensional array of transform coefficients. In addition to the residual transform coefficient data, the entropy decoding engine 1130 may apply entropy decoding to the bitstream to reconstruct various syntax elements useful for decoding the encoded image in the bitstream.

The scan pattern selection unit 1110 selects a scan pattern to be used by the scan unit 1120 among available scan patterns (e.g., diagonal, horizontal, and vertical scan patterns). In some examples, the scan pattern selector 1110 can determine the scan pattern based on the signaling in the bitstream. For example, the scan pattern selector 1110 may receive a syntax element (e.g., an index) that explicitly signals the scan pattern from the entropy decoding engine. In some examples, the scan pattern selector 1110 can deduce a scan pattern without receiving signaling. The scan pattern may be based on prediction type, block size, whether to convert, or other characteristics of the encoded image. In some embodiments, if the current block to be decoded is inter-predicted, the scan pattern may be a predetermined scan pattern (e.g., a diagonal scan). In some embodiments, when the current block to be decoded is intra-predicted, a plurality of blocks (e.g., a block of a long horizontal rectangle, a vertically long rectangle, a square, or a block larger than a predetermined threshold value) It can be deduced from available scan patterns. The scan pattern memory 1140 may store instructions, tables and / or data defining scan patterns.

The scan unit 1120 receives a scan pattern from the scan pattern selection unit 1110 and generates a block of two-dimensional transform coefficients from the one-dimensional array of the transform coefficients by applying the received scan pattern.

12 is a flowchart illustrating a method of determining a scan pattern by the image decoding apparatus according to an embodiment of the present invention. The method illustrated in FIG. 12 may be performed, for example, by the entropy decoding unit 1010 of the video decoding apparatus of FIG. In other instances, one or more other units of the video decoding apparatus may additionally be involved in performing the techniques of the present disclosure.

As a preliminary step, the video decoding apparatus entropy-decodes the bitstream received from the video encoding apparatus, extracts information related to the block division, and determines a current block to be decoded.

The video decoding apparatus extracts information on the prediction type from the bitstream to determine whether the current block is intra-predicted or inter-predicted (S1210).

If it is determined that the information on the predictive type is inter-predicted (No in S1210), the image decoding apparatus determines the diagonal scan pattern as a scan pattern to be applied to the transform coefficients of the current block (S1250).

If it is determined that the prediction type information is intra-predicted (Yes in S1210), the video decoding apparatus determines whether the width and height of the current block are smaller than the threshold value th (S1220). If at least one of the width and the height of the current block is not smaller than the threshold value (NO at S1220), the image decoding apparatus determines the diagonal scan pattern as a scan pattern to be applied to the transform coefficients of the current block (S1250) . If the width and height of the current block are respectively less than the threshold value (Yes in S1220), the image decoding apparatus judges whether the width and height of the current block are the same (i.e., whether the current block is a square) (S1230).

If the width and height of the current block are the same (i.e., the current block is a square, YES in S1230), the image decoding apparatus determines a scan pattern to be applied to the transform coefficients of the current block (S1250).

If the width and height of the current block are not equal to each other (i.e., the current block is not a square), the image decoding apparatus determines whether the height of the current block is greater than the width S1240) is further determined.

If the current block height is greater than the width (i.e., if the current block is a vertically long rectangle; " Yes " in S1240), the image decoding apparatus sets the vertical scan pattern to a scan pattern (S1260).

If the height of the current block is smaller than the width (i.e., if the current block is a horizontally elongated rectangle; " NO " in S1240), the image decoding apparatus adds a horizontal scan pattern to a scan pattern (S1270).

The video decoding apparatus determines, based on the determined scan pattern, syntax elements relating to positions of valid coefficients (e.g., valid (i.e., non-zero) coefficients on transform coefficients, syntax elements on levels of valid coefficients , And / or syntax elements for the signs of significant coefficients).

The video decoding apparatus may be configured to generate syntax elements (e.g., valid (i.e., non-zero)) for transform coefficients based on a determined scan pattern to reconstruct a block of 2-dimensional quantized transform coefficients (i.e., Syntax elements for the positions of coefficients, syntax elements for the levels of significant coefficients, and / or syntax elements for the signs of significant coefficients).

13 is a flowchart illustrating a method of determining a scan pattern by an image decoding apparatus according to another embodiment of the present invention. The scan pattern determination method of this embodiment uses a predetermined default scan pattern if the position of the last valid coefficient according to the scan pattern in the block of transform coefficients is sufficiently close to the position of the left offset pixel (0,0) in the transform block, (0, 0), the scan pattern selected by the image encoding apparatus among the plurality of available scan patterns is explicitly signaled.

The video decoding apparatus extracts, from the bit stream, information indicating the position of the last valid coefficient according to the scan pattern in the coefficient block. The last effective coefficient may be identified by the variables last_significant_coeff_x and last_significant_coeff_y (S1310).

The video decoding apparatus determines whether the position of the last valid coefficient is larger than a preset threshold position (S1320). The critical position may be a function of the width and height of the current block and may be a specific constant independent of the width and height of the current block. For example, the threshold associated with last_significant_coeff_x may be "width of current block / 2" and the threshold associated with last_signicant_coeff_y may be "height of current block / 2".

If the position of the last valid coefficient is not larger than the predetermined threshold value (NO in S1320), the image decoding apparatus sets a scan pattern (for example, diagonal scanning) set in default to a scan (S1350).

If the position of the last valid coefficient is larger than a predetermined threshold (YES in S1320), the image decoding apparatus parses a syntax element indicating a scan pattern selected by the encoder among the set of scan pattern candidates (S1330). The image decoding apparatus determines the scan pattern indicated by the syntax element as a scan pattern to be applied to the transform coefficients of the current block. The syntax element may include an index value of a scan pattern selected from a set of scan pattern candidates. TU (truncated unary) coding may be used for the binarization method of representing the scan pattern. For example, diagonal scanning can be expressed as "0", vertical scanning can be expressed as "10", and horizontal scanning can be expressed as "11". As another example, when there are four scan patterns, each scan pattern can be represented by a fixed length (FL) code of 2 bits. As another example, when there are two scan patterns, each scan pattern can be represented by a fixed length (FL) code of 1 bit.

The image decoding apparatus generates syntax elements (e.g., valid (i. E., Valid (i. E., Valid (i. E. , Syntax elements for the positions of the coefficients, syntax elements for the levels of significant coefficients, and / or syntax elements for the signs of the significant coefficients).

The foregoing description is merely illustrative of the technical idea of the present embodiment, and various modifications and changes may be made to those skilled in the art without departing from the essential characteristics of the embodiments. Therefore, the present embodiments are to be construed as illustrative rather than restrictive, and the scope of the technical idea of the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment should be construed according to the following claims, and all technical ideas within the scope of equivalents thereof should be construed as being included in the scope of the present invention.

CROSS-REFERENCE TO RELATED APPLICATION

This patent application is a continuation-in-part of U.S. Patent Application No. 10-2017-0136811 filed on October 20, 2017, which is incorporated herein by reference in its entirety, and U.S. Patent Application No. Priority is claimed to 10-2018-0001860.

Claims (12)

1. A method for determining a scan pattern to be applied to transform coefficients of a current block in a video decoding process,

   Determining whether a current block is coded in an intra prediction mode or an inter prediction mode;

   Comparing a width and a height of the current block with each other when the current block is coded in an intra prediction mode; And

   If the current block is a vertically long rectangle, a vertical scan pattern is determined as a scan pattern to be applied to the transform coefficients. If the current block is a horizontally long rectangle, a horizontal scan pattern is determined as a scan pattern to be applied to the transform coefficients And if the block of transform coefficients is square, determining a diagonal scan pattern as a scan pattern to be applied to the transform coefficients;

   / RTI >
2. The method according to claim 1,

   Comparing a width and a height of the current block with predetermined threshold values, respectively; And

   Determining a diagonal scan pattern as a scan pattern to be applied to the transform coefficients if the width and height of the current block are not less than a predetermined threshold, respectively.
3. The method according to claim 1,

   When the current block is coded by the inter prediction method,

   Determining a diagonal scan pattern as a scan pattern to be applied to the transform coefficients.
4. An apparatus for determining a scan pattern to be applied to transform coefficients of a current block in an image decoding process,

   Memory; And

   Comprising one or more processors,

   The one or more processors,

   Determining whether a current block is coded in an intra prediction mode or an inter prediction mode;

   Comparing a width and a height of the current block with each other when the current block is coded in an intra prediction mode; And

   If the current block is a vertically long rectangle, a vertical scan pattern is determined as a scan pattern to be applied to the transform coefficients. If the current block is a horizontally long rectangle, a horizontal scan pattern is determined as a scan pattern to be applied to the transform coefficients And if the block of transform coefficients is square, determining a diagonal scan pattern as a scan pattern to be applied to the transform coefficients.
5. 5. The method of claim 4,

   The one or more processors,

   Comparing a width and a height of the current block with predetermined threshold values, respectively; And

   And to determine a diagonal scan pattern as a scan pattern to be applied to the transform coefficients if the width and height of the current block are not less than predetermined threshold values respectively.
6. 5. The method of claim 4,

   The one or more processors,

   And a step of determining a diagonal scan pattern as a scan pattern to be applied to the transform coefficients when the current block is coded by the inter prediction method.
7. A method for determining a scan pattern to be applied to transform coefficients of a current block in a video decoding process,

   Parsing a syntax element indicating a location of a last significant coefficient within the block of transform coefficients;

   Determining whether a position of the last significant coefficient is greater than a preset threshold value;

   Parsing a syntax element indicating a scan pattern for a one-dimensional array of the transform coefficients from a plurality of available scan patterns if the position of the last effective coefficient is greater than a predetermined threshold; And

   Determining a scan pattern set as a default among the plurality of available scan patterns as a scan pattern for the one-dimensional array of the transform coefficients if the position of the last effective coefficient is not greater than a preset threshold value;

   ≪ / RTI >
8. 8. The method of claim 7,

   The predetermined threshold value may be a predetermined threshold value,

   And a function of the magnitude of the transform coefficients.
9. 8. The method of claim 7,

   The scan pattern set to the default,

   Gt; is a diagonal scan pattern. ≪ / RTI >
10. An apparatus for determining a scan pattern to be applied to transform coefficients of a current block in an image decoding process,

    A memory for storing instructions; And

    Comprising one or more processors,

    Wherein the instructions, when performed by the one or more processors,

    Parsing a syntax element indicating a location of a last significant coefficient within the block of transform coefficients;

    Determining whether a position of the last significant coefficient is greater than a preset threshold value;

    Parsing a syntax element indicating a scan pattern for a one-dimensional array of the transform coefficients from a plurality of available scan patterns if the position of the last effective coefficient is greater than a predetermined threshold; And

    Determining a scan pattern set as a default among the plurality of available scan patterns as a scan pattern for the one-dimensional array of the transform coefficients if the position of the last effective coefficient is not greater than a preset threshold value;

    And to perform the processing.
11. 11. The method of claim 10,

    The predetermined threshold value may be a predetermined threshold value,

    Is a function of the magnitude of the transform coefficients.
12. 11. The method of claim 10,

    The scan pattern set to the default,

    Wherein the device is a diagonal scan pattern.

Images