KR102476541B1 - Method and apparatus of image encoding/decoding using reference pixel composition in intra prediction - Google Patents

Abstract

An image encoding and decoding method for performing intra-prediction is disclosed. Determining whether a boundary pixel in the neighboring block can be used as a reference pixel of the current block in consideration of a prediction mode of each of the current block and neighboring blocks, obtaining a reference pixel of the current block according to the determination result and generating a prediction block within a picture based on the obtained reference pixel, and encoding the current block using the generated prediction block. Therefore, prediction accuracy can be improved by improving the reference pixel configuration in intra-prediction. In addition, encoding efficiency of an image can be improved by increasing prediction accuracy in intra-prediction.

Description

Encoding/decoding method and apparatus for reference pixel configuration in intra-prediction

The present invention relates to image encoding and decoding technology, and more particularly, to an encoding/decoding method and apparatus for a reference pixel configuration in intra-prediction.

With the spread of the Internet and portable terminals and the development of information and communication technology, the use of multimedia data is rapidly increasing. Therefore, in order to perform various services or tasks through image prediction in various systems, the need for improving the performance and efficiency of image processing systems is significantly increasing, but research and development results that can respond to this atmosphere are insufficient.

As described above, in the video encoding/decoding method and apparatus of the related art, improvement in performance of video processing, particularly video encoding or video decoding, is required.

An object of the present invention to solve the above problems is to provide a method of encoding/decoding an image by improving a reference pixel configuration.

Another object of the present invention is to provide an apparatus for encoding/decoding an image by improving a reference pixel configuration.

In order to achieve the above object of the present invention, an image encoding method for performing intra-prediction according to an aspect of the present invention considers a prediction mode of each of a current block and a neighboring block, and selects a boundary pixel in a neighboring block as a current block. Determining whether a block can be used as a reference pixel, obtaining a reference pixel of the current block according to the determination result, generating a predicted prediction block within a picture based on the obtained reference pixel, and generating the prediction block It may include encoding the current block using .

Here, in the step of acquiring the reference pixel, if it is determined that the boundary pixel is unavailable as the reference pixel of the current block, a reference pixel configured with a preset pixel value may be obtained.

Here, the step of determining whether the block is usable as a reference pixel may indicate whether a preset flag is to consider the prediction mode of a neighboring block.

Here, in the step of determining whether the block can be used as a reference pixel, if the prediction mode of the neighboring block is intra-prediction, it may be determined that the boundary pixel can be used as a reference pixel of the current picture.

Here, in the step of determining whether the pixel can be used as a reference pixel, if the prediction mode of the neighboring block is inter-prediction, it may be determined that the boundary pixel is unavailable as a reference pixel of the current picture.

Here, the step of determining whether a border pixel can be used as a reference pixel is, when the prediction mode of the neighboring block is inter-prediction, considering the reference picture of the neighboring block, whether or not the boundary pixel can be used as a reference pixel of the current picture. can decide

Here, the reference picture is selected from list 0, which stores data for reference pictures preceding the current picture, or list 1, which stores data for reference pictures subsequent to the current picture, and the current picture is in list 0 or list 1. can be included

Here, when the reference picture of the neighboring block is the current picture, it can be determined that the boundary pixel can be used as a reference pixel of the current picture.

Here, if the reference picture of the neighboring block is not the current picture, it may be determined that the boundary pixel is unavailable as a reference pixel of the current picture.

Here, when the reference picture of the neighboring block is an I picture, a boundary pixel may be determined to be usable as a reference pixel of the current picture.

In order to achieve the above-described other object of the present invention, a video decoding method for performing intra-prediction according to another aspect of the present invention includes a bit stream including data about a current block and a prediction mode of a block adjacent to the current block. Receiving, extracting data from the received bitstream, and checking the prediction mode of the neighboring block, considering the prediction mode of the neighboring block, whether a boundary pixel in the neighboring block can be used as a reference pixel of the current block. Determining whether or not, obtaining a reference pixel of the current block according to the determined result, generating a predicted prediction block in the picture based on the obtained reference pixel, and decoding the current block using the generated prediction block It includes steps to

Here, in the step of acquiring the reference pixel, if it is determined that the boundary pixel is unavailable as the reference pixel of the current block, a reference pixel configured with a preset pixel value may be obtained.

Here, the step of determining whether a block can be used as a reference pixel may indicate whether a preset flag is to consider a prediction mode of a neighboring block.

Here, in the step of determining whether the block can be used as a reference pixel, if the prediction mode of the neighboring block is intra-prediction, it may be determined that the boundary pixel can be used as a reference pixel of the current picture.

Here, in the step of determining whether the pixel can be used as a reference pixel, if the prediction mode of the neighboring block is inter-prediction, it may be determined that the boundary pixel is unavailable as a reference pixel of the current picture.

Here, the step of determining whether a border pixel can be used as a reference pixel is, when the prediction mode of the neighboring block is inter-prediction, considering the reference picture of the neighboring block, whether or not the boundary pixel can be used as a reference pixel of the current picture. can decide

Here, the reference picture is selected from list 0, which stores data for reference pictures preceding the current picture, or list 1, which stores data for reference pictures subsequent to the current picture, and the current picture is in list 0 or list 1. can be included

Here, when the reference picture of the neighboring block is the current picture, it can be determined that the boundary pixel can be used as a reference pixel of the current picture.

Here, when the reference picture of the neighboring block is not the current picture, pixels in the neighboring block may be determined to be unavailable as reference pixels of the current picture.

Here, when the reference picture of the neighboring block is an I picture, pixels in the neighboring block may be determined to be usable as reference pixels of the current picture.

In an image decoding apparatus including one or more processors according to another aspect of the present invention for achieving the above-described other object of the present invention, the one or more processors transmit data about a current block and a prediction mode of a block adjacent to the current block. After receiving a bit stream containing a bit stream, extracting data from the received bit stream, checking the prediction mode of a neighboring block, considering the prediction mode of the neighboring block, a boundary pixel in the neighboring block is selected as a reference pixel of the current block. It is determined whether it is available as , according to the determined result, a reference pixel of the current block is obtained, a predicted prediction block within the picture is generated based on the obtained reference pixel, and the current block is decoded using the generated prediction block. can do.

In the case of using the video encoding/decoding method and apparatus according to the embodiment of the present invention as described above, prediction accuracy can be improved by improving the reference pixel configuration in intra prediction.

In addition, encoding efficiency of an image can be improved by increasing prediction accuracy in intra-prediction.

1 is a conceptual diagram of an image encoding and decoding system according to an embodiment of the present invention.
2 is a configuration diagram of an image encoding apparatus according to an embodiment of the present invention.
3 is a configuration diagram of a video decoding apparatus according to an embodiment of the present invention.
4 is an exemplary diagram illustrating inter prediction of P slices in an image encoding and decoding method according to an embodiment of the present invention.
5 is an exemplary diagram illustrating inter prediction of a B slice in an image encoding and decoding method according to an embodiment of the present invention.
6 is an exemplary diagram for explaining a case of generating a prediction block in one direction in an image encoding and decoding method according to an embodiment of the present invention.
7 is an exemplary view of constructing a reference picture list in an image encoding and decoding method according to an embodiment of the present invention.
8 is an exemplary diagram illustrating another example of performing inter prediction from a reference picture list in an image encoding and decoding method according to an embodiment of the present invention.
9 is an exemplary diagram for explaining intra-prediction in an image encoding method according to an embodiment of the present invention.
10 is an exemplary diagram for explaining a prediction principle in a P slice or a B slice in an image encoding method according to an embodiment of the present invention.
11 is an exemplary diagram for explaining a process of obtaining a prediction block.
12 is an exemplary diagram for explaining a main process of an image encoding method according to an embodiment of the present invention with syntax in a coding unit.
13 is an exemplary diagram for explaining an example of supporting symmetric type splitting or asymmetric type splitting as in inter prediction when a prediction block is generated through block matching in a current picture.
14 is an exemplary diagram for explaining that 2N×2N and N×N can be supported in inter prediction (Inter).
15 is an exemplary view for explaining an encoding mode of a neighboring block in intra prediction according to an embodiment of the present invention.
16A is an exemplary diagram for explaining whether to use a reference pixel in consideration of an encoding mode of a neighboring block in intra prediction according to an embodiment of the present invention.
16B is an exemplary diagram for explaining whether or not a reference pixel is used without considering the coding mode of a neighboring block in intra prediction according to an embodiment of the present invention.
17 is another exemplary diagram for explaining an encoding mode of a neighboring block in intra prediction according to an embodiment of the present invention.
18A is an exemplary view illustrating a case in which an encoding mode of a neighboring block is intra prediction in intra prediction according to an embodiment of the present invention.
18B is an exemplary view illustrating consideration of coding modes and reference pictures of neighboring blocks in intra prediction according to an embodiment of the present invention.
18C is another exemplary diagram for explaining a case in which encoding modes of neighboring blocks are not considered in intra prediction according to an embodiment of the present invention.
19A is an exemplary view illustrating a method of obtaining a reference pixel when an available neighboring block is a block in the lower left corner.
19B is an exemplary view illustrating a method of obtaining a reference pixel when an available neighboring block is a block in the upper right corner.
19C is an exemplary diagram explaining a method of obtaining a reference pixel when available neighboring blocks are blocks in lower left and upper right corners.
19D is an exemplary view illustrating a method of obtaining a reference pixel when available neighboring blocks are upper left and upper right blocks.
20 is a flowchart of an image encoding method according to an embodiment of the present invention.

Since the present invention can make various changes and have various embodiments, specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

Terms such as first, second, A, and B may be used to describe various components, but the components should not be limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention. The terms and/or include any combination of a plurality of related recited items or any of a plurality of related recited items.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle.

Terms used in this application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, the terms "include" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in this application, they should not be interpreted in an ideal or excessively formal meaning. don't

In general, a moving picture may be composed of a series of pictures, and each picture may be divided into predetermined areas such as blocks. In addition, the divided region may be referred to as a block, as well as various sizes or terms such as a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), and a transform unit (TU). Each unit may be composed of one luminance block and two chrominance blocks, which may be configured differently according to color formats. Also, the size of the luminance block and the chrominance block may be determined according to the color format. For example, in the case of 4:2:0, the size of the chrominance block may have a length equal to half the length and width of the luminance block. For this unit, existing terms such as HEVC or H.264/AVC may be referred to. In the present invention, block and the above terms are used interchangeably, but may be understood differently according to standard technologies, and should be understood as corresponding terms or units according to encoding and decoding processes according to such standard technologies.

In addition, a picture, block, or pixel referred to for encoding or decoding a current block or current pixel is referred to as a reference picture, reference block, or reference pixel. In addition, those of ordinary skill in the art to which this embodiment pertains that the term "picture" described below may be replaced with other terms having equivalent meanings such as image and frame. You can understand when you grow up

Hereinafter, with reference to the accompanying drawings, preferred embodiments of the present invention will be described in more detail. In order to facilitate overall understanding in the description of the present invention, the same reference numerals are used for the same components in the drawings, and redundant descriptions of the same components are omitted.

1 is a conceptual diagram of an image encoding and decoding system according to an embodiment of the present invention.

Referring to FIG. 1, an image encoding device 12 and a decoding device 11 include a personal computer (PC), a notebook computer, a personal digital assistant (PDA), and a portable multimedia player (PMP). Player), PlayStation Portable (PSP: PlayStation Portable), wireless communication terminal (Wireless Communication Terminal), smart phone (Smart Phone), TV, etc., or server terminals such as application servers and service servers, and various devices or A communication device such as a communication modem to communicate with a wired/wireless communication network, a memory (memory, 18) to store various programs and data for encoding or decoding images, or inter or intra prediction for encoding or decoding, and program execution It may include various devices having a processor (processor, 14) for calculation and control. In addition, the video encoded as a bitstream by the video encoding device 12 is transmitted through a wired or wireless communication network such as the Internet, a local area wireless communication network, a wireless LAN network, a WiBro network, or a mobile communication network in real time or non-real time, or through a cable or universal It can be transmitted to an image decoding device through various communication interfaces such as a serial bus (USB: Universal Serial Bus), decoded in the image decoding device, and restored and reproduced as an image.

Also, an image encoded into a bitstream by an image encoding device may be transmitted from an encoding device to a decoding device through a computer-readable recording medium.

2 is a block diagram of an image encoding apparatus according to an embodiment of the present invention. 3 is a block diagram of an image decoding apparatus according to an embodiment of the present invention.

As shown in FIG. 2 , the video encoding apparatus 20 according to the present embodiment includes a prediction unit 200, a subtraction unit 205, a transform unit 210, a quantization unit 215, and an inverse quantization unit 220. , an inverse transform unit 225, an adder 230, a filter unit 235, a decoded picture buffer 240, and an entropy encoding unit 245.

In addition, as shown in FIG. 3, the video decoding apparatus 30 according to this embodiment includes an entropy decoding unit 305, a prediction unit 310, an inverse quantization unit 315, an inverse transform unit 320, and an adder 325, a filter unit 330, and a decoded picture buffer 335.

The above-described image encoding device 20 and image decoding device 30 may be separate devices, but may be made into one image encoding and decoding device depending on implementation. In this case, the prediction unit 200, the inverse quantization unit 220, the inverse transform unit 225, the adder 230, the filter unit 235, and the memory 240 of the video encoding apparatus 20 generate the image in the order described. The prediction unit 310, the inverse quantization unit 315, the inverse transform unit 320, the adder 325, the filter unit 330, and the memory 335 of the decoding apparatus 30 are substantially the same technical elements, and at least the same structure, or at least implemented to perform the same function. In addition, the entropy encoder 245 may correspond to the entropy decoder 305 when performing its function in reverse. Therefore, in the detailed description of the following technical elements and their operating principles, redundant descriptions of the corresponding technical elements will be omitted.

Also, since the image decoding device 30 corresponds to a computing device that applies the image encoding method performed by the image encoding device 20 to decoding, the following description will focus on the image encoding device 20.

A computing device may include a memory for storing a program or software module implementing an image encoding method and/or an image decoding method, and a processor connected to the memory to execute the program. Also, the image encoding device may be referred to as an encoder, and the image decoding device may be referred to as a decoder, respectively.

Each component of the video encoding apparatus of the present embodiment will be described in detail as follows.

Here, the video encoding device 20 may further include a division unit. The division unit divides the input image into blocks (M×N) of a predetermined size. Here, M or N is an arbitrary natural number greater than or equal to 1. In detail, the division unit may include a picture division unit and a block division unit. The size or shape of the block may be determined according to the characteristics and resolution of the image, and the size or shape of the block supported through the picture segmentation unit is an M×N square shape in which the horizontal and vertical lengths are expressed as powers of 2 (256 × 256, 128 × 128, 64 × 64, 32 × 32, 16 × 16, 8 × 8, 4 × 4, etc.) or M × N rectangular shape. For example, an input image may be divided into sizes such as 256×256 for an 8k UHD video having a high resolution, 128×128 for a 1080p HD video, and 16×16 for a WVGA video.

Information on the size or shape of such blocks can be set in units such as sequences, pictures, and slices, and related information can be transmitted to a decoder. That is, it can be set in units of a sequence parameter set, a picture parameter set, a slice header, or a combination thereof.

Here, a sequence refers to a unit composed of several related scenes. And a picture is a term that refers to a series of luminance (Y) components or luminance + chrominance (Y, Cb, Cr) components in one scene or picture, and the range of one picture is one frame or one field, depending on the case. can be

A slice may refer to one independent slice segment and a plurality of subordinate slice segments existing in the same access unit. An access unit means a set of network abstraction layer (NAL) units related to one coded picture. A NAL unit is a syntax structure that configures a video compression bitstream in a network-friendly format in the H.264/AVC and HEVC standards. It is common to configure one slice unit as one NAL unit, and the system standard generally regards NAL or NAL sets constituting one frame as one access unit.

Returning to the description of the picture division unit, the information on the block size or shape (M×N) may consist of explicit flags, specifically block shape information, one length information when the block is square, and rectangular shape information. In this case, each length information or difference value information between the horizontal and vertical lengths may be included. For example, if M and N are composed of the powers of k (assuming that k is 2) (M=2 ^m , N=2 ⁿ ), information about m and n is unary binarized or truncated unary binarized. It is possible to transmit related information to a decoding device by encoding in various ways such as. Alternatively, the minimum size allowed for division (Minblksize) supported by the picture division unit is I×J (assuming I=J for convenience of description. In case I=2 ⁱ , J=2 ^j ), mi or nj information is transmitted. can As another example, when M and N are different, the difference between m and n (|mn|) may be transmitted. Alternatively, information on im or nj is transmitted if the maximum size (Maxblksize) supported by the picture segmentation unit is I×J (assuming that I=J for convenience of description. In case of I=2 ⁱ , J=2 ^j ), im or nj can

In the case of an implicit situation, for example, when a syntax for related information exists but cannot be confirmed by an encoder or a decoder, the encoder or decoder may follow a pre-prepared default setting. For example, if the related syntax cannot be checked in the step of checking the block shape information, the block shape can be set to the default square shape. Alternatively, in the step of checking the block size information, in detail, in the step of checking the block size information through the difference value from the minimum size allowed for division (Minblksize) as in the above example, syntax related to the difference value can be checked, but the minimum size allowed for division If the (Minblksize)-related syntax cannot be verified, it can be obtained from the default setting value related to the minimum size (Minblksize) prepared in advance.

In this way, the size or shape of a block in the picture segmentation unit may be implicitly determined according to the characteristics and resolution of an image or the related information may be explicitly transmitted by an encoder or a decoder.

Blocks divided and determined through the picture division unit as described above may be used as a basic coding unit. In addition, a block divided and determined through a picture division unit may be a minimum unit constituting a high-level unit such as a picture, slice, tile, or the like, and may be a coding block, a prediction block, or a transform block. ), quantization block, entropy block, and inloop filter (inloopfiltering block), but some blocks are not limited thereto and exceptions are possible.

For example, some of them, such as in-loop filtering blocks, may be applied in units larger than the block size described above.

The block divider performs division on blocks such as encoding, prediction, transformation, quantization, entropy, and in-loop filters. The division part is included in each component and performs a function. For example, a transform block division unit in the transform unit 210 and a quantization block division unit in the quantization unit 215 may be included. The size or shape of the initial block of the block divider may be determined by a result of division of a previous step or higher level block. For example, in the case of a coding block, a block acquired through a picture segmentation unit, which is a previous step, may be set as an initial block. Alternatively, in the case of a prediction block, a block obtained through a division process of a coding block, which is a higher level of the prediction block, may be set as an initial block. Alternatively, in the case of a transform block, a block obtained through a division process of a coding block, which is a higher level of a transform block, may be set as an initial block. Conditions for determining the size or shape of the initial block are not always fixed, and some may be changed or there may be exceptions. In addition, the division state of the previous level or higher level block (eg, size of coding block, type of coding block, etc.) and setting conditions of the current level (eg, size of supported transform block, type of transform block, etc.) ) may affect the division operation of the current level (eg whether division is possible or not, block type that can be divided, etc.) according to a combination of at least one or more factors.

The block partitioning unit may support a quad tree-based partitioning scheme. That is, it can be divided into 4 blocks each having a horizontal and vertical length of 1/2 of the previous block. This is based on the first block (dep_0), and is the limit of the allowable division depth (dep_k, k means the number of allowable divisions, and the block size at the allowable division depth limit (dep_k) is (M >> k, N >> k)) The division can be repeated up to

In addition, a binary tree-based partitioning method may be supported. This indicates that it can be divided into two blocks each having a horizontal or vertical length equal to 1/2 of the length of the block before division. In the case of the quad tree partitioning and the binary tree partitioning, symmetric partitioning or asymmetric partitioning may be used, and which partitioning method to follow may be determined according to the setting of the encoder or decoder. In the video encoding method of the present invention, the description will be focused on the symmetric division method.

Whether or not each block is divided can be indicated through a division flag (div_flag). If the corresponding value is 1, division is performed, and if the value is 0, division is not performed. Alternatively, if the corresponding value is 1, division is performed and additional division is possible, and if the value is 0, division is not performed and further division is not allowed. Depending on conditions such as a minimum size allowed for division and a limit on depth allowed for division, the flag may be considered only for whether to divide or not, and whether or not to additionally divide.

Splitting flags can be used in quadtree partitioning, and can also be used in binary tree partitioning. In binary tree splitting, the splitting direction can be one of the block's splitting depth, encoding mode, prediction mode, size, shape, type (encoding, prediction, transformation, quantization, entropy, in-loop filter, etc.) or luminance or chrominance. may be one), and may be determined according to at least one factor or a combination thereof, such as a slice type, an allowable division depth limit, and minimum and maximum sizes allowed for division.

In this case, according to the division flag, according to the corresponding division direction, that is, only the block may be divided in half horizontally or only vertically. For example, assuming that a block supports horizontal partitioning as M×N (M>N), where M is greater than N, and that the current partitioning depth (dep_curr) is smaller than the partitioning depth limit, further partitioning is possible. The division flag is allocated as 1 bit, and if the corresponding value is 1, horizontal division is performed, and if the value is 0, no further division may be performed. As for the splitting depth, one splitting depth may be provided for the quad tree and binary tree splits, or each split depth may be provided for the quad tree and binary tree splits. In addition, as for the splitting allowable depth limit, one splitting allowable depth limit may be placed on quad tree and binary tree splitting, or splitting allowable depth limits may be set on each of the quad tree and binary tree splitting.

As another example, if the block is M×N (M>N) and N is equal to the preset minimum allowable splitting size and does not support horizontal splitting, the above splitting flag is assigned as 1 bit, and if the value is 1, vertical splitting is performed, , if it is 0, no division is performed.

In addition, flags (div_h_flag and div_v_flag) for horizontal division or vertical division may be supported, respectively, and binary division may be supported according to the flags. Horizontal or vertical division of each block can be indicated through the horizontal division flag (div_h_flag) or vertical division flag (div_v_flag). If the horizontal division flag (div_h_flag) or vertical division flag (div_v_flag) is 1, horizontal or vertical division is performed. If set to 0, horizontal or vertical division is not performed. Alternatively, if each flag is 1, horizontal or vertical division is performed and additional horizontal or vertical division is possible. If the value is 0, horizontal or vertical division is not performed and additional horizontal or vertical division is not allowed. . Depending on conditions such as the minimum size allowed for division and the limit on the depth allowed for division, the flag may consider whether to divide or not and may not consider whether to additionally divide. Alternatively, a flag (div_flag/h_v_flag) for horizontal division or vertical division may be supported, and binary division may be supported according to the flag. The division flag (div_flag) may indicate horizontal or vertical division, and the division direction flag (h_v_flag) may indicate a horizontal or vertical division direction. If the division flag (div_flag) is 1, division is performed and horizontal or vertical division is performed according to the division direction flag (h_v_flag). If it is 0, horizontal or vertical division is not performed. Alternatively, if the value is 1, horizontal or vertical division is performed according to the division direction flag (h_v_flag) and additional horizontal or vertical division is possible. If the value is 0, no horizontal or vertical division is performed and further horizontal or vertical division is possible. can be considered disallowed. Depending on conditions such as the minimum size allowed for division and the limit on the depth allowed for division, the flag may consider whether to divide or not and may not consider whether to additionally divide.

Horizontal and vertical splitting may also be supported, and binary tree splitting may be supported according to the flag. In addition, when the division direction is predetermined, only one of the two division flags or both division flags may be used as in the above example.

For example, if all of the above flags are allowed, the possible block types can be divided into one of MxN, M/2xN, MxN/2, and M/2xN/2. In this case, the flags may be coded as 00, 01, 10, and 11 (in order of div_h_flag/div_v_flag). The above case is an example of a setting in which the split flags can be overlapped and used, and a setting in which the split flags cannot be overlapped and used is also possible. For example, the split block form can be divided into M×N, M/2×N, and M×N/2, in which case the above flags are coded as 00, 01, 10 (in order of div_h_flag/div_v_flag), or , (in order of div_flag/h_v_flag. h_v_flag is a flag representing horizontal or vertical division direction) and can be coded as 0, 10, or 11. Here, the meaning of overlapping may mean performing horizontal and vertical division at the same time. Either of the aforementioned quad tree partitioning and binary tree partitioning may be used alone or in combination according to the setting of an encoder or a decoder. For example, quad tree or binary tree partitioning may be determined according to a block size or shape. That is, when the block type is M×N and M is greater than N, binary tree splitting can be supported according to horizontal splitting, and when the block type is M×N and N is larger than M, vertical splitting can be supported. is M×N, and if N and M are the same, quadtree partitioning may be supported.

As another example, if the size of the block (M×M) is greater than or equal to the block partitioning boundary value Thrblksize, binary tree partitioning may be supported, and if smaller than that, quadtree partitioning may be supported.

* As another example, if M or N of the block (M×N) is smaller than or equal to the first splitting maximum size (Maxblksize1) and greater than or equal to the first splitting minimum size (Minblksize1), quad tree splitting is supported, Binary tree splitting may be supported when M or N of blocks (M×N) is smaller than or equal to the maximum size allowed for second splitting (Maxblksize2) and larger than or equal to the minimum size allowed for second splitting (Minblksize2). If the first division support range and the second division support range, which can be defined by the maximum size allowed for division and the minimum size allowed for division, overlap, the first or second division method takes precedence according to the setting of the sub or decoder. A ranking may be given. In this example, the first division method is quad tree division, and the second division method is binary tree division. For example, if the minimum size allowed for the first division (Minblksize1) is 16, the maximum size allowed for the second division (Maxblksize2) is 64, and the block before division is 64×64, the first division support range and the second division support range Since they all belong, quad tree splitting and binary tree splitting are possible. If priority is given to the first division method (quad tree division in this example) according to a preset setting, if the division flag (div_flag) is 1, quad tree division is performed and additional quad tree division is possible. does not perform quad-tree splitting and can be considered as not performing quad-tree splitting anymore. Depending on conditions such as a minimum size allowed for division and a limit on depth allowed for division, the flag may be considered only for whether to divide or not, and whether or not to additionally divide. If the division flag (div_flag) is 1, it is divided into 4 blocks each having a size of 32×32 and is larger than the first minimum size allowed for division (Minblksize1), so quad tree division can be continued. If it is 0, additional quad tree splitting is not performed, and binary tree splitting can be performed because the current block size (64x64) falls within the second splitting support range. When the division flags (in order of div_flag/h_v_flag) are 0, no further division is performed, and when they are 10 or 11, horizontal division or vertical division may be performed. If the block before splitting is 32×32 and the splitting flag (div_flag) is 0, quad tree splitting is not performed and the maximum size allowed for second splitting (Maxblksize2) is 16, the size of the current block (32×32) is Since it does not fall within the scope of 2 division support, further division may not be supported. In the above description, the priority of the segmentation method may be determined according to at least one factor or a combination thereof among slice types, encoding modes, luminance and color difference components, and the like.

As another example, various settings may be supported according to luminance and color difference components. For example, the quad tree or binary tree partitioning structure determined in the luminance component can be used as it is without additional information or decoding in the chrominance component. Alternatively, when independent division of the luminance component and the chrominance component is supported, quad tree+binary tree division may be supported for the luminance component and quad tree division may be supported for the chrominance component. Alternatively, quad tree + binary tree division is supported in the luminance and chrominance components, but the division support range may be equal to or proportional to the luminance and chrominance components, or not. For example, when the color format is 4:2:0, the division support range of the chrominance component may be N/2 of the division support range of the luminance component.

As another example, different settings may be set according to slice types. For example, quad tree partitioning can be supported in I slice, binary tree partitioning can be supported in P slice, and quad tree + binary tree partitioning can be supported in B slice.

As in the above example, quad tree splitting and binary tree splitting can be set and supported according to various conditions. The above examples are not specific to the above-mentioned cases, and may include cases where conditions of each other are reversed, and may include cases where one or more factors mentioned in the above examples or a combination thereof are also included, and variations are also possible as examples of other cases. do. The above limit on the permissible depth of division may be determined according to at least one factor or a combination thereof, such as a division method (quad tree, binary tree), slice type, luminance and color difference components, and an encoding mode. In addition, the division support range may be determined according to at least one factor or a combination thereof in a division method (quad tree, binary tree), slice type, luminance and chrominance components, encoding mode, etc., and the related information is the division support range It can be expressed as a maximum value and a minimum value. When this information is configured as an explicit flag, length information of each of the maximum and minimum values or difference information between the minimum and maximum values may be expressed. For example, when the maximum value and the minimum value are composed of the power of k (assuming that k is 2), the exponential information of the maximum and minimum values may be encoded through various binarization and transmitted to a decoding device. Alternatively, the difference between the exponents of the maximum and minimum values can be transmitted. At this time, the transmitted information may be index information of the minimum value and difference value information of the index.

According to the above description, flag-related information may be generated and transmitted in units such as sequences, pictures, slices, tiles, and blocks.

With the split flags presented as examples above, block split information can be represented through a combination of quad tree, binary tree, or two tree methods, and the split flag is encoded using various methods such as unary binarization and truncated unary binarization to decode related information. can be passed to the device. A bitstream structure of a partition flag for expressing partition information of the block may be selected from one or more scan methods. For example, a bitstream of split flags may be configured based on a split depth order (from dep0 to dep_k), and a bitstream of split flags may be configured based on whether or not to split. In the split depth order criterion method, split information at the current level depth based on the first block is acquired and then split information at the next level depth is obtained. This means a method of preferentially obtaining additional segmentation information, and in addition to this, other scan methods not presented in the above example may be included and selected.

Also, depending on the implementation, the block division unit may generate and express index information for a block candidate group of a predefined form, rather than the above-described division flag. The shape of the block candidate group is, for example, the shape of a split block that can be had in a block before splitting, M×N, M/2×N, M×N/2, M/4×N, 3M/4×N, M×N/4, M×3N/4, M/2×N/2, and the like. As described above, when a candidate group of split blocks is determined, index information for the split block shape may be encoded through various methods such as fixed-length binarization, truncated-type binarization, and truncated-type binarization. As with the splitting flags described above, at least one or a combination of factors such as block splitting depth, encoding mode, prediction mode, size, shape, type, and slice type, splitting depth limit, splitting minimum and maximum size, etc. Accordingly, a partition block candidate group may be determined. For the following description, (M×N, M/2×N/2) is candidate list1, (M×N, M/2×N, M×N/2, M/2×N/2) is candidate list2, (M×N, M/2×N, M×N/2) as candidate list3, (M×N, M/2×N, M×N/2, M/4×N, 3M/4×N, It is assumed that M×N/4, M×3N/4, and M/2×N/2) are candidate list4. For example, when describing based on M×N, when (M=N), a split block candidate of candidate list2 may be supported, and when (M≠N), a split block candidate of candidate list3 may be supported.

As another example, if M or N of M×N is greater than or equal to the boundary value (blk_th), the split block candidate of candidate list2 may be supported, and if smaller than that, the split block candidate of candidate list4 may be supported. Alternatively, when M or N is greater than or equal to the first boundary value (blk_th_1), the split block candidate of candidate list1 is selected, and is smaller than the first boundary value (blk_th_1) but greater than or equal to the second boundary value (blk_th_2). When the split block candidate of list2 is smaller than the second boundary value (blk_th_2), the split block candidate of candidate list4 may be supported.

As another example, when the encoding mode is intra prediction, the split block candidate of candidate list2 may be supported, and when the encoding mode is inter prediction, the split block candidate of candidate list4 may be supported.

Even if the split block candidate as described above is supported, the bit configuration according to binarization in each block may be the same or different. For example, if supported split block candidates are limited according to the block size or shape, as in the above split flag application, the bit configuration according to binarization of the corresponding block candidate may be changed. For example, in the case of (M>N), block types according to horizontal partitioning, that is, M × N, M × N / 2, and M / 2 × N / 2 can be supported, and the partition block candidates (M × N, M / Binary bits of the index according to M×N/2 in 2×N, M×N/2, and M/2×N/2) and M×N/2 of the current condition may be different. According to the type of block used for encoding, prediction, transformation, quantization, entropy, in-loop filtering, etc., information about the partition and shape of the block may be expressed using either a split flag or a split index method. In addition, the block size limit and the allowable depth limit for splitting and supporting the shape of the block may be different according to each block type.

In the block-by-block encoding or decoding process, after a coding block is determined, encoding or decoding may be performed according to processes such as a prediction block determination, a transform block determination, a quantization block determination, an entropy block determination, and an in-loop filter determination. The order of the encoding or decoding process is not always fixed, and some order may be changed or excluded. The size and shape of each block may be determined according to the encoding cost for each candidate of the size and shape of the block, and segmentation related information such as image data of each determined block and size and shape of each determined block may be encoded.

The prediction unit 200 may be implemented using a prediction module, which is a software module, and may generate a prediction block using an intra-prediction method or an inter-prediction method with respect to a block to be encoded. Here, the prediction block is a block that is understood to closely match the block to be encoded in terms of pixel differences, and can be determined by various methods including Sum of Absolute Difference (SAD) and Sum of Square Difference (SSD). In addition, at this time, various syntaxes that can be used when decoding image blocks may be generated. A prediction block may be classified into an intra-picture block and an inter-picture block according to an encoding mode.

Intra prediction is a prediction technique using spatial correlation, and refers to a method of predicting a current block using reference pixels of previously encoded, decoded, and reconstructed blocks within a current picture. That is, in the intra-prediction, brightness values reconstructed through prediction and restoration may be used as reference pixels in the encoder and decoder. Intra-picture prediction can be effective for a flat region with continuity and a region with a constant direction, and can be used for the purpose of guaranteeing random access and preventing error diffusion because it uses spatial correlation.

Inter prediction uses a compression technique that removes data redundancy by using temporal correlation by referring to images encoded in one or more past or future pictures. That is, inter prediction can generate a prediction signal having high similarity by referring to one or more past or future pictures. An encoder using inter-prediction may search for a block having a high correlation with a block to be currently encoded in a reference picture, transmit location information and a residual signal of the selected block to a decoder, and the decoder may transmit selection information of the transmitted image. A reconstructed image may be constructed by generating the same prediction block as the encoder and compensating for the transmitted residual signal using .

4 is an exemplary diagram illustrating inter prediction of P slices in an image encoding and decoding method according to an embodiment of the present invention. 5 is an exemplary diagram illustrating inter prediction of a B slice in an image encoding and decoding method according to an embodiment of the present invention.

In the video encoding method of the present embodiment, since inter-prediction generates prediction blocks from previously encoded pictures having high temporal correlation, encoding efficiency can be increased. Current(t) may mean the current picture to be encoded, and the first temporal distance (t -1) and a second reference picture (t-2) having a second temporal distance (t-2) prior to the first temporal distance.

That is, as shown in FIG. 4, inter-prediction that can be employed in the video encoding method of this embodiment is based on the current block of the current picture (current(t)) and the reference pictures (t-1, t-2). Through block matching of reference blocks, motion estimation may be performed to find an optimal prediction block from reference pictures (t-1, t-2) in which a block having a high correlation has previously been coded. After performing an interpolation process based on a structure in which at least one or more sub-pixels are arranged between two adjacent pixels as necessary for precise estimation, motion compensation is performed after finding an optimal prediction block to obtain the final prediction block can be found.

In addition, as shown in FIG. 5, inter-prediction that can be employed in the video encoding method of the present embodiment involves reference pictures (which have already been coded) temporally in both directions based on the current picture (current(t)). A prediction block can be generated from t-1, t+1). Also, two prediction blocks can be generated from one or more reference pictures.

When video encoding is performed through inter-prediction, motion vector information for an optimal prediction block and information for a reference picture are encoded. In this embodiment, when a prediction block is generated in a unidirectional or bidirectional manner, a reference picture list may be configured differently and a prediction block may be generated from the corresponding reference picture list. Basically, a reference picture existing before the current picture in time is assigned to list 0, and a reference picture existing after the current picture is assigned to list 1 and managed. When constructing the reference picture list 0, if the allowed number of reference pictures in the reference picture list 0 is not filled, a reference picture existing after the current picture may be allocated. Similarly, when constructing the reference picture list 1, if the number of reference pictures in the reference picture list 1 cannot be filled up to the allowed number, a reference picture existing before the current picture can be allocated.

6 is an exemplary diagram for explaining a case of generating a prediction block in one direction in an image encoding and decoding method according to an embodiment of the present invention.

Referring to FIG. 6, in the video encoding and decoding method according to the present embodiment, prediction blocks can be found from previously encoded reference pictures (t-1, t-2) as in the past, and in addition to this, a current picture (current ( In t)), a prediction block may be found from an area where encoding has already been completed.

That is, in the video encoding and decoding method according to the present embodiment, prediction blocks not only generated from previously encoded pictures (t-1, t-2) having high temporal correlation, but also spatially highly correlated prediction blocks It can be implemented to find together. Finding such a spatially correlated prediction block may correspond to finding the prediction block by way of intra-prediction. In order to perform block matching from an encoded region in the current picture, the video encoding method of the present embodiment may be mixed with an intra prediction mode to configure syntax for information related to prediction candidates.

For example, if n (n is an arbitrary natural number) intra-prediction modes are supported, one mode is added to the intra-prediction candidate group to support n+1 modes, and 2 ^M-1 ≤ n+1 A prediction mode can be encoded using M fixed bits satisfying <2 ^M. In addition, it may be implemented to select from a candidate group of highly probable prediction modes, such as the most probable mode (MPM) of high efficiency video coding (HEVC). In addition, encoding may be preferentially performed in a higher stage of prediction mode encoding.

When a prediction block is generated through block matching in the current picture, the video encoding method of the present embodiment may be mixed with an inter prediction mode to configure syntax for related information. As additional related prediction mode information, motion or displacement related information may be used. Motion or motion-related information may include information about an optimal candidate among several vector candidates, a difference value between an optimal candidate vector and a real vector, a reference direction, reference picture information, and the like.

7 is an exemplary view of constructing a reference picture list in an image encoding and decoding method according to an embodiment of the present invention. 8 is an exemplary diagram illustrating another example of performing inter prediction from a reference picture list in an image encoding and decoding method according to an embodiment of the present invention.

Referring to FIG. 7 , in the video encoding method according to the present embodiment, inter-prediction may be performed on a current block of a current picture (current(t)) from reference picture lists (reference list 0 and reference list 1), respectively. there is.

Referring to FIGS. 7 and 8 , the reference picture list 0 can be composed of reference pictures prior to the current picture t, where t-1 and t-2 are picture order counts of the current picture t, respectively. Indicates reference pictures having a first temporal distance (t-1) and a second temporal distance (t-2) prior to Count, POC. In addition, the reference picture list 1 can be composed of reference pictures subsequent to the current picture (t), where t+1 and t+2 are the first temporal distance (t+1) subsequent to the POC of the current picture (t), respectively. ), indicating reference pictures having a second temporal distance (t+2).

The above examples of constructing a reference picture list show an example of constructing a reference picture list with reference pictures having a difference in temporal distance (based on POC in this example) of 1, but constructing a different temporal distance difference between reference pictures. You may. That is, it means that the index difference between reference pictures and the temporal distance difference between reference pictures may not be proportional. In addition, the list construction order may not be configured based on temporal distance. Information on this can be confirmed in an example of a reference picture list configuration to be described later.

Prediction can be performed from a reference picture in the list according to the slice type (I, P or B). In addition, when a prediction block is generated from the current picture (current(t)) through block matching, the current picture can be added to the reference picture list (reference list 0, reference list 1) to perform encoding in an inter-prediction method. .

As shown in FIG. 8 , the current picture t can be added to reference picture list 0 or the current picture t can be added to reference picture list 1. That is, reference picture list 0 can be constructed by adding a reference picture that is a temporal distance (t) to a reference picture before the current picture (t), and reference picture list 1 is formed by adding a reference picture that is temporally distant (t) to a reference picture after the current picture (t). It can also be constructed by adding a reference picture with a constant distance (t).

For example, when constructing reference picture list 0, a reference picture prior to the current picture may be assigned to reference picture list 0, and then the current picture (t) may be assigned. A reference picture can be assigned to reference picture list 1, and then the current picture (t) can be assigned. Alternatively, when constructing the reference picture list 0, the current picture (t) may be allocated and then a reference picture prior to the current picture may be allocated. When constructing the reference picture list 1, the current picture (t) may be allocated and then the current picture Subsequent reference pictures can be assigned. Alternatively, when constructing the reference picture list 0, a reference picture prior to the current picture may be allocated, a reference picture subsequent to the current picture may be allocated, and the current picture (t) may be allocated. Similarly, when constructing the reference picture list 1, a reference picture subsequent to the current picture may be allocated, a reference picture prior to the current picture may be allocated, and the current picture (t) may be allocated. The above examples are not specific to the above-described case, and may include cases in which mutual conditions are reversed, and modifications are also possible as examples of other cases.

Whether to include the current picture in each reference picture list (e.g., not added to any list, added only to list 0, added only to list 1, or added to lists 0 and 1 together) requires the same setting in the encoder or decoder. It is possible, and information about this can be transmitted in units such as sequences, pictures, and slices. This information may be encoded through a method such as fixed length binarization, truncated binarization, or truncated binarization.

Unlike the method of FIG. 7, the video encoding and decoding method of this embodiment selects a prediction block by performing block matching on the current picture t, constructs a reference picture list including information related to the prediction block, and , there is a difference in using this reference picture list for video encoding and decoding.

In constructing a reference picture list, the order and rules for constructing each list and the allowable number of reference pictures in each list can be set differently. whether or not), slice type, list reconstruction parameter (may be applied to lists 0 and 1 respectively, or may be applied to lists 0 and 1 together), location in a group of pictures (GOP), temporal layer information (temporal id) etc. may be determined according to at least one or a combination of factors, and related information may be explicitly transmitted in units of sequences, pictures, and the like. For example, in the case of a P slice, the reference picture list 0 may follow list construction rule A regardless of whether the current picture is included in the list, and in the case of a B slice, the reference picture list 0 including the current picture in the list has a list construction Rule B, list construction rule C may be followed for reference picture list 1, list construction rule D may be followed for reference picture list 0 that does not include the current picture, list construction rule E may be followed for reference picture list 1, and list construction rules Among them, B and D, C and E may be the same. List construction rules may be constructed in the same way as described in the above reference picture list construction example or in a modified manner. As another example, when the current picture is included in the list, the allowed number of first reference pictures may be set, and when the current picture is not included in the list, the allowed number of second reference pictures may be set. The allowed number of first reference pictures and the allowed number of second reference pictures may be the same or different, and the default setting may be that the difference between the allowed number of first reference pictures and the allowed number of second reference pictures is 1. As another example, when the current picture is included in the list and the list reconstruction parameter is applied, all reference pictures may be list reconstruction candidates in slice A, and only some reference pictures may be included in the list reconstruction candidate group in slice B. At this time, slice A or B can be classified as whether it is included in the list of the current picture, temporal layer information, slice type, location in a picture set (Group of Pictures, GOP), etc. It may be determined by a picture order count (POC) or reference picture index, a reference prediction direction (before and after the current picture), whether or not the current picture is present, and the like.

According to the configuration described above, since a reference block coded by inter-prediction in the current picture can be used, inter-prediction can be allowed or used even in I-slice motion prediction.

In addition, when constructing a reference picture list, the order of index allocation or list construction may be different according to the slice type. In the case of an I slice, fewer indices (eg, idx = 0, 1, 2) are used in the current picture (current (t)) by increasing the priority as in the above reference picture list configuration example, and corresponding The amount of bits in video encoding can be reduced through binarization (fixed-length binarization, truncation-type binarization, truncated-type binarization, etc.) in which the allowable number (C) of reference pictures in the reference picture list is set to the maximum value. In addition, in the case of a P or B slice, if it is determined that the probability of selecting a reference picture of the current block as a prediction candidate by performing block matching on the current picture is lower than the probability of selecting a prediction candidate through other reference pictures, the current picture Binarization of various methods that sets the priority for block matching of to the maximum value by using a higher index (eg, idx = C, C-1) to maximize the number of allowed reference pictures in the reference picture list It is possible to reduce the amount of bits in video encoding through In the above example, the priority setting of the current picture may be configured in the same way as described in the reference picture list construction example or in a modified manner. In addition, it is possible to omit information on a reference picture by not constructing a reference picture list according to a slice type (eg, I slice). For example, a prediction block may be generated through existing inter prediction, but inter prediction information may be expressed with the rest excluding reference picture information from motion information in the inter prediction mode.

The method of performing block matching in the current picture may determine whether to support it according to the slice type. For example, block matching in the current block may be set to be supported in I slice but not supported in P slice or B slice, and other modifications are also possible. In addition, the method for supporting block matching in the current picture may be determined in units of pictures, slices, tiles, etc., or may be determined according to positions in a group of pictures (GOP), temporal layer information (temporal ID), and the like. there is. Such setting information may be transmitted in units of sequences, pictures, slices, or the like during an image encoding process or from an encoder to a decoder. In addition, when setting information or syntax related to the above exists in a higher level unit and the same setting information or syntax as above exists in a lower level unit even in a situation where a setting related operation is turned on, the setting information in a lower level unit is set at a higher level. Priority can be given to setting information in units. For example, if the same or similar setting information is processed in sequences, pictures, and slices, a picture unit may have priority over a sequence unit, and a slice unit may have priority over a picture unit.

9 is an exemplary diagram for explaining intra-prediction in an image encoding method according to an embodiment of the present invention.

Referring to FIG. 9 , the intra prediction method according to the present embodiment includes reference sample padding, reference sample filtering, intra prediction, and boundary filtering. It may include a series of steps.

The reference pixel filling step may be an example of the reference pixel construction step, the reference pixel filtering step may be referred to as a reference pixel filtering unit, and intra-prediction may include a prediction block generating step and a prediction mode encoding step, and boundary filtering. may be an example of an embodiment of a post-processing filter step.

That is, intra-prediction executed by the video encoding apparatus of the present embodiment may include a reference pixel configuration step, a reference pixel filtering step, a prediction block generation step, a prediction mode encoding step, and a post-processing filtering step. Depending on various environmental factors, such as block size, block shape, block position, prediction mode, prediction method, quantization parameter, etc., one or part of the above-described processes may be omitted or other processes may be added, as described above. It can be changed in a different order than the order.

The above-described reference pixel configuration step, reference pixel filtering step, prediction block generation step, prediction mode encoding step, and post-processing filtering step may be implemented in a form in which a processor connected to the memory executes software modules stored in a memory. Therefore, in the following description, for convenience of explanation, a functional unit generated by a combination of a software module implementing each step and a processor executing the same or a component performing the function of the functional unit is a reference pixel configuration unit, a reference pixel filter unit, A prediction block generation unit, a prediction mode encoding unit, and a post-processing filter unit will be referred to as execution subjects of each stage.

Describing each component in more detail, the reference pixel constructing unit configures a reference pixel to be used for prediction of the current block through reference pixel filling. If the reference pixel does not exist or is unavailable, the reference pixel filling may be used for the reference pixel by copying a value from an available nearby pixel. For value copying, a decoded picture buffer (DPB) may be used.

That is, intra-prediction is performed using reference pixels of previously coded blocks of the current picture. To this end, in the reference pixel configuration step, neighboring blocks of the current block, that is, neighboring pixels such as left, upper left, lower left, upper, and upper right blocks are mainly used as reference pixels. The candidate group of neighboring blocks for the reference pixel is only an example when the encoding order of blocks follows raster scan or z-scan, and scans such as inverse z-scan In the case of using the encoding order scan method, adjacent pixels such as the right block, the lower right block, and the lower block may be used as reference pixels in addition to the blocks above.

Also, depending on the implementation, additional pixels other than immediately adjacent pixels may be used as replacements or mixed with existing reference pixels according to the step-by-step configuration of intra-prediction.

In addition, when prediction is performed in a directional mode among intra-prediction modes, reference pixels in fractional units may be generated through linear interpolation of reference pixels in integer units. Modes that perform prediction through reference pixels existing at integer unit positions include some modes having vertical, horizontal, 45 degrees, and 135 degrees, and for the above prediction modes, a process of generating reference pixels in decimal units is required. may not For prediction modes with other directions other than the prediction mode, interpolated reference pixels are interpolated by powers of 1/2, such as 1/2, 1/4, 1/8, 1/16, 1/32, and 1/64. It may have precision, or it may have precision of a multiple of 1/2. This is because interpolation accuracy may be determined according to the number of supported prediction modes or the prediction direction of the prediction mode. A fixed interpolation precision may always be supported in a picture, slice, tile, block, etc., or adaptive interpolation precision may be supported according to the size of a block, the shape of a block, the prediction direction of a supported mode, and the like. At this time, the prediction direction of the mode may be expressed as tilt information or angle information of a direction indicated by the mode with a specific line reference (eg, the x-axis of positive <+> on the coordinate plane).

As an interpolation method, linear interpolation is performed through immediately adjacent integer pixels, but other interpolation methods may be supported. For interpolation, one or more types of filters and the number of taps, for example, a 6-tap Wiener filter and an 8-tap Kalman filter can be supported, and the type of interpolation to be performed can be determined according to the block size and prediction direction. there is. In addition, related information may be transmitted in units of sequences, pictures, slices, blocks, and the like.

The reference pixel filter unit may perform filtering on the reference pixel for the purpose of increasing prediction efficiency by reducing deterioration remaining in an encoding process after constructing the reference pixel. The reference pixel filter unit may implicitly or explicitly determine the type of filter and whether to apply filtering according to the size, shape, and prediction mode of a block. That is, even filters of the same tap may differently determine filter coefficients according to filter types. For example, a 3-tap filter such as [1,2,1]/4 or [1,6,1]/8 can be used.

Also, the reference pixel filter unit may determine whether or not to transmit additional bits to determine whether to apply filtering. For example, in the implicit case, the reference pixel filter unit may determine whether or not to apply filtering according to characteristics (variance, standard deviation, etc.) of pixels in neighboring reference blocks.

In addition, the reference pixel filter unit may determine whether to apply filtering when a related flag satisfies a predetermined hiding condition, such as a residual coefficient or an intra-prediction mode. The number of taps of the filter is, for example, 3-tap such as [1,2,1]/4 for small blocks (blk), [2,3,6,3,2]/16 and [2,3,6,3,2]/16 for large blocks (blk). It can be set to the same 5-tap, and the number of times of application can be determined by not performing filtering, filtering once, or filtering twice.

Also, the reference pixel filter unit may basically apply filtering to the nearest reference pixel of the current block. In addition to the nearest reference pixel, additional reference pixels may also be considered in the filtering process. For example, filtering may be applied to additional reference pixels by replacing the nearest reference pixel, or filtering may be applied by mixing additional reference pixels to nearest reference pixels.

The filtering may be applied statically or adaptively, which includes the size of the current block or the size of the neighboring block, the coding mode of the current block or the neighboring block, the block boundary characteristics of the current block and the neighboring block (eg, the boundary of the coding unit). It may be determined according to at least one factor or a combination thereof among factors such as boundary of a perceptual transformation unit, etc.), a prediction mode or direction of a current block or a neighboring block, a prediction method of the current block or a neighboring block, a quantization parameter, and the like. . This decision may have the same settings in the encoder or decoder (implicit), or may be determined in consideration of encoding cost (explicit). The filter that is basically applied is a low pass filter, and the number of filter taps, filter coefficients, whether or not to code the filter flag, and the number of filter applications can be determined according to the various factors specified above. It can be set in units such as slices and blocks, and related information can be transmitted to the decoder.

The prediction block generation unit uses an extrapolation or extrapolation method through reference pixels in prediction within a screen, an interpolation method such as the average value (DC) of reference pixels or a planar mode, or a copy of reference pixels. ) method to generate a prediction block. In the case of copying a reference pixel, one reference pixel may be copied to generate one or more prediction pixels, or one or more reference pixels may be copied to generate one or more prediction pixels, and the number of copied reference pixels is the number of copied reference pixels. It may be equal to or less than the number of predicted pixels.

In addition, according to the prediction method, it can be classified into a directional prediction method and a non-directional prediction method, and in detail, the directional prediction method can be classified into a linear directional method and a curved directional method. The linear directivity method borrows an extrapolation or extrapolation method, but the pixels of the prediction block are generated through reference pixels lying on the prediction direction line, and the curve directivity method borrows the extrapolation or extrapolation method, but the pixels of the prediction block are generated through the reference pixels lying on the prediction direction line. It is generated through pixels, but it means a method in which a partial prediction direction change in pixel units is allowed in consideration of the detailed directionality (eg, edge <Edge>) of the block. In the case of the directional prediction mode in the video encoding and decoding method of the present invention, the description will be focused on the linear directional method. In addition, in the case of the directional prediction method, intervals between adjacent prediction modes may be equal or non-uniform, which may be determined according to the size or shape of a block. For example, when a block having a size and shape of M×N is obtained through a block divider, when M and N are the same, the interval between prediction modes may be equal, and when M and N are different, In , intervals between prediction modes may be non-uniform. As another example, if M is greater than N, modes with vertical direction may allocate finer intervals between prediction modes close to the vertical mode (90 degrees), and allocate wider intervals to prediction modes far from the vertical mode. When N is greater than M, modes having horizontal directionality may allocate finer intervals between prediction modes closer to the horizontal mode (180 degrees) and allocate wider intervals to prediction modes farther from the horizontal mode. The above examples are not specific to the above-described case, and may include cases in which mutual conditions are reversed, and modifications are also possible as examples of other cases. In this case, the interval between prediction modes may be calculated based on a numerical value indicating the directionality of each mode, and the directionality of the prediction modes may be digitized with direction gradient information or angle information.

In addition, a prediction block may be generated by using other methods other than the above method using spatial correlation. For example, a reference block using an inter prediction method such as motion search and compensation may be generated as a prediction block using a current picture as a reference picture. In the prediction block generating step, a prediction block may be generated using a reference pixel according to the prediction method. That is, according to the prediction method, a prediction block can be generated through a directional prediction or a non-directional prediction method such as extrapolation, interpolation, copy, average, etc. of the existing intra prediction method, and the prediction block can be generated using the inter prediction method. can be generated, and other additional methods may be used.

The intra-prediction method may be supported under the same setting of an encoder or a decoder, and may be determined according to a slice type, block size, block shape, and the like. The intra-prediction scheme may be supported according to at least one or a combination of the above-mentioned prediction schemes. An intra-prediction mode may be configured according to the supported prediction method. The number of prediction modes within a supported picture may be determined according to the prediction method, slice type, block size, block shape, and the like. The related information can be set and transmitted in units such as sequences, pictures, slices, and blocks.

In the prediction mode encoding step performed by prediction mode encoding, a mode having an optimal encoding cost according to each prediction mode may be determined as a prediction mode of the current block in terms of encoding cost.

For example, the prediction mode encoder may use modes of one or more neighboring blocks for prediction of a current block mode for the purpose of reducing prediction mode bits. A mode (most_probable_mode, MPM) with a high probability of being identical to the mode of the current block may be included as a candidate group, and modes of neighboring blocks may be included in the above candidate group. For example, prediction modes of blocks such as the left, upper left, lower left, upper, and upper right blocks of the current block may be included in the candidate group.

The prediction mode candidate group is selected from among factors such as location of neighboring blocks, priorities of neighboring blocks, priorities in divided blocks, sizes or shapes of neighboring blocks, predetermined specific modes, and prediction modes of luminance blocks (in the case of chrominance blocks). It can be configured according to at least one factor or a combination thereof, and related information can be transmitted in units such as sequences, pictures, slices, and blocks.

For example, when a current block and a neighboring block are divided into two or more blocks, the mode of which block among the divided blocks is to be included as a mode prediction candidate for the current block can be determined under the same setting of the encoder or decoder. there is. For example, the block to the left of the neighboring blocks of the current block (M×M) is composed of three split blocks by performing quad tree split in the block splitter, and M/2×M/2, M/2×M/2, M/2×M/2 from top to bottom. When a block of 4×M/4 or M/4×M/4 is included, the prediction mode of the M/2×M/2 block based on the block size may be included as a mode prediction candidate of the current block. As another example, the upper block among the neighboring blocks of the current block (N×N) performs binary tree splitting in the block divider, and consists of three split blocks, N/4×N, N/4× from left to right. When N, N/2×N blocks are included, the prediction mode of the first N/4×N block from the left is used as a mode prediction candidate for the current block according to a preset order (priorities are assigned from left to right). can include

As another example, when the prediction mode of a block adjacent to the current block is a directional prediction mode, a prediction mode adjacent to the prediction direction of the current block (in terms of gradient information or angle information of the direction of the mode) is included in the mode prediction candidate group of the current block. can do. In addition, preset modes (planar, DC, vertical, horizontal, etc.) may be preferentially included according to the configuration or combination of prediction modes of neighboring blocks. Also, among the prediction modes of neighboring blocks, a prediction mode with a high frequency of occurrence may be preferentially included. The priority may mean not only the possibility of being included in the mode prediction candidate group of the current block, but also the possibility of being assigned a higher priority or index (that is, a higher probability of being assigned fewer bits in the binarization process) in the candidate group configuration. have.

As another example, the maximum number of mode prediction candidates for the current block is k, the left block is composed of m blocks whose length is smaller than the vertical length of the current block, and the upper block is composed of n blocks whose length is smaller than the horizontal length of the current block. Once configured, when the sum of split blocks (m+n) of neighboring blocks is greater than k, candidates can be filled in a predetermined order (from left to right, top to bottom), and the sum of split blocks (m+n) of neighboring block splits ) is greater than the maximum value (k) of the candidate group, the prediction mode of the neighboring block (left block, upper block) includes other neighboring blocks (eg, lower left, upper left, upper right, etc.) A prediction mode of the same block may also be included in the mode prediction candidate group of the current block. The above examples are not specific to the above-described case, and may include cases in which mutual conditions are reversed, and modifications are also possible as examples of other cases.

In this way, the candidate block for prediction of the mode of the current block is not limited to a specific block location, and prediction mode information can be used from at least one block among blocks located on the left side, upper left side, lower left side, upper side, and upper right side, As in the above example, the prediction mode candidate group of the current block may be configured by considering various factors.

The prediction mode encoder can classify a mode (MPM) candidate group with a high probability of being the same as the mode of the current block (referred to as candidate group 1 in this example) and a mode candidate group that is not (referred to as candidate group 2 in this example), and the current block A prediction mode encoding process may vary depending on which of the candidate groups the prediction mode of . The total prediction mode may be composed of the sum of the prediction modes of candidate group 1 and the prediction modes of candidate group 2, and the number of prediction modes of candidate group 1 and the number of prediction modes of candidate group 2 are the total number of prediction modes, slice type, block size, It may be determined according to one or more factors or a combination thereof among factors such as the shape of the block. Depending on the candidate group, the same binarization or different binarization may be applied. For example, fixed-length binarization may be applied to candidate group 1 and truncation-type binarization may be applied to candidate group 2. Although the number of candidate groups is exemplified as two in the above description, the first mode candidate group having a high probability of being the same as the mode of the current block, the second mode candidate group having a high probability of being identical to the mode of the current block, and the mode candidate group having a high probability of being identical to the mode of the current block It is possible to expand, and its modification is also possible.

In the post-processing filtering step performed by the post-processing filter unit, some predicted pixels of the prediction block generated in the previous process are considered in consideration of characteristics of high correlation between reference pixels adjacent to the boundary between the current block and neighboring blocks and pixels in the adjacent current block. can be replaced with a value generated by filtering one or more reference pixels adjacent to the boundary and one or more predicted pixels, and a value obtained by digitizing characteristics between reference pixels adjacent to the boundary of the block (eg, pixel value difference , gradient information, etc.) can be replaced with a value generated by applying a filtering process, and in addition to the above method, other methods having a similar purpose (correction of some predicted pixels of a prediction block through reference pixels) can be used. can be added In the post-processing filter unit, the type of filter and whether filtering is applied may be implicitly or explicitly determined, and the location and number of reference pixels and current pixels used in the post-processing filter unit, and the type of prediction mode to be applied may be determined. It can be set in an encoder or a decoder, and related information can be transmitted in units such as sequences, pictures, and slices.

In addition, in the post-processing filtering step, an additional post-processing process may be performed after generating a prediction block, such as the block boundary filtering. In addition, similar to the boundary filtering above, post-processing is performed on the reconstructed current block by combining the residual signal and the prediction signal obtained through transformation, quantization, and the reverse process after obtaining the residual signal, considering the characteristics of the pixels of the adjacent reference block. can also be done

Finally, a prediction block is selected or obtained through the above-described process, and information generated in this process may include information related to a prediction mode, and is transmitted to the transform unit 210 for encoding a residual signal after obtaining the prediction block. can

10 is an exemplary diagram for explaining a prediction principle in a P slice or a B slice in an image encoding method according to an embodiment of the present invention.

Referring to FIG. 10 , the video encoding method according to the present embodiment may include motion estimation module and interpolation steps. Information about a motion vector, a reference picture index, and a reference direction generated in the motion prediction step may be transferred to the interpolation step. In the motion prediction step and the interpolation step, values stored in a decoded picture buffer (DPB) may be used.

That is, the video encoding apparatus may perform motion estimation to find a block similar to the current block in previously encoded pictures. In addition, the image encoding apparatus may perform interpolation of reference pictures for prediction more precise than fractional precision. Finally, the video encoding apparatus obtains a prediction block through a predictor, and information generated in this process includes a motion vector, a reference picture index (reference picture index), and a reference direction. ), etc., and then coding of the residual signal can be performed.

In this embodiment, since intra prediction is performed even in P slices or B slices, it is possible to implement a combination method as shown in FIG. 9 supporting inter prediction and intra prediction.

11 is an exemplary diagram for explaining a process of obtaining a prediction block.

Referring to FIG. 11 , the image encoding method according to the present embodiment includes reference sample padding, reference sample filtering, intra prediction, boundary filtering, motion It may include steps of motion estimation and interpolation.

When the image encoding apparatus supports block matching in the current picture, the prediction method in I slices can be implemented with the configuration shown in FIG. 11 instead of the configuration shown in FIG. 9 . That is, the video encoding apparatus may use information such as a motion vector, a reference picture index, a reference direction, and the like generated only in a P slice or a B slice as well as a prediction mode in an I slice to generate a prediction block. However, information that can be partially omitted may exist due to the characteristic that the reference picture is current. For example, when the reference picture is the current picture, the reference picture index and reference direction can be omitted.

In addition, when interpolation is applied, the video encoding apparatus may not require block matching up to decimal units due to the characteristics of artificial images such as computer graphics, so whether or not to perform this may also be set by the encoder. It is also possible to set units such as sequences, pictures, and slices.

For example, the video encoding apparatus may not perform interpolation of reference pictures used for inter-prediction according to the setting of the encoder, and may not perform interpolation only when block matching is performed in the current picture. can do. That is, the video encoding apparatus of the present embodiment may set whether to perform interpolation of reference pictures. In this case, it may be determined whether to perform interpolation on all or some reference pictures constituting the reference picture list. For example, the image encoding apparatus does not perform interpolation when it is not necessary to perform block matching in decimal units because the characteristics of an image in which a reference block exists in a certain current block are artificial images, and block matching in decimal units because it is a natural image. When necessary, it can operate to perform interpolation.

In addition, the video encoding apparatus may set whether to apply block matching to a reference picture in which interpolation is performed block by block. For example, when a natural image and an artificial image are mixed, interpolation is performed on a reference picture, but when an optimal motion vector can be obtained by searching a part of the artificial image, a constant unit (assuming an integer unit here) is performed. A motion vector can be expressed, and if an optimal motion vector can be obtained by selectively searching a part of a natural video, the motion vector can be expressed in another constant unit (herein, 1/4 unit is assumed).

12 is an exemplary diagram for explaining a main process of an image encoding method according to an embodiment of the present invention with syntax in a coding unit.

Referring to FIG. 12, curr_pic_BM_enabled_flag means a flag that allows block matching in the current picture, and can be defined and transmitted in units of sequences and pictures. In this case, the process of generating a prediction block by performing block matching in the current picture is This may mean a case of operating through prediction. In addition, it can be assumed that cu_skip_flag, which is an inter-screen technology that does not encode a residual signal, is a flag supported only in P slices or B slices excluding I slices. In this case, when curr_pic_BM_enabled_flag is on, block matching (BM) can be supported in the inter-prediction mode even in the I slice.

That is, the video encoding apparatus of the present embodiment can support skip when a prediction block is generated through block matching in the current picture, and can support skip even in the case of an in-screen description other than block matching. In addition, skip may not be supported in I slices according to conditions. Whether or not to skip this may be determined according to encoder settings.

For example, when skip is supported in I slice, a prediction block can be directly restored as a reconstructed block through block matching without encoding a residual signal by connecting to prediction_unit(), which is a prediction unit, through a specific flag if (cu_skip_flag). . In addition, the video encoding apparatus may classify a method of using a prediction block through block matching in a current picture as an inter-prediction technique, and process such classification through pred_mode_flag, which is a specific flag.

That is, the video encoding device may set the prediction mode to inter prediction mode (MODE_INTER) if pred_mode_flag is 0, and to intra prediction mode (MODE_INTRA) if it is 1. This is a similar intra-screen technique to the existing one, but it can be classified as an inter-screen technique or an intra-screen technique in I slices to distinguish it from the existing structure. That is, the video encoding apparatus of the present embodiment does not use temporal correlation in I slices, but may use a structure of temporal correlation. part_mode means information about the size and shape of a block divided in a coding unit.

13 is an exemplary diagram for explaining an example of supporting symmetric type splitting or asymmetric type splitting as in inter prediction when a prediction block is generated through block matching in a current picture.

Referring to FIG. 13, in the video encoding method according to the present embodiment, when a prediction block is generated through block matching in a current picture, a symmetric type such as 2N×2N, 2N×N, and N×2N is used as in inter prediction. (symmetric) partitioning may be supported, or asymmetric partitioning such as nL×2N, nR×2N, 2N×nU, and 2N×nD may be supported. Various block sizes and shapes may be determined according to the division method of the block divider.

14 is an exemplary diagram for explaining that 2N×2N and N×N can be supported in inter prediction (Inter).

Referring to FIG. 14 , the video encoding method according to the present embodiment can support 2N×2N and N×N like prediction block types used for existing intra-prediction. This is an example in which the block division unit supports a square shape through a quad-tree division method or a division method according to a predefined block candidate group, and a binary tree division method or a rectangular shape in a predefined block candidate group in intra-prediction. Other block types can be supported by adding a shape, and the setting for this can be set in the encoder. Also, during intra prediction, the encoder can set whether to apply skip only when block matching is performed with the current picture, to apply to existing intra prediction, or to other new intra prediction. Information about this may be transmitted in units of sequences, pictures, slices, and the like.

The subtraction unit 205 may generate a residual block by deriving a pixel difference value by subtracting pixel values of the prediction block generated by the prediction unit 200 from the pixel values of the current block to be encoded.

The transform unit 210 receives a residual block, which is a difference value between a current block and a predicted block generated through intra-prediction or inter-prediction from the subtractor 205, and converts the residual block into a frequency domain. Through the transform process, each pixel of the residual block corresponds to the transform coefficient of the transform block. The size and shape of the transform block may have a size equal to or smaller than that of the coding unit. Also, the size and shape of the transform block may be the same as or smaller than the prediction unit. An image encoding device may perform transformation processing by bundling several prediction units.

The size or shape of the transform block may be determined by the block divider, and a square or rectangular transform may be supported according to the block divider. The block division operation may be influenced according to transform-related settings supported by an encoder or a decoder (size, shape, etc. of a supported transform block).

The size and shape of each transform block is determined according to the encoding cost for each candidate of the size and shape of the transform block, and video data of each determined transform block and partition information such as the determined size and shape of each transform block can be encoded. .

The transform can be transformed by a one-dimensional transform matrix. (DCT, DST, etc.) Each transformation matrix can be used adaptively in horizontal and vertical units. Examples of adaptive use may be determined by various factors such as block size, block shape, block type (luminance and color difference), encoding mode, prediction mode information, quantization parameter, and encoding information of neighboring blocks.

For example, in the case of intra prediction, when the prediction mode is horizontal, a DCT-based transformation matrix may be used in the vertical direction and a DST-based transformation matrix may be used in the horizontal direction. In the vertical case, a DCT-based transformation matrix may be used in the horizontal direction and a DST-based transformation matrix may be used in the vertical direction. Transformation matrices are not limited to those from the above description. This information may be determined using an implicit or explicit method, and may be determined according to one or more factors or a combination of factors such as block size, block shape, encoding mode, prediction mode, quantization parameter, and encoding information of a neighboring block. may be determined, and the related information may be transmitted in units of sequences, pictures, slices, blocks, and the like.

Here, considering the case of using the explicit method, if two or more transformation matrices for the horizontal and vertical directions are set as candidates, information on which transformation matrix was used for each direction may be sent, respectively, Alternatively, information on which transformation matrices are used in the horizontal and vertical directions may be transmitted by bundling them into one pair for which transformation matrices are used in the horizontal and vertical directions, and setting two or more pairs as candidates.

In addition, partial transformation or overall transformation may be omitted in consideration of characteristics of an image. For example, one or both of the horizontal or vertical components may be omitted. This is because when intra-prediction or inter-prediction is not performed well and a large difference occurs between the current block and the predicted block (ie, when the residual component is large), the resulting encoding loss may increase when converting it. This may be determined according to at least one or a combination of factors such as encoding mode, prediction mode information, block size, block shape, block type (luminance and color difference), quantization parameter, and encoding information of neighboring blocks. . Depending on the above conditions, it can be expressed using an implicit or explicit method, and information about this can be transmitted in units such as sequences, pictures, and slices.

The quantization unit 215 performs quantization of the residual component transformed by the transform unit 210 . The quantization parameter is determined in units of blocks, and the quantization parameters may be set in units of sequences, pictures, slices, and blocks.

For example, the quantization unit 215 may predict a current quantization parameter using one or more quantization parameters derived from neighboring blocks such as left, upper left, upper, upper right, and lower left of the current block.

In addition, the quantization unit 215 performs a difference between a basic parameter and a basic parameter transmitted in units of sequences, pictures, slices, etc., when a quantization parameter predicted from a neighboring block does not exist, that is, when a block is at a boundary between a picture and a slice. values can be output or transmitted. If there is a quantization parameter predicted from a neighboring block, a difference value may be transmitted using the quantization parameter of the corresponding block.

The priority of blocks from which quantization parameters are to be derived may be set in advance, or may be transmitted in units of sequences, pictures, slices, and the like. The residual block may be quantized through dead zone uniform threshold quantization (DZUTQ), a quantization weighted matrix, or an improved technique thereof. This can set one or more quantization techniques as candidates and can be determined by encoding mode, prediction mode information, and the like.

For example, the quantization unit 215 may set a quantization weight matrix to be applied to inter-screen coding, intra-coding unit, etc., and may also set different weight matrices according to intra-prediction modes. The quantization weight matrix has a size of M×N, and can be constructed by changing quantization coefficients for each position of each frequency component when it is assumed that the size of a block is the same as the size of a quantization block. The quantization unit 215 may select one of several existing quantization methods or may be used under the same settings of an encoder or a decoder. This information can be transmitted in units such as sequences, pictures, and slices.

Meanwhile, the inverse quantization units 220 and 315 and the inverse transform units 225 and 320 shown in FIGS. 2 and 3 may be implemented by performing the above processes in the transform unit 210 and the quantization unit 215 in reverse. That is, the inverse quantization unit 220 may inversely quantize the quantized transform coefficients generated by the quantization unit 215, and the inverse transform unit 225 may generate a reconstructed residual block by inversely transforming the inverse quantized transform coefficients. have.

The adders 230 and 324 shown in FIGS. 2 and 3 add pixel values of the prediction block generated from the predictor to pixel values of the reconstructed residual block to generate a reconstructed block. The reconstructed block may be stored in the encoding and decoding picture buffers 240 and 335 and provided to the prediction unit and the filter unit.

The filter unit may apply an in-loop filter such as a deblocking filter, a sample adaptive offset (SAO), or an adaptive loop filter (ALF) to the reconstructed block. The deblocking filter may filter the reconstructed block to remove distortion between block boundaries generated during encoding and decoding processes. SAO is a filter process for restoring a difference between an original image and a reconstructed image as an offset on a pixel-by-pixel basis for a residual block. ALF may perform filtering to minimize the difference between a predicted block and a reconstructed block. ALF may perform filtering based on a comparison value between a block reconstructed through a deblocking filter and a current block.

The entropy encoding unit 245 may entropy encode the transform coefficients quantized by the quantization unit 215 . For example, techniques such as context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), and probability interval partitioning entropy (PIPE) coding may be performed. can

The entropy encoding unit 245 may include in encoded data a bit stream obtained by encoding a quantization coefficient and various pieces of information required to decode the encoded bit stream. Encoded data may include an encoded block type, quantization coefficients, a bit stream in which the quantization block is encoded, and information necessary for prediction. In the case of quantization coefficients, 2-dimensional quantization coefficients can be scanned in 1-dimensional. The distribution of the quantization coefficient may vary according to the characteristics of an image. In particular, in the case of intra-prediction, since the distribution of coefficients may have a specific distribution according to the prediction mode, a scan method may be set differently.

Also, the entropy encoding unit 245 may be set differently according to the size of a block to be encoded. The scan pattern may be preset or set as a candidate for at least one of various patterns such as zigzag, diagonal, raster, etc., and may be determined by encoding mode, prediction mode information, etc., under the same settings of the encoder and decoder. can be used This information can be transmitted in units such as sequences, pictures, and slices.

The size of the quantized block (hereinafter referred to as quantization block) input to the entropy encoding unit 245 may be equal to or smaller than the size of the transform block. In addition, a quantization block may be divided into two or more sub-blocks, and in case of division, the scan pattern of the divided block may be set to be the same as or different from that of the existing quantization block.

For example, if the scan pattern of an existing quantization block is zigzag, the zigzag pattern may be applied to all subblocks, or the zigzag pattern may be applied to a subblock located at the upper left of a block including a DC component, Diagonal patterns can be applied to other blocks. This may also be determined according to encoding mode, prediction mode information, and the like.

In addition, the starting position of the scan pattern in the entropy encoding unit 245 basically starts from the upper left corner, but may start from the upper right corner, lower right corner, or lower left corner depending on the characteristics of the image, and any one of two or more candidate groups is selected. Information on whether or not it has been performed can be transmitted in units of sequences, pictures, slices, and the like. Entropy encoding technology may be used as the encoding technology, but is not limited thereto.

On the other hand, the inverse quantization of the inverse quantization unit 220 and the inverse transformation of the inverse transform unit 225 shown in FIGS. 2 and 3 reverse the structure of the quantization of the quantization unit 215 and the transformation of the transform unit 210. And it can be implemented by combining the basic filter units 235 and 330.

Hereinafter, the aforementioned reference pixel configuration step will be described in more detail.

When performing intra prediction according to an embodiment of the present invention, it is necessary to check the availability of reference pixels of neighboring blocks. It may be determined by at least one factor or a combination thereof among factors such as and boundary characteristics of neighboring blocks. For example, an M×M block is divided into 4 M/2×M/2 blocks (in this example, 0, 1, 2, and 3 indexes are assigned in the raster scan direction) through the block divider, and the current block is 3 In the case of M/2×M/2 blocks in number, the neighboring blocks located in a specific direction (in this example, the upper right and lower left) relative to the current block belong to the same unit of picture, slice, or tile as the current block, but are not yet Since the block is not encoded, it is determined that the block is unavailable, and thus the reference pixel of the corresponding block cannot be used. As another example, whether to use a reference pixel of a neighboring block may be determined according to a boundary characteristic (a boundary such as a slice or a tile) between a current block and a neighboring block.

As another example of whether or not to use reference pixels of neighboring blocks, in the case of P or B slices, determine whether to use data with a high probability of error accumulation in the encoder/decoder for the purpose of avoiding error propogation over time settings can be supported. Depending on this setting information, it is possible to determine whether or not to use data with a high probability of error accumulation for prediction block generation. The flag may determine whether to use a neighboring block coded in an inter-prediction mode having a high possibility of error accumulation among neighboring blocks referred to for generation of an intra-prediction block. If the flag is 0, reference pixels of blocks encoded by inter prediction cannot be used, and if the flag is 1, reference pixels of blocks encoded by inter prediction can be used.

An example to be described later describes a case of generating a prediction block when the flag is not 0. By additional setting of the flag, regardless of the possibility of error accumulation, whether or not the reference pixel can be used is disabled so that no reference pixel can be used for prediction. In this case, since the prediction block cannot use any reference pixel, the prediction block may be generated by filling in a predetermined value (eg, a median value of a pixel value range of a bit depth). Alternatively, a prediction block to which edge information or the like is applied may be generated based on a preset value. For example, if the preset value is A, the right area including a random pixel is filled with (A+B) and the other areas (A-C) based on a random pixel in the M?N block to form a prediction block. In this case, B and C mean parameters used to express edge information.

In addition, by additionally setting the flag, the possibility of error accumulation may be determined based on the number of neighboring blocks coded through inter-prediction, the number of reference pixels of neighboring blocks, and the like. Depending on the above determination, it may be determined whether or not reference pixels of blocks encoded through intra-prediction are also used for prediction. For example, if less than N blocks among M neighboring blocks adjacent to the current block are blocks encoded by inter-prediction, reference pixels of the blocks cannot be used for prediction block generation, and within (M-N) number of blocks Reference pixels of prediction-encoded blocks may also be unavailable for generation of prediction blocks. Since the number of blocks as the criterion may be affected by the size or shape of the block, it may be replaced with the number of reference pixels existing at a position to be used for prediction of the current block, and other factors may also be considered.

Depending on whether or not the reference pixel is available, a generated prediction block may be affected. For example, if at least one of the neighboring blocks of the current block is available according to the availability of the reference pixel determined by the various factors, a pixel derived (eg, copied, averaged, etc.) from the reference pixel of the available block A reference pixel of an unavailable block can be replaced with . As another example, if there is no block available according to whether or not the reference pixel can be used, the reference pixel of the unavailable block may be replaced with a preset value (eg, a median value of a pixel value range having a bit depth).

Substitution of a value derived from the available block may be prohibited according to settings of an encoder or decoder, and in this case, a prediction mode derived from the corresponding block cannot be used.

Hereinafter, these contents will be described in more detail with reference to the drawings.

15 is an exemplary view for explaining an encoding mode of a neighboring block in intra prediction according to an embodiment of the present invention.

Referring to FIG. 15 , it is possible to determine whether to use a reference pixel of a corresponding block to generate a prediction block according to an encoding mode of a block adjacent to a current block. In this case, the neighboring block may have a square shape of M? M or a rectangular shape of M? N obtained through the block divider, and may be encoded as either intra prediction or inter prediction according to an encoding mode. As shown in FIG. 15, if the current block is the middle block, the upper left block is inter prediction (Inter), the upper two blocks are intra prediction (Intra), and the upper right block is again In it, it is possible to assume a situation coded as inter-prediction, intra-prediction, and intra-prediction from the left.

Blocks encoded by the inter-prediction method may be additionally classified according to reference pictures. 15, coded by inter prediction (Inter), and expressed as ref = t, the reference picture indicates the current picture.

16A and 16B will be described under the above premise.

*

16A is an exemplary diagram for explaining whether to use a reference pixel in consideration of an encoding mode of a neighboring block in intra prediction according to an embodiment of the present invention. 16B is an exemplary diagram for explaining whether or not a reference pixel is used without considering the coding mode of a neighboring block in intra prediction according to an embodiment of the present invention.

In an example to be described later, whether or not to consider the encoding mode of a neighboring block is described using a CIP flag. Specifically, when the CIP flag indicates 1, a block encoded in a specific encoding mode (inter prediction in this example) cannot be used for prediction of the current block, and when the CIP flag indicates 0, a block in a certain encoding mode of the current block can be used for prediction.

* Referring to FIG. 16A, when the CIP flag is 1 and a neighboring block is coded by inter prediction (Inter), the reference pixel of the corresponding block cannot be used for prediction of the current block. A reference pixel of a block adjacent to a block coded by intra prediction (Intra), indicated by hatched lines, may be used for prediction of the current block.

Here, a pixel adjacent to a current block within a block usable as a reference pixel may be used as a reference pixel, and other additional pixels may also be included.

Referring to FIG. 16B, when the CIP flag is 0, reference pixels of an encoded block may be used for prediction of a current block. That is, pixels adjacent to the current block in the upper left, upper, upper right, left, and lower left blocks of the current block may be used as reference pixels.

17 is another exemplary diagram for explaining an encoding mode of a neighboring block in intra prediction according to an embodiment of the present invention.

Referring to FIG. 17, a neighboring block is coded with either intra prediction (Intra) or inter prediction (Inter), and when the neighboring block is coded with inter prediction, as an indication of a reference picture used at this time, ref=t indicates that the reference picture is the current picture, ref=t-1 indicates that the reference picture is the picture immediately preceding the current picture, and ref=t-2 indicates that the reference picture is the second previous picture of the current picture. and other reference pictures may be added according to the reference direction of inter-prediction. The upper right block means a block located outside the picture range.

An example of determining whether to use as a reference pixel on the premise of an encoding state as shown in FIG. 17 will be described with reference to FIGS. 18A to 18C.

18A is an exemplary view illustrating a case in which an encoding mode of a neighboring block is intra prediction in intra prediction according to an embodiment of the present invention. 18B is an exemplary view illustrating consideration of coding modes and reference pictures of neighboring blocks in intra prediction according to an embodiment of the present invention. 18C is another exemplary diagram for explaining a case in which encoding modes of neighboring blocks are not considered in intra prediction according to an embodiment of the present invention.

Referring to FIG. 18A, neighboring blocks previously encoded by intra prediction in the encoding mode of FIG. 17 may be indicated by hatched lines.

Since a block indicated by hatching is an intra-prediction, a pixel within the block may be used as a reference pixel.

Also, even in this case, whether or not to consider the encoding mode of a neighboring block may be indicated according to a CIP flag, and this may be an example of a case in which the CIP flag is 1.

Referring to FIG. 18B , as described above, a pixel in a block encoded by intra prediction in the encoding mode of FIG. 17 can be used as a reference pixel.

However, in addition to this in FIG. 18B, even if a neighboring block is coded by inter-prediction, if the reference picture of the neighboring block is the current picture, pixels in the neighboring block can be used as reference pixels. Alternatively, if the reference picture of a neighboring block is the current picture, pixels of the corresponding block may not be used as reference pixels.

This can be used because if the reference picture of a neighboring block is the current picture, error propagation does not occur between pictures according to the time lapse. Conversely, if a block referenced by the current picture is a block coded from a previous or subsequent picture, it may be unavailable because error accumulation may occur.

As above, whether or not to use the reference pixel can be determined by at least one of factors such as the location of the neighboring block, the encoding mode of the neighboring block, the location of the current block in the divided block, and the boundary characteristics between the current block and the neighboring block, or a combination thereof. and a reference picture may also be added as the factor. In detail, it is possible to determine whether to use reference pixels of a corresponding block for prediction of the current block by considering the index of the reference picture of the neighboring block or the picture order count (POC). Although the above example specified that the reference picture is the current picture, extension to a reference picture having a different index or picture order count (POC) is possible. In addition, even when the reference directions of inter prediction are unidirectional prediction (L0, L1) and bidirectional prediction, various settings for reference picture indices (L0, L1) may be configured.

Accordingly, not only blocks indicated by hatching in FIG. 18A (intra-predicted blocks), but also pixels in blocks that have been coded through inter-prediction and whose reference picture is the current picture can be used as reference pixels.

Also, even in this case, whether or not to consider the encoding mode of a neighboring block may be indicated according to the CIP flag, so this may be an example describing a case in which the flag is 1.

Referring to FIG. 18C , in principle, adjacent pixels in adjacent blocks can all be used as reference pixels.

However, since the area expressed at the upper right of the current block located in the middle is a part without image data or an area outside the image, it may correspond to an unavailable area in any case.

Examples described above with reference to FIGS. 15 to 18C are summarized as follows.

First, availability of reference pixels for reference pixels of neighboring blocks of the current block may be determined.

If it is determined that reference pixels in all neighboring blocks are not available, reference pixels configured with preset pixel values may be obtained. For example, a reference pixel may be configured as an intermediate value of a pixel expression range. More specifically, values can be assigned to reference pixels as 1 << (bit depth - 1). Since bit depth means the number of bits used to represent a pixel, the example value is pixels. It may mean the median value of the expression range of

When determining that at least one neighboring block is usable, the encoding mode of the neighboring block (left, upper, upper left, upper right, lower left block, etc.) may be considered.

In addition, a preset flag indicating whether to consider the encoding mode can be used.

For example, all pixels in neighboring blocks may be used as reference pixels without considering the encoding mode, and here, a preset flag may indicate that the encoding mode is not considered.

Here, when the coding mode of the neighboring block is inter-prediction, it is determined that the neighboring block is unusable, and pixels in the neighboring block may not be used as reference pixels.

In addition, even if the coding mode of the neighboring block is inter-prediction, a reference picture of the neighboring block can be checked and used as a reference pixel if it is the current picture.

Here, the reference picture is selected from list 0, which stores data for reference pictures preceding the current picture, or list 1, which stores data for reference pictures subsequent to the current picture, and the current picture is selected from list 0 or the list. 1 can be included.

Also, even if the coding mode of the neighboring block is inter-prediction, the reference picture of the neighboring block is checked and if it is an I picture (excluding P or B), it can be used as a reference pixel.

However, the above constraints should be understood as examples for determining whether a pixel in a neighboring block can be used as a reference pixel in consideration of a specific condition. That is, the coding mode of a neighboring block is inter-prediction, and whether or not it can be used as a reference pixel can be determined by referring to motion vectors, reference picture information, etc., which are coding information for the neighboring block. In this case, the sequence and picture , The conditions exemplified above can be set in units such as slices.

Determination of availability of a reference pixel according to an embodiment of the present invention may follow the following flow.

First, it can be checked whether a block adjacent to the current block is located beyond the boundary of a picture, slice, tile, or the like. After that, it is possible to check whether neighboring blocks are encoded. Thereafter, the slice type of the neighboring block is checked, and in the case of an I slice, a boundary pixel in the neighboring block may be set to be available as a reference pixel of the current block. If it is a P slice, since it is unidirectional prediction, the reference picture is checked by checking the reference picture list 0 (ref_idx_l0). If the reference picture is the current picture, it can be described as being available as a reference pixel of the current picture. However, this should be understood as an example. If the current picture is not a reference picture, it can be set as not available. If it is a B slice, it is possible to check whether the prediction mode is unidirectional_L0, unidirectional_L1, or bidirectional by checking whether the current picture is set as a reference picture, and then whether it is available as a reference pixel.

19A is an exemplary view illustrating a method of obtaining a reference pixel when an available neighboring block is a block in the lower left corner. 19B is an exemplary view illustrating a method of obtaining a reference pixel when an available neighboring block is a block in the upper right corner. 19C is an exemplary diagram explaining a method of obtaining a reference pixel when available neighboring blocks are blocks in lower left and upper right corners. 19D is an exemplary view illustrating a method of obtaining a reference pixel when available neighboring blocks are upper left and upper right blocks.

Referring to FIGS. 19A to 19D , hatched pixels may mean adjacent pixels in a block in which the adjacent block is determined to be usable, and an arrow direction may mean a direction in which a reference pixel is configured.

As described above, a neighboring block may be determined to be available, but may be determined to be unavailable. Accordingly, a pixel in a block determined to be unavailable cannot be used as a reference pixel. In this case, a problem may arise as to how to configure adjacent pixels in the unavailable block.

Referring to FIGS. 19A and 19B , when only pixels in the upper right or lower left of the current block are available as reference pixels, pixels adjacent to the unavailable area may be copied and filled in pixels adjacent to the current block among the unavailable areas. have.

Referring to FIG. 19C, if there are available areas in both the upper right and lower left of the current block, it is possible to copy from pixels in either available area (<bottom to top and left to right> or <left to right> Right, and top to bottom> or <bottom to top, right to left>), the average value of pixels on both sides can be filled, or it can be filled by linear interpolation.

Referring to FIG. 19D, if the area usable as a reference pixel is mixed, the value can be copied and filled in either direction (left direction, downward direction), and there is an area available on both sides, such as the top of the current block It can be filled with a value generated by utilizing pixels on both sides, such as interpolation or average, or it can be copied and filled where there is an area where only one side is available, such as the left or bottom left of the current block.

In more detail with reference to FIG. 19D , pixels located on the right including P(-1, -1) and P(blk_size, -1) at the top right of the current block may be available. Values from P(0, -1) to P(blk_size-1, -1) may not be available. In this case, if we assign the average value, P(x, y){x is from 0 to blk_size-1, y is 1} = {P(-1, -1) + P(blk_size, -1)} + 1 /2 can be assigned to the value of In this case, blk_size may mean a block size.

Pixels located below, including P(-1,0), may be not available. At this time, P(x, y) {x is 1, y is 0 to 2*blk_size-1} = P(-1, -1).

In addition, it is possible to determine whether to perform interpolation according to the position of the reference pixel. As shown in FIG. 19C , even if available areas exist on both sides, interpolation may be limited if the positions of the available areas are not connected with vertical lines. Specifically, when comparing the coordinates at both ends of the available area, interpolation can be restricted when neither x nor y matches, such as P(-1, a) or P(b, -1). . Interpolation can be performed only when one of x and y matches, such as P(a, -1) and P(b, -1).

In addition, as shown in FIG. 19D , interpolation may be performed when the location of the available area lies on a vertical line, even if there is an available area on both sides.

20 is a flowchart of an image encoding method according to an embodiment of the present invention.

Referring to FIG. 20, an image encoding method for performing intra-prediction determines whether a boundary pixel in a neighboring block can be used as a reference pixel of a current block in consideration of a prediction mode of each of a current block and neighboring blocks. Step (S110), obtaining a reference pixel of the current block according to the determined result (S120), generating a predicted prediction block in the picture based on the obtained reference pixel (S130), and using the generated prediction block and encoding the current block (S140).

For a description of each step, reference may be made to FIGS. 15 to 18C, so a detailed description thereof will be omitted.

An image decoding method for performing intra-prediction according to an embodiment of the present invention includes the steps of receiving a bit stream including data about a current block and prediction modes of blocks adjacent to the current block, and extracting data from the received bit stream. extracting and confirming the prediction mode of the neighboring block, determining whether a boundary pixel in the neighboring block can be used as a reference pixel of the current block in consideration of the prediction mode of the neighboring block, according to the determined result, The method may include acquiring a reference pixel of the current block, generating a prediction block within a picture based on the obtained reference pixel, and decoding the current block using the generated prediction block.

Here, in the step of obtaining a reference pixel, if it is determined that a boundary pixel is unavailable as a reference pixel of the current block, a reference pixel configured with a preset pixel value may be obtained.

Here, the step of determining whether a block can be used as a reference pixel may indicate whether a preset flag is to consider a prediction mode of a neighboring block.

Here, in the step of determining whether the block can be used as a reference pixel, if the prediction mode of the neighboring block is intra-prediction, it may be determined that the boundary pixel can be used as a reference pixel of the current picture.

Here, in the step of determining whether the pixel can be used as a reference pixel, if the prediction mode of the neighboring block is inter-prediction, it may be determined that the boundary pixel is unavailable as a reference pixel of the current picture.

Here, the step of determining whether a border pixel can be used as a reference pixel is, when the prediction mode of the neighboring block is inter-prediction, considering the reference picture of the neighboring block, whether or not the boundary pixel can be used as a reference pixel of the current picture. can decide

Here, the reference picture is selected from list 0, which stores data for reference pictures preceding the current picture, or list 1, which stores data for reference pictures subsequent to the current picture, and the current picture is in list 0 or list 1. can be included

Here, when the reference picture of the neighboring block is the current picture, a boundary pixel may be determined to be usable as a reference pixel of the current picture.

Here, when the reference picture of the neighboring block is not the current picture, pixels in the neighboring block may be determined to be unavailable as reference pixels of the current picture.

Here, when the reference picture of the neighboring block is an I picture, pixels in the neighboring block may be determined to be usable as reference pixels of the current picture.

In the video decoding apparatus including one or more processors according to an embodiment of the present invention, the one or more processors receive a bit stream including data about a current block and prediction modes of blocks adjacent to the current block, and receive the received bits. After extracting data from the stream, checking the prediction mode of the neighboring block, determining whether or not a boundary pixel in the neighboring block can be used as a reference pixel of the current block in consideration of the prediction mode of the neighboring block, and based on the determined result. Accordingly, a reference pixel of the current block may be obtained, a prediction block within a picture may be generated based on the acquired reference pixel, and the current block may be decoded using the generated prediction block.

According to the above-described embodiment, international codecs such as MPEG-2, MPEG-4, H.264 or other codecs in which intra-prediction technology is used, media using these codecs, and high performance generally available to the video industry A high-efficiency video encoding and decoding technology can be provided.

In addition, in the future, it is expected to be applied to the field of image processing using standard codecs and intra-prediction such as the current high-efficiency video coding technology (HEVC) and H.264/AVC.

Although described with reference to the above embodiments, those skilled in the art will understand that the present invention can be variously modified and changed without departing from the spirit and scope of the present invention described in the claims below. You will be able to.

Claims (12)

delete delete delete delete delete delete delete In the video decoding method,
decoding current picture reference information indicating whether block matching prediction based on the current picture is allowed for the current picture;
generating reference picture list 0 and reference picture list 1 for the current block, the current picture being assigned to the reference picture list 0 and the reference picture list 1 according to the current picture reference information;
determining a reference picture for the current block based on the reference picture list 0 and the reference picture list 1; and
Performing prediction on the current block using the reference picture;
The current picture is assigned to the reference picture list 0 after a first reference picture is assigned to the reference picture list 0;
The first reference picture is displayed before the current picture,
The current picture is assigned to reference picture list 1 after a second reference picture is assigned to reference picture list 1;
The second reference picture is displayed after the current picture.
According to claim 8,
The reference picture list 0 is generated by sequentially allocating the first reference picture, the second reference picture, and the current picture,
The reference picture list 1 is characterized in that it is generated by sequentially allocating the second reference picture, the first reference picture, and the current picture.
According to claim 8,
The video decoding method, characterized in that the current picture reference information is transmitted in units of pictures or sequences.
In the video encoding method,
Generating reference picture list 0 and reference picture list 1 for the current block, according to whether block matching prediction based on the current picture is allowed for the current picture, the current picture is the reference picture list 0 and the reference picture list assigned to 1;
determining a reference picture for the current block based on the reference picture list 0 and the reference picture list 1;
performing prediction on the current block using the reference picture; and
Encoding current picture reference information indicating whether block matching prediction based on the current picture is allowed for the current picture,
The current picture is assigned to the reference picture list 0 after a first reference picture is assigned to the reference picture list 0;
The first reference picture is displayed before the current picture,
The current picture is assigned to reference picture list 1 after a second reference picture is assigned to reference picture list 1;
Wherein the second reference picture is displayed after the current picture.
In a computer-readable recording medium storing a bitstream encoded by an image encoding method in a computer-readable recording medium storing a bitstream encoded by an image encoding method, the image encoding method comprises:
Generating reference picture list 0 and reference picture list 1 for the current block, according to whether block matching prediction based on the current picture is allowed for the current picture, the current picture is the reference picture list 0 and the reference picture list assigned to 1;
determining a reference picture for the current block based on the reference picture list 0 and the reference picture list 1;
performing prediction on the current block using the reference picture; and
Encoding current picture reference information indicating whether block matching prediction based on the current picture is allowed for the current picture,
The current picture is assigned to the reference picture list 0 after a first reference picture is assigned to the reference picture list 0;
The first reference picture is displayed before the current picture,
The current picture is assigned to reference picture list 1 after a second reference picture is assigned to reference picture list 1;
The second reference picture is displayed after the current picture, computer-readable recording medium.

Images