KR20220024304A - Image encoding/decoding method and image decoding apparatus using motion vector precision - Google Patents

Abstract

An image encoding/decoding method and image decoding apparatus using motion vector precision are disclosed. A motion vector candidate selection method comprises the steps of: constructing a spatial motion vector candidate (a first candidate); determining whether the reference picture of a current block exists in a current picture; and when the determination result of the determining step is YES, adding a spatial motion vector candidate (a second candidate) in another block of the current picture encoded prior to the current block.

Description

Image encoding and decoding method and image decoding apparatus using motion vector precision

The present invention relates to image encoding and decoding technology, and more particularly, to an image encoding and decoding method and an image decoding apparatus using motion vector precision.

With the spread of the Internet and mobile terminals and the development of information and communication technologies, the use of multimedia data is rapidly increasing. Accordingly, in order to perform various services or tasks through image prediction in various systems, the need for improving the performance and efficiency of an image processing system is significantly increasing.

On the other hand, in the existing image encoding and decoding technology, according to the inter prediction method, motion information for neighboring blocks of the current block is predicted in at least one reference picture before or after the current picture according to the inter prediction method, or the current picture is predicted according to the intra prediction method. A motion vector for the current block is estimated by acquiring motion information from my reference block.

However, the conventional inter prediction has a disadvantage of high computational complexity because a prediction block is generated using a temporal prediction mode between pictures, and the intra prediction has a disadvantage of high coding complexity.

As such, in the image encoding and decoding method of the prior art, performance improvement for image encoding or image decoding is still required.

An object of the present invention for solving the above problems is to improve motion vector precision when selecting a prediction candidate from reference blocks of a reference picture including the current picture in order to derive motion information for the current block in video encoding and decoding. An object of the present invention is to provide an image encoding and decoding method used.

Another object of the present invention is to provide an image decoding apparatus using motion vector precision.

In one aspect of the present invention for achieving the above object, when the interpolation precision of the reference picture has a first value, the motion vector of the first neighboring block of the current block referring to the reference picture is the same as the first value, or searching with a precision of a second value greater than the first value; searching for a motion vector of a second neighboring block of the current block with precision of a third value greater than the second value; and encoding first information on the motion vectors of the first block and the second block and matching information on the precision of the motion vectors.

In another aspect of the present invention for achieving the above object, when the interpolation precision of the reference picture has a first value, the motion vector of the first neighboring block of the current block referring to the reference picture is the same as the first value, or searching with a precision of a second value greater than the first value, searching for a motion vector of a second neighboring block of the current block with a precision of a third value greater than the second value, and precision of the motion vectors An image decoding method is provided, comprising decoding an image based on matching information for , and information on motion vectors of the first block and the second block.

Here, the current picture or information on the current picture may be added to the end of the reference picture list 0 and the reference picture list 1.

Here, the first value may be a true fraction. Also, the second value or the third value may be an integer.

Here, if the frequency of the second value is greater than the frequency of the third value in the index including the matching information, the number of binary bits of the second value may be shorter than the number of binary bits of the third value.

Here, if the frequency of the third value is the greatest in the index, the third value may have a precision of zero, that is, zero.

Here, the first neighboring block or the second neighboring block may place and position a block spatially different from the current block. The first neighboring block or the second neighboring block may be a block encoded by inter prediction prior to the current block.

Here, the reference picture of the first neighboring block or the second neighboring block may be a current picture. The motion vector of the first neighboring block or the second neighboring block may be searched by inter prediction.

In another aspect of the present invention for achieving the above object, a memory for storing a program or program code for image decoding, and a processor connected to the memory, wherein the processor interpolates a reference picture by the program When the precision has a first value, a motion vector of a first neighboring block of the current block referring to the reference picture is searched for with a precision of a second value equal to or greater than the first value, and the second value is greater than the first value. A motion vector of a second neighboring block of the current block is searched for with a precision of a third value greater than a value of 2, and matching information about the precision of the motion vectors and motion vectors of the first block and the second block are obtained. An image decoding apparatus for decoding an image based on information is provided.

In another aspect of the present invention for achieving the above object, there is provided an image encoding method related to a reference pixel configuration in intra prediction, comprising: obtaining a reference pixel of the current block from a neighboring block in intra prediction for the current block; A step of adaptively filtering a reference pixel, generating a prediction block of the current block by using the reference pixel to which the adaptive filtering is applied as an input value according to the prediction mode of the current block, post-processing filter adaptive to the prediction block An image encoding method is provided, comprising the step of applying

Here, the obtaining may include obtaining a reference pixel of the current block from the neighboring block.

Here, the obtaining step may be determined according to the availability of a neighboring block.

Here, the availability of the neighboring block may be determined by the position of the neighboring block and/or a specific flag (constrained_intra_pred_flag). As an example, a specific flag may have a value of 1 when a neighboring block is available. This may mean that when the prediction mode of the neighboring block is the inter-picture mode, the reference pixel of the corresponding block cannot be used for prediction of the current block.

Here, the specific flag constrained_intra_pred_flag is determined according to the prediction mode of the neighboring block, and the prediction mode may be either intra prediction or inter prediction.

Here, when the specific flag (constrained_intra_pred_flag) is 0, the availability of the neighboring block becomes 'true' regardless of the prediction mode of the neighboring block, and when the specific flag (constrained_intra_pred_flag) is 1, the prediction mode of the neighboring block is 'true' if the prediction mode is intra prediction. ', and in the case of inter-picture prediction, the availability of neighboring blocks may be 'false'.

Here, the inter prediction may generate a prediction block by referring to one or more reference pictures.

Here, the reference picture is managed through the reference picture list 0 (List 0) and the reference picture list 1 (List 1), and one or more past pictures, future pictures, and current pictures may be included in List 0 and List 1.

Here, in List 0 and List 1, whether to put the current picture in the reference picture list may be adaptively determined.

Here, information for determining whether to put the current picture into the reference picture list may be included in a sequence, a reference picture parameter set, and the like.

In another aspect of the present invention for achieving the above object, there is provided an image decoding method performed in a computing device, the method comprising: obtaining, from an input bitstream, a flag regarding the availability of reference pixels of a neighboring block in units of sequences or pictures; determining the reference pixel availability of a neighboring block when performing intra prediction according to the flag; When the flag is 0, the reference pixel of the neighboring block is used for prediction of the current block regardless of the prediction mode of the neighboring block, and when the flag is 1, the reference pixel of the neighboring block is used when the prediction mode of the neighboring block is intra prediction. Provided is an image decoding method that is used for prediction of a current block and does not use reference pixels of a neighboring block for prediction of the current block when the prediction mode of the neighboring block is inter prediction.

Here, the inter prediction may generate a prediction block based on block matching in the reference picture.

Here, the reference picture may be managed through List 0 in the P picture and List 0 and List 1 in the B picture.

Here, the current picture may be included in List 0 in inter prediction.

Here, the current picture may be included in List 1 in inter prediction.

Here, the inclusion of the current picture in List 0 and List 1 may be determined based on a flag transmitted from a sequence parameter.

Here, the inclusion of the current picture in List 0 and List 1 may be determined based on a flag transmitted from a picture parameter.

When the image encoding and decoding method and the image decoding apparatus using motion vector precision according to the embodiment of the present invention as described above are used, the performance of the image prediction system can be improved. That is, performance in image encoding and decoding can be improved by copying a reference block in the current picture and using it as a prediction block.

In addition, the current picture in the reference picture list 0 (List 0) and the reference picture list 1 (List 1) of the motion vector while using intra block copy or block matching to extend the precision of the motion vector performance and efficiency in encoding and decoding can be improved by adding

1 is a diagram for explaining a system using an image encoding apparatus and/or an image decoding apparatus of the present invention.
2 is a block diagram of an image encoding apparatus according to an embodiment of the present invention.
3 is a block diagram of an image decoding apparatus according to an embodiment of the present invention.
4 is an exemplary diagram illustrating inter prediction of a P slice in an image encoding and decoding method according to an embodiment of the present invention.
5 is an exemplary diagram illustrating inter prediction of a B slice in a video encoding and decoding method according to an embodiment of the present invention.
6 is an exemplary diagram for explaining a case of generating a prediction block unidirectionally in an image encoding and decoding method according to an embodiment of the present invention.
7 is an exemplary diagram of configuring a reference picture list in an image encoding and decoding method according to an embodiment of the present invention.
8 is an exemplary diagram illustrating another example of performing inter prediction from a reference picture list in an image encoding and decoding method according to an embodiment of the present invention.
9 is an exemplary diagram for explaining intra prediction in an image encoding method according to an embodiment of the present invention.
10 is an exemplary diagram for explaining a prediction principle in a P slice or a B slice in an image encoding method according to an embodiment of the present invention.
11 is an exemplary diagram for explaining a case in which interpolation is performed in the image encoding method of FIG. 10 .
12 is a diagram for explaining a main process of an image encoding method according to an embodiment of the present invention with syntax in a coding unit.
13 is a diagram illustrating an example of supporting symmetric type partitioning or asymmetric type partitioning as in inter prediction when a prediction block is generated through block matching in the current picture used in FIG. 12 . It is also an example.
14 is an exemplary diagram for explaining that 2Nx2N and NxN can be supported in the inter prediction (Inter) like the intra prediction (Intra) of FIG. 9 .
15 is a diagram for explaining a process of performing a horizontal 1D filter on pixels at positions a, b, and c (assumed to be x) of an image in the image encoding method according to an embodiment of the present invention.
16 is an exemplary diagram of a current block and neighboring blocks that can be employed in an image encoding method according to an embodiment of the present invention.
17 is an example of a reference structure for a random access mode in an image encoding and decoding method according to an embodiment of the present invention.
18 is a diagram for explaining that one picture may have two or more interpolation accuracies in an image encoding and decoding method according to an embodiment of the present invention.
19 is a diagram illustrating a reference picture list when the current picture is an I picture in FIG. 18 .
20 is a diagram illustrating a reference picture list when the current picture is a P picture in FIG. 18 .
21 is a diagram illustrating a reference picture list when a current picture is B(2) in a video encoding and decoding method according to an embodiment of the present invention.
22 is a diagram illustrating a reference picture list when the current picture is B(5) in the video encoding and decoding method according to an embodiment of the present invention.
23 is a diagram for explaining a process in which motion vector precision of each block is determined according to interpolation precision of a reference picture in a video encoding and decoding method according to an embodiment of the present invention.
24 is a diagram for explaining a process in which motion vector precision of each block is adaptively determined when the interpolation precision of each reference picture is fixed in the video encoding and decoding method according to an embodiment of the present invention.

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, and it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention.

Terms such as first, second, A, and B may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

When an element is referred to as being “connected” or “connected” to another element, it is understood that it may be directly connected or connected to the other element, but other elements may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as "comprises" or "having" are intended to designate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in a commonly used dictionary should be interpreted as being consistent with the meaning of the context of the related art, and are not interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application.

In general, a moving picture may be composed of a series of pictures, and each picture may be divided into a predetermined area such as a frame or a block. In addition, the divided region includes a block as well as a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), and a transform unit (TU). It can be referred to by various sizes or terms, such as Each unit may include one luminance block and two color difference blocks, which may be configured differently according to a color format. Also, sizes of the luminance block and the chrominance block may be determined according to the color format. For example, in the case of 4:2:0, the size of the chrominance block may have a length equal to 1/2 of the width and length of the luminance block. For this unit and terminology, conventional terms such as high efficiency video coding (HEVC) or H.264/advanced video coding (H.264/AVC) may be referred to.

In addition, a picture, block, or pixel referenced for encoding or decoding a current block or a current pixel is referred to as a reference picture, a reference block, or a reference pixel. In addition, it is common knowledge in the technical field to which this embodiment belongs that the term “picture” described below may be used in place of other terms having the same meaning, such as an image and a frame. Grow up to understand

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. In describing the present invention, in order to facilitate the overall understanding, the same reference numerals are used for the same components in the drawings, and duplicate descriptions of the same components are omitted.

1 is a diagram for explaining a system using an image encoding apparatus and/or an image decoding apparatus of the present invention.

1 , a system using an image encoding apparatus and/or an image decoding apparatus includes a personal computer (PC), a notebook computer, a personal digital assistant (PDA), a portable multimedia player, PMP), PlayStation portable (playstation portable, PSP), wireless communication terminal (wireless communication terminal), smart phone (smart phone), a user terminal 11 such as a television (TV) or a server terminal such as an application server and a service server ( 12) can be Such a system may be referred to as a computing device.

In addition, the computing device is a communication device such as a communication modem for performing communication with various devices or a wired/wireless network, encoding or decoding an image, or inter or intra prediction for encoding and decoding. It may include various devices including a memory 18 for storing various programs and data, a processor 14 for executing and controlling the program, and the like.

In addition, the computing device transmits an image encoded as a bitstream by an image encoding device in real time or in non-real time through a wired or wireless communication network such as the Internet, a local area wireless network, a wireless LAN network, a WiBro network, or a mobile communication network, or through a cable, general purpose It may be transmitted to an image decoding apparatus through various communication interfaces such as a serial bus (USB: Universal Serial Bus), and decoded in the image decoding apparatus and reproduced as a restored image. Also, an image encoded as a bitstream by the image encoding apparatus may be transmitted from the encoding apparatus to the decoding apparatus through a computer-readable recording medium.

2 is a block diagram of an image encoding apparatus according to an embodiment of the present invention. 3 is a block diagram of an image decoding apparatus according to an embodiment of the present invention.

As shown in FIG. 2 , the image encoding apparatus 20 according to the present embodiment includes a prediction unit 200 , a subtraction unit 205 , a transform unit 210 , a quantization unit 215 , and an inverse quantization unit 220 . , an inverse transform unit 225 , an adder 230 , a filter unit 235 , a decoded picture buffer (DPB) 240 , and an entropy encoder 245 . Also, the image encoding apparatus 20 may further include a division unit 190 .

In addition, as shown in FIG. 3 , the image decoding apparatus 30 according to the present embodiment includes an entropy decoding unit 305 , a prediction unit 310 , an inverse quantization unit 315 , an inverse transform unit 320 , and an addition unit. 325 , a filter unit 330 , and a decoded picture buffer 335 may be included.

The above-described image encoding apparatus 20 and image decoding apparatus 30 may be separate apparatuses, respectively, but may be made into one image encoding and decoding apparatus according to implementation. In this case, the prediction unit 200 , the inverse quantization unit 220 , the inverse transform unit 225 , the adder 230 , the filter unit 235 , and the decoded picture buffer 240 of the video encoding apparatus 20 are in the order described. As a technical element substantially the same as the prediction unit 310, the inverse quantization unit 315, the inverse transform unit 320, the adder 325, the filter unit 330, and the memory 335 of the image decoding apparatus 30, It may be implemented to include at least the same structure or to perform at least the same function. Also, when the entropy encoder 245 performs its function in reverse, it may correspond to the entropy decoder 305 . Therefore, in the detailed description of the following technical elements and their operating principles, overlapping descriptions of corresponding technical elements will be omitted.

In addition, since the image decoding apparatus corresponds to a computing device that applies the image encoding method performed by the image encoding apparatus to decoding, the image encoding apparatus will be mainly described in the following description.

The computing device may include a memory that stores a program or software module implementing the image encoding method and/or the image decoding method, and a processor connected to the memory to execute the program. In addition, the image encoding apparatus may be referred to as an encoder, and the image decoding apparatus may be referred to as a decoder, respectively.

Each component of the video encoding apparatus of the present embodiment will be described in more detail as follows.

The division unit 190 divides the input image into blocks (M×N) of a predetermined size. Here, M or N is any natural number greater than or equal to 1.

In detail, the dividing unit 190 may include a picture dividing unit and a block dividing unit. The size or shape of the block may be determined according to the characteristics and resolution of the image, and the size or shape of the block supported through the picture divider is an M×N square shape (256) in which the horizontal and vertical lengths are expressed as powers of two. ×256, 128×128, 64×64, 32×32, 16×16, 8×8, 4×4, etc.) or M×N rectangular shape. For example, the input image may be divided into sizes such as 256×256 for a high-resolution 8k UHD video, 128×128 for a 1080p HD video, and 16×16 for a WVGA video.

Information on the size or shape of such a block can be set in units such as sequence, picture, and slice, and related information can be transmitted to the decoder. That is, it may be set in a sequence parameter set, a picture parameter set, a slice header, or a combination unit thereof.

Here, a sequence refers to a structural unit configured by collecting several related scenes. In addition, a picture is a term that refers to a series of luminance (Y) components or all luminance and chrominance (Y, Cb, Cr) components in one scene or picture, and the range of one picture is one frame or one field in some cases. can be

A slice may refer to one independent slice segment and a plurality of dependent slice segments existing in the same access unit. The access unit refers to a set of network abstraction layer (NAL) units related to one coded picture. The NAL unit is a syntax structure that composes a video compression bitstream in a network-friendly format in the H.264/AVC and HEVC standards. It is common to configure one slice unit as one NAL unit, and the system standard generally regards the NAL or NAL set constituting one frame as one access unit.

Returning to the description of the picture division unit, information on the block size or shape (M×N) may consist of an explicit flag, specifically block shape information, one length information in case the block is a square, and a rectangular one. In this case, each length information or information on a difference value between horizontal and vertical lengths may be included. For example, when M and N are composed of k to the power of k (assuming k is 2) (M=2 ^m , N=2 ⁿ ), information about m and n is converted into unary binarization and truncation unary binarization. The related information may be transmitted to the decoding apparatus by encoding in various methods such as.

In addition, the minimum allowable division size (Minblksize) supported by the picture division unit is I×J (for convenience of explanation, it is assumed that I=J. In the case of I=2 ⁱ ,J=2 ^j ), information on mi or nj is transmitted. can As another example, when M and N are different, the difference value (|mn|) between m and n may be transmitted. Alternatively, the maximum allowable division size (Maxblksize) supported by the picture division unit is I×J (for convenience of explanation, it is assumed that I=J. If I=2 ⁱ ,J=2 ^j ), information about im or nj is transmitted. can

In an implicit situation, for example, when syntax for related information exists but cannot be confirmed by the encoder/decoder, the encoder or decoder may follow a pre-prepared default setting. For example, when the related syntax cannot be checked in the step of checking the block shape information, the block shape may be set to a basic square shape. In addition, in the step of checking the block size information, more specifically, in the step of checking the block size information through the difference value from the minimum allowable partition size (Minblksize) as in the above example, syntax related to the difference value can be checked, but the division If the syntax related to the minimum allowable size (Minblksize) cannot be confirmed, it can be obtained from a preset value related to the minimum allowable partition size (Minblksize).

In this way, the size or shape of a block in the picture divider may be implicitly determined according to characteristics and resolution of an image or the like by explicitly transmitting related information from an encoder and/or a decoder.

As described above, the block divided and determined through the picture divider may be used as a basic coding unit. In addition, the block divided and determined through the picture divider may be a minimum unit constituting a higher-level unit such as a picture, a slice, and a tile, and may include a coding block, a prediction block, and a transform block. (transform block), quantization block (quantization block), entropy block (entropy block), may be a maximum unit such as an inloop filtering block (inloopfiltering block), some blocks are not limited thereto, and exceptions are possible. For example, some such as in-loop filtering blocks may be applied in units larger than the block size described above.

The block divider divides blocks such as encoding, prediction, transformation, quantization, entropy, and in-loop filter. The division unit 190 is included in each component to perform a function. For example, the transform unit 210 may include a transform block division unit, and the quantization unit 215 may include a quantization block division unit. The size or shape of the initial block of the block division unit may be determined by the division result of a previous stage or higher-level block.

For example, in the case of a coding block, a block obtained through a picture dividing unit, which is a previous step, may be set as an initial block. Alternatively, in the case of a prediction block, a block obtained through a process of dividing a coding block that is a higher level of the prediction block may be set as an initial block. Alternatively, in the case of a transform block, a block obtained through a process of dividing a coding block that is a higher level of the transform block may be set as an initial block.

The conditions for determining the size or shape of the initial block are not always fixed, and some may be changed or there may be exceptions. In addition, the partition state of the previous stage or higher level block (eg, the size of the coding block, the type of the coding block, etc.) and the setting condition of the current level (eg, the size of the supported transform block, the type of the transform block, etc.) ) may affect the current level division operation (whether division is possible, the type of a block that can be divided, etc.) according to a combination of at least one or more factors, respectively.

The block divider may support a quad tree-based division method. That is, it can be divided into four blocks each having a length of 1/2 each in width and length from the block before division. This is based on the initial block (dep_0), and the limit of the allowed depth of division (dep_k, k means the number of allowed divisions, and the block size when the limit of the allowed depth of division (dep_k) is (M >> k, N >> k)) The division can be repeated until

In addition, a binary tree-based partitioning method may be supported. This indicates that it can be divided into two blocks having a length of one-half compared to the block before division, either horizontally or vertically. In the case of the quad-tree partitioning and binary tree partitioning, symmetric partitioning or asymmetric partitioning may be used, and it is possible to determine which partitioning method to follow according to the setting of the encoder/decoder. In the image encoding method of the present invention, the description will be focused on the symmetric division method.

Whether or not each block is divided can be indicated through the division flag (div_flag). If the corresponding value is 1, division is performed, and if the value is 0, division is not performed. Alternatively, if the corresponding value is 1, partitioning is performed and further partitioning is possible. If the value is 0, partitioning is not performed and further partitioning may not be allowed. According to conditions such as the minimum allowable division size and the limit of the allowable division depth, the flag may only consider whether to divide and not consider whether to further divide.

The split flag can be used for quad-tree splitting and can also be used for binary tree splitting. In binary tree partitioning, the partitioning direction may be one of the partition depth, encoding mode, prediction mode, size, shape, type (coding, prediction, transformation, quantization, entropy, in-loop filter, etc.) may be one) and may be determined according to at least one or a combination of factors such as a slice type, a partition allowable depth limit, and a partition allowable minimum/maximum size. In addition, according to the division flag and/or according to the division direction, that is, only the width of the block may be divided into 1/2 or only the length of the block may be divided into 1/2.

For example, if a block is M×N(M>N), and horizontal division is supported when M is greater than N, and the current division depth (dep_curr) is smaller than the division allowable depth limit, further division is possible. The division flag is allocated as 1 bit, and if the corresponding value is 1, horizontal division is performed, and if it is 0, no further division may be performed. As for the division depth, one division depth may be provided for quad tree and binary tree division, and each division depth may be provided for quad tree and binary tree division. In addition, as for the allowable partitioning depth limit, one partitioning allowable depth limit may be placed on quad tree and binary tree partitioning, and each partitioning allowable depth limit may be placed on quad tree and binary tree partitioning.

As another example, if the block is M×N (M>N) and N is equal to the preset minimum allowable size for division, so horizontal division is not supported, the above division flag is assigned as 1 bit and if the value is 1, vertical division is performed, , if it is 0, no partitioning is performed.

In addition, flags div_h_flag and div_h_flag for horizontal division or vertical division may be supported, respectively, and binary division may be supported according to the flags. Horizontal or vertical division of each block can be indicated through the horizontal division flag (div_h_flag) or vertical division flag (div_v_flag). If the horizontal division flag (div_h_flag) or vertical division flag (div_v_flag) is 1, horizontal or vertical division is performed. If it is 0, horizontal or vertical division is not performed.

In addition, if each flag is 1, horizontal or vertical division is performed and additional horizontal or vertical division is possible. If the value is 0, horizontal or vertical division is not performed and further horizontal or vertical division is not allowed. . Depending on conditions such as the minimum allowable partition size and the limit of the allowable partitioning depth, the flag may consider whether to split and not consider whether to further split.

In addition, flags (div_flag/h_v_flag) for horizontal division or vertical division may be supported, and binary division may be supported according to the flag. The division flag div_flag may indicate horizontal or vertical division, and the division direction flag h_v_flag may indicate a horizontal or vertical division direction.

When the division flag div_flag is 1, division is performed, horizontal or vertical division is performed according to the division direction flag h_v_flag, and when 0, horizontal or vertical division is not performed. In addition, if the value is 1, horizontal or vertical division is performed according to the division direction flag (h_v_flag) and additional horizontal or vertical division is possible. If the value is 0, horizontal or vertical division is not performed and further horizontal or vertical division is possible. may be considered as not permitted.

Depending on conditions such as the minimum allowable partition size and the limit of the allowable partitioning depth, the flag may consider whether to split and not consider whether to further split.

These division flags may also be supported for horizontal and vertical division, respectively, and binary tree division may be supported according to the flag. In addition, when the division direction is predetermined, as in the above example, only one of the division flags or both division flags may be used.

For example, if all of the above flags are allowed, the possible block shape may be divided into any one of M×N, M/2×N, M×N/2, and M/2×N/2. In this case, the flags may be encoded as 00, 10, 01, and 11 in the order of the horizontal division flag or the vertical division flag (div_h_flag/div_v_flag).

In the above case, it is an example in a setting in which the division flags can be overlapped and used, and a setting in which the division flags cannot be overlapped and used is also possible. For example, the divided block shape may be divided into M×N, M/2×N, and M×N/2. In this case, the flags above are encoded as 00, 01, and 10 in the order of horizontal or vertical division flags. Alternatively, a division flag (div_flag) and a horizontal-vertical flag (h_v_flag, this flag indicating a horizontal or vertical division direction) may be encoded as 0, 10, and 11 in the order. Here, the meaning of overlap may mean performing horizontal division and vertical division at the same time.

Any one of the above-described quad-tree splitting and binary tree splitting may be used alone or in combination according to settings of an encoder and/or a decoder. For example, the quadtree or binary tree division may be determined according to the size or shape of the block. That is, when the block shape is M×N and M is greater than N, horizontal division, the block shape is M×N, and when N is greater than M, binary tree division may be supported according to vertical division, and block type In the case where is M×N and N and M are the same, quadtree splitting may be supported.

As another example, binary tree splitting may be supported when the size of a block (M×M) is greater than or equal to a block splitting boundary value thrblksize, and quadtree splitting may be supported if it is smaller than that.

As another example, when M or N of a block (M×N) is less than or equal to the first allowable maximum size for partitioning (Maxblksize1) and greater than or equal to the first minimum allowable size for partitioning (Minblksize1), quad tree partitioning is supported, and the block Binary tree splitting may be supported when M or N of (M×N) is less than or equal to the second allowable maximum size for splitting (Maxblksize2) and greater than or equal to the second minimum allowable size for splitting (Minblksize2).

If the first division support range and the second division support range that can be defined by the maximum division size and the minimum division size overlap, the first or the second division method takes precedence according to the setting of the encoder/decoder. A ranking may be given. In this embodiment, the first partitioning method may be quad-tree partitioning, and the second partitioning method may be binary tree partitioning, for example. For example, if the first partition allowable minimum size (Minblksize1) is 16, the second partition allowable maximum size (Maxblksize2) is 64, and the block before division is 64×64, the first partition support range and the second partition support range are Since they belong to all, quad tree splitting and binary tree splitting are possible.

If priority is given to the first division method (quad tree division in this embodiment) according to a preset, when the division flag (div_flag) is 1, quad tree division is performed and additional quad tree division is possible, and 0 In this case, quad-tree splitting is not performed and it can be considered that quad-tree splitting is no longer performed. According to conditions such as the minimum allowable division size and the limit of the allowable division depth, the flag may only consider whether to divide and not consider whether to further divide.

When the division flag div_flag is 1, it is divided into four blocks having a size of 32×32 and is larger than the first allowable minimum size for division (Minblksize1), so that quad tree division can be continued. When the split flag is 0, additional quad tree splitting is not performed, and since the current block size (64×64) falls within the second split support range, binary tree splitting can be performed. When the division flag (in the order of div_flag/h_v_flag) is 0, no further division is performed, and when it is 10 or 11, horizontal division or vertical division can be performed.

If the block before division is 32x32 and the division flag (div_flag) is 0 so that no quad tree division is performed and the second division maximum size (Maxblksize2) is 16, the size of the current block (32x32) is Since it does not belong to the 2 partition support range, it may not support further partitioning. In the above description, the priority of the partitioning method may be determined according to at least one factor among a slice type, an encoding mode, a luminance/chrominance component, or the like, or a combination thereof.

As another example, various settings may be supported according to luminance and chrominance components. For example, a quad-tree or binary-tree split structure determined in the luminance component may be used as it is in the chrominance component without additional information encoding/decoding. Alternatively, if independent division of the luminance component and the chrominance component is supported, a quad tree and a binary tree may be supported together for the luminance component, and quad tree division may be supported for the chrominance component.

In addition, quad tree division and binary tree division are supported in the luminance and chrominance components, but the division support range may or may not be the same or proportional to the luminance and chrominance components. For example, when the color format is 4:2:0, the division support range of the chrominance component may be N/2 of the division support range of the luminance component.

As another example, different settings may be set according to the slice type. For example, I slice may support quad tree splitting, P slice may support binary tree splitting, and B slice may support quad tree splitting and binary tree splitting together.

As in the above example, quad tree splitting and binary tree splitting may be set and supported according to various conditions. The above examples are not specific only to the above cases, and may include cases in which each other's conditions are reversed, and may include one or more factors mentioned in the above examples or a combination thereof. possible. The above partition allowable depth limit may be determined according to at least one factor or a combination thereof, such as a partitioning method (quad tree, binary tree), a slice type, a luminance/chrominance component, an encoding mode, and the like.

In addition, the division support range may be determined according to at least one factor or a combination thereof in the division method (quad tree, binary tree), slice type, luminance/chrominance component, encoding mode, etc. It can be expressed as a maximum value and a minimum value. When this information is configured as an explicit flag, information on the length of each of the maximum/minimum values or information on the difference between the minimum and maximum values may be expressed.

For example, when the maximum value and the minimum value are configured to the power of k (assuming k is 2), exponent information of the maximum value and the minimum value may be encoded through various binarizations and transmitted to the decoding apparatus. Alternatively, a difference value between the exponents of the maximum value and the minimum value may be transmitted. In this case, the transmitted information may be index information of the minimum value and information on the difference value of the index.

According to the above description, flag-related information may be generated and transmitted in units of a sequence, a picture, a slice, a tile, a block, and the like.

The split flags presented as examples above can indicate block split information through a quad tree or binary tree or a mixture of both tree methods, and the split flag is encoded using various methods such as unary binarization and truncation unary binarization to decode related information. can be passed to the device. A bitstream structure of a partition flag for expressing partition information of the block may be selected from one or more scan methods.

For example, a bitstream of the division flags may be configured based on the division depth order (in dep0 to dep_k order), and a bitstream of division flags may be configured based on whether division is performed. In the division depth order reference method, division information at the current level is obtained based on the first block, and division information at the next level is obtained. It refers to a method of preferentially acquiring additional segmentation information, and in addition to this, other scan methods not presented in the above example may be included and selected.

In addition, depending on implementation, the block partitioning unit may generate and express index information on a predefined type of block candidate group instead of the aforementioned partitioning flag. The shape of the block candidate group is, for example, the shape of a divided block that can be in the block before division, such as M×N, M/2×N, M×N/2, M/4×N, 3M/4×N, It may include M×N/4, M×3N/4, M/2×N/2, and the like.

When the candidate group of the divided block is determined as described above, the index information of the divided block type can be encoded through various methods such as fixed-length binarization, short-cut binarization, and truncated binarization. Like the partition flag described above, at least one or a combination of factors such as the partition depth, encoding mode, prediction mode, size, shape, type, and slice type, partition allowable depth limit, and partition allowable minimum/maximum size of the block Accordingly, a partition block candidate group may be determined.

For the following description, (M×N, M×N/2) is a candidate list 1 (list1), (M×N, M/2×N, M×N/2, M/2×N/2) is a candidate List 2 (list2), (M×N, M/2×N, M×N/2) to candidate list 3 (list3), (M×N, M/2×N, M×N/2, M/ 4xN, 3M/4xN, MxN/4, Mx3N/4, M/2xN/2) are assumed as candidate list 4 (list4). For example, when describing based on M×N, when (M=N), the partition block candidate of candidate list2 can be supported, and when (M≠N), the partition block candidate of candidate list3 can be supported.

As another example, when M or N of M×N is greater than or equal to the boundary value blk_th, the divided block candidate of candidate list2 may be supported, and when smaller than that, the divided block candidate of candidate list4 may be supported. Also, when M or N is greater than or equal to the first boundary value blk_th_1, the partition block candidate of the candidate list1 is selected as a candidate when smaller than the first boundary value blk_th_1 but greater than or equal to the second boundary value blk_th_2. When the divided block candidate of list2 is smaller than the second boundary value blk_th_2, the divided block candidate of candidate list4 may be supported.

As another example, if the encoding mode is intra prediction, the split block candidate of candidate list2 may be supported, and if the encoding mode is inter prediction, the split block candidate of candidate list4 may be supported.

Even if the split block candidates as described above are supported, the bit configuration according to binarization in each block may be the same or different. For example, if the supported partition block candidates are limited according to the block size or shape as in the application of the partition flag above, the bit configuration according to the binarization of the corresponding block candidate may vary. For example, in the case of (M>N), block types according to horizontal division, that is, M×N, M×N/2, M/2×N/2 can be supported, and a group of divided block candidates (M×N, M/ The binary bits of the index according to M×N/2 in 2×N, M×N/2, M/2×N/2) and M×N/2 of the current condition may be different from each other.

According to the type of block, for example, the type of block used for encoding, prediction, transformation, quantization, entropy, in-loop filtering, etc., information on the division and shape of the block may be expressed using either a division flag or a division index method. In addition, depending on the type of each block, a block size limit for partitioning and block shape support, a partition allowable depth limit, etc. may be different.

In the block-by-block encoding and decoding process, after a coding block is first determined, encoding and decoding may be performed according to processes such as prediction block determination, transform block determination, quantization block determination, entropy block determination, and in-loop filter determination. The order of the above encoding and decoding processes is not always fixed, and some orders may be changed or excluded. The size and shape of each block may be determined according to the encoding cost of each candidate for the size and shape of the block, and partition-related information such as image data of each determined block and the determined size and shape of each block may be encoded.

The prediction unit 200 may be implemented using a prediction module, which is a software module, and may generate a prediction block with respect to a block to be encoded using an intra prediction method or an inter prediction method. Here, the prediction block is a block that is understood to closely match the block to be encoded in terms of pixel difference, and may be determined by various methods including sum of absolute difference (SAD) and sum of square difference (SSD). Also, in this case, various syntaxes that can be used when decoding image blocks may be generated. The prediction block may be classified into an intra-picture block and an inter-picture block according to an encoding mode.

Intra prediction is a prediction technique using spatial correlation, and refers to a method of predicting a current block using reference pixels of previously encoded, decoded, and reconstructed blocks within the current picture. That is, the brightness value reconstructed through intra prediction and restoration may be used as a reference pixel in the encoder and the decoder. The intra prediction may be effective for a flat area having continuity and a region having a constant direction, and may be used for the purpose of ensuring random access and preventing error diffusion because spatial correlation is used.

Inter prediction uses a compression technique that removes data duplication by using temporal correlation with reference to an image encoded in one or more past or future pictures. That is, the inter prediction may generate a prediction signal with high similarity by referring to one or more past or future pictures. An encoder using inter prediction may search a reference picture for a block having a high correlation with a block to be currently encoded, and transmit location information and a residual signal of the selected block to a decoder, and the decoder may select information on the transmitted image can be used to generate the same prediction block as the encoder and compensate for the transmitted residual signal to compose a reconstructed image.

4 is an exemplary diagram illustrating inter prediction of a P slice in an image encoding and decoding method according to an embodiment of the present invention. 5 is an exemplary diagram illustrating inter prediction of a B slice in a video encoding and decoding method according to an embodiment of the present invention.

In the video encoding method of the present embodiment, since inter prediction generates a prediction block from a previously encoded picture having high temporal correlation, encoding efficiency may be increased. Current(t) may mean a current picture to be encoded, and based on a temporal flow of video pictures or a picture order count (POC), a first temporal distance (t-1) before the POC of the current picture It may include a first reference picture having a and a second reference picture having a second temporal distance t-2 before the first temporal distance.

That is, as shown in FIG. 4 , inter prediction that can be employed in the video encoding method of the present embodiment is performed between the current block of the current picture current(t) and the reference pictures t-1 and t-2. Through block matching of reference blocks, motion estimation may be performed to find an optimal prediction block from reference pictures (t-1, t-2) in which blocks with high correlation have been previously encoded. After performing an interpolation process based on a structure in which at least one or more sub-pixels are arranged between two adjacent pixels as needed for precise estimation, an optimal prediction block is found and motion compensation is performed to obtain a final prediction block can be found

In addition, as shown in FIG. 5 , the inter prediction that can be employed in the video encoding method of the present embodiment includes reference pictures ( A prediction block may be generated from t-1, t+1). Also, two prediction blocks may be generated from one or more reference pictures.

When an image is encoded through inter prediction, motion vector information on an optimal prediction block and information on a reference picture are encoded. In the present embodiment, when a prediction block is generated unidirectionally or bidirectionally, a reference picture list may be configured differently to generate a prediction block from the reference picture list. Basically, a reference picture existing temporally before the current picture may be managed by allocating it to a list 0 (L0), and a reference picture existing after the current picture may be allocated and managed by allocating it to a list 1 (L1).

When composing the reference picture list 0, if the reference picture list 0 does not fill up to the allowable number of reference pictures, reference pictures existing after the current picture may be allocated. Similarly, when composing the reference picture list 1, if the reference picture list 1 does not fill up to the allowable number of reference pictures, a reference picture existing before the current picture may be allocated.

6 is an exemplary diagram for explaining a case of generating a prediction block unidirectionally in an image encoding and decoding method according to an embodiment of the present invention.

Referring to FIG. 6 , in the image encoding and decoding method according to the present embodiment, a prediction block can be found from previously encoded reference pictures t-1 and t-2, and in addition to the current picture (current( t)), the prediction block can be found from the region that has already been coded.

That is, in the video encoding and decoding method according to the present embodiment, not only the prediction block is generated from the previously coded pictures t-1 and t-2 having high temporal correlation, but also the prediction block with high spatial correlation can be implemented to find together. Finding such a spatially highly correlated prediction block may correspond to finding the prediction block in the manner of intra prediction. In order to perform block matching from an encoded region in the current picture, the image encoding method of the present embodiment may be mixed with an intra prediction mode to construct a syntax for information related to a prediction candidate.

For example, when n (n is an arbitrary natural number) intra-prediction modes are supported, one mode is added to the intra-prediction candidate group to support n+1 modes, and 2 ^M-1 ≤ n+1 A prediction mode may be coded using M fixed bits satisfying <2 ^M. In addition, it may be implemented to select from a candidate group of a highly probable prediction mode, such as the most probable mode (MPM) of HEVC. In addition, encoding may be preferentially performed in a higher stage of prediction mode encoding.

When a prediction block is generated through block matching in the current picture, the image encoding method of the present embodiment may configure a syntax for related information by mixing with the inter prediction mode. As additional related prediction mode information, motion or displacement related information may be used. The motion or movement-related information may include optimal candidate information among multiple vector candidates, a difference value between an optimal candidate vector and an actual vector, a reference direction, reference picture information, and the like.

7 is an exemplary diagram of configuring a reference picture list in an image encoding and decoding method according to an embodiment of the present invention. 8 is an exemplary diagram illustrating another example of performing inter prediction from a reference picture list in an image encoding and decoding method according to an embodiment of the present invention.

Referring to FIG. 7 , in the image encoding method according to the present embodiment, a first reference picture list (reference list 0, L0) and a second reference picture list (reference list) with respect to a current block of a current picture (current(t)) 1, L1), inter prediction can be performed.

Referring to FIGS. 7 and 8 , the reference picture list 0 may consist of reference pictures before the current picture (t), where t-1 and t-2 are the first first before the POC of the current picture (t), respectively. Reference pictures having a temporal distance t-1 and a second temporal distance t-2 are indicated. In addition, the reference picture list 1 may consist of reference pictures after the current picture (t), where t+1 and t+2 are respectively a first temporal distance (t+1) after the POC of the current picture (t). ), indicating reference pictures having a second temporal distance (t+2).

The above-described examples of the reference picture list configuration show an example of configuring the reference picture list with reference pictures having a difference in temporal distance (based on POC in this example) of 1, but the difference in temporal distance between reference pictures is configured differently You may. That is, it means that the index difference of the reference pictures and the temporal distance difference of the reference pictures may not be proportional. In addition, the list construction order may not be configured based on a temporal distance. This can be confirmed in the reference picture list configuration example to be described later.

Prediction may be performed from reference pictures in the list according to the slice type (I, P, or B). And when a prediction block is generated through block matching in the current picture (current(t)), encoding is performed in an inter prediction method by adding the current picture to a reference picture list (reference list 0 and/or reference list 1). can

As shown in FIG. 8 , a current picture t may be added to a reference picture list 0 or a current picture current(t) may be added to a reference picture list 1 . That is, the reference picture list 0 can be configured by adding a reference picture having a temporal distance (t) to the reference picture before the current picture (t), and the reference picture list 1 is the reference picture after the current picture (t) in time It can also be configured by adding a reference picture that is a specific distance t.

For example, when constructing reference picture list 0, a reference picture before the current picture may be assigned to reference picture list 0, and then the current picture (t) may be assigned, and when constructing reference picture list 1, a reference picture after the current picture The reference picture may be allocated to the reference picture list 1, and then the current picture (t) may be allocated. Alternatively, the current picture (t) may be allocated when configuring the reference picture list 0, and then the reference picture before the current picture may be allocated, and when configuring the reference picture list 1, the current picture (t) is allocated and then the current picture Subsequent reference pictures may be allocated.

Also, when constructing the reference picture list 0, a reference picture before the current picture may be allocated, then a reference picture after the current picture may be allocated, and the current picture t may be allocated. Similarly, when constructing the reference picture list 1, a reference picture after the current picture may be allocated, then a reference picture before the current picture may be allocated, and the current picture t may be allocated. The above examples are not specific only to the above-described case, and may include a case in which each other's conditions are reversed, and variations in other cases are also possible.

Whether or not to include the current picture in each reference picture list (e.g., do not add to any list or add to list 0 only or add to list 1 only or add to list 0 and 1 together) has the same setting in the encoder/decoder possible, and information on this can be transmitted in units of sequences, pictures, slices, and the like. This information may be encoded through a method such as fixed-length binarization, short cut-type binarization, or cut-type binarization.

Unlike the method of FIG. 7, the image encoding and decoding method of this embodiment selects a prediction block by performing block matching on the current picture t, and constructs a reference picture list including information related to the prediction block, , there is a difference in using this reference picture list for video encoding and decoding.

In the reference picture list configuration, the settings for each list configuration order and rule, and the allowable number of reference pictures in each list can be changed, which is whether the current picture is included in the list (whether the current picture is included as a reference picture in inter prediction) or not), slice type, list reconstruction parameters (which may be applied to lists 0 and 1, respectively, or may be applied to lists 0 and 1 together), location in a GOP (Group of Picture), temporal hierarchical information (temporal id), etc. It may be determined according to at least one of the factors or a combination thereof, and related information may be explicitly transmitted in units of sequences, pictures, or the like.

For example, in the case of a P slice, the reference picture list 0 may follow the list construction rule A regardless of including the current picture in the list, and in the case of a B slice, the reference picture list 0 including the current picture in the list consists of a list Rule B, the list construction rule C may be followed for the reference picture list 1, the list construction rule D for the reference picture list 0 that does not contain the current picture, and the list construction rule E for the reference picture list 1 may be followed, and the list construction rule Among them, B and D and C and E may be the same.

The list construction rule may be configured in the same or modified manner as described in the reference picture list construction example. As another example, when the current picture is included in the list, the allowable number of first reference pictures may be set, and when the current picture is not included in the list, the allowed number of second reference pictures may be set. The allowable number of first reference pictures and the allowable number of second reference pictures may be the same or different, and a difference between the allowable number of first reference pictures and the allowable second reference pictures may be set as a default setting.

As another example, when the current picture is included in the list and the list reconstruction parameter is applied, all reference pictures may become a list reconstruction candidate group in slice A, and only some reference pictures may be included in the list reconstruction candidate group in slice B. At this time, slice A or B can be divided into whether or not the list of current pictures is included, temporal hierarchical information, slice type, position in the GOP, etc. It may be determined by the direction (before/after the current picture), whether the current picture is present, or the like.

According to the above configuration, since the reference block encoded by inter prediction in the current picture can be used, inter prediction can be allowed or used even in I-slice motion prediction.

In addition, when constructing the reference picture list, index allocation or list construction order may be changed according to slice types. In the case of I slice, the current picture (current(t)) uses fewer indexes (eg, idx=0, 1, 2) by increasing the priority as in the reference picture list configuration example above, and the corresponding The amount of bits in image encoding can be reduced through binarization (fixed-length binarization, short-cut binarization, truncated binarization, etc.) in which the allowable number of reference pictures (C) of the reference picture list is the maximum value.

In addition, in the case of a P or B slice, if it is determined that the probability of selecting the reference picture of the current block as a prediction candidate by performing block matching on the current picture is lower than the probability of selecting a prediction candidate through other reference pictures, the current picture Binarization of various methods to set the priority for block matching of It is possible to reduce the amount of bits in video encoding.

In the above example, the priority setting of the current picture may be configured in the same or modified manner as described in the reference picture list configuration example. In addition, it is possible to omit information about the reference picture by not configuring the reference picture list according to the slice type (eg, I slice). For example, a prediction block may be generated through the existing inter prediction, but the inter prediction information may be expressed by excluding the reference picture information from the motion information in the inter prediction mode.

A method of performing block matching in the current picture may determine whether to support it according to a slice type. For example, it may be set that block matching in the current block is supported in the I slice but not in the P slice or the B slice, and a modification to another example is also possible. In addition, the method of supporting block matching in the current picture may be determined in units of pictures, slices, tiles, etc., or may be determined according to a location in a GOP, temporal hierarchical information (temporal ID), and the like. Such setting information may be transmitted in units of sequences, pictures, slices, etc., during an image encoding process or from an encoder to a decoder.

In addition, even when the above-related setting information or syntax exists in the upper level unit and the setting-related operation is on, when the same setting information or syntax as above exists in the lower level unit, the setting information in the lower level unit is displayed at the upper level The setting information in the unit may be given priority. For example, if the same or similar configuration information is processed in a sequence, picture, or slice, a picture unit may have priority over a sequence unit, and a slice unit may have priority over a picture unit.

9 is an exemplary diagram for explaining intra prediction in an image encoding method according to an embodiment of the present invention.

Referring to FIG. 9 , the intra prediction method according to the present embodiment includes reference sample padding, reference sample filtering, intra prediction, and boundary filtering. It may include a series of steps.

The reference pixel filling step may be an example of the reference pixel construction step, the reference pixel filtering step may be executed by the reference pixel filter unit, and the intra prediction may include a prediction block generation step and a prediction mode encoding step, Filtering may be an example of one embodiment of a post-processing filter step.

That is, the intra prediction performed in the image encoding method of the present embodiment may include a reference pixel construction step, a reference pixel filtering step, a prediction block generation step, a prediction mode encoding step, and a post-processing filtering step. Depending on various environmental factors, such as block size, block shape, block position, prediction mode, prediction method, quantization parameter, etc., one or some of the above-described processes may be omitted, and other processes may be added, and the order described above may be changed in an order other than .

The above-described reference pixel construction step, reference pixel filtering step, prediction block generation step, prediction mode encoding step, and post-processing filtering step may be implemented in a form in which a processor connected to the memory executes software modules stored in the memory. Therefore, in the following description, for convenience of explanation, a function unit generated by a combination of a software module implementing each step and a processor executing the same and/or a component performing the function of such a function unit, respectively, a reference pixel component and a reference pixel filter The unit, the prediction block generator, the prediction mode encoder, and the post-processing filter will be referred to as execution subjects of each step.

When each component is described in more detail, the reference pixel component configures a reference pixel to be used for prediction of the current block by filling the reference pixel. If the reference pixel does not exist or is unavailable, the reference pixel fill can be used for the reference pixel, such as by copying a value from an available nearby pixel. A reconstructed picture buffer or a decoded picture buffer (DPB) may be used for copying a value.

That is, the intra prediction is performed using reference pixels of previously encoded blocks of the current picture. To this end, in the reference pixel construction step, neighboring blocks of the current block, that is, adjacent pixels such as left, upper left, lower left, upper, and upper right blocks, are mainly used as reference pixels.

However, the candidate group of the neighboring blocks for the reference pixel is only an example of a case in which the encoding order of the blocks follows a raster scan or a z-scan, and a scan such as an inverse z-scan When used in this coding order scan method, adjacent pixels such as the right, lower right, and lower blocks in addition to the above blocks may be used as reference pixels.

In addition, depending on implementation, additional pixels other than immediately adjacent pixels may be used in combination with replacement or existing reference pixels according to the stepwise configuration of intra prediction.

In addition, when prediction is performed in a directional mode among intra prediction modes, a decimal unit reference pixel may be generated through linear interpolation of an integer unit reference pixel. Modes for performing prediction through reference pixels existing at integer unit positions include some modes having vertical, horizontal, 45 degrees, and 135 degrees, and for the above prediction modes, the process of generating a decimal unit reference pixel is may not be necessary.

A reference pixel interpolated in prediction modes having a direction other than the prediction mode is 1/2, 1/4, 1/8, 1/16, 1/32, 1/64 of an exponential power of 1/2. It may have interpolation precision, or it may have a precision of multiples of 1/2.

This is because interpolation precision may be determined according to the number of supported prediction modes or prediction directions of prediction modes. A picture, a slice, a tile, a block, etc. may always support a fixed interpolation precision, or an adaptive interpolation precision may be supported according to a size of a block, a shape of a block, a prediction direction of a supported mode, and the like. In this case, the prediction direction of the mode may be expressed as inclination information or angle information of the direction indicated by the mode with respect to a specific line reference (eg, positive <+> x-axis on the coordinate plane).

As the interpolation method, linear interpolation is sometimes performed through immediately adjacent integer pixels, but other interpolation methods may be supported. One or more types of filters and the number of taps for interpolation For example, a 6-tap Wiener filter, an 8-tap Kalman filter, etc. can be supported, and it is possible to determine which interpolation to perform according to the size of the block, the prediction direction, etc. there is. In addition, related information may be transmitted in units of sequences, pictures, slices, blocks, and the like.

The reference pixel filter unit may perform filtering on the reference pixel in order to increase prediction efficiency by reducing deterioration remaining in the encoding process after configuring the reference pixel. The reference pixel filter unit may implicitly or explicitly determine the type of filter and whether to apply the filtering according to the size, shape, and prediction mode of the block. That is, filter coefficients may be determined differently depending on the filter type even for filters of the same tap. For example, a 3-tap filter such as [1,2,1]/4 or [1,6,1]/8 can be used.

In addition, the reference pixel filter unit may determine whether to apply filtering by determining whether to additionally transmit a bit or not. For example, in the implicit case, the reference pixel filter unit may determine whether to apply filtering according to characteristics (variance, standard deviation, etc.) of pixels in the neighboring reference block.

Also, the reference pixel filter unit may determine whether to apply filtering when the related flag satisfies a hiding condition preset for a residual coefficient, an intra prediction mode, or the like. The number of taps of the filter is, for example, 3-tap such as [1,2,1]/4 for a small block (blk), and [2,3,6,3,2]/16 and [2,3,6,3,2]/16 for a large block (blk). It may be set to the same 5-tap, and the number of times of application may be determined by not performing filtering, filtering once, filtering twice, and the like.

Also, the reference pixel filter unit may basically apply filtering to the nearest reference pixel of the current block. In addition to the nearest reference pixel, additional reference pixels may also be considered in the filtering process. For example, filtering may be applied to additional reference pixels by replacing the closest reference pixel, or filtering may be applied by mixing additional reference pixels to the closest reference pixel.

The above filtering can be applied statically or adaptively, which includes the size of the current block or the size of the neighboring block, the coding mode of the current block or the neighboring block, and the block boundary characteristics of the current block and the neighboring block (eg, the size of the coding unit). It may be determined according to at least one or a combination of factors such as the boundary or the boundary of the transform unit), the prediction mode or direction of the current block or neighboring block, the prediction method of the current block or neighboring block, and a quantization parameter. there is. A decision on this may have the same setting in the encoder/decoder (implicit), or may be determined in consideration of encoding cost and the like (explicit). The basically applied filter is a low pass filter, and the number of filter taps, filter coefficients, filter flag encoding or not, the number of times to apply the filter, etc. can be determined according to several factors specified above. It can be set in units such as slices and blocks, and related information can be transmitted to the decoder.

In intra prediction, the prediction block generator uses an extrapolation or extrapolation method through a reference pixel, an interpolation method such as an average value (DC) of reference pixels or a planar mode, or a copy of the reference pixel. ) to generate a prediction block.

In the case of copying of reference pixels, one reference pixel may be copied to generate one or more prediction pixels, and one or more reference pixels may be copied to generate one or more prediction pixels, and the number of copied reference pixels is equal to the number of copied reference pixels. It may be equal to or less than the number of prediction pixels.

Also, it is possible to classify the directional prediction method and the non-directional prediction method according to the prediction method, and in detail, the directional prediction method can be classified into a linear direction method and a curved direction method. The linear directional method borrows extrapolation or extrapolation method, but the pixels of the prediction block are generated through reference pixels lying on the prediction direction line. This refers to a method in which a partial prediction direction change in pixel units is allowed in consideration of the detailed directionality (eg, edge<Edge>) of a block although generated through pixels.

In the case of the directional prediction mode in the image encoding and decoding method of the present embodiment, the description will be focused on the linear directional method.

In addition, in the case of the directional prediction method, the spacing between adjacent prediction modes may be uniform or non-uniform, which may be determined according to the size or shape of a block. For example, when a block having a size and shape of M×N is obtained through the block divider, when M and N are the same, the interval between prediction modes may be uniform, and when M and N are different Intervals between prediction modes may be non-uniform.

As another example, when M is greater than N, modes having vertical direction may allocate a finer interval between prediction modes close to the vertical mode (90 degrees), and allocate a wide interval to prediction modes farther from the vertical mode. . When N is greater than M, a finer interval may be allocated between prediction modes close to the horizontal mode (180 degrees) for modes having horizontal directivity, and a wide interval may be allocated to a prediction mode far from the horizontal mode.

The above examples are not specific only to the above-described case, and may include a case in which each other's conditions are reversed, and variations in other cases are also possible. In this case, the interval between the prediction modes may be calculated based on a numerical value indicating the directionality of each mode, and the directionality of the prediction mode may be quantified as inclination information or angle information of the direction.

Also, in addition to the above method, the prediction block may be generated by using other methods using spatial correlation. For example, by using the current picture as a reference picture, a reference block using an inter prediction method such as motion search and compensation may be generated as a prediction block.

The generating of the prediction block may generate the prediction block by using the reference pixel according to the prediction method. That is, according to the prediction method, a prediction block can be generated through a directional prediction method such as extrapolation, interpolation, copying, and averaging of the existing intra prediction method or a non-directional prediction method, and a prediction block can be generated using an inter prediction method can be created, and other additional methods may also be used.

The intra prediction method may be supported under the same setting of an encoder/decoder, and may be determined according to a slice type, a size of a block, a shape of a block, and the like. The intra prediction method may be supported according to at least one of the above-mentioned prediction methods or a combination thereof. The intra prediction mode may be configured according to the supported prediction method. The number of supported intra prediction modes may be determined according to the prediction method, slice type, block size, block type, and the like. The related information can be set and transmitted in units such as sequence, picture, slice, and block.

In the prediction mode encoding step of performing prediction mode encoding, a mode having an optimal encoding cost according to each prediction mode in terms of encoding cost may be determined as the prediction mode of the current block.

For example, the prediction mode encoder may use the modes of one or more neighboring blocks for the current block mode prediction for the purpose of reducing the prediction mode bits. A mode (most_probable_mode, MPM) having a high probability of being identical to the mode of the current block may be included as a candidate group, and modes of a neighboring block may be included in the above candidate group. For example, prediction modes of blocks such as upper-left, lower-left, upper, and upper-right of the current block may be included in the upper candidate group.

The candidate group of the prediction mode is selected from among factors such as the position of the neighboring block, the priority of the neighboring block, the priority in the divided block, the size or shape of the neighboring block, a predetermined specific mode, and the prediction mode of the luminance block (in case of a color difference block). It may be configured according to at least one factor or a combination thereof, and related information may be transmitted in units of sequences, pictures, slices, blocks, and the like.

For example, when the current block and the neighboring block are divided into two or more blocks, which block mode among the divided blocks to be included as a mode prediction candidate of the current block can be determined under the same setting of the encoder/decoder. there is. Also, for example, the left block among the neighboring blocks of the current block (M×M) consists of three divided blocks by performing quad tree division in the block division unit, and M/2×M/2, M/2×M/2, When the blocks of M/4×M/4 and M/4×M/4 are included, the prediction mode of the M/2×M/2 block may be included as a mode prediction candidate of the current block based on the block size.

As another example, the upper block among neighboring blocks of the current block (N×N) consists of three divided blocks by performing binary tree division in the block division unit, and N/4×N, N/4 from left to right When blocks of ×N and N/2×N are included, the prediction mode of the first N/4×N block from the left is the mode prediction candidate of the current block according to a preset order (priorities are allocated from left to right) can be included as

As another example, when the prediction mode of the block adjacent to the current block is the directional prediction mode, the prediction mode adjacent to the prediction direction of the corresponding mode (the slope information or angle information side of the direction of the mode) is added to the mode prediction candidate group of the current block. may include In addition, preset modes (planar, DC, vertical, horizontal, etc.) may be preferentially included according to the configuration or combination of prediction modes of neighboring blocks.

Also, a prediction mode having a high frequency of occurrence among prediction modes of a neighboring block may be preferentially included. The priority means not only the possibility of being included in the mode prediction candidate group of the current block, but also the possibility of being assigned a higher priority or index (that is, a high probability of receiving fewer bits in the binarization process) in the candidate group configuration. there is.

As another example, the maximum number of mode prediction candidates of the current block is k, the left block is composed of m blocks having a length smaller than the vertical length of the current block, and the upper block is n blocks having a length smaller than the horizontal length of the current block. , when the divided block sum (m+n) of the neighboring blocks is greater than k, the candidate group can be filled according to a preset order (left to right, top to bottom), and the divided block sum of the neighboring block divisions (m+ If n) is greater than the maximum value (k) of the candidate group, a neighboring block (eg, lower left, upper left, upper right, etc.) other than the position of the neighboring block in the prediction mode of the neighboring block (left block, upper block) A prediction mode of a block such as may also be included in the mode prediction candidate group of the current block. The above examples are not specific only to the above-described case, and may include a case in which each other's conditions are reversed, and variations in other cases are also possible.

As such, the candidate block for mode prediction of the current block is not limited to a specific block position, and prediction mode information can be utilized from at least one block among blocks located on the left, upper left, lower left, upper, and upper right, As in the above example, the prediction mode of the current block may be configured as a candidate group in consideration of various factors.

The prediction mode encoder can classify a mode (MPM) candidate group (referred to as candidate group 1 in this example) and a mode candidate group (referred to as candidate group 2 in this example) that are highly probable to be the same as the mode of the current block. A prediction mode encoding process may vary according to which candidate group among the candidate groups the prediction mode of the block belongs to.

The total prediction mode may consist of the sum of the prediction modes of the candidate group 1 and the prediction modes of the candidate group 2. It may be determined according to one or more factors, such as the shape of the block, or a combination thereof. Depending on the candidate group, the same binarization may be applied or a different binarization may be applied.

For example, fixed-length binarization may be applied to candidate group 1 and short cut-type binarization may be applied to candidate group 2. In the above description, the number of candidate groups is two as an example, but the mode first candidate group with a high probability of being identical to the mode of the current block, the mode second candidate group with high probability of being identical to the mode of the current block, and mode candidates not having the same probability as the mode of the current block, etc. It can be extended, and its modifications are also possible.

In the post-processing filtering step performed by the post-processing filter unit, some prediction pixels among the prediction blocks generated in the previous process in consideration of the characteristic of high correlation between the reference pixel adjacent to the boundary between the current block and the neighboring block and the pixel in the adjacent current block. may be replaced with a value generated by filtering one or more reference pixels and one or more prediction pixels adjacent to the boundary, and a value obtained by quantifying the characteristics between reference pixels adjacent to the boundary of the block (eg, the difference in pixel values, gradient information, etc.) can be applied to the filtering process to replace the predicted pixel with a value generated can be

In the post-processing filter unit, the type of filter and whether or not to apply the filtering may be determined implicitly or explicitly, and the positions and number of reference pixels and current pixels used in the post-processing filter unit, and the type of prediction mode to be applied, etc. It can be set in the encoder/decoder, and related information can be transmitted in units of sequences, pictures, slices, and the like.

Also, in the post-processing filtering step, an additional post-processing process may be performed after the prediction block is generated, such as block boundary filtering. In addition, post-processing filtering is performed in consideration of the characteristics of pixels of adjacent reference blocks, similar to the boundary filtering above, for the current block restored by adding the residual signal and the prediction signal obtained through the transform/quantization process and the inverse process after obtaining the residual signal. can also be done

Finally, a prediction block is selected or acquired through the above-described process, and information from this process may include prediction mode-related information, and is transmitted to the transform unit 210 for encoding the residual signal after the prediction block is obtained. can

10 is an exemplary diagram for explaining a prediction principle in a P slice or a B slice in an image encoding method according to an embodiment of the present invention. 11 is an exemplary diagram for explaining a case in which interpolation is performed in the image encoding method of FIG. 10 .

Referring to FIG. 10 , the image encoding method according to the present embodiment may include motion estimation module and interpolation steps. Information on the motion vector, reference picture index, and reference direction generated in the motion prediction step may be transferred to the interpolation step. In the motion prediction step and the interpolation step, a value stored in a decoded picture buffer (DPB) may be used.

That is, the image encoding apparatus may perform motion estimation to find a block similar to the current block in previously encoded pictures. Also, the image encoding apparatus may perform interpolation of the reference picture for more precise prediction than the precision of a fractional unit. Finally, the image encoding apparatus obtains a prediction block through a predictor, and information from this process includes a motion vector, a reference picture index or reference index, and a reference direction. ) and the like, and thereafter, residual signal encoding may be performed.

In the present embodiment, since intra prediction is also performed in P slice or B slice, a combination method as shown in FIG. 11 supporting inter prediction and intra prediction is possible.

11 , the image encoding method according to the present embodiment includes reference sample padding, reference sample filtering, intra prediction, boundary filtering, It may include steps of motion estimation, and interpolation.

When the video encoding apparatus supports block matching in the current picture, the prediction method in the I slice may be implemented with the configuration shown in FIG. 11 instead of the configuration shown in FIG. 9 . That is, the video encoding apparatus may use information such as a motion vector, a reference picture index, and a reference direction that occur only in a P slice or a B slice as well as a prediction mode in the I slice to generate the prediction block. However, information that can be partially omitted may exist due to the characteristic that the reference picture is current. For example, when the reference picture is the current picture, the reference picture index and the reference direction may be omitted.

In addition, when the image encoding apparatus applies interpolation, since block matching up to decimal units may not be required due to the characteristics of the image, for example, due to the characteristics of an artificial image such as computer graphics, whether to perform this is also set in the encoder. In this regard, it is also possible to set units such as sequences, pictures, and slices.

For example, the image encoding apparatus may not perform interpolation of reference pictures used for inter prediction according to the setting of the encoder, and may perform various settings such as not performing interpolation only when block matching is performed in the current picture. can That is, the image encoding apparatus of the present embodiment may set whether to perform interpolation of reference pictures. In this case, it may be determined whether interpolation is performed on all reference pictures or some reference pictures constituting the reference picture list.

For example, the image encoding apparatus does not perform interpolation when it is not necessary to perform block matching in decimal units because the characteristic of an image in which a reference block exists in a certain current block is an artificial image, and blocks matching in decimal units because it is a natural image. It can operate to perform interpolation when it is necessary to do

Also, the image encoding apparatus may set whether block matching is applied to a reference picture that has been interpolated in units of blocks. For example, if a natural image and an artificial image are mixed, interpolation is performed on the reference picture, but if the optimal motion vector can be obtained by searching for a part of the artificial image, the A motion vector can be expressed, and when an optimal motion vector can be obtained by selectively searching a part of a natural image, the motion vector can be expressed in another predetermined unit (assuming that it is a quarter unit).

12 is a diagram for explaining a main process of an image encoding method according to an embodiment of the present invention with syntax in a coding unit.

Referring to FIG. 12 , curr_pic_BM_enabled_flag means a flag that allows block matching in the current picture, and can be defined and transmitted in sequence and picture units. It may mean a case of operating through inter-prediction. In addition, it may be assumed that cu_skip_flag, which is an inter-picture technique that does not encode a residual signal, is a flag supported only in P slices or B slices except for I slices. In that case, when curr_pic_BM_enabled_flag is turned on, block matching (BM) can be supported in the inter prediction mode even in I slices.

The image encoding method according to the present embodiment may support skip when a prediction block is generated through block matching in the current picture, and may support skip in the case of an intra-picture technique other than block matching. Also, skip may not be supported in the I slice depending on conditions. Whether to skip this may be determined according to an encoder setting.

For example, when skip is supported in I slice, the prediction block can be directly restored to the reconstructed block through block matching without encoding the residual signal by connecting to prediction_unit(), which is the prediction unit through a specific flag if(cu_skip_flag). . In addition, the image encoding apparatus may classify a method of using a prediction block through block matching in the current picture as an inter prediction technique, and may process such classification through a specific flag, pred_mode_flag.

Also, the video encoding apparatus according to the present embodiment may set the prediction mode to the inter prediction mode (MODE_INTER) if pred_mode_flag is 0, and may set the prediction mode to the intra prediction mode (MODE_INTRA) if 1 is 1. This is an intra-picture technique similar to the existing one, but it can be classified as an inter-picture technique or an intra-picture technique in I slice to distinguish it from the existing structure. That is, the video encoding apparatus of the present embodiment does not use temporal correlation in the I slice, but may use a temporal correlation structure. part_mode means information on the size and shape of a block divided in a coding unit.

Some syntaxes used in connection with the present embodiment are exemplified as follows.

In the sequence parameter, the syntax sps_curr_pic_ref_enabled_flag may be a flag indicating whether IBC is used.

The syntax pps_curr_pic_ref_enabled_flag in the picture parameter may be a flag indicating whether IBC is used in units of pictures.

In addition, it may be determined whether to increase NumPicTotalCurr for the current picture according to on/off of the syntax pps_curr_pic_ref_enabled_flag.

In the process of creating the reference picture list 0, it may be determined whether to put the current picture in the reference picture according to pps_curr_pic_ref_enabled_flag.

In the process of creating the reference picture list 1, it may be determined whether to add the reference picture list of the current picture according to pps_curr_pic_ref_enabled_flag.

According to the above configuration, it may be determined whether the current picture is included in the reference picture list in inter prediction according to the flags.

13 is a diagram illustrating an example of supporting symmetric type partitioning or asymmetric type partitioning as in inter prediction when a prediction block is generated through block matching in the current picture used in FIG. 12 . It is an example diagram.

Referring to FIG. 13 , in the video encoding method according to the present embodiment, when a prediction block is generated through block matching in the current picture, 2N×2N, 2N×N, N×2N, N× as in inter prediction. A symmetric partition such as N may be supported, or an asymmetric partition such as nL×2N, nR×2N, 2N×nU, or 2N×nD may be supported.

14 is an exemplary diagram for explaining that 2N×2N and N×N can be supported in the inter prediction (Inter) like the intra prediction (Intra) of FIG. 9 . Various block sizes and shapes may be determined according to the division method of the block division unit.

Referring to FIG. 14 , the video encoding method according to the present embodiment may support 2N×2N and N×N like the form of a prediction block used for conventional intra prediction. This is an example in which the block division unit supports a square shape through a quadtree division method or a division method according to a predefined block candidate group, and even in intra prediction, a binary tree division method or a rectangular shape in a predefined block candidate group Other block types can also be supported by adding a shape, and the setting for this can be set in the encoder.

In addition, among intra prediction, whether skip is applied only when block matching to the current picture is performed (ref_idx = curr), whether it is applied to existing intra prediction, or other partition types (FIG. 13) Reference), it is possible to set in the encoder whether or not to apply the prediction block to the new intra prediction. Information on this may be transmitted in units of sequences, pictures, slices, and the like.

The subtraction unit 205 (refer to FIG. 2 ) may generate a residual block by subtracting pixel values of the prediction block generated by the prediction unit 200 from the pixel values of the current block to be encoded to derive a pixel difference value.

The transform unit 210 (refer to FIG. 2 ) receives the residual block that is the difference between the current block and the prediction block generated through intra prediction or inter prediction from the subtraction unit 205 and transforms it into the frequency domain. Through the transform process, each pixel of the residual block corresponds to a transform coefficient of the transform block. The size and shape of the transform block may be the same as or smaller than the coding unit. Also, the size and shape of the transform block may be the same as or smaller than the prediction unit. The image encoding apparatus may perform transformation processing by bundling several prediction units.

The size or shape of the transform block may be determined through the block dividing unit, and a square shape or rectangular shape transform may be supported according to the block division. The block division operation may be affected according to the transform related settings supported by the encoder/decoder (size, shape, etc. of a supported transform block).

The size and shape of each transform block is determined according to the encoding cost for each candidate of the size and shape of the transform block, and segmentation information such as image data of each determined transform block and the determined size and shape of each transform block can be encoded. .

The transform may be transformed by a one-dimensional transform matrix. For example, each transform matrix may be adaptively used in discrete cosine transform (DCT), discrete cosine transform (DST), horizontal and vertical units. The adaptive use may include, for example, determining the size of a block, the shape of the block, the type of block (luminance/color difference), encoding mode, prediction mode information, quantization parameters, and determining based on various factors such as encoding information of neighboring blocks. there is.

For example, in the case of intra prediction, when the prediction mode is horizontal, a DCT-based transform matrix may be used in a vertical direction and a DST-based transform matrix may be used in a horizontal direction. Also, when the prediction mode is vertical, a DCT-based transform matrix may be used in a horizontal direction and a DST-based transform matrix may be used in a vertical direction.

The transformation matrix is not limited to that from the description above. Information on this may be determined using an implicit or explicit method, and one or more factors among factors such as block size, block shape, encoding mode, prediction mode, quantization parameter, and encoding information of a neighboring block, or a combination thereof , and the related information may be transmitted in units of sequences, pictures, slices, blocks, and the like.

Here, considering the case of using an explicit method, if two or more transformation matrices for horizontal and vertical directions are placed as candidates, information on which transformation matrix is used in each direction may be sent, or horizontal, Information on which transformation matrix is used in the horizontal and vertical directions may be transmitted by grouping two or more pairs as a candidate group with respect to which transformation matrix is used in the vertical direction, respectively.

In addition, partial transformation or total transformation may be omitted in consideration of the characteristics of the image. For example, either or both of the horizontal and vertical components may be omitted. This is because, when the difference between the current block and the prediction block is large because intra prediction or inter prediction is not performed well, that is, when a residual component is large, a corresponding encoding loss may increase when transforming it. This may be determined according to at least one factor or a combination of factors such as encoding mode, prediction mode, block size, block shape, block type (luminance/color difference), quantization parameter, and encoding information of a neighboring block. . According to the above conditions, it can be expressed using an implicit or explicit method, and information about this can be transmitted in units of sequences, pictures, slices, and the like.

The quantization unit 215 (refer to FIG. 2 ) quantizes the residual component transformed by the transform unit 210 . The quantization parameter is determined in units of blocks, and the quantization parameters may be set in units of sequences, pictures, slices, blocks, and the like.

For example, the quantization unit 215 may predict the current quantization parameter by using one or more quantization parameters derived from neighboring blocks such as left, upper left, upper, right upper, lower left, etc. of the current block.

In addition, the quantization unit 215 determines the difference from the basic parameters transmitted in units of sequences, pictures, slices, etc. when the quantization parameter predicted from the neighboring block does not exist, that is, when the block is at the boundary between pictures, slices, etc. values can be printed or transmitted. When a quantization parameter predicted from a neighboring block exists, a difference value may be transmitted using the quantization parameter of the corresponding block.

The priority of the block from which the quantization parameter is to be derived may be set in advance, or may be transmitted in units of sequences, pictures, slices, and the like. The residual block may be quantized through Dead Zone Uniform Threshold Quantization (DZUTQ), a quantization weighted matrix, or an improved technique thereof. This may include one or more quantization techniques as candidates, and may be determined by encoding mode, prediction mode information, and the like.

For example, the quantization unit 215 may set a quantization weight matrix to be applied to inter-picture coding, an intra-coding unit, etc., and may provide a different weight matrix according to an intra prediction mode. The quantization weight matrix has a size of M×N, and assuming that the size of the block is the same as the size of the quantization block, the quantization coefficient can be configured differently for each position of each frequency component. In addition, the quantization unit 215 may select one of several existing quantization methods, or may be used under the same setting of an encoder/decoder. Information on this can be transmitted in units of sequences, pictures, slices, and the like.

Meanwhile, the inverse quantization units 220 and 315 and the inverse transform units 225 and 320 illustrated in FIGS. 2 and 3 may be implemented by performing the above processes in the transform unit 210 and the quantization unit 215 inversely. That is, the inverse quantization unit 220 may inverse quantize the quantized transform coefficient generated by the quantizer 215, and the inverse transform unit 225 may inversely transform the inverse quantized transform coefficient to generate a reconstructed residual block. there is.

The adders 230 and 324 shown in FIGS. 2 and 3 may generate a reconstructed block by adding the pixel value of the prediction block generated from the prediction unit to the pixel value of the reconstructed residual block. The reconstructed block may be stored in the encoding and decoding picture buffers 240 and 335 and provided to the prediction unit and the filter unit.

The filter unit 235 (refer to FIG. 2 ) may apply an in-loop filter such as a deblocking filter, a sample adaptive offset (SAO), or an adaptive loop filter (ALP) to the reconstructed block. The deblocking filter may filter the reconstructed block to remove distortion between block boundaries generated during encoding and decoding. SAO is a filter process that restores the difference between the original image and the reconstructed image as an offset for the residual block in units of pixels. The ALF may perform filtering to minimize the difference between the prediction block and the reconstruction block. The ALF may perform filtering based on a comparison value between the block restored through the deblocking filter and the current block.

The entropy encoder 245 (refer to FIG. 2 ) may entropy-encode the transform coefficients quantized through the quantizer 215 . For example, to be implemented using a coding scheme other than context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax based context adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy coding ( PIPE) coding may be performed.

The entropy encoder 245 may include, in the encoded data, a bit string obtained by encoding a quantization coefficient and various pieces of information necessary for decoding the encoded bit string. The encoded data may include an encoded block type, a quantization coefficient, and a bit stream in which the quantized block is encoded, and information required for prediction. In the case of a quantization coefficient, a two-dimensional quantization coefficient may be scanned in one dimension. The distribution of the quantization coefficients may vary depending on the characteristics of the image. In particular, in the case of intra prediction, since the distribution of coefficients may have a specific distribution depending on the prediction mode, a scanning method may be set differently.

Also, the entropy encoder 245 may be set differently according to the size of a block to be encoded. The scan pattern may be preset or set as a candidate to at least one of various patterns such as zigzag, diagonal, raster, etc., and may be determined by encoding mode and prediction mode information, etc. can be used Information on this can be transmitted in units of sequences, pictures, slices, and the like.

The size of the quantized block (hereinafter, quantized block) input to the entropy encoder 245 may be equal to or smaller than the size of the transform block. In addition, the quantization block may be divided into two or more sub-blocks, and when divided, the scan pattern in the divided block may be set to be the same as that of the existing quantization block or may be set differently.

For example, if the scan pattern of the existing quantization block is called zigzag, zigzag may be applied to all subblocks, or a zigzag pattern may be applied to a subblock located at the upper left of a block including an average value (DC) component. You can apply a diagonal pattern to other blocks. This may also be determined according to encoding mode, prediction mode information, and the like.

In addition, the start position of the scan pattern in the entropy encoder 245 basically starts from the upper left, but may start from the upper right, lower right, or lower left depending on the characteristics of the image, and any one of two or more candidate groups is selected Information on whether or not it has been performed may be transmitted in units of sequences, pictures, slices, and the like. As the encoding technology, an entropy encoding technology may be used, but is not limited thereto.

On the other hand, the inverse quantization of the inverse quantization unit 220 and the inverse transformation of the inverse transform unit 225 shown in FIGS. 2 and 3 reverse the quantization of the quantization unit 215 and the transformation structure of the transform unit 210. And it can be implemented by combining the basic filter units (235, 330).

Next, interpolation that can be employed in the video encoding apparatus of the present invention will be briefly described as follows.

In order to increase the accuracy of prediction through block matching, interpolation is performed with a resolution of a fractional unit that is more precise than an integer unit. A DCT-IF technique is used as an interpolation method in high efficiency video coding (HEVC). For example, a pixel is generated in units of 1/2 and 1/4 between integers, a reference picture is interpolated, and a block is referred to. A prediction block may be generated by performing matching.

Table 1 and Table 2 show the filter coefficients used for the luminance component and the chrominance component, respectively. For the luminance component, an 8-tap DCT-IF filter is used and for the chrominance component, a 4-tap DCT-IF filter is used. A filter may be applied differently to a color difference component according to a color format. In the case of 4:2:0 of YCbCr, the filter shown in Table 2 can be applied, and in 4:4:4, the filter shown in Table 1 or other filters can be applied instead of Table 2, and 4:2 In the case of :2, the horizontal 1D 4-tap filter shown in Table 2 and the vertical 1D 8-tap filter shown in Table 1 can be applied.

15 is a diagram for explaining a process of performing a horizontal 1D filter on pixels at positions a, b, and c (assumed to be x) of an image in the image encoding method according to an embodiment of the present invention.

15, a horizontal 1D filter can be applied to sub-pixels located at positions a, b, and c (assuming x) between the first pixel (G) and the second pixel (H) adjacent thereto. there is. This can be expressed as a formula as follows.

x = ( f1*E + f2*F + f3*G + f4*H + f5*I + f6*J + 32 ) / 64

Next, a vertical 1D filter can be applied to sub-pixels at positions d, h, and n (assuming y). This can be expressed as a formula as follows.

y = ( f1*A + f2*C + f3*G + f4*M + f5*R + f6*T + 32 ) / 64

In addition, a 2D separable filter may be applied to sub-pixels e, f, g, i, j, k, p, q, r in the center. Taking the sub-pixel e as an example, pixels in a direction perpendicular to a are first interpolated and then interpolated using the pixels. Then, horizontal 1D filter is performed as if interpolating a between G and H, and vertical 1D filter is performed on the resulting sub-pixels to obtain e value. In addition, a similar operation may be performed with respect to a color difference signal.

The above explanation is only a partial explanation of interpolation. Filters other than DCT-IF can also be used, and the type of filter applied to each decimal unit and the number of taps can be varied. For example, an 8-tap Kalman filter for 1/2, a 6-tap Wiener filter for 1/4, a 2-tap linear filter for 1/8, a fixed coefficient, or filter coefficients by calculating the filter coefficients as DCT-IF. It can also be encoded. As described above, one interpolation filter may be used for a picture, a different interpolation filter may be used for each region according to the characteristics of the image, and two or more reference pictures to which a plurality of interpolation filters are applied may be created and one of them may be selected. there is.

Different filters may be applied according to encoding information such as the type of the reference picture, the temporal layer, and the state of the reference picture (eg, whether it is a current picture or not). The above-mentioned information can be set in units of sequences, pictures, slices, etc., and can be transmitted in units of those units.

Next, before describing in detail improved techniques for motion estimation, motion compensation, and motion prediction that can be employed in the image encoding method according to the present embodiment, basic meanings of these terms is defined as follows.

Motion estimation refers to a process of dividing an image frame into small blocks in predicting the motion of a current block to be encoded and estimating from which block in a temporally previous or subsequent code-coded frame (reference frame) it is moved. That is, motion estimation can be said to be a process of finding a block most similar to a target block when encoding a current block to be compressed. Block-based motion estimation may refer to a process of estimating to which position a video object or a screen processing unit block (such as a macro block) moved in time.

In order to encode the current image, motion compensation obtains at least a partial region of a previously encoded reference image to obtain motion information (motion vector, reference picture index) for the optimal prediction block found in the motion estimation process to predict the current image. It means to generate a prediction block of the current block based on the That is, motion compensation may refer to a process of creating an error block with a difference between a current block to be encoded and a reference block found to be the most similar block.

Motion prediction means finding a motion vector during encoding for motion compensation. Major techniques of motion prediction include skip, temporal prediction, and spatial prediction. In skip, the size of a motion vector predicted by the video encoding apparatus is zero because the motion of the screen is constant, or the residual is small enough. If it can be ignored, it means skipping the encoding of the corresponding video block. Temporal prediction may be mainly used for inter prediction, and spatial prediction or inter-view prediction may be mainly used for intra prediction.

The information output through inter prediction may include information for discriminating a reference picture list direction (unidirectional (L0, L1), bidirectional), an index for discriminating reference pictures in the reference picture list, a motion vector, and the like. Since the temporal correlation is used, motion information can be efficiently encoded by using the characteristic in which the motion vectors of the current block and the neighboring blocks are the same or similar.

16 is an exemplary diagram of a current block and neighboring blocks that can be employed in an image encoding method according to an embodiment of the present invention.

As shown in FIG. 16 , the setting of whether to refer to the candidate group for the neighboring block of the current block may be determined according to information such as the type of the current picture and the temporal identifier (temporal id), and this information is fixed and used or It may be transmitted in units of sequences, pictures, slices, and the like.

Here, the setting of whether to refer to the candidate group is, for example, using only spatially adjacent blocks (A, B, C, D, E) or temporally adjacent to spatially adjacent blocks (A, B, C, D, E). It can include a setting using blocks (H, I, J) or using blocks (F, G) that are spatially separated from blocks (A, B, C, D, E) that are spatially adjacent to each other.

The following describes the setting of “block matching can be applied to the current picture”.

First, if the case of the I picture is described, for example, a block that is spatially adjacent may be prioritized first, and other blocks may be set as a candidate group. As an example, availability of reference blocks may be checked in the order of E → D → C → B → A → H → I → J. Availability is for determining whether availability is available, and may be compared with a preset reference value or may be compared with other values checked for availability. Availability may be determined by the coding mode of the candidate block, motion information, the position of the candidate block, and the like. The motion information may include a motion vector, a reference direction, a reference picture index, and the like.

Since the current picture is an I picture, motion information exists only when the encoding mode is set to intra prediction (hereinafter, INTER) of the present embodiment. Therefore, in order of priority, first check if it is INTER. For example, if n is 3 and E is encoded as INTER, E is excluded from the candidate group and the next D is checked. If D is encoded as INTER, it has motion information because block matching is performed on the current picture, and D is added to the candidate group based on this motion information. Then there are two n left. Then, the image encoding apparatus may check the priority again. In this way, when the final three candidates are filled, the search for candidate groups is stopped.

Availability is not only used in the encoding mode, but can also be used in the case of a boundary between a picture, a slice, and a tile. In case of boundary, availability is checked as not available. In addition, when the result of determining that the candidate is the same as or similar to the already filled candidate is obtained, the corresponding block is excluded from the candidate, and the reference pixel constituent unit checks the availability of the next candidate.

Here, INTER is different from the existing inter prediction (inter). That is, the INTER mode of the present embodiment is different from inter prediction in which a prediction block is generated from a reference picture because a prediction block is generated from a current picture only by using an inter structure. That is, in the encoding mode of the present embodiment, the block matching method in the current picture can be applied by classifying the INTER mode and intra (same as the existing intra).

Hereinafter, motion vector copy (MVC) and motion vector prediction (MVP) will be separately described. The reason is that whether or not the scaling process is included is different.

The motion vector prediction (MVP) will be described first.

In the case of a P picture or B picture

In addition to the above-mentioned candidates (A, B, C, D, E, F, G, H, I, J), temporal candidates (F, G) are also included in the description. In this embodiment, it is assumed that candidates are spatially searched, temporally searched, searched by constructing a mixed list, and performed in the described order to search for constant candidates.

First, we prioritize candidates and check availability accordingly. It is assumed that the number of motion vector candidates (n) is set to 2, and the priority is as described in parentheses.

For example, when searching spatially, it can be classified into the following groups.

Group 1_1 = {A, B, C, I, J}, ( C → B → A → I → J)

Group 1_2 = {D, E, H}, (D → E → H)

In this embodiment, among the two groups, group 1_1 includes blocks immediately above, above, left, and above the current block, and group 1_2 is immediately adjacent to the left and not immediately adjacent to the current block. , and the blocks in the lower left.

As another embodiment, it is possible to spatially search for a candidate block of a motion vector by dividing it into three groups. The three groups can be classified as follows.

group 1_1 = {A, B, C}, (C → B → A)

group 1_2 = {D, E}, (D → E)

Group 1_3 = {H, I, J}, (J → I → H)

In the present embodiment, among the three groups, group 1_1 includes blocks immediately adjacent to the upper left, adjacent upper left, and adjacent upper right with respect to the current block, and group 1_2 is immediately adjacent to the left and immediately adjacent to the current block with respect to the current block. It includes the blocks in the lower left, and group 1_3 includes non-adjacent blocks with one or more block intervals from the current block.

As another embodiment, a candidate block of a motion vector may be spatially searched for by dividing it into three groups in another method. The three groups can be classified as follows.

group 1_1 = {B}

group 1_2 = {D}

Group 1_3 = {A, C, E}, (E → C → A)

In the present embodiment, among the three groups, group 1_1 includes blocks positioned in the vertical direction with respect to the current block, group 1_2 includes adjacent blocks positioned in the horizontal direction with respect to the current block, and group 1_3 includes the current block Based on the block, the remaining adjacent blocks are included.

As discussed above, in the P picture or the B picture, since there is a lot of referenceable information such as a reference direction or a reference picture, a candidate group can be set accordingly. A candidate block having a different reference picture from the current block may be included in the candidate group based on the current block. It is also possible to add the vector of the corresponding block to the scaled candidate group. Also, a scaled candidate group may be added according to which picture the reference picture of the current block is. In addition, when the temporal distance between the reference picture of the current block and the reference picture of the candidate block exceeds a certain distance, it may be excluded from the candidate group, and when the temporal distance is less than or equal to the reference picture of the current block, the scaled block may be included in the candidate group.

The aforementioned similarity check is a process of comparing and determining how similar a motion vector already included in the prediction candidate group is to a motion vector to be newly added. According to definition, it may be set to be true when the x and y components perfectly match, or set to be true when the difference is less than or equal to a certain threshold value.

In this embodiment, the condition that the reference picture points to the current picture is taken as an example, but it may be extended to a case in which the reference picture is not the current picture. For example, a setting such as 'a block using a picture farther than the picture pointed to by the current block is excluded' may be used. In this embodiment, even if the current block and the reference picture are different, they can be included in the candidate group through scaling.

mixed list

When it is assumed that each reference picture in the reference picture list (L0, L1) has motion information by performing bidirectional prediction of the current block. Availability (availability) is checked according to the priority of the preset candidate group. In this case, the priority is checked first that is encoded by bi-prediction.

If each reference picture is different, scaling is performed. When searching spatially and temporally, when only bi-predicted blocks were put in the candidate group, and when the maximum number of candidates was not exceeded, blocks coded by uni-prediction among the previously performed candidate blocks were put into the preliminary candidate group and then a combination of these candidates was used. Candidates for bidirectional prediction can be created.

First, it is assumed that motion information of bi-prediction of the current block is referenced in reference pictures 1 in L0 and 0 in L1. In Table 5(a), the motion information of the first candidate block is (mvA ₁ , ref1) and (mvA ₂ , ref0), and the motion information of the second candidate block is (mvB ₁ ', ref1). ) and (mvB ₂ ', ref0). Here, the meaning of the apostrophe (') is a scaled vector. If it is assumed that the number of candidates after spatial and temporal search is two, however, assuming that n is 5, blocks predicted unidirectionally in the previous step may be put as preliminary candidates according to preset priorities.

In Table 3(a), since the maximum number of candidates has not yet been filled, a new candidate can be added by combining the scaled unidirectional candidates using the remaining motion vectors mvC, mvD, and mvE.

In Table 3(b), motion information of each unidirectionally predicted block is scaled according to the reference picture of the current block. Here, an example of creating a new combination with unidirectional candidates is presented, but a new candidate combination may be possible with motion information of each of the already added bidirectional reference pictures L0 and L1. This part is not performed in situations such as unidirectional prediction. Also, it is not performed even when the reference picture of the current block is the current picture.

constant candidate

If, through the above process, the candidate blocks with the maximum number of candidates of n (in this embodiment, it is assumed as 2) cannot be formed, a constant candidate having preset fixed coordinates may be added. Fixed candidates having fixed coordinates such as (0,0), (-a,0), (-2*a,0), (0,-b) can be used, and the number of fixed candidates according to the maximum number of candidates can be set.

The above fixed coordinates may be set, or through the above process, at least two motion vectors included in the candidate group may be added as fixed candidates through the process of average, weighted average, and median. If n is 5 and so far 3 candidates have been registered as {(mvA_x, mvA_y), (mvB_x, mvB_y), (mvC_x, mvC_y)}, fixed candidates with predetermined priorities are selected to fill the remaining two candidates. It is possible to add a fixed candidate according to the priority according to the included candidate group. The fixed candidate group is, for example, {(mvA_x + mvB_x)/2, (mvA_y + mvB_y)/2), ((mvA_x + mvB_x + mvC_x)/3, (mvA_y + mvB_y + mvC_y)/3), (median(mvA_x) , mvB_x, mvC_x), median(mvA_y, mvB_y, mvC_y)) may include fixed candidates.

In addition, a fixation candidate may be set differently according to the reference picture of the current block. For example, when the current picture is a reference picture, fixed candidates such as (-a,0), (0,-b), (-2*a,0) may be set, and when the current picture is not a reference picture, (0,0), (-a,0), (average(mvA_x, …), average(mvA_y, …)) can also be set. The corresponding information may be preset in an encoder or a decoder, or may be transmitted in units of sequences, pictures, slices, and the like.

Hereinafter, motion vector copy (MVC) will be described in more detail.

Description of P picture or B picture

In this embodiment, it is assumed that temporal candidates (F, G) are also included. Candidate groups include A, B, C, D, E, F, G, H, I, and J. Although the search order is not fixed, it is assumed here that MVC candidates are searched spatially, temporally searched, searched by constructing a mixed list, and added in the order of constant candidates.

In other words, the above-described part is that the search order is arbitrarily determined as described above, and does not use a predetermined order. Set priorities and check availability accordingly. Assume that n is 5 and the priority is the same as in parentheses.

In the following description, only the difference part in the aforementioned motion vector prediction (MVP) will be described. The MVP part can be written by attaching the following contents except for the scaling part in the previous part. For spatial candidates, availability can be checked while omitting the scaling process. However, similar to MVC, the type of the reference picture, the distance from the current picture or the current block to the reference picture, etc. may be excluded from the candidate group.

When a mixed list exists, as shown in Table 4 below, a candidate for bidirectional prediction can be created by a combination of candidates added so far.

As shown in Table 4(a), a new candidate may be added to the motion vector candidate group by combining a candidate using the reference list LO and a candidate using the reference list L1. If the predetermined number of motion vectors of 5 is not satisfied, as shown in Table 4(b), a candidate using L1 and a candidate next to L0 may be combined to be newly added to the candidate.

As described above, encoding may be performed according to a mode such as MVP or MVC for finding an optimal motion information candidate.

In the skip mode, encoding may be performed using MVC. That is, information on an optimal motion vector candidate can be encoded after skip flag processing. If there is only one candidate, this part may be omitted. A residual component that is a difference between a current block and a prediction block may be encoded through processes such as transformation and quantization without separately encoding the motion vector difference.

If it is not skip, it is first checked whether or not motion information is processed through MVC in priority, and if it is correct, information on a candidate group of an optimal motion vector can be encoded. If motion information is not to be processed through MVC, motion information may be processed through MVP. In the case of MVP, information on an optimal motion vector candidate may be encoded. Here, when there is only one candidate, motion information processing may be omitted. In addition, information such as a difference value from the motion vector of the current block, a reference direction, and a reference picture index may be encoded, a residual component may be obtained, and then encoded through processes such as transformation and quantization.

Codecs such as entropy and post-processing filtering will be omitted to avoid overlap with the above description.

17 is an example of a reference structure for a random access mode in an image encoding and decoding method according to an embodiment of the present invention.

Referring to FIG. 17, when encoding is sequentially performed from a picture having a low temporal identifier (temporalID), I picture and P picture having a low ID are encoded respectively, and then B(2) having a high ID is encoded. Encoding proceeds in the order of encoding. It is assumed that a picture having a lower ID is used as a reference picture without referring to an ID higher than or equal to itself.

First, referring to FIG. 17, interpolation precision of pictures used as reference pictures (for example, any one of integer, 1/2, 1/4, 1/8, etc.) can be fixed and used, and depending on the implementation Interpolation precision of pictures used as reference pictures may be set differently. For example, in the case of an I picture having an ID of 0, when used as a reference picture of a picture, the distance between pictures is calculated and has a distance of 1, 4, or 8 (in the case of B4, B2, and P1 pictures). In the case of a P picture, it becomes 4, 2, 1 (B2, B6, B8). And in the case of B2 having an ID of 1, it has a distance of 2, 1, 1, 2 (B3, B5, B7, B6), and when the ID is B3, it has a distance of 1, 1 (B4, B5).

If the interpolation precision of each reference picture can be determined based on the average distance to the referenced picture. That is, it is judged that the closer the reference picture is, the smaller the difference in motion is, so fine interpolation is required. In the case of more than that, the interpolation precision can be lowered and used. For example, it can be reduced to 1/4 (see Case 1 in Table 5 below). Conversely, since it is determined that finer interpolation precision is required as the reference picture is farther away, in that case, the opposite to the above example can be applied.

In addition, finer interpolation may be performed on pictures that are often referenced. For example, interpolation may be performed by increasing the interpolation precision to B2, or a different interpolation precision may be applied to a picture that is rarely referenced (refer to Case 2 of Table 5 below).

In addition, the interpolation precision may be applied differently according to the temporal layer. For example, finer interpolation may be performed for a picture having an ID of 0, and precision may be lowered for a picture having a different ID or vice versa (see Case 3 in Table 5 below).

As described above, inter prediction can be performed by setting the interpolation precision in units of pictures. Information related thereto may be set under an agreement with the encoder or decoder, or may be transmitted in units of sequences, pictures, or the like.

FIG. 18 is a diagram for explaining that one picture may have two or more interpolation precisions in an image encoding and decoding method according to an embodiment of the present invention.

Referring to FIG. 18 , a case in which block matching is performed in the current picture is exemplified. That is, FIG. 18 shows that self-referencing is possible at I0, P1, and B2.

In the present embodiment, a picture having the same or lower ID may be used as a reference picture without referring to an ID higher than the self. In fact, although not shown in FIG. 18 , up to temporalID 2 or 3 may be included when the current picture is used as a reference picture. This also means that the current picture can be used according to the temporalID. In addition to the temporal identifier (temporalID), the use of the reference picture may be determined according to the type of the picture.

The case of FIG. 18 is generally similar to the case of FIG. 17, but indicates that the interpolation precision in the case of using the current picture as a reference picture when encoding the current picture can be set to the same as the interpolation precision in the general case. In addition, the case of FIG. 18 indicates that the interpolation precision can be set differently only when the current picture is a reference picture. That is, it indicates that one picture may have two or more interpolation precisions.

Up to this point, as the interpolation precision of the reference picture is determined, the technique in which the precision of the motion vector is also determined accordingly has been described. Hereinafter, a technique in which motion vector precision is determined regardless of interpolation precision will be described.

Although the interpolation precision (for example, 1/4) of a picture is fixed, the motion vector precision of a block may be set in units of blocks of a picture to be encoded. The default setting may be to use a predetermined motion vector precision, but in this embodiment, it means that two or more candidates are supported. For example, if the interpolation precision of the reference picture is 1/4, the motion vector of the block referring to this picture should be expressed in units of 1/4. However, in the present embodiment, the motion vector may be basically expressed in units of 1/4, additionally in units of 1/2, or may be expressed in units of integers according to blocks.

Table 6 describes the matching according to the precision of a motion vector of a constant constant. For example, to express 1, it corresponds to 1 in integer unit, but corresponds to 2 in 1/2 and 4 in 1/4. In the case of expressing 2, it corresponds to 2 in integer units, 4 in 1/2 units, and 8 in 1/4 units.

When the precision is doubled based on the current precision of the block referring to a specific picture, the integer 1 becomes 1/2 and 1/2 becomes 1/4, so the number for expressing this is doubled. When expressing this as various binarizations such as unary binarization, rice truncated rice, k-th order exp-golomb, etc., the higher the precision, the more bits to be expressed.

Even if the interpolation precision is increased, for example, from 1/2 to 1/4, if the motion vector is found in a low precision unit (eg, an integer or 1/2), the encoding efficiency is reduced due to an increase in the amount of bits accordingly. can fall To prevent this, in the present embodiment, information is expressed on which precision unit the motion vector is found in block units, and this information is used for encoding and/or decoding. In that case, encoding and decoding can be performed more efficiently.

For example, when encoding a motion vector using unary binarization, if the x component of the vector is 8/4 and the y component is 4/4, 111111110 + 11110 binarization bits are required to represent this. Do. However, according to the present embodiment, the above binarization bit can be expressed in 1/2 units (4/2, 2/2), and in that case, it can be expressed as 11110 + 110. In addition, it can be expressed as an integer unit (2,1), and in that case, it can be expressed as 110 + 10. And if information on which precision unit is used for encoding the motion vector (for example, 0 for 1/4, 10 for 1/2, 11 for integer) is sent, 11 (precision unit) in the above case Since it can be expressed as + 110 + 10, it is possible to use a smaller amount of bits than 111111110 + 11110 in 1/4 unit. As described above, by transmitting/receiving information indicating that the motion vector precision of the corresponding block is expressed in integer units together with the motion vector encoding result of the present embodiment, an effect of reducing transmitted/received encoding bits can be obtained.

Also, as an example, if the motion vector x and y components are 3/4 or 1/4, they cannot be expressed in integers or 1/2 units, so they can be expressed as 1110 + 10 in 1/4 units. In this case, encoding and decoding according to the present embodiment can be performed by transmitting information indicating that the motion vector precision of the corresponding block is expressed by 1/4. As described above, the precision of the motion vector may be adaptively determined in units of blocks while the interpolation precision in units of pictures is fixed. The maximum precision of the motion vector in units of blocks is the same as the interpolation precision of the reference picture.

In addition, the group of precision supported in the present embodiment may be configured in various ways. For example, assuming that the interpolation precision of the reference picture is 1/8, the precision integers 1/2, 1/4, 1/8 as above may be used, and (1/2, 1/4, 1/8) ), (1/4, 1/8), (integer, 1/2, 1/8), (1/4, 1/8), etc., can be used by configuring two or more precisions. These may be set in units of pictures, slices, etc., and may be determined in consideration of encoding cost. Information on this can be transmitted in units of sequences, pictures, slices, and the like. Also, in decoding, one set from among the received groups may be selected and used to adaptively determine motion vector precision in units of blocks.

In addition, in the present embodiment, the index for precision selection can be expressed as fixed length binarization, unary binarization, etc. according to the number of constituting candidate groups. A short bit may be allocated to a unit that is most likely to occur or occur based on the precision of a reference picture, and vice versa, a long bit may be allocated.

For example, if 1/8 units are expected to occur the most, 0 can be assigned to 1/8, 10 to 1/4, 110 to 1/2, and 111 to integers. That is, the shortest bit may be allocated to 1/8 and the longest bit may be allocated to an integer. In addition, when 1/4 units occur most frequently, it is possible to allocate the shortest bits to 1/4. In addition, a fixed length may be allocated regardless of the probability of occurrence according to implementation. For example, a fixed length may be assigned according to a statistical frequency of occurrence, such as 00 for integers, 01 for 1/2, 10 for 1/4, and 11 for 1/8.

Also, in the present embodiment, when the reference picture is a specific picture, that is, when the reference picture is a current picture, a high priority can be given to a specific precision. When the reference picture is the current picture, a high priority may be given to an integer. You can also assign the shortest bit to the default precision of 1/8, and the next shortest bit to the integer. That is, 0 can be assigned to 1/8, 10 to an integer, 110 to 1/4, and 111 to 1/2. Also, depending on the implementation, the shortest bit may be assigned to an integer. For example, 0 can be assigned to an integer, 10 can be assigned to 1/8, 110 can be assigned to 1/4, and 111 can be assigned to 1/2.

The reason for the above allocation is that, in the case of an image including screen content, motion search is rarely performed for the corresponding part to a fractional unit, but motion search is often performed for a natural image part to a fractional unit. am. That is, the image encoding and decoding method of this embodiment can be more effectively applied to the case of an image in which a part of the image is a screen content region such as a computer captured screen and at least another part of the image is a general natural image region, in which the two regions are mixed. .

In addition, in the present embodiment, information on a reference block used for motion information prediction of the current block may be utilized. The reference block used in this case may utilize information of a spatially adjacent first block. The first block may include at least one or more blocks among upper-left, upper-right, upper-right, and lower-left based on the current block.

In addition, information on a reference block (co-located block) present at a position corresponding to the current block in the selected reference picture may be used. A reference block includes at least one block among the top-left, top, top-right, left, bottom-left, bottom, bottom-right, and right of the reference block (center block) in the same position as the current block as well as the co-located block. can be considered as candidates. In this case, the position included in the candidate group may be determined according to the picture type, encoding-related parameters such as the size, mode, motion vector, and reference direction of the current block, and correlation with spatially adjacent candidate blocks. In addition, the selected reference picture may mean a picture in which a distance between pictures existing before or after the current picture is 1 or more.

Also, in the present embodiment, information on blocks that are not adjacent to the current block but located in the same space may be used. The blocks may include, as candidates, a block including at least one block between a current block determined based on information such as an encoding mode, a reference picture index, or preset coordinates and the corresponding block. The preset coordinates may be set to have a predetermined distance horizontally and vertically from the upper left coordinates of the current block.

For example, it is assumed that the left block and the upper block are referenced when expressing the precision of the motion vector of the current block. And the precision of the motion vector is expressed in a truncated unary binarization method. The bit configuration may be something like 0 - 10 - 110 - 111. If the precision is determined in 1/2 units for the motion vector of the left block and 1/2 units for the upper block, it is determined that the motion vector is most likely to appear in the current block in 1/2 units. By allocating short bits, precision-related information can be binarized. The various units may include integers, 1/2, 1/4, and 1/8 units supported by the device, and 1/8 may be set as the reference unit.

As another example, when the left block is 1/4 and the upper block is 1/8, relatively short bits may be allocated to 1/4 and 1/8, and relatively long bits may be allocated to other units. In this case, when 1/8, which is the current reference unit, is additionally included, the shortest bit may be allocated to 1/8. In addition, when a reference unit is not included, such as 1/4 on the left and 1/2 on the top, the shortest bit may be allocated to a fixed position so that the fixed position is used first. For example, assuming that the left has priority, the shortest bit can be allocated to 1/4.

Also, in this embodiment, motion information of a candidate block may be used. The motion information may include a reference picture index, a reference direction, etc. in addition to the motion vector. For example, when the left side is 1/4 and the top is an integer, when the reference picture is checked, the left side has t-1 and the top side has t-2 as a reference picture, and when the current block is t-2, the same reference A block having a picture may be given priority, and the shortest bit may be allocated to an integer that is the precision of a motion vector of the corresponding block.

A brief description of the above embodiment is as follows 1) to 4).

1) (1/2, 1/2) 1/2 - 1/8 - 1/4 - integer

2) (1/4, 1/8) 1/8 - 1/4 - 1/2 - integer

3) (1/4, 1/2) 1/4 - 1/2 - 1/8 - integer

4) (1/4, integer) integer - 1/4 - 1/8 - integer

In addition, in the present embodiment, motion vector precision can be adaptively or fixedly supported on a block-by-block basis according to information such as the type and type of the current picture while the interpolation precision is fixed on a picture-by-picture basis. 18, when the current picture is a reference picture, motion vector precision may be adaptively used, and when the distance between reference pictures is short, such as B4, B5, B7, or B9, for the purpose of finding a more detailed motion Block-wise motion vector precision may be adaptively supported for a picture, and motion vector precision may be fixedly supported for other pictures.

That is, when the temporal hierarchical information (Temporal ID) is 3, an integer, 1/2, or 1/4 is selected and supported, and for the remaining IDs, only 1/4 is used fixedly, or a reference picture and a reference picture are used. In the case of a P picture with a long distance from , it is assumed that there may be regions with fine motion and regions without fine motion, and adaptive motion vector precision can be determined on a block-by-block basis. there is.

In addition, depending on the implementation, two or more sets with different configurations of precision may be provided and supported. For example, (1/2, 1/4), (integer, 1/2, 1/4), (1/2, 1/4, 1/8), (integer, 1/4), etc. at least 2 More than one precision can be candidates (see Table 7).

When the interpolation precision can be set on a picture-by-picture basis

19 is a diagram illustrating a reference picture list when the current picture is an I picture in FIG. 18 .

Referring to FIG. 19 , a prediction block may be generated through block matching in the current picture. An I picture is added to the reference picture (N). Expression of I*(0) means that the current picture is used as a reference picture when the current picture is encoded.

When the interpolation precision is set on a picture-by-picture basis, precision is allowed only up to an integer unit for I pictures, which corresponds to a case where interpolation is not performed. The image encoding and decoding apparatus may refer to only reference picture list 0 (reference list 0, L0).

In addition, if the use of the current picture as a reference picture is restricted according to the picture type or temporal identifier (temporalID), for example, block matching may be allowed for the current picture only in the case of an I picture, and block matching for other pictures is not allowed. , reference pictures of an asterisk (*) may be omitted.

20 is a diagram illustrating a reference picture list when the current picture is a P picture in FIG. 18 .

Referring to FIG. 20 , a P picture is also added to the reference picture. The expression P*(1) means that the P picture is used as the current picture when the current picture is encoded. Here, I(0) has a different meaning from I*(0) above. That is, since I*(0) performs motion search during encoding, motion search can be performed on a picture to which post-processing filtering such as a deblocking filter is not applied or partially applied, and I(0) indicates that the encoding is finished. Since it is a picture to which post-processing filtering is applied, it may be a picture having the same POC, but different pictures due to a difference in filtering application.

When the previous I picture is used as a reference picture, since the interpolation precision of the corresponding picture is 1/8, motion search is also performed up to 1/8 unit to find the optimal motion vector. When the current picture (P*(1)) is used as a reference picture, since the interpolation precision of the corresponding picture is an integer, motion search may be performed in units of integers.

21 is a diagram illustrating a reference picture list when a current picture is B(2) in an image encoding and decoding method according to an embodiment of the present invention. 22 is a diagram illustrating a reference picture list when the current picture is B(5) in the video encoding and decoding method according to an embodiment of the present invention.

Referring to FIG. 21 , when the current picture is B(2), an I picture (I(0)) and a P picture (P(1)) are added as reference pictures, and motion search is also performed according to the interpolation precision of each picture. It shows that it can perform up to the precision unit of the picture. In the present embodiment, a motion vector may be searched for with an accuracy of 1/4 unit in a P picture, and a motion vector may be searched with an accuracy of 1/8 unit in an I picture.

Also, referring to FIG. 22 , when the current picture is B(5), neither of the I picture and the P picture is included as a reference picture, but a plurality of reference pictures {B(2), B(4), B (3)}, it is possible to search for a motion vector with different precision according to the interpolation precision.

FIG. 23 is a diagram for explaining a process in which motion vector precision of each block is determined according to interpolation precision of a reference picture in an image encoding and decoding method according to an embodiment of the present invention. 24 is a diagram for describing a process in which motion vector precision of each block is adaptively determined when the interpolation precision of each reference picture is fixed in the video encoding and decoding method according to an embodiment of the present invention.

Referring to FIG. 23 , the interpolation precision of the current picture is an integer (Int), and the interpolation precision of three reference pictures (t-1, t-2, t-3) is 1/4, 1/2, and 1, respectively. When /8, when motion vectors for neighboring blocks of the current block referring to the reference picture are searched, the motion vectors of each neighboring block may be searched with a precision corresponding to the interpolation precision of the corresponding reference picture.

Referring to FIG. 24 , when the interpolation precision of the current picture and the reference pictures is equal to 1/4, when the motion vectors for neighboring blocks of the current block referencing the reference picture are searched, Motion vectors of neighboring blocks can be searched with precision. Of course, even if the precision of the block unit motion vector is determined according to implementation, the interpolation precision of the reference pictures may be set respectively.

In the above-described embodiment, it has been described that the precision of the motion vector is determined according to the interpolation precision of the reference picture, or the precision of the motion vector is determined in units of blocks while the interpolation precision is fixed. However, the present invention is not limited to such configurations. It is also possible to mix cases in which the interpolation precision of the reference picture is also set, and the precision of the motion vector is determined in units of blocks. In this case, the maximum precision of the motion vector is determined according to the precision of the referenced picture.

According to the above-described embodiment, interpolation precision may be adaptively applied in units of pictures or blocks in inter prediction. In addition, adaptive interpolation precision may be supported according to a temporal layer in the GOP structure, an average distance of referenced pictures, and the like. In addition, it is possible to reduce the amount of encoded information in encoding index information according to precision, and to enable encoding of index information in various ways.

In the above-described embodiment, the image encoding method may be used instead of the image decoding method when the motion vector precision described above is used to decode the encoded image. Also, it goes without saying that the image encoding/decoding method may be executed by an image processing apparatus or an image encoding and decoding apparatus having at least one means for encoding and decoding or a component performing a function corresponding to the means.

According to the above-described embodiment, international codecs such as MPEG-2, MPEG-4, H.264, etc. or other codecs using in-picture prediction technology, media using these codecs, and high performance generally available to the video industry It is possible to provide a high-efficiency video encoding/decoding technology. In addition, in the future, it is expected to be applied to the image processing field using the current high-efficiency image encoding technology (HEVC) and standard codecs such as H.264/AVC and intra prediction.

Although the above has been described with reference to the preferred embodiment of the present invention, those skilled in the art can variously modify and change the present invention within the scope without departing from the spirit and scope of the present invention as described in the claims below. You will understand that it can be done.

Claims (3)

decoding reference picture information indicating whether a current picture including a current block can be used as a reference picture and precision information indicating motion vector precision, wherein the reference picture information is decoded from a picture parameter set;
generating a reference picture list 0, wherein the reference picture list 0 includes a picture decoded prior to the current picture, and when the reference picture information indicates that the current picture can be used as a reference picture, the current picture further comprising;
determining a reference picture of the current block from the reference picture list 0;
determining motion vector precision of the current block based on the precision information and the determined reference picture;
obtaining a motion vector of the current block based on the motion vector precision; and
Including the step of deriving a prediction sample of the current block based on the motion vector,
The motion vector precision of the current block is determined based on whether the determined reference picture is the current picture,
The video decoding method of claim 1, wherein the current picture is allocated after the first reference picture before the current picture is allocated to the reference picture list 0.
determining whether a current picture including the current block can be used as a reference picture and motion vector precision of the current block;
generating a reference picture list 0, wherein the reference picture list 0 includes a picture decoded prior to the current picture, and when it is determined that the current picture can be used as a reference picture, further includes the current picture ;
determining a reference picture of the current block from the reference picture list 0;
obtaining a motion vector of the current block based on the motion vector precision; and
Based on whether the current picture can be used as a reference picture and motion vector precision of the current block, encoding reference picture information and precision information of the current block using a picture parameter set,
The motion vector precision of the current block is determined based on whether the determined reference picture is the current picture,
The video encoding method, characterized in that the current picture is allocated after the first reference picture before the current picture is allocated to the reference picture list 0.
In a non-transitory storage medium including a bitstream,
determining whether a current picture including the current block can be used as a reference picture and motion vector precision of the current block;
generating a reference picture list 0, wherein the reference picture list 0 includes a picture decoded prior to the current picture, and when it is determined that the current picture can be used as a reference picture, further includes the current picture ;
determining a reference picture of the current block from the reference picture list 0;
obtaining a motion vector of the current block based on the motion vector precision; and
Based on whether the current picture can be used as a reference picture and motion vector precision of the current block, encoding reference picture information and precision information of the current block using a picture parameter set,
The motion vector precision of the current block is determined based on whether the determined reference picture is the current picture,
A non-transitory storage medium including a bitstream generated by the video encoding method, wherein the current picture is allocated after the first reference picture before the current picture is allocated to the reference picture list 0.

Images