KR102447872B1 - Method and apparatus of image encoding/decoding using adaptive deblocking filtering - Google Patents

Abstract

An encoding/decoding method and apparatus for adaptive deblocking filtering are disclosed. An image encoding method for performing inter prediction and adaptive filtering using a current picture as a reference picture includes the steps of performing inter prediction on a current block to be encoded to search for a reference block; generating a flag indicating whether it is in the current picture, encoding and reconstructing the current block using the found reference block, and adaptively applying an in-loop filter to the reconstructed current block with reference to the flag and storing the current block to which the in-loop filter is applied or not applied in a decoded picture buffer (DPB).

Description

Encoding/decoding method and apparatus for adaptive deblocking filtering

The present invention relates to video encoding and decoding technology, and more particularly, to an encoding/decoding method and apparatus for adaptive deblocking filtering.

With the spread of the Internet and mobile terminals and the development of information and communication technologies, the use of multimedia data is rapidly increasing. Accordingly, the need for improving the performance and efficiency of an image processing system in order to perform various services or tasks through image prediction in various systems is significantly increasing, but research and development results that can respond to such an atmosphere are insufficient.

In addition, image encoding is divided into prediction, transformation, quantization, and entropy coding processes. In the quantization process, blocking artifacts and ringing occur in the reconstructed picture due to a quantization error. As a result, the reconstructed image There is a problem in that the subjective image quality of

Therefore, in the prior art image encoding and decoding technology using quantization, the subjective image quality is improved by performing filtering on the reconstructed picture. The prior art related to the filtering of the reconstructed picture is largely divided into a post-processing filter method and an in-loop filter method depending on whether the filtered picture is used as a reference picture in the inter prediction mode.

The post-processing filter method performs filtering immediately before the display output of the reconstructed image from outside the image decoder, and the in-loop filter method applies filtering to the reconstructed picture and inserts it into a decoded picture buffer (DPB) between screens. This method is used as a reference picture in the prediction mode.

On the other hand, the in-loop filter method, that is, the deblocking filter method, performs filtering after loading restored pixels stored in memory when filtering is performed, and stores the filtered pixels in memory again, so frequent memory accesses causes In addition, the deblocking filter also has a complex filtering operation process itself, and has a disadvantage in that it occupies a fairly large complexity of 20-30% in the decoder due to the computational complexity and overhead for memory access. As such, in the image encoding and decoding technology, an effective filtering method for the reconstructed picture is required.

An object of the present invention for solving the above problems is to provide a method for encoding/decoding an image by improving deblocking filtering.

Another object of the present invention is to provide an apparatus for encoding/decoding an image by improving deblocking filtering.

In order to achieve the above object, an image encoding method for performing adaptive filtering in inter prediction according to an aspect of the present invention includes the steps of performing inter prediction on a current block to be encoded to search for a reference block; Generating a flag indicating whether a block is within a current picture in which the current block is located, encoding and reconstructing the current block using the found reference block, referring to the flag, and an in-loop filter for the reconstructed current block adaptively applying and storing the current block to which the in-loop filter is not applied or not in a reconstructed picture buffer (Decoded Picture Buffer, DPB).

Here, the in-loop filter may include at least one of a deblocking filter and an adaptive sample offset (SAO).

Here, the step of adaptively applying the in-loop filter may not apply the in-loop filter to the reconstructed current block when the flag indicates that the reference block exists in the current picture.

Here, in the step of adaptively applying the in-loop filter, if the flag indicates that there is no reference block in the current picture, the in-loop filter may be applied to the reconstructed current block.

An image encoding method for performing inter prediction according to another aspect of the present invention for achieving the above object is an image signal including a reference block located in the same picture as a current block and matching blocks during inter prediction of the current block A step of reconstructing after encoding, storing the reconstructed image signal in a temporary memory separate from the reconstructed picture buffer (DPB), and using a reference block included in the reconstructed image signal to block the current block and performing inter-screen prediction through matching.

Here, in the step of storing in the temporary memory, the restored image signal may be stored in a separate temporary memory through an in-loop filter and a separate filter.

Here, a separate filter may perform filtering on the left and upper boundaries of the reference block included in the reconstructed image signal.

Here, the separate filter may perform filtering on at least one of a transform block boundary of the reconstructed image signal, a prediction block boundary, and a common block boundary between the transform block and the prediction block.

An image decoding method for performing adaptive filtering in inter prediction according to another aspect of the present invention for achieving the above object is an image including, from a bit stream, a reference block that is block-matched during inter prediction of a current block to be decoded reconstructing a signal, obtaining a flag indicating whether a reference block is in a current picture in which the current block is located, reconstructing the current block using the reference block, reconstructed based on the obtained flag and adaptively applying the in-loop filter to the current block, and storing the current block to which the in-loop filter is applied or not applied in a decoded picture buffer (DPB).

Here, the in-loop filter may include at least one of a deblocking filter and an adaptive sample offset (SAO).

Here, the step of adaptively applying the in-loop filter may not apply the in-loop filter to the reconstructed current block when the flag indicates that the reference block exists in the current picture.

Here, in the step of adaptively applying the in-loop filter, if the flag indicates that there is no reference block in the current picture, the in-loop filter may be applied to the reconstructed current block.

In order to achieve the above object, an image decoding method for performing inter prediction according to another aspect of the present invention includes a reference block located in the same picture as a current block from a bitstream and matching blocks during inter prediction of the current block screen through block matching with respect to the current block using the restored image signal and the restored image signal is stored in a temporary memory separate from the restored picture buffer (Decoded Picture Buffer, DPB) performing inter-prediction.

Here, in the storing in the temporary memory, the restored image signal may be stored in the temporary memory through an in-loop filter and a separate filter.

Here, a separate filter may perform filtering on the left and upper boundaries of the reference block included in the reconstructed image signal.

Here, the separate filter may perform filtering on at least one of a transform block boundary of the reconstructed image signal, a prediction block boundary, and a common block boundary between the transform block and the prediction block.

In an image decoding apparatus including one or more processors that perform adaptive filtering in inter prediction according to another aspect of the present invention for achieving the above object, the one or more processors are configured to: Reconstructing an image signal including a block-matched reference block during inter-prediction, obtaining a flag indicating whether the reference block is in a current picture in which the current block is located, from a bit stream, using the reference block to select the current block Reconstructing, adaptively applying an in-loop filter to the reconstructed current block based on the obtained flag, and reconstructing a current block to which the in-loop filter is applied or not applied to a decoded picture buffer (DPB) Steps to save to

Here, the in-loop filter may include at least one of a deblocking filter and an adaptive sample offset (SAO).

Here, the step of adaptively applying the in-loop filter may not apply the in-loop filter to the reconstructed current block when the flag indicates that the reference block exists in the current picture.

Here, in the step of adaptively applying the in-loop filter, if the flag indicates that there is no reference block in the current picture, the in-loop filter may be applied to the reconstructed current block.

When using the encoding method, the decoding method, and the decoding apparatus for performing adaptive deblocking filtering according to the embodiment of the present invention as described above, it is possible to improve the encoding and decoding efficiency by improving the performance of the deblocking filtering.

In addition, by applying an adaptive in-loop filter, additional memory consumption can be prevented.

In addition, prediction efficiency can be improved by applying a filter that is separately distinguished from the in-loop filter.

1 is a conceptual diagram of an image encoding and decoding system according to an embodiment of the present invention.
2 is a block diagram of an image encoding apparatus according to an embodiment of the present invention.
3 is a block diagram of an image decoding apparatus according to an embodiment of the present invention.
4 is an exemplary diagram illustrating inter prediction of a P slice in an image encoding and decoding method according to an embodiment of the present invention.
5 is an exemplary diagram illustrating inter prediction of a B slice in a video encoding and decoding method according to an embodiment of the present invention.
6 is an exemplary diagram for explaining a case in which a prediction block is generated unidirectionally in an image encoding and decoding method according to an embodiment of the present invention.
7 is an exemplary diagram configured from a reference picture list in a video encoding and decoding method according to an embodiment of the present invention.
8 is an exemplary diagram illustrating another example of performing inter prediction from a reference picture list in an image encoding and decoding method according to an embodiment of the present invention.
9 is an exemplary diagram for explaining intra prediction in an image encoding method according to an embodiment of the present invention.
10 is an exemplary diagram for explaining a prediction principle in a P slice or a B slice in an image encoding method according to an embodiment of the present invention.
11 is an exemplary diagram for explaining a process of obtaining a prediction block.
12 is an exemplary diagram for explaining a main process of an image encoding method according to an embodiment of the present invention with syntax in a coding unit.
13 is an exemplary diagram for explaining an example of supporting symmetric type division or asymmetric type division as in inter prediction when a prediction block is generated through block matching in the current picture.
14 is an exemplary diagram for explaining that 2N×2N and N×N can be supported in inter prediction (Inter).
15 is an exemplary diagram of generating a prediction block through block matching using a reference block in a current picture.
16 is an exemplary diagram illustrating a reference block of an already encoded region used for block matching in a current picture.
17 is a block diagram of a video encoding apparatus to which a filtering skip check unit is added.
18 is a block diagram of an image decoding apparatus to which a filtering skip check unit is added.
19 to 22 are exemplary diagrams for explaining flag transmission in blocks of various sizes in an image encoding method according to an embodiment of the present invention.
23 is an exemplary diagram for explaining a method of transmitting a flag indicating whether a reference block is used.

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention.

Terms such as first, second, A, and B may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

When a component is referred to as being “connected” or “connected” to another component, it may be directly connected or connected to the other component, but it is understood that other components may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It is to be understood that this does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

In general, a moving picture may be composed of a series of pictures, and each picture may be divided into a predetermined area such as a block. In addition, the divided region may be referred to by various sizes or terms such as a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), and a transform unit (TU) as well as a block. Each unit may include one luminance block and two color difference blocks, which may be configured differently according to a color format. Also, sizes of the luminance block and the chrominance block may be determined according to the color format. For example, in the case of 4:2:0, the size of the chrominance block may have a length equal to 1/2 of the width and length of the luminance block. For this unit, conventional terms such as HEVC or H.264/AVC may be referred to. Although block and the above terms are used interchangeably in the present invention, it may be understood differently according to standard technology, and it should be understood as a corresponding term or unit according to the encoding and decoding process according to such standard technology.

In addition, a picture, block, or pixel referenced for encoding or decoding the current block or the current pixel is referred to as a reference picture, a reference block, or a reference pixel. In addition, it is common knowledge in the technical field to which this embodiment belongs that the term "picture" described below may be used instead of other terms having the same meaning, such as an image and a frame. Grow up to understand

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. In describing the present invention, in order to facilitate the overall understanding, the same reference numerals are used for the same components in the drawings, and duplicate descriptions of the same components are omitted.

1 is a conceptual diagram of an image encoding and decoding system according to an embodiment of the present invention.

Referring to FIG. 1 , an image encoding apparatus 12 and a decoding apparatus 11 include a personal computer (PC), a notebook computer, a personal digital assistant (PDA), and a portable multimedia player (PMP). Player), PlayStation Portable (PSP: PlayStation Portable), wireless communication terminal, smart phone, TV, etc., or server terminals such as application servers and service servers, and various devices or A communication device such as a communication modem for performing communication with a wired or wireless communication network, a memory (memory, 18) for storing various programs and data for encoding or decoding an image or inter or intra prediction for encoding or decoding, and executing a program and may include various devices including a processor 14 for calculating and controlling the operation. In addition, the video encoded as a bitstream by the video encoding device 12 is transmitted in real time or in non-real time through a wired/wireless network such as the Internet, a local area network, a wireless LAN network, a WiBro network, a mobile communication network, etc., or through a cable, general purpose It may be transmitted to an image decoding apparatus through various communication interfaces such as a serial bus (USB: Universal Serial Bus), decoded in the image decoding apparatus, restored as an image, and reproduced.

Also, an image encoded as a bitstream by the image encoding apparatus may be transmitted from the encoding apparatus to the decoding apparatus through a computer-readable recording medium.

2 is a block diagram of an image encoding apparatus according to an embodiment of the present invention. 3 is a block diagram of an image decoding apparatus according to an embodiment of the present invention.

As shown in FIG. 2 , the image encoding apparatus 20 according to the present embodiment includes a prediction unit 200 , a subtraction unit 205 , a transform unit 210 , a quantization unit 215 , and an inverse quantization unit 220 . , an inverse transform unit 225 , an adder 230 , a filter unit 235 , a decoded picture buffer 240 , and an entropy encoder 245 .

In addition, as shown in FIG. 3 , the image decoding apparatus 30 according to the present embodiment includes an entropy decoding unit 305 , a prediction unit 310 , an inverse quantization unit 315 , an inverse transform unit 320 , and an adder unit. 325 , a filter unit 330 , and a decoded picture buffer 335 may be included.

The above-described image encoding apparatus 20 and image decoding apparatus 30 may be separate apparatuses, respectively, but may be made into one image encoding and decoding apparatus according to implementation. In this case, the prediction unit 200 , the inverse quantization unit 220 , the inverse transform unit 225 , the adder 230 , the filter unit 235 , and the memory 240 of the image encoding apparatus 20 operate the image in the order described. The prediction unit 310 , the inverse quantization unit 315 , the inverse transform unit 320 , the addition unit 325 , the filter unit 330 , and the memory 335 of the decoding apparatus 30 have substantially the same technical elements as at least the same technical elements. It may be implemented to include a structure or to perform at least the same function. Also, when the entropy encoder 245 performs its function in reverse, it may correspond to the entropy decoder 305 . Therefore, in the detailed description of the following technical elements and their operating principles, overlapping descriptions of corresponding technical elements will be omitted.

And since the image decoding apparatus 30 corresponds to a computing device that applies the image encoding method performed by the image encoding apparatus 20 to decoding, the image encoding apparatus 20 will be mainly described in the following description.

The computing device may include a memory that stores a program or software module implementing the image encoding method and/or the image decoding method, and a processor connected to the memory to execute the program. In addition, the image encoding apparatus may be referred to as an encoder, and the image decoding apparatus may be referred to as a decoder, respectively.

Each component of the video encoding apparatus of the present embodiment will be described in more detail as follows.

Here, the image encoding apparatus 20 may further include a division unit. The division unit divides the input image into blocks (M×N) of a predetermined size. Here, M or N is any natural number greater than or equal to 1. In detail, the dividing unit may include a picture dividing unit and a block dividing unit. The size or shape of the block may be determined according to the characteristics and resolution of the image, and the size or shape of the block supported through the picture divider is an M×N square shape (256) in which the horizontal and vertical lengths are expressed by the power of two. ×256, 128×128, 64×64, 32×32, 16×16, 8×8, 4×4, etc.) or M×N rectangular shape. For example, the input image can be divided into sizes such as 256×256 for a high-resolution 8k UHD video, 128×128 for a 1080p HD video, and 16×16 for a WVGA video.

Information on the size or shape of such a block can be set in units such as sequence, picture, slice, and the like, and related information can be transmitted to the decoder. That is, it may be set in a sequence parameter set, a picture parameter set, a slice header, or a combination unit thereof.

Here, a sequence refers to a structural unit configured by collecting several related scenes. In addition, a picture is a term that refers to a series of luminance (Y) components or all luminance + chrominance (Y, Cb, Cr) components in one scene or picture, and the range of one picture is one frame or one field in some cases can be

A slice may refer to one independent slice segment and a plurality of dependent slice segments existing in the same access unit. The access unit refers to a set of network abstraction layer (NAL) units related to one coded picture. The NAL unit is a syntax structure that composes a video compression bitstream in a network-friendly format in the H.264/AVC and HEVC standards. It is common to configure one slice unit as one NAL unit, and the system standard generally regards the NAL or NAL set constituting one frame as one access unit.

Returning to the description of the picture division unit, information on the block size or shape (M×N) may consist of an explicit flag, specifically block shape information, one length information when the block is square, and rectangular. In this case, it may include information on each length, or information on a difference value between horizontal and vertical lengths. For example, when M and N are composed of k to the power of k (assuming k is 2) (M=2 ^m , N=2 ⁿ ), information about m and n is converted into unary binarization and truncation unary binarization. The related information may be transmitted to the decoding device by encoding in various methods such as Alternatively, the minimum allowable division size (Minblksize) supported by the picture division unit is I×J (for convenience of explanation, it is assumed that I=J. In the case of I=2 ⁱ ,J=2 ^j ), information about mi or nj is transmitted. can As another example, when M and N are different, the difference value (|mn|) between m and n may be transmitted. Alternatively, the maximum allowable division size (Maxblksize) supported by the picture division unit is I×J (for convenience of explanation, it is assumed that I=J. If I=2 ⁱ ,J=2 ^j ), information about im or nj is transmitted. can

In an implicit situation, for example, when a syntax for related information exists but cannot be confirmed by an encoder or a decoder, the encoder or decoder may follow a pre-prepared default setting. For example, when the related syntax cannot be checked in the step of checking the block shape information, the block shape may be set to a default square shape. Alternatively, in the step of checking the block size information, specifically, in the step of checking the block size information through the difference value from the minimum allowable partition size (Minblksize) as in the above example, the syntax related to the difference value can be checked, but the minimum partition allowable size If the syntax related to (Minblksize) cannot be confirmed, it can be obtained from a preset value related to the minimum allowable partition size (Minblksize).

In this way, the size or shape of a block in the picture divider may be implicitly determined according to characteristics and resolution of an image, or the like by explicitly transmitting related information from an encoder or a decoder.

As described above, the block divided and determined through the picture divider may be used as a basic coding unit. In addition, a block divided and determined through the picture divider may be a minimum unit constituting a higher-level unit such as a picture, a slice, and a tile, and may include a coding block, a prediction block, and a transform block. ), quantization block, entropy block, and inloopfiltering block may be the maximum unit of a block, but some blocks are not limited thereto, and exceptions are possible.

For example, some such as in-loop filtering blocks may be applied in units larger than the block size described above.

The block divider divides blocks such as encoding, prediction, transformation, quantization, entropy, and in-loop filter. The division part is included in each component to perform a function. For example, the transform unit 210 may include a transform block division unit and the quantization unit 215 may include a quantization block division unit. The size or shape of the initial block of the block division unit may be determined by the division result of a previous stage or higher-level block. For example, in the case of a coding block, a block obtained through a picture dividing unit, which is a previous step, may be set as an initial block. Alternatively, in the case of a prediction block, a block obtained through a process of dividing a coding block that is a higher level of the prediction block may be set as an initial block. Alternatively, in the case of a transform block, a block obtained through a process of dividing a coding block that is a higher level of the transform block may be set as an initial block. Conditions for determining the size or shape of the initial block are not always fixed, and some may be changed or there may be exceptions. In addition, the partition state of the previous stage or higher level block (eg, the size of the coding block, the type of the coding block, etc.) and the setting condition of the current level (eg, the size of the supported transform block, the type of the transform block, etc.) ) each may affect the current level division operation (whether division is possible, the type of block that can be divided, etc.) according to a combination of at least one or more factors.

The block divider may support a quad tree-based division method. That is, it can be divided into four blocks having a length of 1/2 each in width and length from the block before division. This is based on the initial block (dep_0) and the allowable division depth limit (dep_k, k means the number of allowed divisions, and the block size when the division allowable depth limit (dep_k) is (M >> k, N >> k)) The division can be repeated until

In addition, a binary tree-based partitioning method may be supported. This indicates that it can be divided into two blocks having a length of one-half compared to the block before division, either horizontally or vertically. In the case of the quad tree partitioning and binary tree partitioning, symmetric partitioning or asymmetric partitioning may be used, and it is possible to determine which partitioning method to follow according to the setting of an encoder or a decoder. In the video encoding method of the present invention, the description will be focused on the symmetric segmentation method.

Whether or not each block is divided can be indicated through the division flag (div_flag). If the corresponding value is 1, division is performed, and if the value is 0, division is not performed. Alternatively, if the corresponding value is 1, partitioning is performed and further partitioning is possible. If the value is 0, partitioning is not performed and further partitioning may not be allowed. According to conditions such as the minimum allowable division size and the limit of the allowable division depth, the flag may only consider whether to divide and not consider whether to further divide.

The split flag can be used for quad-tree splits and can also be used for binary tree splits. In binary tree partitioning, the partitioning direction may be one of the partition depth, encoding mode, prediction mode, size, shape, type (encoding, prediction, transformation, quantization, entropy, in-loop filter, etc.) may be one), and may be determined according to at least one or a combination of factors such as a slice type, a partition allowable depth limit, and a partition allowable minimum and maximum size.

In that case, according to the division flag, only the width of the block may be divided into 1/2 or only the length of the block may be divided according to a corresponding division direction. For example, if a block is M×N (M>N), which supports horizontal division when M is greater than N, and the current division depth (dep_curr) is smaller than the division allowable depth limit, further division is possible. The division flag is allocated as 1 bit, and if the corresponding value is 1, horizontal division is performed, and if it is 0, no further division may be performed. As for the split depth, one split depth may be placed in the quad tree and binary tree split, and each split depth may be set in the quad tree and binary tree split. In addition, the partition allowable depth limit may set one partition allowable depth limit for quad tree and binary tree partitioning, and may set each partition allowable depth limit for quad tree and binary tree partitioning.

As another example, if the block is M×N (M>N) and N is equal to the preset minimum allowable partition size, so horizontal partitioning is not supported, the above partitioning flag is assigned as 1 bit and if the corresponding value is 1, vertical partitioning is performed, , if it is 0, no partitioning is performed.

In addition, flags div_h_flag and div_v_flag for horizontal division or vertical division may be supported, respectively, and binary division may be supported according to the flags. Horizontal or vertical division of each block can be indicated through the horizontal division flag (div_h_flag) or vertical division flag (div_v_flag). If the horizontal division flag (div_h_flag) or vertical division flag (div_v_flag) is 1, horizontal or vertical division is performed. If it is 0, horizontal or vertical division is not performed. Alternatively, if each flag is 1, horizontal or vertical division is performed and additional horizontal or vertical division is possible. If the value is 0, horizontal or vertical division is not performed and further horizontal or vertical division is not allowed. . According to conditions such as the minimum allowable partition size and the limit of the allowable partitioning depth, whether the flag is divided may be considered and whether or not to be further divided may not be considered. Alternatively, a flag (div_flag/h_v_flag) for horizontal division or vertical division may be supported, and binary division may be supported according to the flag. The division flag div_flag may indicate horizontal or vertical division, and the division direction flag h_v_flag may indicate a horizontal or vertical division direction. When the division flag div_flag is 1, division is performed, horizontal or vertical division is performed according to the division direction flag h_v_flag, and when 0, horizontal or vertical division is not performed. Alternatively, if the value is 1, horizontal or vertical division is performed according to the division direction flag (h_v_flag) and additional horizontal or vertical division is possible. If the value is 0, horizontal or vertical division is not performed and no further horizontal or vertical division is performed. may be considered not to be allowed. According to conditions such as the minimum allowable partition size and the limit of the allowable partitioning depth, whether the flag is divided may be considered and whether or not to be further divided may not be considered.

These split flags may also be supported for horizontal and vertical splitting, and binary tree splitting may be supported according to the flag. In addition, when the division direction is predetermined, as in the above example, only one of the two division flags may be used, or both division flags may be used.

For example, if all of the above flags are allowed, the possible block shape may be divided into any one of M×N, M/2×N, M×N/2, and M/2×N/2. In this case, the flags may be coded as 00, 01, 10, and 11 (in the order of div_h_flag/div_v_flag). The above is an example of a setting in which the division flags can be overlapped and used, and a setting in which the division flags cannot be overlapped and used is also possible. For example, the divided block shape may be divided into M×N, M/2×N, and M×N/2. In this case, the flags above are encoded as 00, 01, 10 (in the order of div_h_flag/div_v_flag) or , (in the order of div_flag/h_v_flag. h_v_flag is a flag indicating a horizontal or vertical division direction) 0, 10, and 11 may be encoded. Here, the meaning of overlap may mean performing horizontal division and vertical division at the same time. Any one of the above-described quad tree splitting and binary tree splitting may be used alone or in combination according to the setting of an encoder or a decoder. For example, a quadtree or binary tree division may be determined according to a block size or shape. That is, when the block shape is M×N and M is greater than N, horizontal division, the block shape is M×N, and when N is greater than M, binary tree division may be supported according to vertical division, and the block type In the case where is M×N and N and M are the same, quadtree splitting may be supported.

As another example, binary tree splitting may be supported when the size of the block (M×M) is greater than or equal to the block splitting boundary value Thrblksize, and quadtree splitting may be supported if it is smaller than that.

* As another example, when M or N of a block (M×N) is less than or equal to the first allowable maximum size for partitioning (Maxblksize1) and greater than or equal to the first minimum allowable size for partitioning (Minblksize1), quad-tree partitioning is supported, When M or N of a block (M×N) is less than or equal to the second maximum allowable size for partitioning (Maxblksize2) and greater than or equal to the second minimum allowable size for partitioning (Minblksize2), binary tree partitioning may be supported. If the first division support range and the second division support range that can be defined by the maximum allowable division size and the minimum division allowable size overlap, the first or the second division method takes precedence according to the setting of the encoder or decoder. A ranking may be given. In this example, the first partitioning method may be quad-tree partitioning, and the second partitioning method may be binary tree partitioning. For example, if the first partition allowable minimum size (Minblksize1) is 16, the second partition allowable maximum size (Maxblksize2) is 64, and the block before division is 64×64, the first partition support range and the second partition support range are Since they belong to all of them, quad tree splitting and binary tree splitting are possible. If priority is given to the first division method (quad-tree division in this example) according to a preset, when the division flag (div_flag) is 1, quad-tree division is performed and additional quad-tree division is possible. does not perform quad-tree splitting, and it can be considered that no more quad-tree splits are performed. Alternatively, if the split flag (div_flag) is 1, quad tree split is performed and additional quad tree or binary tree split is possible. If it is 0, quad tree split is not performed and no quad tree split is performed, but binary The tree can be considered as splittable.

According to conditions such as the minimum allowable division size and the limit of the allowable division depth, the flag may only consider whether to divide and not consider whether to further divide. If the division flag div_flag is 1, since the division is divided into four blocks having a size of 32×32 and is larger than the first allowable minimum size for division (Minblksize1), quad tree division can be continued. If it is 0, additional quad tree splitting is not performed, and since the current block size (64×64) falls within the second split support range, binary tree splitting can be performed. When the division flag (in the order of div_flag/h_v_flag) is 0, no further division is performed, and when it is 10 or 11, horizontal division or vertical division can be performed. If the block before division is 32x32 and the division flag (div_flag) is 0 so that no more quad tree division is performed and the maximum size of the second division (Maxblksize2) is 16, the size of the current block (32x32) is Since it does not belong to the 2 division support range, it may not support further division. In the above description, the priority of the partitioning method may be determined according to at least one factor among a slice type, an encoding mode, a luminance and a chrominance component, or a combination thereof.

As another example, various settings may be supported according to luminance and chrominance components. For example, a quad-tree or binary-tree split structure determined in the luminance component may be used as it is in the chrominance component without additional information unit or decoding. Alternatively, when independent division of the luminance component and the chrominance component is supported, quad tree + binary tree division for the luminance component and quad tree division for the chrominance component may be supported. Alternatively, quad tree + binary tree division is supported in the luminance and chrominance components, but the division support range may or may not be the same or proportional to the luminance and chrominance components. For example, when the color format is 4:2:0, the division support range of the chrominance component may be N/2 of the division support range of the luminance component.

As another example, different settings may be set according to the slice type. For example, an I slice may support quad tree partitioning, a P slice may support binary tree partitioning, and a B slice may support quad tree + binary tree partitioning.

As in the above example, quad tree splitting and binary tree splitting may be set and supported according to various conditions. The above examples are not specific only to the above cases, and may include cases in which each other's conditions are reversed, and may include one or more factors mentioned in the above examples or a combination thereof. do. The above partition allowable depth limit may be determined according to at least one factor or a combination thereof, such as a partitioning method (quad tree, binary tree), slice type, luminance and chrominance components, encoding mode, and the like. In addition, the division support range may be determined according to at least one factor or a combination thereof, such as a division method (quad tree, binary tree), slice type, luminance and chrominance components, encoding mode, etc. It can be expressed as a maximum value and a minimum value. When this information is configured as an explicit flag, length information of each of the maximum value and the minimum value or information on the difference value between the minimum value and the maximum value may be expressed. For example, when the maximum value and the minimum value are configured to the power of k (assuming that k is 2), exponent information of the maximum value and the minimum value may be encoded through various binarizations and transmitted to the decoding apparatus. Alternatively, a difference value between the exponents of the maximum value and the minimum value may be transmitted. In this case, the transmitted information may be index information of the minimum value and information on the difference value of the index.

According to the above description, flag-related information may be generated and transmitted in units of a sequence, a picture, a slice, a tile, a block, and the like.

The split flags presented as examples above can indicate block split information through a quad tree or binary tree or a mixture of two tree methods. can be passed to the device. A bitstream structure of a partition flag for expressing partition information of the block may be selected from one or more scan methods. For example, a bitstream of the division flags may be configured based on the division depth order (in dep0 to dep_k order), and a bitstream of division flags may be configured based on whether division is performed. In the division depth order reference method, the division information at the current level is obtained based on the first block and then division information at the next level is obtained. It refers to a method of preferentially acquiring additional segmentation information, and in addition to this, other scan methods not presented in the above example may be included and selected.

Also, according to implementation, the block partitioning unit may generate and express index information on a predefined type of block candidate group instead of the aforementioned partitioning flag. The shape of the block candidate group is, for example, the shape of a divided block that can be in the block before division, such as M×N, M/2×N, M×N/2, M/4×N, 3M/4×N, It may include M×N/4, M×3N/4, M/2×N/2, and the like. When the candidate group of the divided block is determined as described above, the index information for the divided block type can be encoded through various methods such as fixed-length binarization, short-cut binarization, and truncated binarization. Like the partition flag described above, at least one or a combination of factors such as the partition depth, encoding mode, prediction mode, size, shape, type and slice type, partition allowable depth limit, and minimum and maximum partition size of the block. Accordingly, a partition block candidate group may be determined. For the following explanation, (M×N, M/2×N/2) is candidate list1, (M×N, M/2×N, M×N/2, M/2×N/2) is candidate list2, (M×N, M/2×N, M×N/2) candidate list3, (M×N, M/2×N, M×N/2, M/4×N, 3M/4×N, It is assumed that M×N/4, M×3N/4, M/2×N/2) are candidate list4. For example, when explaining based on M×N, when (M=N), the partition block candidate of candidate list2 can be supported, and when (M≠N), the partition block candidate of candidate list3 can be supported.

As another example, when M or N of M×N is greater than or equal to the boundary value blk_th, the partition block candidate of candidate list2 may be supported, and when smaller than that, the partition block candidate of candidate list4 may be supported. Alternatively, when M or N is greater than or equal to the first boundary value (blk_th_1), the partition block candidate of the candidate list1 is selected as a candidate when smaller than the first boundary value (blk_th_1) but greater than or equal to the second boundary value (blk_th_2). When the divided block candidate of list2 is smaller than the second boundary value blk_th_2, the divided block candidate of candidate list4 may be supported.

As another example, when the encoding mode is intra prediction, the split block candidate of candidate list2 may be supported, and when the encoding mode is inter prediction, the split block candidate of candidate list4 may be supported.

Even if the split block candidates as described above are supported, the bit configuration according to binarization in each block may be the same or different. For example, if the supported partition block candidates are limited according to the block size or shape as applied to the partition flag above, the bit configuration according to the binarization of the corresponding block candidate may vary. For example, in the case of (M>N), block types according to horizontal division, that is, M×N, M×N/2, M/2×N/2 can be supported, and a group of divided block candidates (M×N, M/ In 2×N, M×N/2, M/2×N/2), binary bits of an index according to M×N/2 and M×N/2 of the current condition may be different from each other. According to the type of block, for example, the type of block used for encoding, prediction, transformation, quantization, entropy, in-loop filtering, etc., information on the division and shape of the block may be expressed using either a division flag or a division index method. In addition, depending on the type of each block, a block size limit for partitioning and block shape support, a partition allowable depth limit, etc. may be different.

In the block-by-block encoding or decoding process, after a coding block is first determined, encoding or decoding may be performed according to processes such as prediction block determination, transform block determination, quantization block determination, entropy block determination, and in-loop filter determination. The order of the encoding or decoding process is not always fixed, and some orders may be changed or excluded. The size and shape of each block may be determined according to the encoding cost of each candidate for the size and shape of the block, and segmentation related information such as determined image data of each block and the determined size and shape of each block may be encoded.

The prediction unit 200 may be implemented using a prediction module, which is a software module, and may generate a prediction block with respect to a block to be encoded using an intra prediction method or an inter prediction method. Here, the prediction block is a block that is understood to closely match the block to be encoded in terms of pixel difference, and may be determined by various methods including sum of absolute difference (SAD) and sum of square difference (SSD). Also, in this case, various syntaxes that can be used when decoding image blocks may be generated. The prediction block may be classified into an intra-picture block and an inter-picture block according to an encoding mode.

Intra prediction is a prediction technique using spatial correlation, and refers to a method of predicting a current block using reference pixels of previously encoded, decoded, and reconstructed blocks in the current picture. That is, in the intra prediction, the brightness value reconstructed by prediction and restoration may be used as a reference pixel in the encoder and the decoder. The intra prediction may be effective for a flat area with continuity and an area with a constant direction, and may be used for the purpose of ensuring random access and preventing error propagation because spatial correlation is used.

Inter prediction uses a compression technique that removes data duplication by using temporal correlation with reference to an image encoded in one or more past or future pictures. That is, the inter prediction may generate a prediction signal with high similarity by referring to one or more past or future pictures. An encoder using inter prediction may search a reference picture for a block having a high correlation with a block to be currently encoded in a reference picture, and transmit location information and residual signal of the selected block to a decoder, and the decoder may transmit selection information of the transmitted image can be used to generate the same prediction block as the encoder and compensate for the transmitted residual signal to compose a reconstructed image.

4 is an exemplary diagram illustrating inter prediction of a P slice in an image encoding and decoding method according to an embodiment of the present invention. 5 is an exemplary diagram illustrating inter prediction of a B slice in a video encoding and decoding method according to an embodiment of the present invention.

In the video encoding method of the present embodiment, since inter prediction generates a prediction block from a previously encoded picture having high temporal correlation, encoding efficiency may be increased. Current(t) may mean a current picture to be encoded, and a first temporal distance (t) before the POC of the current picture based on a temporal flow of video pictures or a picture order count (POC). It may include a first reference picture t-1 having -1) and a second reference picture t-2 having a second temporal distance t-2 that is earlier than the first temporal distance.

That is, as shown in FIG. 4 , the inter prediction that can be employed in the video encoding method of the present embodiment is performed between the current block of the current picture current(t) and the reference pictures t-1 and t-2. Through block matching of reference blocks, motion estimation may be performed to find an optimal prediction block from reference pictures t-1 and t-2 in which blocks with high correlation have been previously encoded. For precise estimation, an interpolation process is performed based on a structure in which at least one or more sub-pixels are arranged between two adjacent pixels as needed, and then an optimal prediction block is found and motion compensation is performed to make a final prediction block can be found

In addition, as shown in FIG. 5 , the inter prediction that can be employed in the video encoding method of the present embodiment includes reference pictures ( A prediction block may be generated from t-1, t+1). Also, two prediction blocks may be generated from one or more reference pictures.

When an image is encoded through inter prediction, motion vector information on an optimal prediction block and information on a reference picture are encoded. In the present embodiment, when the prediction block is generated unidirectionally or bidirectionally, the prediction block may be generated from the reference picture list by configuring the reference picture list differently. Basically, a reference picture existing temporally before the current picture may be allocated to list 0, and a reference picture existing after the current picture may be allocated to and managed from list 1. When composing the reference picture list 0, if the reference picture list 0 does not fill up to the allowable number of reference pictures, reference pictures existing after the current picture may be allocated. Similarly, when composing the reference picture list 1, if the reference picture list 1 does not fill up to the allowable number of reference pictures, a reference picture existing before the current picture may be allocated.

6 is an exemplary diagram for explaining a case in which a prediction block is generated unidirectionally in an image encoding and decoding method according to an embodiment of the present invention.

Referring to FIG. 6 , in the image encoding and decoding method according to the present embodiment, a prediction block can be found from previously encoded reference pictures t-1 and t-2, and in addition to the current picture (current( t)), the prediction block can be found from the region that has already been coded.

That is, in the video encoding and decoding method according to the present embodiment, not only the prediction block is generated from the previously coded pictures t-1 and t-2 having high temporal correlation, but also the prediction block with high spatial correlation can be implemented to find together. Finding such a spatially highly correlated prediction block may correspond to finding the prediction block in the manner of intra prediction. In order to perform block matching from an encoded region in the current picture, the image encoding method of the present embodiment may be mixed with an intra prediction mode to construct a syntax for information related to a prediction candidate.

For example, if n (n is an arbitrary natural number) intra-prediction modes are supported, one mode is added to the intra-prediction candidate group to support n+1 modes, and 2 ^M-1 ≤ n+1 A prediction mode may be coded using M fixed bits satisfying <2 ^M. In addition, it may be implemented to select from a candidate group of a highly probable prediction mode, such as a most probable mode (MPM) of high efficiency video coding (HEVC). In addition, encoding may be preferentially performed in a higher stage of prediction mode encoding.

When a prediction block is generated through block matching in the current picture, the image encoding method of the present embodiment may be mixed with an inter prediction mode to configure a syntax for related information. As additional related prediction mode information, motion or displacement related information may be used. The motion or movement-related information may include optimal candidate information among multiple vector candidates, a difference between an optimal candidate vector and an actual vector, a reference direction, reference picture information, and the like.

7 is an exemplary diagram configured from a reference picture list in a video encoding and decoding method according to an embodiment of the present invention. 8 is an exemplary diagram illustrating another example of performing inter prediction from a reference picture list in an image encoding and decoding method according to an embodiment of the present invention.

Referring to FIG. 7 , in the image encoding method according to the present embodiment, inter prediction can be performed from reference picture lists (reference list 0, reference list 1) with respect to a current block of a current picture (current(t)). have.

Referring to FIGS. 7 and 8 , the reference picture list 0 may consist of a reference picture before the current picture t, where t-1 and t-2 are picture order counts of the current picture t, respectively. Count, POC) indicates reference pictures having a first temporal distance (t-1) and a second temporal distance (t-2) before. In addition, the reference picture list 1 may consist of reference pictures after the current picture (t), where t+1 and t+2 are respectively a first temporal distance (t+1) after the POC of the current picture (t). ), indicating reference pictures having a second temporal distance (t+2).

The above-described examples of the reference picture list configuration show an example of configuring the reference picture list with reference pictures having a difference in temporal distance (the POC standard in this example) of 1, but the difference in temporal distance between the reference pictures is configured differently You may. That is, it means that the index difference of the reference pictures and the temporal distance difference of the reference pictures may not be proportional. In addition, the list construction order may not be configured based on a temporal distance. This can be confirmed in the reference picture list configuration example to be described later.

Prediction may be performed from the reference picture in the list according to the slice type (I, P, or B). And when a prediction block is generated through block matching in the current picture (current(t)), the current picture is added to the reference picture lists (reference list 0, reference list 1) to perform encoding in an inter prediction method. .

As shown in FIG. 8 , a current picture t may be added to a reference picture list 0 or a current picture t may be added to a reference picture list 1 . That is, the reference picture list 0 can be configured by adding a reference picture having a temporal distance (t) to the reference picture before the current picture (t), and the reference picture list 1 is the reference picture after the current picture (t) in time It can also be configured by adding a reference picture that is a specific distance t.

For example, when constructing reference picture list 0, a reference picture before the current picture may be assigned to reference picture list 0 and then the current picture (t) may be assigned, and when constructing reference picture list 1, a reference picture after the current picture The reference picture may be allocated to the reference picture list 1 and then the current picture (t) may be allocated. Alternatively, the current picture (t) may be allocated when composing the reference picture list 0, and then the reference picture before the current picture may be allocated. Subsequent reference pictures may be allocated. Alternatively, when constructing the reference picture list 0, a reference picture before the current picture may be allocated, then a reference picture after the current picture may be allocated, and the current picture t may be allocated. Similarly, when configuring the reference picture list 1, a reference picture after the current picture may be allocated, then a reference picture before the current picture may be allocated, and the current picture t may be allocated. The above examples are not specific only to the above-described case, and may include a case in which each other's conditions are reversed, and variations in other cases are also possible.

Whether or not to include the current picture in each reference picture list (e.g., do not add to any list or add to list 0 only or add to list 1 only or add to lists 0 and 1 together) has the same setting in the encoder or decoder. possible, and information on this can be transmitted in units of sequences, pictures, slices, and the like. This information may be encoded through a method such as fixed-length binarization, short cut-type binarization, or cut-type binarization.

Unlike the method of FIG. 7, the image encoding and decoding method of this embodiment selects a prediction block by performing block matching on the current picture t, and constructs a reference picture list including information related to the prediction block, , there is a difference in using this reference picture list for video encoding and decoding.

In the reference picture list configuration, the settings for each list configuration order and rule, and the allowable number of reference pictures in each list can be changed, which is whether the current picture is included in the list (whether the current picture is included as a reference picture in inter prediction) Whether or not), slice type, list reconstruction parameter (which may be applied to lists 0 and 1, respectively, or may be applied to lists 0 and 1, respectively), location in a picture set (Group of Picture, GOP), temporal hierarchical information (temporal id) It may be determined according to at least one factor or a combination thereof among various factors, such as, and the related information may be explicitly transmitted in units of sequences, pictures, and the like. For example, in the case of a P slice, the reference picture list 0 may follow the list construction rule A regardless of whether the current picture is included in the list, and in the case of a B slice, the reference picture list 0 including the current picture in the list consists of a list Rule B, the list construction rule C may be followed for the reference picture list 1, the list construction rule D for the reference picture list 0 that does not contain the current picture, and the list construction rule E for the reference picture list 1 may be followed, and the list construction rule Among them, B and D and C and E may be the same. The list construction rule may be configured in the same or modified manner as described in the reference picture list construction example. As another example, when the current picture is included in the list, the allowed number of first reference pictures may be set, and when the current picture is not included, the allowed number of second reference pictures may be set. The allowable number of first reference pictures and the allowable number of second reference pictures may be the same or different, and a difference between the allowable number of first reference pictures and the allowable second reference pictures may be set as a default setting. As another example, when the current picture is included in the list and the list reconstruction parameter is applied, in slice A, all reference pictures may become a list reconstruction candidate group, and in slice B, only some reference pictures may be included in the list reconstruction candidate group. At this time, slice A or B may be classified according to whether the list of current pictures is included, temporal hierarchical information, slice type, position in a group of pictures (GOP), etc. It may be determined by a picture order count (POC) or a reference picture index, a reference prediction direction (before and after the current picture), whether a current picture is present, or the like.

According to the configuration described above, since the reference block encoded by inter prediction in the current picture can be used, inter prediction can be allowed or used even in I-slice motion prediction.

In addition, when constructing the reference picture list, index assignment or list construction order may be different depending on the slice type. In the case of I slice, the current picture (current(t)) uses fewer indexes (eg, idx=0, 1, 2) by increasing the priority as in the reference picture list configuration example above, and the corresponding The amount of bits in video encoding can be reduced through binarization (fixed-length binarization, truncated binarization, truncated binarization, etc.) in which the allowable number of reference pictures (C) of the reference picture list is the maximum value. In addition, in the case of a P or B slice, if it is determined that the probability of selecting the reference picture of the current block as a prediction candidate by performing block matching on the current picture is lower than the probability of selecting a prediction candidate through other reference pictures, the current picture Binarization of various methods to set the priority for block matching of Through this, it is possible to reduce the amount of bits in video encoding. In the above example, the priority setting of the current picture may be configured in the same or modified manner as described in the reference picture list configuration example. Also, it is possible to omit information on the reference picture by not configuring the reference picture list according to the slice type (eg, I slice). For example, the prediction block may be generated through the existing inter prediction, but the inter prediction information may be expressed by excluding the reference picture information from the motion information in the inter prediction mode.

A method of performing block matching in the current picture may determine whether to support it according to a slice type. For example, block matching in the current block may be set to be supported in I slice but not supported in P slice or B slice, and a modification to another example is also possible. In addition, the method of supporting block matching in the current picture may be determined in units of pictures, slices, tiles, etc., or may be determined according to a location in a group of pictures (GOP), temporal hierarchical information (temporal ID), etc. have. Such setting information may be transmitted in units of sequences, pictures, slices, etc., during an image encoding process or from an encoder to a decoder. In addition, even if the above-related setting information or syntax exists in the upper level unit and the setting-related operation is on, when the same setting information or syntax as above exists in the lower level unit, the setting information in the lower level unit is displayed at the upper level The setting information in the unit may be given priority. For example, if the same or similar configuration information is processed in a sequence, a picture, or a slice, a picture unit rather than a sequence unit and a slice unit rather than a picture unit may have priority. . For example, the flag supporting block matching in the current picture in the sequence parameter may be sps_curr_pic_BM_enabled_flag, and the flag supporting block matching in the current picture in the picture parameter may be pps_curr_pic_BM_enabled_flag. If sps_curr_pic_BM_enabled_flag is on and pps_curr_pic_BM_enabled_flag is off, block matching may not be supported in the current picture. Whether the current picture is included in the reference picture list 0 configuration may be determined according to the flag, and whether the current picture is included in the reference picture list 1 configuration may be determined.

* FIG. 9 is an exemplary diagram for explaining intra prediction in an image encoding method according to an embodiment of the present invention.

Referring to FIG. 9 , the intra prediction method according to the present embodiment includes reference sample padding, reference sample filtering, intra prediction, and boundary filtering. It may include a series of steps.

The reference pixel filling step may be an example of the reference pixel construction step, the reference pixel filtering step may be referred to as a reference pixel filter unit, and intra prediction may include a prediction block generation step and a prediction mode encoding step, and boundary filtering may be an example of an embodiment of a post-processing filter step.

That is, the intra prediction executed in the image encoding apparatus of the present embodiment may include a reference pixel construction step, a reference pixel filtering step, a prediction block generation step, a prediction mode encoding step, and a post-processing filtering step. Depending on various environmental factors, such as block size, block shape, block position, prediction mode, prediction method, quantization parameter, etc., one or some of the above-described processes may be omitted, and another process may be added, and the It may be changed in an order other than the order.

The above-described reference pixel construction step, reference pixel filtering step, prediction block generation step, prediction mode encoding step, and post-processing filtering step may be implemented in a form in which a processor connected to the memory executes software modules stored in the memory. Therefore, in the following description, for convenience of explanation, a reference pixel component, a reference pixel filter unit, A prediction block generator, a prediction mode encoder, and a post-processing filter will be referred to as execution subjects of each step.

When each component is described in more detail, the reference pixel component configures a reference pixel to be used for prediction of the current block by filling the reference pixel. If the reference pixel does not exist or is unavailable, the reference pixel fill can be used for the reference pixel, such as by copying a value from an available nearby pixel. A decoded picture buffer (DPB) may be used for copying the value.

That is, the intra prediction is performed using reference pixels of previously encoded blocks of the current picture. To this end, in the reference pixel construction step, neighboring blocks of the current block, ie, adjacent pixels such as left, upper-left, lower-left, upper, and upper-right blocks, are mainly used as reference pixels. The candidate group of the neighboring blocks for the reference pixel is only an example of a case in which the encoding order of the blocks follows a raster scan or a z-scan, and a scan such as an inverse z-scan is used. In the case of using the coding order scan method, adjacent pixels such as the right, lower right, and lower blocks in addition to the above blocks may be used as reference pixels.

In addition, depending on implementation, additional pixels other than immediately adjacent pixels may be used in combination with replacement or existing reference pixels according to the stepwise configuration of intra prediction.

In addition, when prediction is performed in a directional mode among intra-prediction modes, a decimal unit reference pixel may be generated through linear interpolation of an integer unit reference pixel. Modes for performing prediction through reference pixels existing in integer unit positions include some modes having vertical, horizontal, 45 degrees, and 135 degrees, and for the above prediction modes, the process of generating a fractional reference pixel is necessary may not In the prediction modes having a direction other than the prediction mode, the interpolated reference pixel is 1/2, 1/4, 1/8, 1/16, 1/32, 1/64, such as 1/2, 1/4, 1/8, 1/16, 1/32, 1/64. It may have precision, or it may have a precision of multiples of 1/2. This is because interpolation precision may be determined according to the number of supported prediction modes or prediction directions of prediction modes. A picture, a slice, a tile, a block, etc. may always support a fixed interpolation precision, or an adaptive interpolation precision may be supported according to a size of a block, a shape of a block, a prediction direction of a supported mode, and the like. In this case, the prediction direction of the mode may be expressed as inclination information or angle information of the direction indicated by the mode with respect to a specific line reference (eg, positive <+> x-axis on the coordinate plane).

As the interpolation method, linear interpolation is sometimes performed through immediately adjacent integer pixels, but other interpolation methods may be supported. One or more types of filters and the number of taps for interpolation For example, a 6-tap Wiener filter, an 8-tap Kalman filter, etc. can be supported, and it is possible to determine which interpolation to perform according to the size of the block, the prediction direction, etc. have. In addition, related information may be transmitted in units of sequences, pictures, slices, blocks, and the like.

The reference pixel filter unit may perform filtering on the reference pixel in order to increase prediction efficiency by reducing deterioration remaining in the encoding process after configuring the reference pixel. The reference pixel filter unit may implicitly or explicitly determine the type of filter and whether to apply the filtering according to the size, shape, and prediction mode of the block. That is, even filters having the same tap may determine different filter coefficients depending on the filter type. For example, a 3-tap filter such as [1,2,1]/4 or [1,6,1]/8 can be used.

In addition, the reference pixel filter unit may determine whether to apply filtering by determining whether to additionally transmit a bit or not. For example, in the implicit case, the reference pixel filter unit may determine whether to apply filtering according to characteristics (variance, standard deviation, etc.) of pixels in the neighboring reference block.

Also, the reference pixel filter unit may determine whether filtering is applied when the related flag satisfies a hiding condition preset for a residual coefficient, an intra prediction mode, or the like. The number of taps of the filter is, for example, 3-tap such as [1,2,1]/4 for a small block (blk), and [2,3,6,3,2]/16 and [2,3,6,3,2]/16 for a large block (blk). It may be set to the same 5-tap, and the number of times of application may be determined by not performing filtering, filtering once, filtering twice, and the like.

Also, the reference pixel filter unit may basically apply filtering to the nearest reference pixel of the current block. In addition to the nearest reference pixel, additional reference pixels may also be considered in the filtering process. For example, filtering may be applied to additional reference pixels by replacing the closest reference pixel, or filtering may be applied by mixing additional reference pixels to the closest reference pixel.

The filtering may be applied statically or adaptively, which includes the size of the current block or the size of the neighboring block, the encoding mode of the current block or the neighboring block, and block boundary characteristics of the current block and the neighboring block (eg, the boundary of the coding unit). It may be determined according to at least one or a combination of factors such as the boundary of the cognitive transformation unit, etc.), the prediction mode or direction of the current block or the neighboring block, the prediction method of the current block or the neighboring block, and a quantization parameter. . Determination for this may have the same setting in the encoder or decoder (implicit), or may be determined in consideration of encoding cost and the like (explicit). The basically applied filter is a low pass filter, and the number of filter taps, filter coefficients, filter flag encoding, the number of times the filter is applied, etc. can be determined according to various factors specified above. It can be set in units such as slices and blocks, and related information can be transmitted to the decoder.

The prediction block generator performs an extrapolation or extrapolation method through a reference pixel in intra prediction, an interpolation method such as an average value (DC) of reference pixels or a planar mode, or a copy of the reference pixel. ) to generate a prediction block. In the case of copying of reference pixels, one reference pixel may be copied to generate one or more prediction pixels, and one or more reference pixels may be copied to generate one or more prediction pixels, and the number of copied reference pixels is It may be equal to or less than the number of prediction pixels.

Also, it is possible to classify the directional prediction method and the non-directional prediction method according to the prediction method, and in detail, the directional prediction method can be classified into a linear direction method and a curve direction method. The linear directional scheme borrows extrapolation or extrapolation scheme, but the pixels of the prediction block are generated by reference pixels lying on the prediction direction line, and the curved directional scheme borrows the extrapolation or extrapolation scheme, but the pixels of the prediction block are reference lying on the prediction direction line. This refers to a method in which a partial prediction direction change in units of pixels is allowed in consideration of the detailed directionality (eg, edge<Edge>) of a block while being generated through pixels. In the case of the directional prediction mode in the image encoding and decoding method of the present invention, the description will be focused on the linear directional method. In addition, in the case of the directional prediction method, intervals between adjacent prediction modes may be uniform or non-uniform, which may be determined according to a size or shape of a block. For example, when a block having a size and shape of M×N is obtained through the block divider, when M and N are the same, the interval between prediction modes may be uniform, and when M and N are different Intervals between prediction modes may be non-uniform. As another example, when M is greater than N, a finer interval may be allocated between prediction modes close to the vertical mode (90 degrees) for modes having vertical directionality, and a wide interval may be allocated to a prediction mode far from the vertical mode. When N is greater than M, a finer interval may be allocated between prediction modes close to the horizontal mode (180 degrees) for modes having horizontal directivity, and a wide interval may be allocated to a prediction mode far from the horizontal mode. The above examples are not specific only to the above-described case, and may include a case in which each other's conditions are reversed, and variations in other cases are also possible. In this case, the interval between the prediction modes may be calculated based on a numerical value indicating the directionality of each mode, and the directionality of the prediction mode may be quantified as gradient information or angle information of the direction.

In addition, in addition to the above method, it is possible to generate the prediction block by including other methods using spatial correlation. For example, by using the current picture as a reference picture, a reference block using an inter prediction method such as motion search and compensation may be generated as a prediction block. The generating of the prediction block may generate the prediction block by using the reference pixel according to the prediction method. That is, according to the prediction method, a prediction block can be generated through a directional prediction method such as extrapolation, interpolation, copying, averaging, etc. or a non-directional prediction method of the existing intra prediction method according to the prediction method, and the prediction block can be generated using the inter prediction method can be created, and other additional methods may also be used.

The intra prediction method may be supported under the same setting of an encoder or a decoder, and may be determined according to a slice type, a size of a block, a block shape, and the like. The intra prediction method may be supported according to at least one of the above-mentioned prediction methods or a combination thereof. The intra prediction mode may be configured according to the supported prediction method. The number of supported intra prediction modes may be determined according to the prediction method, slice type, block size, block type, and the like. The related information can be set and transmitted in units such as sequence, picture, slice, and block.

In the prediction mode encoding step performed by the prediction mode encoding, a mode having an optimal encoding cost according to each prediction mode in terms of encoding cost may be determined as the prediction mode of the current block.

For example, the prediction mode encoder may use the modes of one or more neighboring blocks for the current block mode prediction for the purpose of reducing the prediction mode bits. A mode (most_probable_mode, MPM) having a high probability of being the same as the mode of the current block may be included as a candidate group, and modes of a neighboring block may be included in the above candidate group. For example, prediction modes of blocks such as the left, upper left, lower left, upper, and upper right of the current block may be included in the above candidate group.

The candidate group of the prediction mode is selected from among factors such as the position of the neighboring block, the priority of the neighboring block, the priority in the divided block, the size or shape of the neighboring block, a predetermined specific mode, and the prediction mode of the luminance block (in case of a chrominance block). It may be configured according to at least one factor or a combination thereof, and related information may be transmitted in units of sequences, pictures, slices, blocks, and the like.

For example, when the current block and the neighboring block are divided into two or more blocks, it is possible to determine which mode of the divided blocks to be included as a mode prediction candidate of the current block under the same setting of the encoder or the decoder. have. For example, the left block among neighboring blocks of the current block (M×M) consists of three divided blocks by performing quad tree division in the block division unit, and M/2×M/2, M/ When 4×M/4 and M/4×M/4 blocks are included, the prediction mode of the M/2×M/2 block may be included as a mode prediction candidate of the current block based on the block size. As another example, the upper block among the neighboring blocks of the current block (N×N) consists of three divided blocks by performing binary tree division in the block division unit, and N/4×N, N/4× from left to right. When blocks of N and N/2×N are included, the prediction mode of the first N/4×N block from the left is used as the mode prediction candidate of the current block according to a preset order (priorities are allocated from left to right). may include

As another example, when the prediction mode of the block adjacent to the current block is the directional prediction mode, the prediction mode adjacent to the prediction direction of the corresponding mode (the slope information or angle information side of the direction of the mode) is included in the mode prediction candidate group of the current block can do. Also, preset modes (planar, DC, vertical, horizontal, etc.) may be preferentially included according to the configuration or combination of prediction modes of neighboring blocks. Also, a prediction mode having a high frequency of occurrence among prediction modes of a neighboring block may be preferentially included. The priority means not only the possibility of being included in the mode prediction candidate group of the current block, but also the possibility of being assigned a higher priority or index (that is, the probability of receiving fewer bits in the binarization process) in the candidate group configuration. have.

As another example, the maximum number of mode prediction candidates of the current block is k, the left block is composed of m blocks having a length smaller than the vertical length of the current block, and the upper block is composed of n blocks having a length smaller than the horizontal length of the current block. If configured, the candidate group can be filled according to a preset order (left to right, top to bottom) when the sum (m+n) of the divided blocks of the neighboring blocks is greater than k, and the sum of the divided blocks of the neighboring blocks (m+n) ) is greater than the maximum value (k) of the candidate group, in the prediction mode of the neighboring block (left block, upper block), other neighboring blocks (eg, lower left, upper left, upper right, etc.) The prediction mode of the same block may also be included in the mode prediction candidate group of the current block. The above examples are not specific only to the above-described case, and may include a case in which each other's conditions are reversed, and variations in other cases are also possible.

As such, the candidate block for mode prediction of the current block is not limited to a specific block position, and prediction mode information can be utilized from at least one block among blocks located on the left, upper left, lower left, upper, and upper right, As in the above example, the prediction mode candidate group of the current block may be configured in consideration of various factors.

The prediction mode encoder can classify a mode (MPM) candidate group (referred to as candidate group 1 in this example) and a mode candidate group (referred to as candidate group 2 in this example) that are highly probable to be the same as the mode of the current block, and the current block A prediction mode encoding process may vary according to which candidate group among the candidate groups belongs to. The total prediction mode may be composed of the sum of the prediction modes of the candidate group 1 and the prediction modes of the candidate group 2. It may be determined according to one or more factors, such as the shape of the block, or a combination thereof. Depending on the candidate group, the same binarization may be applied or a different binarization may be applied. For example, fixed-length binarization may be applied to candidate group 1 and short cut-type binarization may be applied to candidate group 2. In the above description, the number of candidate groups is two as an example, but the mode 1 candidate group having a high probability of being the same as the mode of the current block, the mode 2 candidate group having a high probability of being the same as the mode of the current block, and the mode candidate group not having a high probability of being the same as the mode of the current block It is extensible, and its modifications are also possible.

In the post-processing filtering step performed by the post-processing filter unit, some prediction pixels among the prediction blocks generated in the previous process in consideration of the characteristic of high correlation between the reference pixel adjacent to the boundary between the current block and the neighboring block and the pixel in the adjacent current block. may be replaced with a value generated by filtering one or more reference pixels and one or more prediction pixels adjacent to the boundary, and a value obtained by quantifying characteristics between reference pixels adjacent to the boundary of the block (eg, a difference in pixel values) , gradient information, etc.) can be replaced with a value generated by applying the filtering process, and other methods having a similar purpose (correcting some prediction pixels of a prediction block through reference pixels) are available in addition to the above method. can be added. In the post-processing filter unit, the type of filter and whether or not to apply the filtering may be determined implicitly or explicitly, and the positions and number of reference pixels and current pixels used in the post-processing filter unit, and the type of prediction mode to be applied, etc. It can be set in the encoder or decoder, and related information can be transmitted in units of sequences, pictures, slices, and the like.

In addition, in the post-processing filtering step, an additional post-processing process may be performed after the prediction block is generated like the block boundary filtering. In addition, post-processing filtering is performed in consideration of the characteristics of pixels of adjacent reference blocks, similar to the boundary filtering above, for the current block restored by adding the residual signal and the prediction signal obtained through the transformation, quantization process and the reverse process after obtaining the residual signal. can also be done

Finally, a prediction block is selected or acquired through the above-described process, and the information from this process may include prediction mode related information, and is transmitted to the transform unit 210 for encoding the residual signal after the prediction block is obtained. can

10 is an exemplary diagram for explaining a prediction principle in a P slice or a B slice in an image encoding method according to an embodiment of the present invention.

Referring to FIG. 10 , the image encoding method according to the present embodiment may include motion estimation module and interpolation steps. Information on the motion vector, reference picture index, and reference direction generated in the motion prediction step may be transferred to the interpolation step. In the motion prediction step and the interpolation step, a value stored in a reconstructed picture buffer (DPB) may be used.

That is, the image encoding apparatus may perform motion estimation to find a block similar to the current block in previously encoded pictures. Also, the image encoding apparatus may perform interpolation of the reference picture for more precise prediction than the precision of a fractional unit. Finally, the image encoding apparatus obtains a prediction block through a predictor, and information from this process includes a motion vector, a reference picture index or a reference index, and a reference direction. ), and thereafter, residual signal encoding may be performed.

In the present embodiment, since intra prediction is also performed in P slice or B slice, a combination method as shown in FIG. 9 supporting inter prediction and intra prediction is possible.

11 is an exemplary diagram for explaining a process of obtaining a prediction block.

Referring to FIG. 11 , the image encoding method according to the present embodiment includes reference sample padding, reference sample filtering, intra prediction, boundary filtering, and motion. It may include steps of motion estimation, and interpolation.

When the video encoding apparatus supports block matching in the current picture, the prediction method in the I slice may be implemented with the configuration shown in FIG. 11 instead of the configuration shown in FIG. 9 . That is, the image encoding apparatus may use information such as a motion vector, a reference picture index, and a reference direction that occur only in a P slice or a B slice as well as a prediction mode in the I slice to generate the prediction block. However, information that can be partially omitted may exist due to the characteristic that the reference picture is current. For example, when the reference picture is the current picture, the reference picture index and the reference direction may be omitted.

In addition, when the image encoding apparatus applies interpolation, since block matching up to decimal units may not be required due to the characteristics of the image, for example, due to the characteristics of an artificial image such as computer graphics, whether to perform this is also set in the encoder. In this regard, it is also possible to set units such as sequences, pictures, and slices.

For example, the image encoding apparatus may not perform interpolation of reference pictures used for inter prediction according to the setting of the encoder, and perform various settings such as not performing interpolation only when block matching is performed in the current picture. can do. That is, the image encoding apparatus of the present embodiment may set whether to perform interpolation of reference pictures. In this case, it may be determined whether interpolation is performed on all reference pictures or some reference pictures constituting the reference picture list. For example, the image encoding apparatus does not perform interpolation when it is not necessary to perform block matching in decimal units because the characteristic of an image in which a reference block exists in a certain current block is an artificial image, and blocks matching in decimal units because it is a natural image. It can be operated to perform interpolation when it is necessary to do

Also, the image encoding apparatus may set whether block matching is applied to a reference picture that has been interpolated in units of blocks. For example, when a natural image and an artificial image are mixed, interpolation is performed on the reference picture, but when the optimal motion vector can be obtained by searching for a part of the artificial image, the A motion vector can be expressed, and when an optimal motion vector can be obtained by selectively searching a part of a natural image, the motion vector can be expressed in another predetermined unit (assuming that it is a quarter unit).

12 is an exemplary diagram for explaining a main process of an image encoding method according to an embodiment of the present invention with syntax in a coding unit.

12, curr_pic_BM_enabled_flag means a flag that allows block matching in the current picture, and can be defined and transmitted in sequence and picture units. In this case, the process of generating a prediction block by performing block matching in the current picture is inter-picture. It may mean a case of operating through prediction. In addition, it may be assumed that cu_skip_flag, which is an inter-picture technique that does not encode a residual signal, is a flag supported only in P slices or B slices except for I slices. In that case, when curr_pic_BM_enabled_flag is turned on, block matching (BM) can be supported in the inter prediction mode even in I slices.

That is, the image encoding apparatus of the present embodiment may support skip when generating a prediction block through block matching in the current picture, and may support skip in the case of intra-picture techniques other than block matching. And depending on conditions, skip may not be supported in the I slice. Whether to skip this may be determined according to an encoder setting.

As an example, when skip is supported in I slice, the prediction block can be directly restored as a reconstructed block through block matching without encoding the residual signal by connecting to prediction_unit(), which is the prediction unit through a specific flag if(cu_skip_flag). . Also, the apparatus for encoding an image may classify a method of using a prediction block through block matching in the current picture as an inter prediction technique, and may process such classification through a specific flag, pred_mode_flag.

That is, the video encoding apparatus may set the prediction mode to the inter prediction mode (MODE_INTER) if pred_mode_flag is 0, and may set the prediction mode to the intra prediction mode (MODE_INTRA) if 1 is 1. This is an intra-picture technique similar to the existing one, but it can be classified as an inter-picture technique or an intra-picture technique in the I slice to distinguish it from the existing structure. That is, the video encoding apparatus of the present embodiment does not use temporal correlation in the I slice, but may use a temporal correlation structure. part_mode means information on the size and shape of a block divided in a coding unit. . For example, the flag supporting block matching in the current picture in the sequence parameter may be sps_curr_pic_BM_enabled_flag, and the flag supporting block matching in the current picture in the picture parameter may be pps_curr_pic_BM_enabled_flag. If sps_curr_pic_BM_enabled_flag is on and pps_curr_pic_BM_enabled_flag is off, block matching may not be supported in the current picture. Whether the current picture is included in the reference picture list 0 configuration may be determined according to the flag, and whether the current picture is included in the reference picture list 1 configuration may be determined.

13 is an exemplary diagram for explaining an example of supporting symmetric type division or asymmetric type division as in inter prediction when a prediction block is generated through block matching in the current picture.

Referring to FIG. 13 , in the video encoding method according to the present embodiment, when a prediction block is generated through block matching in a current picture, symmetric types such as 2N×2N, 2N×N, and N×2N as in inter prediction. (symmetric) partitioning may be supported, or asymmetric partitioning such as nL×2N, nR×2N, 2N×nU, or 2N×nD may be supported. Various block sizes and shapes may be determined according to the division method of the block division unit.

14 is an exemplary diagram for explaining that 2N×2N and N×N can be supported in inter prediction (Inter).

Referring to FIG. 14 , the video encoding method according to the present embodiment may support 2N×2N and N×N like the form of a prediction block used for conventional intra prediction. This is an example in which the block division unit supports a square shape through a quadtree division method or a division method according to a predefined block candidate group, and even in intra prediction, a binary tree division method or a rectangular shape in a predefined block candidate group. Other block types can also be supported by adding a shape, and the setting for this can be set in the encoder. In addition, the encoder determines whether skip is applied only when block matching is performed on the current picture during intra prediction, whether to apply skip to existing intra prediction, or whether to apply skip to other new intra predictions. can be set in Information on this may be transmitted in units of sequences, pictures, slices, and the like.

The subtraction unit 205 (refer to FIG. 2 ) may generate a residual block by subtracting pixel values of the prediction block generated by the prediction unit 200 from the pixel values of the current block to be encoded to derive a pixel difference value.

The transform unit 210 (refer to FIG. 2 ) receives the residual block that is the difference between the current block and the prediction block generated through intra prediction or inter prediction from the subtraction unit 205 and transforms it into the frequency domain. Through the transform process, each pixel of the residual block corresponds to a transform coefficient of the transform block. The size and shape of the transform block may be the same as or smaller than the coding unit. Also, the size and shape of the transform block may be the same as or smaller than the prediction unit. The image encoding apparatus may perform transformation processing by bundling several prediction units.

The size or shape of the transform block may be determined through the block dividing unit, and a square shape or rectangular shape transform may be supported according to the block division. The block division operation may be affected according to the transform related settings supported by the encoder or the decoder (size, shape, etc. of a transform block supported).

The size and shape of each transform block is determined according to the encoding cost for each candidate of the size and shape of the transform block, and segmentation information such as image data of each determined transform block and the determined size and shape of each transform block can be encoded. .

The transform may be transformed by a one-dimensional transform matrix. (DCT, DST, etc.) Each transform matrix can be adaptively used in horizontal and vertical units. Examples of adaptive use may be determined by several factors, such as a size of a block, a block shape, a block type (luminance and color difference), an encoding mode, prediction mode information, a quantization parameter, and encoding information of a neighboring block.

For example, in the case of intra prediction, when the prediction mode is horizontal, a DCT-based transform matrix may be used in a vertical direction and a DST-based transform matrix may be used in a horizontal direction. In the vertical case, a DCT-based transform matrix may be used in a horizontal direction and a DST-based transform matrix may be used in a vertical direction. The transformation matrix is not limited to that from the description above. Information on this may be determined using an implicit or explicit method, depending on one or more factors or a combination of factors such as block size, block shape, encoding mode, prediction mode, quantization parameter, encoding information of a neighboring block, etc. may be determined, and the related information may be transmitted in units of sequences, pictures, slices, blocks, and the like.

Here, considering the case of using an explicit method, when two or more transformation matrices for horizontal and vertical directions are in a candidate group, information on which transformation matrix is used in each direction may be sent, respectively, Alternatively, information on which transformation matrix is used in the horizontal and vertical directions may be transmitted by grouping two or more pairs as a candidate group for each transformation matrix used in the horizontal and vertical directions.

In addition, partial transformation or total transformation may be omitted in consideration of the characteristics of the image. For example, one or both of the horizontal or vertical components may be omitted. This is because, when the difference between the current block and the prediction block is large (that is, when the residual component is large) because intra prediction or inter prediction is not performed well, the resulting encoding loss may increase when transforming it. This may be determined according to at least one or a combination of factors such as encoding mode, prediction mode information, block size, block shape, block type (luminance and chrominance), quantization parameters, and encoding information of neighboring blocks. . According to the above conditions, it can be expressed using an implicit or explicit method, and information on this can be transmitted in units of sequences, pictures, slices, and the like.

The quantization unit 215 (refer to FIG. 2 ) quantizes the residual component transformed by the transform unit 210 . The quantization parameter is determined in units of blocks, and the quantization parameters may be set in units of sequences, pictures, slices, blocks, and the like.

For example, the quantization unit 215 may predict the current quantization parameter using one or more quantization parameters derived from neighboring blocks such as the left, upper left, upper, right upper, lower left, etc. of the current block.

In addition, the quantization unit 215 determines the difference from the basic parameters transmitted in units of sequences, pictures, slices, etc. when there is no quantization parameter predicted from a neighboring block, that is, when a block is at a boundary between pictures, slices, etc. values can be printed or transmitted. When a quantization parameter predicted from a neighboring block exists, a difference value may be transmitted using the quantization parameter of the corresponding block.

The priority of the block from which the quantization parameter is to be derived may be set in advance, or may be transmitted in units of sequences, pictures, slices, and the like. The residual block may be quantized through Dead Zone Uniform Threshold Quantization (DZUTQ), a quantization weighted matrix, or an improved technique thereof. This may include one or more quantization techniques as candidates, and may be determined by encoding mode, prediction mode information, and the like.

For example, the quantization unit 215 may set the quantization weight matrix to be applied to inter-picture coding, intra-picture coding units, etc., and may also provide a different weight matrix according to the intra prediction mode. The quantization weight matrix has a size of M×N, and assuming that the size of the block is the same as the size of the quantization block, the quantization coefficient can be configured by different quantization coefficients for each position of each frequency component. In addition, the quantization unit 215 may select one of several existing quantization methods, or may be used under the same setting of an encoder or a decoder. Information on this can be transmitted in units of sequences, pictures, slices, and the like.

Meanwhile, the inverse quantization units 220 and 315 and the inverse transform units 225 and 320 illustrated in FIGS. 2 and 3 may be implemented by performing the above processes in the transform unit 210 and the quantization unit 215 inversely. That is, the inverse quantization unit 220 may inverse quantize the quantized transform coefficient generated by the quantizer 215, and the inverse transform unit 225 may inversely transform the inverse quantized transform coefficient to generate a reconstructed residual block. have.

The adders 230 and 324 shown in FIGS. 2 and 3 may generate a reconstructed block by adding the pixel value of the prediction block generated from the prediction unit to the pixel value of the reconstructed residual block. The reconstructed block may be stored in the encoding and decoding picture buffers 240 and 335 and provided to the prediction unit and the filter unit.

The filter unit may apply an in-loop filter such as a deblocking filter, a sample adaptive offset (SAO), or an adaptive loop filter (ALF) to the reconstructed block. The deblocking filter may filter the reconstructed block to remove distortion between block boundaries generated during encoding and decoding. SAO is a filter process that restores the difference between the original image and the reconstructed image as an offset for the residual block in units of pixels. The ALF may perform filtering to minimize the difference between the prediction block and the reconstruction block. The ALF may perform filtering based on a comparison value between the block restored through the deblocking filter and the current block.

The entropy encoder 245 (refer to FIG. 2 ) may entropy-encode the transform coefficients quantized through the quantizer 215 . For example, to perform techniques such as context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, etc. can

The entropy encoding unit 245 may include, in the encoded data, a bit string obtained by encoding a quantization coefficient and various information necessary for decoding the encoded bit string. The coded data may include an coded block type, a quantization coefficient, and a bit stream in which the quantized block is coded, and information required for prediction. In the case of a quantization coefficient, a two-dimensional quantization coefficient may be scanned in one dimension. The distribution of the quantization coefficients may vary depending on the characteristics of the image. In particular, in the case of intra prediction, since the distribution of coefficients may have a specific distribution depending on the prediction mode, a different scanning method may be set.

Also, the entropy encoder 245 may be set differently according to the size of a block to be encoded. The scan pattern may be preset or set as a candidate at least one or more of various patterns such as zigzag, diagonal, raster, etc., and may be determined by encoding mode and prediction mode information, etc. can be used Information on this can be transmitted in units of sequences, pictures, slices, and the like.

The size of the quantized block (hereinafter, quantized block) input to the entropy encoder 245 may be equal to or smaller than the size of the transform block. In addition, the quantization block may be divided into two or more sub-blocks, and when divided, the scan pattern in the divided block may be set to be the same as that of the existing quantization block or may be set differently.

For example, if the scan pattern of the existing quantization block is called zigzag, the zigzag may be applied to all subblocks, or the zigzag pattern is applied to the subblock located at the upper left of the block including the DC component, A diagonal pattern can be applied to other blocks. This may also be determined according to encoding mode, prediction mode information, and the like.

In addition, the start position of the scan pattern in the entropy encoder 245 basically starts from the upper left, but may start from the upper right, lower right, or lower left depending on the characteristics of the image, and any one of two or more candidate groups is selected. Information on whether or not it has been performed may be transmitted in units of a sequence, a picture, a slice, or the like. As the encoding technology, an entropy encoding technology may be used, but is not limited thereto.

On the other hand, the inverse quantization of the inverse quantization unit 220 and the inverse transformation of the inverse transform unit 225 shown in FIGS. 2 and 3 reverse the quantization of the quantization unit 215 and the transformation structure of the transform unit 210 . And it can be implemented by combining the basic filter units (235, 330).

On the other hand, in the encoding or decoding method of image data, data-level parallelization divides data to be processed in a parallelized program into several units and then allocates the divided data to different cores or threads to perform the same task in parallel way to do it. Theoretically, parallelism is one of the important factors for performance because image processing speed increases as parallelization increases within the performance limit of a core or thread.

For data units of such parallel processing, parallelization in units of frames, slices, and blocks is frequently used. Hereinafter, it will be described on the premise that the prediction between screens is performed by such parallel processing.

15 is an exemplary diagram of generating a prediction block through block matching using a reference block in a current picture.

Referring to FIG. 15 , the middle four blocks may be referred to as current blocks to be encoded, and an arrow indicates a reference block, indicating a case in which the reference blocks are present in the current picture. The present embodiment may be applied to a case where block matching is performed on the current picture and is encoded through an inter-picture encoding structure.

16 is an exemplary diagram illustrating a reference block of an already encoded region used for block matching in a current picture.

Referring to FIG. 16 , a shaded block indicates a reference block of an coded region. As described above, in such a reference block, blocking deterioration may occur because the in-loop filter is not applied.

As such, in the case of a region that has already been encoded prior to the current block, since the deblocking filter and in-loop filter such as Sample Adaptive Offset (SAO) are not applied yet, when block matching is performed in the current picture, parallel From a processing point of view, problems can arise. As such, for block matching, additional memory may be required before filtering is applied.

In addition, in the case of an already-encoded region, since the in-loop filter is not applied, it may be difficult to generate an optimal prediction block due to deterioration between blocks.

According to the present embodiment, when the memory for the current picture and another block in the current picture are referred to in the decoded picture buffer (DPB), an additional memory may be required. In this case, the additional memory may be another memory for the current picture in addition to the existing memory for the current picture, or a temporary memory that may be used in the encoding or decoding process of the current picture. In addition, in parallel processing, a very large capacity of an additional memory may be required.

In addition, when an in-loop filter such as a deblocking filter and an adaptive sample offset (SAO) is applied, it may be a memory for the current picture, and when the in-loop filter is not applied, it may be an additional memory. . In addition, the additional memory may not be required according to the in-loop filter operation setting in units of a sequence parameter set, a picture parameter set, a slice header, and the like. For example, the additional memory may not be required to set the in-loop filter operation such as deblocking filter off, SAO off, and ALF off. Some or other components of the above components may be included.

As described above, a reference block referenced to another block in the current picture may be stored in an additional memory. This results in the loss of additional memory and there is a need to prevent this. Additional memory protection through adaptive filtering will be described below.

17 is a block diagram of a video encoding apparatus to which a filtering skip check unit is added.

Referring to FIG. 17 , a prediction unit 200 , an adder 205 , a transform unit 210 , a quantization unit 215 , an inverse quantization unit 220 , and an inverse transform unit , 225 ), a subtraction unit 230 , a decoded picture buffer (DPB) 240 , an entropy encoder 245 , a filtering skip check unit 250 , a skip selection circuit 260 , and a filter unit 290 ). may include

Here, the prediction unit 200 may include a first prediction means 201 for intra prediction and a second prediction means 202 for inter prediction. The first prediction unit may be referred to as an intra prediction unit, and the second prediction unit may be referred to as an inter prediction unit. Also, the addition unit 205 and the subtraction unit 230 may be referred to as a first addition/subtraction unit and a second addition/subtraction unit, respectively.

In addition, the filtering skip check unit 250 is located between the second addition/subtract unit 230 and the filter unit 290 , and the skip selection circuit 260 is located between the filtering skip check unit 250 and the filter unit 290 , and It is located between the filtering skip check unit 250 and the decoded picture buffer 240 . The filtering skip check unit 250 may adaptively perform deblocking filtering by controlling the skip selection circuit 260 based on selection information based on the filtering skip flag.

In addition, the filter unit 290 may be referred to as an in-loop filter unit, and may include at least any one or more of a deblocking filter 270 and a sample adaptive offset (SAO, 280). . The filter unit 290 may perform filtering on the reconstructed image.

The deblocking filter 270 will be described in more detail as follows. That is, a quantization error may occur in the quantization step during the previous prediction, transformation, quantization, and entropy encoding processes. This is adjusted by the quantization parameter value. If the quantization parameter value is small, the transform coefficient is densely quantized, so that the quantization error is relatively small. If the quantization parameter value is large, the quantization error may occur relatively large. In order to improve this problem, it is possible to reduce image quality deterioration by performing filtering on the reconstructed picture.

As confirmed in FIG. 17 , by additionally configuring the filtering skip check unit 250 , the filtering skip check unit 250 applies and skips the filter unit 290 based on a flag according to whether or not it is used as a reference block. The skip selection circuit 26 may be controlled to select . Accordingly, parallel processing can be performed using one DPB 240 without using additional memory.

For this purpose, the encoder and the decoder may have the same configuration information, and may be transmitted between the encoder and the decoder in units of sequences, pictures, slices, and the like.

18 is a block diagram of an image decoding apparatus to which a filtering skip check unit is added.

Referring to FIG. 18 , the image decoding apparatus according to the present embodiment includes an entropy encoder, an inverse quantization unit, an inverse transform unit, an intra prediction unit, and an inter prediction unit. It may include an inter prediction, a deblocking filter, a sample adaptive offset (SAO), a reconstructed picture buffer (DPB), and a filtering skip checker.

Here, for the existing configuration, since reference may be made to FIG. 3 described above, a duplicate description is omitted. Referring to FIG. 18 , it can be confirmed that a filtering skip checker is added to the existing image decoding apparatus.

That is, the image decoding apparatus may instruct a switching operation as to whether to skip filtering based on the flag transmitted from the image encoding apparatus. If filtering is skipped, the corresponding data is immediately stored in the reconstructed picture buffer DPB. Otherwise, the corresponding data may be stored in the reconstructed picture buffer DPB after filtering is applied.

19 to 22 are exemplary diagrams for explaining flag transmission in blocks of various sizes in an image encoding method according to an embodiment of the present invention.

That is, FIGS. 19 to 22 describe examples in which a flag indicating whether the in-loop filter is adaptively applied or whether it is used as a reference block of another picture in the current picture may be transmitted in units of various blocks. can do.

First, referring to FIG. 19 , when the maximum coding unit is 64×64, it can be transmitted in the maximum coding unit. The maximum coding unit may correspond to a coding tree unit (CTU).

Referring to FIG. 20 , the maximum coding unit may be transmitted as a divided coding unit. When generating flags in divided units, tree-based division is possible. The present embodiment shows that the largest coding unit can be split into four or at least one of the four coding units can be split into four again.

Referring to FIG. 21 , one or more maximum coding units may be transmitted in a bundle unit. This embodiment shows that four maximum coding units can be transmitted as one bundle unit.

Referring to FIG. 22 , it is possible to combine transmission of one or more largest coding units in a bundle unit and transmission of one or more largest coding units in split coding units obtained by splitting the largest coding units into a plurality of units.

The above-described FIGS. 19 to 22 exemplify quad tree division as the division method, but may be divided into various block sizes and shapes according to the division method in the division unit (see FIG. 2 ).

Information on the above-described maximum block size to minimum block size can be set equally to the encoder and the decoder, and related information can be transmitted in units of sequences, pictures, slices, and the like.

23 is an exemplary diagram for explaining a method of transmitting a flag indicating whether a reference block is used.

Referring to FIG. 23 , a circle denotes a split flag, a square denotes a filtering applied flag, a slash denotes split or filtering is applied, and a colorless denotes that no split or filtering is applied.

Here, if at least one split block is referenced, the corresponding block is split, and when it is no longer split, the on/off (on, off) flag of the in-loop filter may be transmitted. A depth for the division of a block may be determined by the size of a maximum block, a picture type, etc. However, if the supported depth is 0, the division may not be performed. Information on this may be transmitted in units of sequences, pictures, slices, and the like. In addition, here, the maximum block size is 64x64 and the supported depth is 3 (here, up to 8x8).

At this time, since the size of the block is flexible, it is necessary to first transmit whether or not to split the block. If it is 1, the block is split. If it is 0, it is transmitted that the block is no longer divided. , information can be transmitted in the lower-left block order (or jet scan direction).

First, 1 (64 × 64 division) indicating division is transmitted to the 64 × 64 block (top of the tree structure) indicated by CTU_64, and then 1 (32 × 32 division) is transmitted to the upper left block (32 × 32). It transmits a split box by sending, and 0 (16×16 crystals) 0 (16×16 crystals) 0 (16×16 crystals) 0 (16×16 crystals) can be transmitted for the four blocks (16×16) inside it. have. Subsequently, 0 (32×32 crystal) may be transmitted to the upper right block to indicate that no more division is to be made, and 0 (32×32 crystal) may be transmitted to the lower left block. The division is transmitted by transmitting 1 (32 × 32 division) to the lower right block (32 × 32), and 1 (16 × 16 division) is transmitted to the upper left block (16 × 16) inside it to indicate that it is divided. indicates. In the case of the upper-left block (16×16), the size (8×8) of the divided block is the same as the supported depth (8×8) as it is divided, so the division flag is not transmitted to the divided block (8×8). does not After that, 0 (16×16 crystal), 0 (16×16 crystal), and 0 (16×16 crystal) can be transmitted for the remaining blocks (16×16). That is, 1100000011000 may be transmitted as indication data indicating whether to divide.

Through the above process, information indicating whether to divide or not may be generated, and thereafter, information on whether to apply filtering may be generated. Through the above process, 0, 0, 1, 1, 0, 1, 1, 0, 1, 0, 0, 1, 0 may be transmitted in the jet scan direction for flags indicating whether filtering is applied to the 13 blocks.

A bitstream structure of a partition flag for expressing a flag for partition information of the block may be selected from one or more scan methods. As in the above example, it is possible to preferentially obtain additional partitioning information in a block divided based on the initial block based on the jet scan. Alternatively, as a jet scan-based depth-depth order reference method, it is a method of acquiring partition information at a depth depth of a current level on the basis of an initial block and then acquiring partition information at a depth depth of a next level. When using the above method, 1 (64 × 64 division), 1 (32 × 32 division), 0 (32 × 32 division), 0 (32 × 32 division), 1 (32 × 32 division), 0 (16 × 16 crystals), 0 (16×16 crystals), 0 (16×16 crystals), 0 (16×16 crystals), 1 (16×16 divisions), 0 (16×16 crystals), 0 (16×16 crystals) ), 0 (16×16 crystals) can be transmitted. That is, 1100100001000 may be transmitted as indication data indicating whether to divide. In addition to this, other scanning methods not presented in the above example may be included and selected.

In the present embodiment, since the in-loop filter has not yet been applied to data in the current picture that has already been encoded, there is deterioration between blocks. The memory containing such data may be a temporary memory stored only during the encoding or decoding process of the current picture. When referring to data with deterioration through block matching, there may be a disadvantage in that prediction accuracy is lowered due to deterioration between blocks. In this case, block deterioration can be reduced by applying filtering that reduces block deterioration to the left and upper boundaries of the current block after the encoding of the block is completed and before proceeding to the next block.

Also, the right and bottom boundaries can be processed in the block to be coded next. In which block boundary the filtering is to be applied, the filtering application may be determined according to a predetermined condition, for example, a type of a block boundary, a minimum block size, and the like. Filtering may be performed at the transform block boundary, filtering may be performed at the prediction block boundary, or may be performed at a common block boundary between the transform block and the prediction block. The size of a minimum block on which filtering is performed can also be set, and related information may be generated or transmitted in units of sequences, pictures, slices, and the like.

Also, it is possible to set whether or not filtering is applied, a pixel to which filtering is applied, a filter type, and the like, by analyzing characteristics of blocks placed on the block boundary, for example, coding mode, block boundary characteristics, prediction information, coding coefficients, and the like. The related configuration information may be set identically to the encoder and the decoder, and may be transmitted in units of sequences, pictures, slices, and the like. The filtering employed in the present embodiment described above may use the same configuration as the existing deblocking filter, or may use a different configuration.

According to the above-described embodiment, international codecs such as MPEG-2, MPEG-4, H.264, etc. or other codecs using in-picture prediction technology, media using these codecs, and high performance generally available to the video industry It is possible to provide a high-efficiency image encoding/decoding technology.

In addition, in the future, it is expected to be applied to the image processing field using the current high-efficiency video encoding technology (HEVC) and standard codecs such as H.264/AVC and intra prediction.

Although the above has been described with reference to preferred embodiments of the present invention, those skilled in the art can variously modify and change the present invention within the scope without departing from the spirit and scope of the present invention as set forth in the claims below. You will understand that it can be done.

Claims (4)

In the video encoding method for performing inter prediction,
searching for a reference block by performing inter prediction on the current block;
encoding a flag indicating whether to include a current picture including the current block in a reference picture list in units of picture parameter sets according to the searched reference block;
generating a prediction block for the current block by using the searched reference block;
generating a residual block for the current block based on the prediction block;
reconstructing the current block based on the prediction block and the residual block; and
Storing the current picture in a restored picture buffer (Decoded Picture Buffer, DPB),
The reconstructed picture buffer is a first allocated for storage of a current picture to which the in-loop filter is applied, according to whether the current picture is included in the reference picture list and whether an in-loop filter is applied to the current picture. and a second memory allocated for storage of a memory and a current picture to which the in-loop filter is not applied,
A filtering skip flag indicating whether the in-loop filter is applied to the current picture is encoded in the bitstream,
The reference picture list is configured based on whether the current picture is included in the reference picture list and a slice type.
In the image decoding method for performing inter prediction,
decoding a flag indicating whether a current picture including a current block is included in a reference picture list from a picture parameter set;
generating a reference picture list for the current block;
generating a prediction block for the current block by using the reference picture list;
reconstructing the current block based on the prediction block and the residual block; and
Comprising the step of storing the restored current picture in a restored picture buffer (Decoded Picture Buffer, DPB),
The reconstructed picture buffer is a first allocated for storage of a current picture to which the in-loop filter is applied, according to whether the current picture is included in the reference picture list and whether an in-loop filter is applied to the current picture. and a second memory allocated for storage of a memory and a current picture to which the in-loop filter is not applied,
Whether the in-loop filter is applied to the current picture is determined based on a filtering skip flag obtained from a bitstream,
The reference picture list is configured based on whether the current picture is included in the reference picture list and a slice type.
A computer-readable medium for storing a bitstream generated by an image encoding method performed by an image encoding apparatus, comprising:
The method is
searching for a reference block by performing inter prediction on the current block;
encoding a flag indicating whether to include a current picture including the current block in a reference picture list in units of picture parameter sets according to the searched reference block;
generating a prediction block for the current block by using the searched reference block;
generating a residual block for the current block based on the prediction block;
reconstructing the current block based on the prediction block and the residual block; and
Storing the current picture in a restored picture buffer (Decoded Picture Buffer, DPB),
The reconstructed picture buffer is a first allocated for storage of a current picture to which the in-loop filter is applied, according to whether the current picture is included in the reference picture list and whether an in-loop filter is applied to the current picture. and a second memory allocated for storage of a memory and a current picture to which the in-loop filter is not applied,
A filtering skip flag indicating whether the in-loop filter is applied to the current picture is encoded in the bitstream,
The reference picture list is configured based on whether the current picture is included in the reference picture list and a slice type. delete

Images