KR20160146591A - Method and apparatus of image encoding/decoding using adaptive deblocking filtering - Google Patents

Abstract

Disclosed are a method and apparatus for encoding/decoding using adaptive deblocking filtering. The method for encoding an image, which performs adaptive filtering and prediction between screens using a current picture as a reference picture, includes the steps of: searching a reference block by performing the prediction between the screens with regard to a current block to be encoded; generating a flag to indicate whether the searched reference block is located within the current picture in which the current block is located; encoding the current block by using the searched reference block and then decoding the current block; adaptively applying an in-loop filter with regard to the decoded current block by referring to the flag; and storing the current block to which the in-loop filter is applied or is not applied in a decoded picture buffer (DPB). Accordingly, the present invention can improve encoding and decoding efficiency by improving the performance of deblocking filtering.

Description

TECHNICAL FIELD The present invention relates to adaptive deblocking filtering, and more particularly, to a method and apparatus for adaptive deblocking filtering.

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to image encoding and decoding techniques, and more particularly, to an encoding / decoding method and apparatus for adaptive deblocking filtering.

With the spread of the Internet and mobile terminals and the development of information and communication technology, the use of multimedia data is increasing rapidly. Therefore, in order to perform various services or tasks through image prediction in various systems, there is a great need for improving the performance and efficiency of the image processing system, but the research and development result that can respond to the atmosphere is insufficient.

In addition, image encoding is divided into prediction, transformation, quantization, and entropy coding. In the quantization process, blocking artifacts and ringing phenomenon occur in a reconstructed picture due to a quantization error. As a result, There is a problem that the subjective quality of the image is deteriorated.

Therefore, in the conventional image encoding and decoding techniques using quantization, filtering is performed on the reconstructed picture to improve the subjective picture quality. The prior art related to the filtering of the reconstructed picture is largely divided into a post-processing filter method and an in-loop filter method depending on whether a filtered picture is used as a reference picture in the inter-picture prediction mode.

In the post-processing filter method, filtering is performed right before the display output of the restored image outside the video decoder. In the in-loop filtering method, after filtering is applied to the restored picture, the filtered picture buffer is inserted into the decoded picture buffer And is used as a reference picture in the prediction mode.

On the other hand, the in-loop filter method, that is, the deblocking filter method, performs filtering after loading restored pixels stored in the memory when performing filtering, and stores the filtered pixels in the memory again. . In addition, the deblocking filter has a complicated filtering operation itself, and has a disadvantage in that it takes a considerably large complexity of 20 to 30% in the decoder due to the computational complexity and overhead for memory access. Thus, in the image coding and decoding technology, an effective filtering method for the reconstructed picture is required.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a method of decoding / decoding an image by improving deblocking filtering.

It is another object of the present invention to provide an apparatus for encoding / decoding an image by improving deblocking filtering.

According to an aspect of the present invention, there is provided an image encoding method for performing adaptive filtering in inter-picture prediction according to an aspect of the present invention includes searching for a reference block by performing inter-picture prediction on a current block to be coded, Generating a flag indicating whether a block is present in a current picture in which the current block is located, encoding and restoring the current block using the searched reference block, referring to the flag, And storing the current block to which the in-loop filter is applied or not in a reconstructed picture buffer (Decoded Picture Buffer, DPB).

Here, the in-loop filter may include at least one of a deblocking filter, an adaptive sample offset (SAO).

Here, adaptively applying the in-loop filter may not apply the in-loop filter to the restored current block if the flag indicates that there is a reference block in the current picture.

Here, adaptively applying the in-loop filter may apply an in-loop filter to the restored current block if the flag indicates that there is no reference block in the current picture.

According to another aspect of the present invention, there is provided an image encoding method for performing inter-picture prediction, the method comprising the steps of: inputting a video signal including a reference block located in the same picture as a current block, Decoding the reconstructed image signal, storing the reconstructed image signal in a temporary memory separate from the decoded picture buffer (DPB), and reconstructing the reconstructed image signal using a reference block included in the reconstructed image signal, And performing inter-picture prediction through matching.

Here, in the step of storing in the temporary memory, the restored video signal may be stored in a separate temporary memory via an in-loop filter and a separate filter.

Here, the separate filter may perform filtering on the left and upper boundaries of the reference block included in the reconstructed video signal.

Here, the separate filter may perform filtering at least one of a transform block boundary, a predictive block boundary, and a common block boundary between the transform block and the predictive block of the reconstructed image signal.

According to another aspect of the present invention, there is provided an image decoding method for performing adaptive filtering in inter-picture prediction according to another aspect of the present invention, including the steps of: Obtaining a flag indicating whether a reference block is present in the current picture in which the current block is located, restoring the current block using the reference block, restoring the current block based on the obtained flag, Adaptively applying an in-loop filter to the current block, and storing the current block to which the in-loop filter is applied or not, in a reconstructed picture buffer (Decoded Picture Buffer, DPB).

Here, the in-loop filter may include at least one of a deblocking filter, an adaptive sample offset (SAO).

Here, adaptively applying the in-loop filter may not apply the in-loop filter to the restored current block if the flag indicates that there is a reference block in the current picture.

Here, adaptively applying the in-loop filter may apply an in-loop filter to the restored current block if the flag indicates that there is no reference block in the current picture.

According to another aspect of the present invention, there is provided an image decoding method for performing inter-picture prediction according to another aspect of the present invention includes a reference block located in the same picture as a current block from a bit stream and block- A step of reconstructing the reconstructed image signal, a step of storing the restored reconstructed image signal in a temporary memory separate from the reconstructed picture buffer (DPB), and a step of performing block matching on the current block And performing inter-prediction.

Here, in the step of storing in the temporary memory, the restored video signal may be stored in the temporary memory through an in-loop filter and a filter separate from the in-loop filter.

Here, the separate filter may perform filtering on the left and upper boundaries of the reference block included in the reconstructed video signal.

Here, the separate filter may perform filtering at least one of a transform block boundary, a predictive block boundary, and a common block boundary between the transform block and the predictive block of the reconstructed image signal.

According to another aspect of the present invention, there is provided an image decoding apparatus for performing adaptive filtering in inter-picture prediction according to another aspect of the present invention, the one or more processors comprising: Obtaining a flag indicating whether a reference block is included in a current picture in which a current block is located, obtaining a flag indicating whether a reference block is present in the current picture in which the current block is located, A step of adaptively applying an in-loop filter to the restored current block based on the obtained flag, and a step of decoding a current block to which the in-loop filter is applied or not, using a reconstructed picture buffer (Decoded Picture Buffer, DPB) .

Here, the in-loop filter may include at least one of a deblocking filter, an adaptive sample offset (SAO).

Here, adaptively applying the in-loop filter may not apply the in-loop filter to the restored current block if the flag indicates that there is a reference block in the current picture.

Here, adaptively applying the in-loop filter may apply an in-loop filter to the restored current block if the flag indicates that there is no reference block in the current picture.

When the coding method, the decoding method, and the decoding apparatus for performing the adaptive deblocking filtering according to the embodiment of the present invention as described above are used, the coding and decoding efficiency can be improved by improving the performance of deblocking filtering.

Furthermore, by applying an adaptive in-loop filter, additional memory consumption can be prevented.

In addition, prediction efficiency can be improved by applying a filter that is different from the in-loop filter.

1 is a conceptual diagram of an image encoding and decoding system according to an embodiment of the present invention.
2 is a block diagram of an image encoding apparatus according to an embodiment of the present invention.
3 is a configuration diagram of an image decoding apparatus according to an embodiment of the present invention.
FIG. 4 is a diagram illustrating an inter-picture prediction of a P slice in an image encoding and decoding method according to an embodiment of the present invention.
FIG. 5 is a diagram illustrating an inter-picture prediction of a B slice in an image encoding and decoding method according to an embodiment of the present invention.
FIG. 6 is a diagram illustrating an example of generating a prediction block in a unidirectional manner in the image encoding and decoding method according to an embodiment of the present invention. Referring to FIG.
FIG. 7 is a diagram illustrating a reference picture list constructed in a method of encoding and decoding images according to an exemplary embodiment of the present invention. Referring to FIG.
FIG. 8 is a diagram illustrating an example of performing inter-picture prediction from a reference picture list in an image encoding and decoding method according to an embodiment of the present invention.
9 is an exemplary diagram for explaining intra prediction in the image encoding method according to an embodiment of the present invention.
10 is an exemplary diagram for explaining a prediction principle in a P slice or a B slice in the image encoding method according to an embodiment of the present invention.
11 is an exemplary diagram for explaining a process of acquiring a prediction block.
FIG. 12 is an exemplary diagram illustrating a main process of the image encoding method according to an embodiment of the present invention with a syntax in a coding unit.
FIG. 13 is an exemplary diagram for explaining an example of supporting a symmetric type division or an asymmetric type division as in inter picture prediction when a prediction block is generated through block matching in the current picture.
14 is an exemplary diagram for explaining that 2N x 2N and N x N can be supported in inter-picture prediction (Inter).
15 is an exemplary diagram for generating a prediction block through block matching using a reference block in the current picture.
FIG. 16 is an exemplary view for explaining a reference block of an already-coded region used in block matching in the current picture.
17 is a configuration diagram of a video encoding apparatus to which a filtering skip confirmation section is added.
18 is a configuration diagram of an image decoding apparatus to which a filtering skip confirmation section is added.
19 to 22 are diagrams for explaining flag transmission in units of blocks of various sizes in the image encoding method according to the embodiment of the present invention.
23 is an exemplary diagram for explaining a method of transmitting a flag indicating whether a reference block is utilized or not.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It is to be understood, however, that the invention is not to be limited to the specific embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

The terms first, second, A, B, etc. may be used to describe various elements, but the elements should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. And / or < / RTI > includes any combination of a plurality of related listed items or any of a plurality of related listed items.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The terminology used in this application is used only to describe a specific embodiment and is not intended to limit the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In the present application, the terms "comprises" or "having" and the like are used to specify that there is a feature, a number, a step, an operation, an element, a component or a combination thereof described in the specification, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the relevant art and are to be interpreted in an ideal or overly formal sense unless explicitly defined in the present application Do not.

In general, a moving picture can be composed of a series of pictures, and each picture can be divided into a predetermined area such as a block. In addition, the divided area may be referred to as various sizes or terms such as a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), and a transform unit (TU) as well as a block. Each unit may consist of one luminance block and two color difference blocks, which may be otherwise configured according to the color format. Further, the size of the luminance block and the color difference block can be determined according to the color format. For example, in the case of 4: 2: 0, the size of the color difference block may have a length of 1/2 of the luminance block. For this unit, terms such as existing HEVC or H.264 / AVC can be referred to. Although blocks and terms are used interchangeably in the present invention, they may be otherwise understood in accordance with standard techniques and should be understood as corresponding terms or units in accordance with the encoding and decoding processes according to such standard techniques.

A reference picture, a reference block, or a reference pixel is referred to as a reference picture, a block, or a pixel to be referred to in encoding or decoding a current block or a current pixel. It is also to be understood that the term "picture" described below may be used in place of other terms having equivalent meanings such as image, frame, etc., I can understand if I am a child

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In order to facilitate the understanding of the present invention, the same reference numerals are used for the same constituent elements in the drawings and redundant explanations for the same constituent elements are omitted.

1 is a conceptual diagram of an image encoding and decoding system according to an embodiment of the present invention.

1, the image encoding apparatus 12 and the decryption apparatus 11 may be implemented by a personal computer (PC), a notebook computer, a personal digital assistant (PDA), a portable multimedia player (PMP) Player, a PlayStation Portable (PSP), a wireless communication terminal, a smart phone, a TV, a server terminal such as an application server and a service server, A communication device such as a communication modem for performing communication with a wired / wireless communication network, a memory (memory) 18 for storing various programs for inter- or intra-prediction for encoding or decoding an image or for encoding or decoding an image, A processor 14 for computing and controlling the operation of the system 10, and the like. The. The video encoded by the video encoding device 12 can be transmitted in real time or in non-real time through a wired or wireless communication network such as the Internet, a local area wireless communication network, a wireless LAN network, a WiBro network, a mobile communication network, (Universal Serial Bus) or the like to be decoded in an image decoding apparatus and restored and reproduced as an image.

In addition, an image encoded by a video encoding apparatus by a bit stream may be transferred from a coding apparatus to a decoding apparatus via a computer-readable recording medium.

2 is a block diagram of an image encoding apparatus according to an embodiment of the present invention. 3 is a block diagram of an image decoding apparatus according to an embodiment of the present invention.

2, the image encoding apparatus 20 according to the present embodiment includes a predictor 200, a subtractor 205, a transformer 210, a quantizer 215, an inverse quantizer 220, An inverse transform unit 225, an adder 230, a filter unit 235, a decoded picture buffer 240, and an entropy encoding unit 245.

3, the image decoding apparatus 30 according to the present embodiment includes an entropy decoding unit 305, a predicting unit 310, an inverse quantizing unit 315, an inverse transforming unit 320, A decoded picture buffer 325, a filter unit 330, and a decoded picture buffer 335.

The image encoding apparatus 20 and the image decoding apparatus 30 may be separate apparatuses, but may be implemented as one image encoding and decoding apparatus according to an implementation. In this case, the prediction unit 200, the inverse quantization unit 220, the inverse transformation unit 225, the addition unit 230, the filter unit 235, and the memory 240 of the image encoding apparatus 20 are arranged in the order At least the same technical elements as the prediction unit 310, the inverse quantization unit 315, the inverse transformation unit 320, the addition unit 325, the filter unit 330 and the memory 335 of the decoding apparatus 30 Structure or at least perform the same function. In addition, the entropy coding unit 245 may correspond to the entropy decoding unit 305 when performing its function inversely. Therefore, a detailed description of the following technical elements and their operating principles will not be repeated.

Since the image decoding apparatus 30 corresponds to a computing apparatus that applies the image encoding method performed in the image encoding apparatus 20 to decryption, the following description will be made with reference to the image encoding apparatus 20 as an example.

The computing device may include a memory for storing a program or a software module implementing the image encoding method and / or the image decoding method, and a processor connected to the memory for executing the program. The video encoding apparatus may be referred to as an encoder, and the video decoding apparatus may be referred to as a decoder.

Each component of the image encoding apparatus of the present embodiment will be described in more detail as follows.

Here, the image encoding apparatus 20 may further include a division unit. The dividing unit divides the input image into blocks (MxN) of a predetermined size. Here, M or N is an arbitrary natural number of 1 or more. Specifically, the dividing section may be composed of a picture dividing section and a block dividing section. The size or shape of the block can be determined according to the characteristics and resolution of the image, and the size or shape of the block supported through the picture partitioning unit is M × N square shape in which the length and the length are represented by an exponent of 2 (256 X 256, 128 x 128, 64 x 64, 32 x 32, 16 x 16, 8 x 8, 4 x 4, etc.), or an M x N rectangular shape. For example, an input image can be divided into 256 × 256 for an 8 k UHD image having a high resolution, 128 × 128 for a 1080p HD image, and 16 × 16 for a WVGA image.

Information on the size or shape of such a block can be set in units of a sequence, picture, slice, or the like, and related information can be transmitted to the decoder. That is, a sequence parameter set, a picture parameter set, a slice header, or a combination thereof.

Here, a sequence indicates a constituent unit formed by collecting a number of related scenes. A picture is a term that refers to a whole series of luminance (Y) components or luminance + chrominance (Y, Cb, Cr) components in one scene or picture. The range of one picture is one frame or one field .

A slice can refer to one independent slice segment and a number of dependent slice segments that are in the same access unit. An access unit means a set of network abstraction layer (NAL) units associated with a single coded picture. The NAL unit is a syntax structure composed of a video compression bitstream in a network-friendly format in the H.264 / AVC and HEVC standards. It is common to configure one slice unit as one NAL unit, and system standards generally regard a set of NALs or NALs constituting one frame as one access unit.

Returning to the description of the picture division section, the information on the block size or type (M × N) can be made up of explicit flags, specifically block type information, one length information when the block is square, The length information, or the difference value information between the horizontal and vertical lengths. For example, if M and N are composed of exponential powers of k (assuming k is 2) (M = 2 ^m , N = 2 ⁿ ), information on m and n can be unary binarization, Or the like, and transmit the related information to the decoding apparatus. Alternatively, if the division allowable minimum size (Minblksize) supported by the picture division unit is IxJ (assuming I = J for convenience of explanation, I = 2 ⁱ , J = 2 ^j ), information about mi or nj . As another example, when M and N are different, it is possible to convey the difference value (| mn |) between m and n. Alternatively, information about im or nj may be conveyed if the division allowable maximum size (Maxblksize) supported by the picture division unit is IxJ (assuming I = J for convenience of explanation, I = 2 ⁱ , J = 2 ^j ) .

In case of an implied situation, for example, if a syntax for related information exists but can not be confirmed by an encoder or a decoder, the encoder or decoder may follow preset default settings. For example, if the related syntax can not be confirmed at the step of confirming the block type information, the block type may be set to a square shape, which is the default setting. Alternatively, the step of checking the block size information may be confirmed in the step of checking the block size information through the difference value from the minimum allowable minimum size (Minblksize) as in the above example. However, (Minblksize) related syntax can be obtained from the preliminarily set default values for the Minblksize.

As described above, the size or shape of the block in the picture division unit can be implicitly determined by explicitly transmitting the related information in the encoder or the decoder or according to the characteristics and resolution of the image.

As described above, a block divided and determined through the picture division unit can be used as a basic encoding unit. A block divided and determined by the picture division unit may be a minimum unit constituting a high level unit such as a picture, a slice, and a tile, and may be a unit such as a coding block, a prediction block, a transform block A quantization block, an entropy block, and an inloopfiltering block. However, some blocks are not limited to this and some exceptions are possible.

For example, some such as an in-loop filtering block may be applied in units larger than the block sizes described above.

The block division unit performs division on blocks such as coding, prediction, conversion, quantization, entropy, and in-loop filter. The divider may be included in each configuration to perform its functions. For example, the transforming unit 210 may include a transform block dividing unit and the quantizing unit 215 may include a quantization block dividing unit. The size or shape of the initial block of the block partition may be determined by the result of division of the previous level or higher level block. For example, in the case of a coded block, a block obtained through a picture division unit which is a previous stage can be set as an initial block. Alternatively, in the case of a prediction block, a block obtained through a division process of a coding block which is a higher level of the prediction block may be set as an initial block. Alternatively, in the case of a transform block, a block obtained through the division process of an encoding block, which is a higher level of the transform block, may be set as an initial block. The condition for determining the size or type of the initial block is not always fixed, and there may be a case where a part is changed or an exception is made. In addition, it is also possible to set the setting level of the current level (for example, the size of the supported conversion block, the type of the conversion block, etc.) of the previous level or the higher level block (for example, the size of the encoding block, ) May affect the division operation of the current level (division possibility, divisible block type, etc.) according to a combination of at least one or more factors.

The block partition may support a quad tree based partitioning scheme. That is, the block may be divided into four blocks each having a length of 1/2 in the horizontal and vertical directions in the block before division. This means that the block size limit (dep >> k, n >> k) means the allowable division depth limit (dep >> k, k >>) while the block size limit (dep_k) Can be repeatedly performed.

In addition, a binary tree-based partitioning scheme can be supported. This indicates that the length of one of the horizontal and vertical lengths can be divided into two blocks having a length of 1/2 as compared with the pre-division block. In the case of quad tree partitioning and binary tree partitioning, symmetric partitioning or asymmetric partitioning may be used, and it may be determined according to a partitioning scheme according to the setting of an encoder or a decoder. In the image encoding method of the present invention, a symmetric division method will be mainly described.

The division flag (div_flag) indicates whether or not each block is divided. If the corresponding value is 1, the division is performed. If the value is 0, the division is not performed. Alternatively, if the value is 1, the segmentation is performed and further segmentation is possible. If the value is 0, the segmentation is not performed, and the segmentation is not allowed any more. Depending on the conditions such as the minimum allowable division size and the limit of the allowable division depth, the flag may be considered only for the division, and the additional division may not be considered.

The split flag can be used in a quadtree partition, and also in a binary tree partition. In binary tree segmentation, the dividing direction may be one of the block depth, encoding mode, prediction mode, size, type, and type (encoding, prediction, And may be determined based on at least one of factors such as a slice type, a division allowable depth limit, a division allowable minimum and maximum size, or a combination thereof.

In this case, the divided flag may be divided according to the dividing direction, that is, divided into a width of only 1/2 of the block or a length of 1/2. For example, suppose that a block supports M × N (M> N) and M is greater than N, and that the current partition depth (dep_curr) is smaller than the partitioning allowable depth limit, The division flag is allocated with 1 bit. If the corresponding value is 1, the horizontal division is performed. If the division value is 0, the division flag can be no longer divided. The split depth can be one divide depth for quad tree and binary tree divisions, and one divide depth for quad tree and binary tree divisions. In addition, the allowable depth limit may have one partition allowable depth limit for the quadtree and the binary tree partition, or a respective partition allowable depth limit for the quadtree and binary tree partition.

 As another example, if the block is M × N (M> N) and N is equal to the predetermined minimum allowable division size, and the horizontal division is not supported, the division flag is allocated to 1 bit. If the value is 1, If it is 0, no division is performed.

Also, flags (div_h_flag, div_v_flag) for horizontal division or vertical division can be respectively supported, and binary division can be supported according to the flags. It is possible to indicate whether each block is horizontally or vertically divided through the horizontal division flag (div_h_flag) or the vertical division flag (div_v_flag). If the horizontal division flag (div_h_flag) or the vertical division flag (div_v_flag) If 0, no horizontal or vertical division is performed. Or, if each flag is 1, horizontal or vertical division is performed and horizontal or vertical division is possible. If the value is 0, horizontal division or vertical division is not performed and further division of horizontal or vertical division may not be allowed . The flag may be considered for the division depending on the conditions such as the division allowable minimum size and the division allowable depth limit, and the flag may not be considered. Alternatively, a flag (div_flag / h_v_flag) for horizontal division or vertical division may be supported, and binary division may be supported according to the flag. The division flag div_flag may indicate whether it is horizontally or vertically divided, and the division direction flag h_v_flag may indicate a horizontal or vertical division direction. If the division flag (div_flag) is 1, the division is performed and the horizontal or vertical division is performed according to the division direction flag (h_v_flag). If 0, the horizontal or vertical division is not performed. If the value is 1, horizontal or vertical division is performed according to the division direction flag (h_v_flag), and horizontal or vertical division is possible. If the value is 0, horizontal or vertical division is not performed. It can be regarded as not permitting. The flag may be considered for the division depending on the conditions such as the division allowable minimum size and the division allowable depth limit, and the flag may not be considered.

These division flags can also support for horizontal and vertical division, respectively, and can support binary tree division according to the flags. When the dividing direction is predetermined, only one of the two division flags may be used, or both of the two division flags may be used as in the above example.

For example, if the above flags are allowed, the possible block types can be divided into any one of M × N, M / 2 × N, M × N / 2, and M / 2 × N / 2. In this case, the flags can be encoded as 00, 01, 10, and 11 (in order of div_h_flag / div_v_flag). In the above case, it is an example of a setting in which the division flag can be used in a superimposed manner, and a setting in which the division flag can not be used overlapping is also possible. For example, the split block type may be divided into M × N, M / 2 × N, and M × N / 2, where the above flags are encoded as 00, 01, 10 (in order of div_h_flag / div_v_flag) , (in the order of div_flag / h_v_flag, h_v_flag can be encoded as 0, 10, and 11 flags indicating the horizontal or vertical direction of division). Here, the meaning of the overlap may mean that the horizontal division and the vertical division are performed at the same time. The quad tree division and the binary tree division described above can be used singly or in combination according to the setting of the encoder or the decoder. For example, a quadtree or binary tree partition may be determined depending on the block size or type. That is, when the block type is M x N and M is larger than N, the horizontal division, the block type is M x N, and if N is larger than M, the binary tree division can be supported according to the vertical division, Is M x N, and quad tree partitioning is supported if N and M are the same.

In another example, binary tree partitioning may be supported if the size of the block (MxM) is greater than or equal to the block partitioning boundary value (Thrblksize), and quad tree partitioning may be supported if the size is smaller.

As another example, if M or N of the block (MxN) is equal to or smaller than the first allowable maximum size (Maxblksize1) and equal to or greater than the first allowable minimum size (Minblksize1), quad tree partitioning is supported, A binary tree segmentation may be supported if M or N of (M x N) is less than or equal to the second allowable maximum size Maxblksize2 and greater than or equal to the second allowable minimum size Minblksize2. If the first division supporting range and the second division supporting range, which can be defined by the division allowable maximum size and the division allowable minimum size, overlap, the priority of the first or second division method Rank can be given. In this example, quad tree partitioning is used for the first partitioning method, and binary tree partitioning is used for the second partitioning method. For example, when the first division allowable minimum size (Minblksize1) is 16, the second division allowable maximum size (Maxblksize2) is 64, and the pre-division block is 64x64, the first division support range and the second division support range Quad tree partitioning and binary tree partitioning are possible because it belongs to all. If divide flag (div_flag) is 1, quad tree partitioning is performed and additional quadtree partitioning is possible, if the priority is given by the first partitioning method (quad tree partitioning in this example) according to the setting, Can be regarded as not performing a quadtree partition and no longer performing a quadtree partition. Or, if the divide flag (div_flag) is 1, quad tree partitioning is performed and additional quadtree or binary tree partitioning is possible. If 0, no quadtree partitioning is performed and no quad tree partitioning is performed. It can be regarded as a tree divisible.

Depending on the conditions such as the minimum allowable division size and the limit of the allowable division depth, the flag may be considered only for the division, and the additional division may not be considered. If the division flag div_flag is 1, it is divided into 4 blocks having a size of 32x32 and is larger than the first division allowable minimum size (Minblksize1), so that the quadtree partitioning can be continued. If it is 0, no additional quadtree partitioning is performed, and the current block size (64x64) belongs to the second partitioning support range, so that the binary tree partitioning can be performed. When the division flag (in the order of div_flag / h_v_flag) is 0, no further division is performed, and in the case of 10 or 11, a horizontal division or a vertical division can be performed. If the size of the current block (32x32) is smaller than the size of the current block (32x32) if the block before division is 32x32 and the division flag (div_flag) is 0, Since it is not in the scope of 2-partition support, it may not support further partitioning. In the above description, the priority of the division method may be determined according to at least one of a slice type, a coding mode, a luminance and chrominance components, or a combination thereof.

As another example, various settings can be supported depending on the luminance and chrominance components. For example, a quadtree or binary tree division structure determined from the luminance component can be used as it is without further information or decoding in the chrominance component. Alternatively, if independent division of the luminance component and the chrominance component is supported, quad tree + binary tree may be included in the luminance component, and quad tree division may be supported in the chrominance component. Alternatively, the quad tree + binary tree division is supported in the luminance and chrominance components, but the division support range may or may not be the same or proportional to the luminance and chrominance components. For example, when the color format is 4: 2: 0, the division support range of the chrominance component may be N / 2 of the division support range of the luminance component.

As another example, different settings can be made depending on the slice type. For example, I slices can support quadtree partitioning, P slices can support binary tree partitioning, and B slices can support quadtree + binary tree partitioning.

Quad tree partitioning and binary tree partitioning can be set and supported according to various conditions as in the above example. It should be noted that the above-described examples are not limited to the above-described cases and may include cases where the conditions of each other are reversed, and may include one or more factors mentioned in the above examples or combinations thereof. Do. The above allowable division depth limit can be determined according to at least one factor or a combination of at least one of division method (quad tree, binary tree), slice type, luminance and chrominance components, encoding mode and the like. The division support range may be determined according to at least one factor or a combination thereof in a division mode (quad tree, binary tree), a slice type, a luminance and color difference component, a coding mode, Maximum value, and minimum value. When information on this is configured with an explicit flag, length information of each of the maximum value and the minimum value, or difference value information between the minimum value and the maximum value can be expressed. For example, when the maximum value and the minimum value are composed of exponentiation powers of k (assuming k is 2), the exponential information of the maximum and minimum values can be encoded through various binarization and transmitted to the decoding apparatus. Alternatively, the difference value between the maximum value and the minimum value can be transmitted. The information transmitted at this time may be the difference information of the exponent information and the exponent of the minimum value

In accordance with the above description, information related to flags can be generated and transmitted in units of sequences, pictures, slices, tiles, blocks, and the like.

The block division information may be represented by a quad tree, a binary tree, or a combination of two tree schemes with the division flags shown in the above examples. The division flags may be encoded by various methods such as unary binarization and cutting type unary binarization, Device. The bitstream structure of the division flag for representing the division information of the block may be selected from one or more scanning methods. For example, a bitstream of division flags can be configured based on the division depth order (from dep0 to dep_k), and a bitstream of division flags can be configured based on division. In the division depth order reference method, division information at the current level is acquired on the basis of the first block, and division information is acquired at the next level. In the division method, It means a method of preferentially acquiring additional division information, and other scanning methods not shown in the above example may be included and selected.

Also, according to the implementation, the block dividing unit may generate index information for a block candidate group of a predetermined type, which is not the division flag described above, and express it. M × N, M × N, M × N / 2, M / 4 × N, 3M / 4 × N, and M × N are the types of block candidates, M × N / 4, M × 3N / 4, M / 2 × N / 2, and the like. When the candidate group of the divided block is determined as described above, the index information for the divided block type can be encoded by various methods such as fixed length binarization, truncated binarization, truncated binarization, and the like. As with the division flag described above, at least one factor or a combination of factors such as a block depth, a coding mode, a prediction mode, a size, a type, a type and a slice type, a division allowable depth limit, The divided block candidate group can be determined. (M × N, M / 2 × N, M × N / 2, and M / 2 × N / 2) to the candidate lists 2, (M × N, M / 2 × N, M × N / 2, M / 4 × N, 3M / 4 × N, M × N / 4, M × 3N / 4, and M / 2 × N / 2) are assumed to be candidate list4. For example, when describing M × N as a reference, a divided block candidate of candidate list 2 can be supported when (M = N), and a divided block candidate of candidate list 3 if (M ≠ N).

As another example, if M × N of M or N is greater than or equal to the boundary value blk_th, the divided block candidate of the candidate list 2 can be supported, and if it is smaller than that, the divided block candidate of the candidate list 4 can be supported. Alternatively, when the M or N is equal to or greater than the first threshold value blk_th_1, if the divided block candidate of the candidate list 1 is smaller than the first threshold value blk_th_1 but equal to or greater than the second threshold value blk_th_2, If the divided block candidate of list2 is smaller than the second boundary value blk_th_2, the divided block candidate of the candidate list 4 can be supported.

As another example, it is possible to support a divided block candidate of the candidate list 2 when the coding mode is intra-picture prediction and a divided block candidate of the candidate list 4 when inter prediction is performed.

Even if the divided block candidates are supported, the bit configuration according to the binarization in each block may be the same or different. For example, if the divided block candidates are limited according to the block size or type as in the case of the division flag, the bit configuration according to the binarization of the corresponding block candidate may be changed. M × N, M × N / 2 and M / 2 × N / 2 in the case of (M> N) The binary bits of the index according to the M × N / 2 of the current condition and the M × N / 2 of the current condition may be different from each other. Information on the segmentation and type of the block can be expressed using one of the segmentation flag or the segmentation index method according to the type of the block used for coding, prediction, conversion, quantization, entropy, in-loop filtering, and the like. In addition, block size limitation and division allowable depth limit for the partitioning and block type support may be different depending on each block type.

In the encoding or decoding process on a block-by-block basis, after the encoding block is determined, the encoding or decoding can be performed according to a process such as a prediction block determination, a transform block determination, a quantization block determination, an entropy block determination, and an in-loop filter determination. The order of the encoding or decoding process is not always fixed, and some of the sequences may be changed or excluded. The size and shape of each block are determined according to the coding cost of each candidate of the size and the shape of the block, and coding related information such as image data of each determined block and size and type of each determined block can be encoded.

The prediction unit 200 can be implemented using a prediction module, which is a software module, and can generate a prediction block using an intra-picture prediction scheme or an inter-picture prediction scheme for a block to be encoded. Here, the prediction block is a block which is understood to closely match the block to be encoded in terms of pixel difference, and can be determined by various methods including a Sum of Absolute Difference (SAD) and a Sum of Square Difference (SSD). At this time, various syntaxes that can be used in decoding the image blocks may be generated. The prediction block can be classified into an in-picture block and an inter-picture block according to an encoding mode.

Intra prediction refers to a prediction technique that uses spatial correlation and predicts a current block using reference pixels of previously reconstructed and decoded blocks in the current picture. That is, the intra-picture prediction can use the reconstructed brightness value as prediction and restoration as reference pixels in the encoder and the decoder. Intra prediction can be effective for flat areas with continuity and areas with constant directionality, and it can be used for the purpose of guaranteeing random access and preventing error diffusion because it uses spatial correlation.

Inter prediction uses a compression technique that removes redundancy of data by using temporal correlation with reference to an image encoded in one or more past or future pictures. That is, the inter picture prediction can generate a prediction signal having high similarity by referring to one or more past or future pictures. In the encoder using inter-picture prediction, a block having a high degree of correlation with a block to be currently coded is searched in the reference picture, and the position information and residue signal of the selected block can be transmitted to the decoder. A reconstructed image can be constructed by generating the same prediction block as the encoder and compensating the transmitted residual signal.

FIG. 4 is a diagram illustrating an inter-picture prediction of a P slice in an image encoding and decoding method according to an embodiment of the present invention. FIG. 5 is a diagram illustrating an inter-picture prediction of a B slice in an image encoding and decoding method according to an embodiment of the present invention.

In the image coding method of this embodiment, the inter picture prediction generates a prediction block from a previously coded picture having a high temporal correlation, so that the coding efficiency can be increased. Current (t) may refer to a current picture to be coded and may be a temporal flow of a picture or a first temporal distance t (t) prior to the POC of the current picture on the basis of a picture order count 2 having a first temporal distance t-1 and a second temporal distance t-2 preceding the first temporal distance by a first reference picture t-1.

4, inter-picture prediction that can be employed in the video coding method of this embodiment is the same as that of the current block of the current picture (current (t)) and that of the reference pictures (t-1, t-2) It is possible to perform motion estimation for finding an optimal prediction block from reference pictures (t-1, t-2) that have been previously encoded with a block having a high correlation through block matching of reference blocks. An interpolation process is performed based on a structure in which at least one or more sub-pixels are arranged between two adjacent pixels as necessary for accurate estimation, and after an optimal prediction block is found, motion compensation is performed, Can be found.

5, inter-picture prediction that can be used in the image coding method of the present embodiment is based on the already-coded reference pictures (" current (t) " t-1, t + 1). In addition, two prediction blocks can be generated from one or more reference pictures.

When encoding an image through inter-picture prediction, motion vector information for the optimal prediction block and information about the reference picture are encoded. In this embodiment, when a prediction block is generated in a unidirectional or bidirectional direction, a reference block list may be configured differently to generate a prediction block from the corresponding reference picture list. Basically, the reference pictures temporally present before the current picture are allocated to the list 0, and the reference pictures existing after the current picture are allocated to the list 1 and can be managed. The reference pictures existing after the current picture can be allocated when the reference picture list 0 can not be filled up to the reference picture list number 0 of the reference picture list 0. Similarly, when the reference picture list 1 is constructed, if the reference picture list 1 can not satisfy the reference picture list, the reference pictures existing before the current picture can be allocated.

FIG. 6 is a diagram illustrating an example of generating a prediction block in a unidirectional manner in the image encoding and decoding method according to an embodiment of the present invention. Referring to FIG.

Referring to FIG. 6, in the image encoding and decoding method according to the present embodiment, a prediction block can be found from a previously encoded reference picture t-1, t-2. In addition, a current picture t) from the region where coding has already been completed.

That is, in the image encoding and decoding method according to the present embodiment, not only a prediction block is generated from a previously coded picture (t-1, t-2) having a temporally high correlation, but also a prediction block having a high spatial correlation As shown in FIG. Finding such a spatially highly correlated prediction block can correspond to finding a prediction block in a manner of intra prediction. In order to perform block matching from an area where coding is completed in the current picture, the image coding method of the present embodiment can construct a syntax for information related to the prediction candidate in combination with the intra-picture prediction mode.

For example, when n (n is an arbitrary natural number) branch prediction mode is supported, one mode is added to the intra prediction prediction candidate to support n + 1 modes and ^{2M-1? N} + 1 ≪ 2 < ^M > can be used to encode the prediction mode. In addition, it can be implemented to select among candidates of possible prediction modes such as MPM (most probable mode) of HEVC (high efficiency video coding). It is also possible to preferentially encode at a higher stage of prediction mode encoding.

When a prediction block is generated through block matching in the current picture, the image coding method of the present embodiment may form a syntax for information related to mixing with the inter-picture prediction mode. Additional associated prediction mode information may be motion or displacement related information. The motion or motion-related information may include optimal candidate information among various vector candidates, a difference value between an optimal candidate vector and an actual vector, a reference direction, reference picture information, and the like.

FIG. 7 is a diagram illustrating a reference picture list constructed in a method of encoding and decoding images according to an exemplary embodiment of the present invention. Referring to FIG. FIG. 8 is a diagram illustrating an example of performing inter-picture prediction from a reference picture list in an image encoding and decoding method according to an embodiment of the present invention.

7, an image encoding method according to an embodiment of the present invention can perform inter-picture prediction from a reference picture list (reference list 0) to a current block of a current picture current (t) have.

Referring to FIGS. 7 and 8, the reference picture list 0 may be composed of reference pictures before the current picture t, where t-1 and t-2 are the picture order counts of the current picture t The first temporal distance t-1 and the second temporal distance t-2, which are earlier than the reference picture P0, Count, POC. The reference picture list 1 can be composed of a reference picture after the current picture t and t + 1 and t + 2 respectively represent a first temporal distance t + 1 after the POC of the current picture t ) And a second temporal distance (t + 2).

Although the above-described examples of the reference picture list construction show an example in which the reference picture list is composed of the reference pictures having the difference of the temporal distance (POC standard in this example), the difference in temporal distance between the reference pictures is configured differently You may. That is, the index difference of the reference pictures and the temporal distance difference of the reference pictures may not be proportional. In addition, the list arrangement order may not be constituted based on temporal distance. This can be confirmed in an example of the reference picture list construction to be described later.

The prediction can be performed from the reference picture in the list according to the slice type (I, P, or B). When a prediction block is generated by block matching in the current picture (current (t)), a current picture is added to a reference list (reference list 0), and coding is performed using an inter picture prediction scheme .

The current picture t can be added to the reference list 0 or the current picture t can be added to the reference list 1 as shown in FIG. That is, the reference picture list 0 can be constructed by adding a reference picture which is a temporal distance (t) to the reference picture before the current picture (t), and the reference picture list 1 is composed of time A reference picture that is a distance t may be added.

For example, when the reference picture list 0 is constructed, the reference picture prior to the current picture can be assigned to the reference picture list 0, and the current picture t can be allocated. When the reference picture list 1 is constructed, The reference picture can be assigned to the reference picture list 1 and then the current picture t can be allocated. Alternatively, when the reference picture list 0 is constructed, the current picture t can be allocated and the reference picture prior to the current picture can be allocated. When the reference picture list 1 is constructed, the current picture t is allocated, It is possible to assign subsequent reference pictures. Alternatively, when configuring the reference picture list 0, the reference picture prior to the current picture can be allocated, the reference picture after the current picture can be allocated, and the current picture t can be allocated. Similarly, when constructing the reference picture list 1, reference pictures after the current picture can be allocated, reference pictures before the current picture can be allocated, and the current picture t can be allocated. The above examples are not specific to the above-mentioned case, but may include cases where the conditions of each other are reversed, and examples of other cases may also be modified.

Whether or not to include the current picture in each reference picture list (for example, not adding to any list, adding to list 0 only, or adding to list 1 or adding to lists 0 and 1) Information can be transmitted in units of sequences, pictures, slices, and the like. The information on this can be encoded by a method such as fixed-length binarization, truncation-type binarization, truncation-type binarization, and the like.

The image encoding and decoding method of this embodiment differs from the method of FIG. 7 in that block matching is performed on the current picture t to select a prediction block, and a reference picture list including related information about the prediction block is constructed , There is a difference in that such a reference picture list is used for image coding and decoding.

In the reference picture list configuration, it is possible to set the order and the rule of each list and the allowable number of the reference pictures of the respective lists differently. This includes whether the current picture is included in the list (whether the current picture is included as a reference picture in inter- , A slice type, a list reconstruction parameter (which may be applied to lists 0 and 1 respectively, and may be applied to lists 0 and 1 respectively), a position in a group of pictures (GOP) And the like, or a combination of these factors, and can explicitly transmit related information in units of a sequence, a picture, or the like. For example, in the case of a P slice, the reference picture list 0 can follow the list construction rule A, regardless of whether the current picture is included in the list, and the reference picture list 0 that includes the current picture in the list in the case of B slice, Rule B and the reference picture list 1 can follow the list construction rule C, and the list construction rule D for the reference picture list 0 that does not include the current picture and the list construction rule E for the reference picture list 1, B and D, C and E may be the same. The list configuration rules may be configured in the same or modified manner as described in the above-described reference picture list configuration example. As another example, when the current picture is included in the list, the first reference picture allowable number can be set, and when not included, the second reference picture allowable number can be set. The first reference picture allowable number and the second reference picture allowable number may be equal to or different from each other and the difference between the first reference picture allowable number and the second reference picture allowable number may be set to 1 by default. As another example, if the current picture is included in the list and the list reconstruction parameter is applied, all the reference pictures in the slice A can be the list reconstruction candidates, and the slice B can include only some reference pictures in the list reconstruction candidate group. At this time, the slice A or B can be classified into whether or not the current picture is included in a list, temporal layer information, slice type, position in a group of pictures (GOP), and the like. A picture order count (POC) or a reference picture index, a reference prediction direction (before and after the current picture), whether or not the current picture is present, and the like.

According to the above-described structure, the reference block encoded with the inter-picture prediction can be used in the current picture, so that inter-picture prediction can be allowed or utilized even in the moving prediction of the I-slice.

Further, when constructing the reference picture list, the index allocation or the list construction order may be different according to the slice type. In the case of the I-slice, a lower index (for example, idx = 0, 1, 2) is used in the current picture (current (t)) with a higher priority as in the reference picture list structure example, It is possible to reduce the bit amount in the image encoding through the binarization (fixed length binarization, chopping-type binarization, truncation-type binarization, etc.) with the maximum allowable reference picture number C of the reference picture list. In the case of the P or B slice, when it is judged that the block matching is performed in the current picture and it is judged that it is lower than the probability of selecting the prediction candidate through the reference picture having a different probability of selecting the reference picture of the current block as the prediction candidate, The binarization of various methods of setting the priority for the block matching of the reference picture list to the maximum value using a higher index (for example, idx = C, C-1) The amount of bits in the image encoding can be reduced. In the above example, the priority setting of the current picture can be configured in the same or modified manner as that described in the reference picture list configuration example. In addition, it is possible to omit the information on the reference picture by not constructing the reference picture list according to the slice type (for example, I slice). For example, it is possible to generate a prediction block through the existing inter-view prediction, but to display the inter-view prediction information with the remainder excluding the reference picture information in the motion information in the inter-picture prediction mode.

The method of performing block matching in the current picture can determine whether or not to support according to the slice type. For example, block matching in the current block is supported by I slices, but not P slices or B slices, or other variations are possible. In addition, the method of supporting block matching in the current picture may determine whether to support in units of pictures, slices, tiles, etc., or may be determined based on positions in a group of pictures (GOP), temporal layer information have. Such setting information may be transmitted in units of a sequence, a picture, a slice, or the like in an image encoding process or a decoder in an encoder. Further, even when the above setting information or syntax exists in the upper level unit and the setting related operation is turned on, when the same setting information or syntax exists in the lower level unit, the setting information in the lower level unit is changed to the upper level The setting information in the unit can be prioritized. For example, if the same or similar setting information is processed in a sequence, a picture, or a slice, a picture unit may have priority rather than a sequence unit, and a slice unit may have priority rather than a picture unit. . For example, the flag that supports block matching in the current picture in the sequence parameter may be sps_curr_pic_BM_enabled_flag, and the flag that supports block matching in the current picture in the picture parameter may be pps_curr_pic_BM_enabled_flag. If sps_curr_pic_BM_enabled_flag is on and pps_curr_pic_BM_enabled_flag is off, block matching may not be supported in the current picture. It is possible to determine whether or not the current picture is included in the reference picture list 0 configuration according to the flag and determine whether the current picture is included in the reference picture list 1 configuration.

9 is an exemplary diagram for explaining intra prediction in the image encoding method according to an embodiment of the present invention.

Referring to FIG. 9, the intra-frame prediction method according to the present embodiment includes a reference sample padding, a reference sample filtering, an intra predicting, and a boundary filtering. May comprise a series of steps.

The reference pixel filling step may be an example of a reference pixel forming step, the reference pixel filtering step may be referred to as a reference pixel filter section, the intra prediction may include a prediction block generating step and a prediction mode coding step, May be an example for one embodiment of the post-processing filter step.

That is, the intra-picture prediction performed in the image encoding apparatus of the present embodiment may include a reference pixel configuration step, a reference pixel filtering step, a prediction block generation step, a prediction mode encoding step, and a post-processing filtering step. Depending on various environmental factors such as block size, block type, block location, prediction mode, prediction method, quantization parameter, etc., one or a part of the above processes may be omitted and another process may be added, It can be changed in a different order than the order.

The reference pixel construction step, the reference pixel filtering step, the prediction block generation step, the prediction mode encoding step, and the post-processing filtering step may be implemented by a processor connected to a memory by executing software modules stored in the memory. Therefore, in the following description, for convenience of explanation, a functional unit generated by a combination of a software module implementing each step and a processor executing the function, or a constituent unit performing the function of the functional unit is referred to as a reference pixel unit, The prediction block generation unit, the prediction mode encoding unit, and the post-processing filter unit will be referred to as the execution subject of each step.

More specifically, the reference pixel forming unit forms a reference pixel to be used for predicting the current block through reference pixel filling. If the reference pixel does not exist or is unavailable, the reference pixel filling can be used for the reference pixel by a method such as copying the value from the nearest usable pixel. A decoded picture buffer (DPB) may be used for copying the value.

That is, the intra prediction uses the reference pixels of the blocks that have been encoded before the current picture to perform the prediction. To this end, adjacent pixels such as left, top, bottom left, top, and top right blocks of the current block are mainly used as reference pixels in the reference pixel configuration step. A candidate block of a neighboring block for the reference pixel is an example of a case where a coding sequence of a block is followed by a raster scan or a z-scan, and a scan such as an inverse z-scan In case of using the coding order scanning method, adjacent pixels such as right, lower right, and lower blocks in addition to the above blocks can be used as reference pixels.

Further, according to the implementation, additional pixels other than immediately adjacent pixels may be used as a substitute or mixed with an existing reference pixel according to the stepwise configuration of the intra prediction.

In addition, in the case of predicting a mode having a directionality among the modes of intra-picture prediction, it is possible to generate a reference pixel in decimal units through linear interpolation of reference pixels in integer units. The mode for performing the prediction through the reference pixel existing in the integer unit position includes some modes having vertical, horizontal, 45 degrees, and 135 degrees, and the process of generating reference pixels in decimal units for the above prediction modes is required I can not. In the prediction modes having other directionality except for the prediction mode, the interpolation reference pixels are interpolated at 1/2, 1/4, 1/8, 1/16, 1/32, 1/64, May have precision, and may have a precision of a multiple of 1/2. This is because the interpolation accuracy can be determined according to the number of supported prediction modes or the prediction direction of the prediction mode. A fixed interpolation precision may always be supported in a picture, a slice, a tile, a block, etc., and an adaptive interpolation precision may be supported depending on a block size, a block type, a prediction direction of a supported mode, At this time, the prediction direction of the mode can be expressed by the inclination information or the angle information of the direction indicated by the mode on a specific line reference (for example, x + axis of the positive + on the coordinate plane).

 As an interpolation method, linear interpolation can be performed through adjacent integer pixels, but other interpolation methods can be supported. For interpolation, one or more filter types and the number of taps can be supported. For example, a 6-tap Wiener filter, an 8-tap Kalman filter, and so on can be determined. have. Further, the related information may be transmitted in units of a sequence, picture, slice, block, or the like.

The reference pixel filter unit can filter the reference pixels for the purpose of improving the prediction efficiency by reducing the deterioration in the encoding process after the reference pixels are constructed. The reference pixel filter unit can implicitly or explicitly determine the type of the filter and the application of the filtering according to the size, type, and prediction mode of the block. That is, even if the filter is the same tap, the filter coefficient can be determined differently depending on the filter type. For example, you can use a 3-tap filter such as [1,2,1] / 4, [1,6,1] / 8.

In addition, the reference pixel filter unit can determine whether to apply filtering by determining whether to send additional bits or not. For example, in an implied case, the reference pixel filter unit can determine whether to apply filtering according to the characteristics (dispersion, standard deviation, and the like) of the pixels in the peripheral reference block.

In addition, the reference pixel filter unit may determine whether or not the filtering is applied when the related flag satisfies a predetermined hiding condition such as a residual coefficient, an intra prediction mode, and the like. The number of taps of the filter is, for example, 3-tap such as [1,2,1] / 4 in a small block (blk), [2,3,6,3,2] / 16 in a large block The same 5-tap may be set, and the number of times of application may be determined by whether to perform filtering, whether to perform filtering once, or to perform filtering twice.

Also, the reference pixel filter unit can basically apply filtering to the nearest reference pixel of the current block. Additional reference pixels besides the nearest reference pixel may also be considered in the filtering process. For example, filtering may be applied to additional reference pixels by replacing the nearest reference pixel, or filtering may be applied by mixing additional reference pixels with the nearest reference pixel.

The filtering may be applied either fixedly or adaptively, depending on the size of the current block or the size of the neighboring block, the coding mode of the current block or the neighboring block, the block boundary property of the current block and the neighboring block (for example, A prediction mode or direction of a current block or a neighboring block, a prediction method of a current block or a neighboring block, a quantization parameter, and the like, or a combination thereof . The decision may be made on the same setting (implicit) in the encoder or decoder, and in the encoding cost (explicit). Basically, the filter applied is a low pass filter. Depending on various factors described above, the number of filter taps, the filter coefficient, whether or not the filter flag is encoded, and the number of filter application can be determined. Slice, block, or the like, and can transmit the related information to the decoder.

The prediction block generation unit may perform an extrapolation or an extrapolation scheme through a reference pixel in intra-picture prediction, an interpolation scheme such as an average value (DC) or a planar mode of a reference pixel, a copy of a reference pixel ) Prediction block can be generated. In the case of copying a reference pixel, one reference pixel may be copied to generate one or more prediction pixels, one or more reference pixels may be copied to generate one or more prediction pixels, and the number of copied reference pixels may be copied May be equal to or less than the number of prediction pixels.

In addition, according to the prediction method, it is possible to classify into a directional prediction method and a non-directional prediction method. Specifically, the directional prediction method can be classified into a linear directional method and a curved directional method. The linear direction method uses extrapolation or extrapolation but the pixels of the prediction block are generated through the reference pixels lying on the prediction direction line. The curved directional method uses an extrapolation or an extrapolation method, but the pixels of the prediction block are referred to as a reference Means a scheme in which a partial prediction direction of a pixel is allowed to be changed in consideration of a detailed directionality of a block (for example, edge <Edge>). In the case of the directional prediction mode in the image encoding and decoding method of the present invention, a linear directional method will be mainly described. In the case of the directional prediction method, the intervals between adjacent prediction modes may be equal or unequal, and this may be determined according to the size or type of the block. For example, when a block with the size and shape of M × N is acquired by the block division, if the M and N are the same, the intervals between the prediction modes may be uniform. If M and N are different The intervals between the prediction modes may be unequal. As another example, when M is greater than N, modes with vertical orientation may allocate finer intervals between prediction modes close to the vertical mode (90 degrees), and wider spacings may be allocated to far modes in the vertical mode. When N is greater than M, modes with horizontal orientation may allocate finer intervals between prediction modes close to the horizontal mode (180 degrees), and wide spacing may be allocated to horizontal modes with far prediction mode. The above examples are not specific to the above-mentioned case, but may include cases where the conditions of each other are reversed, and examples of other cases may also be modified. In this case, the interval between the prediction modes can be calculated based on numerical values indicating the directionality of each mode, and the directionality of the prediction mode can be numerically expressed as directional inclination information or angle information.

In addition, a prediction block including other methods using spatial correlation other than the above method can be generated. For example, a reference block using an inter picture prediction method such as motion search and compensation using a current picture as a reference picture can be generated as a prediction block. The prediction block generating step may generate the prediction block using the reference pixel according to the prediction method. That is, it is possible to generate a prediction block through directional prediction or non-directional prediction such as extrapolation, interpolation, copying, and averaging of the existing intra-picture prediction method according to the prediction method, And other additional methods may be used.

The intra-picture prediction scheme may be supported under the same settings of an encoder or a decoder, and may be determined according to a slice type, a block size, a block type, and the like. The intra prediction scheme can be supported according to at least one of the above-mentioned prediction schemes or a combination thereof. The intra prediction mode can be configured according to the supported prediction mode. The number of intra prediction modes supported can be determined according to the prediction method, slice type, block size, block type, and the like. The related information can be set and transmitted in units of a sequence, a picture, a slice, and a block.

The prediction mode encoding step performed by the prediction mode encoding can determine the mode in which the encoding cost according to each prediction mode is optimal in terms of encoding cost as the prediction mode of the current block.

For example, the prediction mode encoding unit may use one or more neighboring block modes for current block mode prediction for the purpose of reducing prediction mode bits. (MPM) candidates that are likely to be the same as the current block mode, and the modes of neighboring blocks may be included in the above candidate group. For example, the prediction mode of the block such as the left, top left, bottom left, top, right top of the current block can be included in the above candidate group.

The candidate group of the prediction mode includes the position of the neighboring block, the priority of the neighboring block, the priority in the divided block, the size or type of the neighboring block, the preset specific mode, and the prediction mode of the luminance block And may be configured according to at least one or more factors or combinations thereof, and the related information may be transmitted in units of a sequence, a picture, a slice, a block, or the like.

For example, when a block adjacent to the current block is divided into two or more blocks, it is possible to decide which block mode of the divided blocks is included as a mode prediction candidate of the current block under the same setting of the encoder or the decoder have. For example, the left block among the neighboring blocks of the current block (M × M) is divided into three blocks by performing a quadtree partition in the block division block, and M / 2 × M / 2 and M / 4 × M / 4 and M / 4 × M / 4, the prediction mode of the M / 2 × M / 2 block can be included as the mode prediction candidate of the current block on the basis of the block size. As another example, the upper block among the neighboring blocks of the current block (N × N) is divided into three blocks by performing the binary tree division in the block division block, and N / 4 × N and N / 4 × N, and N / 2 × N blocks, the prediction mode of the first N / 4 × N block on the left according to a predetermined order (priority assigned from left to right) is set as a mode prediction candidate of the current block .

As another example, if the prediction mode of the block adjacent to the current block is the directional prediction mode, the prediction mode adjacent to the prediction direction of the current mode (inclination information of the direction of the mode or angle information side) can do. In addition, the predetermined modes (planar, DC, vertical, horizontal, etc.) may be preferentially included according to the prediction mode configuration or combination of neighboring blocks. In addition, a prediction mode having a high occurrence frequency among the prediction modes of neighboring blocks can be preferentially included. The priority may be a possibility to be allocated not only to the possibility of being included in the mode prediction candidate group of the current block but also a higher priority or index in the candidate group configuration (that is, a higher probability of allocating fewer bits in the binarization process) have.

As another example, the maximum number of mode prediction candidates of the current block is k, the left block is composed of m blocks whose length is smaller than the length of the current block, and the upper block is n blocks whose length is shorter than the width of the current block (M + n) of neighboring blocks can be filled in according to a predetermined order (left to right, top to bottom) when the divided block sum (m + n) of neighboring blocks is larger than k, (E.g., lower left, upper left, upper right, etc.) other than the neighboring block position in the prediction mode of the neighboring block (left block, upper block) The prediction mode of the same block may be included in the mode prediction candidate group of the current block. The above examples are not specific to the above-mentioned case, but may include cases where the conditions of each other are reversed, and examples of other cases may also be modified.

As described above, the candidate block for predicting the mode of the current block is not limited to a specific block position, and prediction mode information can be utilized from at least one block among left, upper left, lower left, upper and upper right blocks, The prediction mode candidate group of the current block can be configured in consideration of various factors as in the above example.

(MPM) candidate group (referred to as candidate group 1 in this example) and a mode candidate group (referred to as candidate group 2 in this example) having a high probability of being the same as the mode of the current block of the prediction mode encoding unit, The prediction mode encoding process may be changed depending on which candidate group among the candidate groups belongs. The total prediction mode may be composed of the sum of the prediction mode of the candidate group 1 and the prediction mode of the candidate group 2. The number of prediction modes of the candidate group 1 and the prediction group of the candidate group 2 may be the sum of the total number of prediction modes, The shape of the block, and the like, or a combination thereof. The same binarization may be applied or another binarization may be applied according to the candidate group. For example, fixed length binarization may be applied to candidate group 1, and truncation type binarization may be applied to candidate group 2. Although the number of candidate groups is exemplified in the above description, it is assumed that the number of the candidate groups is two, but the number of the candidate groups is not limited to the first candidate group having a high probability of being the same as the current block, the second candidate group having a high probability of being the same as that of the current block, It is expandable, and its variants are also possible.

The post-processing filtering step performed by the post-processing filter unit may include filtering a part of the prediction blocks generated in the previous process in consideration of characteristics having high correlation between the reference block adjacent to the boundary between the current block and the neighboring block, May be replaced with a value generated by filtering one or more reference pixels adjacent to the boundary and one or more of the prediction pixels and a value obtained by digitizing the characteristics of reference pixels adjacent to the boundary of the block (for example, , Slope information, etc.) may be applied to the filtering process to replace the predictive pixel with a generated value, and a similar method (correction of some predictive pixels of the predictive block through a reference pixel) other than the above method Can be added. In the post-processing filter unit, the kind of the filter and whether or not the filtering is applied can be implicitly or explicitly determined. The reference pixel used in the post-processing filter unit, the position and the number of the current pixel, And can be set in an encoder or a decoder, and related information can be transmitted in units of a sequence, a picture, a slice, or the like.

In the post-processing filtering step, an additional post-processing process may be performed after the prediction block is generated, such as the boundary filtering. In addition, after reconstructing the residual signal obtained by acquiring the residual signal and adding the residual signal and the prediction signal obtained through the inverse process of the quantization process and the inverse process, the reconstructed current block is subjected to post-processing filtering in consideration of the characteristics of the neighboring reference block, .

Finally, the prediction block is selected or obtained through the above-described process. Information obtained in this process may include prediction mode related information, and after the prediction block is acquired, it is transmitted to the conversion unit 210 for encoding the residual signal .

10 is an exemplary diagram for explaining a prediction principle in a P slice or a B slice in the image encoding method according to an embodiment of the present invention.

Referring to FIG. 10, the image encoding method according to the present embodiment may include motion estimation module and interpolation steps. Information on the motion vector, the reference picture index, and the reference direction generated in the motion prediction step may be transmitted to the interpolation step. In the motion prediction step and the interpolation step, a value stored in the decoded picture buffer (DPB) can be used.

That is, the image encoding apparatus can perform motion estimation to find a block similar to the current block in the previous encoded pictures. In addition, the image encoding apparatus can perform interpolation of a reference picture for precise prediction with respect to precision in decimal units. Finally, the image coding apparatus acquires a prediction block through a predictor. The information generated in this process includes a motion vector, a reference picture index, a reference direction, ), And the residual signal coding can be proceeded.

In this embodiment, the intra-picture prediction is performed on the P-slice or the B-slice, so that it is possible to implement a combination scheme as shown in FIG. 9 that supports inter-picture prediction and intra-picture prediction.

11 is an exemplary diagram for explaining a process of acquiring a prediction block.

Referring to FIG. 11, an image encoding method according to an exemplary embodiment of the present invention includes reference sample padding, reference sample filtering, intra prediction, boundary filtering, Prediction, motion estimation, and interpolation.

When the image encoding apparatus supports block matching in the current picture, the prediction method in the I-slice can be implemented by the configuration shown in FIG. 11 instead of the configuration shown in FIG. That is, the image encoding apparatus can use information such as a motion vector, a reference picture index, and a reference direction generated in only P-slice or B-slice as well as a prediction mode in I-slice for generating a prediction block. However, information that can be partially omitted may exist due to the characteristic that the reference picture is current. For example, when the reference picture is the current picture, the reference picture index and the reference direction can be omitted.

In addition, when interpolation is applied, the image encoding apparatus may not be required to perform block matching up to a decimal unit due to the characteristics of an artificial image such as a computer graphic due to characteristics of an image. And a unit such as a sequence, a picture, and a slice can be set.

For example, the image encoding apparatus may not perform interpolation of reference pictures used for inter-view prediction according to the setting of the encoder, and may perform various settings such as not interpolating only when block matching is performed in the current picture can do. That is, the image encoding apparatus of the present embodiment can set whether interpolation of reference pictures is performed or not. At this time, it is possible to determine whether to perform interpolation on all the reference pictures or some reference pictures constituting the reference picture list. For example, the image encoding apparatus does not perform interpolation when it is unnecessary to perform block matching on a fractional unit because the characteristic of an image in which a reference block exists in an existing block is an artificial image, Lt; RTI ID = 0.0 &gt; interpolation. &Lt; / RTI &gt;

In addition, the image encoding apparatus can set whether or not block matching is applied in a reference picture interpolated on a block-by-block basis. For example, when a natural image and an artificial image are mixed, interpolation is performed on a reference picture, but when an optimal motion vector can be obtained by searching a part of an artificial image, a predetermined unit (here assumed to be an integer unit) A motion vector can be represented. In addition, when an optimal motion vector can be obtained by selectively searching a portion of a natural image, a motion vector can be expressed by another constant unit (here, 1/4 unit is assumed).

FIG. 12 is an exemplary diagram illustrating a main process of the image encoding method according to an embodiment of the present invention with a syntax in a coding unit.

Referring to FIG. 12, curr_pic_BM_enabled_flag indicates a flag that allows block matching in the current picture, and can be defined and transmitted in units of a sequence and a picture. At this time, a process of generating a prediction block by performing block matching in the current picture It can mean the case of operating through prediction. It can be assumed that cu_skip_flag, which is an inter-screen technique that does not code the residual signal, is a flag that is supported only by the P slice or the B slice excluding the I slice. In this case, when curr_pic_BM_enabled_flag is turned on, block slicing (BM) can be supported in the inter-picture prediction mode also in the I-slice.

That is, the image encoding apparatus of the present embodiment can support skipping when generating a prediction block through block matching on the current picture, and can support skipping in the case of intra-picture description other than block matching. It may not support skipping on I slices depending on the condition. This skipping may be determined according to the encoder setting.

For example, when the I-slice supports skipping, the predicted block may be directly connected to prediction_unit () through a specific flag if (cu_skip_flag) to restore the prediction block to the restoring block through block matching without coding the residual signal . In addition, the image encoding apparatus classifies the method of using the prediction block through block matching in the current picture into the inter-picture prediction technique, and can process the classification through pred_mode_flag, which is a specific flag.

That is, if the pred_mode_flag is 0, the image encoding apparatus sets the prediction mode to the inter-picture prediction mode (MODE_INTER), and if the pred_mode_flag is 1, it can set the intra prediction mode (MODE_INTRA). This is an in-screen technology similar to the existing one, but it can be classified as an inter-screen technique or an in-screen technique in an I-slice to distinguish it from an existing structure. In other words, the image encoding apparatus of the present embodiment can use a temporal correlation structure without using temporal correlation in the I-slice. part_mode indicates information on the size and type of the block divided in the encoding unit. . For example, the flag that supports block matching in the current picture in the sequence parameter may be sps_curr_pic_BM_enabled_flag, and the flag that supports block matching in the current picture in the picture parameter may be pps_curr_pic_BM_enabled_flag. If sps_curr_pic_BM_enabled_flag is on and pps_curr_pic_BM_enabled_flag is off, block matching may not be supported in the current picture. It is possible to determine whether or not the current picture is included in the reference picture list 0 configuration according to the flag and determine whether the current picture is included in the reference picture list 1 configuration.

FIG. 13 is an exemplary diagram for explaining an example of supporting a symmetric type division or an asymmetric type division as in inter picture prediction when a prediction block is generated through block matching in the current picture.

Referring to FIG. 13, in the case of generating a prediction block through block matching in the current picture, the image encoding method according to the present embodiment generates a predictive block such as 2N × 2N, 2N × N, symmetric partitioning, or support asymmetric partitioning such as nL x 2N, nR x 2N, 2N x nU, and 2N x nD. It is possible to determine various block sizes and types according to the division method of the block dividing unit.

14 is an exemplary diagram for explaining that 2N x 2N and N x N can be supported in inter-picture prediction (Inter).

Referring to FIG. 14, the image encoding method according to the present embodiment can support 2N × 2N and N × N as well as a prediction block form used for intra-picture prediction. This is an example in which a square shape is supported through a quadtree division method in a block division unit or a division method according to a predefined block candidate group. In the case of intra-picture prediction, a binary tree division method or a predefined block Other types of blocks can be added by adding a type, and the setting for this can be set in the encoder. It is also possible to determine whether skip is applied only when block matching is performed with respect to the current picture in the intra prediction, whether skipping is applied to the existing intra prediction or skipping is applied to the other intra intra prediction, . Information on this can be transmitted in units of a sequence, a picture, a slice, and the like.

The subtraction unit 205 (see FIG. 2) can generate the residual block by subtracting the pixel values of the prediction block generated from the prediction unit 200 from the pixel values of the current block to be encoded, thereby deriving the pixel difference value.

The transforming unit 210 (see FIG. 2) receives the residual block, which is a difference between the current block and a prediction block generated through intra-picture prediction or inter-picture prediction, in the subtracting unit 205 and converts the residual block into a frequency domain. Through the conversion process, each pixel of the residual block corresponds to the transform coefficient of the transform block. The size and shape of the transform block may be the same or smaller than the encoding unit. In addition, the size and shape of the conversion block may be the same as or smaller than the prediction unit. The image encoding apparatus can perform conversion processing by grouping various prediction units.

The size or type of the transform block can be determined through a block divider and can support square or rectangular transform according to the block division. The block dividing operation can be influenced by the conversion related setting (size, type, etc. of the supported conversion block) supported by the encoder or the decoder.

The size and the shape of each transform block are determined according to the cost of coding the candidates of the size and the shape of the transform block and the division information such as the determined image data of each transform block and the determined size and type of each transform block can be encoded .

The transform can be transformed by a one-dimensional transform matrix. (DCT, DST, etc.) Each transformation matrix can be adaptively used in horizontal and vertical units. Examples of the adaptive use may be determined by various factors such as the size of the block, the type of the block, the type of the block (luminance and chrominance), the encoding mode, the prediction mode information, the quantization parameter, and the encoding information of the neighboring block.

For example, in the case of intra prediction, if the prediction mode is horizontal, a DCT-based transformation matrix may be used in the vertical direction and a DST-based transformation matrix may be used in the horizontal direction. In the case of vertical, a DCT-based transformation matrix may be used in the horizontal direction and a DST-based transformation matrix may be used in the vertical direction. The transformation matrix is not limited to that shown in the above description. The information on this may be determined using an implicit or explicit method and may be determined according to one or more factors or a combination of factors such as the size of the block, the type of the block, the encoding mode, the prediction mode, the quantization parameter, And the related information can be transmitted in units of a sequence, a picture, a slice, a block, or the like.

Here, considering the case of using an explicit method, when two or more transformation matrices for horizontal and vertical directions are used as candidates, information on which transformation matrix is used for each direction may be respectively sent, Or a conversion matrix used for the horizontal and vertical directions, respectively, so that two or more pairs are set as candidates, and information on which conversion matrix is used in the horizontal and vertical directions may be transmitted.

In addition, the partial conversion or the entire conversion can be omitted in consideration of the characteristics of the image. For example, one or both of the horizontal or vertical components may be omitted. Intra-picture prediction or inter-picture prediction are not performed well, so that a coding loss due to the conversion when the difference between the current block and the prediction block is large (that is, when the residual component is large) may become large. This may be determined according to at least one of factors such as an encoding mode, prediction mode information, a size of a block, a type of a block, a type of a block (luminance and chrominance), a quantization parameter, encoding information of a neighboring block, . This can be expressed using an implicit or explicit method according to the above conditions, and information on this can be transmitted in units of sequence, picture, slice, and the like.

The quantization unit 215 (see FIG. 2) performs quantization of the residual component transformed by the transform unit 210. The quantization parameter is determined on a block-by-block basis, and the quantization parameter can be set in units of a sequence, a picture, a slice, and a block.

For example, the quantization unit 215 may predict the current quantization parameters using one or more quantization parameters derived from neighboring blocks such as left, top, top, right top, bottom bottom, etc. of the current block.

If the quantization parameter predicted from the neighboring block does not exist, that is, if the block is located at the boundary of a picture, a slice, or the like, the quantization unit 215 calculates a difference with respect to a basic parameter transmitted in units of a sequence, a picture, Value can be output or transmitted. When the quantization parameter predicted from the neighboring block exists, the differential value may be transmitted using the quantization parameter of the block.

The priority of the block from which the quantization parameter is to be derived may be set in advance or may be transmitted in units of a sequence, a picture, a slice, or the like. The residual block can be quantized through a dead zone uniform threshold quantization (DZUTQ), a quantization weighted matrix, or an improved technique. It can be set to one or more quantization schemes as candidates and can be determined by encoding mode, prediction mode information, and the like.

For example, the quantization unit 215 may set the quantization weight matrix to be applied to inter picture coding, intra picture coding unit, or the like, or may set another weighting matrix according to the intra picture prediction mode. Assuming that the size of the block is equal to the size of the quantization block and the size of the block is M × N, the quantization weight matrix can be configured by varying the quantization coefficient for each frequency component position. The quantization unit 215 may be one of various existing quantization methods and may be used under the same setting of an encoder or a decoder. Information on this can be transmitted in units of a sequence, a picture, a slice, and the like.

The inverse quantization units 220 and 315 and the inverse transform units 225 and 320 shown in FIGS. 2 and 3 may be implemented by reversing the processes in the transform unit 210 and the quantization unit 215 described above. That is, the inverse quantization unit 220 can dequantize the quantized transform coefficients generated in the quantization unit 215, and the inverse transform unit 225 can inversely transform the inversely quantized transform coefficients to generate a reconstructed residual block have.

The adders 230 and 324 shown in FIGS. 2 and 3 may generate a restored block by adding the pixel values of the restored residual block to the pixel values of the predicted block generated from the predictor. The reconstruction block may be stored in the encoding and decoding picture buffers 240 and 335 and provided to the prediction unit and the filter unit.

An in-loop filter, such as a deblocking filter, an adaptive sample offset (SAO), an adaptive loop filter (ALF), etc., may be applied to the restoration block. The deblocking filter may filter the restoration block to remove distortion between block boundaries that occurs during the encoding and decoding processes. SAO is a filtering process that restores the difference between the original image and the reconstructed image by an offset, on a pixel-by-pixel basis, with respect to the residual block. The ALF can perform filtering to minimize the difference between the prediction block and the restoration block. The ALF can perform filtering based on the comparison value between the restored block and the current block through the deblocking filter.

The entropy encoding unit 245 (see FIG. 2) can entropy encode the quantized transform coefficients through the quantization unit 215. For example, techniques such as context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax based context adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) .

The entropy encoding unit 245 may include various types of information necessary for decoding the quantized coefficient bit stream and the encoded bit stream in the encoded data. The encoded data may include a coded block type, a quantization coefficient, a bit string in which the quantization block is coded, and information necessary for prediction. In the case of the quantization coefficient, the two-dimensional quantization coefficient can be scanned in one dimension. The quantization factor may vary depending on the characteristics of the image. In particular, in the case of the intra prediction, since the distribution of coefficients can have a specific distribution according to the prediction mode, the scanning method can be set differently.

In addition, the entropy encoding unit 245 can be set differently according to the size of a block to be encoded. The scan pattern can be preset or candidate as at least one of various patterns such as zigzag, diagonal, and raster, and can be determined by the encoding mode, the prediction mode information, and the like, Can be used. Information on this can be transmitted in units of a sequence, a picture, a slice, and the like.

The size of a quantized block (hereinafter, a quantization block) input to the entropy encoding unit 245 may be equal to or smaller than the size of the transform block. In addition, the quantization block may be divided into two or more subblocks. In the case of division, the scan pattern in the divided block may be set to be the same as or different from that of the existing quantization block.

For example, when a scan pattern of an existing quantization block is a zigzag, zigzag may be applied to all subblocks, or a zigzag pattern may be applied to a subblock located at the upper left of a block including a DC component, A diagonal pattern can be applied to other blocks. May also be determined according to the encoding mode, the prediction mode information, and the like.

The start position of the scan pattern in the entropy encoding unit 245 basically starts from the upper left corner, but may start from the upper right corner, the lower right corner, or the lower left corner depending on the characteristics of the image. Information can be transmitted in units of a sequence, a picture, a slice, or the like. As the encoding technique, an entropy encoding technique may be used, but is not limited thereto.

The inverse quantization and inverse transformation of the inverse quantization unit 225 of the inverse quantization unit 220 shown in FIGS. 2 and 3 is performed by reversing the transforming structure of the quantization unit 215 and the transformation unit 210 And the basic filter units 235 and 330 are combined.

On the other hand, in the method of encoding or decoding video data, data-level parallelization divides data to be processed by a parallelizing program into a plurality of units and then assigns the divided data to different cores or threads to perform the same operations in parallel It is a way to do it. Theoretically, parallelism is one of the important factors for performance because the image processing speed becomes faster as parallelism is made within the performance limits of core or thread.

Parallelization of frame, slice, and block units is often used as a data unit of such parallel processing. The following description will be made on the assumption that inter-picture prediction is performed by such parallel processing.

15 is an exemplary diagram for generating a prediction block through block matching using a reference block in the current picture.

Referring to FIG. 15, the middle four blocks may be referred to as current blocks to be encoded, and arrows indicate reference blocks, indicating that reference blocks exist in the current picture. The present embodiment can be applied to a case where block matching is performed in the current picture, and coding is performed through the inter picture coding structure.

FIG. 16 is an exemplary view for explaining a reference block of an already-coded region used in block matching in the current picture.

Referring to FIG. 16, a hatched block indicates a reference block of a coded area. As described above, such a reference block may not be subjected to an in-loop filter, resulting in blocking deterioration.

Since the in-loop filter, such as the deblocking filter and the adaptive sample offset (SAO), is applied before the current block is coded before the current block, if block matching is performed on the current picture, Problems can arise from a processing point of view. Thereby, for block matching, additional memory may be required before filtering is applied.

Also, in the case of the already coded region, since the in-loop filter is applied, it may be difficult to generate an optimal prediction block due to deterioration between blocks.

According to the present embodiment, additional memory may be required when referring to a memory for the current picture and another block in the current picture in the decoded picture buffer DPB. In this case, the additional memory may be another memory for the current picture in addition to the existing memory for the current picture, and may be temporary memory that can be used in the encoding or decoding of the current picture. In addition, a very large capacity of additional memory may be required for parallel processing.

In addition, if an in-loop filter such as a deblocking filter and an adaptive sample offset (SAO) is applied, it may be a memory for the current picture and may be an additional memory if the in-loop filter is not applied . Further, the additional memory may not be required in accordance with the in-loop filter operation setting in units of a sequence parameter set, a picture parameter set, a slice header, and the like. For example, the additional memory may not be needed for an in-loop filter operation setting such as a deblocking filter off, a SAO off, and an ALF off, Some or all of the above configurations may be included and configured.

As described above, reference blocks referenced to other blocks in the current picture can be stored in additional memory. This leads to additional memory loss and needs to be avoided. Hereinafter, additional memory prevention through adaptive filtering will be described.

17 is a configuration diagram of a video encoding apparatus to which a filtering skip confirmation section is added.

17, a prediction unit 200, an adding unit 205, a transform unit 210, a quantization unit 215, an inverse quantization unit 220, an inverse transform unit 220, A subtractor 230, a decoded picture buffer (DPB) 240, an entropy encoder 245, a filtering skip confirmation unit 250, a skip selection circuit 260, and a filter unit 290, . &Lt; / RTI &gt;

Here, the prediction unit 200 may include a first prediction unit 201 for intra prediction and a second prediction unit 202 for inter prediction. The first prediction means may be referred to as an intra prediction portion, and the second prediction means may be referred to as an inter prediction portion. The addition section 205 and the subtraction section 230 may be referred to as a first addition / subtraction section and a second addition / subtraction section, respectively.

The skip selecting circuit 260 is provided between the filtering skip confirmation unit 250 and the filter unit 290 and between the filtering skip confirmation unit 250 and the filter unit 290, Is located between the filtering skip confirmation unit (250) and the decoding picture buffer (240). The filtering skip confirmation unit 250 may adaptively perform the deblocking filtering by controlling the skip selection circuit 260 based on the selection information by the filtering skip flag.

The filter unit 290 may also be referred to as an in-loop filter unit and may include at least one of a deblocking filter 270 and a sample adaptive offset (SAO) 280 . The filter unit 290 may perform filtering on the reconstructed image.

The deblocking filter 270 will be described in more detail as follows. That is, a quantization error may occur in the quantization step during the prediction, conversion, quantization, and entropy encoding processes. This is adjusted by the quantization parameter value. If the quantization parameter value is small, dense quantization is performed on the transform coefficient so that the quantization error is relatively small, and if the quantization parameter value is large, the quantization error may be relatively large. In order to solve such a problem, it is possible to reduce image deterioration by performing filtering on the restored picture.

17, the filtering skip confirmation unit 250 is additionally constructed so that the filtering skip confirmation unit 250 determines whether or not the filter unit 290 is applied and skipped The skip selection circuit 26 can be controlled. Accordingly, parallel processing can be performed using one DPB 240 without using additional memory.

The setting information for this may be the same in the encoder and the decoder, and may be transmitted in units of a sequence, picture, slice, or the like between the encoder and the decoder.

18 is a configuration diagram of an image decoding apparatus to which a filtering skip confirmation section is added.

Referring to FIG. 18, an image decoding apparatus according to an embodiment of the present invention includes an entropy encoder, an inverse quantization unit, an inverse transform unit, an intra prediction unit, A deblocking filter, an adaptive sample offset (SAO), a reconstructed picture buffer (DPB), and a filtering skip confirmation unit.

Here, with respect to the existing configuration, since the above-described FIG. 3 can be referred to, the duplicated description is omitted. Referring to FIG. 18, it can be seen that the filtering skip confirmation unit is added to the existing image decoding apparatus.

That is, the video decoding apparatus can instruct a switching operation on whether or not to skip the filtering based on the flag transmitted from the video encoding apparatus. If the filtering is skipped, the corresponding data is immediately stored in the reconstructed picture buffer (DPB). Otherwise, the corresponding data can be stored in the reconstructed picture buffer (DPB) after the filtering is applied.

19 to 22 are diagrams for explaining flag transmission in units of blocks of various sizes in the image encoding method according to the embodiment of the present invention.

19 to 22 illustrate examples in which a flag indicating whether or not the in-loop filter is adaptively applied or a flag indicating whether or not the current picture is utilized as a reference block of another picture can be transmitted in various block units can do.

Referring to FIG. 19, if the maximum encoding unit is 64 × 64, the maximum encoding unit can be transmitted. The maximum coding unit may correspond to a coding tree unit (CTU).

Referring to FIG. 20, the maximum encoding unit can be transmitted as a divided encoding unit. Tree-based partitioning is possible when flags are created in split units. In this embodiment, it is shown that the maximum encoding unit may be divided into four, or at least one of the four encoding units may be divided into four again.

Referring to FIG. 21, one or more maximum encoding units may be transmitted in a bundle unit. In this embodiment, it is shown that four maximum encoding units can be transmitted in one bundle unit.

Referring to FIG. 22, it is possible to combine transmission of one or more maximum encoding units in a bundle unit and transmission of a maximum encoding unit in a plurality of divided division units.

19 to 22 illustrate quad tree partitioning by the partitioning method, but may be divided into various block sizes and types according to the partitioning method in the partitioning part (see FIG. 2).

Information on the size of the minimum block from the maximum block size can be set equally in the encoder and the decoder, and the information related thereto can be transmitted in units of a sequence, picture, slice, or the like.

23 is an exemplary diagram for explaining a method of transmitting a flag indicating whether a reference block is utilized or not.

Referring to FIG. 23, a circle indicates a split flag, a square indicates a filtering application flag, a hatched segment indicates that the filtering is applied, and a colorless means that the segmentation is not divided or the filtering is not applied.

Here, if at least one sub-block is referred to, the corresponding sub-block may be divided, and if the sub-block is not further divided, the on / off flag of the in-loop filter may be transmitted. The depth for the partition of the block can be determined by the size of the largest block, the picture type, etc., but not divided if the supported depth is zero. Information on this can be transmitted in units of a sequence, a picture, a slice, and the like. Here, it is assumed that the maximum block size is 64 × 64, and the supported depth is supported up to 3 (up to 8 × 8 in this case).

In this case, since the size of the block is flexible, it is necessary to first transmit whether or not to divide the block. If it is 1, it means that the block is divided. If the block is 0, the block is not divided anymore and the upper left block, , The lower left block order (or the jet scan direction).

First, 64 (64x64 partitions) indicating division are transmitted to 64x64 blocks (uppermost part of the tree structure) indicated by CTU_64, and 1 (32x32 partitions) are allocated to the upper left block (32x32) (16 × 16 decision) 0 (16 × 16 decision) 0 (16 × 16 decision) 0 (16 × 16 decision) can be transmitted for four blocks (16 × 16) have. Subsequently, 0 (32 × 32) is transmitted to the upper right block to transmit no further division, and 0 (32 × 32) can be transmitted to the lower left block. 1 (32 x 32 division) is transmitted for the lower right block (32 x 32) to transmit the division, and 1 (16 x 16 division) is transmitted for the upper left block (16 x 16) . In the case of the upper left block (16 × 16), since the size of the divided block (8 × 8) is equal to the supported depth depth (8 × 8), the division flag is not transmitted to the divided block (8 × 8) Do not. Then, 0 (16 × 16), 0 (16 × 16), and 0 (16 × 16) can be transmitted to the remaining blocks (16 × 16). That is, 1100000011000 can be transmitted as instruction data indicating whether or not to divide.

Through the above process, it is possible to generate information for notifying whether or not to divide, and thereafter, information about whether to apply filtering can be generated. 1, 0, 1, 1, 0, 1, 0, 0, 1, 0 can be transmitted in the jet scan direction with respect to the flag indicating whether the filtering is applied to thirteen blocks.

The bitstream structure of the division flag for expressing the flag for the division information of the block may be selected from one or more scanning methods. It is possible to preferentially obtain the additional division information in the block divided on the basis of the first block on the basis of the jet scan as in the above example. Or a jet scan based depth and depth ordering method, the division information at the current level depth depth is obtained on the basis of the initial block, and the division information is obtained at the depth level of the next level. When the above method is used, 1 (64 × 64 division), 1 (32 × 32 division), 0 (32 × 32 division) (16x16), 0 (16x16), 0 (16x16), 0 (16x16), 0 ), And 0 (16 x 16 decision). That is, 1100100001000 can be transmitted as instruction data indicating whether or not to divide. Other scanning methods not shown in the above examples may be included and selected.

In this embodiment, the data in the already-coded current picture is not yet subjected to the in-loop filter, and there is degradation between the blocks. The memory containing such data may be temporary memory that is only stored during the encoding or decoding of the current picture. When the data with deterioration is referenced through block matching, there may be a disadvantage that the accuracy of prediction is degraded due to deterioration between blocks. In this case, the block deterioration can be reduced by applying filtering that reduces the block deterioration at the left and upper boundaries of the current block before the coding of the block is completed and before proceeding to the next block.

In addition, the right and bottom boundaries can be processed in the block to be coded next. The filtering application can be determined based on predetermined conditions such as the type of the block boundary, the minimum block size, and the like at which block boundary the filtering is applied. Filtering may be performed at the transform block boundary, filtering may be performed at the predict block boundary, or may be performed at a common block boundary between the transform block and the prediction block. The size of the minimum block for which filtering is performed can also be set, and information related thereto can be generated or transmitted in units of a sequence, a picture, a slice, or the like.

In addition, characteristics of a block placed at a block boundary, such as encoding mode, block boundary property, prediction information, encoding coefficient, etc., can be analyzed to determine whether to apply filtering, a pixel to be filtered, The related setting information may be set equally in the encoder and the decoder, and may be transmitted in units of a sequence, a picture, a slice, or the like. The filtering employed in the above-described embodiment can use the same configuration as that of the existing deblocking filter, and other configurations can also be used.

According to the embodiments described above, it is possible to use an international codec such as MPEG-2, MPEG-4, or H.264 or other codecs in which the intra prediction technique is used, a medium using these codecs, A high-efficiency image encoding / decoding technique can be provided.

In future, it is expected to be applied to the current high efficiency image coding technology (HEVC) and standard codec such as H.264 / AVC and the image processing field using intra prediction.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the present invention as defined by the following claims It can be understood that

Claims (20)

An image encoding method for performing adaptive filtering in inter-picture prediction,
Performing inter-picture prediction on a current block to be coded to search for a reference block;
Generating a flag indicating whether the searched reference block is within a current picture in which the current block is located;
Encoding and restoring the current block using the searched reference block;
Adaptively applying an in-loop filter to the restored current block with reference to the flag,
And storing the current block to which the in-loop filter is applied or not in a reconstructed picture buffer (DPB). The method according to claim 1,
Wherein the in-loop filter comprises at least one of a deblocking filter and an adaptive sample offset (SAO). The method according to claim 1,
Adaptively applying the in-loop filter comprises:
Wherein the in-loop filter is not applied to the restored current block if the flag indicates that the reference block is present in the current picture. The method according to claim 1,
Adaptively applying the in-loop filter comprises:
Wherein the in-loop filter is applied to the restored current block if the flag indicates that there is no reference block in the current picture. An image encoding method for performing inter picture prediction,
Reconstructing a video signal located in the same picture as a current block and including a reference block that is block-matched when inter-screen prediction of the current block is coded;
Storing the restored video signal in a temporary memory separate from a decoded picture buffer (DPB); And
And performing inter-picture prediction through block matching on the current block using the reference block included in the reconstructed video signal. The method of claim 5,
Wherein the step of storing in the temporary memory comprises:
Wherein the reconstructed image signal is stored in a separate temporary memory via an in-loop filter and a separate filter. The method of claim 6,
Wherein the separate filter performs filtering on the left and upper boundaries of the reference block included in the reconstructed image signal. The method of claim 6,
Wherein the separate filter performs filtering at least one of a transform block boundary of the reconstructed image signal, a predictive block boundary, and a common block boundary between the transform block and the predictive block. An image decoding method for performing adaptive filtering in inter-picture prediction,
Reconstructing, from a bitstream, a video signal including a reference block that is block-matched when inter-screen prediction of a current block to be decoded;
Obtaining a flag from the bitstream indicating whether the reference block is within a current picture in which the current block is located;
Reconstructing the current block using the reference block;
Adaptively applying an in-loop filter to the restored current block based on the obtained flag; And
And storing the current block to which the in-loop filter is applied or not applied in a reconstructed picture buffer (DPB). The method of claim 9,
Characterized in that the in-loop filter comprises at least one of a deblocking filter, an adaptive sample offset (SAO). The method of claim 9,
Adaptively applying the in-loop filter comprises:
Wherein if the flag indicates that the reference block is present in the current picture, the in-loop filter is not applied to the restored current block. The method of claim 9,
Adaptively applying the in-loop filter comprises:
Wherein the in-loop filter is applied to the restored current block if the flag indicates that there is no reference block in the current picture. An image decoding method for performing inter picture prediction,
Reconstructing a video signal including a reference block located in the same picture as a current block from a bitstream and being block-matched when inter-screen prediction of the current block;
Storing the restored video signal in a temporary memory separate from a decoded picture buffer (DPB); And
And performing inter-picture prediction through block matching on the current block using the reconstructed video signal. 14. The method of claim 13,
Wherein the step of storing in the temporary memory comprises:
Wherein the reconstructed image signal is stored in the temporary memory via an in-loop filter and a separate filter. 15. The method of claim 14,
Wherein the separate filter performs filtering on the left and upper boundaries of the reference block included in the reconstructed image signal. 15. The method of claim 14,
Wherein the separate filter performs filtering at least one of a transform block boundary, a predictive block boundary, and a common block boundary between the transform block and the predictive block of the reconstructed image signal. In an image decoding apparatus that performs adaptive filtering in inter-picture prediction and includes one or more processors,
The one or more processors,
Reconstructing, from a bitstream, a video signal including a reference block that is block-matched when inter-screen prediction of a current block to be decoded;
Obtaining a flag from the bitstream indicating whether the reference block is within a current picture in which the current block is located;
Reconstructing the current block using the reference block;
Adaptively applying an in-loop filter to the restored current block based on the obtained flag; And
And storing the current block to which the in-loop filter is applied or not applied in a reconstructed picture buffer (DPB). 18. The method of claim 17,
Characterized in that the in-loop filter comprises at least one of a deblocking filter, adaptive sample offset (SAO). 18. The method of claim 17,
Adaptively applying the in-loop filter comprises:
Wherein the in-loop filter is not applied to the restored current block if the flag indicates that the reference block is present in the current picture. 18. The method of claim 17,
Adaptively applying the in-loop filter comprises:
And if the flag indicates that there is no reference block in the current picture, applies the in-loop filter to the restored current block.

Images