KR20160146590A - Image encoding method and image decoding method and apparatus using adaptive deblocking filtering - Google Patents

Abstract

The present invention relates to an image encoding method and an image decoding method using adaptive deblocking filtering according to a prediction mode, and a device thereof. The image encoding method includes: a step of generating a restoring signal based on a prediction signal referring to an image decoded after encoding and a residual signal generated through inverse quantization and inverse transform; a step of checking filtering skip for the restoring signal; and a step of performing filtering skip or deblocking filtering. The step of performing deblocking filtering includes a sub step of setting filtering strength and a third sub step of determining whether the filtering is applied.

Description

[0001] IMAGE ENCODING METHOD AND IMAGE DECODING METHOD AND APPARATUS USING ADAPTIVE DEBLOCKING FILTERING [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to image encoding and decoding techniques, and more particularly, to an image encoding method and an image decoding method and apparatus using adaptive deblocking filtering according to a prediction mode.

With the spread of the Internet and mobile terminals and the development of information and communication technology, the use of multimedia data is increasing rapidly. Therefore, in order to perform various services or tasks through image prediction in various systems, there is a great need for improving the performance and efficiency of the image processing system.

Meanwhile, in the conventional image encoding / decoding technique, motion information on neighboring blocks of the current block is predicted in at least one reference picture before or after the current picture according to the inter picture prediction method, The motion information for the current block is estimated by acquiring the motion information from the reference block.

However, existing inter-picture prediction has a drawback in that the calculation complexity is high because a prediction block is generated using a temporal prediction mode between pictures, and in-picture prediction has a disadvantage that it has a large coding complexity.

In addition, image encoding is divided into prediction, transformation, quantization, and entropy coding. In the quantization process, blocking artifacts and ringing phenomenon occur in a reconstructed picture due to a quantization error. As a result, There is a problem that the subjective quality of the image is deteriorated.

Therefore, in the conventional image encoding and decoding techniques using quantization, filtering is performed on the reconstructed picture to improve the subjective picture quality. The prior art related to the filtering of the reconstructed picture is largely divided into a post-processing filter method and an in-loop filter method depending on whether a filtered picture is used as a reference picture in the inter-picture prediction mode.

In the post-processing filter method, filtering is performed right before the display output of the restored image outside the video decoder. In the in-loop filtering method, after filtering is applied to the restored picture, the filtered picture buffer is inserted into the decoded picture buffer And is used as a reference picture in the prediction mode.

On the other hand, the in-loop filter method, that is, the deblocking filter method, performs filtering after loading restored pixels stored in the memory when performing filtering, and stores the filtered pixels in the memory again. . In addition, the deblocking filter has a complicated filtering operation itself, and has a disadvantage in that it takes a considerably large complexity of 20 to 30% in the decoder due to the computational complexity and overhead for memory access. Thus, in the image coding and decoding technology, an effective filtering method for the reconstructed picture is required.

It is an object of the present invention to provide an image encoding method and an image decoding method and apparatus using adaptive deblocking filtering.

It is another object of the present invention to provide an image encoding method and an image decoding method and apparatus capable of improving the performance and efficiency of an image processing system.

According to an aspect of the present invention, there is provided a method for generating a reconstructed signal, the reconstructed signal including a predicted signal with reference to a decoded image and a residual signal generated through an inverse quantization and inverse transform process, And performing deblocking filtering if it is not the filtering skip.

According to another aspect of the present invention, there is provided a method for generating a reconstructed signal, the method including generating a reconstructed signal based on a predicted signal with reference to a decoded image, a residual signal generated through an inverse quantization and inverse transform process, And performing deblocking filtering if it is not the filtering skip.

According to another aspect of the present invention, there is provided an apparatus and method for decoding an audio signal including an entropy decoding unit, an inverse quantization unit, an inverse transform unit, an add / subtract unit, a prediction unit, a filtering skip confirmation unit, wherein the skip selecting circuit is located between the filtering skip confirmation unit and the filter unit or between the filtering skip confirmation unit and the decoding picture buffer, and the filtering skip confirmation unit is located between the filtering unit and the filter unit, And the skip verification unit controls the skip selection circuit based on the selection information by the filtering skip flag.

According to still another aspect of the present invention, there is provided a method for decoding a reconstructed signal, the method including generating a reconstructed signal based on a predictive signal with reference to a decoded image after being encoded and a residual signal generated through an inverse quantization and inverse transform process, And performing deblocking filtering if it is not the filtering skip, wherein performing deblocking filtering comprises: a sub-step of identifying a block boundary; a sub-step of setting a filtering strength; And a sub-step of applying filtering according to whether or not the filtering is applied.

According to another aspect of the present invention, there is provided a method for generating a reconstructed signal, the method including generating a reconstructed signal based on a predictive signal with reference to a decoded image, a residual signal generated through an inverse quantization and inverse transform process, And performing deblocking filtering if it is not the filtering skip, wherein performing the deblocking filtering comprises: a sub-step of identifying a block boundary; a sub-step of setting a filtering strength; And sub-steps of applying filtering according to whether or not the filtering is applied.

According to another aspect of the present invention, there is provided a decoding apparatus for decoding a decoding program for generating a reconstructed signal based on a predictive signal referring to a decoded image, a residual signal generated through an inverse quantization and an inverse transform process, And a processor for executing a program of a decoding program or a filtering program, the processor being connected to the memory for storing the filtering program for skipping or performing the filtering, Generates a reconstructed signal based on a reference signal and a residual signal generated through an inverse quantization and inverse transform process, confirms a filtering skip for a reconstructed signal, performs deblocking filtering if the skip is not a filtering skip, When checking block boundaries A filtering strength is set, whether or not filtering is applied is determined, and filtering is applied according to whether filtering is applied or not.

According to another aspect of the present invention for achieving the above object, there is provided a video encoding method for a reference pixel configuration in intra-frame prediction, comprising: obtaining a reference pixel of a current block from intra- Generating a prediction block of a current block by using a reference pixel to which adaptive filtering is applied as an input value according to a prediction mode of the current block, generating a post-processing filter adaptive to the prediction block, The method comprising the steps of:

Here, the acquiring step may acquire a reference pixel of the current block from the neighboring block.

Here, the acquiring step may be determined according to availability of neighboring blocks.

Here, the availability of the neighboring block may be determined by the position of the neighboring block and / or a specific flag (constrained_intra_pred_flag). In one example, a particular flag may have a value of 1 when neighboring blocks are available. That is, when the prediction mode of the neighboring block is the inter-view mode, it can mean that the reference pixel of the block can not be used for prediction of the current block.

Here, the specific flag (constrained_intra_pred_flag) may be determined according to the prediction mode of the neighboring block, and the prediction mode may be one of intra prediction or inter prediction.

Here, when the specific flag (constrained_intra_pred_flag) is 0, the availability of the neighboring block becomes 'true' irrespective of the prediction mode of the neighboring block. If the prediction mode of the neighboring block is intra- , And if the prediction is inter-picture prediction, the availability of the neighboring block may be 'false'.

Here, the inter picture prediction can generate a prediction block with reference to one or more reference pictures.

Here, the reference picture is managed through the reference picture list 0 (List 0) and the reference picture list 1 (List 1), and one or more past picture, future picture, and current picture can be included in the List 0 and List 1.

Here, List 0 and List 1 can be adaptively determined whether to insert the current picture into the reference picture list.

Here, the information for determining whether to insert the current picture into the reference picture list may be included in a sequence, a picture parameter set, or the like.

According to still another aspect of the present invention, there is provided a method of decoding an image performed in a computing device, the method comprising: obtaining a flag for a reference pixel availability of a neighboring block in a sequence or picture unit from an input bitstream; Determining a reference pixel availability of a neighboring block when intra prediction is performed according to a flag; When the flag is 0, the reference pixel of the neighboring block is used for prediction of the current block regardless of the prediction mode of the neighboring block. If the prediction mode of the neighboring block is intra prediction when the flag is 1, And the reference pixel of the neighboring block is not used for prediction of the current block when the prediction mode of the neighboring block is the inter-view prediction.

Here, the inter picture prediction can generate a prediction block based on block matching in the reference picture.

Here, the reference picture can be managed by List 0 in the P picture, and by List 0 and List 1 in the B picture.

Here, in the inter picture prediction, the current picture can be included in List 0.

Here, the current picture can be included in List 1 in inter-picture prediction.

Here, the inclusion of the current picture in the List 0 and List 1 can be determined based on the flag transmitted in the sequence parameter.

Here, the inclusion of the current picture in the List 0 and List 1 can be determined based on the flag transmitted from the picture parameter.

In the case of using the image coding method and the image decoding method and apparatus using the adaptive deblocking filtering according to the embodiment of the present invention as described above, the performance of the deblocking filtering is improved and the performance of the image processing system or the image prediction system is improved. Can be improved.

In addition, according to the present invention, it is possible to improve the subjective image quality by reducing the blocking effect for the intra-picture prediction block and the inter-picture prediction block, and to improve the coding efficiency.

1 is a view for explaining a system using an image encoding apparatus and / or an image decoding apparatus according to the present invention.
2 is a block diagram of an image encoding apparatus according to an embodiment of the present invention.
3 is a block diagram of an image decoding apparatus according to an embodiment of the present invention.
FIG. 4 is a diagram illustrating an inter-picture prediction of a P slice in an image encoding and decoding method according to an embodiment of the present invention.
FIG. 5 is a diagram illustrating an inter-picture prediction of a B slice in an image encoding and decoding method according to an embodiment of the present invention.
FIG. 6 is a diagram illustrating an example of generating a prediction block in a unidirectional manner in the image encoding and decoding method according to an embodiment of the present invention. Referring to FIG.
FIG. 7 is a diagram illustrating a reference picture list in the image encoding and decoding method according to an embodiment of the present invention. Referring to FIG.
FIG. 8 is a diagram illustrating an example of performing inter-picture prediction from a reference picture list in an image encoding and decoding method according to an embodiment of the present invention.
9 is an exemplary diagram for explaining intra prediction in the image encoding method according to an embodiment of the present invention.
10 is an exemplary diagram for explaining a prediction principle in a P slice or a B slice in the image encoding method according to an embodiment of the present invention.
11 is an exemplary diagram for explaining a case where interpolation is performed in the image encoding method of FIG.
FIG. 12 is a diagram for explaining a main procedure of an image coding method according to an embodiment of the present invention with a syntax in a coding unit.
13 is a diagram illustrating an example of supporting a symmetric type partition or an asymmetric type partition as in inter picture prediction when a prediction block is generated through block matching in the current picture used in FIG. Fig.
FIG. 14 is an exemplary diagram for explaining that 2Nx2N and NxN can be supported in inter-picture prediction (Inter) as in intra-picture prediction (Intra) in FIG.
FIG. 15 is a diagram for explaining a process of performing a horizontal 1D filter on pixels located at positions a, b, and c of an image (assuming x) in the image encoding method according to an embodiment of the present invention.
16 is an exemplary diagram of a current block and neighboring blocks that can be employed in the image encoding method according to an embodiment of the present invention.
17 is an exemplary diagram for explaining generation of a prediction block through block matching in a current block in the image encoding method according to an embodiment of the present invention.
FIG. 18 is an exemplary diagram for explaining reference blocks in an already encoded region used as a prediction block through at least one block matching in the image encoding method according to an embodiment of the present invention. FIG.
19 is a block diagram for explaining an image encoding apparatus according to an embodiment of the present invention.
20 is a block diagram for explaining an image decoding apparatus according to an embodiment of the present invention.
21 is an exemplary diagram for explaining transmission in units of blocks of various sizes in an image encoding method according to an embodiment of the present invention.
22 is an exemplary diagram for explaining an example of flag transmission in the image encoding method according to an embodiment of the present invention.
23 is a flowchart illustrating a filtering procedure in an image encoding method and an image decoding method according to an embodiment of the present invention.
FIG. 24 is an illustration of a prediction block or a transform block that can be employed in the image encoding method of FIG. 23. FIG.
25 is an exemplary diagram for explaining a boundary for setting a filtering strength in the image encoding method of FIG.
FIG. 26 is an exemplary diagram for explaining a pattern of pixel values in an image in the image encoding method of FIG. 25; FIG.
FIG. 27 is a flowchart for explaining a filtering intensity setting process in the image encoding method of FIG. 23. FIG.
28 is a flowchart for explaining another example of the filtering intensity setting process in the image encoding method of FIG.
29 is a flowchart for explaining another example of the filtering intensity setting process in the image encoding method of FIG.
30 is a flowchart for explaining another example of the filtering intensity setting process in the image encoding method of FIG.
31 is an exemplary diagram for explaining a filtering application determination process in the image encoding method of FIG.
32 is an exemplary diagram for explaining another example of the filtering application determination process in the image encoding method of FIG.
FIG. 33 is an exemplary diagram for explaining another example of the filtering application determination process in the image encoding method of FIG. 23. FIG.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It is to be understood, however, that the invention is not to be limited to the specific embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

The terms first, second, A, B, etc. may be used to describe various elements, but the elements should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. And / or < / RTI > includes any combination of a plurality of related listed items or any of a plurality of related listed items.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The terminology used in this application is used only to describe a specific embodiment and is not intended to limit the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In the present application, the terms "comprises ", or" having ", and the like, specify that the presence of stated features, integers, steps, operations, elements, parts, or combinations thereof, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, are meant to be equivalent to those generally understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries are to be interpreted as being consistent with the meanings in the context of the relevant art and are not to be construed as ideal or overly formal meanings unless explicitly defined in the present application.

The moving picture may be generally composed of a series of pictures, and each picture may be divided into a predetermined area such as a frame or a block. In addition, the divided area includes not only a block but also a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), a transform unit (TU) As shown in FIG. Each unit may consist of one luminance block and two color difference blocks, which may be otherwise configured according to the color format. Further, the size of the luminance block and the color difference block can be determined according to the color format. For example, in the case of 4: 2: 0, the size of the color difference block may have a length of 1/2 of the luminance block. For these units and terms, terms such as conventional high efficiency video coding (HEVC) or advanced video coding (H.264 / AVC) can be referred to.

A reference picture, a reference block, or a reference pixel is referred to as a reference picture, a block, or a pixel to be referred to in encoding or decoding a current block or a current pixel. It is also to be understood that the term "picture" described below may be used in place of other terms having equivalent meanings such as image, frame, etc., If you are a child, you can understand.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In order to facilitate the understanding of the present invention, the same reference numerals are used for the same constituent elements in the drawings and redundant explanations for the same constituent elements are omitted.

1 is a view for explaining a system using an image encoding apparatus and / or an image decoding apparatus according to the present invention.

1, a system using an image encoding apparatus and / or an image decoding apparatus includes a personal computer (PC), a notebook computer, a personal digital assistant (PDA), a portable multimedia player A user terminal 11 such as a PMP, a Playstation Portable (PSP), a wireless communication terminal, a smart phone, a TV or the like, or a server terminal such as an application server and a service server 12). Such a system may be referred to as a computing device.

In addition, the computing device may be a communication device such as a communication modem for performing communication with various devices or a wired / wireless communication network, an inter or intra prediction for coding or decoding an image, A memory 18 for storing various programs and data for executing programs, a processor 14 for executing and calculating programs and the like, and the like.

In addition, the computing device can transmit a bitstream-encoded image by a video encoding device through a wired or wireless communication network such as the Internet, a short-range wireless communication network, a wireless LAN network, a WiBro network, a mobile communication network, (Universal Serial Bus), and the like, and can be decoded by the image decoding apparatus and reproduced as a reconstructed image. In addition, an image encoded by a video encoding apparatus by a bit stream may be transferred from a coding apparatus to a decoding apparatus via a computer-readable recording medium.

2 is a block diagram of an image encoding apparatus according to an embodiment of the present invention. 3 is a block diagram of an image decoding apparatus according to an embodiment of the present invention.

2, the image encoding apparatus 20 according to the present embodiment includes a predictor 200, a subtractor 205, a transformer 210, a quantizer 215, an inverse quantizer 220, An inverse transform unit 225, an adding unit 230, a filter unit 235, a decoded picture buffer (DPB) 240, and an entropy coding unit 245. In addition, the image encoding apparatus 20 may further include a segmentation unit 190.

3, the image decoding apparatus 30 according to the present embodiment includes an entropy decoding unit 305, a predicting unit 310, an inverse quantizing unit 315, an inverse transforming unit 320, A decoded picture buffer 325, a filter unit 330, and a decoded picture buffer 335.

The image encoding apparatus 20 and the image decoding apparatus 30 may be separate apparatuses, but may be implemented as one image encoding and decoding apparatus according to an implementation. In this case, the prediction unit 200, the inverse quantization unit 220, the inverse transformation unit 225, the addition unit 230, the filter unit 235, and the decoding picture buffer 240 of the image coding apparatus 20 Substantially the same technology as the predicting unit 310, the inverse quantizing unit 315, the inverse transforming unit 320, the adding unit 325, the filter unit 330 and the decoded picture buffer 335 of the image decoding apparatus 30 Elements may be implemented to include at least the same structure or at least perform the same function. In addition, the entropy coding unit 245 may correspond to the entropy decoding unit 305 when performing its function inversely. Therefore, a detailed description of the following technical elements and their operating principles will not be repeated.

Since the image decoding apparatus corresponds to a computing apparatus that applies the image encoding method performed in the image encoding apparatus to decoding, the following description will be made with reference to the image encoding apparatus.

The computing device may include a memory for storing a program or a software module implementing the image encoding method and / or the image decoding method, and a processor connected to the memory for executing the program. The video encoding apparatus may be referred to as an encoder, and the video decoding apparatus may be referred to as a decoder.

Each component of the image encoding apparatus of this embodiment will be described in more detail as follows.

The division unit 190 divides the input image into blocks (MxN) of a predetermined size. Here, M or N is an arbitrary natural number of 1 or more.

In detail, the division unit 190 may be constituted of a picture division unit and a block division unit. The size or shape of the block can be determined according to the characteristics and resolution of the image, and the size or shape of the block supported through the picture partitioning unit is M × N square shape in which the length and the length are represented by an exponent of 2 (256 X 256, 128 x 128, 64 x 64, 32 x 32, 16 x 16, 8 x 8, 4 x 4, etc.), or an M x N rectangular shape. For example, an input image can be divided into 256 × 256 for an 8 k UHD image having a high resolution, 128 × 128 for a 1080p HD image, and 16 × 16 for a WVGA image.

Information on the size or shape of such a block can be set in units of a sequence, picture, slice, or the like, and related information can be transmitted to the decoder. That is, a sequence parameter set, a picture parameter set, a slice header, or a combination thereof.

Here, a sequence indicates a constituent unit formed by collecting a number of related scenes. A picture is a term that refers to a series of luminance (Y) components or luminance and chrominance (Y, Cb, Cr) components in one scene or picture. The range of one picture is one frame or one field .

A slice can refer to one independent slice segment and a number of dependent slice segments that are in the same access unit. An access unit means a set of network abstraction layer (NAL) units associated with a single coded picture. The NAL unit is a syntax structure composed of a video compression bitstream in a network-friendly format in the H.264 / AVC and HEVC standards. It is common to configure one slice unit as one NAL unit, and system standards generally regard a set of NALs or NALs constituting one frame as one access unit.

Returning to the description of the picture division section, information on the block size or type (M × N) can be made of an explicit flag, specifically, block type information, one length information when the block is square, The length information, or the difference value information between the horizontal and vertical lengths. For example, if M and N are composed of exponential powers of k (assuming k is 2) (M = 2 ^m , N = 2 ⁿ ), information on m and n can be unary binarization, Or the like, and transmit the related information to the decoding apparatus.

Further, if the division allowable minimum size (Minblksize) supported by the picture division unit is IxJ (assuming I = J for convenience of explanation, I = 2 ⁱ , J = 2 ^j ), information about mi or nj . As another example, when M and N are different, it is possible to convey the difference value (| mn |) between m and n. Alternatively, information about im or nj may be conveyed if the division allowable maximum size (Maxblksize) supported by the picture division unit is IxJ (assuming I = J for convenience of explanation, I = 2 ⁱ , J = 2 ^j ) .

In case of an implied situation, for example, if a syntax for related information exists but can not be confirmed by an encoder or a decoder, the encoder or decoder may follow preset default settings. For example, if the related syntax can not be confirmed at the step of confirming the block type information, the block type may be set to a square shape, which is the default setting. In addition, the step of checking the block size information may check the difference value related syntax in the step of checking the block size information through the difference value from the minimum allowable size (Minblksize) as in the above example, If the syntax related to the MinBlkSize is not available, it can be obtained from the default settings related to the MinBlkSize which is prepared in advance.

As such, the size or shape of the block in the picture partitioning unit can be implicitly determined by explicitly transmitting the related information in the encoder and / or the decoder or according to the characteristics and resolution of the image.

As described above, a block divided and determined through the picture division unit can be used as a basic encoding unit. The block divided and determined through the picture division unit may be a minimum unit that constitutes a high level unit such as a picture, a slice, and a tile, and may be a coding block, a prediction block, a quantization block, an entropy block, an inloopfiltering block, and the like, but some blocks are not limited to this and may be an exception. For example, some such as an in-loop filtering block may be applied in units larger than the block sizes described above.

The block division unit performs division on blocks such as coding, prediction, conversion, quantization, entropy, and in-loop filter. The partitioning unit 190 also functions in each configuration. For example, the transforming unit 210 may include a transform block dividing unit and the quantizing unit 215 may include a quantization block dividing unit. The size or shape of the initial block of the block partition may be determined by the result of division of the previous level or higher level block.

For example, in the case of a coded block, a block obtained through a picture division unit which is a previous stage can be set as an initial block. Alternatively, in the case of a prediction block, a block obtained through a division process of a coding block which is a higher level of the prediction block may be set as an initial block. Alternatively, in the case of a transform block, a block obtained through a dividing process of an encoding block, which is a higher level of the transform block, may be set as an initial block.

The condition for determining the size or shape of the initial block is not always fixed and there may be some cases where the block is changed or an exception is made. In addition, it is also possible to set the setting level of the current level (for example, the size of the supported conversion block, the type of the conversion block, etc.) of the previous level or the higher level block (for example, the size of the encoding block, ) May affect the division operation of the current level (division possibility, divisible block type, etc.) according to a combination of at least one or more factors.

The block partitioning part can support a quad tree based partitioning method. That is, the block may be divided into four blocks each having a length of 1/2 in the horizontal and vertical directions in the block before division. This means that the block size limit (dep >> k, n >> k) means the allowable division depth limit (dep >> k, k >>) while the block size limit (dep_k) Can be repeatedly performed.

In addition, the block dividing unit can support a binary tree-based partitioning scheme. This indicates that the length of one of the horizontal and vertical lengths can be divided into two blocks having a length of 1/2 as compared with the pre-division block. In the case of the quadtree partitioning and the binary tree partitioning, a symmetric partition or an asymmetric partition may be used, and the partitioning scheme may be determined according to the setting of the encoder / decoder. The image encoding method of this embodiment will be mainly described with respect to the symmetric division method.

The division flag (div_flag) indicates whether or not each block is divided. If the corresponding value is 1, the division is performed. If the value is 0, the division is not performed. Alternatively, if the value is 1, the segmentation is performed and further segmentation is possible. If the value is 0, the segmentation is not performed, and the segmentation is not allowed any more. Depending on the conditions such as the minimum allowable division size and the limit of the allowable division depth, the flag may be considered only for the division, and the additional division may not be considered.

The split flag can be used in a quadtree partition, and also in a binary tree partition. In binary tree segmentation, the dividing direction may be one of the block depth, encoding mode, prediction mode, size, type, and type (encoding, prediction, And may be determined depending on at least one of factors such as a slice type, a division allowable depth limit, a division allowable minimum / maximum size, or a combination thereof. In addition, it can be divided according to the division flag and / or according to the division direction, that is, divided into only 1/2 of the width of the block or 1/2.

For example, suppose that a block supports M × N (M> N) and M is greater than N, and that the current partition depth (dep_curr) is smaller than the partitioning allowable depth limit, The division flag is allocated with 1 bit. If the corresponding value is 1, the horizontal division is performed. If the division value is 0, the division flag can be no longer divided. The split depth can be one divide depth for quad tree and binary tree divisions, and one divide depth for quad tree and binary tree divisions. In addition, the allowable depth limit may have one partition allowable depth limit for the quadtree and the binary tree partition, or a respective partition allowable depth limit for the quadtree and binary tree partition.

 As another example, if the block is M × N (M> N) and N is equal to the predetermined minimum allowable division size, and the horizontal division is not supported, the division flag is allocated to 1 bit. If the value is 1, If it is 0, no division is performed.

Also, flags (div_h_flag, div_h_flag) for horizontal division or vertical division can be respectively supported, and binary division can be supported according to the flags. It is possible to indicate whether each block is horizontally or vertically divided through the horizontal division flag (div_h_flag) or the vertical division flag (div_v_flag). If the horizontal division flag (div_h_flag) or the vertical division flag (div_v_flag) If 0, no horizontal or vertical division is performed.

In addition, if each flag is 1, horizontal or vertical division is performed and horizontal or vertical division is possible. If the value is 0, horizontal division or vertical division is not performed and further division of horizontal or vertical division may not be allowed . The flag may be considered for the division depending on the conditions such as the division allowable minimum size and the division allowable depth limit, and the flag may not be considered.

In addition, a flag (div_flag / h_v_flag) for horizontal division or vertical division can be supported, and binary division can be supported according to the flag. The division flag div_flag may indicate whether it is horizontally or vertically divided, and the division direction flag h_v_flag may indicate a horizontal or vertical division direction.

If the division flag (div_flag) is 1, the division is performed and the horizontal or vertical division is performed according to the division direction flag (h_v_flag). If 0, the horizontal or vertical division is not performed. If the value is 1, horizontal or vertical division is performed according to the division direction flag (h_v_flag), and horizontal or vertical division can be further performed. If the value is 0, horizontal or vertical division is not performed, It can be regarded as not permitting.

The flag may be considered for the division depending on the conditions such as the division allowable minimum size and the division allowable depth limit, and the flag may not be considered.

These division flags can also support for horizontal and vertical division, respectively, and can support binary tree division according to the flags. When the dividing direction is predetermined, only one of the two division flags may be used, or both of the two division flags may be used as in the above example.

For example, if the above flags are allowed, the possible block types can be divided into any one of M × N, M / 2 × N, M × N / 2, and M / 2 × N / 2. In this case, the flags can be encoded as 00, 10, 01, and 11 in the order of the horizontal division flag or the vertical division flag (div_h_flag / div_v_flag).

In the above case, the setting in which the division flag is used in the overlapping example is exemplified, and the setting in which the division flag can not be used overlapping is also possible. For example, the split block type may be divided into M × N, M / 2 × N, and M × N / 2. In this case, the above flags are encoded as 00, 01, Or the division flag div_flag and the horizontal-vertical flag h_v_flag, and the flags may be encoded as 0, 10, and 11 in the order of the horizontal or vertical division direction. Here, the meaning of the overlap may mean that the horizontal division and the vertical division are performed at the same time.

The quad tree division and the binary tree division described above can be used singly or in combination according to the setting of the encoder and / or the decoder. For example, a quadtree or binary tree partition may be determined depending on the size or type of the block. That is, when the block type is M x N and M is larger than N, the horizontal division, the block type is M x N, and if N is larger than M, the binary tree division can be supported according to the vertical division, Is M x N, and quad tree partitioning is supported if N and M are the same.

As another example, a binary tree partition may be supported if the size of the block (M x M) is greater than or equal to the block partition boundary value (thrblksize), and a quad tree partition may be supported if the size is smaller.

As another example, if M or N of the block (MxN) is equal to or smaller than the first allowable maximum size (Maxblksize1) and equal to or larger than the first allowable minimum size (Minblksize1), quad tree partitioning is supported, A binary tree segmentation may be supported if M or N of (M x N) is less than or equal to the second allowable maximum size Maxblksize2 and greater than or equal to the second allowable minimum size Minblksize2.

If the first division supporting range and the second division supporting range which can be defined as the division allowable maximum size and the division allowable minimum size overlap with each other, priority of the first or second division method is set according to the setting of the subdecoder Rank can be given. In the present embodiment, quad tree partitioning is used for the first partitioning method, and binary tree partitioning is used for the second partitioning method. For example, when the first division allowable minimum size (Minblksize1) is 16, the second division allowable maximum size (Maxblksize2) is 64, and the pre-division block is 64x64, the first division support range and the second division support range Quad tree partitioning and binary tree partitioning are possible because it belongs to all.

If divide flag (div_flag) is 1, quad tree partitioning is performed and additional quadtree partitioning is possible, if the priority is given by the first partitioning method (quad tree partitioning in this embodiment) , It can be regarded that no quadtree partitioning is performed and no further quad tree partitioning is performed. In addition, if the division flag (div_flag) is 1, the quadtree division is performed and the additional quadtree or binary tree division is possible. If the division flag (div_flag) is 0, the quadtree division is not performed. However, it can be regarded as a binary tree partitionable.

Depending on the conditions such as the minimum allowable division size and the limit of the allowable division depth, the flag may be considered only for the division, and the additional division may not be considered.

When the division flag (div_flag) is 1, it is divided into 4 blocks having a size of 32x32 and is larger than the first allowable minimum size (Minblksize1), so that the quadtree partitioning can be continued. When the division flag is 0, no additional quadtree division is performed, and the current block size (64x64) belongs to the second division support range, so that the binary tree division can be performed. When the division flag (in the order of div_flag / h_v_flag) is 0, no further division is performed, and in the case of 10 or 11, a horizontal division or a vertical division can be performed.

If the size of the current block (32x32) is smaller than the size of the current block (32x32) if the block before division is 32x32 and the division flag (div_flag) is 0 and no quad tree partitioning is performed and the second division allowable maximum size (Maxblksize2) Since it is not in the scope of 2-partition support, it may not support further partitioning. In the above description, the priority of the division method may be determined according to at least one of a slice type, a coding mode, a luminance / chrominance component, or a combination thereof.

As another example, various settings can be supported depending on the luminance and chrominance components. For example, the quadtree or binary tree partition structure determined by the luminance component can be used as it is without additional information / decoding in the color difference component. Alternatively, if independent division of the luminance component and the chrominance component is supported, the luminance component may support the quad tree and the binary tree, and the chrominance component may support the quad tree division.

In addition, quad tree division and binary tree division are supported for luminance and chrominance components, but the division support range may or may not be the same or proportional to luminance and chrominance components. For example, when the color format is 4: 2: 0, the division support range of the chrominance component may be N / 2 of the division support range of the luminance component.

As another example, different settings can be made depending on the slice type. For example, I slices can support quadtree partitioning, P slices can support binary tree partitioning, and B slices can support quadtree partitioning and binary tree partitioning.

Quad tree partitioning and binary tree partitioning can be set and supported according to various conditions as in the above example. It should be noted that the above-mentioned examples are not limited to the above-mentioned cases and may include cases in which mutual conditions are reversed, and may include one or more factors mentioned in the above-mentioned examples or combinations thereof, It is possible. The above allowable division depth limit can be determined according to at least one factor or a combination of at least one of division method (quad tree, binary tree), slice type, luminance / chrominance component, encoding mode and the like.

The division support range may be determined according to at least one factor or a combination thereof in a division mode (quad tree, binary tree), a slice type, a luminance / chrominance component, a coding mode, Maximum value, and minimum value. When information on this is configured with an explicit flag, length information of each of the maximum value / minimum value, or difference value information between the minimum value and the maximum value can be expressed.

For example, when the maximum value and the minimum value are composed of exponentiation powers of k (assuming k is 2), the exponential information of the maximum and minimum values can be encoded through various binarization and transmitted to the decoding apparatus. Alternatively, the difference value between the maximum value and the minimum value can be transmitted. The information transmitted at this time may be the difference information of the exponent information and the exponent of the minimum value

In accordance with the above description, information related to flags can be generated and transmitted in units of sequences, pictures, slices, tiles, blocks, and the like.

The block division information may be represented by a quad tree, a binary tree, or a combination of two tree schemes with the division flags shown in the above examples. The division flags may be encoded by various methods such as unary binarization and cutting type unary binarization, Device. The bitstream structure of the division flag for representing the division information of the block may be selected from one or more scanning methods.

For example, a bitstream of division flags can be configured based on the division depth order (from dep0 to dep_k), and a bitstream of division flags can be configured based on division. In the division depth order reference method, division information at the current level is acquired on the basis of the first block, and division information is acquired at the next level. In the division method, It means a method of preferentially acquiring additional division information, and other scanning methods not shown in the above example may be included and selected.

Also, according to the implementation, the block dividing unit may generate index information for a block candidate group of a predetermined type, which is not the division flag described above, and express it. M × N, M × N, M × N / 2, M / 4 × N, 3M / 4 × N, and M × N are the types of block candidates, M × N / 4, M × 3N / 4, M / 2 × N / 2, and the like.

When the candidate group of the divided block is determined as described above, the index information for the divided block type can be encoded by various methods such as fixed length binarization, truncated binarization, truncated binarization, and the like. At least one of factors such as the division depth of a block, a coding mode, a prediction mode, a size, a type, a type and a slice type, a division allowable depth limit, a division allowable minimum / maximum size, The divided block candidate group can be determined.

(M × N, M × N / 2, M / 2 × N / 2) as candidate lists 1 (list 1) (M × N, M / 2 × N, M × N / 2, M / N × 2) of the list 2 (list 2) 4 × N, 3M / 4 × N, M × N / 4, M × 3N / 4 and M / 2 × N / 2) are assumed to be the candidate list 4 (list4). For example, when describing M × N as a reference, a divided block candidate of candidate list 2 can be supported when (M = N), and a divided block candidate of candidate list 3 if (M ≠ N).

As another example, if M × N of M or N is greater than or equal to the boundary value blk_th, the divided block candidate of the candidate list 2 can be supported, and if it is smaller than that, the divided block candidate of the candidate list 4 can be supported. When M or N is equal to or greater than the first threshold value blk_th_1, if the divided block candidate of the candidate list 1 is smaller than the first threshold value blk_th_1 but equal to or greater than the second threshold value blk_th_2, If the divided block candidate of list2 is smaller than the second boundary value blk_th_2, the divided block candidate of the candidate list 4 can be supported.

As another example, it is possible to support a divided block candidate of the candidate list 2 when the coding mode is intra-picture prediction and a divided block candidate of the candidate list 4 when inter prediction is performed.

Even if the divided block candidates are supported, the bit configuration according to the binarization in each block may be the same or different. For example, if the divided block candidates are limited according to the block size or type as in the case of the division flag, the bit configuration according to the binarization of the corresponding block candidate may be changed. M × N, M × N / 2 and M / 2 × N / 2 in the case of (M> N) The binary bits of the index according to the M × N / 2 of the current condition and the M × N / 2 of the current condition may be different from each other.

Information on the segmentation and type of the block can be expressed using one of the segmentation flag or the segmentation index method according to the type of the block used for coding, prediction, conversion, quantization, entropy, in-loop filtering, and the like. In addition, block size limitation and division allowable depth limit for the partitioning and block type support may be different depending on each block type.

The encoding and decoding processes on a block-by-block basis can be performed in accordance with a process of determining a prediction block, determining a transform block, determining a quantization block, determining an entropy block, and determining an in-loop filter. The order of the encoding and decoding processes is not always fixed, and some of the sequences may be changed or excluded. The size and shape of each block are determined according to the coding cost of each candidate of the size and the shape of the block, and coding related information such as image data of each determined block and size and type of each determined block can be encoded.

The prediction unit 200 can be implemented using a prediction module, which is a software module, and can generate a prediction block using an intra-picture prediction scheme or an inter-picture prediction scheme for a block to be encoded. Here, the prediction block is a block which is understood to closely match the block to be encoded in terms of pixel difference, and can be determined by various methods including a sum of absolute difference (SAD) and a sum of square difference (SSD). Also, at this time, various syntaxes that can be used in decoding the image blocks may be generated. The prediction block can be classified into an in-picture block and an inter-picture block according to an encoding mode.

Intra prediction refers to a prediction technique that uses spatial correlation and predicts a current block using reference pixels of previously reconstructed and decoded blocks in the current picture. That is, the brightness value reconstructed by intra prediction and reconstruction can be used as reference pixels in the encoder and the decoder. Intra prediction can be effective for flat areas with continuity and areas with constant directionality, and it can be used for the purpose of guaranteeing random access and preventing error diffusion because it uses spatial correlation.

Inter prediction uses a compression technique that removes redundancy of data by using temporal correlation with reference to an image encoded in one or more past or future pictures. That is, the inter picture prediction can generate a prediction signal having high similarity by referring to one or more past or future pictures. In the encoder using inter-picture prediction, a block having a high degree of correlation with a block to be currently coded is searched in the reference picture, and the position information and residue signal of the selected block can be transmitted to the decoder. A reconstructed image can be constructed by generating the same prediction block as the encoder and compensating the transmitted residual signal.

FIG. 4 is a diagram illustrating an inter-picture prediction of a P slice in an image encoding and decoding method according to an embodiment of the present invention. FIG. 5 is a diagram illustrating an inter-picture prediction of a B slice in an image encoding and decoding method according to an embodiment of the present invention.

In the image coding method of this embodiment, the inter picture prediction generates a prediction block from a previously coded picture having a high temporal correlation, so that the coding efficiency can be increased. Current (t) may refer to the current picture to be coded and may be a first temporal distance (t-1) prior to the POC of the current picture based on temporal flow of the picture or POC (picture order count) And a second reference picture having a second temporal distance (t-2) before the first temporal distance.

4, inter-picture prediction that can be employed in the video coding method of this embodiment is the same as that of the current block of the current picture (current (t)) and that of the reference pictures (t-1, t-2) It is possible to perform motion estimation for finding an optimal prediction block from reference pictures (t-1, t-2) that have been previously encoded with a block having a high correlation through block matching of reference blocks. An interpolation process based on a structure in which at least one or more subpixels are arranged between adjacent two pixels as necessary for precise estimation is performed and then an optimal prediction block is found and motion compensation is performed, Can be found.

5, the inter-picture prediction that can be employed in the image coding method of the present embodiment is based on the already-coded reference pictures (" current (t) " t-1, t + 1). In addition, two prediction blocks can be generated from one or more reference pictures.

When encoding an image through inter-picture prediction, motion vector information for the optimal prediction block and information about the reference picture are encoded. In this embodiment, when a prediction block is generated in a unidirectional or bidirectional direction, a reference block list may be configured differently to generate a prediction block from the corresponding reference picture list. Basically, the reference pictures temporally present before the current picture are managed by being allocated to the list 0 (L0), and the reference pictures existing after the current picture can be allocated to the list 1 (L1) and managed.

The reference pictures existing after the current picture can be allocated when the reference picture list 0 can not be filled up to the reference picture list number 0 of the reference picture list 0. Similarly, when the reference picture list 1 is constructed, if the reference picture list 1 can not satisfy the reference picture list, the reference pictures existing before the current picture can be allocated.

FIG. 6 is a diagram illustrating an example of generating a prediction block in a unidirectional manner in the image encoding and decoding method according to an embodiment of the present invention. Referring to FIG.

Referring to FIG. 6, in the image encoding and decoding method according to the present embodiment, a prediction block can be found from a previously encoded reference picture t-1, t-2. In addition, a current picture t) from the region where coding has already been completed.

That is, in the image encoding and decoding method according to the present embodiment, not only a prediction block is generated from a previously coded picture (t-1, t-2) having a temporally high correlation, but also a prediction block having a high spatial correlation As shown in FIG. Finding such a spatially highly correlated prediction block can correspond to finding a prediction block in a manner of intra prediction. In order to perform block matching from an area where coding is completed in the current picture, the image coding method of the present embodiment can construct a syntax for information related to the prediction candidate in combination with the intra-picture prediction mode.

For example, when n (n is an arbitrary natural number) kinds of intra prediction modes are supported, one mode is added to the intra prediction prediction candidates to support n + 1 modes, and ^2M-1 ?? N + It is possible to encode the prediction mode using M fixed bits satisfying 1 < 2 ^M. Also, it can be implemented to select among candidate groups of probable prediction modes, such as MPV (most probable mode) of HEVC. It is also possible to preferentially encode at a higher stage of prediction mode encoding.

When a prediction block is generated through block matching in the current picture, the image coding method of the present embodiment may form a syntax for information related to mixing with the inter-picture prediction mode. Additional associated prediction mode information may be motion or displacement related information. The motion or motion-related information may include optimal candidate information among various vector candidates, a difference value between an optimal candidate vector and an actual vector, a reference direction, reference picture information, and the like.

FIG. 7 is a diagram illustrating a reference picture list in the image encoding and decoding method according to an embodiment of the present invention. Referring to FIG. FIG. 8 is a diagram illustrating an example of performing inter-picture prediction from a reference picture list in an image encoding and decoding method according to an embodiment of the present invention.

7, a method of encoding an image according to an embodiment of the present invention includes a first reference picture list (L0) and a second reference picture list (L0) for a current block of a current picture (current (t) 1, &lt; / RTI &gt; L1).

Referring to FIGS. 7 and 8, the reference picture list 0 can be composed of reference pictures before the current picture t, where t-1 and t-2 are respectively the first , The temporal distance (t-1), and the second temporal distance (t-2). The reference picture list 1 can be composed of a reference picture after the current picture t and t + 1 and t + 2 respectively represent a first temporal distance t + 1 after the POC of the current picture t ) And a second temporal distance (t + 2).

Although the above-described examples of the reference picture list construction show an example in which the reference picture list is composed of the reference pictures having the difference of the temporal distance (POC standard in this example), the difference in temporal distance between the reference pictures is configured differently You may. That is, the index difference of the reference pictures and the temporal distance difference of the reference pictures may not be proportional. In addition, the list arrangement order may not be constituted based on temporal distance. This can be confirmed in an example of the reference picture list construction to be described later.

The prediction can be performed from the reference picture in the list according to the slice type (I, P, or B). When a prediction block is generated by block matching in the current picture (current (t)), a current picture is added to a reference picture list (reference list 0 and / or reference list 1) .

The current picture t can be added to the reference list 0 or the current picture current t can be added to the reference list 1 as shown in FIG. That is, the reference picture list 0 can be constructed by adding a reference picture which is a temporal distance (t) to the reference picture before the current picture (t), and the reference picture list 1 is composed of time A reference picture that is a distance t may be added.

For example, when the reference picture list 0 is constructed, the reference picture prior to the current picture can be assigned to the reference picture list 0, and the current picture t can be allocated. When the reference picture list 1 is constructed, The reference picture can be assigned to the reference picture list 1 and then the current picture t can be allocated. Alternatively, when the reference picture list 0 is constructed, the current picture t can be allocated and the reference picture prior to the current picture can be allocated. When the reference picture list 1 is constructed, the current picture t is allocated, It is possible to assign subsequent reference pictures.

Also, when the reference picture list 0 is constructed, the reference picture before the current picture can be allocated, the reference picture after the current picture can be allocated, and the current picture t can be allocated. Similarly, when constructing the reference picture list 1, reference pictures after the current picture can be allocated, reference pictures before the current picture can be allocated, and the current picture t can be allocated. The above examples are not specific to the above-mentioned case, but may include cases where the conditions of each other are reversed, and examples of other cases may also be modified.

Whether or not to include the current picture in each reference picture list (for example, not adding to any list, adding to list 0 only, or adding to list 1 or adding to lists 0 and 1) Information can be transmitted in units of sequences, pictures, slices, and the like. The information on this can be encoded by a method such as fixed-length binarization, truncation-type binarization, truncation-type binarization, and the like.

The image encoding and decoding method of this embodiment differs from the method of FIG. 7 in that block matching is performed on the current picture t to select a prediction block, and a reference picture list including related information about the prediction block is constructed , There is a difference in that such a reference picture list is used for image coding and decoding.

In the reference picture list configuration, it is possible to set the order and the rule of each list and the allowable number of the reference pictures of the respective lists differently. This includes whether the current picture is included in the list (whether the current picture is included as a reference picture in inter- (E.g., whether or not it is applicable to the list 0 or 1), a slice type, a list reconstruction parameter (which may be applied to lists 0 and 1 respectively, or may be applied to lists 0 and 1 respectively), a position in a group of pictures (GOP) At least one of the factors, or a combination thereof, and can explicitly transmit the related information in units of a sequence, picture, or the like.

For example, in the case of a P slice, the reference picture list 0 can follow the list construction rule A, regardless of whether the current picture is included in the list, and the reference picture list 0 that includes the current picture in the list in the case of B slice, Rule B and the reference picture list 1 can follow the list construction rule C, and the list construction rule D for the reference picture list 0 that does not include the current picture and the list construction rule E for the reference picture list 1, B and D, C and E may be the same.

The list configuration rules may be configured in the same or modified manner as described in the above-described reference picture list configuration example. As another example, when the current picture is included in the list, the first reference picture allowable number can be set, and when not included, the second reference picture allowable number can be set. The first reference picture allowable number and the second reference picture allowable number may be equal to or different from each other and the difference between the first reference picture allowable number and the second reference picture allowable number may be set to 1 by default.

As another example, if the current picture is included in the list and the list reconstruction parameter is applied, all the reference pictures in the slice A can be the list reconstruction candidates, and the slice B can include only some reference pictures in the list reconstruction candidate group. At this time, the slice A or B can be classified into whether or not the current picture is included in the list, temporal layer information, slice type, position in the GOP, and the like. The POC or reference picture index of the reference picture, Direction (before / after the current picture), whether or not the current picture is present, and the like.

According to the above-described structure, the reference block encoded with the inter-picture prediction can be used in the current picture, so that inter-picture prediction can be allowed or utilized even in the moving prediction of the I-slice.

Further, when constructing the reference picture list, the index allocation or the list construction order may be different according to the slice type. In the case of the I-slice, a lower index (for example, idx = 0, 1, 2) is used in the current picture (current (t)) with a higher priority as in the reference picture list structure example, It is possible to reduce the bit amount in the image encoding through the binarization (fixed length binarization, chopping-type binarization, truncation-type binarization, etc.) with the maximum allowable reference picture number C of the reference picture list.

In the case of the P or B slice, when it is judged that the block matching is performed in the current picture and it is judged that it is lower than the probability of selecting the prediction candidate through the reference picture having a different probability of selecting the reference picture of the current block as the prediction candidate, The binarization of various methods of setting the priority for the block matching of the reference picture list to the maximum value using a higher index (for example, idx = C, C-1) The amount of bits in the image encoding can be reduced.

In the above example, the priority setting of the current picture can be configured in the same or modified manner as that described in the reference picture list configuration example. In addition, it is possible to omit the information on the reference picture by not constructing the reference picture list according to the slice type (for example, I slice). For example, it is possible to generate a prediction block through the existing inter-view prediction, but to display the inter-view prediction information with the remainder excluding the reference picture information in the motion information in the inter-picture prediction mode.

The method of performing block matching in the current picture can determine whether or not to support according to the slice type. For example, block matching in the current block is supported by I slices, but not P slices or B slices, or other variations are possible. In addition, the method of supporting block matching in the current picture may determine whether to support in units of pictures, slices, tiles, etc., or may be determined according to the position in the GOP, temporal layer information, or the like. Such setting information may be transmitted in units of a sequence, a picture, a slice, or the like in an image encoding process or a decoder in an encoder.

Further, even when the above setting information or syntax exists in the upper level unit and the setting related operation is turned on, when the same setting information or syntax exists in the lower level unit, the setting information in the lower level unit is changed to the upper level The setting information in the unit can be prioritized. For example, if the same or similar setting information is processed in a sequence, a picture, or a slice, a picture unit may have priority rather than a sequence unit, and a slice unit may have priority rather than a picture unit.

For example, the flag that supports block matching in the current picture in the sequence parameter may be sps_curr_pic_BM_enabled_flag, and the flag that supports block matching in the current picture in the picture parameter may be pps_curr_pic_BM_enabled_flag. If sps_curr_pic_BM_enabled_flag is on and pps_curr_pic_BM_enabled_flag is off, block matching may not be supported in the current picture. It is possible to determine whether or not the current picture is included in the reference picture list 0 configuration according to the flag and determine whether the current picture is included in the reference picture list 1 configuration.

9 is an exemplary diagram for explaining intra prediction in the image encoding method according to an embodiment of the present invention.

Referring to FIG. 9, the intra-frame prediction method according to the present embodiment includes a reference sample padding, a reference sample filtering, an intra predicting, and a boundary filtering. May comprise a series of steps.

The reference pixel filling step may be an example of the reference pixel forming step, the reference pixel filtering step may be executed by the reference pixel filter part, and the intra picture prediction may include a prediction block generating step and a prediction mode coding step, The filtering may be an example for one embodiment of the post-processing filter step.

That is, the intra-picture prediction performed in the image encoding method of this embodiment may include a reference pixel configuration step, a reference pixel filtering step, a prediction block generation step, a prediction mode encoding step and a post-processing filtering step. Depending on various environmental factors such as block size, block type, block location, prediction mode, prediction method, quantization parameter, etc., one or a part of the above processes may be omitted and other processes may be added, May be changed in a different order.

The reference pixel construction step, the reference pixel filtering step, the prediction block generation step, the prediction mode encoding step, and the post-processing filtering step may be implemented by a processor connected to a memory by executing software modules stored in the memory. Therefore, in the following description, for convenience of description, a functional unit generated by a combination of a software module that implements each step and a processor that executes the function, and / or a component that performs the function of the functional unit, A prediction block generation unit, a prediction mode encoding unit, and a post-processing filter unit will be referred to as an execution subject of each step.

More specifically, the reference pixel forming unit forms a reference pixel to be used for predicting the current block through reference pixel filling. If a reference pixel does not exist or is unavailable, reference pixel filling can be used for the reference pixel through a method such as copying a value from the nearest available pixel. A reconstructed picture buffer or a decoded picture buffer (DPB) may be used for copying a value or the like.

That is, the intra prediction uses the reference pixels of the blocks that have been encoded before the current picture to perform the prediction. To this end, adjacent pixels such as left, top, bottom left, top, and top right blocks of the current block are mainly used as reference pixels in the reference pixel configuration step.

However, a candidate block of a neighboring block for the reference pixel is an example of a case in which a coding sequence of a block is followed by a raster scan or a z-scan, such as an inverse z-scan When the scan is used in the coding order scanning method, neighboring pixels such as right, lower right, and lower blocks in addition to the above blocks can also be used as reference pixels.

Further, according to the implementation, additional pixels other than immediately adjacent pixels may be used as a substitute or mixed with an existing reference pixel according to the stepwise configuration of the intra prediction.

In addition, in the case of predicting a mode having a directionality among the modes of intra-picture prediction, it is possible to generate a reference pixel in decimal units through linear interpolation of reference pixels in integer units. The mode for performing the prediction through the reference pixel existing in the integer unit position includes some modes having vertical, horizontal, 45 degrees, and 135 degrees, and the process of generating reference pixels in decimal units for the above prediction modes It may not be necessary.

The reference pixels to be interpolated in the prediction modes having the direction other than the prediction mode are 1/2, 1/4, 1/8, 1/16, 1/32, 1/64, It may have interpolation precision, or it may have accuracy of a multiple of 1/2.

This is because the interpolation accuracy can be determined depending on the number of supported prediction modes or the prediction direction of the prediction mode and the like. A fixed interpolation precision may always be supported in a picture, a slice, a tile, a block, etc., and an adaptive interpolation precision may be supported depending on a block size, a block type, a prediction direction of a supported mode, At this time, the prediction direction of the mode can be expressed by the inclination information or the angle information of the direction indicated by the mode on a specific line reference (for example, x + axis of the positive + on the coordinate plane).

As an interpolation method, linear interpolation can be performed through adjacent integer pixels, but other interpolation methods can be supported. For interpolation, one or more filter types and the number of taps can be supported. For example, a 6-tap Wiener filter, an 8-tap Kalman filter, and so on can be determined. have. Further, the related information may be transmitted in units of a sequence, picture, slice, block, or the like.

The reference pixel filter unit can filter the reference pixels for the purpose of improving the prediction efficiency by reducing the deterioration in the encoding process after the reference pixels are constructed. The reference pixel filter unit can implicitly or explicitly determine the type of the filter and the application of the filtering according to the size, type, and prediction mode of the block. That is, even if the filter is the same tap, the filter coefficient can be determined differently depending on the filter type. For example, you can use a 3-tap filter such as [1,2,1] / 4, [1,6,1] / 8.

In addition, the reference pixel filter unit can determine whether to apply filtering by determining whether to send additional bits or not. For example, in an implied case, the reference pixel filter unit can determine whether to apply filtering according to the characteristics (dispersion, standard deviation, and the like) of the pixels in the peripheral reference block.

In addition, the reference pixel filter unit may determine whether or not the filtering is applied when the related flag satisfies a predetermined hiding condition such as a residual coefficient, an intra prediction mode, and the like. The number of taps of the filter is, for example, 3-tap such as [1,2,1] / 4 in a small block (blk), [2,3,6,3,2] / 16 in a large block The same 5-tap may be set, and the number of times of application may be determined by whether to perform filtering, whether to perform filtering once, or to perform filtering twice.

Also, the reference pixel filter unit can basically apply filtering to the nearest reference pixel of the current block. Additional reference pixels besides the nearest reference pixel may also be considered in the filtering process. For example, filtering may be applied to additional reference pixels by replacing the nearest reference pixel, or filtering may be applied by mixing additional reference pixels with the nearest reference pixel.

The filtering may be applied either fixedly or adaptively, depending on the size of the current block, the size of the neighboring block, the encoding mode of the current block or neighboring block, the block boundary property of the current block and the neighboring block (e.g., A prediction mode or direction of a current block or a neighboring block, a prediction method of a current block or a neighboring block, a quantization parameter, and the like, or a combination thereof. have. The decision can be made on the encoder / decoder with the same setting (implicit), or on the basis of the coding cost (explicit). Basically, the filter applied is a low pass filter. Depending on various factors described above, the number of filter taps, the filter coefficient, whether or not the filter flag is encoded, and the number of filter application can be determined. Slice, block, or the like, and can transmit the related information to the decoder.

The prediction block generation unit may perform an extrapolation or an extrapolation scheme through a reference pixel in intra-picture prediction, an interpolation scheme such as an average value (DC) or a planar mode of a reference pixel, a copy of a reference pixel ) Prediction block can be generated.

In the case of copying a reference pixel, one reference pixel may be copied to generate one or more prediction pixels, one or more reference pixels may be copied to generate one or more prediction pixels, and the number of copied reference pixels may be copied May be equal to or less than the number of prediction pixels.

In addition, according to the prediction method, it is possible to classify into a directional prediction method and a non-directional prediction method. Specifically, the directional prediction method can be classified into a linear directional method and a curved directional method. The linear direction method uses extrapolation or extrapolation but the pixels of the prediction block are generated through the reference pixels lying on the prediction direction line. The curved directional method uses an extrapolation or an extrapolation method, but the pixels of the prediction block are referred to as a reference Means a scheme in which a partial prediction direction of a pixel is allowed to be changed in consideration of a detailed directionality of a block (for example, edge <Edge>).

In the case of the directional prediction mode in the image encoding and decoding method of the present embodiment, a description will be made mainly on a linear directional method.

In the case of the directional prediction method, the intervals between adjacent prediction modes may be equal or unequal, and this may be determined according to the size or type of the block. For example, when a block with the size and shape of M × N is acquired by the block division, if the M and N are the same, the intervals between the prediction modes may be uniform. If M and N are different The intervals between the prediction modes may be unequal.

As another example, when M is greater than N, modes with vertical orientation may allocate more precise intervals between prediction modes that are close to vertical mode (90 degrees), and wide spacing may be allocated to vertical modes with far prediction mode . When N is greater than M, modes with horizontal orientation may allocate finer intervals between prediction modes close to the horizontal mode (180 degrees), and wide spacing may be allocated to horizontal modes with far prediction mode.

The above examples are not specific to the above-mentioned case, but may include cases where the conditions of each other are reversed, and examples of other cases may also be modified. In this case, the interval between the prediction modes can be calculated based on numerical values indicating the directionality of each mode, and the directionality of the prediction mode can be numerically expressed by the inclination information of the direction or the angle information.

In addition, a prediction block including other methods using spatial correlation other than the above method can be generated. For example, a reference block using an inter prediction scheme such as motion search and compensation can be generated as a prediction block using the current picture as a reference picture.

The prediction block generating step may generate the prediction block using the reference pixel according to the prediction method. That is, it is possible to generate a prediction block through directional prediction or non-directional prediction such as extrapolation, interpolation, copying, and averaging of the existing intra-picture prediction method according to the prediction method, And other additional methods may be used.

The intra-picture prediction scheme may be supported under the same settings of the encoder / decoder, and may be determined according to a slice type, a block size, a block type, and the like. The intra prediction scheme can be supported according to at least one of the above-mentioned prediction schemes or a combination thereof. The intra prediction mode can be configured according to the supported prediction mode. The number of intra prediction modes supported can be determined according to the prediction method, slice type, block size, block type, and the like. The related information can be set and transmitted in units of a sequence, a picture, a slice, and a block.

The prediction mode encoding step for performing the prediction mode encoding can determine the mode with the best encoding cost according to each prediction mode as the prediction mode of the current block in terms of encoding cost.

For example, the prediction mode encoding unit may use one or more neighboring block modes for current block mode prediction for the purpose of reducing prediction mode bits. (MPM) candidates that are likely to be the same as the current block mode, and the modes of neighboring blocks may be included in the above candidate group. For example, the prediction mode of a block such as the upper left corner, the lower left corner, the upper right corner, and the upper right corner of the current block may be included in the above candidate group.

The candidate group of the prediction mode includes the position of the neighboring block, the priority of the neighboring block, the priority in the divided block, the size or type of the neighboring block, the preset specific mode, and the prediction mode of the luminance block And may be configured according to at least one or more factors or combinations thereof, and the related information may be transmitted in units of a sequence, a picture, a slice, a block, or the like.

For example, when a block adjacent to the current block is divided into two or more blocks, it is possible to determine which block mode among the divided blocks is included as a mode prediction candidate of the current block under the same setting of the encoder / decoder have. For example, the left block among the neighboring blocks of the current block (M × M) is divided into three blocks by performing quad tree partitioning in the block division block, and M / 2 × M / 2, M / 4 × M / 4 and M / 4 × M / 4, the prediction mode of M / 2 × M / 2 block may be included as a mode prediction candidate of the current block based on the block size.

As another example, the upper block among the neighboring blocks of the current block (NxN) is divided into three blocks by performing the binary tree division in the block division block, and N / 4 × N and N / 4 N × N blocks, the prediction mode of the first N / 4 × N block from the left according to a predetermined order (priority assigned from left to right) is referred to as a mode prediction candidate of the current block As shown in FIG.

As another example, when the prediction mode of the block adjacent to the current block is the directional prediction mode, the prediction mode adjacent to the prediction direction of the corresponding mode (the tilt information of the direction of the mode or the angle information side) . In addition, predetermined modes (planar, DC, vertical, horizontal, etc.) may be preferentially included according to the prediction mode configuration or combination of neighboring blocks.

In addition, a prediction mode having a high occurrence frequency among the prediction modes of neighboring blocks can be preferentially included. The priority may be a possibility to be allocated not only to the possibility of being included in the mode prediction candidate group of the current block but also a higher priority or index in the candidate group configuration (that is, a higher probability of allocating fewer bits in the binarization process) have.

As another example, the maximum value of the mode prediction candidate group of the current block is k, the left block is composed of m blocks whose length is shorter than the vertical length of the current block, and the upper block is composed of n blocks (M + n) of neighboring blocks is greater than k, it is possible to fill the candidate group according to a preset order (left to right, from top to bottom), and when the divided block sum (m + (e.g., lower left, upper left, upper right, etc.) other than the neighboring block position in the prediction mode of the neighboring block (left block, upper block) May also be included in the mode prediction candidate group of the current block. The above examples are not specific to the above-mentioned case, but may include cases where the conditions of each other are reversed, and examples of other cases may also be modified.

As described above, the candidate block for predicting the mode of the current block is not limited to a specific block position, and prediction mode information can be utilized from at least one block among left, upper left, lower left, upper and upper right blocks, The prediction mode of the current block can be configured as a candidate group considering various factors as in the above example.

The prediction mode encoding unit can classify the candidate MPM candidate group (referred to as the candidate group 1 in this example) and the mode candidate group (referred to as the candidate group 2 in this example) having a high probability of being the same as the current block mode, The prediction mode encoding process may be changed depending on which candidate group among the candidate groups the prediction mode of the block belongs to.

The total prediction mode may be composed of the sum of the prediction mode of the candidate group 1 and the prediction mode of the candidate group 2. The number of prediction modes of the candidate group 1 and the prediction group of the candidate group 2 may be the sum of the total number of prediction modes, The shape of the block, and the like, or a combination thereof. The same binarization may be applied or another binarization may be applied according to the candidate group.

For example, fixed length binarization may be applied to candidate group 1, and truncation type binarization may be applied to candidate group 2. Although the number of candidate groups is exemplified in the above description, it is assumed that the number of the candidate groups is two, but the number of the candidate groups is not limited to the first candidate group having a high probability of being the same as the current block, the second candidate group having a high probability of being the same as that of the current block, It is expandable, and its variants are also possible.

The post-processing filtering step executed by the post-processing filter unit may include a step of performing a post-processing filtering step of performing a post- May be replaced with a value generated by filtering one or more reference pixels adjacent to the boundary and one or more prediction pixels, and a value obtained by quantifying characteristics between reference pixels adjacent to the boundary of the block (for example, Gradient information, etc.) may be applied to the filtering process to replace the predictive pixel with the generated value, and another method having a similar purpose (correcting a part of the predictive pixel of the predictive block through the reference pixel) .

In the post-processing filter unit, the kind of the filter and whether or not the filtering is applied can be implicitly or explicitly determined. The reference pixel used in the post-processing filter unit, the position and the number of the current pixel, And can be set in an encoder / decoder, and related information can be transmitted in units of a sequence, a picture, a slice, and the like.

Further, in the post-processing filtering step, an additional post-processing process can be performed after generating a prediction block such as boundary filtering. In addition, the reconstructed current block, which is obtained by adding the residual signal and the prediction signal obtained through the conversion / quantization process after the acquisition of the residual signal and the inverse process, is subjected to post-processing filtering considering the characteristics of the pixels of the reference block, .

Finally, the prediction block is selected or obtained through the above-described process. Information obtained in this process may include prediction mode related information, and after the prediction block is acquired, it is transmitted to the conversion unit 210 for encoding the residual signal .

10 is an exemplary diagram for explaining a prediction principle in a P slice or a B slice in the image encoding method according to an embodiment of the present invention. 11 is an exemplary diagram for explaining a case where interpolation is performed in the image encoding method of FIG.

Referring to FIG. 10, the image encoding method according to the present embodiment may include motion estimation module and interpolation steps. Information on the motion vector, the reference picture index, and the reference direction generated in the motion prediction step may be transmitted to the interpolation step. In the motion prediction step and the interpolation step, a value stored in the decoded picture buffer (DPB) can be used.

That is, the image encoding apparatus can perform motion estimation to find a block similar to the current block in the previous encoded pictures. In addition, the image encoding apparatus can perform interpolation of a reference picture for precise prediction with respect to precision in decimal units. Finally, the image coding apparatus acquires a prediction block through a predictor. The information generated in this process includes a motion vector, a reference picture index, a reference direction, ), And the residual signal coding can be proceeded.

In the present embodiment, in-picture prediction is performed in P-slice or B-slice, so that it is possible to implement a combination scheme as shown in FIG. 11 that supports inter-picture prediction and intra-picture prediction.

As shown in FIG. 11, the image encoding method according to the present embodiment includes reference sample padding, reference sample filtering, intra prediction, boundary filtering, Motion estimation, and interpolation. &Lt; RTI ID = 0.0 &gt;

When the image encoding apparatus supports block matching in the current picture, the prediction method in the I-slice can be implemented by the configuration shown in FIG. 11 instead of the configuration shown in FIG. That is, the image encoding apparatus can use information such as a motion vector, a reference picture index, and a reference direction, which are generated not only in the prediction mode in the I-slice but also in the P-slice or B-slice, for generating the prediction block. However, information that can be partially omitted may exist due to the characteristic that the reference picture is current. For example, when the reference picture is the current picture, the reference picture index and the reference direction can be omitted.

In addition, when interpolation is applied, the image encoding apparatus may not be required to perform block matching up to a decimal unit due to the characteristics of an artificial image such as a computer graphic due to characteristics of an image. And a unit such as a sequence, a picture, and a slice can be set.

For example, the image encoding apparatus may not perform interpolation of reference pictures used for inter-view prediction according to the setting of the encoder, and may perform various settings such as not performing interpolation only when block matching is performed in the current picture . That is, the image encoding apparatus of the present embodiment can set whether interpolation of reference pictures is performed or not. At this time, it is possible to determine whether to perform interpolation on all reference pictures or some reference pictures constituting the reference picture list.

For example, the image encoding apparatus does not perform interpolation when it is unnecessary to perform block matching on a fractional unit because the characteristic of an image in which a reference block exists in an existing block is an artificial image, Lt; RTI ID = 0.0 &gt; interpolation. &Lt; / RTI &gt;

In addition, the image encoding apparatus can set whether or not block matching is applied in a reference picture interpolated on a block-by-block basis. For example, when a natural image and an artificial image are mixed, interpolation is performed on a reference picture, but when an optimal motion vector can be obtained by searching a part of an artificial image, a predetermined unit (here assumed to be an integer unit) A motion vector can be represented. In addition, when an optimal motion vector can be obtained by selectively searching a portion of a natural image, a motion vector can be expressed by another constant unit (here, 1/4 unit is assumed).

FIG. 12 is a diagram for explaining a main procedure of an image coding method according to an embodiment of the present invention with a syntax in a coding unit.

Referring to FIG. 12, curr_pic_BM_enabled_flag denotes a flag that allows block matching in the current picture. The curr_pic_BM_enabled_flag can be defined and transmitted in units of a sequence and a picture. At this time, a process of generating a prediction block by performing block matching in the current picture, It can mean the case where it operates through inter prediction. It can be assumed that cu_skip_flag, which is an inter-screen technique that does not code the residual signal, is a flag that is supported only by the P slice or the B slice excluding the I slice. In this case, when curr_pic_BM_enabled_flag is turned on, block slicing (BM) can be supported in the inter-picture prediction mode also in the I-slice.

The image encoding method according to the present embodiment can support skipping in the case of generating a prediction block through block matching on the current picture and can support skipping in the case of an in-screen technique other than block matching. It may not support skipping on I slices depending on the condition. This skipping may be determined according to the encoder setting.

For example, when the I-slice supports skipping, the predicted block may be directly connected to prediction_unit () through a specific flag if (cu_skip_flag) to restore the prediction block to the restoring block through block matching without coding the residual signal . In addition, the image encoding apparatus classifies the method of using the prediction block through block matching in the current picture into the inter-picture prediction technique, and can process the classification through pred_mode_flag, which is a specific flag.

In addition, if the pred_mode_flag is 0, the image encoding apparatus according to the present embodiment sets the prediction mode to the inter-picture prediction mode (MODE_INTER), and if the pred_mode_flag is 1, it can set the intra prediction mode (MODE_INTRA). This is an in-screen technology similar to the existing one, but it can be classified as an inter-screen technique or an in-screen technique in an I-slice to distinguish it from an existing structure. In other words, the image encoding apparatus of the present embodiment can use a temporal correlation structure without using temporal correlation in the I-slice. part_mode indicates information on the size and type of the block divided in the encoding unit.

13 is a diagram illustrating an example of supporting a symmetric type partition or an asymmetric type partition as in inter picture prediction when a prediction block is generated through block matching in the current picture used in FIG. Fig.

Referring to FIG. 13, in the case of generating a prediction block through block matching in the current picture, the image encoding method according to the present exemplary embodiment may use 2N × 2N, 2N × N, N × 2N, N, or support asymmetric partitioning such as nL x 2N, nR x 2N, 2N x nU, and 2N x nD.

FIG. 14 is an exemplary diagram for explaining that 2N × 2N and N × N can be supported in inter-picture prediction (Inter) as in intra-picture prediction (Intra) in FIG. It is possible to determine various block sizes and types according to the division method of the block dividing unit.

Referring to FIG. 14, the image encoding method according to the present embodiment can support 2N × 2N and N × N as well as a prediction block form used for intra-picture prediction. This is an example in which a square shape is supported through a quadtree division method in a block division unit or a division method according to a predetermined block candidate group and the like. In the intra-picture prediction, a binary tree division method or a rectangular block division method Other types of blocks can be added by adding a type, and the setting for this can be set in the encoder.

In addition, it is determined whether to skip only when block matching is performed on the current picture, skip to the existing intra-picture prediction, or skip to the new intra-picture prediction in other cases (else) The encoder can be set to apply. Information on this can be transmitted in units of a sequence, a picture, a slice, and the like.

The subtraction unit 205 (see FIG. 2) can generate the residual block by subtracting the pixel values of the prediction block generated from the prediction unit 200 from the pixel values of the current block to be encoded, thereby deriving the pixel difference value.

The transforming unit 210 (see FIG. 2) receives the residual block, which is a difference between the current block and a prediction block generated through intra-picture prediction or inter-picture prediction, in the subtracting unit 205 and converts the residual block into a frequency domain. Through the conversion process, each pixel of the residual block corresponds to the transform coefficient of the transform block. The size and shape of the transform block may be the same or smaller than the encoding unit. In addition, the size and shape of the conversion block may be the same as or smaller than the prediction unit. The image encoding apparatus can perform conversion processing by grouping various prediction units.

The size or type of the transform block can be determined through a block divider and can support square or rectangular transform according to the block division. The block dividing operation can be affected according to the conversion related setting (size, type, etc. of the supported conversion block) supported by the encoder / decoder.

The size and the shape of each transform block are determined according to the cost of coding the candidates of the size and the shape of the transform block and the division information such as the determined image data of each transform block and the determined size and type of each transform block can be encoded .

The transform can be transformed by a one-dimensional transform matrix. For example, discrete cosine transforms (DCT), discrete cosine transforms (DST), and horizontal and vertical units may be adaptively used for each transformation matrix. The adaptive use may include, for example, determining based on various factors such as the size of the block, the type of the block, the type of the block (luminance / chrominance), the encoding mode, the prediction mode information, the quantization parameter, have.

For example, in the case of intra-picture prediction, when the prediction mode is horizontal, a DCT-based transformation matrix may be used in the vertical direction, and a DST-based transformation matrix may be used in the horizontal direction. If the prediction mode is vertical, a DCT-based transformation matrix may be used in the horizontal direction and a DST-based transformation matrix may be used in the vertical direction.

The transformation matrix is not limited to that shown in the above description. Information on this may be determined using an implicit or explicit method, and may include one or more of the factors such as the size of the block, the type of the block, the encoding mode, the prediction mode, the quantization parameter, the encoding information of the neighboring block, And the related information may be transmitted in units of a sequence, a picture, a slice, a block, or the like.

Here, considering the case of using the explicit method, when two or more transformation matrices for the horizontal and vertical directions are used as candidate groups, information on which transformation matrix is used for each direction may be sent, It is also possible to transmit information on which conversion matrix is used in the horizontal and vertical directions by putting two or more pairs as a candidate group by grouping them into one pair each of which conversion matrix is used for the vertical direction.

In addition, the partial conversion or the entire conversion can be omitted in consideration of the characteristics of the image. For example, one or both of the horizontal and vertical components may be omitted. Since intra prediction or inter prediction is not performed well, a large difference between the current block and the prediction block, that is, when the residual component is large, the encoding loss due to the conversion can be increased. At least one factor among factors such as an encoding mode, a prediction mode, a block size, a block type, a block type (luminance / chrominance), a quantization parameter, a neighboring block coding information, . This can be expressed using an implicit or explicit method according to the above conditions, and information on this can be transmitted in units of sequence, picture, slice, and the like.

The quantization unit 215 performs quantization of the residual component that is transformed by the transform unit 210. The quantization parameter is determined on a block-by-block basis, and the quantization parameter can be set in units of a sequence, a picture, a slice, and a block.

For example, the quantization unit 215 may predict the current quantization parameters using one or more quantization parameters derived from neighboring blocks such as left, top, top, right top, bottom bottom, etc. of the current block.

If the quantization parameter predicted from the neighboring block does not exist, that is, if the block is located at the boundary of a picture, a slice, or the like, the quantization unit 215 calculates a difference with respect to a basic parameter transmitted in units of a sequence, a picture, Value can be output or transmitted. When the quantization parameter predicted from the neighboring block exists, the differential value may be transmitted using the quantization parameter of the block.

The priority of the block from which the quantization parameter is to be derived may be set in advance or may be transmitted in units of a sequence, a picture, a slice, or the like. The residual block can be quantized through a dead zone uniform threshold quantization (DZUTQ), a quantization weighted matrix, or an improved technique. It can be set to one or more quantization schemes as candidates and can be determined by encoding mode, prediction mode information, and the like.

For example, the quantization unit 215 may set the quantization weight matrix to be applied to inter picture coding, intra picture coding unit, or the like, or may set another weighting matrix according to the intra picture prediction mode. Assuming that the size of the block is equal to the size of the quantization block, the quantization weight matrix may be configured by varying the quantization coefficients for each frequency component position, assuming that the size of the block is MxN. The quantization unit 215 may be one of various existing quantization methods and may be used under the same setting of the encoder / decoder. Information on this can be transmitted in units of a sequence, a picture, a slice, and the like.

The inverse quantization units 220 and 315 and the inverse transform units 225 and 320 shown in FIGS. 2 and 3 may be implemented by reversing the processes in the transform unit 210 and the quantization unit 215 described above. That is, the inverse quantization unit 220 can dequantize the quantized transform coefficients generated in the quantization unit 215, and the inverse transform unit 225 can inversely transform the inversely quantized transform coefficients to generate a reconstructed residual block have.

The adders 230 and 324 shown in FIGS. 2 and 3 may generate a restored block by adding the pixel values of the restored residual block to the pixel values of the predicted block generated from the predictor. The reconstruction block may be stored in the encoding and decoding picture buffers 240 and 335 and provided to the prediction unit and the filter unit.

An in-loop filter such as a deblocking filter, a sample adaptive offset (SAO), an ALF (Adaptive Loop Filter), or the like may be applied to the restoring block. The deblocking filter may filter the restoration block to remove distortion between block boundaries that occurs during the encoding and decoding processes. SAO is a filtering process that restores the difference between the original image and the reconstructed image by an offset, on a pixel-by-pixel basis, with respect to the residual block. The ALF can perform filtering to minimize the difference between the prediction block and the restoration block. The ALF can perform filtering based on the comparison value between the restored block and the current block through the deblocking filter.

The entropy encoding unit 245 can entropy-encode the transform coefficients quantized through the quantization unit 215. For example, be implemented using other coding schemes than context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax based context adaptive binary arithmetic coding (SBAC), and probability interval partitioning entropy coding PIPE) coding can be performed.

The entropy encoding unit 245 may include various types of information necessary for decoding the quantized coefficient bit stream and the encoded bit stream in the encoded data. The encoded data may include a coded block type, a quantization coefficient, a bit string in which the quantization block is coded, and information necessary for prediction. In the case of the quantization coefficient, the two-dimensional quantization coefficient can be scanned in one dimension. The quantization factor may vary depending on the characteristics of the image. In particular, in the case of the intra prediction, since the distribution of coefficients can have a specific distribution according to the prediction mode, the scanning method can be set differently.

In addition, the entropy encoding unit 245 can be set differently according to the size of a block to be encoded. The scan pattern can be preset or candidate as at least one of various patterns such as zigzag, diagonal, and raster, and can be determined by the encoding mode, the prediction mode information, and the like, Can be used. Information on this can be transmitted in units of a sequence, a picture, a slice, and the like.

The size of a quantized block (hereinafter, a quantization block) input to the entropy encoding unit 245 may be equal to or smaller than the size of the transform block. In addition, the quantization block may be divided into two or more subblocks. In the case of division, the scan pattern in the divided block may be set to be the same as or different from that of the existing quantization block.

For example, when a scan pattern of an existing quantization block is called zigzag, zigzag may be applied to all subblocks, or a zigzag pattern may be applied to a subblock located at the upper left of a block including an average value (DC) , And diagonal patterns can be applied to the other blocks. May also be determined according to the encoding mode, the prediction mode information, and the like.

The start position of the scan pattern in the entropy encoding unit 245 basically starts from the upper left corner, but may start from the upper right corner, the lower right corner, or the lower left corner depending on the characteristics of the image. Information can be transmitted in units of a sequence, a picture, a slice, or the like. As the encoding technique, an entropy encoding technique may be used, but is not limited thereto.

Inversely, the inverse quantization and inverse transform unit 225 of the inverse quantization unit 220 shown in FIGS. 2 and 3 reverses the transformation structure of the quantization and transformation unit 210 of the quantization unit 215 It is possible to combine the basic filter units 235 and 330.

Next, interpolation that can be employed in the image encoding apparatus of the present invention will be briefly described as follows.

In order to improve the accuracy of the prediction through block matching, interpolation is performed with a finer resolution than that of an integer unit. The interpolation method includes a DCT-IF (discrete cosine transform based interpolation filter) technique. For example, a DCT-IF technique is used as an interpolation method in HEVC (high efficiency video coding). For example, pixels are generated in units of 1/2 and 1/4 between integers to interpolate reference pictures, Matching can be performed to generate a prediction block.

Table 1 and Table 2 show the filter coefficients used for the luminance component and the chrominance component, respectively. For the luminance components, 8-tap and 4-tap DCT-IF filters are used for chrominance components. For color difference components, filters can be applied differently according to the color format. In the case of 4: 2: 0 of YCbCr, the filter shown in Table 2 can be applied. In the case of 4: 4: 4, the filter shown in Table 1 and other filters can be applied. : In case of 2, a horizontal 1D 4-tap filter as shown in Table 2 and a vertical 1D 8-tap filter as shown in Table 1 can be applied.

FIG. 15 is a diagram for explaining a process of performing a horizontal 1D filter on pixels located at positions a, b, and c of an image (assuming x) in the image encoding method according to an embodiment of the present invention.

As shown in FIG. 15, a horizontal 1D filter can be applied to a sub-pixel located at a position (assumed to be x) of a, b, c between the first pixel G and a second pixel H adjacent thereto have. The equation is expressed as follows.

f + f2 * F + f3 * G + f4 * H + f5 * I + f6 * J + 32/64

Next, a vertical 1D filter can be applied to the sub-pixels in positions d, h, and n (assuming y). The equation is expressed as follows.

f / mo> 4 + M + f / mo> 5 R + f / mo> 6 + T / mo> 32 + 64 / mo>

A 2D separable filter can be applied to the center sub-pixels e, f, g, i, j, k, p, q and r. For example, in the case of the sub-pixel e, the pixels in the vertical direction with a are interpolated first, and then interpolated using the pixels. Then, a horizontal 1D filter is performed as interpolating a between G and H, and a vertical 1D filter is performed on the sub-pixels resulting from the horizontal 1D filter, thereby obtaining an e value. Similar operations can be performed on color difference signals.

The above description is only a partial explanation of interpolation. Filters other than DCT-IF can also be used, and the number of filters and the number of tabs to be applied per decimal unit can be different. For example, a coefficient or filter coefficient, such as 8-tap Kalman filter for 1/2, 6-tap Wiener filter for 1/4, 2-tap linear filter for 1/8, fixed like DCT-IF, Or may be encoded. As described above, one interpolation filter may be used for a picture, or an interpolation filter which is different for each region according to the characteristics may be used for an image. Alternatively, two or more reference pictures to which a plurality of interpolation filters are applied may be generated, have.

Different filters may be applied according to the encoding information such as the type of the reference picture, the temporal layer, the state of the reference picture (for example, whether it is the current picture) or the like. The above information can be set in units of sequence, picture, slice, etc., and can be transmitted in that unit.

Next, before describing in detail the improved techniques for motion estimation, motion compensation, and motion prediction that can be employed in the image encoding method according to the present embodiment, Is defined as follows.

Motion estimation refers to a process of estimating motion of a current block to be encoded by dividing an image frame into small blocks and estimating which blocks are temporally shifted from previous or later base-delayed frames (reference frames). That is, the motion estimation is a process of finding the block most similar to the target block at the time of coding the current block to be compressed. Block-based motion estimation is a process of estimating the position of a video object or screen processing unit block (macro block, etc.) in time.

The motion compensation includes motion information (motion vector, reference picture index) for the optimal prediction block found in the motion estimation process in order to predict at least a part of the previously encoded reference image to encode the current image and to predict the current image Which means that a prediction block of the current block is generated. That is, the motion compensation is a process of making an error block as a difference between a current block to be encoded and a reference block that is found to be the most similar block.

Motion prediction means finding a motion vector at the time of coding for motion compensation. Skipping, temporal prediction, spatial prediction, and the like are main techniques of motion prediction. Skipping is because the motion of the screen is constant and the size of the motion vector predicted by the image coding apparatus is zero or the residual is small enough If it can be ignored, it means that the coding of the corresponding image block is skipped. Temporal prediction can be mainly used for inter-picture prediction, and spatial prediction or inter-view prediction can be mainly used for intra-picture prediction.

Information derived from inter-picture prediction may include information (unidirectional (L0, L1), bidirectional) for distinguishing reference picture list directions, an index for distinguishing reference pictures in the reference picture list, a motion vector, and the like. Since temporal correlation is used, motion information can be efficiently coded when the motion vector of the current block is the same as or similar to the motion vector of the neighboring block.

16 is an exemplary diagram of a current block and neighboring blocks that can be employed in the image encoding method according to an embodiment of the present invention.

As shown in FIG. 16, the setting of whether or not the candidate group is referenced to the neighboring block of the current block may be determined according to information such as the type of the current picture, the temporal ID, and the like, A sequence, a picture, a slice, or the like.

Here, the reference to the candidate group may be set using only spatially adjacent blocks A, B, C, D, and E, or may be set to be temporally adjacent to spatially adjacent blocks A, B, C, D, May include settings using blocks H, I, J or spatially separated blocks F, G from spatially adjacent blocks A, B, C, D,

The following is a description of the setting of "block matching is applicable to the current picture".

First, in the case of an I picture, for example, a spatially adjacent block may be placed in a priority order and other blocks may be set as a candidate group. For example, the availability of reference blocks can be checked in the order of E → D → C → B → A → H → I → J. Availability is to determine whether or not it is available and can be contrasted with other values that are compared with a preset reference value or are availability checked. The availability can be determined by the encoding mode of the candidate block, the motion information, the position of the candidate block, and the like. The motion information may include a motion vector, a reference direction, a reference picture index, and the like.

Since the current picture is an I picture, motion information exists only when the coding mode is intra prediction (hereinafter referred to as INTER) of the present embodiment. Therefore, when you see it in priority order, it is confirmed that it is INTER first. For example, if n is 3 and E is encoded as INTER, E is excluded from the candidate group and D is checked. If D is coded as INTER, it carries motion information because it performs block matching in the current picture, and adds D to the candidate group based on the motion information. Then n remains two. Then, the image encoding apparatus can check the priority again. So when the last three candidates are filled, we stop looking for candidates.

The availability can be used not only in the encoding mode but also in the case of boudnary of pictures, slices, tiles, and the like. If it is a border, availability is checked as not available. If it is determined that the candidate is the same as or similar to the already filled candidate, the corresponding block is excluded from the candidate and the reference pixel unit checks the availability of the next candidate.

Here, INTER differs from the existing inter-picture prediction (inter). That is, the INTER mode of the present embodiment differs from inter-picture prediction in which a predictive block is generated in a reference picture because a predictive block is generated in the current picture only by utilizing an inter-picture prediction (inter) structure. That is, in the encoding mode of the present embodiment, a method of block matching in the current picture can be classified into INTER mode and intra (same as existing intra).

Hereinafter, the motion vector copy (MVC) and the motion vector prediction (MVP) will be separately described. This is because whether or not the scaling process is included is different.

The motion vector prediction (MVP) will be described first.

P Picture I  B Picture  Occation

The temporal candidates (F, G) will be described in addition to the above-mentioned candidates (A, B, C, D, E, F, G, H, I and J). In this embodiment, it is assumed that candidates are spatially searched, temporally searched, mixed lists are constructed and searched, and fixed candidates are searched in the described order.

First, the candidates are prioritized and the availability is checked accordingly. It is assumed that the number of candidates (n) of the motion vectors is set to 2, and the priority is as described in parentheses.

For example, when searching spatially, you can group them into the following groups.

Group 1_1 = {A, B, C, I, J}, (C? B? A? I? J)

Group 1_2 = {D, E, H}, (D? E? H)

In this embodiment, the group 1_1 of the two groups includes the blocks immediately above, the upper left, and the upper right with respect to the current block, and the group 1_2 includes blocks immediately adjacent to the current block, , And the lower left blocks.

In another embodiment, candidate blocks of motion vectors can be spatially searched by dividing the motion vectors into three groups. The three groups can be categorized as follows.

Group 1_1 = {A, B, C}, (C? B? A)

Group 1_2 = {D, E}, (D? E)

Group 1_3 = {H, I, J}, (J? I? H)

In the present embodiment, the group 1_1 includes blocks immediately adjacent to the current block, adjacent upper left, and upper right adjacent to the current block. Group 1_2 includes blocks immediately adjacent to the immediately adjacent block on the basis of the current block And the group 1_3 includes the current block and non-contiguous blocks having one or more block intervals.

In another embodiment, the candidate blocks of the motion vector can be spatially searched by dividing the motion vectors into three groups in another manner. The three groups can be categorized as follows.

Group 1_1 = {B}

Group 1_2 = {D}

Group 1_3 = {A, C, E}, (E? C? A)

In this embodiment, the group 1_1 includes blocks located in the vertical direction with respect to the current block, the group 1_2 includes adjacent blocks positioned in the horizontal direction with respect to the current block, And includes the remaining adjacent blocks with respect to the block.

As discussed above, since there are many referenceable information such as a reference direction and a reference picture in the P picture or the B picture, the candidate group can be set accordingly. A candidate block having a current block different from that of a current block may be included in a candidate block having a different reference block from the current block or a picture of count (POC) between a reference block of a current block and a reference block of a candidate block The vector of the corresponding block may be scaled and added to the candidate group. It is also possible to add a scaled candidate group according to which picture the reference picture of the current block is. When the temporal distance between the reference picture of the current block and the reference picture of the candidate block exceeds a predetermined distance, the candidate group is excluded. If the temporal distance is less than the predetermined distance, the scaled block may be included in the candidate group.

The similarity check described above is a process of comparing and determining how similar a motion vector already included in a prediction candidate group is and a motion vector to be newly added. Depending on the definition, it can be set to be true when the x and y components are perfectly matched, or to be true when there is a difference below the theshold value range.

In this embodiment, although the condition that the reference picture indicates the current picture is taken as an example, the reference picture can be expanded to the case where the current picture is not the current picture. For example, you can use a setting such as 'Exclude blocks that use pictures that are farther than the current block'. In the present embodiment, even if the current block and the reference picture are different, they can be included in the candidate group through scaling.

Mixed list

It is assumed that bidirectional prediction of the current block is performed and each motion information is present in a reference picture existing in the reference picture list (L0, L1). And checks the availability according to the priority order of the preset candidates. In this case, the priority is first confirmed as being encoded in bidirectional prediction.

If each reference picture is different, scaling is performed. In the case where only the bidirectional predicted blocks are included in the candidate group and the maximum number of candidate blocks is not exceeded, the blocks coded by unidirectional prediction among the candidate blocks previously performed are put into the preliminary candidate groups, You can create a candidate for bidirectional prediction.

First, it is assumed that the motion information of the bidirectional prediction of the current block is referred to in the reference picture at L0 and at the reference picture at L1. In the case of Table 5 (a), the motion information of the first candidate block is (mvA ₁ , ref1), (mvA ₂ , ref0), and the motion information of the second candidate block is mvB ₁ ', ref1 ) And (mvB ₂ ', ref0). Where the apostrophe (') is the scaled vector. Assuming that the number of candidates after the spatial and temporal search is two, if n is 5, the unidirectionally predicted blocks can be put into the preliminary candidates according to the preset priority in the preceding step.

In Table 3 (a), since the maximum number of candidates has not been filled yet, new candidates can be added by combining scaled unidirectional candidates using the remaining motion vectors mvC, mvD, and mvE.

In Table 3 (b), the motion information of the unidirectionally predicted block is scaled according to the reference picture of the current block. Here, an example of creating a new combination with unidirectional candidates is presented, but it is possible to combine new candidates with motion information of each of the bidirectional reference pictures L0 and L1 already added. This part is not performed in situations such as unidirectional prediction. It is also not performed when the reference picture of the current block is the current picture.

Constant candidate

If a candidate block of n maximum candidates (assumed to be 2 in this embodiment) can not be constructed through the above process, a constant candidate having a preset fixed coordinate may be added. (0, 0), (-a, 0), (-2 * a, 0), (0, -b), and the number of fixed candidates Can be set.

The above fixed coordinates can be set. Through the above process, it is possible to add as a fixed candidate through processes such as average, weighted average, and median value of at least two motion vectors included in the candidate group. If n is 5 and thus far three are registered as candidates {(mvA_x, mvA_y), (mvB_x, mvB_y), (mvC_x, mvC_y)}, the fixed candidates having a predetermined priority to fill the remaining two candidates You can add a fixed candidate based on the priority of the candidate to be included. (MvA_x + mvB_x) / 2, (mvA_y + mvB_y) / 2), ((mvA_x + mvB_x + mvC_x) / 3, (mvA_y + mvB_y + mvC_y) / 3) , mvB_x, mvC_x), median (mvA_y, mvB_y, mvC_y)), and the like.

In addition, the fixed candidate can be set differently according to the reference picture of the current block. For example, when the current picture is a reference picture, fixed candidates may be set as (-a, 0), (0, -b), (-2 * a, 0), and when the current picture is not a reference picture (0,0), (-a, 0), (average (mvA_x, ...), average (mvA_y, ...)). The information can be preset in an encoder or a decoder, or can be transmitted in units of a sequence, picture, slice, or the like.

Hereinafter, the motion vector copy (MVC) will be described in more detail.

P Picture I  B In picture  Explanation for

In this embodiment, it is assumed that temporal candidates (F, G) are also included. The candidate group includes A, B, C, D, E, F, G, H, I, Although the search order is not fixed, it is assumed that the MVC candidate is searched spatially, temporally searched, a mixed list is searched, and a constant candidate is added.

In other words, the above-described portion arbitrarily sets the search order like this, and does not mean that a predetermined order is used. Priority is set and availability is checked accordingly. Let n be 5, and the priority is assumed to be the same as in parentheses.

In the following description, only differences in the motion vector prediction (MVP) will be described. The MVP part can be written with the following content, with the exception of the part for scaling in the previous section. Spatial candidates can be checked for availability without omitting the scaling process. However, similar to MVC, the type of the reference picture, the distance between the current picture and the reference picture of the current block, and the like may be excluded from the candidate group.

If there is a mixed list, candidates for bidirectional prediction can be created with a combination of the candidates added up to now as shown in Table 4 below.

As shown in Table 4 (a), a candidate using the reference list LO and a candidate using the reference list L1 can be combined to add a new candidate to the motion vector candidate group. If it is not possible to fill five motion vectors in a predetermined number, as shown in Table 4 (b), the next candidate of L0 and the candidate using L1 may be combined and added to the candidate.

MVP, MVC, and the like, which find candidates of the optimal motion information as described above.

In the case of the skip mode, encoding can be performed using MVC. That is, information on the optimal motion vector candidate can be encoded after the skip flag process. If there is one candidate, this part can be omitted. It is possible to encode residual components, which are the difference values between the current block and the prediction block, through a process such as transformation and quantization without encoding the motion vector difference values separately.

If it is not a skip, it is first checked whether to process motion information through MVC with a priority order, and if it is correct, information about a candidate group of an optimal motion vector can be encoded. If MVP does not handle motion information, MVP can handle motion information. In the case of MVP, information on the optimal motion vector candidates can be encoded. Here, if there is one candidate, motion information processing can be omitted. Then, information such as a difference value with respect to a motion vector of the current block, a reference direction, a reference picture index, and the like can be coded and the residual component can be obtained and then encoded through transformation and quantization.

The subsequent codec such as entropy and post-processing filtering is omitted in order to avoid redundancy with the above description.

On the other hand, in the method of encoding or decoding video data, data-level parallelization divides data to be processed by a parallelizing program into a plurality of units and then assigns the divided data to different cores or threads to perform the same operations in parallel It is a way to do it. Theoretically, parallelism is one of the important factors for performance because the image processing speed becomes faster as parallelism is made within the performance limits of core or thread.

Parallelization of frame, slice, and block units is often used as a data unit of such parallel processing. The following description will be made on the assumption that inter-picture prediction is performed by such parallel processing.

17 is an exemplary diagram for explaining generation of a prediction block through block matching in a current block in an image encoding method according to an embodiment of the present invention. FIG. 18 is an exemplary diagram for explaining reference blocks in an already encoded region used as a prediction block through at least one block matching in the image encoding method according to an embodiment of the present invention. FIG.

The present embodiment can be applied to a case where block matching is performed in the current picture, and coding is performed through the inter picture coding structure.

In FIG. 17, the middle four blocks can be referred to as current blocks B_t to be encoded, and the arrow indicates a reference block, indicating that the corresponding reference blocks exist in the current picture.

Since the in-loop filter such as deblocking filter and SAO is still applied in the case of the already coded area before the current block, a problem may arise from the viewpoint of parallel processing when performing block matching in the current picture. As such, additional memory may be required before filtering for block matching is applied.

Also, in the case of the already coded region, since the in-loop filter is applied, it may be difficult to generate an optimal prediction block due to deterioration between blocks.

According to the present embodiment, additional memory may be required when referring to a memory for the current picture and another block in the current picture in the decoded picture buffer DPB. In this case, the additional memory may be another memory for the current picture in addition to the existing memory for the current picture, and may be temporary memory that can be used in the encoding or decoding of the current picture. In addition, a very large capacity of additional memory may be required for parallel processing.

In addition, if an in-loop filter such as a deblocking filter and SAO is applied, it may be a memory for a current picture, or may be an additional memory if the in-loop filter is not applied. Further, the additional memory may not be required in accordance with the in-loop filter operation setting in units of a sequence parameter set, a picture parameter set, a slice header, and the like. For example, the additional memory may not be needed for an in-loop filter operation setting such as a deblocking filter off, a SAO off, and an ALF off, Some or all of the above configurations may be included and configured.

As described above, the reference block (see reference character rfb in Fig. 18) referenced to another block in the current picture can be stored in an additional memory. This leads to additional memory loss and needs to be avoided. Hereinafter, additional memory prevention through adaptive filtering will be described.

19 is a block diagram for explaining an image encoding apparatus according to an embodiment of the present invention.

19, the image encoding apparatus according to the present embodiment includes a predictor 200, an adder 205, a transform unit 210, a quantization unit 215, an inverse quantization unit a quantization unit 220, an inverse transform unit 225, a subtractor 230, a decoded picture buffer (DPB) 240, an entropy encoder 245, a filtering skip confirmation unit 250, (260), and a filter unit (290).

Here, the prediction unit 200 may include a first prediction unit 201 for intra prediction and a second prediction unit 202 for inter prediction. The first prediction unit may be referred to as an intra prediction unit, and the second prediction unit may be referred to as an inter prediction unit. The addition section 205 and the subtraction section 230 may be referred to as a first addition / subtraction section and a second addition / subtraction section, respectively.

The skip selecting circuit 260 is provided between the filtering skip confirmation unit 250 and the filter unit 290 and between the filtering skip confirmation unit 250 and the filter unit 290, Is located between the filtering skip confirmation unit (250) and the decoding picture buffer (240). The filtering skip confirmation unit 250 may adaptively perform in-loop filtering by controlling the skip selection circuit 260 based on the selection information by the filtering skip flag.

The filter unit 290 may be referred to as an in-loop filter unit and may include at least one of the configurations of the deblocking filter 270, the SAO 280, and the in-loop filter unit such as the ALF. The filter unit 290 may perform filtering on the reconstructed image.

The deblocking filter 270 will be described in more detail as follows. That is, a quantization error may occur in the quantization step during the prediction, conversion, quantization, and entropy encoding processes. This is adjusted by the quantization parameter value. If the quantization parameter value is small, dense quantization is performed on the transform coefficient so that the quantization error is relatively small, and if the quantization parameter value is large, the quantization error may be relatively large. In order to solve such a problem, it is possible to reduce image deterioration by performing filtering on the restored picture. For this, a post-processing filter or an in-loop filter method using a deblocking filter can be applied to the image encoding method and apparatus according to the present embodiment. The in-loop filter can function similar to the HEVC's basic codec and so on.

As shown in FIG. 19, the filtering skip confirmation unit 250 is further configured. The filtering skip confirmation unit 250 determines whether or not the filter unit 290 is applied and skipped The skin selection circuit 260 can be controlled / switched to be selected. Accordingly, parallel processing can be performed using one DPB 240 without using additional memory. The setting information for this may be the same in the encoder and the decoder, and may be transmitted in units of a sequence, picture, slice, or the like between the encoder and the decoder.

20 is a block diagram for explaining an image decoding apparatus according to an embodiment of the present invention.

Referring to FIG. 20, an image decoding apparatus according to an embodiment of the present invention includes an entropy encoder 305, an inverse quantization unit 315, an inverse transform unit 320, A prediction unit 310, a filter unit 330, a DPB 335a, a filtering skip confirmation unit 350, and a skip selection circuit 360. [

The prediction unit 310 may include an intra prediction unit 311 for intra prediction and an inter prediction unit 312 for inter prediction. The filter unit 330 may include a deblocking filter 370 and a SAO 380.

The skip selecting circuit 360 is located between the filtering skip verifying unit 350 and the filter unit 330 and between the filtering skip verifying unit 350 and the filtering skip verifying unit 350. [ 350 and the decoded picture buffer 335a. The filtering skip confirmation unit 350 may adaptively perform the deblocking filtering according to the coding block, the prediction block, or the conversion block by controlling the skip selection circuit 360 based on the selection information by the filtering skip flag.

That is, in the decoder of the present embodiment, it is possible to instruct a switching operation on whether or not to skip the filtering based on the flag transmitted from the encoder. If the filtering is skipped, the corresponding data is directly stored in the DPB 335a. Otherwise, the filtered data is output to the display device or the like to be displayed or stored in the DPB 335a. In other words, the image decoding apparatus can obtain the reconstructed picture, apply filtering, and then output it to the playback apparatus, and store the reconstructed picture in the DPB 335a so that the reconstructed picture can be used as a reference picture in future inter-picture prediction.

21 is an exemplary diagram for explaining transmission in units of blocks of various sizes in an image encoding method according to an embodiment of the present invention.

The flags transmitted for the above-described filtering may be transmitted in block units of various sizes. For example, as shown in Fig. 21 (a), when the maximum encoding unit is 64 x 64, the maximum encoding unit can be transmitted. The maximum coding unit may correspond to a coding tree unit (CTU).

In addition, as shown in Fig. 21 (b), the maximum encoding unit can be transmitted as a divided encoding unit. Tree-based partitioning is possible when flags are created in split units. In this embodiment, it is shown that the maximum encoding unit may be divided into four, or at least one of the four encoding units may be divided into four again.

In addition, as shown in Fig. 21 (c), one or more of the maximum encoding units can be transmitted in bundle units. In this embodiment, it is shown that four maximum encoding units can be transmitted in one bundle unit.

As shown in Fig. 21 (d), it is also possible to combine transmission of one or more of the maximum encoding units in units of a bundle and transmission of the maximum encoding units in a plurality of divisional encoding units.

In the case of FIG. 21, although the quad tree partitioning is exemplified in the partitioning method, it can be divided into various block sizes and types according to the partitioning method in the partitioning part.

Information on the size of the minimum block from the maximum block size can be set equally in the encoder and the decoder, and the information related thereto can be transmitted in units of a sequence, a picture, a slice, and the like.

22 is an exemplary diagram for explaining an example of flag transmission in the image encoding method according to an embodiment of the present invention.

Referring to FIG. 22, a flag for adaptive filtering may be transmitted in the image encoding method or the image decoding method according to the present embodiment.

For example, if at least one sub-block is referred to, that is, if there is at least one block to which the in-loop filter is not applied, the sub-block is divided. Off (on / off) flag can be transmitted. The depth of division of the block can be determined by the size of the maximum block, the picture type, and the like. However, if the supported depth is 0, it may not be divided. The information related thereto can be transmitted in units of a sequence, a picture, a slice, and the like. In this embodiment, the maximum block size is 64 × 64, and the supported depth is supported up to 3 (8 × 8).

In this case, when the related flag is 1, it means that the block is divided. When the related flag is 1, the block is not divided, and the division information can be generated according to the jet scanning direction.

First, 1 (64x64 division) indicating division is transmitted to 64x64 block (uppermost part of the tree structure), and 1 (32x32 division) is transmitted to upper left block (32x32) It is possible to transmit divided information 0 (16 × 16), divided into four blocks (16 × 16), 0 (16 × 16), 0 (16 × 16), and 0 (16 × 16). Subsequently, 0 (32 × 32) is transmitted to the upper right block to transmit no further division, and 0 (32 × 32) can be transmitted to the lower left block. 1 (32x32 division) is transmitted for the lower right block (32x32), and 1 is transmitted for the upper left block (16x16) inside the divided block. In the case of the upper left block (16 × 16), since the size of the divided block (8 × 8) is equal to the supported depth depth (8 × 8), the division flag is not transmitted to the divided block (8 × 8) Do not. 0 (16 × 16), 0 (16 × 16), and 0 (16 × 16) can be transmitted for the remaining blocks (16 × 16). That is, 1100000011000 can be transmitted as instruction data indicating whether or not to divide.

Through the above process, it is possible to generate information for notifying whether or not to divide, and thereafter, information about whether to apply filtering can be generated. 1, 0, 1, 1, 0, 1, 0, 0, 1, 0 can be transmitted in the jet scan direction with respect to the flag indicating whether the filtering is applied to thirteen blocks.

The bitstream structure of the division flag for expressing the flag for the division information of the block may be selected from one or more scanning methods. It is possible to preferentially obtain the additional division information in the block divided on the basis of the first block on the basis of the jet scan as in the above example. Or a jet scan based depth and depth ordering method, the division information at the current level depth depth is obtained on the basis of the initial block, and the division information is obtained at the depth level of the next level. When the above method is used, 1 (64 × 64 division), 1 (32 × 32 division), 0 (32 × 32 division) (16x16), 0 (16x16), 0 (16x16), 0 (16x16), 0 ), And 0 (16 x 16 decision). That is, 1100100001000 can be transmitted as instruction data indicating whether or not to divide. Other scanning methods not shown in the above examples may be included and selected.

In Fig. 22, the circles indicate the division flag, the square indicates the filtering application flag, the hatching indicates that the divided or the filtering is applied, and the colorless means that the division is not applied or the filtering is not applied.

In this embodiment, the data in the already-coded current picture is not yet subjected to the in-loop filter, and there is degradation between the blocks. The memory containing such data may be a temporary memory that is stored during the encoding or decoding of the current picture. When the data with deterioration is referenced through block matching, there may be a disadvantage that the accuracy of prediction is degraded due to deterioration between blocks. In this case, the block deterioration can be reduced by applying filtering that reduces the block deterioration at the left and upper boundaries of the current block before the coding of the block is completed and before proceeding to the next block.

In addition, the right and bottom boundaries can be processed in the block to be coded next. The filtering application can be determined based on predetermined conditions such as the type of the block boundary, the minimum block size, and the like at which block boundary the filtering is applied. Filtering may be performed at the transform block boundary, filtering may be performed at the predict block boundary, or may be performed at a common block boundary between the transform block and the prediction block. The size of the minimum block for which filtering is performed can also be set, and information related thereto can be generated or transmitted in units of a sequence, a picture, a slice, or the like.

In addition, characteristics of a block placed at a block boundary, such as encoding mode, block boundary property, prediction information, encoding coefficient, and the like, can be analyzed to determine whether to apply filtering, a pixel to be filtered, and a filter type. The related setting information may be set equally in the encoder and the decoder, and may be transmitted in units of a sequence, a picture, a slice, or the like. The filtering employed in the above-described embodiment can use the same configuration as that of the existing deblocking filter, and other configurations can also be used.

23 is a flowchart illustrating a filtering procedure in an image encoding method and an image decoding method according to an embodiment of the present invention.

Referring to FIG. 23, the image encoding method or image decoding method according to the present embodiment includes filtering steps.

That is, the filtering steps of the present embodiment may include a block boundary checking step S231, a filtering strength setting step S233, a filtering applying determining step S235, and a filtering applying step S237.

The block boundary checking step S231, the filtering intensity setting step S233, the filtering application determining step S235 and the filtering applying step S237 may be performed in the memory for storing the respective programs, program codes or software modules (See reference numeral 18 in Fig. 1) connected to the memory and executing a program.

That is, the image encoding apparatus or the image decoding apparatus according to the present embodiment may include a processor that executes a program code stored in a neighboring memory or transmitted from a remote memory, or a filtering control unit that is at least a part of or a component of the processor, A filtering application determining unit for implementing a filtering application determining step S235 and a filtering applying step S237 for implementing a filtering step S233, a block boundary checking unit for implementing a checking step S231, a filtering strength setting unit for implementing a filtering strength setting step S233, And a filtering application unit which implements the filtering function.

Hereinafter, the filtering step will be described in detail.

FIG. 24 is an illustration of a prediction block or a transform block that can be employed in the image encoding method of FIG. 23. FIG.

Referring to FIG. 24, a block boundary check unit performing a block boundary check step (S231) in the image encoding / decoding apparatus according to the present embodiment compares the block boundary to which a block boundary is to be applied with a predetermined condition, , The filtering is applied to the boundary, and if not, the filtering may not be applied. The predetermined condition may include setting information such as the boundary of a certain block, the minimum block size, and the like.

The block boundary checker may perform filtering on the boundary between the block where the conversion is performed or the block where the prediction is executed. The setting information for performing filtering at the conversion block boundary, filtering at the prediction block boundary, or both may be set in units of a sequence, a picture, a slice, and the like. It is also possible to transmit the setting information of &quot;

In the block boundary checking step S231, the size (MxN) of the minimum block to be filtered can be set. These settings can also be transmitted in units of sequences, pictures, slices, and the like. It is possible to determine the block size and shape of the in-loop filter (deblocking filter in this embodiment) through the process of the block dividing section. It may also be determined according to the size and shape of other kinds of blocks. For example, the size of the minimum block may be greater than or equal to the smallest prediction unit or conversion unit. The prediction block or transform block may include a size of 64x64, 32x32, 16x16, 8x8, 4x4, etc. and may also include a rectangular shape such as MxN.

In FIG. 24, the block boundaries VEB1, VEB2, VEB3, and VEB4 represent vertical edge boundaries to which filtering is applied. Of course, in FIG. 24, the block boundaries may further include a horizontal edge boundary to which filtering may be applied. In FIG. 24, when the size of the largest block is 64 × 64, the size of the minimum block to be filtered is 16 × 16. In this case, the minimum block size may be set according to the color format of the color difference component or according to the luminance component, and may be set for each color difference component.

In Figure 24, when filtering is applied only to block boundaries at the vertical edge boundaries, the first coding unit (CU # 1) has two prediction units (64 × 64 CUs) PUs), the second coding unit (CU # 2) is divided into one prediction unit (PU) of 64x64 size and the third coding unit (CU # 3) is divided into 64x64 The CU is divided into two prediction units PUs of a size of 32x64 and the fourth coding unit CU # 4 is divided into two prediction units PUs of a size of 32x64 . Of course, when filtering is applied to the horizontal edge boundaries, each coding unit can be divided into more prediction units except for the second coding unit.

25 is an exemplary diagram for explaining a boundary for setting a filtering strength in the image encoding method of FIG.

Referring to FIG. 25, the filtering intensity setting unit for performing the filtering intensity setting step S233 in the image encoding / decoding apparatus of the present embodiment can determine which filter is used by analyzing the characteristics of the blocks located on both sides of the block boundary . That is, the filtering strength setting unit can determine the filtering strength depending on which filter is used.

That is, the filtering strength setting unit can use the information on the coding mode, the block boundary characteristics, the prediction information, the coding coefficients, and the like of the block as the information used for setting the filtering strength. The block boundary property includes information on whether a boundary between blocks is an encoding unit or a conversion unit, and the prediction information may include information on an intra prediction mode, a motion vector, a reference picture, and the like.

The filtering intensity setting unit can determine, based on the above-described information, which intensity is to be filtered or not to be applied to the block boundary. That is, the filtering intensity setting unit can determine the filtering intensity based on the pixel value, but it is not limited to this, and can be determined based on the information of the corresponding block rather than the actual pixel value based.

When the filtering strength is determined based on the pixel value, the pixel unit of the boundary where the filtering strength is set may be at least one row or at least one column. This is because, due to the nature of the block boundary, it can be determined in proportion to the width and length of the minimum filtering applied block. Of course, the filtering strength may be set in units of fixed pixels in a picture, a slice, or the like depending on the implementation. The related information can be set equally in the encoder and the decoder, and information on the information can be transmitted in units of a sequence, a picture, a slice, and the like.

In the case of the chrominance component, as shown in Fig. 25, boundary strength (BS) is obtained through encoding information of two blocks (for example, P block and Q block) on both sides of the block boundary B5 ) Or the filtering strength can be determined. For example, the luminance component of each pixel of the P block, that is, the set {p (3,0), p (2,0), p (1,0), p (0,0) , p (2,1), p (1,1), p (0,1), p (2,2) the filtering intensity at the pixel corresponding to each element of p (3,3), p (2,3), p (1,3), p (0,3)}, Q (2,0), q (0,0), q (3,1), q (2,1), q (1,1) , q (3, 2), q (2, 2), q q (0, 3)} with reference to the filtering strength at the pixel corresponding to each element.

For example, when the size of both blocks is equal to or larger than a certain size, the luminance component is referred to, and if not, the filtering intensity can be set by referring to the color difference component. In the case of chrominance components, the type and number of filtering supported may be the same as or different from the luminance components. For example, in the luminance component, the filtering may be applied at the intensity of 0, 1, 2, and the chrominance component may be applied at the intensity of 0, 1.

26 is an exemplary diagram illustrating a pattern of pixel values in an image in the image encoding / decoding method of Fig.

The filtering application determination unit of the image encoding / decoding apparatus according to the present embodiment determines a filtering intensity setting unit based on various information of both blocks (see blocks on both sides of the block boundary B5 in FIG. 25) After the determination, the filtering strength obtained through the filtering strength setting unit can be determined based on the restored pixel placed on the block boundary.

As shown in FIGS. 26A and 26B, when there is no change or variation between adjacent pixels, since the change between adjacent pixels takes a linear form, the quantization error can be reduced by applying filtering have.

On the other hand, as shown in Fig. 26 (c), when a step edge is formed between adjacent pixels p2,0 and p1,0, the actual edge can be blurred when filtering is applied Therefore, performing the filtering may hinder the encoding efficiency. Therefore, in the present embodiment, it is possible to determine the presence or absence of the filtering application or to determine the filtering application pixel by checking the amount of change with respect to the pixel placed at the boundary of both blocks.

FIG. 27 is a flowchart for explaining a filtering intensity setting process in the image encoding / decoding method of FIG.

It is assumed that the image coding / decoding method according to the present embodiment sets the filtering strength for the vertical boundary. It is assumed that the filtering strength is 0 to 2.

Referring to FIG. 27, the filtering control unit of the image encoding / decoding apparatus implementing the image encoding / decoding method according to the present embodiment may determine whether any of the blocks includes intra-coded blocks (S271). As a result of the determination, if yes, the BS or the filtering strength may be set to 2 (S272). Here, the higher the number, the stronger the filtering strength may be.

If it is determined in step S271 that both of the blocks are coded in Inter, the filtering control unit may determine the filtering strength by comparing the coding coefficients of both blocks. That is, the filtering control unit can determine whether there is one or more non-zero coefficients to be encoded even in one block (P or Q) (S273). If the determination is YES, the filtering controller may set the BS to 1 (S277).

If it is determined in step S273 that there is no coefficient to be coded by the two blocks, the filtering controller can determine whether the reference picture (reference or ref) does not match or is different (S274). If the determination is YES, the filtering controller may set the BS to 1 (S277).

If it is determined in step S274 that the two blocks do not have different reference pictures, the filtering controller may determine whether the number of motion vectors of the two blocks is different from each other (S275). If the determination is YES, the filtering controller may set the BS to 1 (S277).

If it is determined in step S275 that the number of motion vectors of the two blocks is the same, the filtering controller may determine whether there is a difference of at least a predetermined magnitude between the x component and the y component of each motion vector (S276). If the determination is YES, the filtering controller may set the BS to 1 (S277). Assuming that the motion vector precision is 1/4, the meaning of the number 4 in the 1/4 unit can correspond to 1 in the integer unit. Therefore, this boundary strength or boundary value may vary depending on the precision of the motion vector.

For example, when the motion vector precision of one of two blocks is an integer and the motion vector precision of another block is 1/4, the accuracy of a motion vector of a block having an accuracy of an integer unit is changed to 1/4, Step S276 may be performed. When the motion vector precision of one of the two blocks is 1/4 and the accuracy of the motion vector of the other block is 1/8, the difference value setting between the x component and the y component in step S276 is changed from 4 to 8 . In addition, when the motion vector precision of two blocks is an integer, the motion vector precision of the two blocks may be changed to 1/4 and the step S276 may be performed.

On the other hand, if the determination result in step S276 is NO, the filtering control unit may determine that both blocks have high similarity, and may set the BS or the filtering strength to zero. Here, a filtering strength of 0 may mean that no filtering is performed.

In this embodiment, the series of steps S271 to S278 can be modified into various forms. For example, the step of confirming whether another reference picture is used (S274) may be omitted by performing scaling according to the distance of the reference picture. The step of checking the difference between the number of motion vectors (S275) can be variously modified according to the image characteristics. Also, unlike in the step S73, even if a coefficient having a non-zero coding coefficient exists in both blocks, it can be compared or determined under the same conditions as in the case of having a frequency coefficient of a certain size or less. Here, in the case of being located in a specific region of the frequency band, in the case of being located at the upper left of the low-frequency component including DC (direct current) component in the NxN frequency block, <0,0>, <1,0> , 1 &gt;, a non-zero coefficient occurs at a position such as 1 &lt; 1 &gt;, and no other case occurs, and other variations are possible.

28 is a flowchart for explaining another example of the filtering intensity setting process in the image encoding method of FIG.

The present embodiment is an example of a case where a current picture is included as a reference picture and is implemented by an inter picture coding method as compared with the embodiment of FIG. Description of parts overlapping with the embodiment of FIG. 27 is omitted.

Referring to FIG. 28, the filtering control unit determines whether there is an intra-coded block in any of the two blocks P and Q (S271). If the determination result is yes, BS) or the filtering strength is set to 2, and if the determination result is no, it is possible to determine whether any of the two blocks is the current picture (S271a).

If the determination result of step S271a is YES, the filtering control unit may set the BS to 2. If the determination result is NO, the filtering controller may determine whether there is one or more non-zero coefficients to be encoded even in one block (P or Q) (S273). If the determination result of step S273 is YES, the filtering control unit may set the BS to 2. If the determination result of step S273 is NO, the filtering control unit may sequentially execute the steps S274 to S278 of FIG.

As described above, according to the image encoding / decoding method of the present embodiment, the filtering strength can be set differently depending on which picture the reference picture of each block is. On the other hand, in the present embodiment, in the case where both blocks are coded by intra, it is determined whether a current block is used as a reference picture even in one block. However, the present invention is not limited to such a configuration, Do.

29 is a flowchart for explaining another example of the filtering intensity setting process in the image encoding / decoding method of FIG.

Referring to FIG. 29, the filtering control unit of the image encoding / decoding apparatus implementing the image encoding / decoding method according to the present embodiment, when both blocks have the current picture as a reference picture, And to set the intensity to 1 (S276a).

In the filtering process of this embodiment, a step S276a of determining whether both blocks (P & Q) have a current picture as a reference picture (ref) May be substantially the same as the embodiment of FIG. 27, except that it is determined at step S276 whether or not there is a difference of at least a certain size in either one of the blocks.

FIG. 30 is a flowchart for explaining another example of the filtering intensity setting process in the image encoding / decoding method of FIG.

The present embodiment is an example in which a current picture is included as a reference picture and is implemented by an intra-picture coding scheme.

Referring to FIG. 30, the filtering control unit of the image encoding / decoding apparatus implementing the image encoding / decoding method according to the present embodiment may determine whether any of the blocks is intra-coded (S271).

If the determination result in step S271 is NO, the filtering control unit can determine whether there is one or more non-zero coefficients to be encoded even in one block (P or Q) (S273 ). If the result of the determination in step S273 is NO, the filtering controller may determine whether the reference picture (reference or ref) does not match or is different (S274). If the determination result of step S274 is NO, the filtering control unit may determine whether the number of motion vectors of the two blocks is different from each other (S275). If the result of the determination in step S275 is NO, the filtering controller may determine whether there is a difference of at least a predetermined size between the x component and the y component of each motion vector (S276). If the result of the determination in step S276 is NO, the filtering control unit determines that both blocks have high similarity, and may set the BS or the filtering strength to zero (S278). Meanwhile, if the determination result of each of the steps S273, S274, S275, and S276 is YES, the filtering control unit may set the BS, i.e., the filtering strength, to 1 (S277).

If the result of the determination in step S271 is YES, the filtering controller may determine whether both P and Q perform intra block matching (intra-MB) (S271b). If the determination result in step S271b is Io, the filtering control unit may set the BS or the filtering strength to 2 (S272).

On the other hand, if the result of step S271b is YES, the filtering control unit can determine whether there is one or more non-zero coefficients to be encoded even in one block (P or Q) (S273b) . If the determination result of step S273b is NO, the filtering control unit may determine whether the number of motion vectors of the two blocks is different from each other (S275b). If the result of the determination in step S275b is NO, the filtering controller may determine whether there is a difference of at least a predetermined size between the x component and the y component of each motion vector (S276b). If the result of step S276b is NO, it is determined that both blocks of the filtering control unit have high similarity, and the BS or the filtering strength may be set to 0 (S278). If the determination result of each of the steps S273b, S275b, and S276b is YES, the filtering control unit may set the BS, i.e., the filtering strength, to 1 (S277).

According to the present embodiment, when either of the blocks is intra-coded, the filtering strength may be set to 2 if both blocks are not blocks coded by block matching in the current picture. Also, if both blocks are blocks coded by block matching in the current picture, the filtering strength is set to 1 if any of the two blocks has non-zero coding coefficients. Also, if there is no coefficient to be coded in both blocks, the number of motion vectors of both blocks may be compared and the filtering strength may be set to 1 in other cases. In addition, if both blocks use the same number of motion vectors, and the difference between the x and y components of each motion vector is equal to or larger than a certain size, the filtering strength can be set to 1. If the difference is less than the predetermined size, it is determined that the similarity of the two blocks is high, and the filtering strength can be set to zero. That is, in this embodiment, the filtering strength may be set differently depending on which reference picture is used.

FIGS. 31 to 33 are exemplary diagrams for explaining a filtering application determination process in the image encoding method of FIG.

31 to 33, the filtering control unit may calculate the amount of change in each block and the amount of change in the boundary between two blocks in order to determine the filtering application. The left block of both blocks of the block boundary B5 is referred to as a P block, and the right block is referred to as a Q block.

The amount of change in the P block is shown below.

dp0 = | p (0,0) - p (1,0) x2 + p (2,0) |

dp1 = | p (0,1) - p (1,1) x2 + p (2,1) |

dp2 = | p (0,2) - p (1,2) x2 + p (2,2) |

dp3 = | p (0,3) - p (1,3) x2 + p (2,3) |

Dpx (where x is 0, 1, 2 or 3) represents the amount of change in the pixel.

The amount of change in the Q block is shown below.

dq0 = q (0,0) - q (1,0) x2 + q (2,0) |

dq1 = q (0,1) - q (1,1) x2 + q (2,1) |

dq2 = q (0,2) - q (1,2) x2 + q (2,2) |

dq3 = q (0,3) - q (1,3) x2 + q (2,3) |

In the above variation, dqx (x is 0, 1, 2, or 3) represents the amount of change in the corresponding pixel.

And the amount of change in the block boundary (B5) of the P and Q blocks is as follows.

dpq0 = | p (0,0) - q (0,0) |

dpq1 = | p (0, 1) - q (0, 1) |

dpq2 = p (0,2) - q (0,2) |

dpq3 = | p (0,3) - q (0,3) |

In the above variation, dpqx (x is 0, 1, 2 or 3) represents the amount of change in the pixel.

As described above, the amount of change can be measured for each pixel line unit, and the amount of change can be measured in units of two or more pixel lines. In the case of whether filtering is applied in units of a bundled pixel line, filtering strength, filtering applied pixels, filter coefficients and the like can be determined in the next step.

At least one of the above variations may be considered, and additional variations other than the above three variations may be considered depending on the implementation. If we consider the above three variations, we can decide whether to apply filtering by comparing the variation with each boundary value.

For example, when the first line is defined for explanation, when the amount of change in the P block, the amount of change in the Q block, and the amount of change in the PQ block boundary B5 are smaller than the threshold values (threshold_P, threshold_Q, threshold_PQ) And if filtering is larger than the threshold value, filtering may not be applied.

As another example, filtering may not be applied if the amount of change at the block boundary is greater than the threshold value and the amount of change of at least one of the blocks is less than the threshold value.

As another example, when the amount of change of the P block is larger than the boundary value and the amount of change of the Q block and the amount of change of the PQ block boundary are smaller than the boundary value, the step edge portion where the amount of change of the P block occurs is detected, The filtering can be performed only on the portion where the step edge is detected.

Filtering as described above may be applied in the filtering application. In the above case, the boundary value may be set by a quantization parameter, a bit depth, or the like. If the quantization parameters of both blocks are different, the quantization parameter of the two blocks may be selected or set to an average value. When setting the average value, it is possible to use any of rounding, rounding down, and rounding to process the decimal unit.

In addition, it is possible to determine whether filtering is applied based on the pixel value. In this case, it is possible to determine using the encoding information of both blocks. In this case, as shown in Fig. 31, the coding mode of both blocks is intra, and the prediction mode information of both blocks can coincide with each other. The boundary to which the filtering is applied is a horizontal edge boundary, and it may be a boundary due to a difference in the conversion unit but belonging to a prediction unit of two block modes.

Also, as shown in FIG. 32, the filtering control unit can determine that both blocks are likely to be blocks having a correlation in the horizontal direction continuously through the mode of both blocks. In this case, the filtering control unit may decide to apply the unconditional filtering without measuring the amount of change unlike the description of FIG. 31 above.

33, when the coding mode of both blocks is intra and the prediction mode information of both blocks coincides with each other, the filtering control section measures the amount of change as described above because the probability of occurrence of a step edge is high Can be determined pixel by pixel for the filtering application.

According to the above embodiment, the filtering control unit or the image encoding / decoding apparatus including the filtering control unit can determine whether to apply the filtering considering the correlation between the two blocks based on the encoding mode and the prediction mode information of both blocks. Since the encoding coefficient may also have an influence on the application of the filtering, the filtering control unit may consider the encoding coefficient as well.

In addition, when the encoding mode of both blocks is intra, the prediction mode of both blocks is DC mode on the left side and the horizontal mode is on the right side, and the coding coefficient of the left block is not 0, one or more coefficients exist When the coding coefficient of the right block is non-zero and there is no coefficient, it can be judged that the left block is a complicated area, and the variation amount of the corresponding block may exceed the threshold value set above. However, in this case, it is possible to apply the filtering even when the amount of change exceeds the threshold value because there is a possibility that the amount of change is high due to the complexity of the image characteristic, not the edge region. In addition to measuring the amount of change, the prediction mode information, coding coefficients, and the like can be confirmed and the application of the filtering can be determined.

On the other hand, in the above-described embodiment, the case where the filtering strength is 0, 1, or 2 has been described. It can be changed according to the setting of the encoder / decoder and can be transmitted in units of a sequence, picture, slice, or the like.

Referring again to FIG. 23, the filtering application unit performing the filtering application step S237 may apply the filtering according to the determination result of the strength of the filtering and the application of the filtering. The filtering application unit can determine the filter and the pixel to be applied to the filtering according to the intensity of the filtering determined in advance, and can further determine the filter and the pixel to be applied to the filtering based on the intensity of the filtering determined above. This determination can be performed following the progression based on the pixel line applied in the previous step.

In addition, the filtering application unit can apply the filtering through analysis of the amount of change for the pixel placed at the block boundary. For example, the intensity and the applied pixel of the filtering can be determined according to the change amount of the pixel (the change amount in the P, Q block, and the PQ boundary) as in the following 1) to 3). It is possible to apply filtering by selecting at least one filter according to the result such as whether the amount of change is larger or smaller than the boundary value. Here, the boundary values may be applied to different boundary values in each case. At this time, the boundary value can be determined by quantization parameters of both blocks, bit depth, and the like. And the filter may include, for example, a strong filter, a weak filter, and the like.

1) | p (0,0) - p (1,0) x2 + p (2,0) | + | q (0, 0) - q (1, 0) x 2 + q (2, 0) |

2) | p (0,0) - p (3,0) | + | q (0,0) - q (3,0) |

3) | p (0,0) - q (0,0) |

In this case, the number of pixels to be applied to the filtering may be determined according to the above results. That is, the filtering application unit may apply filtering to all preset candidate pixels, and may set pixels that do not perform filtering according to the above results. The filter may have the characteristics of a low pass filter in the form of a finite impulse response (FIR). The input pixels of the filter may be adaptively determined according to the intensity of the filter, and / or the position of the pixel.

In addition, the filtering application unit may determine the applied pixels of the filter according to the encoding mode of both blocks. In general, the filtering is applied to the pixels placed in the vertical direction at the horizontal boundary of the block and the pixels positioned in the horizontal direction at the vertical boundary. In the present embodiment, the coding mode of both blocks is intra, If the prediction mode is a vertical mode and the boundary of the current block is a vertical boundary, filtering may be performed on the horizontal pixels placed at the horizontal boundary to reduce the quantization error.

In addition, since the correlation between pixels in the vertical direction is high in both blocks, when the pixels placed in the vertical direction are filtered, it is possible to reduce the quantization error more efficiently than when the filtering is performed in the horizontal direction. In this case, the change amount of the pixel can be obtained for the pixels lying in the vertical direction of the vertical boundary as in the following 1) and 2), and at least one filter is selected according to the result such as whether it is larger or smaller than the boundary value Filtering can be applied. In addition, the number of pixels to be applied to the filtering can also be determined according to the result immediately above.

1) | p (0,0) - p (0,1) | + | p (0, 1) - p (0, 2) | + | p (0, 2) - p (0, 3) |

2) | q (0,0) - q (0,1) | + | q (0, 1) - q (0, 2) | + | q (0,2) - q (0,3) |

In addition, the filtering application unit may apply the filtering by measuring the amount of change between the pixels placed at the block boundary for the purpose of reducing the block deterioration between the both blocks.

According to the embodiments described above, the strength of the deblocking filter can be determined according to whether the current picture is block-matched to generate a prediction block, or to process the current block as an intra-prediction mode or an inter-prediction mode. In case of processing in the inter picture prediction mode, in addition to the conditions such as the encoding mode of both blocks, the presence of the same reference picture and the motion vector, and the influence of the intensity of the existing deblocking filter, You can decide.

In the above-described embodiments, the image encoding method may be replaced with the image decoding method when the adaptive deblocking filtering is used in the process of decoding the encoded image. It is needless to say that the image encoding / decoding method may be performed by an image processing apparatus or an image encoding / decoding apparatus having at least one means for encoding and decoding or a component performing a function corresponding to the means.

According to the embodiments described above, it is possible to use an international codec such as MPEG-2, MPEG-4, or H.264 or other codecs in which the intra prediction technique is used, a medium using these codecs, A high-efficiency image encoding / decoding technique can be provided. In future, it is expected to be applied to the current high efficiency image coding technology (HEVC) and standard codec such as H.264 / AVC and the image processing field using intra prediction.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the present invention as defined by the following claims It can be understood that

Claims (20)

Generating a reconstructed signal based on a prediction signal that refers to a decoded image after being encoded and a residual signal that is generated through inverse quantization and inverse transform;
Confirming a filtering skip for the restored signal; And
Performing deblocking filtering if it is not the filtering skip,
Wherein the deblocking filtering includes a first sub-step of identifying a block boundary, a second sub-step of setting a filtering strength, and a third sub-step of determining whether to apply filtering. The method according to claim 1,
Wherein the first sub-step sets a minimum block size according to a luminance component or a chrominance component according to a color format of a chrominance component. The method according to claim 1,
Wherein the second sub-step comprises analyzing characteristics of two blocks lying on both sides of the block boundary to determine which filter to use and determining the filtering strength according to the determined filter to use. The method of claim 3,
Wherein the information used to determine which filter to use includes information on at least one of a coding mode of each of the two blocks, a boundary characteristic between each block or two blocks, prediction information, Image encoding method. The method of claim 3,
Wherein the second sub-step determines the filtering strength based on an actual pixel value or information of the corresponding block. The method according to claim 1,
Wherein the third sub-step determines whether filtering is applied by applying the filtering strength obtained in the second sub-step to reconstructed pixels placed at a block boundary. The method of claim 6,
Wherein the third sub-step determines whether the filtering is to be applied based on a change amount determined for a pixel located at a boundary between the two blocks. Generating a reconstructed signal based on a predicted signal referring to the decoded image and a residual signal generated through inverse quantization and inverse transform;
Confirming a filtering skip for the restored signal; And
Performing deblocking filtering if it is not the filtering skip,
Wherein performing deblocking filtering comprises a first sub-step of identifying a block boundary, a second sub-step of setting a filtering strength, and a third sub-step of determining whether to apply filtering. The method of claim 8,
Wherein the first sub-step sets a minimum block size according to a luminance component or a chrominance component according to a color format of a chrominance component. The method of claim 8,
Wherein the second substep analyzes characteristics of two blocks lying on both sides of the block boundary to determine which filter to use and determines the filtering strength according to the determined filter to use. The method of claim 10,
Wherein the information used to determine which filter to use includes information on at least one of a coding mode of each of the two blocks, a boundary characteristic between each block or two blocks, prediction information, Video decoding method. The method of claim 10,
Wherein the second sub-step determines the filtering strength based on an actual pixel value or information of the corresponding block. The method of claim 8,
Wherein the third sub-step determines whether filtering is to be applied by applying the filtering strength obtained in the second sub-step to reconstructed pixels placed at block boundaries. 14. The method of claim 13,
Wherein the third sub-step determines whether or not the filtering is applied based on a change amount determined for a pixel located at a boundary between the two blocks. A decoding program for generating a reconstructed signal based on a residual signal generated through an inverse quantization and an inverse transform process, and a filtering program for checking skipping of a reconstructed signal and skipping or performing filtering, A memory for storing; And
And a processor connected to the memory for executing a decryption program or a program code of a filtering program,
The processor, by means of the program code,
Generating a reconstructed signal based on a predicted signal referring to the decoded image and a residual signal generated through an inverse quantization and inverse transform process, checking a filtering skip for a reconstructed signal, and performing deblocking filtering if the skipped is not a filtering skip, Wherein when blocking filtering is performed, the block boundary is checked, the filtering strength is set, and whether or not the filtering is applied is determined. 16. The method of claim 15,
Wherein the processor sets a minimum block size according to a luminance component or a chrominance component according to a color format of a chrominance component when checking the block boundary. 16. The method of claim 15,
Wherein the processor is further configured to analyze the characteristics of two blocks lying on either side of the block boundary to determine which filter to use and to determine the filtering strength according to the determined filter to use, Device. 18. The method of claim 17,
Wherein the information used to determine which filter to use includes information on at least one of a coding mode of each of the two blocks, a boundary characteristic between each block or two blocks, prediction information, Image decoding apparatus. 16. The method of claim 15,
Wherein the processor determines whether the filtering is applied by applying a filtering strength obtained when the filtering strength is set for a restored pixel placed at a block boundary when determining whether to apply the filtering. The method of claim 19,
Wherein the processor determines whether or not the filtering is applied based on a change amount determined as a target of a pixel located at a boundary between the two blocks when determining whether to apply the filtering.

Images