KR102045983B1 - Method and apparatus for encoding and decoding based on merge - Google Patents

Abstract

A method and apparatus for decoding based on merge are disclosed. The merge motion prediction method determines whether there is a merge candidate block that is not available in the current block, and when there is a merge candidate block that is not available in the current block, the candidate for the merge motion prediction based on the corresponding lower layer block. Deriving motion prediction information and performing merge motion prediction on the current block based on the merge motion candidate list including the candidate motion prediction information, wherein the merge candidate block is a candidate motion for merge motion prediction. A block for deriving prediction information, and a corresponding lower layer block may exist in a lower layer than a layer including a current block.

Description

Method and apparatus for decoding based on merge {Method and apparatus for encoding and decoding based on merge}

The present invention relates to an image decoding method and apparatus, and more particularly, to an inter prediction method.

Recently, the demand for high resolution and high quality images such as high definition (HD) and ultra high definition (UHD) images is increasing in various applications. As the video data becomes higher resolution and higher quality, the amount of data increases relative to the existing video data. Therefore, when the video data is transmitted or stored using a medium such as a conventional wired / wireless broadband line, The storage cost will increase. High efficiency image compression techniques can be used to solve these problems caused by high resolution and high quality image data.

An inter-screen prediction technique for predicting pixel values included in the current picture from a picture before or after the current picture using an image compression technique, an intra prediction technique for predicting pixel values included in a current picture using pixel information in the current picture, Various techniques exist, such as an entropy encoding technique for allocating a short code to a high frequency of appearance and a long code to a low frequency of appearance, and the image data can be effectively compressed and transmitted or stored.

In addition, video encoding technology for efficiently transmitting data to various transmission environments and various terminals by generating one unified data capable of supporting various spatial resolutions and various frame rates. As a result, standardization for Scalable HEVC (SHVC) based on HEVC is in progress. In SHVC, image information of a higher layer may be encoded through image information of a lower layer. Here, the lower layer may mean a base layer.

An object of the present invention is to provide an inter prediction method for increasing image encoding and decoding efficiency.

Another object of the present invention is to provide an apparatus for performing an inter prediction method for increasing image encoding and decoding efficiency.

According to an aspect of the present invention, there is provided a method of predicting a merge motion in which a merge candidate block is not present in a current block, and the merge candidate is not available in the current block. If there is a block, deriving candidate motion prediction information for the merge motion prediction based on a corresponding lower layer block and merging the current block based on a merge motion candidate list including the candidate motion prediction information; The method may include performing motion prediction, wherein the merge candidate block is a block for deriving candidate motion prediction information for the merge motion prediction, and the corresponding lower layer block is lower than a layer including the current block. May exist in The candidate motion prediction information may be motion prediction information of a merge candidate block of the corresponding lower layer block. The merge motion prediction method may further include determining whether to perform the merge motion prediction on the current block based on the size of the current block and depth information of the coding block including the current block. The current block may be included in an enhanced layer, and the corresponding lower layer block may be included in a base layer. The candidate motion prediction information may be motion prediction information of the corresponding lower layer block. The merge motion prediction method may further include determining whether to perform the merge motion prediction method on the current block based on the size of the current block and depth information of the coding block including the current block. The current block may be included in an enhanced layer, and the corresponding lower layer block may be included in a base layer.

In the image decoding apparatus according to an aspect of the present invention for achieving the above object of the present invention, the image decoding apparatus may include a motion compensation unit, the motion compensation unit is a merge candidate block that is not available to the current block If there is a merge candidate block present in the current block, the candidate motion prediction information for the merge motion prediction is derived based on a corresponding lower layer block, and the candidate motion prediction information is determined. It may be implemented to perform the merge motion prediction for the current block based on a merge motion candidate list including, wherein the merge candidate block is a block for deriving candidate motion prediction information for the merge motion prediction, the corresponding Lower layer blocks are lower than the layer containing the current block. The present in the layer may be a colo locate blocks of the current block. The candidate motion prediction information may be motion prediction information of a merge candidate block of the corresponding lower layer block. The motion compensator may be implemented to determine whether to perform the merge motion prediction on the current block based on the size of the current block and depth information of a coding block including the current block. The current block may be included in an enhanced layer, and the corresponding lower layer block may be included in a base layer. The candidate motion prediction information may be motion prediction information of the corresponding lower layer block. The motion compensator may be implemented to determine whether to perform the merge motion prediction method on the current block based on the size of the current block and depth information of a coding block including the current block. The current block may be included in an enhanced layer, and the corresponding lower layer block may be included in a base layer.

As described above, according to the decoding-based decoding method and apparatus according to the embodiment of the present invention, one coding unit (CU) is divided into a plurality of prediction units (PU) to be encoded or decoded. When there is a motion candidate that is not available for one prediction unit, encoding efficiency may be increased by using a motion candidate in a lower layer image corresponding to the candidate in the upper layer.

1 is a block diagram illustrating a configuration of an image encoding apparatus according to an embodiment.
2 is a block diagram illustrating a configuration of an image decoding apparatus according to an embodiment.
3 is a conceptual diagram schematically illustrating an embodiment of a scalable video coding structure using multiple layers to which the present invention can be applied.
4 is a conceptual diagram illustrating neighboring blocks of a current block used as a merge motion candidate list.
5 is a conceptual diagram illustrating a method of setting a merge motion candidate list when one CU is divided into two PUs.
6 is a conceptual diagram of a 2N × 2N CU including a plurality of CUs.
7 is a conceptual diagram illustrating a method of performing parallel merge motion prediction.
8 is a conceptual diagram illustrating an example in which several blocks are bundled with MER.
9 is a conceptual diagram illustrating candidate regions in a MER.
10 is a conceptual diagram illustrating a merge motion prediction method according to an embodiment of the present invention.
11 is a conceptual diagram illustrating a merge motion prediction method according to an embodiment of the present invention.
12 is a conceptual diagram illustrating a merge motion prediction method according to an embodiment of the present invention.

As the invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. In describing the drawings, similar reference numerals are used for similar elements.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The term and / or includes a combination of a plurality of related items or any item of a plurality of related items.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may exist in the middle. Should be. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in between.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprises" or "having" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Hereinafter, with reference to the accompanying drawings, it will be described in detail a preferred embodiment of the present invention. Hereinafter, the same reference numerals are used for the same components in the drawings, and duplicate descriptions of the same components are omitted.

1 is a block diagram illustrating a configuration of an image encoding apparatus according to an embodiment. A scalable video encoding / decoding method or apparatus may be implemented by an extension of a general video encoding / decoding method or apparatus that does not provide scalability, and the block diagram of FIG. 1 is scalable. An embodiment of an image encoding apparatus that may be the basis of a video encoding apparatus is illustrated.

Referring to FIG. 1, the image encoding apparatus 100 may include a motion predictor 111, a motion compensator 112, an intra predictor 120, a switch 115, a subtractor 125, and a converter 130. And a quantization unit 140, an entropy encoding unit 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filter unit 180, and a reference image buffer 190.

The image encoding apparatus 100 may perform encoding in an intra mode or an inter mode on an input image and output a bit stream. Intra prediction means intra prediction and inter prediction means inter prediction. In the intra mode, the switch 115 is switched to intra, and in the inter mode, the switch 115 is switched to inter. The image encoding apparatus 100 may generate a prediction block for an input block of an input image and then encode a difference between the input block and the prediction block.

In the intra mode, the intra predictor 120 may generate a prediction block by performing spatial prediction using pixel values of blocks that are already encoded around the current block.

In the inter mode, the motion predictor 111 may obtain a motion vector by searching for a region that best matches an input block in the reference image stored in the reference image buffer 190 during the motion prediction process. The motion compensator 112 may generate a prediction block by performing motion compensation using the motion vector and the reference image stored in the reference image buffer 190.

The subtractor 125 may generate a residual block by the difference between the input block and the generated prediction block. The transform unit 130 may output a transform coefficient by performing transform on the residual block. The quantization unit 140 may output the quantized coefficient by quantizing the input transform coefficient according to the quantization parameter.

The entropy encoding unit 150 entropy encodes a symbol according to a probability distribution based on values calculated by the quantization unit 140 or encoding parameter values calculated in the encoding process, thereby generating a bit stream. You can print The entropy encoding method is a method of receiving a symbol having various values and expressing it in a decodable column while removing statistical redundancy.

Here, the symbol means a syntax element, a coding parameter, a residual signal value, or the like that is to be encoded / decoded. The encoding parameter is a parameter necessary for encoding and decoding, and may include information that may be inferred in the encoding or decoding process as well as information encoded by the encoding apparatus and transmitted to the decoding apparatus, such as syntax elements. It means the information you need when you do. Coding parameters may be, for example, intra / inter prediction modes, moving / motion vectors, reference picture indexes, coding block patterns, presence or absence of residual signals, transform coefficients, quantized transform coefficients, quantization parameters, block sizes, block partitioning information, or the like. May include statistics. In addition, the residual signal may mean a difference between the original signal and the prediction signal, and a signal in which the difference between the original signal and the prediction signal is transformed or a signal in which the difference between the original signal and the prediction signal is converted and quantized It may mean. The residual signal may be referred to as a residual block in block units.

When entropy encoding is applied, a small number of bits are allocated to a symbol having a high probability of occurrence and a large number of bits are allocated to a symbol having a low probability of occurrence, whereby the size of the bit string for the symbols to be encoded is increased. Can be reduced. Therefore, compression performance of image encoding may be increased through entropy encoding.

For entropy coding, coding methods such as exponential golomb, context-adaptive variable length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC) may be used. For example, the entropy encoder 150 may store a table for performing entropy encoding, such as a variable length coding (VLC) table, and the entropy encoder 150 may store the stored variable length encoding. Entropy encoding may be performed using the (VLC) table. In addition, the entropy encoder 150 derives a binarization method of a target symbol and a probability model of a target symbol / bin, and then performs entropy encoding using the derived binarization method or a probability model. You may.

The quantized coefficients may be inversely quantized by the inverse quantizer 160 and inversely transformed by the inverse transformer 170. The inverse quantized and inverse transformed coefficients are added to the prediction block through the adder 175 and a reconstruction block can be generated.

The reconstruction block passes through the filter unit 180, and the filter unit 180 applies at least one or more of a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the reconstructed block or reconstructed picture. can do. The reconstructed block that has passed through the filter unit 180 may be stored in the reference image buffer 190.

2 is a block diagram illustrating a configuration of an image decoding apparatus according to an embodiment.

As described above with reference to FIG. 1, a scalable video encoding / decoding method or apparatus may be implemented by extension of a general video encoding / decoding method or apparatus that does not provide scalability, and the block diagram of FIG. 2 is scalable video decoding. An embodiment of an image decoding apparatus that may be the basis of an apparatus is shown.

2, the image decoding apparatus 200 may include an entropy decoder 210, an inverse quantizer 220, an inverse transformer 230, an intra predictor 240, a motion compensator 250, and a filter. 260 and a reference picture buffer 270.

The image decoding apparatus 200 may receive a bitstream output from the encoding apparatus, perform decoding in an intra mode or an inter mode, and output a reconstructed image, that is, a reconstructed image. In the intra mode, the switch may be switched to intra, and in the inter mode, the switch may be switched to inter. The image decoding apparatus 200 may generate a reconstructed block, that is, a reconstructed block by obtaining a residual block reconstructed from the received bitstream, generating a prediction block, and adding the reconstructed residual block and the prediction block.

The entropy decoder 210 may entropy decode the input bitstream according to a probability distribution to generate symbols including symbols in the form of quantized coefficients. The entropy decoding method is a method of generating each symbol by receiving a binary string. The entropy decoding method is similar to the entropy coding method described above.

The quantized coefficients are inversely quantized by the inverse quantizer 220 and inversely transformed by the inverse transformer 230, and as a result of the inverse quantization / inverse transformation of the quantized coefficients, a reconstructed residual block may be generated.

In the intra mode, the intra predictor 240 may generate a predictive block by performing spatial prediction using pixel values of an already encoded block around the current block. In the inter mode, the motion compensator 250 may generate a prediction block by performing motion compensation using the motion vector and the reference image stored in the reference image buffer 270.

The reconstructed residual block and the prediction block are added through the adder 255, and the added block passes through the filter unit 260. The filter unit 260 may apply at least one or more of the deblocking filter, SAO, and ALF to the reconstructed block or the reconstructed picture. The filter unit 260 outputs a reconstructed image, that is, a reconstructed image. The reconstructed picture may be stored in the reference picture buffer 270 to be used for inter prediction.

The entropy decoder 210, the inverse quantizer 220, the inverse transformer 230, the intra predictor 240, the motion compensator 250, and the filter 260 included in the image decoding apparatus 200. And components directly related to the decoding of an image in the reference image buffer 270, for example, an entropy decoder 210, an inverse quantizer 220, an inverse transformer 230, an intra predictor 240, and motion compensation. The unit 250, the filter unit 260, and the like may be distinguished from other components and expressed as a decoder or a decoder.

Also, the image decoding apparatus 200 may further include a parsing unit (not shown) which parses information related to an encoded image included in a bitstream. The parser may include the entropy decoder 210 or may be included in the entropy decoder 210. Such a parser may also be implemented as one component of the decoder.

3 is a conceptual diagram schematically illustrating an embodiment of a scalable video coding structure using multiple layers to which the present invention can be applied. In FIG. 3, a GOP (Group of Picture) represents a picture group, that is, a group of pictures.

In order to transmit image data, a transmission medium is required, and its performance varies depending on the transmission medium according to various network environments. A scalable video coding method may be provided for application to such various transmission media or network environments.

The scalable video coding method is a coding method that improves encoding / decoding performance by removing inter-layer redundancy by using texture information, motion information, and residual signals between layers. The scalable video coding method may provide various scalability in terms of spatial, temporal, and image quality according to ambient conditions such as a transmission bit rate, a transmission error rate, and a system resource.

Scalable video coding may be performed using multiple layers structure to provide a bitstream applicable to various network situations. For example, the scalable video coding structure may include a base layer that compresses and processes image data by using a general image encoding method, and compresses the image data by using the encoding information of the base layer and a general image encoding method together. May include an enhancement layer for processing.

In this case, a layer is an image and a bit divided based on spatial (eg, image size), temporal (eg, coding order, image output order, frame rate), image quality, complexity, and the like. Means a set of bitstreams. In addition, the base layer may mean a lower layer, a reference layer or a base layer, and the enhancement layer may mean an upper layer and an enhancement layer. In addition, the plurality of layers may have a dependency between each other.

Referring to FIG. 3, for example, the base layer may be defined as a standard definition (SD), a frame rate of 15 Hz, and a 1 Mbps bit rate, and the first enhancement layer may be a high definition (HD), a frame rate of 30 Hz, and a 3.9 Mbps bit rate. The second enhancement layer may be defined as an ultra high definition (4K-UHE), a frame rate of 60 Hz, and a bit rate of 27.2 Mbps. The format, frame rate, bit rate, etc. are exemplary and may be determined differently as necessary. In addition, the number of hierarchies used is not limited to this embodiment and may be determined differently according to a situation.

For example, if the transmission bandwidth is 4 Mbps, the frame rate of the first enhancement layer HD may be reduced and transmitted at 15 Hz or less. The scalable video coding method can provide temporal, spatial and image quality scalability by the method described above in the embodiment of FIG. 3.

In general, inter-screen prediction may use at least one of a previous picture or a subsequent picture of the current picture as a reference picture, and perform prediction on the current block based on the reference picture. An image used for prediction of the current block is called a reference picture or a reference frame.

The region in the reference picture may be represented using a reference picture index refIdx, a motion vector, etc. indicating the reference picture.

The inter prediction may select a reference picture corresponding to the current block in the reference picture and the reference picture, and generate a prediction block for the current block.

In the inter prediction, the encoding apparatus and the decoding apparatus may derive the motion information of the current block and then perform the inter prediction and / or motion compensation based on the derived motion information. In this case, the encoding apparatus and the decoding apparatus may move a coll block corresponding to the current block in a neighboring block and / or a collocated picture that has already been restored. By using the information, the encoding / decoding efficiency can be improved.

Here, the reconstructed neighboring block is a block in the current picture that is already encoded and / or decoded and reconstructed, and may include a block adjacent to the current block and / or a block located at an outer corner of the current block. In addition, the encoding apparatus and the decoding apparatus may determine a predetermined relative position based on a block existing at a position spatially corresponding to the current block in the call picture, and the predetermined relative position (spatially corresponding to the current block) The call block may be derived based on the location of the inside and / or outside of the block existing at the location. Here, as an example, the call picture may correspond to one picture among the reference pictures included in the reference picture list.

In inter prediction, a prediction block may be generated such that a residual signal with a current block is minimized and a motion vector size is also minimized.

Meanwhile, the motion information derivation scheme may vary depending on the prediction mode of the current block. Prediction modes applied for inter prediction may include Advanced Motion Vector Predictor (AMVP), merge, and the like.

As an example, when an advanced motion vector predictor (AMVP) is applied, the encoding apparatus and the decoding apparatus may generate a predicted motion vector candidate list using the motion vector of the reconstructed neighboring block and / or the motion vector of the call block. . That is, the motion vector of the reconstructed neighboring block and / or the motion vector of the call block may be used as the prediction motion vector candidate. The encoding apparatus may transmit a predicted motion vector index indicating the optimal predicted motion vector selected from the predicted motion vector candidates included in the list to the decoding apparatus. In this case, the decoding apparatus may select the prediction motion vector of the current block from the prediction motion vector candidates included in the prediction motion vector candidate list by using the prediction motion vector index.

The encoding apparatus may obtain a motion vector difference (MVD) between the motion vector of the current block and the predictive motion vector, and may encode the same and transmit the encoded motion vector to the decoding apparatus. In this case, the decoding apparatus may decode the received motion vector difference, and may derive the motion vector of the current block through the sum of the decoded motion vector difference and the predicted motion vector.

The encoding apparatus may also transmit a reference picture index or the like indicating the reference picture to the decoding apparatus.

The decoding apparatus may predict the motion vector of the current block using the motion information of the neighboring block, and may derive the motion vector for the current block using the residual received from the encoding apparatus. The decoding apparatus may generate a prediction block for the current block based on the derived motion vector and the reference picture index information received from the encoding apparatus.

As another example, when merge is applied, the encoding apparatus and the decoding apparatus may generate the merge candidate list using the motion information of the reconstructed neighboring block and / or the motion information of the call block. That is, when there is motion information of the reconstructed neighboring block and / or the call block, the encoding apparatus and the decoding apparatus may use this as a merge candidate for the current block.

The encoding apparatus may select a merge candidate capable of providing an optimal encoding efficiency among the merge candidates included in the merge candidate list as motion information for the current block. In this case, a merge index indicating the selected merge candidate may be included in the bitstream and transmitted to the decoding apparatus. The decoding apparatus may select one of the merge candidates included in the merge candidate list by using the transmitted merge index, and determine the selected merge candidate as motion information of the current block. Therefore, when the merge mode is applied, the motion information of the restored neighboring block and / or the call block may be used as the motion information of the current block. The decoding apparatus may reconstruct the current block by adding the prediction block and the residual transmitted from the encoding apparatus.

In the above-described AMVP and merge mode, the motion information of the reconstructed neighboring block and / or the motion information of the call block may be used to derive the motion information of the current block.

In the case of a skip mode, which is one of other modes used for inter prediction, information of neighboring blocks may be used as is in the current block. Accordingly, in the skip mode, the encoding apparatus does not transmit syntax information such as residual to the decoding apparatus other than the information indicating which block motion information to use as the motion information of the current block.

The encoding apparatus and the decoding apparatus may generate a prediction block of the current block by performing motion compensation on the current block based on the derived motion information. Here, the prediction block may mean a motion compensated block generated as a result of performing motion compensation on the current block. Also, the plurality of motion compensated blocks may constitute one motion compensated image.

The decoding apparatus may check and derive motion information necessary for inter prediction of the current block, for example, a skip flag, a merge flag, and the like received from the encoding apparatus, corresponding to the information about a motion vector, a reference picture index, and the like.

The processing unit in which the prediction is performed and the processing unit in which the prediction method and the details are determined may be different. For example, a prediction mode may be determined in units of prediction blocks, and prediction may be performed in units of transform blocks, or a prediction mode may be determined in units of prediction blocks, and intra prediction may be performed in units of transform blocks.

The merge motion prediction method, that is, motion information used for merge prediction, is information including at least one of a motion vector, an index with respect to a reference image, and a prediction direction (unidirectional, bidirectional, etc.). The prediction direction may be largely divided into unidirectional prediction and bidirectional prediction according to the use of a reference picture list (RefPicList).

* For unidirectional prediction, it is divided into forward prediction (Pred_L0; Prediction L0) using forward reference picture list (LIST 0) and backward prediction (Pred_L1; Prediction L1) using backward reference picture list (LIST 1).

In addition, bidirectional prediction (Pred_BI) uses both a forward reference picture list (LIST 0) and a reverse reference picture list (LIST 1), and may mean that both forward prediction and backward prediction exist.

In addition, the case in which two forward predictions exist by copying the forward reference picture list LIST 0 to the backward reference picture list LIST 1 may also be included in the range of bidirectional prediction. Such prediction direction may be expressed using flag information such as predFlagL0 and predFlagL1.

In an embodiment, in the case of unidirectional prediction and forward prediction, predFlagL0 may be '1' and predFlagL1 may be '0'. Also, in the case of unidirectional prediction and reverse prediction, predFlagL0 may be '0' and predFlagL1 may be '1'. In addition, in the bidirectional prediction, predFlagL0 may be '1' and predFlagL1 may be “1”.

Merged motion prediction may be performed for each coding unit (CU), or may be performed for each prediction unit (PU).

In the case of performing a merge movement for each CU or PU (for convenience of description, a CU is a 'coding block', a PU is a 'prediction block', and a CU and a PU collectively, a 'block'), a block partition Information about whether to perform a merge motion prediction for each block and a neighboring block adjacent to the current block (left neighbor block of the current block, upper neighbor block of the current block, temporal neighbor block of the current block, etc.) Information on whether to do this should be signaled.

The merge motion candidate list represents a list in which motion information is stored and is generated before merge motion prediction is performed. The motion information stored in the merge motion candidate list may be motion information of a neighboring block adjacent to the current block or motion information of a block collocated with the current block in the reference image. Also, the motion information stored in the merge motion candidate list may be new motion information created by combining motion information already present in the merge motion candidate list.

4 is a conceptual diagram illustrating neighboring blocks of a current block used as a merge motion candidate list.

Referring to FIG. 4, the merge motion candidate list is a candidate block H at the same position as the neighboring blocks A 410, B 420, C 430, D 440, and E 450 of FIG. 4. For 460 (or M 470), it may be determined whether motion information of the corresponding block can be used for prediction of the merge motion of the current block. When the motion information of the block is available for the merge motion prediction of the current block, the motion information of the block may be input to the merge motion candidate list. A block used for merge motion prediction may be defined in terms of a merge motion prediction candidate block.

In addition, if each neighboring block has the same motion information by checking whether the neighboring blocks have the same motion information, the motion information of the neighboring block may not be included in the merge motion candidate list. For example, when generating the merge motion candidate list for the X block that is the current block in FIG. 1, after neighboring block A 410 is available and included in the merge motion candidate list, neighboring block B 420 is Only when it is not the same motion information as the neighboring block A (410), it may be included in the merge motion candidate list. In the same manner, the neighboring block C 430 may be included in the merge motion candidate list only when the neighboring block C 430 is not the same motion information as the neighboring block B 420. The same method may be applied to the neighboring block D 440 and the neighboring block E 450. Here, the same motion information may mean that the motion vectors are the same, use the same reference picture, and use the same prediction directions (unidirectional (forward, reverse), and bidirectional). Finally, in FIG. 1, the merge motion candidate list for the X block 400 is in a predetermined order, for example, A 410 → B 420 → C 430 → D 440 → E 450 → H ( 460 (or M 470) may be added to the list in block order. Here, A 410 may be referred to as A1, B 420 to B1, C 430 to B0, D 440 to A0, and E 450 to B2. Therefore, A (410) → B (420) → C (430) → D (440) → E (450) → H (460) (or M (470)) is A1 → B1 → B0 → A0 → B2 → H (Or M).

The position of the pixel in the upper left of the current block is called (xP, yP), and the width of the current block is defined by variables called nPSW and height as nPSH. MinSize, a variable for representing a merge motion prediction candidate block, indicates the size of the largest block unit that can be used.

Hereinafter, in the embodiment of the present invention, the merge motion prediction candidate block of the current block may include a block including a pixel present in (xP-1, yP + nPSH) and the first left block (block D 440), (xP−). 1, yP + nPSH-MinSize) is used to define a block including a pixel present in the second left block (block A (410)). Also, a block including a pixel located at (xP + nPSW, yP-1) may include a pixel located at a top first block (block C 430) and (xP + nPSW-MinSize, yP-1). A block including a pixel located at an upper second block (block B 420) and (xP-MinSize, yP-1) is defined and used as a term of an upper third block (block E 450). .

The merge motion prediction candidate block includes a left first block (block D 440), a left second block (block A 410), a top first block (block C 430), and a top second block (block B ( 420), and the upper third block (block E 450). One group including a left first block (block D 440) and a left second block (block A 410) is defined as a first merge motion prediction candidate block group and an upper first block (block C 430). ), One group including an upper second block (block B 420) and an upper third block (block E 450) may be defined as a second merge motion prediction candidate block group.

The merge motion prediction candidate block may be a block corresponding to another picture in time. For example, the prediction unit includes the pixel at the position (xP + nPSW, yP + nPSH) in the call-picture of the current prediction unit based on the pixel position (xP, yP) in the picture including the current block, or (xP + nPSW). If a prediction unit including a pixel at a position of yP + nPSH) is not available, it may be a prediction unit including a pixel at a position of (xP + nPSW / 2-1, yP + nPSH / 2-1).

The position and number of the merge motion prediction candidate blocks disclosed in FIG. 4 are arbitrary, and the position and number of the merge motion prediction candidate blocks, which are not deviated from the essence of the present invention, may vary, and are preferred when constructing the merge motion candidate list. The position and candidate prediction group of the merge motion prediction candidate block scanned with may also change. That is, the position, number, scan order, candidate prediction group, and the like of the merge motion prediction candidate blocks used when constructing the merge motion candidate list described in the embodiments of the present invention are exemplified in the nature of the present invention as an embodiment. Unless you change it.

5 is a conceptual diagram illustrating a method of setting a merge motion candidate list when one CU is divided into two PUs.

5 illustrates a method of setting a merge motion candidate list when one CU (2Nx2N) is divided into two PUs (Nx2N).

Although not illustrated in FIG. 5, the neighboring block may be motion information of a neighboring block adjacent to the current block or motion information of a block collocated with the current block in the reference image. The left side of FIG. 5 shows neighboring blocks for constructing the merge motion candidate list for the first PU and the right side of FIG. 5 shows neighboring blocks used for constructing the merge motion candidate list for the second PU.

In order to configure the merge motion candidate list of the second PU in parallel with respect to the second PU located on the right side of FIG. 5, the A1 block located on the first PU on the right side of FIG. 5 is not included in the merge motion candidate list of the second PU. Do not. That is, the A1 block located in the first PU on the right side of FIG. 5 is not available for constructing the merge motion candidate list of the second PU, and may mean that motion information of the A1 block cannot be accessed (or the motion information is May not exist, or it may mean that there is no availability of block motion information).

6 is a conceptual diagram of a 2N × 2N CU including a plurality of CUs.

Referring to FIG. 6, the PU constructs a merge motion candidate list to perform encoding / decoding on motion information. In order to construct a merge motion candidate list, motion information about a neighboring PU must exist. However, when the encoder performs merge motion estimation in parallel in a 2Nx2N CU including a plurality of CUs as shown in FIG. 6, a specific PU () (shown as shaded) among PUs located in a CU of 2Nx2N CU is represented. In the case of PU), merge motion estimation may be performed only. Other PUs cannot perform merge motion prediction in parallel. The reason for this is that each PU must perform merge motion prediction after constructing a merge motion candidate list. In the case of the PUs which are not shaded, there is no motion information on the PUs of neighboring neighboring blocks. Can not. Therefore, PUs that are not shaded cannot perform merge motion prediction in parallel. In order to perform the merge motion prediction method in parallel on the PUs not shaded in FIG. 6, a parallel merge motion prediction (PME) method may be used.

7 is a conceptual diagram illustrating a method of performing parallel merge motion prediction.

Referring to FIG. 7, in the case of “PU0”, motion information available for neighboring blocks may be

Since it already exists, you can create a merge movement list. In the case of “PU1” in FIG. 7, since no neighboring block is encoded (or decoded), no motion information is available. Therefore, the merge motion method cannot be used in parallel with the existing method. In the case of PU1 of FIG. 7, the merge motion method can be used in parallel by not including motion information that is not usable in the neighboring block in the merge motion candidate list. That is, the merge motion candidate list may be configured in parallel by configuring the merge motion candidate list using only the motion information of the neighboring blocks available for the current PU.

The allowable range of the parallel merge motion prediction (PME) method is expressed as a merge motion prediction range (MER; merge motion prediction region) in FIG. 7. For example, in the case of “PU2” in FIG. 7, motion information of neighboring blocks existing in the same MER may not be included in the merge motion candidate list, and only motion information of neighboring blocks existing in another MER is merged candidate list. Can be included in

As another example, FIG. 8 is a conceptual diagram illustrating an example in which several blocks are bundled with MER.

PME is performed only in this region, and since the encoding / decoding is performed in this region in parallel, the motion candidate included in the region is determined to be 'unavailable or unavailable' and excluded from the candidate ( That is, it is not put in the motion candidate list). The method using MER provides parallelism, but the coding efficiency decreases considerably because many candidates cannot be used in the same MER region.

9 is a conceptual diagram illustrating candidate regions in a MER.

In FIG. 9, for example, in the same case of PU0 in FIG. 9, since all candidates exist outside the MER in which they are included, these candidates may be used. However, in case of PU1, all spatial candidates cannot be used because all candidates exist in the MER including them. In the case of PU1, the merge motion candidate list of PU1 may include one temporal candidate and a zero vector. When the motion vector of the neighboring block cannot be used as a spatial candidate, it becomes more prominent as the size of the MER increases. That is, the larger the size of the MER, the higher parallelism is provided, but the coding efficiency may be reduced due to the reduced number of candidates.

The embodiment of the present invention solves a problem of reducing coding efficiency due to the limitation of the use of spatial candidates in the merge motion prediction region, which occurs when using parallel processing in a block-based encoding and decoding structure, such as a scalable HEVC (SHVC). It offers the same parallelism. When using parallel processing for this purpose, if the spatial motion candidate is limited, we propose a method used to predict the upper layer using the motion information of the lower layer.

According to an embodiment of the present invention, for a candidate PU determined to be unavailable when performing parallel merge motion prediction (PME) in a higher layer, a PU (or co) of a lower layer image corresponding to the candidate PU It may refer to the motion prediction information from the -located unit.

10 is a conceptual diagram illustrating a merge motion prediction method according to an embodiment of the present invention.

Referring to FIG. 10, when performing a PME in a higher layer, a PU corresponding to a merge candidate block that may be used as a candidate for merge motion prediction around a current PU may be unavailable. Hereinafter, according to an embodiment of the present invention, a candidate block for performing merge motion prediction is a candidate block for providing motion prediction information for merge motion prediction as a current block, a block located around the current block, or a block located in another picture. The motion prediction information derived from the merge candidate block may be expressed using the term candidate motion prediction information. Hereinafter, in an embodiment of the present invention, a block may be used to mean a PU.

As described above, when performing merge motion prediction, some merge candidate blocks of the merge candidate blocks may be included in the same MER as the current block or may not be decoded or encoded yet. In this case, some of the merge candidate blocks of the merge candidate blocks may not be available as merge candidate blocks for merge motion prediction for the current block.

According to an embodiment of the present invention, in such a case, in SHVC, a merge candidate block of a lower layer block corresponding to the current block may be used instead of a merge candidate block of a current block in a lower layer of the layer in which the current block is located. The lower layer block corresponding to the current block in the lower layer may be a block existing at a co-located position with the current block. Hereinafter, in the embodiment of the present invention, the lower layer block corresponding to the current block in the lower layer is defined and used as the term corresponding lower layer block.

That is, the motion prediction information derived from the merge candidate block of the lower layer block may be used as the candidate motion prediction information instead of the candidate motion prediction information that cannot be derived from the unusable merge candidate block.

For example, in FIG. 10, the merge candidate blocks of the upper layer blocks PU1-0 (1000) may be blocks included in already encoded or decoded MERs rather than the same MERs. Thus, the merge candidate blocks of block PU1-1000 can be used to derive candidate motion prediction information. In this case, candidate motion prediction information may not be derived to the merge candidate block of the lower layer block.

On the other hand, in the upper layer block PU1-1 1010, the A0 merge candidate block and the A1 merge candidate block exist in the same MER. In this case, candidate motion prediction information may not be derived from the A0 merge candidate block and the A1 merge candidate block. Therefore, the A0 merge candidate block and the A1 merge candidate block may be determined to be unavailable blocks. In this case, the merge candidate blocks A0 'and M1' merge candidate blocks, which are the merge candidate blocks of the blocks PU0-1 1015, which are the corresponding lower layer blocks of the block PU1-1 1010 of the upper layer, are block PU1-1 1010. It may be used as a block for deriving candidate motion prediction information instead of the A0 merge candidate block and the A1 merge candidate block.

As another example, in the case of the block PU1-2 1020 of the upper layer, the A0 merge candidate block, the A1 merge candidate block, the B0 merge candidate block, the B1 merge candidate block, and the B2 merge candidate block are present in the same MER or are not yet decoded. May be present in a non-MER. Accordingly, the A0 merge candidate block, the A1 merge candidate block, the B0 merge candidate block, the B1 merge candidate block, and the B2 merge candidate block may be determined to be unavailable blocks for deriving candidate motion prediction information. In this case, A0 'merge candidate block, A1' merge candidate block, B0 'merge candidate block, and B1', which are merge candidate blocks of block PU0-2 1025, which are lower layer blocks of block PU1-2 1020 of the upper layer. Merge candidate block, B2 'merge candidate block to merge candidate motion prediction information instead of A0 merge candidate block, A1 merge candidate block, B0 merge candidate block, B1 merge candidate block, and B2 merge candidate block in block PU1-2 Can be used as a candidate block.

If the resolution of the upper layer image is different from the resolution of the lower layer image, the motion prediction information may be referred to after correction of the motion prediction information based on the ratio using each resolution.

11 is a conceptual diagram illustrating a merge motion prediction method according to an embodiment of the present invention.

In FIG. 11, when some merge candidate blocks of merge candidate blocks are not available when performing merge motion prediction, in SHVC, motion information of a corresponding lower layer block of the current block may be used as candidate prediction motion information.

For example, in the upper layer block PU1-1 1110 in FIG. 11, the A0 merge candidate block and the A1 merge candidate block exist in the same MER. In this case, candidate motion prediction information may not be derived from the A0 merge candidate block and the A1 merge candidate block. Therefore, the A0 merge candidate block and the A1 merge candidate block may be determined to be unavailable blocks. In this case, the motion prediction information of the block PU0-1 1115, which is the corresponding lower layer block of the block PU1-1 1110 of the upper layer, may be used as the candidate motion prediction information.

Instead of the motion prediction information derived based on the A0 merge candidate block and the A1 merge candidate block, the motion prediction information of the lower layer block corresponding to the current block may be used. therefore. Instead of the motion prediction information derived based on the A0 merge candidate block and the A1 merge candidate block, motion prediction information of the corresponding lower layer block may be added as candidate motion prediction information to the final motion candidate list. If the resolution of the upper layer image is different from the resolution of the lower layer image, the motion prediction information may be referred to after correction of the motion prediction information based on the ratio using each resolution.

As another example, in the case of the block PU1-2 1120 of the upper layer, the A0 merge candidate block, the A1 merge candidate block, the B0 merge candidate block, the B1 merge candidate block, and the B2 merge candidate block are present in the same MER or are not yet decoded. May be present in a non-MER. Accordingly, the A0 merge candidate block, the A1 merge candidate block, the B0 merge candidate block, the B1 merge candidate block, and the B2 merge candidate block may be determined to be unavailable blocks for deriving candidate motion prediction information. In this case, the merge motion prediction of the higher layer block PU1-2 1120 is performed using the motion prediction information of the block PU0-2 1125, which is the lower layer block of the block PU1-2 1120 of the higher layer, as the candidate motion prediction information. Can be performed.

12 is a conceptual diagram illustrating a merge motion prediction method according to an embodiment of the present invention.

12 illustrates a method of referring to motion information from a corresponding lower layer block when there is a merge candidate block determined to be unavailable when performing motion prediction based on a PME in a block of an upper layer.

Referring to FIG. 12, unlike any of the methods of FIGS. 10 and 11 described above, if any merge candidate block of the merge candidate blocks of the current block is set to be unavailable, the candidate prediction motion information of the current block may be set to the corresponding lower layer block. It can be derived from the motion prediction information.

In FIG. 12, when the current block is PU1-1 1210, A0 merge candidate blocks and A1 merge candidate blocks are determined to be unusable blocks among the merge candidate blocks, and all remaining merge candidate blocks are non-unavailable. It may be determined as a block.

According to an embodiment of the present invention, since the A0 merge candidate block and the A1 merge candidate block are not available, the motion prediction is performed from the corresponding lower layer block PU0-1 1215 without deriving candidate motion prediction information from the merge candidate block of the current block. Based on the information, motion prediction for the current block can be performed. If the resolution of the upper layer image is different from the resolution of the lower layer image, it may be referred to after correcting motion information based on the ratio using each resolution.

The methods described above with reference to FIGS. 10 to 12 may all set an application range differently according to a block size or a CU depth. The variable that determines the scope of application (ie, size or depth information) may be set so that the encoder and decoder use a predetermined value, or may cause the encoder to use a predetermined value according to a profile or level, and the encoder In the bitstream, the decoder may obtain and use this value from the bitstream. When varying the application range according to the CU depth, as shown in Table 1 below, apply only to a depth above a given depth (method A), apply only to a given depth below (method B), and apply only to a given depth. Can be used (method C).

Table 1 exemplarily shows a range determination method for applying the methods of the present invention when a given CU depth is two.

TABLE 1

In Table 1, 'O' indicates that the merge motion prediction method according to the embodiment of the present invention is used at the corresponding depth, and 'X' indicates that the merge motion prediction method according to the embodiment of the present invention is not applied at the corresponding depth. do.

When the methods of the present invention are not applied to all depths, they may be represented by using an arbitrary flag, or may be represented by signaling a value greater than one of the maximum value of the CU depth as a CU depth value indicating an application range. have.

In addition, the above-described method may be differently applied to the chrominance block according to the size of the luminance block, and may be differently applied to the luminance signal image and the chrominance image.

Table 2 below shows whether the merge motion prediction method is applied according to the size of the luminance block and the size of the chrominance block.

TABLE 2

Referring to the method “I 1” in Table 2, when the size of the luminance block is 8 (8x8, 8x4, 2x8, etc.), and the size of the color difference block is 4 (4x4, 4x2, 2x4), the present invention is implemented. The merge motion prediction method according to an example may be applied to a luminance signal, a color difference signal, a horizontal signal, and a vertical signal.

As another example, the method “wave 2” of the modified methods is a case where the size of the luminance block is 16 (16x16, 8x16, 4x16, etc.), and the size of the color difference block is 4 (4x4, 4x2, 2x4). The merge motion prediction method according to the embodiment of the present invention may be applied to the luminance signal, the color difference signal, and the horizontal signal, but not to the vertical signal.

In another modified method, the merge motion prediction method according to the embodiment of the present invention may be applied only to the luminance signal and not to the chrominance signal. On the contrary, the merge motion prediction method according to the embodiment of the present invention may be applied only to the color difference signal and not to the luminance signal.

By using the merge motion prediction method described above with reference to FIGS. 1 to 12, not only a better prediction efficiency can be obtained than the existing inter prediction method, but also a better efficiency in terms of encoding time and decoding time required for encoding and decoding. have. In addition, by using this intra-screen prediction method, if it is necessary to have better prediction efficiency according to the profile, it can be used as an additional intra-screen prediction mode. By using different intra-screen prediction methods according to the implementation device where the codec is used, the degree of freedom of implementation Can raise. In addition, the above-described image encoding and image decoding method may be implemented in each component of each of the image encoder and the image decoder apparatus described above with reference to FIGS. 1 and 2.

Although described with reference to the embodiments above, those skilled in the art will understand that the present invention can be variously modified and changed without departing from the spirit and scope of the invention as set forth in the claims below. Could be.

Claims (20)

In a parallel merge motion estimation (PME) method of an image supporting multiple layers,
Determining whether there is a parallel merge candidate block that is not available around an encoding target block existing in a parallel merge motion prediction range (MER) of an upper layer;
If the unavailable parallel merge candidate block exists in the current block existing within the parallel merge motion prediction range, candidate motion for the parallel merge motion prediction based on a corresponding lower layer block of the unavailable parallel merge candidate block. Deriving prediction information; And
The parallel merge motion prediction for the current block is performed using at least one candidate motion prediction information from candidate motion prediction information derived from a corresponding lower layer block and candidate motion prediction information of parallel merge candidate blocks available around the current block. Include steps to be performed,
And wherein the unavailable parallel merge candidate block includes a parallel merge candidate block included in the same parallel merge motion prediction range as the current block, or a decoded or uncoded parallel merge candidate block. The method of claim 1,
And the corresponding lower layer block is a collocated block of the current block existing in a lower layer than the layer including the current block. The method of claim 1,
And determining whether to perform the parallel merge motion prediction on the current block based on the size of the current block and the depth information of the coding block including the current block. The method of claim 3,
The current block is included in an enhanced layer, and the corresponding lower layer block is included in a base layer. The method of claim 1,
And the candidate motion prediction information is motion prediction information of the corresponding lower layer block. The method of claim 1,
The corresponding lower layer block is a collocated block of unavailable parallel merge candidate blocks around the current block that exists in a lower layer than a layer including the unavailable parallel merge candidate blocks around the current block. . The method of claim 1,
When the resolutions of the upper layer and the lower layer of the image supporting the plurality of layers are different from each other, the motion information of the lower layer is corrected based on a ratio of the resolution of the upper layer and the resolution of the lower layer, And using the corrected lower layer motion information as candidate motion prediction information for parallel merge motion prediction for the current block. In the image decoding apparatus supporting multiple layers, the decoding apparatus includes a motion compensation unit,
The motion compensator determines whether an unavailable parallel merge candidate block exists around an encoding target block existing in a parallel merge motion prediction range (MER) of an upper layer,
If the unavailable parallel merge candidate block exists in the current block existing within the parallel merge motion prediction range, candidate motion for the parallel merge motion prediction based on a corresponding lower layer block of the unavailable parallel merge candidate block. Derive prediction information,
The parallel merge motion prediction for the current block is performed using at least one candidate motion prediction information from candidate motion prediction information derived from a corresponding lower layer block and candidate motion prediction information of parallel merge candidate blocks available around the current block. Implemented to do so,
The unavailable parallel merge candidate block includes a parallel merge candidate block included in the same parallel merge motion prediction range as the current block, or a decoded or uncoded parallel merge candidate block. The method of claim 8,
And the corresponding lower layer block is a collocated block of the current block existing in a lower layer than a layer including the current block. The method of claim 8, wherein the motion compensation unit,
And determine whether to perform the parallel merge motion prediction on the current block based on the size of the current block and depth information of a coding block including the current block. The method of claim 10,
The current block is included in an enhanced layer, and the corresponding lower layer block is included in a base layer. The method of claim 8,
And the candidate motion prediction information is motion prediction information of the corresponding lower layer block. The method of claim 8,
And the corresponding lower layer block is a collocated block of unavailable parallel merge candidate blocks around the current block that exists in a lower layer than a layer including an unavailable parallel merge candidate block around the current block. The method of claim 8,
The motion compensator compares the resolutions of the upper layer and the lower layer of the image supporting the plurality of layers, and when they are different, the motion information of the lower layer based on a ratio of the resolution of the upper layer and the resolution of the lower layer. And correct the motion information of the corrected lower layer as parallel merge motion prediction information for the current block. In a parallel merge motion estimation (PME) method of an image supporting multiple layers,
Determining whether there is a parallel merge candidate block that is not available around an encoding target block existing in a parallel merge motion prediction range (MER) of an upper layer; And
If at least one non-available parallel merge candidate block exists in a current block existing within the parallel merge motion prediction range, applying motion information of a corresponding lower layer block as motion prediction information for the current block; Including,
And wherein the unavailable parallel merge candidate block includes a parallel merge candidate block included in the same parallel merge motion prediction range as the current block, or a decoded or uncoded parallel merge candidate block. The method of claim 15,
And the corresponding lower layer block is a collocated block of the current block existing in a lower layer than the layer including the current block. The method of claim 15,
When the resolutions of the upper layer and the lower layer of the image supporting the plurality of layers are different from each other, the motion information of the lower layer is corrected based on a ratio of the resolution of the upper layer and the resolution of the lower layer, And using the corrected lower layer motion information as motion prediction information for the current block. In the image decoding apparatus supporting multiple layers, the decoding apparatus includes a motion compensation unit,
The motion compensator determines whether an unavailable parallel merge candidate block exists around an encoding target block existing in a parallel merge motion prediction range (MER) of an upper layer,
When at least one non-available parallel merge candidate block exists in a current block existing within the parallel merge motion prediction range, the motion information of a corresponding lower layer block is applied to the motion prediction information for the current block. Including parallel merging motion prediction devices,
The unavailable parallel merge candidate block includes a parallel merge candidate block included in the same parallel merge motion prediction range as the current block, or a decoded or uncoded parallel merge candidate block. The method of claim 18,
And the corresponding lower layer block is a collocated block of the current block existing in a lower layer than a layer including the current block. The method of claim 18,
The motion compensator compares the resolutions of the upper layer and the lower layer of the image supporting the plurality of layers, and when they are different, the motion information of the lower layer based on a ratio of the resolution of the upper layer and the resolution of the lower layer. And correct the motion information of the corrected lower layer as motion prediction information for the current block.

Images