RU2784483C1 - Method for image decoding, method for image encoding and machine-readable information carrier - Google Patents

Abstract

FIELD: image encoding/decoding.

SUBSTANCE: group of inventions relates to technologies of encoding/decoding of three-dimensional images. A method for image decoding is proposed. The method contains a stage of determination of movement information related to a current block. Next, according to the method, an elementary value of prediction for the current block is determined based on movement information related to the current block. Moreover, the stage of determination of movement information related to the current block includes: determination of whether a central subblock corresponding to a central position of the current block has movement information in a reference block.

EFFECT: increase in the efficiency of image encoding/decoding by means of elimination of dependence on data, when determining movement information.

5 cl, 37 dwg, 5 tbl

Description

This application is separated from application No. 2016125782 for the grant of a patent of the Russian Federation for an invention, filed on 01/05/2015, claiming priority by the filing date of application KR 10-2014-0000527, filed on 01/03/2014, application KR 10-2014-0001531, filed on 01/06/2014 , and application KR 10-2015-0000578 filed on 01/05/2015.

The field of technology to which the invention belongs

The present invention relates to devices and methods for encoding/decoding three-dimensional images, and more specifically, to methods and devices for encoding/decoding images in which inter-view motion information is determined in parallel over a plurality of prediction sub-elements.

State of the art

In the growing information technology industry, HD (high definition) broadcasting services have become ubiquitous, and HD video viewing is becoming commonplace for more and more users.

Accordingly, users are demanding higher quality and higher resolution images, and to meet these expectations, several organizations are racing to develop the next generation of imaging devices as soon as possible. As a result, users have already been able to experience systems capable of handling FHD (Full HD) and UHD (Ultra HD) images.

However, user needs go beyond what has been achieved and go a step further towards three-dimensional (3D) images that can create a sense or effects of three-dimensionality. To meet these user needs, various organizations have created systems for working with three-dimensional images.

However, 3D images contain more data than 2D images, since they contain not only the image itself (texture), but also depth map information. Accordingly, encoding/decoding of 3D images using existing image encoding/decoding methods is not efficient enough.

Disclosure of invention

An object of the present invention is to propose an apparatus and a method for determining motion information related to a block for which encoding/decoding is performed.

Yet another object of the present invention is to provide an apparatus and a method for eliminating data dependency in determining motion information related to a block for which encoding/decoding is performed.

Yet another object of the present invention is to provide an apparatus and a method for improving image encoding/decoding efficiency by eliminating data dependency in determining motion information related to a block for which encoding/decoding is performed from prediction sub-elements.

Yet another object of the present invention is to provide an apparatus and a method for improving image encoding/decoding efficiency by using motion information related to a reference block in determining motion information related to a block to be coded/decoded from prediction sub-elements.

Technical solution

In accordance with an embodiment of the present invention, a three-dimensional (3D) image encoding method can be provided, including:

determining an inter prediction mode as a prediction mode for the current block;

determining whether the reference block corresponding in the reference frame to the current block has motion information;

determining motion information related to the current block for each prediction subblock in the current block if the reference block has motion information; and

determining a prediction chip value for the current block based on motion information related to the current block.

Here, the current block and the reference block may be prediction blocks.

Here, motion information related to the reference block may be located at the center of the reference block.

Wherein, in the step of determining the motion information related to the current block, for each prediction subblock in the current block, if the prediction subblock in the reference block corresponding to the prediction subblock in the current block has motion information, then the motion information related to the prediction subblock of the current block, can be defined as motion information present in the reference block prediction subblock.

If the prediction subblock in the reference block corresponding to the prediction subblock in the current block has no motion information, motion information related to the prediction subblock of the current block is determined as motion information of the reference block.

According to yet another embodiment of the present invention, a three-dimensional (3D) image coding apparatus can be provided, comprising a storage module configured to determine an inter-frame prediction mode as the prediction mode for a current block, and capable of determining whether a reference block has a corresponding reference frame to the current block, motion information; and a determining unit, configured, if the reference block has motion information, to determine motion information related to the current block for each prediction subblock in the current block, and to determine a prediction chip value for the current block based on the motion information related to the current block. block.

Here, the current block and the reference block may be prediction blocks.

Here, motion information related to the reference block may be located at the center of the reference block.

Here, in the determining unit, if the prediction sub-block in the reference block corresponding to the prediction sub-block in the current block has motion information, then the motion information related to the prediction sub-block of the current block can be determined as the motion information present in the prediction sub-block of the reference block.

If the prediction subblock in the reference block corresponding to the prediction subblock in the current block has no motion information, then the motion information related to the prediction subblock of the current block can be determined as the motion information of the reference block.

According to yet another embodiment of the present invention, a method for decoding a three-dimensional (3D) image can be provided, including: determining an inter-frame prediction mode as a prediction mode for a current block; determining whether the reference block corresponding in the reference frame to the current block has motion information; determining motion information related to the current block for each prediction subblock in the current block if the reference block has motion information; and determining a prediction chip value for the current block based on motion information related to the current block.

Here, the current block and the reference block may be prediction blocks.

Here, motion information related to the reference block may be located at the center of the reference block.

Wherein, in the step of determining the motion information related to the current block, for each prediction subblock in the current block, if the prediction subblock in the reference block corresponding to the prediction subblock in the current block has motion information, then the motion information related to the prediction subblock of the current block, can be defined as motion information present in the reference block prediction subblock.

If the prediction sub-block in the reference block corresponding to the prediction sub-block in the current block has no motion information, the motion information related to the prediction sub-block of the current block can be determined as motion information of the reference block.

According to yet another embodiment of the present invention, a three-dimensional (3D) image decoding apparatus can be provided, comprising a storage module configured to determine an inter-frame prediction mode as the prediction mode for a current block, and capable of determining whether a reference block has a corresponding reference frame to the current block, motion information; and a determining unit, configured, if the reference block has motion information, to determine motion information related to the current block for each prediction subblock in the current block, and to determine a prediction chip value for the current block based on the motion information related to the current block. block.

Here, the current block and the reference block may be prediction blocks.

Here, motion information related to the reference block may be located at the center of the reference block.

Here, in the determining unit, if the prediction subblock in the reference block corresponding to the prediction subblock in the current block has motion information, then the motion information related to the prediction subblock of the current block can be determined as motion information present in the reference block prediction subblock.

If the prediction subblock in the reference block corresponding to the prediction subblock in the current block has no motion information, then the motion information related to the prediction subblock of the current block can be determined as the motion information of the reference block.

Technical result

The present invention makes it possible to determine motion information related to a block for which encoding/decoding is performed.

The present invention makes it possible to eliminate data dependency when determining motion information related to a block for which encoding/decoding is performed.

The present invention makes it possible to improve the efficiency of image encoding/decoding by eliminating the data dependency in determining motion information related to a block for which encoding/decoding is performed by prediction sub-elements.

The present invention makes it possible to improve the efficiency of encoding/decoding an image using motion information related to a reference block by eliminating data dependency in determining motion information related to the block for which encoding/decoding is performed on prediction subelements.

Brief description of the drawings

Fig. 1 is a schematic illustration of the basic structure of a 3D video system.

Fig. 2 is an example of an image of balloons and an example of an image of a depth information map.

Fig. 3 is a schematic illustration of the structure into which an image is partitioned during encoding and decoding.

Fig. 4 illustrates prediction elements that may be contained in a coding unit (EC).

Fig. 5 illustrates an example of an inter-view prediction structure in a 3D codec.

Fig. 6 illustrates an example of the operation of encoding and/or decoding the image itself (texture view) and the depth information map (depth view) in a 3D image encoder and/or decoder.

Fig. 7 is a block diagram illustrating the configuration of an image encoder according to an embodiment of the present invention.

Fig. 8 is a block diagram illustrating the configuration of an image decoder according to an embodiment of the present invention.

Fig. 9 is an illustration of an example of a prediction structure for a 3D codec.

Fig. 10 illustrates an example where adjacent blocks are used to build a list of merge candidates for the current block.

Fig. 11 is an illustration of an example of an operation of determining motion information related to the current block using motion information from an adjacent view.

Fig. 12 is an illustration of an example in which one prediction element (PE) is split into multiple prediction sub-elements.

Fig. 13 is an illustration of an example of an operation of determining motion information related to the current block using a reference block.

Fig. 14 is an illustration of an example of a reference block used to determine motion information related to the current block.

Fig. 15a to 15e are schematic illustrations of an example of motion information determination operation using motion information stored in the storage.

Fig. 16a-16g are schematic illustrations of yet another example of motion information determination operation using motion information stored in the storage.

Fig. 17 is a flowchart illustrating a method for determining motion information related to a prediction sub-element of a current block using a prediction sub-element of a reference block in accordance with an embodiment of the present invention.

Fig. 18 is an illustration of an example of an operation of parallel determination of motion information related to the prediction sub-element of the current block using the reference block prediction sub-element.

Fig. 19 is an illustration of an example operation of searching for a usable prediction sub-element when the prediction sub-element is located in the lower right corner of the reference block.

Fig. 20 is a schematic illustration of the time required to determine motion information by prediction sub-elements.

Fig. 21 is a block diagram illustrating the configuration of an inter prediction unit according to the present invention.

Fig. 22 is a flowchart schematically illustrating a method for determining motion information related to a prediction subelement of a current block using a reference block according to an embodiment of the present invention.

Fig. 23 is a flowchart schematically illustrating a method for determining motion information related to a prediction sub-element of a current block, according to another embodiment of the present invention.

Fig. 24 is an illustration of an example of an operation of determining motion information related to a prediction sub-element of the current block using motion information from a position.

Fig. 25 is a flowchart illustrating a method for determining motion information related to a prediction sub-element of the current block using a certain value of motion information in accordance with yet another embodiment of the present invention.

Fig. 26 is an illustration of an example of an operation of determining motion information related to a prediction sub-element of the current block using some motion information.

Fig. 27 is a schematic illustration of the time required to determine motion information in accordance with the present invention.

Implementation of the invention

Next, with reference to the accompanying drawings, embodiments of the present invention are described in detail. In order to more clearly disclose the present invention, a detailed description of known systems or functions relevant to the present invention is not given.

If a component is said to be connected or connected to another component, then this means that this component can be connected or connected to another component either directly or through other components. If this document says that the present invention contains or includes a certain component, then this means that the present invention does not exclude the presence of other components, and it is possible that additional components are also present in the technical essence of the present invention or in embodiments of the present invention.

The terms "first" and "second" may be used to describe the various components, but these terms are not limited in any way. These terms are only used to distinguish one component from another. For example, without departing from the spirit of the present invention, the first component may be referred to as the second component, and the second component may be referred to as the first component.

In this document, components may be shown as independent for the purpose of representing their respective individual characteristics, but this does not mean that each component must be a separate hardware or software element. In other words, for ease of description, the components are shown separately. At least two components may be combined to form one component to perform a function, or any component may be divided into multiple components. Such combination or separation, without being a departure from the idea of the present invention, is also within the scope of protection of the present invention.

Some components may not be essential components to perform the essential functions of the present invention, but optional components that serve to improve performance. Only the essential components that implement the idea of the present invention are sufficient for the implementation of the present invention, and the components used to improve the quality of operation can be omitted, and such a configuration is also within the scope of the present invention.

A 3D image on a 3D stereoscopic display creates a stereoscopic effect similar to what the user sees and feels in the real world. In this regard, the JCT-3V group (The Joint Collaborative Team on 3D Image Coding Extension Development) from the MPEG (Moving Picture Experts Group) at ISO / IEC (International Organization for Standardization - International Electrotechnical Commission) and the VCEG (Video Coding Experts Group ) in the ITU-T (International Telecommunication Union - Telecommunication Standardization Sector) are working on developing standards for three-dimensional images.

Fig. 1 is a schematic illustration of the basic structure of a 3D imaging system.

As shown in FIG. 1, a 3D video (3DV) system may include a transmitter and a receiver. Meanwhile, the 3D video system shown in FIG. 1 may be a basic 3D video system represented by 3D imaging standards, which may include standards for advanced data formats and related technologies that may support the reproduction of autostereoscopic images as well as stereoscopic images using a texture and its corresponding map. depth information.

The transmitter may generate information material with an image containing a plurality of angles. Specifically, at the transmission side, using a stereo camera and a multi-angle camera, image information can be generated, and using a depth information camera, a depth information map (or depth view) can be generated. The transmitter may convert the 2D image to a 3D image using a converter. Using the generated image information and the depth information map, the transmitter can generate information material with an image containing N views (N ≥ 2), i.e. with multi-angle image. Meanwhile, the N-view image information material may contain image information, depth map information of the image, and additional information related to the camera. Said N-view image information material may be compressed by a 3D image encoder using a multi-view image coding scheme, and the compressed image information material (bitstream) may be transmitted via a network to a receiver terminal.

The receiver is configured to decode information material with an image received from the transmitter and to output a multiview image. Specifically, an image decoder included in the receiver (for example, a 3D image decoder, a stereoscopic image decoder, or a 2D image decoder) for the purpose of recovering an N-view image from a bitstream is configured to decode the received bitstream using a multi-view image decoding scheme. At the same time, the image decoder is configured to generate N (or more)-view virtual images using the reconstructed N-view image and a rendering operation based on image depth information (depth image-based rendering, DIBR). The generated N (or more)-angle virtual images are displayed by various 3D displays (eg, N-angle display, stereoscopic display, or 2D display) that give the user a 3D experience.

Fig. 2 is an example of an image of balloons and an example of an image of a depth information map. In FIG. 2(a) shows an image of balloons that is commonly used in the MPEG (International Standards Organization) 3D image coding standard. In FIG. 2(b) shows an image of a depth information map corresponding to the image of balloons shown in FIG. 2(a). This depth information map image is obtained by visualizing depth information by displaying its representation with eight bits per pixel encoding on a screen.

The depth information map is used to generate virtual view images and is constructed by representing the distance between the camera and the real object in real space (the depth information corresponding to each pixel at the same resolution as the texture) by a predetermined number of bits. In this case, the depth information map can be built using a camera to obtain a depth information map or using the actual conventional image (texture).

The depth information map that is built using the depth information map camera for stationary objects or scenes provides the most reliable depth information, however, this camera is only operable over a predetermined distance interval. To overcome this shortcoming, the camera can use a measurement scheme using a laser beam or structured light, or a time-of-flight (TFL) measurement scheme to obtain a depth information map.

The depth information map can also be generated using the normal image (texture) itself and the residual vector. The residual vector is information representing the difference between the views represented by two regular images. The residual vector can be obtained by comparing a pixel from the current view and pixels from other views to find the pixel that is most similar to the pixel from the current view and measure the distance between the pixel from the current view and that most similar pixel.

A texture and its depth information map may be an image(s) taken by one or more cameras. Images acquired by multiple cameras may be independently encoded and a generic codec for encoding/decoding 2D images may be used to encode/decode them. Views in images taken by multiple cameras are correlated, and for this reason, in order to improve coding efficiency, they may be coded using prediction using multi-view prediction (inter-view prediction).

Fig. 3 is a schematic illustration of the structure into which an image is partitioned during encoding and decoding.

With rational partitioning, the image can be encoded and decoded separately in each coding element (EC). The term "element" means a block containing a syntactic element and elementary image values (image element values, samples, samples). The expression "element is broken" can mean that the block corresponding to the specified element is broken.

As shown in FIG. 3, the picture 300 is sequentially partitioned into largest bins (LCEs) and a partition structure of each LEC is determined. In the present document, BCEC can be understood as a coding tree element (EDC). The partitioning structure can be understood to mean the placement of coding units (ECs) in each of the BCSs 310 in order to rationally encode the image; this placement can be determined by whether one CI is split into four CIs, each of which is half the size of the original CI in both the horizontal and vertical directions. In the same way, the EC obtained as a result of splitting can also be sequentially divided into four ECs, each of which is half the size of the original EC both in the horizontal and vertical directions.

The splitting of the coding element may be performed sequentially up to a predetermined splitting depth. Depth information refers to information indicating the size of the CI; this information can be stored for each CI. For example, the split depth of the SBEC may be equal to 0, and the split depth of the smallest coding unit (SMEC) may be equal to a predetermined largest split depth. Here, SBEC is the largest coding unit as above, and CMEC is the smallest coding unit.

Whenever the LCU 310 is bisected both horizontally and vertically, the split depth of that CI is incremented by one. For example, if at a certain depth L of partitioning, the size of the EC is 2Nx2N, then the size of the EC resulting from splitting this EC is reduced to NxN. In this case, the partitioning depth of such an EC with the size NxN is equal to L+1. In other words, the value of N, corresponding to the size of the EC, decreases by half with each increase in the partitioning depth by one.

As shown in FIG. 3, the size of the largest coding unit (SBEC) corresponding to the smallest bin depth of 0 may be 64×64 pixels, and the size of the smallest coding unit (SBEC) corresponding to the smallest tiling depth of 3 may be 8×8 pixels. . In this case, the splitting depth of the 64×64 pixel EC (SBEC) can be set equal to 0, the 32×32 pixel EC equal to 1, the 16×16 pixel EC equal to 2, and the 8×8 pixel EC (SMEC) equal to 3.

Information about whether a particular CI is split can be represented by one-bit split information of the given CI. This split information can be contained in all but the smallest CIs. For example, if a CI is not split, then 0 can be stored in the split information of this CI, and if it is split, then 1 can be stored in the split information of this CI.

Fig. 4 illustrates prediction elements that may be contained in a coding unit (EC).

A bin of coding units obtained by splitting the SBEC that is not subject to further splitting may be split or further divided into one or more prediction bins.

The prediction element (hereinafter EP) is the basic element in which the prediction is performed. The prediction element is encoded and decoded in skip mode, inter-frame mode, or intra-frame mode. Depending on the mode, the prediction element can be split in various ways, shown in FIG. four.

In the skip mode, it is possible to support mode 410, in which the ES has a size of 2Nx2N, the same as that of an unsplit EC.

In interframe mode, eight types of CIs can be supported, such as 410 2Nx2N mode, 415 2NxN mode, 420 Nx2N mode, 425 NxN mode, 430 2NxnU mode, 435 2NxnD mode, 440 nLx2N mode, and 445 NRx2N mode.

In intra frame mode, 410 2Nx2N mode and 425 NxN mode can be supported.

Fig. 5 illustrates an example of an inter-view prediction structure in a 3D codec.

Inter-view prediction for view 1 and view 2 can be performed using view 0 as a reference picture, but view 0 must be encoded before view 1 and view 2.

In this case, view 0 can be encoded independently of other views, so view 0 is called an independent view. View 1 and view 2, which must use view 0 as a reference image, are called dependent views. An independent-view image may be encoded using a conventional 2D image codec. Dependent-view images must be subject to inter-view prediction, so these views may be encoded using a 3D image codec capable of inter-view prediction.

To improve coding efficiency, view 1 and view 2 may be encoded using a depth information map. For example, the texture and the depth information map when encoding can be encoded and/or decoded independently of each other. In addition, the texture and the depth information map when encoding can be encoded and/or decoded depending on each other, as shown in FIG. 6.

Fig. 6 illustrates an example of the operation of encoding and/or decoding the image itself (texture view) and the depth information map (depth view) in a 3D image encoder and/or decoder.

As shown in FIG. 6, the 3D image encoder may include a texture encoder (texture encoder) for encoding the actual image (texture view) and a depth information map encoder (depth encoder) for encoding a depth information map (depth view).

At the same time, the texture encoder is configured to encode the texture using the depth information map encoded by the depth information map encoder. At the same time, the depth information map encoder is configured to encode the depth information map using the texture encoded by the texture encoder.

The 3D image decoder may comprise an image proper decoder (texture decoder) for texture decoding and a depth information map decoder for decoding a depth information map.

Wherein, the texture decoder is configured to decode the texture using the depth information map decoded by the depth information map decoder. At the same time, the depth information map decoder is configured to decode the depth information map using the texture decoded by the texture decoder.

Fig. 7 is a block diagram illustrating the configuration of an image encoder according to an embodiment of the present invention.

Fig. 7 illustrates an example of an image encoder applicable to a multi-view structure, which can be implemented by modifying the single-view image encoder. Meanwhile, the image encoder shown in FIG. 7 may be used in a texture encoder and/or a depth information map encoder as shown in FIG. 6, and this encoder may be understood as an encoder.

As shown in FIG. 7, the image encoder 700 includes an inter prediction module 710, an intra prediction module 720, a switch 715, a subtractor 725, a transform module 730, a quantization module 740, an entropy encoding module 750, a dequantization module 760, an inverse transform module 770, an adder 775, a filter 780, and buffer 790 reference frame.

The image encoder 700 is configured to encode the input image in an intra frame mode or in an inter frame mode with the formation of a bit stream.

Intra-frame prediction refers to prediction on a single frame, and inter-frame prediction refers to prediction on different frames or different angles. Intra-frame or inter-frame mode is set by switch 715.

The image encoder 700 is configured to generate a prediction block for a block of the input frame (the current block) and then encode the difference between the current block and this prediction block.

The intra prediction module 720 is configured to intra frame use as its reference pixel a pixel value from an already encoded block adjacent to the current block. The intra prediction module 720 is configured to generate prediction chips for the current block using this reference pixel.

The inter-frame prediction module 710 is configured to obtain, in inter-frame mode, a motion vector indicating a reference block corresponding to an input block (current block) in the reference frame stored in the reference frame buffer 790 . The inter-frame prediction module 710 is configured to generate a prediction block for the current block by performing motion compensation using a reference frame stored in the reference frame buffer 790 and a specified motion vector.

In the multi-view structure, the inter-picture prediction applied in the inter-frame mode may comprise the inter-view prediction. The inter-frame prediction module 710 is configured to construct an inter-view reference frame by selecting chip values from the reference-view frame. The inter-frame prediction module 710 is configured to perform inter-view prediction using a reference frame list containing the specified inter-view reference frame. The relative relationship between views may be communicated by information indicating the dependency of one view on another view.

If the current-view frame and the reference-view frame have the same dimensions, then the selection of elementary values applied to the reference-view frame can be understood as the generation of the reference elementary value by copying the elementary value from the reference-view frame or by interpolation in the reference-view frame. If the frame of the current view and the frame of the reference view have different sizes, then the selection of elementary values applied to the frame of the reference view can be understood as the selection of elementary values with upscaling or downscaling. For example, if the views have different resolutions, then to build an inter-view key frame, the selection of elementary values from the restored frame of the reference view with upscaling can be performed.

The view from which a frame is to be used to construct an inter-view reference frame may be determined in view of, for example, encoding cost. The encoder may send to the decoder information indicating the view to which the frame to be used as the inter-view reference frame belongs.

The frame used to predict the current block in the view that is being inter-viewed - that is, in the reference view - may be the same as the frame of the same Access Element (ED) as the current frame (the frame in the current view for which prediction is made).

Subtractor 725 is configured to generate a residual block (residual signal) by calculating a difference between the current block and the prediction block.

Transform module 730 converts the residual block into a transform coefficient. In the transform skip mode, transform module 730 may skip the specified residual block transform.

Quantization unit 740 quantizes the specified transform coefficient, generating a quantized coefficient in accordance with the quantization parameters.

The entropy encoding unit 750 entropy encodes the values obtained by the quantization unit 740 or the encoding parameters obtained during the encoding process into a bitstream according to a probability distribution. Entropy encoding unit 750 is also configured to entropy encode, in addition to image pixel information, information (eg, syntax element) for image decoding.

Said encoding parameters, as information necessary for encoding and decoding, may contain information that can be determined during encoding or decoding, as well as information such as, for example, a syntax element encoded by an encoder and transmitted to a decoder.

The residual signal can be understood as the difference between the original signal and the predicted signal, the signal obtained by transforming the difference between the original signal and the predicted signal, or the signal obtained by transforming the difference between the original signal and the predicted signal and then quantizing the transformed difference. When considered at the block level, the residual signal may be referred to as a residual block.

When entropy coding is applied, characters can be represented such that a character with a higher probability of being present is encoded with fewer bits, and another character with a lower probability of being present is encoded with more bits, thus the amount of bitstream can be reduced for the characters to be encoded. . Thus, encoding an image using entropy encoding makes it possible to obtain better compression.

The coding scheme used in entropy coding can be, for example, exponential Golomb, context-adaptive variable length coding (CAVLC), or context-adaptive binary arithmetic coding (CABAC, context-adaptive binary). arithmetic coding). For example, entropy encoding unit 750 may perform entropy encoding using a variable length coding/code coding table (VLC tables). The entropy encoding unit 750 is configured to select a binning method, a target symbol, and a probability model of the specified target symbol/slot, and is configured to implement entropy encoding using the selected binning method and the probability model.

The quantized coefficient may be dequantized by dequantization module 760 and may be inversely transformed by inverse transform module 770 . An adder 775 adds the dequantized and inversely transformed coefficient to the prediction block, resulting in a reconstructed block.

The reconstructed block passes through filter 780. At filter 780, at least one of the following filters may be applied to the reconstructed block or frame: a deblocking filter; a sample adaptive offset (SAO) filter and an adaptive loop filter (ALF). The reconstructed block after passing through the filter 780 may be stored in the buffer 790 of the reference frame.

Fig. 8 is a block diagram illustrating the configuration of an image decoder according to an embodiment of the present invention. Fig. 8 illustrates an example of an image decoder applicable to a multi-view structure, which can be implemented by modifying the single-view image decoder.

The picture decoder shown in FIG. 8 may be used in the texture decoder and/or the depth information map decoder shown in FIG. 6. For ease of description in this document, the terms "decryption" and "decoding" can be used interchangeably; the terms "decoder" and "decoder" may also be used interchangeably.

As shown in FIG. 8, the image decoder 800 includes an entropy decoding unit 810, a dequantization unit 820, an inverse transform unit 830, an intra prediction unit 840, an inter prediction unit 850, a filter 860, and a reference frame buffer 870.

The image decoder 800 is configured to receive a bitstream from the encoder, decode the bitstream either intra-frame or inter-frame, and output a reconstructed image, i. restored image.

Intra-frame or inter-frame mode can be selected with a switch.

The image decoder 800 is configured to obtain a residual block recovered from the received bitstream, to generate a prediction block, and to add the recovered residual block to this prediction block, resulting in a reconstructed block, i. restored block.

The entropy decoding module 810 is configured to convert the received bitstream into information, such as a quantized coefficient and a syntax element, by entropy decoding according to a probability distribution.

The quantized coefficient is dequantized by dequantizing module 820 and subjected to an inverse transform performed by inverse transform module 830. The quantized coefficient can be converted into a reconstructed residual block by dequantization/inverse transformation.

The intra-prediction module 840 is configured to intra-frame generate a prediction block for the current block using a pixel value from an already encoded block adjacent to the current block.

The inter-frame prediction module 850 is configured to inter-frame a prediction block for the current block by performing motion compensation using a reference frame stored in the reference frame buffer 870 and a motion vector.

In the multi-view structure, the inter-picture prediction applied in the inter-frame mode may comprise the inter-view prediction. The inter-frame prediction module 850 is configured to construct an inter-view reference frame by selecting chip values from the reference-view frame. The inter-frame prediction module 850 is configured to perform inter-view prediction using a reference frame list containing the specified inter-view reference frame. The relative relationship between views may be communicated by information indicating the dependency of one view on another view.

If the frame of the current view (current frame) and the frame of the reference view have the same dimensions, then the selection of elementary values applied to the frame of the reference view can be understood as the formation of the reference elementary value by copying the elementary value from the frame of the reference view or by interpolation in the frame of the reference view. If the frame of the current view and the frame of the reference view have different sizes, then the selection of elementary values applied to the frame of the reference view can be understood as the selection of elementary values with upscaling or downscaling.

For example, if inter-picture prediction is applied to views having different resolutions, then upscaling of the reconstructed reference-view frame may be performed to construct an inter-view key frame.

In this case, information indicating the view to which the frame to be used as the inter-view reference frame belongs can be transmitted from the encoder to the decoder.

The frame used to predict the current block in the view that is being inter-viewed - that is, in the reference view - may be the same as the frame of the same Access Element (ED) as the current frame (the frame in the current view for which prediction is made).

The recovered residual block and the prediction block are summed by an adder 855 to form a recovered block. In other words, by adding the residual chip and the prediction chip, a recovered chip or frame is generated.

The reconstructed frame is filtered by filter 860. At filter 860, at least one of the following filters may be applied to the reconstructed block or reconstructed frame: deblocking filter, adaptive bias filter, adaptive loop filter. Filter 860 outputs a recreated (modified) or filtered reconstructed frame (recreated frame). This reconstructed frame is stored in the reference frame buffer 870 for use in inter prediction.

Although in the embodiment described in conjunction with FIG. 7 and 8, these modules perform their respective functions independently of each other, the present invention is not limited to this. For example, one module can perform two or more functions. For example, the operations performed respectively by the intra prediction module and the inter prediction module shown in FIG. 7 and 8 may be executed by one module (predictor module).

As indicated above in connection with FIG. 7 and 8, one encoder/decoder performs encoding/decoding of all views from a plurality of views. However, this is only for ease of description, and a separate encoder/decoder may be provided for each view of the plurality of views.

In such a case, the encoder/decoder for the current view may perform encoding/decoding of the current view using the information related to the other view. For example, a predictor (inter-picture prediction module) for the current view may perform intra-prediction or inter-picture prediction for the current block using the specified pixel information or reconstructed frame information of another view.

Although inter-view prediction is described here, the current layer can be encoded/decoded using information related to another view, regardless of whether an encoder/decoder is provided for each view or multiple views are processed by one device.

The description relating to views in accordance with the present invention can equally apply to zoom-enabled layers. For example, the view described herein may be a level.

Fig. 9 is an illustration of an example of a prediction structure for a 3D codec. To simplify the description of FIG. 9 illustrates a prediction structure for encoding textures acquired by three cameras and depth information maps corresponding to said textures.

As shown in FIG. 9, the three textures obtained from the three cameras, respectively, are labeled T0, T1, and T2 according to the angles, and the three depth information maps corresponding to the three textures are labeled D0, D1, and D2, according to the angles. Here, T0 and D0 are images taken from view 0, T1 and D1 from view 1, and T2 and D2 from view 2. Shown in FIG. 9 quadrangles represent images (frames).

These pictures (frames), depending on the types of coding/decoding, are divided into type I frames (frames with intra prediction), type P frames (frames with prediction with reference to one frame) and frames of type B (frames with prediction with reference to two frames ), and each frame may be encoded/decoded according to its encoding/decoding type. To obtain type I frames, the images are encoded by themselves, without the use of inter-picture prediction. Only reference pictures from one direction can be used to obtain P-type frames in inter-picture prediction, while reference pictures from two directions can be used to obtain B-type frames in inter-picture prediction. The arrows in Fig. 9 indicate directions of prediction. In other words, the texture and its depth information map can be encoded/decoded in relation depending on the prediction directions.

To encode/decode an image using inter prediction, motion information related to the current block is needed. To obtain motion information related to the current block, the following methods can be used: a method using motion information related to a block adjacent to the current block; a method using temporal correlation at the same point in time; and a method using inter-view correlation at a close point in time. For one frame, these inter-frame prediction methods may be used in combination. The current block here is the block for which the prediction is being performed. The motion information may refer to a motion vector, a reference frame number, and/or a prediction direction (eg, whether the prediction is unidirectional prediction or bidirectional prediction, whether the prediction uses temporal correlation or interview correlation, etc.).

The prediction direction can generally be classified into unidirectional prediction or bidirectional prediction depending on whether a reference frame list (RefPicList) is used. Bidirectional prediction is divided into forward reference prediction (Pred_L0: Prediction L0), which uses a list (LIST 0, L0) of reference frames with forward reference, backward reference prediction (Pred_L1: Prediction L1), which uses a list (LIST 1, L1) reference frames with backward reference, and prediction with both forward and backward reference (Pred_BI: Prediction BI), which uses both a list (LIST 0) of key frames with a forward link and a list (LIST 1) of key frames with a backward link. The case in which the forward reference frame list (LIST 0) is copied to the backward reference frame list (LIST 1), resulting in two forward reference prediction operations, can also be categorized as bidirectional prediction.

The direction of prediction may be determined using flags predFlagL0 and predFlagL1. Here, predFlagL0 is an indicator indicating whether a forward reference frame list is used (LIST 0), and predFlag1 is an indicator indicating whether a backward reference frame list is used (LIST 1). For example, in the case of unidirectional prediction and forward reference prediction, predFlagL0 may be equal to 1 and predFlagL1 may be equal to 0; in the case of unidirectional prediction and backward reference prediction, predFlagL0 is 0 and predFlagL1 is 1; and in the case of bidirectional prediction, predFlagL0 is 1 and predFlagL1 is 1.

Fig. 10 illustrates an example where adjacent blocks are used to build a list of merge candidates for the current block.

Merging is a method for performing inter-picture prediction. In the merge mode, motion information related to blocks adjacent to the current block can be used as motion information related to the current block (for example, at least one of the following list can be used as indicated: motion vector, reference frame list, reference frame index). The use of motion information related to neighboring blocks as motion information related to the current block is called merging or motion merging.

In the merge mode, it is possible to merge the movement by coding elements (EC) and by prediction elements (EP).

To merge a motion performed block by block (for example, by coding elements or by prediction elements, hereinafter, to simplify the description, "by blocks"), information is needed on whether the motion is merged on a part of the block, and information on which of the adjacent blocks of the current block are merged motion.

To perform a motion merge, a list of merge candidates may be built.

The merge candidate list refers to a list of motion information items that may be generated prior to performing the merge. Meanwhile, the motion information of the merge candidate list may be motion information relating to neighboring blocks of the current block, or motion information newly created by combining motion information items already contained in said merge candidate list. Motion information related to neighboring blocks (eg, motion vector and/or reference frame index) may be motion information indicated by neighboring blocks or motion information stored in neighboring blocks (or used to decode neighboring blocks).

As shown in FIG. 10, adjacent blocks may include adjacent blocks A, B, C, D, and E spatially adjacent to the current block, and a candidate block H or M correspondingly located corresponding in time to the current block. By a suitably located candidate block is meant a block located at a corresponding position in a correspondingly located frame, corresponding in time to the current frame containing the current block. If block H is available in a correspondingly located frame, this block H can be taken as a correspondingly located candidate block, if block H is not available, then block M in a correspondingly located frame can be taken as a correspondingly located candidate block.

When constructing the merge candidate list, it is checked whether the motion information related to neighboring blocks (A, B, C, D and E) and respectively located candidate block (H or M) can be used as a merge candidate for building a merge candidate list, related to the current block. In other words, motion information related to blocks that can be used for inter prediction with respect to the current block can be added to the merge candidate list as a merge candidate.

For example, the following method may be used as a method for constructing a merge candidate list for block X. 1) If neighbor block A is available, then that neighbor block A is added to the merge candidate list. 2) Then, only in the case where the motion information related to the neighboring block B is different from the motion information related to the neighboring block A, this neighboring block B is added to the merge candidate list. 3) Similarly, only in the case where the motion information related to neighboring block C is different from the motion information related to neighboring block B, neighboring block C is added to the merge candidate list. neighboring block C, the neighboring block D is added to the merge candidate list. merger candidates. 6) Finally, the neighboring block H (or M) is added to the merge candidate list. As a result, neighboring blocks can be added to the list of possible merge blocks in the order A→B→C→D→E→H (or M). Here, the same motion information can be understood as using the same motion vector, the same reference frame, and the same prediction direction (unidirectional or bidirectional).

Here, to simplify the description, the expressions "adding a neighboring block to the merge candidate list as a merge candidate" and "adding motion information to the merge candidate list as a merge candidate" are intermingled, although the two expressions mean essentially the same thing. For example, when referring to a neighboring block as a merge candidate, this may mean motion information related to said block.

Fig. 11 is an illustration of an example of an operation of determining motion information related to the current block using motion information from an adjacent view.

For the purpose of simplifying the description associated with FIG. 11, it uses only one view to determine the motion information related to the current block. However, two or more adjacent views may be used.

As shown in FIG. 11, in order to efficiently encode/decode motion information, the 3D video system can use the motion information from the adjacent view. Specifically, for the current block shown in FIG. 11 (of the block at the current location X), in order to determine the motion information related to the current block, the target block in the adjacent view (at the reference position X _R ) is searched. In this case, the target block in the neighboring view is understood to be the block corresponding to the current block. Since in the current frame the only difference between the current view and the reference view is the position of the cameras, the target block in the adjacent view can be determined from the disparity vector (DV) as described above.

Fig. 12 is an illustration of an example in which one prediction element (PE) is split into multiple prediction sub-elements.

In the example shown in FIG. 12, a 64×64 prediction element is divided into prediction sub-elements, each of which is 8×8 in size. The prediction element size is 64x64 in FIG. 12 is chosen for simplicity of description only, and without limitation, the specified size may be 32×32, 16×16, 8×8, or 4×4. In a 3D video system, one prediction element may be split into multiple prediction sub-elements. In this case, the determination of motion information using the residual vector is performed on prediction subelements. The prediction sub-element may have a predetermined size (eg, 4x4, 8x8, or 16x16), and the prediction sub-element size may be set upon encoding. Prediction subelement size information may be included and reported in an extended Video Parameter Set (VPS) syntax.

Fig. 13 is an illustration of an example of an operation of determining motion information related to the current block using a reference block.

By the operation of determining the motion information related to the current block, it is meant to set, as the motion information related to the current block, the motion information contained in the reference block. However, for efficient encoding/decoding of the motion information, the 3D video system can, for the current block X located in the current frame in the current view, determine the motion information from the prediction sub-elements.

In other words, the 3D video system may assign the value of the motion information contained in the prediction sub-element of the reference block X _R to the motion information related to the prediction sub-element of the current block X. In this case, the reference block X _R may be understood to be the reference block X _R located in the current frame in baseline. The specific motion information determination operation will be described next.

Fig. 14 is an illustration of an example of a reference block used to determine motion information related to the current block.

As shown in FIG. 14, a reference block may refer to a prediction element (PE), and one reference block may contain 16 prediction sub-elements in total. Meanwhile, the motion information related to each prediction sub-element in the current block can be determined from the motion information contained in the reference block prediction sub-elements.

Next, with reference to FIG. 15a-15e and figs. 16a-16g, a method for determining motion information related to prediction subelements of a current block using a reference block is described.

Fig. 15a to 15e are schematic illustrations of an example of motion information determination operation using motion information stored in the storage. Meanwhile, the support block used in FIG. 15a-15e may be the support block shown in FIG. fourteen.

When the prediction sub-elements of the current block are assigned motion information related to the reference block prediction sub-elements, it is possible that not all of the reference block prediction sub-elements contain motion information. In other words, there may be prediction sub-elements in the reference block from which motion information cannot be obtained. Accordingly, if there are prediction subitems from which motion information cannot be obtained, motion information relating to the previous or subsequent prediction subitem relative to the currently referenced prediction subitem can be used to eliminate the inability to determine motion information from the prediction subitem of the current block. For example, motion information related to a prediction sub-item available in a reference block may be stored in advance in order to prepare for a situation in which motion information cannot be determined from another prediction sub-item of the reference block, and this previously stored motion information may be input into the prediction sub-item. of the current block to determine motion information related to the current block.

For a better understanding of the above method, the following describes with reference to the drawings each step of an exemplary method for determining motion information related to a prediction sub-element of the current block in the case where the first prediction sub-element of a reference block contains its motion information, and the second or following reference block prediction subelements may not contain it.

Fig. 15a is an illustration of the initial state of the prediction sub-elements of the current block and storage.

In FIG. 15a Ref denotes a reference block, and Ref 0, 1, 2, and 3, respectively, denote prediction sub-elements in the reference block. In other words, Ref 0 indicates reference block prediction sub-element 0 (first reference block prediction sub-element), Ref 1 indicates reference block prediction sub-element 1 (second reference block prediction sub-element), Ref 2 indicates reference block prediction sub-element 2 (third reference block prediction sub-element) , and Ref 3 denotes reference block prediction sub-element 3 (fourth reference block prediction sub-element). Cur denotes the current block, and Cur 0, 1, 2, and 3, respectively, denote prediction sub-elements in the current block. In other words, Cur 0 indicates current block prediction sub-element 0 (the first prediction sub-element of the current block), Cur 1 indicates the current block prediction sub-element 1 (the second prediction sub-element of the current block), Cur 2 indicates the current block prediction sub-element 2 (the third prediction sub-element of the current block) , and Cur 3 denotes prediction sub-element 3 (the fourth prediction sub-element of the current block).

The symbol X at position Ref 2 in FIG. 15a denotes motion information that cannot be determined using reference block prediction sub-element 2.

Fig. 15b represents the first step of determining motion information from the reference block prediction sub-element.

As shown in FIG. 15b, motion information for prediction sub-element 0 of the current block is determined from prediction sub-element 0 of the reference block. Here, since it is possible to determine the motion information from the reference block prediction sub-element 0, the motion information related to the reference block prediction sub-element 0 is stored in the storage. In this case, the motion information stored in the storage may be designated as motion information 0; this motion information can be used when motion information cannot be determined from any of the other reference block prediction sub-elements.

Fig. 15c represents the second step of determining motion information from the reference block prediction sub-element.

As shown in FIG. 15c, motion information for the prediction sub-element 1 of the current block is determined from the prediction sub-element 1 of the reference block. Here, since it is possible to determine the motion information from the reference block prediction sub-item 1, the motion information related to the reference block prediction sub-item 1 is stored in the storage. In this case, the stored motion information related to prediction sub-item 1 may be designated as motion information 1; this motion information 1 may be stored in the storage instead of the motion information 0. Motion information 1 may be used when motion information cannot be determined from any other reference block prediction sub-element.

Fig. 15d represents the third step of determining motion information from the reference block prediction sub-element.

As shown in FIG. 15d, an attempt is made to determine motion information for the prediction sub-element 2 of the current block from the prediction sub-element 2 of the reference block. However, since the motion information cannot be determined from the prediction sub-item 2 of the reference block, the motion information related to the prediction sub-item 2 of the current block is determined from the motion information stored in the storage. In this case, the motion information stored in the storage may be motion information 1.

Fig. 15e represents the fourth step of determining motion information from the reference block prediction sub-element.

As shown in FIG. 15e, motion information for the prediction sub-element 3 of the current block is determined from the prediction sub-element 3 of the reference block. Here, since it is possible to determine motion information from the reference block prediction sub-item 3, the motion information related to the reference block prediction sub-item 3 is stored in the storage. In this case, the stored motion information related to the prediction subitem 3 may be called motion information 3; this motion information 3 may be stored in the storage instead of motion information 1 . Motion information 3 may be used when motion information cannot be determined from any other reference block prediction sub-element.

Fig. 16a-16g are schematic illustrations of yet another example of motion information determination operation using motion information stored in the storage.

Fig. 16a-16g illustrate an example of motion information determination operation in the case where there is a reference block prediction sub-element from which motion information cannot be determined and followed by another reference block prediction sub-element from which motion information can be determined.

Fig. 16a is an illustration of the initial state of the prediction sub-elements of the current block and storage.

In FIG. 16a, Ref denotes a reference block, and Ref 0, 1, 2, and 3, respectively, denote prediction sub-elements in the reference block. In other words, Ref 0 indicates reference block prediction sub-element 0, Ref 1 indicates reference block prediction sub-element 1, Ref 2 indicates reference block prediction sub-element 2, and Ref 3 indicates reference block prediction sub-element 3. Cur denotes the current block, and Cur 0, 1, 2, and 3, respectively, denote prediction sub-elements in the current block. In other words, Cur 0 indicates prediction sub-element 0 of the current block, Cur 1 indicates prediction sub-element 1 of the current block, Cur 2 indicates prediction sub-element 2 of the current block, and Cur 3 indicates prediction sub-element 3 of the current block. The symbol X at position Ref 0 in FIG. 16a denotes motion information that cannot be determined using the reference block prediction sub-element 0 and the reference block prediction sub-element 1.

Fig. 16b represents the first step of determining motion information from the reference block prediction sub-element.

As shown in FIG. 16b, an attempt is made to determine motion information for prediction sub-element 0 of the current block from prediction sub-element 0 of the reference block. However, as mentioned above, the motion information cannot be determined from the reference block prediction sub-element 0, nor is there motion information stored in the storage. Accordingly, the second step is performed.

Fig. 16c represents the second step of determining motion information from the reference block prediction sub-element.

As shown in FIG. 16c, an attempt is made to determine motion information for the prediction sub-element 1 of the current block from the prediction sub-element 1 of the reference block. However, as mentioned above, the motion information cannot be determined from the reference block prediction sub-element 1, and there is no motion information stored in the storage. Accordingly, the third step is performed.

Fig. 16d represents the third step of determining motion information from the reference block prediction sub-element.

As shown in FIG. 16d, the motion information for the prediction sub-element 2 of the current block is determined from the prediction sub-element 2 of the reference block. Here, since it is possible to determine motion information from the reference block prediction sub-item 2, the motion information related to the reference block prediction sub-item 2 is stored in the storage. In this case, the motion information stored in the storage may be designated as motion information 2; this motion information can be used when motion information cannot be determined from any of the other reference block prediction sub-elements.

Fig. 16e represents the fourth step of determining motion information from the reference block prediction sub-element.

As shown in FIG. 16e, the motion information for the prediction sub-element 0 of the current block is determined using the motion information 2 stored in the storage.

Fig. 16f represents the fifth step of determining motion information from the reference block prediction sub-element.

As shown in FIG. 16f, the motion information for the prediction subitem 1 of the current block is determined using the motion information 2 stored in the storage.

Fig. 16g represents the sixth step of determining motion information from the reference block prediction sub-element.

As shown in FIG. 16g, the motion information for the prediction sub-element 3 of the current block is determined from the prediction sub-element 3 of the reference block. Here, since it is possible to determine motion information from the reference block prediction sub-item 3, the motion information related to the reference block prediction sub-item 3 is stored in the storage. In this case, the stored motion information related to the prediction subitem 3 may be called motion information 3; this motion information 3 may be stored in the storage instead of the motion information 2 . Motion information 3 may be used when motion information cannot be determined from any other reference block prediction sub-element.

Fig. 17 is a flowchart illustrating a method for determining motion information related to a prediction sub-element of a current block using a prediction sub-element of a reference block in accordance with an embodiment of the present invention. Each operation in the method shown in FIG. 17 may be performed by an encoder and/or decoder or an encoder and/or decoder inter prediction module, such as intra prediction module 720 shown in FIG. 7 or the inter prediction module 850 shown in FIG. eight.

First with reference to FIG. 17 describes the operation when the reference block prediction sub-element contains its motion information. The inter prediction unit checks whether the reference block prediction sub-element contains motion information (S1700).

If the reference block prediction sub-item contains motion information, the inter prediction unit inputs the motion information contained in the specified reference block prediction sub-item into the prediction sub-item of the current block for which motion information determination is performed (S1710).

Then, the inter prediction module checks whether the storage contains motion information (S1720). If the storage contains motion information, step S1750 is performed. The store has been detailed above, and in this case, the store contains motion information.

If no motion information is stored in the storage, the inter prediction unit checks whether the prediction sub-element of the current block for which motion information determination is performed is the first prediction sub-element of the current block (S1730). If the prediction sub-element of the current block for which motion information determination is performed is the first prediction sub-element of the current block, the inter prediction unit executes step S1750.

If it is determined in step S1730 that the prediction sub-element of the current block is not the first prediction sub-element, the inter-picture prediction module inputs the motion information contained in the specified reference block prediction sub-element into the prediction sub-element(s) of the current block ahead of the first prediction sub-element of the current block. For example, if the prediction sub-element of the current block for which motion information is determined is the third prediction sub-element, the inter prediction module inputs motion information related to the specified reference block prediction sub-element to the first and second prediction sub-elements of the current block.

The inter prediction unit stores motion information related to the reference block prediction sub-element in storage, replacing the stored information (S1750). A detailed description of the storage and replacement of motion information is given above.

The inter prediction unit checks whether the reference block prediction sub-element for which motion information determination is performed is the last reference block prediction sub-element (S1790). If the reference block prediction sub-element for which motion information determination is performed is the last reference block prediction sub-element, the inter prediction unit terminates this motion information determination operation. On the other hand, if the reference block prediction sub-element for which motion information determination is performed is not the last prediction sub-element of the specified reference block, the inter prediction unit proceeds to the next reference block prediction sub-element processing (S1780). Then, the inter prediction module repeats steps S1700-S1790.

If there are no reference block prediction subelements containing motion information, then the next operation is performed.

The inter prediction unit checks whether the reference block prediction sub-element contains motion information (S1700).

If the reference block prediction sub-element does not contain motion information, then the inter prediction unit checks whether the motion information is contained in the storage (S1770). If the store does not contain motion information, then the inter prediction module executes step S1790.

If the storage contains motion information, the inter prediction unit inputs the motion information stored in the storage into the prediction sub-element of the reference block for which motion information determination is performed (S1750).

After performing the above steps, the inter prediction unit checks whether the reference block prediction sub-element for which motion information determination is performed is the last reference block prediction sub-element (S1790). If the reference block prediction sub-element for which motion information determination is performed is the last reference block prediction sub-element, the inter prediction unit terminates this motion information determination operation. On the other hand, if the reference block prediction sub-element for which motion information determination is performed is not the last prediction sub-element of the specified reference block, the inter prediction unit proceeds to the next reference block prediction sub-element processing (S1780). Then, the inter prediction module repeats steps S1700-S1790.

Thereafter, the inter prediction module determines a prediction chip for the current block based on the motion information related to the current block determined in the above steps. The prediction chip may refer to the prediction signal described above, and the prediction signal may refer to the difference between the original signal and the residual signal as described above.

The above-described motion information determination operation related to the prediction sub-element of the current block may be applied to 3D images in particular as shown in Table 1. As indicated above, the operation presented in Table 1 may be performed by an encoder/decoder or an encoder/decoder predictions.

Table 1

This operation uses the following input data:

- position (xPb, yPb) of the upper left corner of the current prediction element;

- width (nPbW) and height (nPbH) of the current prediction element;

- refViewIdx index of the reference view;

- mvDisp residual vector.

This operation produces the following output:

- flag availableFlagLXinterView to indicate the availability of a possible temporal inter-view motion vector, where LX may be a list of L0 and L1 reference frames;

Here, the definition of "temporal inter-view" is understood in the sense that a frame from a view different from the view of the current frame can be considered as a frame from the same view as the current frame, but corresponding to a different point in time (i.e., having a different sequence number frame, POC).

- a possible temporal interview motion vector mvLXInterView, where LX can be a list of L0 or L1 reference frames;

is a refIdxLXInterView reference index indicating a reference frame contained in the RefPicListLX reference frame list, where LX may be an L0 or L1 list of reference frames.

LX may be a list of L0 or L1 reference frames. The following operation is applied to the LX list:

- the availableFlagLXInterView flag is initialized to 0,

- the mvLXInterView motion vector is assigned an initial value (0,0),

- reference index refIdxLXInterView is initialized to -1.

The variables nSbW and nSbH are initialized as follows:

nSbW=Min(nPbW, SubPbSize);

nSbH=Min(nPbH, SubPbSize).

The ivRefPic variable is initialized to the value corresponding to the frame that has the same ViewIdx value as the refViewIdx in the current accessor. curSubBlockIdx is initialized to 0, lastAvailableFlag is initialized to 0. Changing yBlk from 0 to (nPbH/nSbH-1) and changing xBlk from 0 to (nPbW/nSbW-1) performs the following operation:

- the variable curAvailableFlag is assigned an initial value of 0;

- when X changes from 0 to 1, the following operation is performed;

- the spPredFlagL1[xBlk][yBlk] flag is initialized to 0;

- the motion vector spMvLX is assigned an initial value (0,0);

- the reference index spRefIdxLX[xBlk][yBlk] is assigned the initial value -1;

- the position (xRef, yRef) of the reference block is determined as follows:

xRef=Clip3(0, PicWidthInSamplesL-1), xPb+xBlk*nSbW+nSbW/2+((mvDisp[0]+2)>>2));

yRef=Clip3(0, PicHeightInSamplesL-1, yPb+yBlk*nSbH+nSbH/2+((mvDisp[1]+2)>>2)).

- the ivRefPb variable points to the luminance prediction block at position (xRef, yRef) in the inter-view reference frame indicated by ivRefPic;

- (xIvRefPb, yIvRefPb) points to the position of the upper left corner of the reference block indicated by ivRefPb;

- except when ivRefPb was encoded in intraframe mode, when X changes from 0 to 1, the following operation is performed:

- if the value of X is 0 or the current slice is slice B, then when changing Y from X to (1-X), the following operation is performed:

- the variables refPixListLYIvRef, predFlagLYIvRef[x][y], mvLYIvRef[x][y] and refIdxLYIvRef[x][y] are assigned the values RefPicListLY, PredFlagLY[x][y], MvLY[x][y] and RefIdxLY[x][y] in the frame indicated by ivRefPic;

- if the value of predFlagLYIvRef[xIvRefPb][yIvRefPb] is 1, then when i changes from 0 to num_ref_idx_lX_active_minus1 (the number of reference frames in the reference frame list), the following operation is performed:

- if the frame sequence number (POC) of refPicListLYIvRef[refIdxLYIvRef[xIvRefPb][yIvRefPb]] is the same as RefPicListLX[i] and the value of spPredFlagLX[xBlk][yBlk] is 0, then the following operations are performed:

spMvLX[xBlk][yBlk]=mvLYIvRef[xIvRefPb][yIvRefPb];

spRefIdxLX[xBlk][yBlk]=i;

spPredLfagLX[xBlk][yBlk]=1;

curAvailableFlag=1.

According to the value of curAvailableFlag, the following operation is performed:

- if the value of curAvailableFlag is 1, then the following sequence is used:

1. If lastAvailableFlag is 0, the following operations are performed:

- when X changes from 0 to 1, the following operation is performed:

mxLXInterView=spMvLX[xBlk][yBlk];

refIdxLXInterView=spRefIdxLX[xBlk][yBlk];

availableFlagLXInterview=spPredFlag[xBlk][yBlk].

- if curSubBlockIdx is greater than 0, then when k changes from 0 to (curSubBlockIdx-1), the following operations are performed:

- variables i and j are defined as follows:

i=k%(nPSW/nSbW);

j=k/(nPSW/nSbW).

- when X changes from 0 to 1, the following operation is performed:

spMvLX[i][j]=spMvLX[xBlk][yBlk];

spRefIdxLX[i][j]=spRefIdxLX[xBlk][yBlk];

spPredFlagLX[i][j]=spPredFlagLX[xBlk][yBlk].

2. The lastAvailableFlag variable is set to 1.

3. The xBlk and yBlk values are stored in the xLastAvail and yLastAvail variables, respectively.

- If the value of curAvailableFlag is 0 and the value of lastAvailableFlag is 1, then when X changes from 0 to 1, the following operation is performed:.

spMvLX[xBlk][yBlk]=spMvLX[xLastAvail][yLastAvail];

spRefIdxLX[xBlk][yBlk]=spRefIdxLX[xLastAvail][yLastAvail];

spPredFlagLX[xBlk][yBlk]=spPredFlagLX[xLastAvail][yLastAvail].

- the variable curSubBlockIdx is assigned the value curSubBlockIdx+1.

Next, Table 1 is described in detail.

As shown in Table 1, the position of the upper left corner of the current prediction block, the width and height of the current prediction block, the reference view index, and the residual vector are input to the inter prediction module. In this case, the position of the upper left corner of the current prediction block can be denoted as (xPb, yPb), where xPb can mean the coordinate of the current prediction block on the x axis, and yPb can mean the coordinate of the current prediction block on the y axis. The width of the current prediction block may be referred to as nPbW, and the height of the current prediction block may be referred to as nPbH. The reference view index can be denoted as refViewIdx and the residual vector as mvDisp. Meanwhile, the inter-picture prediction module may correspond to the above-described inter-picture prediction module of the image encoder/decoder.

As shown in FIG. 17, after completion of the operation of determining the motion information related to the prediction sub-element of the current block using the prediction sub-element of the reference block, the inter prediction module outputs a flag signaling that the candidate temporal inter-view motion vector is ready to be used, the candidate temporal inter-view motion vector, and the reference frame contained in the keyframe list. In this case, the flag signaling the readiness of the possible temporal interview motion vector for use may be designated as availableFlagLXInterView, and the specified possible temporal interview motion vector may be designated as mvLXInterView. The specified reference frame list may be referred to as RefPicListLX, and the reference index indicating the reference picture contained in the specified reference frame list may be referred to as refIdxLXInterView. In the notation availableFlagLXInterView, mvLXInterView, RefPicListLX, and refIdxLXInterView, the LX code can be the key frame list 0 code (List 0, L0) or the key frame list 1 code (List 1, L1).

Next, a method for determining motion information related to the prediction sub-element of the current block using the reference block prediction sub-element, for the inter prediction unit to determine the above-described output data from the above-described input data, will be described.

Before determining the motion information related to the prediction sub-element of the current block, using the reference block prediction sub-element, the inter prediction module performs seeding. This sets availableFlagLXInterView to 0, mvLXInterView to (0,0), and refIdxLXInterView to -1. When seeding is performed by the inter prediction module, the width and height of the prediction subelement are also seeded. Here, the prediction sub-element width may be denoted as nSbW and the prediction sub-element height as nSbH. The specific way of assigning initial values to the variables nSbW and nSbH is given by Formula 1 below.

[Formula 1]

.

.

Here, SubPbSize denotes the size (including height and width) of the prediction sub-element indicated by the Picture Parameter Set (VPS), and nuh_layer_id denotes an index identifying the layer (eg, which reference view it is). The Min() function, which produces the smallest of the input variables, can be determined according to formula 2.

[Formula 2]

.

.

The inter-frame prediction module may initialize not only the above variables, but also information identifying the prediction sub-element of the current block and the luminance prediction block at the position (xRef, yRef) in the inter-view reference frame, and information indicating the presence of motion information stored in the storage.

Here, the luminance prediction block at position (xRef, yRef) in the inter-view reference frame is assigned the value of the block in the frame having the same view index as the reference view index in the current access element. The luminance prediction block at position (xRef, yRef) in the inter-frame reference picture is referred to as ivRefPic, and the specified access element is understood to be the element in which image encoding/decoding is performed. This access element contains images from different angles that have the same frame sequence number (picture order count, POC). For example, if there are three views, one access element may contain a normal image and/or image of depth information of the first view, a normal image and/or image of depth information of the second view, and a normal image and/or image of depth information of the third view. The reference view index can be denoted as refViewIdx and the view index as ViewIdx. In this case, ViewIdx can be understood as the view of the current image.

Meanwhile, information for identifying the prediction sub-element of the current block may be set to 0 when assigning initial values, and the identification information of the prediction sub-element of the current block may be designated as curSubBlockIdx. Information is also set to indicate the presence of motion information stored in the storage, which may be designated as lastAvalableFlag, with an initial value of 0.

After initializing the variables described above, the inter prediction module performs the following operation when yBlk changes from 0 to (nPbH/nSbH-1) and xBlk changes from 0 to (nPbW/nSbW-1). Here xBlk denotes the x-coordinate of the block and yBlk denotes the y-coordinate of the block.

First, the inter-picture prediction module assigns an initial value of information for indicating the need to predict motion information from the reference block prediction sub-item, the prediction flag of the specified prediction sub-item, the motion information related to the specified prediction sub-item, and the reference index of the specified prediction sub-item. Specifically, information for indicating the need to predict motion information from the reference block prediction sub-element may be set to 0. Meanwhile, information for indicating the need to predict motion information from the reference block prediction sub-element may be designated as curAvailableFlag. The prediction sub-element prediction flag may be set to 0, and this prediction sub-element prediction flag may be denoted as spPredFlagL1. For the purpose of representing block coordinates, the prediction sub-element flag may be denoted as spPredFlagL1[xBlk][yBlk]. The motion vector of the prediction sub-element is set to (0, 0), and this motion vector of the prediction sub-element may be denoted as spMvLX. The reference index of the specified prediction sub-item may be set to -1, and this prediction sub-item reference index may be denoted as spRefIdxLX. For the purpose of representing block coordinates, the prediction subelement reference index may be denoted as spRefIdxLX[xBlk][yBlk].

The position (xRef, yRef) of the reference block can be set according to Formula 3 below.

[Formula 3]

Here, xRef denotes the x-coordinate of the position of the reference block, and yRef denotes the y-coordinate of the position of the reference block. PicWidthInSamplesL denotes the width in the current frame, PicHeightInSamplesL denotes the height in the current frame. The Clip3() function can be defined according to Formula 4 below.

[Formula 4]

.

.

In the case of encoding an inter-view reference block in intra-frame mode, the following operation is performed when X changes from 0 to 1. An inter-view reference block refers to a luminance prediction block at a position (xRef, yRef) in an inter-view reference frame indicated by ivRefPic, and this inter-view reference block may be designated as ivRefPb. In other words, ivRefPb denotes a luma prediction block at position (xRef, yRef) in the inter-view reference frame indicated by ivRefPic, and ivRefPic denotes an inter-view reference frame. The position of the upper left corner of the reference box indicated by ivRefPb can be set to (xIvRefPb, yIvRefPb).

If X is 0 or the current slice is slice B, then each variable is assigned a new value according to Y (the value of Y changes from X to (1-X)) as follows. The variable refPicListLYIvRef is set to RefPicListLY in the frame indicated by ivRefPic, where RefPicListLY is the list of reference frames. The flag predFlagLYIvRef[x][y] is set to PredFlagLY[x][y] in the frame indicated by ivRefPic, where PredFlagLY is an identifier indicating the list of reference frames. The variable mvLYIvRef[x][y] is set to MvLY[x][y] in the frame indicated by ivRefPic, where MvLY denotes a motion vector. Similarly, the variable refIdxLYIvRef[x][y] is set to RefIdxLY[x][y] in the frame indicated by ivRefPic, where RefIdxLY denotes the reference index.

Meanwhile, if the value of predFlagLYIvRef[xIvRefPb][yIvRefPb] is equal to 1, then Formula 5 below can be applied when changing i from 0 to num_ref_idx_lX_active_minus1 (number of reference frames in the reference frame list).

[Formula 5]

.

.

According to Table 1, in the case where curAvailableFlag is 1 and in the case where curAvailableFalg is 0, the following operations are performed, respectively.

If curAvailableFlag is equal to 1, then the inter prediction module performs the following operation.

1. If lastAvailableFlag is 0, then Formula 6 below can be applied when changing X from 0 to 1.

[Formula 6]

.

.

If lastAvailableFlag is 0 and curSubBlockIdx is greater than 0, then according to formula 7 below, variables i and j can be calculated when changing k from 0 to (curSubBlockIdx - 1).

[Formula 7]

.

.

In this case, Formula 8 below can be applied when changing X from 0 to 1.

[Formula 8]

.

.

2. After executing the above sequence of operations, the inter-frame prediction module changes the value of lastAvailableFlag to the value 1.

3. The interframe prediction module then stores the xBlk and yBlk values in the xLastAvail and yLastAvail variables, respectively.

If the value of curAvailableFlag is 1 and the value of lastAvailableFlag is 1, then the interframe prediction module uses the formula 9 below when X changes from 0 to 1.

[Formula 9]

After all the above operations have been completed, curSubBlockIdx is set to curSubBlockIdx+1.

The method for determining motion information related to a prediction sub-element of the current block described above in conjunction with FIG. 17, if it is not possible to determine the motion information from the reference block prediction sub-element, uses the motion information related to the reference block prediction sub-element that was earlier (or will be later) referenced. Therefore, in the motion information determination method according to FIG. 17, a reference to a prediction sub-element from another reference block must be used, and, accordingly, this method is dependent. The dependent motion information determination method is poorly suited to the parallel processing schemes detailed using FIGS. eighteen.

Fig. 18 is an illustration of an example of an operation of parallel determination of motion information related to the prediction sub-element of the current block using the reference block prediction sub-element.

In FIG. 18, Ref denotes a reference block, and Ref elements 0, 1, 2, 3, 4, 5, 6, and 7 represent reference block prediction sub-elements 0, 1, 2, 3, 4, 5, 6, and 7, respectively. Cur denotes the current block, and Cur 0, 1, 2, 3, 4, 5, 6, and 7 represent prediction subelements 0, 1, 2, 3, 4, 5, 6, and 7 of the current block, respectively. An X symbol at positions Ref 2, 3, 4 and 5 means that prediction sub-elements 2, 3, 4 and 5 of the reference block are not available when determining motion information.

In the embodiment according to FIG. 18, the inter prediction unit for determining motion information from a prediction sub-element from which motion information cannot be determined searches for a prediction sub-element from which motion information can be determined as described above. Accordingly, the inter prediction unit cannot independently determine motion information for each prediction sub-element of the current block, and the above-described motion information determination operation is difficult to perform in parallel.

Fig. 19 is an illustration of an example operation of searching for a usable prediction sub-element when the prediction sub-element is located in the lower right corner of the reference block.

In FIG. 19, each square represents a prediction sub-element; a square with thick lines indicates a prediction sub-element suitable for determining motion information, and a square bounded by thin solid lines indicates prediction sub-elements unsuitable for determining motion information. An arrow formed by a dashed line shows the order in which motion information is searched.

If the prediction sub-element from which motion information can be determined is in the lower right corner of the reference block, as shown in FIG. 19, in order to find a prediction sub-element from which motion information can be determined, it will be necessary to sequentially search through all prediction sub-elements along the dashed arrow starting from the upper left corner of the reference block. It is generally not known which prediction sub-element and in which reference block can be used to determine motion information. Accordingly, in order to find a prediction sub-element that can be used to determine motion information, reference block prediction sub-elements have to be sequentially iterated starting from the first reference block prediction sub-element.

However, the motion information determination method shown in FIG. 19, finding a usable prediction sub-element requires checking all prediction sub-elements in the reference block, which causes frequent memory accesses. If, however, motion information is contained in only a small number of reference block prediction sub-elements, then prediction sub-elements that do not contain useful information are mainly checked. In particular, if none of the prediction sub-elements in the reference block can be used to determine motion information, then finding suitable prediction sub-elements in the reference block only results in useless memory accesses. In this case, "no motion information" means that for the current block it was not possible to detect a similar area in the reference block of the adjacent frame.

Accordingly, when motion information is contained in only a few prediction sub-elements of the reference block or is not contained in any prediction sub-element of the reference block, encoding/decoding of the current block using inter prediction can achieve higher efficiency. In other words, in the case where motion information is contained in only a few reference block prediction sub-elements or is not contained in any reference block prediction sub-element, it may be more efficient to search for a similar area in a neighboring pixel of the current block to perform encoding/decoding of the current block.

Fig. 20 is a schematic illustration of the time required to determine motion information by prediction sub-elements.

As shown in FIG. 20, if the time required to determine motion information from one prediction sub-element is T and the number of prediction sub-elements in a reference block is N, then the time required to determine all motion information from the reference block is NxT. The above method of determining motion information is data-dependent and results in frequent memory access. The data-dependent methods for determining motion information do not allow independent determination of motion information from each prediction sub-item, and therefore, in order to determine motion information from one prediction sub-item, it is necessary to wait for the determination of motion information from another prediction sub-item to be completed. As a consequence, data-dependent methods for determining motion information may lead to encoding/decoding delays.

As a result, the above-described method for determining motion information does not allow the use of parallel data processing to simultaneously determine motion information, and due to its architecture, this method may lead to frequent memory accesses, which reduces the efficiency of memory use.

In order to overcome the above drawbacks, the present invention provides an apparatus and a method for decoupling motion information detection. In FIG. 21 shows a configuration example of an inter prediction module according to the present invention. The motion information determination method according to the embodiment of the present invention will be described in detail with reference to FIG. 22-26.

Fig. 21 is a block diagram illustrating the configuration of an inter prediction unit 2100 according to the present invention.

As shown in FIG. 21, the inter prediction module 2100 may include a storage module 2110 and a determination module 2120. Under the module 2100 inter-picture prediction can be understood as the above-described module 710 inter-picture prediction in the encoder three-dimensional image or module 850 inter-picture prediction in the decoder three-dimensional image. The inter prediction module 2100 shown in FIG. 21 may be used in the above image encoding/decoding operation.

The storage unit 2110 marks the motion information and stores it in storage. The storage unit 2110 may use the motion information available at a position of the reference block to obtain stored motion information. In the present invention, this position may be the center of the reference block or a prediction (sub)element covering the center of the reference block. The motion information stored in the storage unit 2110 may be assigned an initial value. If the motion information cannot be stored in the storage, then the operation of determining the motion information from the prediction sub-elements can be omitted. By eliminating the operation of determining motion information on prediction sub-elements, inter-picture prediction can be performed in accordance with the above description. The storage module 2110 is described in more detail below.

The determining unit 2120 performs an operation of determining motion information from the prediction sub-element of the current block. Meanwhile, the determining unit 2120 is configured to perform basically the above-described motion information determining operation. However, in the case where the reference block prediction sub-element corresponding to the first prediction sub-element of the current block does not contain motion information, the determining unit 2120 of the present invention is configured to search for a reference block prediction sub-element containing motion information, and instead of determining the motion information, related to the first prediction sub-item of the current block from the reference block prediction sub-item containing motion information, then determining the motion information related to the first prediction sub-item of the current block from the motion information stored in the storage unit. The determination module 2120 is described in more detail below.

Next, with reference to the drawings, embodiments of the present invention will be described in detail.

Embodiment 1

Fig. 22 is a flowchart schematically illustrating a method for determining motion information related to a prediction subelement of a current block using a reference block according to an embodiment of the present invention.

In Embodiment 1, motion information related to the prediction sub-element of the current block (current sub-element) is determined based on motion information related to the center of the reference block. Embodiment 1 may be performed in an encoder and decoder, or in a predictive module or in an inter-frame prediction module of an encoder and decoder. Next, to simplify the description, Embodiment 1 is implemented by the inter prediction module 2100 shown in FIG. 21.

As shown in FIG. 22, the inter prediction unit 2100 may determine the position of the center of the reference block (S2200). The position of the center of the reference block can be determined by the following formula 10. In this case, the reference block may be a block located in the reference frame at the same position as the current block, and the reference block may have the same size as the current block.

[Formula 10]

Position X = xPb+(nPbW>>1)

Position Y = yPb+(nPbH>>1) .

In this case, xPb and yPb represent the position of the upper left corner of the current ES, nPbW the width of this ES, and nPbH the height of this ES.

The inter prediction unit 2100 may check for motion information at the center of the reference block (S2210). The position of the center of the support block may be specified as described above.

If there is no motion information that can be used at the center of the reference block, then the inter prediction unit 2100 may terminate this motion information determination operation.

For example, without having motion information at the center of the reference block that can be used, the inter prediction module 2100 may not determine the motion information related to the current block.

If motion information at the center of the reference block is present, then the inter prediction unit 2100 may store this motion information present at the center of the reference block in storage (S2220). The motion information present at the center of the reference block may be motion information related to the prediction block containing the position of the integral chip closest to the center of the reference block. The specific operation of storing motion information by the inter prediction unit 2100 has been described above. The inter prediction unit 2100 may determine the motion information related to the current prediction sub-element based on the specified stored motion information related to the reference block.

The inter prediction unit 2100 may check whether the reference block prediction sub-element corresponding to the current prediction sub-element contains motion information (S2240).

If the reference block prediction sub-item contains motion information, the inter-picture prediction unit 2100 may input motion information related to the specified reference block prediction sub-item (S2250) to the current prediction sub-item (S2250). In other words, the inter prediction unit 2100 may assign the motion information related to the corresponding current prediction sub-item to the value of the motion information related to the reference block prediction sub-item (eg, motion vector, reference frame index).

On the other hand, if the reference block prediction sub-item does not contain available motion information, the inter prediction unit 2100 inputs the reference block motion information stored in the storage into the current prediction sub-item (S2260). In other words, if motion information related to the prediction sub-element of the reference block corresponding to the current prediction sub-element is not available, then the inter prediction unit 2100 may assign the motion information related to the current prediction sub-element to the value of the motion information related to the center of the reference block stored in step S2200.

The inter prediction unit 2100 may check whether the reference block prediction sub-element corresponding to the current prediction sub-element is the last prediction sub-element in that reference block (or, equivalently, whether the current prediction sub-element is the last prediction sub-element in the current block) (S2270 ). If the reference block prediction sub-element is the last prediction sub-element, then the inter prediction unit 2100 may complete this motion information determination operation.

On the other hand, if the reference block prediction sub-element is not the last prediction sub-element, the inter prediction unit 2100 proceeds to determine motion information related to the next prediction sub-element of the current block to continue determining motion information (S2230).

The above-described motion information determination operation according to Embodiment 1 can be applied to the decoding of a 3D image as shown in Table 2.

table 2

This operation uses the following input data:

- position (xPb, yPb) of the upper left corner of the current prediction element;

- width (nPbW) and height (nPbH) of the current prediction element;

- refViewIdx index of the reference view;

- mvDisp residual vector.

This operation produces the following output:

- flag availableFlagLXinterView to indicate the availability of a possible temporal inter-view motion vector, where LX may be a list of L0 and L1 reference frames;

- a possible temporal interview motion vector mvLXInterView, where LX can be a list of L0 or L1 reference frames;

is a refIdxLXInterView reference index indicating a reference frame contained in the RefPicListLX reference frame list, where LX may be an L0 or L1 list of reference frames.

LX may be a list of L0 or L1 reference frames. The following operation is applied to the LX list.

- the availableFlagLXInterView flag is initialized to 0;

- mvLXInterView motion vector is assigned initial value (0,0);

- reference index refIdxLXInterView is initialized to -1.

The variables nSbW and nSbH are initialized as follows:

nSbW=Min(nPbW, SubPbSize);

nSbH=Min(nPbH, SubPbSize),

where SubPbSize is a size including the height and width of the prediction sub-element indicated by the parameter set (VPS).

The ivRefPic variable is initialized to the value corresponding to the frame that has the same ViewIdx value as the refViewIdx in the current accessor.

The variable curSubBlockIdx is initialized to 0.

The reference position (xRef, yRef) can be defined as follows.

.

.

The ivRefPic variable is set to the value corresponding to the frame that has the same ViewIdx value as the refViewIdx in the current accessor. Motion information in a reference frame may be stored in elements that are blocks of 8x8 pixels. The correction factors xRefFull and yRefFull may be the position of the center of the reference block integral element value, indicated using mvDisp.

The ivRefPb block may be a prediction block covering a position (xRef, yRef) in ivRefPic.

Position (xIvRefPb, yIvRefPb) defines the position of the upper left corner of the ivRefPb block.

Except when ivRefPb has been encoded in intra frame mode, changing Y from X to (1-X) may perform the following operation:

- the variables refPixListLYIvRef, predFlagLYIvRef[x][y], mvLYIvRef[x][y] and refIdxLYIvRef[x][y] are assigned, respectively, the values of the variables corresponding to them in the interview reference frame ivRefPic, i.e. RefPicListLY, PredFlagLY[x][y], MvLY[x][y] and RefIdxLY[x][y].

- if the value of predFlagLYIvRef[xIvRefPb][yIvRefPb] is 1, then when i changes from 0 to num_ref_idx_lX_active_minus1 (the number of reference frames in the reference frame list), the following operation can be performed:

- if the frame sequence number (POC) of refPicListLYIvRef[refIdxLYIvRef[xIvRefPb][yIvRefPb]] is the same as RefPicListLX[i] and availableFlagLXInterView is 0, then the following operations are performed:

.

.

If the value of availableFlagL0InterView or the value of availableFlagL1Interview is 1, then the following operation is performed:

- when changing yBlk from 0 to (nPbH/nSbH-1) and when changing xBlk from 0 to (nPbW/nSbW-1), the following operation is performed:

- the variable curAvailableFlag is assigned an initial value of 0;

- when X changes from 0 to 1, the following operation is performed:

-- the spPredFlagL1[xBlk][yBlk] flag is initialized to 0;

- the motion vector spMvLX is assigned an initial value (0,0);

- the reference index spRefIdxLX[xBlk][yBlk] is assigned the initial value -1;

- the position (xRef, yRef) of the reference block is determined as follows:

xRef=Clip3(0, PicWidthInSamplesL-1);

xPb+xBlk*nSbW+nSbW/2+((mvDisp[0]+2)>>2));

yRef=Clip3(0, PicHeightInSamplesL-1);

yPb+yBlk*nSbH+nSbH/2+((mvDisp[1]+2)>>2)).

- the variable ivRefPb points to the luminance prediction block at position (xRef, yRef) in the inter-view reference frame indicated by ivRefPic.

- (xIvRefPb, yIvRefPb) points to the position of the top left corner of the reference block pointed to by ivRefPb.

- except when ivRefPb was encoded in intraframe mode, when X changes from 0 to 1, the following operation is performed:

- if the value of X is 0 or the current slice is slice B, then when changing Y from X to (1-X), the following operation is performed:

- the variables refPixListLYIvRef, predFlagLYIvRef[x][y], mvLYIvRef[x][y] and refIdxLYIvRef[x][y] are assigned the values RefPicListLY, PredFlagLY[x][y], MvLY[x][y] and RefIdxLY[x][y] in the frame indicated by ivRefPic;

- if the value of predFlagLYIvRef[xIvRefPb][yIvRefPb] is 1, then when i changes from 0 to num_ref_idx_lX_active_minus1 (the number of reference frames in the reference frame list), the following operation is performed:

- if the frame sequence number (POC) of refPicListLYIvRef[refIdxLYIvRef[xIvRefPb][yIvRefPb]] is the same as RefPicListLX[i] and the value of spPredFlagLX[xBlk][yBlk] is 0, then the following operations are performed:

spMvLX[xBlk][yBlk]=mvLYIvRef[xIvRefPb][yIvRefPb];

spRefIdxLX[xBlk][yBlk]=i;

spPredLfagLX[xBlk][yBlk]=1;

curAvailableFlag=1.

- according to the value of curAvailableFlag, the following operation is performed:

- if curAvailableFlag is 0, then when X changes from 0 to 1, the following operations are performed:

.

.

- the variable curSubBlockIdx is assigned the value curSubBlockIdx+1.

If the value of availableFlagL0InterView and the value of availableFlagL1InterView are 0, then the operation ends.

Embodiment 1 is described again in Table 2.

As indicated in Table 2, the position of the upper left corner of the current prediction block, the width and height of the current prediction block, the reference view index, and the residual vector are input to the inter prediction module 2100. In this case, the position of the upper left corner of the current prediction block can be denoted as (xPb, yPb). The width of the current prediction block may be denoted as nPbW and the height of the current prediction block as nPbH. The reference view index can be denoted as refViewIdx and the residual vector as mvDisp.

After completion of the operation of determining motion information related to the prediction sub-element of the current block using the reference block prediction sub-element, the inter-frame prediction module 2100 may output a flag signaling an inter-view prediction capability, an inter-view motion vector, and a reference index indicating a reference frame contained in the reference frame list. frames. In this case, the flag signaling the readiness of the possible temporal interview motion vector for use may be designated as availableFlagLXInterView, and the specified possible temporal interview motion vector may be designated as mvLXInterView. The specified reference frame list may be referred to as RefPicListLX, and the reference index indicating the reference picture contained in the specified reference frame list may be referred to as refIdxLXInterView. In the notation availableFlagLXInterView, mvLXInterView, RefPicListLX, and refIdxLXInterView, the LX code can be the key frame list 0 code (List 0, L0) or the key frame list 1 code (List 1, L1).

Next, a method for determining motion information related to a prediction sub-element of the current block is described by obtaining the above-described output from the input.

First, before determining the motion information related to the prediction sub-item of the current block using the prediction sub-item of the reference block, the inter prediction unit 2100 performs seeding. In this case, availableFlagLXInterView can be set to 0, mvLXInterView to (0,0), and refIdxLXInterView to -1. When seeding is performed by the inter prediction module 2100, the width and height of the prediction sub-element may also be seeded. Here, the prediction sub-element width may be denoted as nSbW and the prediction sub-element height as nSbH. An example of how to assign initial values to the variables nSbW and nSbH is given by formula 11.

[Formula 11]

.

.

Here, SubPbSize denotes the size (including height and width) of the prediction sub-element indicated by the Picture Parameter Set (VPS), and nuh_layer_id denotes an index identifying the layer (eg, which reference view it is). Min() is an operator that returns the smallest of the input variables.

The inter-picture prediction unit 2100 can initialize not only the above variables, but also information identifying the prediction sub-element of the current block and the luminance prediction block at position (xRef, yRef) in the inter-view reference frame, and information indicating the presence of motion information stored in the storage .

In this case, a frame having an index of the angle, for example, the index of the reference angle in the current access element, can be taken as the specified inter-view reference frame. The inter-view reference frame may be referred to as ivRefPic, and the luminance prediction block at position (xRef, yRef) in the inter-view reference frame may be referred to as ivRefPb. One access element contains images from different angles having the same frame sequence number (POC). The index of the reference view can be denoted as refViewIdx and the specified view index as ViewIdx.

The indicated reference position may be a position indicating a prediction block covering the center of the reference block according to Embodiment 1. In order to determine the motion information related to the current prediction sub-item, the storage of motion information related to the reference view may be performed. Formula 12 is an example of a method for determining the reference position (xRef, yRef).

[Formula 12]

.

.

Here XRefFull and yRefFull denote the position of an integral elementary value close to the center of the reference block. In other words, xRefFull and yRefFull respectively denote the x-coordinate and y-coordinate of the elementary value in integer position.

The ivRefPb element can be a prediction sub-element or a prediction block covering a point (xRef, yRef). The position (xIvRefPb, yIvRefPb) of the luma element may indicate the upper left corner of ivRefPb.

Except when ivRefPb has been encoded/decoded in intra frame mode, the following operations (1) and (2) can be applied when changing Y from X to (1-X).

refPicListLYIvRef is set to RefPicListLY in the ivRefPic inter-view reference picture, predFlagLYIvRef[x][y] is set to PredFlag[x][y] in the ivRefPic inter-view reference picture, and refIdxLYIvRef[x][y] is set to RefIdxLY[x][ y] in the interview reference picture ivRefPic.

If the value of predFlagLYIvRef[xIvRefPb][yIvRefPb] is 1, then when i changes from 0 to num_ref_idx_lX_active_minus1 (number of reference frames in the X reference frame list), the following operation is performed. If the POC (Frame Ordinal Number) of refPicListLYIvRef[refIdxLYIvRef[xIvRefPb][yIvRefPb]] is RefPicListLX[i] and availableFlagLXInterView is 0, then Equation 13 may apply.

[Formula 13]

.

.

If the value of vailableFlagL0InterView or the value of availableFlagL1InterView is 1, then the inter prediction module 2100 performs the following operation when yBlk changes from 0 to (nPbH/nSbH-1) and xBlk changes from 0 to (nPbW/nSbW-1). Here, xBlk denotes the x-coordinate and yBlk denotes the y-coordinate. In other words, when the motion information available at the center of the reference block can be determined, the inter prediction module 2100 can determine the motion information from the prediction sub-elements.

First, the inter prediction unit 2100 may assign an initial value of information for indicating the need to predict motion information from the reference block prediction sub-item, a prediction sub-item prediction flag, motion information related to the specified prediction sub-item, and a reference index of the prediction sub-item.

Meanwhile, information for indicating the need to predict motion information from the reference block prediction sub-element may be designated as curAvailableFlag, the prediction sub-element prediction flag as spPredFlagLX1, the prediction sub-element flag as spPredFlagLX[xBlk][yBlk], the motion vector of the prediction sub-element as spMvLX, the prediction sub-element reference index as spRefIdxLX, and the specific prediction sub-element reference index as spRefIdxLX[xBlk][yBlk].

The position (xRef, yRef) of the reference block is reset across prediction sub-elements as in Formula 14 below.

[Formula 14]

PicWidthInSamplesL denotes the width of the current image, and PicHeightInSamplesL denotes the height of the current image. The Clip3() function was described above.

Then, in the case of encoding an inter-view reference block in intra-frame mode, the following operation is performed when X changes from 0 to 1.

If X is 0 or the current slice is slice B, then each variable is assigned a new value according to Y (the value of Y changes from X to (1-X)) as follows. The refPicListLYIvRef variable may be set to a value from the RefPicListLY reference frame list for the frame indicated by the ivRefPic variable (ie, for an inter-view reference frame). The variable predFlagLYIvRef[x][y] is set to PredFlagLY[x][y]. predFlagLY[x][y] denotes the list of reference frames that is applied at the point (x,y) in the picture indicated by ivRefPic. The variable mvLYIvRef[x][y] is set to MvLY[x][y]. MvLY[x][y] denotes the motion vector at the point (x,y) in the image indicated by ivRefPic. The variable refIdxLYIvRef[x][y] is set to RefIdxLY[x][y]. RefIdxLY[x][y] denotes the reference pixel at (x,y) in the image indicated by ivRefPic.

In the case where the value of predFlagLYIvRef[xIvRefPb][yIvRefPb] is equal to 1, the following formula 15 can be applied when changing i from 0 to num_ref_idx_lX_active_minus1 (number of reference frames in the reference frame list) if the POC of refPicListLYIvRef[refIdxLYIvRef[xIvRefPb][yIvRefPb]] is RefPicListLX[i] and spPredFlagLX[xBlk][yBlk] is 0.

[Formula 15]

.

.

But if, even after performing the above operation, the value of curAvailableFlag remains equal to the initially set value of 0 (i.e., always, except when spRefIdxLX=i (for example, spRefIdxLx=-1) and spPredFlagLX=1 (for example, spPredFlagLX =-1)), it can be said that the motion information cannot be determined from the prediction subitems. Accordingly, the inter prediction module 2100 may use Formula 16 as X changes from 0 to 1.

In other words, if motion information from the prediction sub-element of the reference block cannot be determined, then the inter-picture prediction unit 2100 may determine motion information related to the prediction sub-element of the current block from motion information related to the center of the reference block.

[Formula 16]

.

.

Finally, after all the above operations are performed, curSubBlockIdx is set to curSubBlockIdx + 1, and if availableFlagL0InterView and availableFlagL1InterView are 0, then the motion information determination operation according to Embodiment 1 ends.

Embodiment 2

Fig. 23 is a flowchart schematically illustrating a method for determining motion information related to a prediction sub-element of a current block, according to another embodiment of the present invention. In the example shown in FIG. 23, motion information related to the prediction sub-element of the current block can be determined using the prediction sub-element located at the center position of the reference block.

In Embodiment 2, motion information related to the prediction sub-element of the current block may be determined based on motion information related to the prediction sub-element covering the center of the reference block.

The example shown in FIG. 23 may be performed in the encoder and decoder or in the encoder and decoder predictor or in the inter prediction module 2100 shown in FIG. 21. Meanwhile, in order to simplify the description, the inter prediction unit 2100 performs each step as shown in FIG. 23.

As shown in FIG. 23, the inter prediction unit 2100 may determine the position of the prediction sub-element located at the center of the reference block (center prediction sub-element) (S2300). The center prediction sub-element located in the reference block refers to the prediction sub-element located in the center of the reference block; the center of the support block discussed above. Formula 17 is an example of determining the position of the reference block prediction center subelement.

[Formula 17]

.X coordinate value of prediction subelement center =

.

.Prediction subelement center Y coordinate value =

.

Here, xPb and yPb indicate the position of the upper left corner of the current prediction element, nPbW the width of the current prediction element, and nPbH the height of the current prediction element.

The inter prediction unit 2100 checks whether the center prediction sub-element of the reference block contains motion information (S2310); the position of the center prediction subelement of the reference block is discussed above. If there is no motion information at the location where the reference block prediction center subelement is located, then the inter prediction unit 2100 may terminate this motion information determination operation.

On the other hand, if there is motion information in the center prediction sub-element of the reference block, the inter prediction unit 2100 may store this motion information available in the center (S2320). The specific operation of storing motion information by the inter prediction unit 2100 has been described above.

The inter prediction module 2100 determines motion information related to the current prediction sub-element. The inter prediction unit 2100 may check whether the reference block prediction sub-element corresponding to the current prediction sub-element contains motion information (S2340).

If the specified reference block prediction sub-item contains motion information, the inter-picture prediction unit 2100 may input to the current prediction sub-item the motion information contained in the specified reference block prediction sub-item (S2350). On the other hand, if the reference block prediction sub-item does not contain motion information, the inter prediction unit 2100 may input the motion information stored in step S2320 into the current prediction sub-item (S2360).

The inter prediction unit 2100 may check whether the prediction sub-element of the reference block for which motion information determination is performed is the last prediction sub-element (S2370). If the specified reference block prediction sub-element is the last prediction sub-element, then the inter prediction unit 2100 may complete this motion information determination operation of the current block. On the other hand, if the specified reference block prediction sub-element is not the last prediction sub-element, the inter prediction unit 2100 proceeds to processing the next prediction sub-element of the current block to continue determining motion information (S2330).

The above-described motion information determination operation according to Embodiment 1 of Embodiment 2 can be applied to decoding 3D images as shown in Table 3.

Table 3

This operation uses the following input data:

- position (xPb, yPb) of the upper left corner of the current prediction element;

- width (nPbW) and height (nPbH) of the current prediction element;

- refViewIdx index of the reference view;

- mvDisp residual vector.

This operation produces the following output:

- flag availableFlagLXInterView to indicate the availability of a possible temporal inter-view motion vector, where LX may be a list of L0 or L1 reference frames;

- a possible temporal interview motion vector mvLXInterView, where LX can be a list of L0 or L1 reference frames;

is a refIdxLXInterView reference index indicating a reference frame contained in the RefPicListLX reference frame list, where LX may be an L0 or L1 list of reference frames.

LX may be a list of L0 or L1 reference frames. The following operation is applied to the LX list.

- the availableFlagLXInterView flag is initialized to 0;

- the mvLXInterView motion vector is assigned an initial value (0, 0);

- reference index refIdxLXInterView is initialized to -1.

The variables nSbW and nSbH are initialized as follows:

.

.

where SubPbSize is a size including the height and width of the prediction sub-element indicated by the parameter set (VPS).

The ivRefPic variable is initialized to the value corresponding to the frame that has the same ViewIdx value as the refViewIdx in the current accessor.

The variable curSubBlockIdx is initialized to 0.

The reference position (xRef, yRef) can be defined as follows:

.

.

The ivRefPic variable is set to the value corresponding to the frame that has the same ViewIdx value as the refViewIdx in the current accessor.

The variable ivRefPb is set to a value corresponding to the prediction block covering the position (xRef, yRef) in ivRefPic.

The variable (xIvRefPb, yIvRefPb) is set to the position of the upper left corner of the reference block specified by ivRefPb.

Except when ivRefPb has been encoded in intra frame mode, changing X from 0 to 1 may apply the following operation:

- if X is 0 or the current slice is slice B, then when Y changes from X to (1-X), the following operation is performed:

- the variables refPixListLYIvRef, predFlagLYIvRef[x][y], mvLYIvRef[x][y] and refIdxLYIvRef[x][y] are assigned, respectively, the values of the variables corresponding to them in the interview reference frame ivRefPic, i.e. RefPicListLY, PredFlagLY[x][y], MvLY[x][y] and RefIdxLY[x][y];

- if the value of predFlagLYIvRef[xIvRefPb][yIvRefPb] is 1, then when i changes from 0 to num_ref_idx_lX_active_minus1 (the number of reference frames in the reference frame list), the following operation can be performed:

- if the frame sequence number (POC) of refPicListLYIvRef[refIdxLYIvRef[xIvRefPb][yIvRefPb]] is the same as RefPicListLX[i] and the value of centerPredFlag is 0, then the following operations are performed:

.

.

If the value of centerAvailableFlag is 1, then the following operation is performed:

- when changing yBlk from 0 to (nPbH/nSbH-1) and when changing xBlk from 0 to (nPbW/nSbW-1), the following operation is performed:

- the variable curAvailableFlag is assigned an initial value of 0;

- when X changes from 0 to 1, the following operation is performed:

- the spPredFlagL1[xBlk][yBlk] flag is initialized to 0;

- the motion vector spMvLX is assigned an initial value (0,0);

- the reference index spRefIdxLX[xBlk][yBlk] is assigned the initial value -1;

- the position (xRef, yRef) of the reference block is determined as follows:

xRef=Clip3(0, PicWidthInSamplesL-1), xPb+xBlk*nSbW+nSbW/2+((mvDisp[0]+2)>>2));

yRef=Clip3(0, PicHeightInSamplesL-1), yPb+yBlk*nSbH+nSbH/2+((mvDisp[1]+2)>>2));

- the ivRefPb variable points to the luminance prediction block at position (xRef, yRef) in the inter-view reference frame indicated by ivRefPic;

- (xIvRefPb, yIvRefPb) points to the position of the upper left corner of the reference block indicated by ivRefPb;

- except when ivRefPb was encoded in intraframe mode, when X changes from 0 to 1, the following operation is performed:

- if the value of X is 0 or the current slice is slice B, then when changing Y from X to (1-X), the following operation is performed:

- the variables refPixListLYIvRef, predFlagLYIvRef[x][y], mvLYIvRef[x][y] and refIdxLYIvRef[x][y] are assigned the values RefPicListLY, PredFlagLY[x][y], MvLY[x][y] and RefIdxLY[x][y] in the frame indicated by ivRefPic.

- if the value of predFlagLYIvRef[xIvRefPb][yIvRefPb] is 1, then when i changes from 0 to num_ref_idx_lX_active_minus1 (the number of reference frames in the reference frame list), the following operation is performed:

- if the frame sequence number (POC) of refPicListLYIvRef[refIdxLYIvRef[xIvRefPb][yIvRefPb]] is the same as RefPicListLX[i] and the value of spPredFlagLX[xBlk][yBlk] is 0, then the following operations are performed:

.

.

- according to the value of curAvailableFlag, the following operation is performed:

- if curAvailableFlag is 0, then when X changes from 0 to 1, the following operations are performed:

.

.

- the variable curSubBlockIdx is assigned the value curSubBlockIdx+1.

Otherwise, i.e. if the value of centerAvailableFlag is 0, the operation ends.

Embodiment 2 is described again in Table 3.

The variables in table 3 are the same as those in table 2.

The inter prediction unit 2100, before determining the motion information related to the current prediction sub-item using the reference block prediction sub-item, performs initialization. This initialization is performed in the same way as described above in relation to Table 2.

The inter-picture prediction module may indicate the position of the center prediction sub-element of the reference block. The position of the referenced block may be determined based on the reference position (xRef, yRef) determined in accordance with Formula 18.

[Formula 18]

ivRefPic is a frame with the same ViewIdx as refViewIdx in the current access element, and ivRefPb is a prediction block or prediction sub-element covering the point (xRef, yRef) defined by formula 19 in ivRefPic.

The point (xIvRefPb, yIvRefPb) specifies the position of the upper left corner of the ivRefPb block.

If ivRefPb has not been intra-coded/decoded and X is 0 or the current slice is slice B, then changing Y from X to (1-X) performs the following operation.

As noted in Table 2 above, refPicListLYIvRef is set to RefPicListLY, predFlagLYIvRef[x][y] is set to PredFlag[x][y], and refIdxLYIvRef[x][y] is set to RefIdxLY[x][y].

If the value of predFlagLYIvRef[xIvRefPb][yIvRefPb] is 1, then when i changes from 0 to num_ref_idx_lX_active_minus1 (the number of reference frames in the X reference frame list), formula 19 is applied in the case when the POC (picture sequence number) for refPicListLYIvRef[refIdxLYIvRef[xIvRefPb] [yIvRefPb]] is RefPicListLX[i] and availableFlagLXInterView is 0.

[Formula 19]

.

.

In formula 1, the flag centerAvailableFlag indicates that the reference block prediction center sub-element can be used, and centerMvLX refers to the motion vector for the reference block prediction center sub-element. Further, centerRefIdxLX denotes a reference index for the prediction center sub-item of the reference block, and centerPredFlagLX denotes the list of reference frames of the prediction center sub-item. Wherein, centerAvailableFlag, centerMvLX, centerRefIdxLX and/or centerPredFlagLX are motion information related to the center prediction subitem. In other words, the inter prediction module 2100 may store motion information related to the reference block prediction center subelement determined by Formula 19 in storage.

After assigning values to the specified variables in accordance with the above description, if the value of centerAvailableFlag is equal to 1, then the inter frame prediction module 2100 performs the following operation when changing yBlk from 0 to (nPbH/nSbH-1) and changing xBlk from 0 to (nPbW/nSbW- one). Here xBlk denotes the x-coordinate of the block and yBlk denotes the y-coordinate of the block. In other words, when it is possible to determine the motion information available from the subblock located in the center of the reference block, the inter prediction module 2100 can determine the motion information related to the current block from the prediction subelements.

First, the inter prediction unit 2100 assigns an initial value of information for indicating the need to predict motion information from the reference block prediction sub-item, a prediction sub-item prediction flag, motion information related to the specified prediction sub-item, and a reference index of the prediction sub-item. This initialization is performed in the same way as described above in relation to Table 2.

Repositioning (xRef, yRef) of the reference block is performed, as can be seen from Formula 20, for each prediction sub-element.

[Formula 20]

Here, xRef denotes the x-coordinate of the position of the reference block, and yRef denotes the y-coordinate of the position of the reference block. PicWidthInSamplesL denotes the width of the current image, and PicHeightInSamplesL denotes the height of the current image. The Clip3() function was described above.

If the inter-view reference block is encoded in intra-frame mode, then the inter-frame prediction module 2100 performs the following operation when X changes from 0 to 1.

If the value of X is 0 or the current slice is slice B, then each variable is assigned a new value according to Y (the value of Y changes from X to (1-X)) as follows. This initialization is performed in the same way as described above in relation to Table 2.

In the case where the value of predFlagLYIvRef[xIvRefPb][yIvRefPb] is equal to 1, the following formula 21 can be applied when changing i from 0 to num_ref_idx_lX_active_minus1 (number of reference frames in the reference frame list), if the POC of refPicListLYIvRef[refIdxLYIvRef[xIvRefPb][yIvRefPb]] is RefPicListLX[i] and spPredFlagLX[xBlk][yBlk] is 0.

[Formula 21]

.

.

But if, even after performing the above operation, the value of curAvailableFlag remains equal to the initially set value of 0 (i.e., always, except when spRefIdxLX=i (for example, spRefIdxLx=-1) and spPredFlagLX=1 (for example, spPredFlagLX =-1)), it can be said that the motion information cannot be determined from the prediction subitems. Accordingly, the inter prediction module 2100 may use Formula 22 as X changes from 0 to 1.

In other words, if motion information from the prediction sub-element of the reference block cannot be determined, then the inter-picture prediction unit 2100 may determine motion information related to the prediction sub-element of the current block from motion information related to the center sub-element.

[Formula 22]

.

.

Finally, after all the above operations are performed, curSubBlockIdx is set to curSubBlockIdx + 1, and if availableFlagL0InterView and availableFlagL1InterView are 0, then the motion information determination operation according to Embodiment 2 ends.

Fig. 24 is an illustration of an example of an operation of determining motion information related to a prediction sub-element of the current block using motion information from a position.

In FIG. 24, the top row blocks are prediction sub-elements of the reference block, and the bottom row blocks are prediction sub-elements of the current block. X stands for some position, and motion information from position X is stored in storage. Here, under the motion information at a certain position in FIG. 24 can be understood as motion information at the center of the reference block as in Embodiment 1, and motion information related to the reference block prediction center subelement as in Embodiment 2.

When determining motion information related to a prediction sub-element of the current block using motion information from a position, motion information from that position may be used for each prediction sub-element in the reference block. In other words, it is possible to simultaneously determine the motion information related to a plurality of prediction sub-elements of the current block using the motion information from a position where a data-dependent deficiency can be eliminated. Accordingly, when using the motion information from a certain position, the inter prediction module 2100 can determine the motion information in parallel.

As mentioned above, in Embodiments 1 and 2, motion information is determined using motion information available at a location. Accordingly, the motion information determination methods according to Embodiments 1 and 2 make it possible to independently determine the motion information related to each prediction sub-element in the reference block. In other words, in Embodiments 1 and 2, in order to find the prediction sub-elements from which motion information can be determined, it is not necessary to sequentially search for prediction sub-elements from which motion information can be determined, and if the first prediction sub-element cannot be used to determine the motion information reference block, in Embodiments 1 and 2, the motion information related to the prediction sub-element of the current block is determined using the predetermined motion information. Thus, in determining the motion information according to Embodiments 1 and 2, the dependence on data is eliminated, which makes it possible to determine the motion information related to each prediction sub-element in parallel. In addition, when determining motion information in accordance with options 1 and 2, the implementation eliminates unnecessary memory accesses inherent in existing methods for determining motion information, thereby reducing the number of memory accesses.

Embodiment 3

Fig. 25 is a flowchart illustrating a method for determining motion information related to a prediction sub-element of the current block using a certain value of motion information in accordance with yet another embodiment of the present invention.

As shown in FIG. 25, Embodiment 3 proposes a method for setting the standard motion information and determining the motion information related to the current prediction sub-element from the standard motion information in the case where the motion information cannot be determined from the reference block prediction sub-element. In this case, the standard motion information can be understood as a zero vector. The specific method for determining motion information according to Embodiment 3 is described next.

The inter prediction module 2100 may store standard motion information in storage (S2500). The specific operation of storing motion information by the inter prediction unit 2100 has been described above.

Then, the inter prediction module 2100 may determine motion information related to the current prediction sub-element. The inter prediction unit 2100 may check whether the reference block prediction sub-element corresponding to the current prediction sub-element contains motion information (S2520).

If the reference block prediction sub-item contains motion information, the inter prediction unit 2100 may input motion information related to the specified reference block prediction sub-item (S2530) to the current prediction sub-item. On the other hand, if the reference block prediction sub-item does not contain motion information, the inter-picture prediction unit 2100 may input the motion information stored in the storage into the current prediction sub-item (S2540).

The inter prediction unit 2100 may check whether the prediction sub-element of the reference block for which motion information determination is performed is the last prediction sub-element (S2550). If the specified reference block prediction sub-element is the last prediction sub-element, then the inter prediction module 2100 may terminate the present motion information determination operation. On the other hand, if the reference block prediction sub-element is not the last prediction sub-element, the inter prediction unit 2100 may proceed to determine motion information related to the next reference block prediction sub-element to continue determining motion information (S2510).

The above motion information determination operation according to Embodiment 3 can be applied to 3D-HEVC project text 2 as shown in Table 4.

Table 4

This operation uses the following input data:

- position (xPb, yPb) of the upper left corner of the current prediction element;

- width (nPbW) and height (nPbH) of the current prediction element;

- refViewIdx index of the reference frame;

- mvDisp residual vector.

This operation produces the following output:

- flag availableFlagLXinterView to indicate the availability of a possible temporal inter-view motion vector, where LX may be a list of L0 and L1 reference frames;

- a possible temporal interview motion vector mvLXInterView, where LX can be a list of L0 or L1 reference frames;

is a refIdxLXInterView reference index indicating a reference frame contained in the RefPicListLX reference frame list, where LX may be an L0 or L1 list of reference frames.

LX may be a list of L0 or L1 reference frames. The following operation is applied to the LX list:

- the availableFlagLXInterView flag is initialized to 0;

- the mvLXInterView motion vector is assigned an initial value (0,0);

- reference index refIdxLXInterView is initialized to -1.

The variables nSbW and nSbH are initialized as follows:

- nSbW=Min(nPbW, SubPbSize);

- nSbH=Min(nPbH, SubPbSize),

where SubPbSize is a size including the height and width of the prediction sub-element indicated by the parameter set (VPS).

The ivRefPic variable is initialized to the value corresponding to the frame that has the same ViewIdx value as the refViewIdx in the current accessor.

The variable curSubBlockIdx is initialized to 0.

The variables availableFlagL0InterView and availableFlagL1Interview are initialized as follows:

.

.

If the current slice is slice B, then

.

.

When yBlk changes from 0 to (nPbH/nSbH-1) and when xBlk changes from 0 to (nPbW/nSbW-1), the following operation is performed:

- the curAvailabeFlag variable is initialized to 0;

- when X changes from 0 to 1, the following operation is performed:

- the spPredFlagL1[xBlk][yBlk] flag is initialized to 0;

- the motion vector spMvLX is assigned an initial value (0, 0);

- the reference index spRefIdxLX[xBlk][yBlk] is assigned the initial value -1;

- the position (xRef, yRef) of the reference block is determined as follows:

.

.

- the ivRefPb variable points to the luminance prediction block at position (xRef, yRef) in the inter-view reference frame indicated by ivRefPic;

- (xIvRefPb, yIvRefPb) points to the position of the upper left corner of the reference block indicated by ivRefPb;

- Except when ivRefPb has been encoded in intra frame mode, when changing X from 0 to 1, the following operation may apply:

- if X is 0 or the current slice is slice B, then when Y changes from X to (1-X), the following operation is performed:

- the variables refPixListLYIvRef, predFlagLYIvRef[x][y], mvLYIvRef[x][y] and refIdxLYIvRef[x][y] are assigned, respectively, the values of the variables corresponding to them in the interview reference frame ivRefPic, i.e. RefPicListLY, PredFlagLY[x][y], MvLY[x][y] and RefIdxLY[x][y];

- if the value of predFlagLYIvRef[xIvRefPb][yIvRefPb] is 1, then when i changes from 0 to num_ref_idx_lX_active_minus1 (the number of reference frames in the reference frame list), the following operation can be performed:

- if the frame sequence number (POC) of refPicListLYIvRef[refIdxLYIvRef[xIvRefPb][yIvRefPb]] is the same as RefPicListLX[i] and the value of centerPredFlag is 0, then the following operations are performed:

.

.

- According to the value of curAvailableFlag, the following operation is performed:

- if curAvailableFlag is 0, then when X changes from 0 to 1, the following operations are performed:

.

.

- the variable curSubBlockIdx is assigned the value curSubBlockIdx+1.

Embodiment 3 is described again in Table 4. The variables in Table 4 are the same as those in Table 2.

The inter prediction unit 2100, before determining the motion information related to the current prediction sub-item using the reference block prediction sub-item, performs initialization. This initialization is performed in the same way as described above in relation to Table 2.

Next, the variables availableFlagLXZero, mvLXZero and refIdxLXZero are assigned values according to formulas 23 and 24. In this case, X takes the values 0 or 1.

[Formula 23]

.

.

[Formula 24]

.

.

Here, availableFlagLXZero denotes an identifier indicating the availability of the standard motion information, mvLXZero denotes the standard motion information, and refIdxLXZero denotes a reference index of the standard motion information.

After assigning values to variables as described above, the inter prediction module 2100 performs the following operation when yBlk changes from 0 to (nPbH/nSbH-1) and when xBlk changes from 0 to (nPbW/nSbW-1). Here xBlk denotes the x-coordinate of the block and yBlk denotes the y-coordinate of the block.

First, the inter prediction unit 2100 assigns an initial value of information for indicating the need to predict motion information from the reference block prediction sub-item, a prediction sub-item prediction flag, motion information related to the specified prediction sub-item, and a reference index of the prediction sub-item. This initialization is performed in the same way as described above in relation to Table 2.

Repositioning (xRef, yRef) of the reference block is performed, as can be seen from Formula 25, for each prediction sub-element.

[Formula 25]

If the inter-view reference block is encoded in intra-frame mode, then the inter-frame prediction module 2100 may perform the following operation when X changes from 0 to 1.

If X is 0 or the current slice is slice B, then each variable is assigned a new value according to Y (the value of Y changes from X to (1-X)), as described above in Table 2.

In this case, if the value of predFlagLYIvRef[xIvRefPb][yIvRefPb] is equal to 1, then the following formula 26 can be applied when changing i from 0 to num_ref_idx_lX_active_minus1 (the number of reference frames in the reference frame list).

[Formula 26]

.

.

After performing the above operation, if the value of curAvailableFlag is 0, then the inter prediction module 2100 may apply Formula 27 when X changes from 0 to 1.

In other words, if it is not possible to determine the motion information from the reference block prediction sub-item, then the inter-picture prediction unit 2100 may determine the motion information related to the prediction sub-item of the current block from the arbitrary standard motion information.

[Formula 27]

.

.

Finally, after all the above operations are performed, curSubBlockIdx is set to curSubBlockIdx + 1, and if availableFlagL0InterView and availableFlagL1InterView are 0, then the motion information determination operation according to Embodiment 3 ends.

Fig. 26 is an illustration of an example of an operation of determining motion information related to a prediction sub-element of the current block using some motion information.

In FIG. 26, the top row blocks are prediction sub-elements of the reference block, and the bottom row blocks are prediction sub-elements of the current block. In addition, standard motion information is stored in the storage. At the same time, under the one shown in Fig. 26, the standard motion information can be understood as the standard motion information arbitrarily set according to Embodiment 3.

When determining the motion information related to the prediction sub-element of the current block using the standard motion information, standard, arbitrary motion information may be used for each prediction sub-element in the reference block. In other words, it is possible to simultaneously determine the motion information related to a plurality of prediction sub-elements of the current block using the standard motion information, and such use of the plurality of prediction sub-elements of the current block can help overcome the disadvantage expressed in terms of data. Accordingly, when using the standard motion information with a certain value, the inter prediction module 2100 can determine the motion information in parallel.

As mentioned above, according to Embodiment 3, the inter prediction module 2100 determines the motion information using the standard motion information that has been assigned a value. Accordingly, the motion information determination method according to Embodiment 3 makes it possible to independently determine the motion information related to each prediction sub-element in the reference block. In other words, in Embodiment 3, in order to find the prediction sub-elements from which motion information can be determined, it is not necessary to sequentially search for prediction sub-elements from which motion information can be determined, and if the first reference block prediction sub-element cannot be used to determine the motion information, then, in Embodiment 3, the motion information related to the prediction sub-element of the current block is determined using the predetermined motion information. Thus, in determining the motion information according to Embodiment 3, the dependence on data is eliminated, which makes it possible to determine the motion information related to each prediction sub-item in parallel. In addition, when determining motion information according to Embodiment 3, unnecessary memory accesses inherent in the existing methods for determining motion information are eliminated, thereby reducing the number of memory accesses.

Fig. 27 is a schematic illustration of the time required to determine motion information in accordance with the present invention.

In FIG. 20 shows that if the time required to determine motion information from one prediction sub-element is T and the number of prediction sub-elements in a reference block is N, then the time required to determine all motion information from the reference block is NxT. However, according to an embodiment of the present invention, the determination of motion information can be parallelized, and therefore the time required to determine the motion information is T, and the encoding/decoding delay of the 3D image is reduced.

The embodiments of the invention described above may have different applicability ranges depending on block sizes, coding element depth (EC) or transform element depth (ECD). As a parameter defining the application range, a value predetermined in the encoder/decoder or a value determined according to a profile or a level may be used; if the encoder specifies the value of the parameter in the bitstream, then the decoder can obtain this value from the bitstream.

For example, in the case where the range of application depends on the depth of the coding elements, a scheme (method A) only applied at a depth not less than a given one, a scheme (method B) only applied at a depth no greater than a given one, or a scheme (method C) can be provided. ), applied only at a given depth. If the methods in accordance with the present invention are not applicable to any of the depths, then an indicator (flag) can be used to indicate this, or by means of the EC depth it can be indicated that the methods in accordance with the present invention are not applicable, while the EC depth can be set to exceed the greatest depth that an EC can have.

Table 5

EC Depth (or EC or EC) representing the application range Method A Method B Method C 0 X 0 0 one X 0 0 2 0 0 0 3 0 X X 4 or more 0 X X

In the above embodiments, the methods are described using flowcharts with sequences of steps or modules, but the present invention is not limited to the order of the steps, and some steps may be performed simultaneously with other steps or in a different order. One of ordinary skill in the art will appreciate that the steps in the flowcharts are not mutually exclusive and that other steps may be added to the flowcharts or some of the steps may be removed from the flowcharts without affecting the scope of the present invention.

In addition, the above embodiments of the invention contain various aspects of the examples. It is not possible to describe all possible combinations to represent various aspects, and those skilled in the art will appreciate that other combinations than those described are possible. Accordingly, the present invention contains all changes, modifications and variations covered by the following claims.

The methods described above in accordance with the present invention may be implemented in a computer executable program that may be stored on a machine-readable, writable medium, examples of which include Read Only Memory (ROM), Random Access Memory (RAM), CD, magnetic tape, floppy disk, optical storage device; said program may also be or be presented as a transmitted signal (eg as a transmission over the Internet).

Said machine-readable writable medium may be distributed across networked computer systems, and the machine-readable codes may be stored and executed distributedly. Functional programs, codes or code segments for implementing the methods described above can be easily developed by programmers familiar with the technical field to which the present invention pertains.

Although the present invention has been presented and described in connection with its preferred embodiments, it is not limited to the said embodiments, and various changes may be made to the present invention without departing from the scope of the following claims, which should not be interpreted individually. apart from the technical essence or scope of the present invention.

Claims (32)

1. An image decoding method, including: determining motion information related to the current block; determining a prediction chip value for the current block based on motion information related to the current block; and generating a recovered chip for the current block based on the prediction chip for the current block, wherein the step of determining motion information related to the current block includes: determining whether the center sub-block corresponding to the center position of the current block has motion information in the reference block; determining motion information related to sub-blocks in the current block from sub-blocks in the reference block if the central sub-block in the reference block has motion information; and terminating the determination of motion information related to the sub-blocks in the current block from the sub-blocks in the reference block if the central sub-block in the reference block has no motion information. 2. The method according to claim 1, wherein the step of determining motion information related to sub-blocks in the current block includes: determining whether the sub-block in the reference block has motion information, wherein the sub-block in the reference block corresponds to the current sub-block in the current block; determining motion information related to the current sub-block in the current block from the sub-block in the reference block if the sub-block in the reference block has motion information; and determining motion information related to the current sub-block in the current block from the center sub-block in the reference block if the sub-block in the reference block has no motion information. 3. An image encoding method, including: determining motion information related to the current block; encoding motion information related to the current block; and generating a recovered chip for the current block based on the prediction chip for the current block, wherein the step of determining motion information related to the current block includes: determining whether the center sub-block corresponding to the center position of the current block has motion information in the reference block; determining motion information related to sub-blocks in the current block from sub-blocks in the reference block if the central sub-block in the reference block has motion information; and terminating the determination of motion information related to the sub-blocks in the current block from the sub-blocks in the reference block if the central sub-block in the reference block has no motion information. 4. The method of claim 3, wherein the step of determining motion information related to sub-blocks in the current block includes: determining whether the sub-block in the reference block has motion information, wherein the sub-block in the reference block corresponds to the current sub-block in the current block; determining motion information related to the current sub-block in the current block from the sub-block in the reference block if the sub-block in the reference block has motion information; and determining motion information related to the current sub-block in the current block from the center sub-block in the reference block if the sub-block in the reference block has no motion information. 5. A computer-readable storage medium that stores a bit stream formed by an image encoding method, including: determining motion information related to the current block; encoding motion information related to the current block; and generating a recovered chip for the current block based on the prediction chip for the current block, wherein the step of determining motion information related to the current block includes: determining whether the center sub-block corresponding to the center position of the current block has motion information in the reference block; determining motion information related to sub-blocks in the current block if the central sub-block in the reference block has motion information; and determining motion information related to the current block if the center subblock in the reference block has no motion information.

Images