ES2906182T3 - Method to decode multiview video - Google Patents

Abstract

A method for decoding a multi-view video, the method comprising: determining motion prediction schemes performed so that a current block and a collocated block corresponding to the current block are decoded, the motion prediction schemes determined to be long-term prediction , a short-term prediction and an interview prediction; and generating a current block motion vector using a collocated block motion vector in accordance with the determined motion prediction schemes, wherein when interview prediction is performed for one of the current block and the collocated block and then prediction is performed a long-term or the short-term prediction for the other block, the current block motion vector is generated using a predetermined vector or the collocated block motion vector, where the collocated block includes a temporary collocated block.

Description

DESCRIPTION

Method to decode multiview video

technical field

The present disclosure relates to encoding/decoding multiview video, and more particularly, to methods and apparatus for performing motion vector prediction and residue prediction for multiview video.

Background art

High Efficiency Video Coding (HEVC), which is known to have approximately twice the compression efficiency of legacy H.264/Advanced Video Coding (H.264/AVC), has recently been standardized. HEVC defines a coding unit (CU), a prediction unit (PU), and a transform unit (TU) in a quad-tree structure, and adopts a sample adaptive shift (SAO) and a loop filter, as a unlock filter. HEVC also increases compression coding efficiency by improving conventional intraprediction and interprediction.

Meanwhile, scalable video coding (SVC) is being standardized as an extension of HEVC, and 3D video coding (3DVC) based on H.264/AV or HEVC is also being standardized through enhanced coding. conventional multiview (MVC).

The group of video experts, MPEG of the international organization for standardization, ISO/IEC, recently began work on the standardization of 3DVC. The 3DVC standardization is based on an existing encoding technique for single-view two-dimensional (2D) video (H.264/AVC), an encoding technique for 2D multi-view video (MVC), and HEVC that has recently been standardized by the Team Joint Collaborative on Video Coding (JCT-VC).

Specifically, MPEG and ITU-T have jointly decided to standardize 3DVC and organized a new collaborative standardization group called the Joint Collaborative Team on 3D Video Coding Extensions (JCT-3V) . . The JCT-3V is defining an advanced syntax for depth encoding/decoding in the conventional MVC and standardizing an encoding/decoding technique for a new color image and depth image based on H.264/AVC and an encoding technique. /decoding for a multiview color image and depth image based on 3D-HEVC.

A variety of techniques are being discussed for the standardization of 3DVC. They usually include an encoding/decoding scheme based on interview prediction. In other words, since the amount of data of a multiview video to encode and transmit increases in proportion to the number of views, there is a need to develop an efficient technique for encoding/decoding a multiview video based on the dependency between views. ZHANG L ET AL: "Test Model 6 of 3D-HEVC and MV-HEVC", 6. JCT-3V MEETING; 10-25-2013 - 11-1-2013; GENEVA; (THE ISO/IEC JTC1/SC29/WG11 AND ITU-T SG.16 3D VIDEO CODING EXTENSION DEVELOPMENT JOINT COLLABORATIVE TEAM); #JCT3V-F1005 discloses encoding of dependent views.

TECH G ET AL: "3D-HEVC Draft Text 2", 6. JCT-3V MEETING; 10-25-2013 - 11-1-2013; GENEVA; (THE ISO/IEC JTC1/SC29/WG11 AND ITU-T SG.16 3D VIDEO CODING EXTENSION DEVELOPMENT JOINT COLLABORATIVE TEAM); No. JCT3V-F1001, specifies multi-view and depth video encoding, called 3D High Efficiency Video Coding (3D-HEVC).

ZHANG L ET AL: "CE4: Further improvements on advanced residual prediction", 6. JCT-3V MEETING; 10-25-2013 -11-1-2013; GENEVA; (THE ISO/IEC JTC1/SC29/WG11 AND ITU-T SG.16 3D VIDEO CODING EXTENSION DEVELOPMENT JOINT COLLABORATIVE TEAM); #JCT3V-F0123 discloses a proposal that is a follow-up to JCT3V-E0124 and improves ARP encoding efficiency.

ZHANG K ET AL: "3D-CE4: Results on improved advanced residual prediction", 6. JCT-3V MEETING; 10-25-2013 -11-1-2013; GENEVA; (THE ISO/IEC JTC1/SC29/WG11 AND ITU-T SG.16 3D VIDEO CODING EXTENSION DEVELOPMENT JOINT COLLABORATIVE TEAM); No. JCT3V-F0108 presents an adaptive disparity vector derivation method for advanced residual prediction.

Divulgation

technical problem

To overcome the above problem, one aspect of the present disclosure is to provide a method and apparatus for encoding and decoding a motion vector for a multi-view video through motion vector prediction.

Another aspect of the present disclosure is to provide a method and apparatus for encoding and decoding a residue for a multi-view video through residue prediction.

technical solution

The invention is defined by the appended claims. In one aspect of the present disclosure, a method of decoding a multi-view video includes determining motion prediction schemes performed to decode a current block and a corresponding block corresponding to the current block, and generating a motion vector predictor of the current block using a motion vector of the corresponding block according to the determined motion prediction schemes.

Determining motion prediction schemes may include acquiring data for video decoding by decoding a received bit stream and determining performed motion prediction schemes for the current block and the corresponding block using the data for video decoding.

Data acquisition for video decoding may include performing entropy decoding, dequantization, and reverse transformation on the received bit stream.

Determining the motion prediction schemes may include identifying the motion prediction schemes using at least one of view ID information, view order information, and flag information to identify a prediction scheme. motion, included in the data for video decoding.

Determining motion prediction schemes may include performing one of long-term prediction, short-term prediction, or interview prediction for each current frame and the corresponding frame, using the data for video decoding.

Generating a current block motion vector predictor may include, when performing a long-term prediction for the current block and the corresponding block, generating the current block motion vector predictor as the corresponding block motion vector .

Generating a current block motion vector predictor may include, when performing a short-term prediction for the current block and the corresponding block, generating the current block motion vector predictor by scaling the corresponding block motion vector using a relationship between an inter-image reference distance of the current block and an inter-image reference distance of the corresponding block.

Generating a current block motion vector predictor may include, when performing an interview prediction for the current block and the corresponding block, generating the current block motion vector predictor by scaling the corresponding block motion vector using a relationship between an interview reference distance of the current block and an interview reference distance of the corresponding block. Generating a current block motion vector predictor may include, when performing different motion prediction schemes for the current block and the corresponding block, not using the corresponding block motion vector.

Generating a current block motion vector predictor may further include, when the corresponding block motion vector is not used due to different motion prediction schemes used for the current block and the corresponding block, generating the motion vector predictor movement of the current block based on a predetermined vector.

The default vector can be (0, 0).

The generation of a motion vector predictor of the current block may include, when an interview prediction is made for one of the current block and the corresponding block and a long-term prediction or a short-term prediction is made for the other block, no use the motion vector of the corresponding block. Generating a motion vector predictor of the current block may include, when a long-term prediction is made for one of the current block and the corresponding block and a short-term prediction is made for the other block, or when a short-term prediction for one of the current block and the corresponding block and long-term prediction is performed for the other block, without using the motion vector of the corresponding block.

The method may further include retrieving a current block motion vector by adding the current block motion vector predictor to a current block motion vector difference included in the data for video decoding.

In another aspect of the present disclosure, a method of decoding a multiview video includes determining a motion prediction scheme performed for a first referenced block referenced for interview prediction of a current block to be decoded, and generating a residual. prediction for the current block according to the motion prediction scheme of the first reference block.

Determining an performed motion prediction scheme for a first reference block may include acquiring data for video decoding by decoding a received bit stream and determining the performed motion prediction scheme for the first reference block. reference, using the data for video decoding.

Data acquisition for video decoding may include performing entropy decoding, dequantization, and reverse transformation on the received bit stream.

Determining a performed motion prediction scheme for a first reference block may include identifying the motion prediction scheme using at least one of View ID information, view order information, and flag information to identify a motion prediction scheme, included in the data for video decoding.

Determining a motion prediction scheme performed for a first reference block may include determining whether a temporal prediction or an interview prediction is performed for the first reference block using the data for video decoding.

Generating a prediction residual for the current block may include generating, as a prediction residual, a difference between a second reference block that is referenced for the current block's temporal motion prediction and a third reference block that is referenced for the current block. is referenced for the first reference block.

The second reference block may belong to a closest image at a time address in a reference list for a current image to which the current block belongs.

Generating a prediction residual for the current block may include, when temporal motion prediction is determined to be performed for the first reference block, generating a scaled motion vector by applying a scale factor to a motion vector. motion used to search for the third reference block, and determining the second reference block using the scaled motion vector.

The scale factor can be generated based on a difference between a number of a reference image to which the first reference block belongs and a number of an image to which the third reference block belongs, which is referenced for temporal motion prediction of the first reference block, and a difference between a number of a picture to which the current block belongs and a number of a picture to which the second reference block belongs.

Generating a prediction residual for the current block may include, when the interview prediction is determined to be performed for the first reference block, determining the second reference block by applying (0, 0) as the motion vector used to search for the second reference block. second reference block.

The method may further include recovering a residual of the current block by adding the prediction residual to a residual difference of the current block included in the data for video decoding.

In another aspect of the present disclosure, an apparatus for decoding a multi-view video includes a processor configured to determine a performed motion prediction scheme for a referenced first referenced block for interview prediction of a current block to be decoded, and generating a prediction residue for the current block according to the motion prediction scheme of the first reference block. Advantageous effects

According to an embodiment of the present disclosure, the method for performing motion vector prediction for a multi-view video enables effective encoding/decoding of a motion vector during encoding/decoding of a multi-view video. That is, a temporary motion vector can be predicted adaptively according to the motion vector prediction schemes used for a current block and a corresponding block.

According to another embodiment (not covered by the claimed invention) of the present disclosure, the method for performing residue prediction for a multiview video enables effective encoding/decoding of a residue during encoding/decoding of a multiview video. That is, an error in calculating a scale factor used for scaling a motion vector can be prevented from occurring during the generation of a prediction residual, thus preventing an error in the prediction residual for a multi-view video.

Brief description of the drawings

Figure 1 is a conceptual view illustrating a motion vector prediction method according to an embodiment of the present disclosure.

Fig. 2 is a conceptual view illustrating a motion vector prediction method according to another embodiment of the present disclosure.

Fig. 3 is a conceptual view illustrating a motion vector prediction method according to another embodiment of the present disclosure.

Fig. 4 is a conceptual view illustrating a motion vector prediction method according to another embodiment of the present disclosure.

Fig. 5 is a conceptual view illustrating a motion vector prediction method according to another embodiment of the present disclosure.

Figure 6 is a flowchart illustrating a motion vector prediction method according to an embodiment of the present disclosure.

Figure 7 is a conceptual view illustrating a residue prediction method according to an embodiment of the present disclosure.

Fig. 8 is a conceptual view illustrating a residue prediction method according to another embodiment of the present disclosure.

Fig. 9 is a conceptual view illustrating a residue prediction method according to another embodiment of the present disclosure.

Figure 10 is a block diagram of apparatus for encoding and decoding multi-view video in accordance with an embodiment of the present disclosure.

Figure 11 is a block diagram of an apparatus for encoding a multi-view video according to an embodiment of the present disclosure.

Figure 12 is a block diagram of an apparatus for decoding a multi-view video according to an embodiment of the present disclosure.

Mode for carrying out the invention

Various modifications may be made to the present disclosure, and the present disclosure may be implemented in various embodiments. Various embodiments of the present disclosure are described with reference to the accompanying drawings. However, the scope of the present disclosure is not intended to be limited to the particular embodiments and it is to be understood that the present disclosure covers all modifications, equivalents and/or alternatives that fall within the scope of the present disclosure. In connection with a description of the drawings, like reference numerals indicate like components.

The term, as used in this disclosure, first, second, A, B and the like can be used to describe various components, without limiting the components. These expressions are used to distinguish one component from another component. For example, a first component may be referred to as a second component and vice versa without departing from the scope of this disclosure. The term and/or includes a combination of a plurality of related elements or any one of a plurality of related elements.

When it is said that a component is "docked with/to" or "connected to" another component, it should be understood that one component is docked or connected to the other component directly or through any other component in between. On the other hand, when it is said that a component is "directly coupled to" or "directly connected to" another component, it should be understood that one component is directly coupled or connected to the other component without any other component in between.

The terms used in this disclosure are provided to describe only specific embodiments, without intending to limit the scope of other embodiments. Singular forms should be understood to include plural referents unless the context clearly dictates otherwise. In this disclosure, the term "includes" or "has" means the presence of a feature, a number, a step, an operation, a component, a part, or a combination of two or more of them as described herein. disclosure, without excluding the presence of one or more characteristics, numbers, stages, operations, components, parts or a combination of two or more of them.

Unless otherwise defined, terms and words including technical or scientific terms used in the following description and claims may have the same meanings as are generally understood by the experts in the art. Terms, as generally defined in dictionaries, may be interpreted to have the same or similar meanings as the contextual meanings of the related technology. Unless otherwise defined, terms should not be construed as having ideal or overly formal meanings.

A video encoding apparatus and a video decoding apparatus as described below may be a personal computer (PC), a laptop, a personal digital assistant (PDA), a portable media player (PMP), a PlayStation Portable (PSP), a wireless communication terminal, a smartphone, and a server terminal, such as a TV application server and a service server, and can cover a wide range of devices, each of which includes a communication, such as a communication modem, to realize communication with a user terminal such as various devices or a wired/wireless communication network, a memory to store programs and data to encode or decode a video or to perform interprediction or intraprediction to encode or decode, and a microprocessor to perform computation and control by executing the programs.

In addition, a video encoded into a bit stream by the video encoding apparatus can be transmitted to the video decoding apparatus in real time or non-real time through a wired/wireless network such as the Internet, a wireless communication network of short range, a Wireless Local Area Network (WLAN), a wireless broadband network (WiBro), a mobile communication network, or various communication interfaces, such as a cable and a universal serial bus (USB). The video decoding apparatus can retrieve and play the video by decoding the received video.

In general, a video can be made up of a series of images, and each image can be divided into predetermined areas, such as frames or blocks. If a picture is divided into blocks, the divided blocks can be largely classified into intra-block and inter-block depending on a coding scheme. An intra-block refers to a block encoded by intra-prediction coding, and intra-prediction coding is a scheme for generating a prediction block by predicting pixels from a current block using pixels from previous blocks retrieved through encoding and decoding in a current image being encoded, and encode the differences between the pixels in the prediction block and those in the current block. An interblock refers to a block encoded by interprediction encoding. Interprediction encoding is a scheme for generating a prediction block by predicting from a current block of a current image by referencing one or more previous or future images and encoding the difference between the prediction block and the current block. A frame that is referenced to encode or decode a current image is called a reference frame. Furthermore, those skilled in the art can understand that the term, as used herein, "image" is interchangeable with other equivalent terms, image, frame, or the like. Furthermore, it can be understood by those skilled in the art that a reference image is a retrieved image.

Furthermore, the term block conceptually covers a coding unit (CU), a prediction unit (PU), and a transform unit (TU) defined in High Efficiency Video Coding (HEVC). In particular, motion estimation can be performed on a PU basis.

Specifically, a process of searching for a PU-like block in a previous coded frame is called Motion Estimation (ME). ME can be a process of finding a block that has the smallest error with a current block, meaning no actual movement of a block.

Furthermore, the present disclosure relates to a video codec technique for a multi-view video, and ME can be applied to a process of referencing an image of a different view. In this specification, the process of referencing an image from a different view may be referred to as interview prediction.

Now, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

First, a description is given of an embodiment of encoding and decoding a motion vector by motion vector prediction, and then an embodiment of encoding and decoding a residue by residue prediction.

Embodiment 1 - Method for encoding and decoding motion vector through motion vector prediction

Motion vector prediction can mean a process of computing a Temporal Motion Vector Predictor (TMVP) using a correlation between temporal motion vectors, or computing a Spatial Motion Vector Predictor (SMVP) using a correlation between motion vectors. space. A value calculated by subtracting a Motion Vector Predictor (MVP) from a current block's motion vector can be called a Motion Vector Difference (MVD).

Figure 1 is a conceptual view illustrating a motion vector prediction method according to an embodiment of the present disclosure.

Referring to Figure 1, a three-dimensional (3D) video can be constructed using images captured from a plurality of views in a multi-view video. A view can be distinguished or identified by a View ID.

Specifically, a multi-view video may include a video of a base view and at least one video of an enhanced view or an extension view.

In Figure 1, View ID 0 may identify an image in a reference view, View ID 1 may include an image (a current image) of a view currently to be encoded or decoded, and View ID 2 may include a image (a corresponding image) of a view that has been encoded or decoded before encoding and decoding the current image. A corresponding block PU ^col may refer to a block located in correspondence with the position of a current block PU ^curr in a picture other than the current picture Pic ^curr that includes the current block PU ^curr . For example, the corresponding PU ^col block may refer to a block co-located with the current PU ^curr block in a different image than the current Pic ^curr image. In addition, a corresponding image Pic ^col may refer to an image that includes the corresponding block PU ^col .

Motion estimation can be performed on the current image Pic ^curr to refer to an image in a different view or another image in the same view.

In the present disclosure, long term prediction may mean referring to an image of the same view apart from a current image by a predetermined time difference or further. Accordingly, referring to an image of the same view apart from a current image with a time difference less than the predetermined time difference can be referred to as short-term prediction.

A result of scaling a motion vector of the corresponding block PU ^col located in correspondence with the current block PU ^curr in the corresponding image Pic ^col can be used as a motion vector predictor (MVP) of the current block PU ^curr . The corresponding image Pic ^col is different from the current image Pic ^curr which includes the current block PU ^curr .

Figure 1 illustrates a case where an image from a different view is referenced for the current block PU ^curr and an image from a different view is also referenced for the corresponding block PU ^col . That is, the interviewed prediction can be performed both for the current block PU ^curr and for the corresponding block PU ^col .

In this case, the interview reference distance of the current block PU ^curr may be different from the interview reference distance of the corresponding block PU ^col . Herein, an interview reference distance may be the difference between the View IDs.

Referring to Figure 1, the current block PU ^curr belongs to View ID 1 and refers to a reference image Pic ^Ref belonging to View ID 0. That is, the inter-view reference distance of the current block PU ^curr is the difference between View IDs, 1.

The corresponding block PU ^col belongs to View ID 2 and refers to a reference image Pic ^Ref belonging to View ID 0. That is, the interview reference distance of the corresponding block PU ^col is the difference between the View IDs, two.

Because the interview reference distance of the current block PU ^curr is different from the interview reference distance of the corresponding block PU ^col , it is necessary to scale the motion vector of the corresponding block PU ^col .

Next, an operation for scaling the motion vector of the corresponding block PU ^col will be described in more detail.

In the illustrated case of Figure 1, the MVP of the current block PU ^curr for encoding or decoding a motion vector MV ^curr of the current block PU ^curr can be acquired by scaling the motion vector MV ^col of the corresponding block PU ^col .

The operation to scale the motion vector MV ^col of the corresponding block PU ^col is detailed below. Equation 1

Diff ^curr = ViewID ^curr - ViewID ^ref

diff ^col = ViewID ^col - ViewID ^colref

In Equation 1, the interview reference distance Diff ^curr of the current block PU ^curr is the difference between the ViewID ViewID ^curr of the current block PU ^curr and the ViewID ViewID ^Ref of the reference block of the current block PU ^curr .

The interview reference distance Diff ^coi of the corresponding block PU ^coi is the difference between the View ID ViewlD ^coi of the corresponding block PU ^col and the View ID ViewlD ^colref of the reference block of the corresponding block PU ^col .

Therefore, a scale factor (ScaleFactor) to be applied to the movement vector MV ^col of the corresponding block PU ^col can be calculated by the following Equation 2.

Equation 2

S ca le F ac to r = ^Diffcurr

diffcoy

Consequently, the motion vector predictor MVP ^curr of the current block PU ^curr can be calculated by multiplying the motion vector MV ^col of the corresponding block PU ^col by the scale factor.

Equation 3

MVP ^curr = ScaleFactor x MV ^col

That is, the motion vector predictor MVP ^curr of the current block PU ^curr can be expressed as Equation 3 above.

Fig. 2 is a conceptual view illustrating a motion vector prediction method according to another embodiment of the present disclosure.

Figure 2 illustrates a case where short-term prediction is performed for the current block PU ^curr and the corresponding block PU ^col . Short-term prediction may mean referring to an image of the same view apart from a current image with a time difference less than a predetermined time difference.

In the illustrated case of Figure 2, the motion vector predictor MVP ^curr of the current block PU ^curr can be generated by scaling the motion vector MV ^col of the corresponding block PU ^col using a relation between the inter-image reference distance of the current block PU ^curr and the interimage reference distance of the corresponding block PU ^col . An inter-image reference distance may be a Picture Order Count (POC) difference in time order.

The operation for scaling the motion vector MV ^col of the corresponding block PU ^col is described in more detail below.

Equation 4

Diff ^curr = POC ^curr - POC ^ref

diff ^col = POC ^col - POC ^colref

In Equation 4, the inter-image reference distance Diff ^curr of the current block PU ^curr is the difference between the POC POC ^curr of the current image to which the current block PU ^curr belongs and the POC POC ^Ref of a reference image to which the current block belongs. which belongs to a reference block that is referenced for the current block PU ^curr .

The interviewed reference distance Diff ^col of the corresponding block PU ^col is the difference between the POC POC ^col of the corresponding image to which the corresponding block PU ^col belongs and the POC POC ^colref of a reference image to which a block of reference referenced for the corresponding block PU ^col . A scale factor to be applied to the motion vector MV ^col of the corresponding block PU ^col can be calculated by the following Equation 5.

Equation 5

S ca le F ac to r = ^Diffcurr

diffcol

Consequently, the motion vector predictor MVP ^curr of the current block PU ^curr can be calculated by multiplying the motion vector MV ^col of the corresponding block PU ^col by the scale factor.

Equation 6

MVP ^curr = ScaleFactor x MV ^col

That is, the MVP MVP ^curr of the current block PU ^curr can be expressed as Equation 6 above.

Fig. 3 is a conceptual view illustrating a motion vector prediction method according to another embodiment of the present disclosure.

Figure 3 illustrates a case where a long-term prediction is performed for the current block PU ^curr and the corresponding block PU ^col . Long-term prediction may mean referring to an image of the same view apart from a current image with a time difference equal to or greater than a predetermined time difference.

When a long-term prediction is made for the current block PU ^curr and the corresponding block PU ^col , the motion vector MV ^col of the corresponding block PU ^col can be generated as the motion vector predictor MVP ^curr of the current block PU ^curr .

Equation 7

MVP ^curr = MV ^col

That is, the motion vector predictor MVP ^curr of the current block PU ^curr can be equal to the motion vector MV ^col of the corresponding block PU ^col , as shown in Equation 7.

It can be concluded that once the motion vector predictor MVP ^curr of the current block PU ^curr is determined according to Figures 1, 2 and 3, a motion vector difference (MVD) MVD ^curr of the current block PU ^curr can be determined. .

Equation 8

MVD ^curr = MV ^curr - MVP ^curr

That is, the MVD MVD ^curr of the current block PU ^curr can be determined by Equation 8. Therefore, the motion vector of the current block can be retrieved by adding the MVP of the current block to the MVD of the current block. Figures 4 and 5 are conceptual views illustrating motion vector prediction methods according to other embodiments of the present disclosure.

Figure 4 illustrates a case where a short-term prediction is made for the current block PU ^curr , and a long-term prediction is made for the corresponding block PU ^col .

Figure 5 illustrates a case where an interview prediction is made for the current block PU ^curr , and a long-term prediction is made for the corresponding block PU ^col .

If different prediction schemes are performed for the current block PU ^curr and the corresponding block PU ^col as illustrated in Figures 4 and 5, the motion vector MV ^col of the corresponding block PU ^col cannot be used to generate the MVP MVP ^curr of the current block PU ^curr .

Figure 6 is a flowchart illustrating a motion vector prediction method according to an embodiment of the present disclosure.

Referring to Figure 6, the motion vector prediction method according to the embodiment of the present disclosure includes determining performed motion prediction schemes for a current block and a corresponding block corresponding to the current block, and generating an MVP of the block based on a motion vector of the corresponding block according to the given motion vector prediction method.

It can be determined that for each current block and the corresponding block, one of long-term prediction, short-term prediction and interview prediction is performed.

That is, data for video decoding can be generated by decoding a received bit stream, and motion prediction schemes performed for the current frame and the corresponding frame using the data for video decoding can be determined. For example, motion prediction schemes may be determined by using at least View ID information, view order information, and flag information to identify a motion prediction scheme, included in the data for video decoding. Data for video decoding may be acquired by performing entropy decoding, dequantization, and inverse transformation on the received bit stream.

If a long-term prediction is made for the current block and the corresponding block, the motion vector of the corresponding block can be generated as the MVP of the current block.

Furthermore, if a short-term prediction is made for the current block and the corresponding block, the MVP of the current block can be generated by scaling the motion vector of the corresponding block using a relation between the interimage reference distance of the current block and the distance reference interimages of the corresponding block.

If an interview prediction is performed for the current block and the corresponding block, the MVP of the current block can be generated by scaling the motion vector of the corresponding block using a relationship between the interview reference distance of the current block and the interview reference distance of the current block. corresponding block.

On the other hand, if an interview prediction is made for one of the current block and the corresponding block and a long-term prediction or a short-term prediction is made for the other block, or if a long-term prediction is made for one of the current block and the corresponding block and a short-term prediction is made for the other block, the motion vector of the corresponding block cannot be used to calculate the MVP of the current block.

In case the motion vector of the corresponding block is not used since the motion prediction schemes for the current block and the corresponding block are different from each other, a predetermined vector can be generated as the MVP of the current block. For example, the default vector can be set equal to (0, 0).

Accordingly, the temporary motion vector predictors (TMVPs) of the current block are listed in [Table 1] below based on the correlation between the motion prediction scheme for the current block and the motion prediction scheme for the block correspondent.

[Table 1]

Before a description of the flowchart illustrated in Figure 6, each parameter is defined as indicated in [Table 2].

[Table 2]

The motion vector prediction method for a multi-view video according to the embodiment of the present disclosure will be described in more detail with reference to Figure 6.

If LT ^curr is different from LT ^col in step S610, this implies that the reference picture indicated by MV ^curr is marked differently from the reference picture indicated by MV ^col . For example, a short-term reference image is referenced for MV ^curr and a long-term reference image is referenced for MV ^col . In this case, a TMVP (S690) cannot be used.

If IV ^curr is different from IV ^col in step S610, this implies that MV ^curr and MV ^col have different properties. For example, MV ^curr is an interview motion vector, and MV ^col is a temporal motion vector. In this case, a TMVP (S690) cannot be used.

If IV ^curr is '1' in step S620, this implies that both MV ^curr and MV ^col are interview motion vectors, and it is possible to scale with the difference between the View IDs (S640).

If IV ^curr is '0' at step S620, this implies that both MV ^curr and MV ^col are time motion vectors, and scaling is possible with the difference between the p 0 c (S630).

In this memory, if BaseViewFlag ^curr is '0', this may mean that Pic ^curr is not a base view.

If Diff ^curr is different from Diff ^col , and IV ^curr is '1' or LT ^curr is '0' in step S650, MV ^col is escalated to TMVP ^curr (S670). If Diff ^curr is equal to Diff ^col in step S650, TMVP ^curr can be set to MV ^col (S660).

If long-term reference pictures are referenced for both the current block PU ^curr and the corresponding block PU ^col and the intervened prediction is not used for the current block PU ^curr and the corresponding block PU ^col in step S650, TMVP ^curr can be set to MV ^col (S660).

Embodiment 2 (not covered by the claimed invention) - Method for Encoding and Decoding Residue via Residue Prediction

A multiview video can be encoded and decoded through residual prediction. Advanced Residue Prediction (ARP) for a multi-view video can mean a process of generating a residue by predicting the motion of a current block and generating a prediction residue by performing a prediction on the generated residue.

Accordingly, a method of encoding and decoding a residue via residue prediction may mean encoding and decoding a residue difference generated by subtracting a prediction residue from a residue of a current block.

Figure 7 is a conceptual view illustrating a residue prediction method according to an embodiment of the present disclosure.

Referring to Figure 7, a view to which a current Curr block belongs is called a current view, and a view that is referenced for the current view is called a reference view.

If a reference block of the same view, which is referenced for the current block Curr, is denoted by CurrRef, a residue signal R can be computed using Equation 9. Herein, the motion vector of the current block Curr can be denoted by Mvcurr.

That is, the residual signal R can be calculated by subtracting the reference block CurrRef from the current block Curr, according to Equation 9.

Equation 9

R(i, j) = Curr(i, j) - CurrRef(i, j)

The residue signal R can further eliminate redundancy by using a similarity between the views. A corresponding block Base corresponding to the current block Curr can be detected using a disparity vector DVderived in a reference view.

A reference block BaseRef referenced to the corresponding Block Base at the temporary address can be detected using MVScaled generated by scaling the motion vector MVcurr of the current block Curr.

In this case, an image to which the reference block BaseRef referenced for the corresponding Block Base belongs may be an image that has the smallest POC difference from an image to which the Block Base belongs. corresponding block, in a list of reference images for the image to which the corresponding Block Base belongs, except for an image that has the same POC as the image to which the corresponding Block Base belongs.

MVScaled can be calculated using Equation 10 and Equation 11.

Equation 10

DiffPOC ^curr = POC ^Curr - POC ^CurrRef

DiffPOC ^Base = POC ^Base = POC ^BaseRef

D i f f P O C B a s e

ScaleFactor

D i f f P O C c u r r

In Equation 10, a temporary reference distance of the current block Curr can be denoted by DiffPOC ^curr , and DiffPOC ^curr can be computed as the difference between the POC POC ^curr of the current block Curr and the POC POC ^currref of the reference block CurrRef to which is referenced for the current Curr block at the temporary address.

In addition, a temporary reference distance of the corresponding block Base can be indicated by DiffPOC ^Base and DiffPOC ^Base can be calculated as the difference between the POC ^Base POC of the corresponding block Base and the POC POC ^baseref of the reference block BaseRef to which reference is made to the corresponding Block Base in the temporary address.

A scaling factor by which to scale the motion vector MV ^curr of the current block Curr can be expressed as a ratio of the time reference distance of the current block Curr to the time reference distance of the corresponding Block Base.

Equation 11

MV ^Scaled = ScaleFactor x MV ^curr

Therefore, MV ^Scaled can be generated by scaling the MV ^curr motion vector of the current Curr block with the scale factor, and the BaseRef referenced block referenced for the corresponding Block Base in the temporary address can be detected using MV ^Scaled .

Equation 12

R'(i, j) = [Base(i, j) — BaseRef (i, j)]

A prediction residue signal R' of the current block Curr can be calculated by Equation 12. That is, the prediction residue signal R' can be calculated by subtracting the reference block BaseRef referenced for Block Base in the time direction from the corresponding Block Base.

In addition, the prediction residue signal R' can be calculated by applying a weight w to the corresponding block Base or reference block BaseRef, or the prediction residue signal R' from Equation 12 can be set to be greater than a default threshold w.

Equation 13

R ^Final (i, j) = R(i, j) - R'(i, j)

Therefore, a residual difference can be calculated by subtracting the prediction residual signal R' of the current block Curr of Equation 12 from the residual signal R of the current block Curr of Equation 9.

Furthermore, the residual prediction depicted in Figure 7 and Equation 9 to Equation 13 can be referred to as Temporal ARP (Advanced Residual Prediction).

Fig. 8 is a conceptual view illustrating a residue prediction method according to another embodiment of the present disclosure.

Referring to Figure 8, let IvRef denote a reference block of a view other than the one referenced for the current Curr block. Then, the residual signal R can be calculated by Equation 14. The current block motion vector Curr can be indicated by MV ^curr , and interview prediction can be performed using MV ^curr .

That is, according to Equation 14 below, the residual signal R of the current block Curr can be calculated by subtracting the reference block IvRef from the current block Curr.

Equation 14

R(i, j) = Curr(i, j) — IvRef(i, j)

Referring to Equation 14, the residual signal R can further eliminate redundancy by using a similarity between the views. The reference block IvRef referenced in interview for the current block Curr can be detected using the motion vector MVcurr of the current block Curr in a reference view. A reference block IvTRef that is referenced to the reference block IvRef of the current block Curr at the temporary address can be detected using dTMV ^Base .

In addition, a reference block TRef referenced for the current block Curr at the temporary address can be detected using dMV ^Scaled generated from the dTMV ^Base scalar used for the reference block IvRef of the current block Curr.

In this case, an image of the reference block TRef referenced for the current block Curr in the time direction may have the smallest POC difference from an image of the current block Curr in a list of reference images for the image of the current Curr block, except for an image that has the same POC as the image of the current Curr block.

MV ^Scaled can be calculated using Equation 15 and Equation 16.

Equation 15

DiffPOC ^curr = POC ^curr - POC ^TRef

DiffPOC ^Base = POC ^lvRef - POC ^lvTRef

ScaleFactor = ^{D lff POC curr}

D'ffPOCBase

In Equation 15, the temporary reference distance of the current block Curr can be denoted by DiffPOC ^curr , and DiffPOC ^curr can be calculated as the difference between the POC POC ^curr of the current block Curr and the POC POC ^TRef of the reference block TRef to which is referenced for the current Curr block at the temporary address.

In addition, the temporary reference distance of the reference block IvRef can be indicated by DiffPOC ^Base , and DiffPOC ^Base can be calculated as the difference between the POC POC ^IvRef of the reference block IvRef and the POC POC ^IvTRef of the reference block IvTRef to which it refers. references to the IvRef reference block at the temporary address.

A scaling factor by which to scale the motion vector dTMV ^Base of the reference block IvRef can be expressed as a ratio of the current block's time reference distance Curr to the time reference distance of the corresponding block's Base.

Equation 16

dTMV ^Scaled = ScaleFactor x dTMV ^Base

Therefore, dTMV ^Scaled can be generated by scaling the ^Base dTMV motion vector of the reference block IvRef with the scale factor, as expressed in Equation 16, and the reference block TRef referenced for the current block Curr in the temporal direction can be detected using dTMV ^Scaled .

Equation 17

R'(i, j) = [TRef(i, j) - IvTRef(i, j)]

The prediction residue signal R' of the current block Curr can be calculated by Equation 17. That is, the prediction residue signal R' can be calculated by subtracting the reference block IvTRef referenced for the reference block IvRef of the current block Curr at the temporary address from the reference block TRef referenced for the current block Curr at the temporary address.

In addition, the prediction residual signal R' can be calculated by applying the weight w to the reference block TRef or IvTRef, or the prediction residual signal R' of Equation 17 can be set to be greater than the predetermined threshold w.

Equation 18

R ^Final (i, j) = R(i, j) - R'(i, j)

Therefore, the residual difference can be calculated by subtracting the prediction residual signal R' of the current block Curr of Equation 17 from the residual signal R of the current block Curr of Equation 14. This residual difference can be called final residue R ^Final as in Equation 18.

In addition, the residue prediction depicted in Figure 8 and Equation 14 to Equation 18 may be referred to as Interview ARP.

Fig. 9 is a conceptual view illustrating a residue prediction method according to another embodiment of the present disclosure.

Referring to Figure 9, a reference block IvRef of View ID 1 referenced in the view can be detected for the current block Curr using the motion vector MVcurr.

Compared to the case of Figure 8, a reference block IvTRef of a view other than the one referenced for the reference block IvRef of the current block Curr can be detected using a motion vector MV ^Ref in the illustrated case of Figure 9.

In this case, a problem occurs in the generation of ^scaled MV by scaling the MV ^Ref used for the IvRef reference block of the current Curr block. That is, this is because MV ^Ref is a motion vector used for interview prediction, while MV ^scaled is a vector used for temporal motion prediction.

More specifically, since MV ^Ref is a motion vector used for interview prediction, a denominator plotted in Equation 15 becomes 0 and, as a result, the scale factor is calculated as an infinite value.

Equation 19

MV scaled = ^ScaleFactor X MV ^Ref

Therefore, since an error may occur during the calculation of MV ^Scaled by Equation 19, the error can be prevented by setting MV ^Scaled to (0, 0).

Figures 8 and 9 will be described below assuming that the reference block IvRef is a first reference block, the reference block TRef is a second reference block, and the reference block IvTRef is a third reference block.

The residual prediction method includes determining a realized motion prediction scheme for the first referenced block referenced for the intervened prediction of the current block, and generating a prediction residual for the current block according to the motion prediction scheme. movement of the first reference block.

It can be determined which of the temporal motion prediction and interview prediction is performed for the first reference block.

The difference between the second reference block referenced for the current block temporal motion prediction and the third reference block referenced for the first reference block may be generated as a prediction residual. In this memory, the second reference block may belong to a picture closest in time address in a reference list for a current picture to which the current block belongs.

If it is determined that temporal motion prediction is performed for the first reference block, a scaled motion vector can be generated by applying a scale vector to a motion vector used to search for the third reference block, and the second reference block. reference can be determined using the scaled motion vector. The scale factor can be generated based on the difference between the number of a reference image to which the first reference block belongs and the number of an image to which the third reference block belongs to which is referenced for the temporal motion prediction of the first reference block, and the difference between the number of a picture to which the current block belongs and the number of a picture to which the second reference block belongs.

On the other hand, if it is determined that interview prediction is performed for the first reference block, the second reference block can be determined by applying (0, 0) as the motion vector used to search for the second reference block.

Figure 10 is a block diagram of an apparatus for encoding a multiview video and an apparatus for decoding a multiview video according to an embodiment of the present disclosure.

Referring to Figure 10, a system for encoding/decoding a multi-view video according to an embodiment of the present disclosure includes a multi-view video encoding apparatus 10 and a multi-view video decoding apparatus 20.

The multiview video encoding apparatus 10 may include a base view video encoder 11 for encoding a base view video and extension view video encoders 12 and 13 for encoding an extension view video. A base view video may be a video to provide a 2D single view video, and an extent view video may be a video to provide an extent view video such as 3D.

For example, multi-view video encoding apparatus 10 may include base-view video encoder 11, first extension-view video encoder 12, and second extension-view video encoder 13. are not limited to the first and second extension view video encoders 12 and 13. Rather, the number of extension view video encoders may increase with the number of views. In addition, the base view video encoder 11 and the extension view video encoders 12 and 13 can encode a color image and a depth image (depth map) separately. The multi-view video encoding apparatus 10 can transmit a bit stream obtained by encoding a multi-view video to the multi-view video decoding apparatus 20.

The multiview video decoding apparatus 20 may include a bitstream extractor 29, a base view video decoder 21, and extension view video decoders 22 and 23.

For example, the multi-view video decoding apparatus 20 may include the base-view video decoder 21, a first extension-view video decoder 22, and a second extension-view video decoder 23. Obviously, the number of decoders extension view video can increase with the number of views.

Specifically, bitstream extractor 29 can separate a bitstream according to views and provide the separated bitstreams to base view video decoder 21 and extension view video decoders 22 and 23, respectively.

According to one embodiment of the present disclosure, a decoded base view video can be displayed on a legacy 2D display, with backwards compatibility. In addition, the decoded base view video and at least one decoded extension view video may be displayed on a stereo screen or a multi-view screen.

Meanwhile, input camera position information may be transmitted as secondary information in a bit stream to the stereo display or multi-view display.

Figure 11 is a block diagram of an apparatus for encoding a multi-view video according to an embodiment of the present disclosure.

Referring to Figure 11, the multiview video encoding apparatus 10 according to the embodiment of the present disclosure may include the base view video encoder 11 and the extension view video encoder 12.

However, the multiview video encoding apparatus 10 may further include another extension view video encoder according to a view.

Each of the base view video encoder 11 and the extension view video encoder 12 includes a subtracter 110 or 110-1, a transformer 120 or 120-1, a quantizer 130 or 130-1, a dequantizer 131 or 131 -1, a 121 or 121-1 reverse transformer, a 140 or 140-1 entropy encoder, a 150 or 150-1 adder, a 160 or 160-1 loop filter unit, a 170 or 170 frame memory- 1, a 180 or 180-1 intrapredictor, and a 190 or 190-1 motion compensator.

Subtractor 110 or 110-1 generates a residual image between a received image to be encoded (a current image) and a prediction image generated by intraprediction or interprediction by subtracting the prediction image from the current image.

Transformer 120 or 120-1 transforms the residual image generated by subtractor 110 or 110-1 from the spatial domain to the frequency domain. Transformer 120 or 120-1 may transform the residual image to the frequency domain by a technique of transforming a spatial video signal into a frequency video signal, such as a Hadamard transform, discrete cosine transform, or alpha transform. discreet breast. Quantizer 130 or 130-1 quantizes the transformed data (frequency coefficients) received from transformer 120 or 120-1. That is, the quantizer 130 or 130-1 quantizes the frequency coefficients being data transformed by the transformer 120 or 120-1 by dividing the frequency coefficients by a quantization step size, and thus obtains quantization result values.

Entropy encoder 140 or 140-1 generates a bit stream by entropy encoding the quantization result values calculated by quantizer 130 or 130-1. In addition, the entropy encoder 140 or 140-1 may entropy encode the quantization result values computed by the quantizer 130 or 130-1 using context-adaptive variable-length coding (CAVLC) or context-adaptive binary arithmetic coding ( CABAC), and more entropy encoding information may be required for video decoding in addition to the quantization result values.

The dequantizer 131 or 131-1 dequantizes the quantization result values calculated by the quantizer 130 or 130-1. That is, the dequantizer 131 or 13-1 retrieves frequency domain values (frequency coefficients) from the quantization result values.

The dequantizer 121 or 121-1 recovers the residual image by transforming the frequency domain values (frequency coefficients) received from the dequantizer 131 or 131-1 to the spatial domain. Adder 150 or 150-1 generates a retrieved image of the input image by adding the residual image retrieved by dequantizer 121 or 121-1 to the prediction image generated by intraprediction or interprediction, and stores the retrieved image in memory 170. or 170-1.

Intrapredictor 180 or 180-1 performs intraprediction, and motion compensator 190 or 190-1 compensates a motion vector for interprediction. The intrapredictor 180 or 180-1 and the motion compensator 190 or 190-1 may collectively be referred to as a prediction unit.

According to one embodiment of the present disclosure, predictors 180-1 and 190-1 included in extension view video encoder 12 can make predictions for a current block of an extension view using prediction information about a reference block of extension view. a reference view. The reference view refers to a view that is referenced for the extension view and can be a base view. Furthermore, the prediction information may include prediction mode information and motion information about a reference block.

Loop filter unit 160 or 160-1 filters the retrieved image. Loop filter unit 160 or 160-1 may include a deblocking filter (DF) and a sample adaptive shift (SAO).

A multiplexer 330 may receive an encoded base-view video bitstream and an encoded spread-view video bitstream and thereby output an extended bitstream.

Particularly, the multi-view video encoding apparatus 10 according to the embodiment of the present disclosure may further include an interview predictor 310 and a residual predictor 320.

Although interview predictor 310 and residual predictor 320 are shown in Figure 11 as residing between base view video encoder 11 and extension view video encoder 12, the present disclosure is not limited to this structure or position.

Interview predictor 310 may interact with motion compensator 190 or 190-1 and encode a motion vector for a multiview video through motion vector prediction according to the above-described first embodiment of the present disclosure.

Residue predictor 320 may interact with motion compensator 190 or 190-1 and intrapredictor 180 or 180-1, and encode a residue for a multi-view video via residue prediction according to the above-described second embodiment herein. divulgation.

Figure 12 is a block diagram of a multi-view video decoding apparatus according to an embodiment of the present disclosure.

Referring to Figure 12, multiview video decoding apparatus 20 may include bitstream extractor 29, base view video decoder 21, and extension view video decoders 22 and 23. The bit stream 29 can separate a bit stream according to views and provide the separate bit streams to base view video decoder 21 and extension view video decoders 22 and 23, respectively.

Each of the base view video decoders 21 and extension view video decoders 22 and 23 may include an entropy decoder 210 or 210-1, a dequantizer 220 or 220-1, a reverse transformer 230 or 230-1 2, an adder 240 or 240-1, a loop filter unit 250 or 250-1, a frame store 260 or 260 1, an intrapredictor 270 or 270-1, and a motion compensator 280 or 280-1. The intrapredictor 270 or 270-1 and the motion compensator 280 or 280-1 may collectively be referred to as a prediction unit.

The multi-view video decoding apparatus 20 according to the embodiment of the present disclosure may further include an interview predictor 410 and a residue predictor 420.

Although the interview predictor 410 and residue predictor 420 are shown in Figure 12 as residing between the base view video decoder 21 and the extension view video decoder 22, the present disclosure is not limited to this structure or position.

Interview predictor 410 may interact with motion compensator 290 or 290-1 and decode a motion vector for a multi-view video through motion vector prediction according to the above-described first embodiment of the present disclosure.

Residue predictor 420 may interact with motion compensator 290 or 290-1 and intrapredictor 280 or 280-1, and decode a residue for a multi-view video via residue prediction according to the above-described second embodiment herein. divulgation.

Meanwhile, each component of the multi-view video decoding apparatus 20 can be understood from its equivalent of the multi-view video encoding apparatus 10 illustrated in Figure 11, and therefore will not be described in detail here.

Furthermore, each component of the multi-view video encoding apparatus 10 and the multi-view video decoding apparatus 20 according to embodiments of the present disclosure has been described as being configured as a separate component, for convenience of description. However, at least two of the components may be incorporated into a single processor, or one component may be partitioned into a plurality of processors, to perform a function. Embodiments for incorporating components or separating a single component also fall within the appended claims without departing from the scope of the present disclosure.

The multi-view video encoding apparatus 10 and the multi-view video decoding apparatus 20 according to the present disclosure may be implemented as a computer-readable program or code on a computer-readable recording medium. Computer-readable recording medium includes any type of recording device that stores data readable by a computer system. Furthermore, the computer-readable recording medium may be distributed to computer systems connected via a network and store and execute a computer-readable program or code in a distributed manner.

The method for performing motion vector prediction for a multi-view video according to the first embodiment of the present disclosure enables effective encoding/decoding of a motion vector during encoding/decoding of a multi-view video. That is, according to the present disclosure, a temporary motion vector can be adaptively predicted according to the motion vector prediction schemes used for a current block and a corresponding block.

The method for performing residue prediction for a multi-view video according to the second embodiment of the present disclosure enables effective encoding/decoding of a residue during encoding/decoding of a multi-view video. That is, an error in calculating a scale factor used for scaling a motion vector can be prevented from occurring during the generation of a prediction residual, thus preventing an error in the prediction residual for a multi-view video.

Although the present disclosure has been described with reference to the preferred embodiments, those skilled in the art will appreciate that various modifications and variations may be made to this disclosure without departing from the scope of the present disclosure described in the appended claims.

Claims (5)

1. A method for decoding a multi-view video, the method comprising: determining motion prediction schemes performed so that a current block and a collocated block corresponding to the current block are decoded, determining that the motion prediction schemes are a long-term prediction, a short-term prediction, and an interval prediction; Y generate a current block motion vector using a co-located block motion vector according to the given motion prediction schemes, where when interview prediction is performed for one of the current block and collocated block and then long-term prediction or short-term prediction is performed for the other block, the motion vector of the current block is generated using a predetermined vector or the motion vector of the collocated block, wherein the co-located block includes a co-located temporary block. The method according to claim 1, wherein generating the current block motion vector comprises, when performing long-term prediction for the current block and collocated block, generating the current block motion vector as the movement of the co-located block. The method according to claim 1, wherein generating the current block motion vector comprises, when short-term prediction is performed for the current block and collocated block, generating the current block motion vector by scaling the motion vector moving the collocated block using a relationship between an inter-image reference distance of the current block and an inter-image reference distance of the collocated block. 4. The method according to claim 1, wherein generating the motion vector of the current block comprises, when the interviewed prediction is performed for the current block and the co-located block, generating the motion vector of the current block by scaling the motion vector of the collocated block using a relationship between an interview reference distance of the current block and an interview reference distance of the collocated block. 5. The method according to claim 1, wherein the predetermined vector is (0, 0).