KR20140043483A - Mvc based 3dvc codec supporting inside view motion prediction (ivmp) mode - Google Patents

Abstract

This disclosure describes features and techniques applicable to three-dimensional (3D) video coding. In one example, the technique includes coding the texture view video block, and coding the depth view video block, wherein the depth view video block is associated with the texture view video block. Coding the depth view video block may include coding a syntax element indicating whether motion information associated with the texture view video block is adopted as motion information associated with the depth view video block.

Description

MVC BASED 3DVC CODEC SUPPORTING INSIDE VIEW MOTION PREDICTION (IVMP) MODE}

[0002]

United States Patent Provisional Application 61 / 561,800, filed November 18, 2011;

United States Patent Provisional Application 61 / 563,771, filed November 26, 2011;

United States Patent Provisional Application 61 / 522,559, filed August 11, 2011;

US Provisional Application No. 61 / 510,738, filed July 22, 2011;

United States Patent Provisional Application 61 / 522,584, filed August 11, 2011;

US Provisional Application No. 61 / 563,772, filed November 26, 2011; And

United States Patent Provisional Application 61 / 624,031, filed August 13, 2011

Is hereby prioritized, the entire contents of each of which are incorporated herein by reference.

This disclosure relates to three-dimensional (3D) video coding.

Digital video capabilities include, but are not limited to, digital television, digital direct broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, e-book readers, A wide variety of devices, including recording devices, digital media players, video gaming devices, video game consoles, cellular or satellite wireless telephones, so-called "smart phones", remote video conferencing devices, video streaming devices, . Digital video devices include MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4, Part 10, AVC (Advanced Video Coding), HEVC (High Efficiency Video Coding) Standards, and techniques described in the standards defined by the extensions of these standards. Video devices may implement such video compression techniques to more efficiently transmit, receive, encode, decode, and / or store digital video information.

Video compression techniques perform spatial (intra-picture) prediction and / or temporal (inter-picture) prediction to reduce or eliminate redundancy inherent in video sequences. In block-based video coding, a video slice (e.g., a portion of a video frame or a video picture) may be partitioned into video blocks, which may also include triblocks, coding units (CUs), and / May be referred to as coding nodes. The video blocks in the intra-coded (I) slice of the picture are encoded using spatial prediction for reference samples in neighboring blocks in the same picture. The video blocks in the inter-coded (P or B) slice of the picture may use spatial prediction for reference samples in neighboring blocks in the same frame or temporal prediction for reference samples in different reference frames . The pictures may be referred to as frames, and the reference pictures may be referred to the reference pictures. Pictures may be referred to as frames, and reference pictures may be referred to as reference frames.

Spatial or temporal prediction results in a prediction block for the block being coded. The residual data represents the pixel differences between the original block being coded and the prediction block. The inter-coded block is encoded according to the motion vector indicating the block of reference samples forming the prediction block, and the residual data indicating the difference between the coded block and the prediction block. The intra-coded block is encoded according to the intra-coding mode and the residual data. For further compression, the residual data may be transformed from the pixel domain to the transform domain, resulting in residual transform coefficients, which may then be quantized. The quantized transform coefficients, initially arranged in a two-dimensional array, may be scanned to generate a one-dimensional vector of transform coefficients, and entropy coding may be applied to achieve even more compression.

Although three-dimensional (3D) video is highly desirable for many applications, many challenges exist for 3D video coding.

This disclosure describes features and techniques applicable to three-dimensional (3D) video coding. In one example, the technique includes coding the texture view video block and coding the depth view video block, the depth view video block being associated with the texture view video block. Coding a depth view video block may include coding a syntax element that indicates whether the texture view video block is adopted as motion information associated with the depth view video block.

The techniques described may correspond to a coding mode referred to herein as inside view motion prediction (IMVP) mode. In this case, the depth view component (eg, depth view video block) may not include any additional delta values for that motion information, but instead employs the motion information of the texture view component as its motion information. You can also By defining a mode that fully adopts the motion information of the texture view as the motion information of the depth view, improved compression may be realized without any signaling of delta values for this motion information.

In another example, this disclosure describes a device for coding 3D video data, the device including one or more processors configured to code a texture view video block and code a depth view video block, wherein the depth view video block is a texture view. Associated with the video block. Coding a depth view video block includes coding a syntax element that indicates whether the texture view video block is adopted as motion information associated with the depth view video block.

In another example, the present disclosure describes a computer readable storage medium having stored thereon instructions that, when executed, cause one or more processors to code a texture view video block and code a depth view video block when executed. The depth view video block is associated with the texture view video block. Coding a depth view video block includes coding a syntax element that indicates whether the texture view video block is adopted as motion information associated with the depth view video block.

In another example, this disclosure describes a device configured to code 3D video data, the device including means for coding a texture view video block and means for coding a depth view video block, wherein the depth view video block is a texture view. The means associated with the video block and coding the depth view video block includes means for coding a syntax element that indicates whether the texture view video block is adopted as motion information associated with the depth view video block.

The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description and drawings, and from the claims.

1 is a block diagram illustrating an example video encoding and decoding system that may utilize the techniques described in this disclosure.
2 is a block diagram illustrating an example video encoder that may implement the techniques described in this disclosure.
3 is a block diagram illustrating an example video decoder that may implement the techniques described in this disclosure.
4 is a conceptual diagram illustrating a bitstream order of video coding layer (VCL) network abstraction layer (NAL) units of view components within one access unit.
5 is a conceptual illustration of a sequence of pictures forming a video sequence, wherein the identified in the motion vector of the co-located macroblock (MB) in the fourth picture of the texture view and the fourth picture of the depth view. Macroblocks (MB) are reused for depth view components.
6 is a conceptual diagram illustrating a prediction structure that may be used by a three-dimensional video coding (3DVC) codec.
7 is a conceptual diagram illustrating a prediction structure of a 3DVC codec that does not allow inter-view prediction for depth view components.
8 is a conceptual diagram illustrating an example of asymmetric inter-view prediction, where the left view (VL) and the right view (VR) have half widths.
9 is a flowchart illustrating a technique that may be performed by a video encoder consistent with this disclosure.
10 is a flowchart illustrating a technique that may be performed by a video decoder consistent with this disclosure.

The techniques of this disclosure are three-dimensional based on the ITU-T H.264 / AVC standard and one or more extensions that support multi-view coding (MVC), such as Annex H of the ITU-T H.264 / AVC standard. (3D) relates to video coding. However, the present techniques may be applied to other video coding standards or techniques, such as emerging HEVC standard, ITU-T H.264 / AVC standards or emerging HEVC standard or proprietary video coding techniques, For example, it can be applied to On2 VP6 / VP7 / VP8.

In 3D video coding, there are a number of different views collectively used to define a 3D video presentation. In addition, each different view may include both a texture view component and a depth view component. Texture view components may be coded into blocks of video data, which are coded as "video blocks" and are commonly referred to as "macro blocks" in the H.264 context. Similarly, depth view components are coded as "video blocks" and are generally referred to as "macro blocks" in the H.264 context. Each texture video block may have a corresponding depth view block. Different video blocks (texture and depth) are however usually coded separately. Other video coding standards may refer to tree blocks or coding units (CUs) for a video block.

In inner coordination, the motion vectors (or motion vector difference values for the motion vector predictor) may be used to define the predictive blocks, which are then used to predict the values of the coded video blocks. In this case, so-called "residual values" or "difference values" are included in the encoded bitstream, along with motion vectors (or motion vector difference values for the motion vector predictor) that identify corresponding prediction blocks. The decoder receives the motion vectors and the residual values and identifies the prediction blocks from the previously decoded video data using the motion vectors. To reconstruct the encoded video blocks, the decoder combines the residual values with the corresponding prediction blocks identified by the motion vectors.

There are many potential problems with 3D video coding. For example, when coding multiview video data, the following problems may need to be solved to realize an effective codec.

1. providing the ability for joint coding of texture and depth components for one or more views;

2. providing the ability to take advantage of motion redundancy between texture and depth;

3. providing the ability to transmit camera parameters in a simple and effective manner;

4. When adopting a view, inter_view_flag may be used to discard the view component, unless it belongs to the view being used for output. However, in the case of asymmetric 3DV, a network abstraction layer (NAL) unit may still be required for prediction of views with different resolution even if the flag is equal to zero.

To solve the above problem, several techniques may be used, including the following.

1. A framework that supports joint coding of depth views and texture views.

2. A new inside view motion prediction (IVMP) mode may be used at the macroblock (or other video block or CU) level to enable reuse of motion vectors between depth and texture views. Aspects of the IVMP mode are described in detail in this disclosure.

 3. Camera parameters and depth ranges may be added in a sequence parameter set (SPS) or added as new supplemental enhancement information (SEI) messages, and if this parameter changes on a picture basis, view parameter set ) Or an SEI message may be added.

4. The network abstraction layer (NAL) unit header indicating whether the semantics of inter_view_flag may change or new syntax elements are also discardable for views with the same resolution, which are not discardable for views with different resolutions. It can also be defined in.

5. In addition to nal_unit_type (eg, 21) to be used by the depth view component, one example may be to add a new new nal_unit_type (eg, 22) for texture view components that are not compatible with H.264 / MVC. It includes more.

This disclosure may use the following definitions.

View component: A coded representation of a view in a single access unit . When a view includes both coded texture and depth representations, the view component consists of a texture view component and a depth view component.

Texture view component: A coded representation of the texture of a view in a single access unit .

Depth view component : A coded representation of the depth of a view in a single access unit .

Coded video coding layer (VCL) network abstraction layer (NAL) units in the depth view component may be assigned nal_unit_type 21 as a new type of coded slice extension specifically for the depth view components. Texture view components and depth view components may also be referred to herein as texture view video blocks and depth view video blocks.

An exemplary bit stream order will be described below. In some examples, in each view component, any coded slice NAL unit (having nal_unit_type 21) of the depth view component should follow all coded slice NAL units of the texture view component. For simplicity, the present application may name coded slice NAL units of the depth view component as depth NAL units.

The depth NAL unit may have the same NAL unit header structure as the NAL unit with nal_unit_type equal to 20. 4 is a conceptual diagram illustrating a bit stream order of VCL NAL units of view components within one access unit.

As shown in FIG. 4, according to this disclosure, an access unit includes multiple NAL units with multiple view components. Each view component may consist of one texture view component and one depth view component. The texture view component of a base view with a view order index (VOIdx) equal to 0 may include one prefix NAL unit (having a NAL unit type such as 4) and one or more AVC VCL NAL units (eg, 1 or 5). Having the same NAL unit type). Texture view components in other views include only MVC VCL NAL units (having a NAL unit type such as 20). In both the base view and non-base views, the depth view components include depth NAL units with a depth NAL unit type such as 21. In any view component, the depth NAL units follow the NAL units of the texture view component in decoding / bitstream order.

Since texture view components and their associated depth view components have similar object silhouettes, they typically have similar object boundaries and movement. Thus, there is redundancy in these motion fields. If the texture view block and depth view block exist in the same NAL unit and / or they correspond to the same (overlapping) spatial and / or temporal instance of 3D video data, the texture view block and depth view block may be “associated”. have. The techniques of this disclosure may utilize this redundancy to a large extent by allowing a mode in which the depth view component fully adopts motion information of the associated texture view component in a manner similar to the so-called "merge" mode. In this case, the depth view component may not include any additional delta values for that motion information, but instead may employ the information of the texture view component as its motion information. By defining a mode that fully adopts the motion information of the texture view as motion information of the depth view, enhanced compression may be realized without any signaling of delta values for such motion information.

In particular, motion prediction from the texture view component for the associated depth view component may be enabled according to a new mode of merging the motion information of the texture view as the motion information of the depth view. In some examples, this so-called inside view motion prediction (IVMP) mode may be enabled only for inter coded MBs with depth view components. In the IVMP mode, motion information including mb_type, sub_mb_type, reference indices, and motion vectors of MBs commonly located in the texture view component are reused by the depth view component of the same view. A flag indicating whether to use the IVMP mode may be signaled in each MB. That is, the flag may be defined at the video block level, eg, macroblock level. The flag may be included with depth video blocks (eg, depth macro blocks). As shown in FIG. 5, the flag may be true for the identified MB in the fourth picture of the depth view, and the motion vector of the MB co-located in the fourth picture of the texture view (identified as the fourth picture) is the depth of field. It is reused for the MB highlighted in the view component. Note that in some examples, the IVMP mode only applies to non-anchor pictures.

Also, compared to techniques for predicting a motion vector for one view based on the motion of another view, the techniques of this disclosure may realize additional compression. For example, some scalable video coding (SVC) techniques may allow motion prediction of the reinforcement view based on the motion information of the base view, and in some cases, the base view may be a texture view and the reinforcement view may be a depth view. . However, in this case, motion vector difference data (eg, delta) is always coded in addition to the prediction information (or flag) indicating that the base view is used to predict the reinforcement view. In contrast, the techniques of this disclosure may use an IVMP mode in which delta information is not coded or allowed (eg, a motion vector difference value is not coded or allowed). Instead, in IVMP mode, the motion information of the texture view is taken as the motion information of the depth view.

When the motion information of the texture view is adopted as the motion information of the depth view, the decoder uses the depth information using the texture information's motion information (e.g., texture block) without receiving or decoding any other motion information for the depth view. A view (eg, a corresponding depth block) can be decoded. In particular, the decoder can be configured to interpret the IVMP flag in this manner. Thus, when the IVMP flag is enabled, motion information from the depth video block may be excluded and the decoder means that the enabled IVMP flag may be obtained from the corresponding test video block with motion information for the depth video block. It can be configured to know that.

An encoder according to this disclosure may generally follow a joint multiview video coding (JMVC) encoder scheme, with views encoded one by one. Inside each view, the texture sequence is encoded first, and the depth sequence is then encoded.

When IVMP mode is enabled, during texture view component encoding, the motion field of each texture view component is recorded in a motion file, and the name can be specified in the configuration file. When encoding the relevant depth sequences of the same view, the motion file can be read for reference.

The decoder may be similar to the JMVC decoder, and in some aspects there are also changes in decoding and outputting the depth sequence for each view. When the IVMP mode is enabled, the motion of each texture view component is stored and adopted as the motion of each corresponding depth view. In any blocks for which IVMP mode is disabled, the depth view may include its motion information or may include some other syntax elements that identify where to obtain, predict, and / or employ each motion information. have. However, if the IVMP mode is enabled, the depth view does not include its motion information, and the motion information is obtained by the decoder from the corresponding texture view component. Thus, when the IVMP mode is enabled, the depth view video block adopts motion information of the corresponding texture view video block so that the depth view video block does not contain its own motion information.

The following description of FIGS. 1, 2 and 3 describe some example scenarios in which the MVC-based 3DVC techniques of this disclosure may be used.

1 is a block diagram illustrating an example video encoding and decoding system 10 that may utilize the techniques described in this disclosure. As shown in FIG. 1, system 10 includes a source device 12 that generates encoded video data to be later decoded by destination device 14. The source device 12 and destination device 14 may be any type of device, such as desktop computers, laptop computers, tablet computers, set-top boxes, telephone handsets such as so- May include any of a wide variety of devices, including pads, televisions, cameras, display devices, digital media players, video gaming consoles, video streaming devices, and the like. In some cases, the source device 12 and the destination device 14 may be mounted for wireless communication.

Destination device 14 may receive the encoded video data over link 16. The link 16 may include any type of media or device capable of moving the encoded video data from the source device 12 to the destination device 14. [ In one example, computer-readable medium 16 may include a communication medium that enables source device 12 to transmit encoded video data directly to destination device 14 in real time. The encoded video data may be modulated according to a communication standard such as a wireless communication protocol and transmitted to the destination device 14. [ The communication medium may comprise any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network, such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful for facilitating communication from the source device 12 to the destination device 14.

Alternatively, the encoded data may be output from the output interface 22 to the storage device 32. Similarly, encoded data may be accessed by the input interface from storage device 32. Storage device 32 may be a hard drive, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other suitable digital storage media for storing encoded video data. It may include any of a variety of distributed or locally accessed data storage media. In a further example, storage device 32 may correspond to a file server or another intermediate storage device that may maintain encoded video generated by source device 12. Destination device 14 may access stored video data via streaming or download from the storage device. The file server may be any kind of server capable of storing encoded video data and transmitting the encoded video data to the destination device 14. [ Exemplary file servers include a web server (e.g., for a web site), an FTP server, network attached storage (NAS) devices, or a local disk drive. The destination device 14 may access the encoded video data via any standard data connection, including an Internet connection. This may include a wireless channel (eg, Wi-Fi connection), a wired connection (eg, DSL, cable modem, etc.), or a combination of both, suitable for accessing encoded video data stored on a file server. The transmission of encoded video data from storage device 32 may be a streaming transmission, a download transmission, or a combination thereof.

The techniques of this disclosure, however, are not necessarily limited to wireless applications or settings. These techniques include over-the-air television broadcasts, cable television transmissions, satellite television transmissions, streaming video transmissions, such as encoding of digital video for storage onto a data storage medium via the Internet, on a data storage medium. It may be applied to video coding with the support of any of a variety of multimedia applications, such as decoding of digital video stored in, or other applications. In some instances, the system 10 may be configured to support one-way or two-way video transmission to support applications, such as video streaming, video playback, video broadcasting, and / or video telephony .

In the example of FIG. 1, the source device 12 includes a video source 18, a video encoder 20, and an output interface 22. In some cases, output interface 22 may include a modulator / demodulator (modem) and / or a transmitter. At source device 12, video source 18 receives a source, such as a video capture device, eg, a video camera, a video archive containing previously captured video, or a video supplied from a video content provider. A video feed interface, and / or a computer graphics system that generates computer graphics data as a source video, a combination of source videos. As an example, if video source 18 is a video camera, source device 12 and destination device 14 may form so-called camera phones or video phones. However, the techniques described in this disclosure may be applicable to video coding in general and may be applied to wireless and / or wired applications.

The captured, pre-captured, or computer-generated video may be encoded by the video encoder 12. The encoded video data may be sent directly to the destination device 14 via the output interface 22 of the source video 20. The encoded video data is also (or alternatively) stored on storage device 32 for later access for decoding and / or playback by destination device 14 or other devices.

Destination device 14 includes input interface 28, video decoder 30, and display device 31. In some cases, input interface 28 may include a receive and / or modem. Input interface 28 of destination device 14 receives encoded video data via link 16. Encoded video data communicated via link 16 or stored on storage device 32 is passed to video encoder 20 for use by a video decoder, such as video decoder 30, in decoding the video data. It may also contain several syntax elements generated by it. Such syntax elements may be included with the encoded video data stored on a file server, stored on a storage medium, or transmitted via a communication medium.

Display device 31 may be integrated with or may be external to destination device 14. In some examples, destination device 14 may include an integrated display and also be configured to interface with an external display device. In other examples, the destination device 14 may be a display device. In general, display device 31 displays decoded video data to a user and may be any of several display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device. have.

Video encoder 20 and video decoder 30 may operate according to a video compression standard, such as the High Efficiency Video Coding (HEVC) standard currently under development, and may conform to the HEVC Test Model (HM). As an alternative, video encoder 20 and video decoder 30 are other proprietary or industrial standards, such as the ITU-T H.264 standard, otherwise referred to as MPEG-4, Part 10, Advanced Video Coding (AVC). Or may operate according to extensions of this standard. However, the techniques of this disclosure are not limited to any particular coding standard. Other examples of video compression standards include MPEG-2 and ITU-T H.263. Proprietary coding techniques, such as those referred to as On2 VP6 / VP7 / VP8, may also implement one or more of the techniques described herein.

 Although not shown in FIG. 1, in some aspects, video encoder 20 and video decoder 30 may be integrated with an audio encoder and decoder, respectively, and may encode both audio and video encodings as a common data stream or a separate data stream Or other hardware and software, for processing with the < RTI ID = 0.0 > MUX-DEMUX < / RTI > If applicable, in some examples, MUX-DEMUX units may conform to the ITU H.223 multiplexer protocol, or other protocols, such as User Datagram Protocol (UDP).

Each of video encoder 20 and video decoder 30 includes one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, It may be implemented in any of a variety of suitable encoder circuits, such as hardware, firmware or any combination thereof. When these techniques are partially implemented in software, the device may store instructions for the software in a suitable non-volatile computer-readable medium to perform the techniques of the present disclosure, and may execute those instructions in hardware using one or more processors have. Each of video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder / decoder (codec) in each device.

JCT-VC is committed to the development of the HEVC standard. HEVC standardization efforts are based on an evolving model of a video coding device referred to as the HEVC test model (HM). HM assumes several additional capabilities of video coding devices, for example, related to existing devices according to ITU-T H.264 / AVC. For example, H.264 provides nine intra-prediction encoding modes, while HM may provide as many as 33 intra-prediction encoding modes.

In general, the working model of HM describes that a video frame or picture may be divided into a sequence of treeblocks or maximum coding units (LCU) that include both luma and chroma samples. The triblock has a purpose similar to the macroblock of the H.264 standard. A slice includes a plurality of consecutive treeblocks in coding order. A video frame or picture may be partitioned into one or more slices. Each treeblock may be divided into coding units (CUs) according to a quadtree. For example, as a root node of the quadtree, a treeblock may be divided into four child nodes, and each child node may then be a parent node or may be divided into four other child nodes. As a leaf node of the quadtree, the last undivided child node contains a coding node, i.e. a coded video block. Syntax data associated with the coded bitstream may define the maximum number of times a treeblock can be split and may also define the minimum size of coding nodes. Treeblocks may be referred to as LCUs in some examples.

The CU includes a coding node, and transformation units (TUs) and prediction units (PUs) associated with the coding node. The size of the CU corresponds to the size of the coding node and should be square in shape. The size of the CU may range from 8 x 8 pixels to the size of the triblock with a maximum of 64 x 64 pixels or more. Each CU may include one or more PUs and one or more TUs. The syntax data associated with the CU may describe, for example, the partitioning of the CU into one or more PUs. The partitioning modes may be different between whether the CU is skipped or direct mode encoded, intra-predicted mode encoded, or inter-predicted mode encoded. PUs may be partitioned into non-square shapes. Syntactic data associated with a CU may also describe partitioning of the CU to one or more TUs according to, for example, a quadtree. The TU may be in the form of a square or a non-square (eg, rectangular).

The HEVC standard allows transforms according to TUs, which may be different for different CUs. TUs are usually sized based on the size of the PUs in a given CU defined for the partitioned LCU, but this is not always true. The TUs are generally the same size or smaller than the PUs. In some instances, the remaining samples corresponding to the CU may be subdivided into smaller units using a quadtree structure known as "residual quadtree " (RQT). The leaf nodes of the RQT may be referred to as conversion units (TUs). The pixel difference values associated with the TUs may be transformed to generate transform coefficients, and the transform coefficients may be quantized.

Generally, the PU contains data relating to the prediction process. For example, when a PU is intra-mode encoded, it may include data describing an intra-prediction mode for the PU. As another example, when the PU is inter-mode encoded, the PU may include data defining a motion vector for the PU. The data defining the motion vector for the PU may be, for example, the horizontal component of the motion vector, the vertical component of the motion vector, the resolution (e.g., 1/4 pixel precision or 1/8 pixel precision) for the motion vector, A reference picture list (eg, list 0, list 1, or list C) for the pointing reference picture and / or motion vector may be described.

In general, a TU is used for transform and quantization processes. A given CU with one or more PUs may or may include one or more transform units (TUs). Following prediction, video encoder 20 may calculate residual values corresponding to the PU. Residual values include pixel difference values that may be transformed, quantized, or scanned using the TUs to produce serialized transform coefficients for entropy coding. This disclosure generally refers to a coding node of a CU using the term “video block”. In some specific cases, this disclosure may also use the term “video block” to refer to a treeblock, that is, an LCU or CU that includes coding nodes and PUs and TUs.

A video sequence generally includes a series or pictures of video frames. A group of pictures (GOP) generally includes one or more series of video pictures. A GOP may include syntax data describing a number of pictures included in a GOP, in a header of the GOP, in one or more headers of pictures, or elsewhere. Each slice of the picture may include slice syntax data that describes the encoding mode for each slice. Video encoder 20 generally operates on video blocks within individual video slices to encode video data. The video block may correspond to a coding node in the CU. The video blocks may have fixed or variable sizes, and may vary in size according to a prescribed coding standard.

As an example, HM supports prediction in several PU sizes. Assuming that the size of a particular CU is 2N x 2N, HM can perform intra-prediction on PU sizes of 2N x 2N or N x N, and intra-prediction on 2N x 2N, 2N x N, N x 2N, or N x N Inter-prediction is supported in symmetric PU sizes. HM also supports asymmetric partitioning for inter-prediction in PU sizes of 2N x nU, 2N x ND, nL x 2N, and nR x 2N. In asymmetric partitioning, one direction of the CU is not partitioned, while the other direction is partitioned by 25% and 75%. The portion of the CU corresponding to the 25% partition is indicated by "n" followed by an indication of "Up", "Down", "Left", or "Right". Thus, for example, “2N × nU” refers to a 2N × 2N CU partitioned horizontally with 2N × 0.5N PU on top and 2N × 1.5N PU on bottom.

In the present disclosure, "N x N" and "N times N" are interchangeable to refer to pixel dimensions of a video block, e.g., 16 x 16 pixels or 16 x 16 pixels in terms of vertical and horizontal dimensions. . In general, a 16 x 16 block will have 16 pixels (y = 16) in the vertical direction and 16 pixels (x = 16) in the horizontal direction. Similarly, an N × N block generally has N pixels in the vertical direction and N pixels in the horizontal direction, where N represents a non-negative integer value. The pixels in the block may be arranged in rows and columns. Moreover, the blocks need not necessarily have the same number of pixels in the horizontal direction as in the vertical direction. For example, the blocks may include N x M pixels, where M does not necessarily have to be equal to N.

After intra-prediction or inter-prediction coding using PUs of the CU, the video encoder 20 may calculate the residual data for the TUs of the CU. The PUs may include pixel data in the spatial domain (also referred to as the pixel domain), and the TUs may be transformed into a transform, such as discrete cosine transform (DCT), integer transform, wavelet transform, or a conceptually similar transform to residual video data And may include coefficients in the transform domain after application. Residual data may correspond to pixel differences between pixels of the unencoded picture and prediction values corresponding to the PUs. The video encoder 20 may form TUs that contain residual data for the CU and then transform those TUs to generate transform coefficients for that CU.

After any transforms that generate transform coefficients, the video encoder 20 may perform quantization of the transform coefficients. Quantization generally refers to a process in which transform coefficients are quantized to reduce as much as possible the amount of data used to represent coefficients, and provides additional compression. The quantization process may reduce the bit depth associated with some or all of its coefficients. For example, the n-bit value may be truncated to an m-bit value during quantization, where n is greater than m.

In some instances, the video encoder 20 may scan the quantized transform coefficients using a predefined scanning order to generate a serialized vector that may be entropy encoded. In other examples, video encoder 20 may perform adaptive scanning. After scanning the quantized transform coefficients to form a one-dimensional vector, video encoder 20 generates the one-dimensional vector, for example, context-adopted variable length coding (CAVLC), context-adopted binary arithmetic coding (CABAC), syntax. Entropy encoding may be performed according to context-adapted binary arithmetic coding (SBAC), Probability Interval Partitioning Entropy (PIPE) coding or another entropy encoding methodology. Video encoder 20 may also entropy encode syntax elements associated with the encoded video data for use by video decoder 30 in decoding the video data.

To perform CABAC, the video encoder 20 may assign a context in the context model to the transmitted symbols. The context may, for example, relate to whether neighboring values of a symbol are non-zero. To perform CAVLC, the video encoder 20 may select a variable length code for the transmitted symbol. The codewords in the VLC may be configured such that relatively shorter codes correspond to more probable symbols, but longer codes correspond to less probable symbols. In this way, the use of VLC may achieve bit savings, for example, beyond using the same-length codewords for each symbol transmitted. The probability determination may be based on the context assigned to the symbol.

2 is a block diagram illustrating an example video encoder 20 that may implement techniques in accordance with this disclosure. Video encoder 20 may perform inter coding of video blocks within video slices. Intra coding relies on spatial prediction to reduce or remove spatial redundancy in video within a given video frame or picture. Inter coding reduces or eliminates temporal redundancy of the video within adjacent frames or pictures of the video sequence depending on the temporal prediction. Intra mode (I mode) may refer to any of several spatial based compression modes. Inter modes, such as unidirectional prediction (P mode) or bidirectional prediction (B mode), may refer to any of several time based compression modes.

In the example of FIG. 2, video encoder 20 is partitioning unit 35, prediction module 41, reference picture memory 64, summer 50, transform module 52, quantization unit 54, and Entropy encoding unit 56. Prediction module 41 includes motion estimation unit 42, motion compensation unit 44, and intra prediction module 46. For video block reconstruction, video encoder 20 also includes inverse quantization unit 58, inverse transform module 60, and summer 62. A deblocking filter (not shown in FIG. 2) may also be included to filter block boundaries to remove block phenomenon artifacts from the reconstructed video. As needed, the deblocking filter typically filters the output of summer 62. Additional loop filters (in loop or post loop) may also be used in addition to the deblocking filter.

As shown in FIG. 2, video encoder 20 receives video data and partitioning unit 35 partitions the data into video blocks. This partitioning may also include partitioning into slices, tiles or other larger units, as well as partitioning of video blocks, eg, according to a quadtree structure of LCUs and CUs. Video encoder 20 generally represents components that encode video blocks within a video slice to be encoded. A slice may be divided into multiple video blocks (and possibly into sets of video blocks referred to as ties). Prediction module 41 may determine one of a plurality of coding modes, such as one of a plurality of intra coding modes, for a current video block based on error results (eg, coding rate and distortion level). One of the coding modes may be selected. Prediction module 41 provides the resulting intra or inter coded block to summer 50 to generate residual block data and to summer 62 to reconstruct the encoded block for use as a reference picture. You may.

Intra prediction module 46 within prediction module 41 may perform intra prediction coding of the current video block on one or more neighboring blocks in the same frame or slice as the current block to be coded to provide spatial compression. Motion estimation unit 42 and motion compensation unit 44 within prediction module 41 perform inter prediction coding of the current video block for one or more prediction blocks in one or more reference pictures to provide temporal compression. It can also be done.

Motion estimation unit 42 may be configured to determine the inter prediction mode for the video slice according to a predetermined pattern for the video sequence. The determined pattern may assign video slices in the sequence as P slices, B slices, or GPB slices. Motion estimation unit 42 and motion compensation unit 44 may be highly integrated, but are described separately for conceptual purposes. The motion estimation performed by motion estimation unit 42 is a process for generating motion vectors, which estimates the motion of the video blocks. For example, the motion vector may indicate the displacement of the PU of the current video frame or video block within the picture relative to the expected block within the reference picture.

The predictive block is the block found to closely match the PU of the video block to be coded in terms of pixel difference, which may be determined by sum of absolute difference (SAD), sum of square difference (SSD) or other different matrices. have. In some examples, video encoder 20 may calculate values for sub-integer pixel locations of reference pictures stored in reference picture memory 64. For example, video encoder 20 may interpolate values of pixel positions of quarter pixel positions, eighth pixel positions, or other portions of a reference picture. Thus, motion estimation unit 42 may perform a motion search for full pixel positions or partial pixel positions and output a motion vector with partial pixel precision.

Motion estimation unit 42 calculates a motion vector for the PU of the video block in the inter coded slice by comparing the position of the PU to the position of the predictive block of the reference picture. The reference picture may be selected from the first reference picture list (List 0) or the second reference picture list (List 1), each of which identifies one or more reference pictures stored in the reference picture memory 64. The motion estimation unit 42 transmits the calculated motion vectors to the entropy encoding unit 56 and the motion compensation unit 44. [

The motion compensation performed by motion compensation unit 44 may include fetching or generating the predictive block based on the motion vector determined by performing motion interpolation on subpixel precision, if possible. Upon reception of the motion vector of the PU of the current video block, motion compensation unit 44 may locate the predictive block to which the motion vector points to one of the reference picture lists. Video encoder 20 forms a residual video block by subtracting pixel values of the predictive block from pixel values of the current video block being coded, thereby forming pixel difference values. The pixel difference values form residual data for the block and may include both luma and chroma difference components. The summer 50 represents a component or components that perform this subtraction operation. Motion compensation unit 44 may also generate syntax elements associated with the video slices and the video blocks for use by video decoder 30 in decoding the video blocks of the video slice.

Intra prediction module 46 may intra predict the current block as an alternative to inter prediction performed by motion estimation unit 42 and motion compensation unit 44, as described above. In particular, intra prediction module 46 may determine an intra prediction mode to use to encode the current block. In some examples, intra prediction module 46 may encode the current block using several intra prediction modes during separate encoding passes, and intra prediction module 46 (or in some examples, mode selection unit 40) The appropriate intra prediction mode may be selected for use from this test mode. For example, intra prediction module 46 may calculate rate distortion values using rate distortion analysis for several tested intra prediction modes, and select an intra prediction mode having the best rate distortion characteristic among the tested modes. It may be. Rate distortion analysis generally includes, in addition to the bit rate (ie, the plurality of bits) used to generate the encoded block, the distortion between the original non-encoded block and the encoded block that was encoded to produce the encoded block. Or error). Intra prediction module 46 may calculate ratios from the distortions and rates for the various encoded blocks to determine which intra prediction mode exhibits the best rate distortion value for the block.

In some cases, prediction module 41 may select an IVMP mode for coding one or more depth video blocks. In this case, motion information for the corresponding texture video block may be adopted for the depth block as described herein. The depth block and the texture block may be coded in the same NAL unit, and the IVMP flag may be encoded such that the decoder can properly decode the depth video block by reusing the motion information of the corresponding texture view video block.

In any case, after selecting the intra prediction mode for the block, intra prediction module 46 may provide entropy encoding unit 56 with information indicating the selected intra prediction mode for the block. Entropy encoding unit 56 may encode the information indicating the selected intra prediction mode according to the techniques of this disclosure. Video encoder 20 may be included in the transmitted bitstream configuration data, which data may include a plurality of intra prediction mode indexes and a plurality of modified intra prediction mode index tables (also referred to as codeword mapping tables), various blocks. Definitions of encoding contexts for, and an indication of the most likely intra prediction mode, an intra prediction mode index table, and a modified intra prediction mode index table for reuse for each of the contexts.

After prediction module 41 generates the predictive block for the current video block through inter prediction or intra prediction, video encoder 20 forms a residual video block by subtracting the prediction block from the current video block. Residual video data in the residual block may be included in one or more TUs and may be applied to transform module 52. Transform module 52 transforms the residual video data into residual transform coefficients using a transform such as a discrete cosine transform (DCT) or a conceptually similar transform. Transform module 52 may transform residual video data from the pixel domain to a transform domain, such as a frequency domain.

The transform module 52 may also send the resulting transform coefficients to the quantization unit 54. [ Quantization unit 54 quantizes the transform coefficients to further reduce bit rate. The quantization process may reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be changed by adjusting the quantization parameter. In some examples, quantization unit 54 may then perform a scan of the matrix including the quantized transform coefficients. As an alternative, entropy encoding unit 56 may perform the scan.

Following quantization, entropy encoding unit 56 may entropy encode the quantized transform coefficients. For example, entropy encoding unit 56 may include context-adopted variable length coding (CAVLC), context-adopted binary arithmetic coding (CABAC), syntax-based context-adopted binary arithmetic coding (SBAC), probability interval partitioning. Probability Interval Partitioning Entropy (PIPE) coding or another entropy encoding methodology or technique may be performed. Following entropy encoding by entropy encoding unit 56, the encoded bitstream may be transmitted to video decoder 30 or archived for retrieval or later transmission by video decoder 30. Entropy encoding unit 56 may also entropy encode other syntax elements and motion vectors for the current video slice being coded.

Inverse quantization unit 58 and inverse transform module 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual block into the pixel domain for later use as a reference block of the reference picture. Motion compensation unit 44 may calculate the reference block by adding a residual block to one prediction block of one of the reference pictures within one of the reference picture lists. Motion compensation unit 44 may also apply one or more interpolation filters to the reconstructed residual block to calculate sub integer pixel values for use in motion estimation. Summer 62 adds the reconstructed residual block to the motion compensated prediction block generated by motion compensation unit 44 to generate a reference block for storage in reference picture memory 64. The reference block may be used as the reference block by motion estimation unit 42 and motion compensation unit 44 to inter predict the block in a subsequent video frame or picture.

3 is a block diagram illustrating an example video decoder 30 that may implement the techniques described in this disclosure. In the example of FIG. 3, video decoder 30 includes entropy decoding unit 80, prediction module 81, inverse quantization unit 86, inverse transform unit 88, summer 90, and reference picture memory ( 92). The prediction module 81 includes a motion compensation unit 82 and an intra prediction module 84. Video decoder 30 may in some examples perform a generally reversible decoding pass for the encoding pass described with respect to video encoder 20 of FIG. 2.

During the decoding process, video decoder 30 receives from video encoder 20 an encoded video bitstream that represents the video blocks and associated syntax elements of the encoded video slice. Entropy decoding unit 80 of video decoder 30 entropy decodes the bitstream to generate quantized coefficients, motion vectors, and other syntax elements. Entropy decoding unit 80 forwards the motion vectors and other syntax elements to prediction module 81. Video decoder 30 may receive syntax elements at a video slice level and / or a video block level.

If the video slice is coded as an intra coding (I) slice, intra prediction module 84 of prediction module 81 is based on the signal from the previously decoded blocks of the current frame or picture and the signaled intra prediction mode. To generate predictive data for a video block of the current video slice. When a video frame is coded into an inter coded (ie, B, P or GPB) slice, motion compensation unit 82 of prediction module 81 may receive motion vectors and other syntax elements received from entropy decoding unit 80. Generate predictive blocks for the video block of the current video slice based on the fields. Predictive blocks may be generated from one of the reference pictures within one of the reference picture lists. Video decoder 30 may construct the reference frame lists, List 0 and List 1 using default construction techniques based on reference pictures stored in reference picture memory 92.

Motion compensation unit 82 determines prediction information for the video block of the current video by parsing motion vectors and other syntax elements, and regenerates the prediction blocks for the current video block being decoded using the prediction information. For example, motion compensation unit 82 may include a prediction mode (eg, intra or inter prediction) used to code video blocks of a video slice, an inter prediction slice type (eg, a B slice, a P slice, or a GPB). Slice), configuration information for one or more of the reference picture lists for the slice, motion vectors for each inter-coded video block of the slice, inter prediction status for each inter-coded video block of the slice, and current video Use some of the received elements to determine other information for decoding the video blocks in the slice.

In some cases, prediction module 81 may interpret the flag in the NAL unit and select an IVMP mode for decoding one or more depth video blocks of the NAL unit. In this case, motion information for the corresponding texture video block may be adopted for the depth block as described herein. The depth block and texture block may be coded in the same NAL unit, and the IVMP flag is decoded from the bitstream so that video decoder 30 can properly decode the depth video block by reusing the motion information of the corresponding texture view video block. May be

Motion compensation unit 82 may also perform interpolation based on interpolation filters. Motion compensation unit 82 may use the interpolation filters used by video encoder 20 to calculate interpolated values for the sub integer pixels of the reference blocks. In this case, motion compensation unit 82 determines, from the received syntax elements, the interpolation filters used by video encoder 20 and uses the interpolation filters to generate the predictive blocks.

Inverse quantization unit 86 is provided in the bitstream to inverse quantize, ie, quantize, the quantized transform coefficients decoded by entropy decoding unit 80. The inverse quantization process may include the use of the quantization parameter calculated by video encoder 20 for each video block in the video slice to determine the degree of quantization and likewise the degree of inverse quantization that should be applied. Inverse transform module 88 applies an inverse transform, eg, an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process to the transform coefficients, to produce a residual block in the pixel domain.

After prediction module 81 generates the predictive block for the current video block based on inter or intra prediction, video decoder 30 generates corresponding prediction blocks generated by prediction module 81 and an inverse transform module. Sum the residual blocks from 88 to form a decoded video block. The summer 90 represents the components or components that perform this summation operation. As needed, a deblocking filter may also be applied to filter the decoded blocks to remove block phenomenon artifacts. Other loop filters (in the coding loop or after the coding loop) may also be used to smooth the pixel transitions or to improve the video quality. Decoded video blocks in a given frame or picture are then stored in reference picture memory 92, which stores reference pictures used for subsequent motion compensation. Reference picture memory 92 also stores decoded video for later presentation on a display device, such as display device 31 of FIG. 1.

In 3D video coding, the texture view component and its associated depth view component may have similar object silhouettes, and these different view components may have similar object boundaries and motion. Thus, there is redundancy in the motion fields of the associated texture view component and the depth view component. The techniques of this disclosure may utilize this redundancy in a wider range than conventional techniques by allowing a mode in which the depth view component fully adopts the motion information of the texture view component, in a manner similar to the so-called "merge" mode. In this case, the depth view component may not include any additional delta values for the motion information (i.e. it may not contain any motion vector difference values), and instead replaces the motion information of the texture view component. It may be adopted as motion information.

Specifically, motion prediction from the texture view component to the associated depth view component may be enabled according to a new mode of merging the motion information of the texture view as the motion information of the depth view. In some examples, this so-called IVMP mode may be enabled only for inter coded MBs for depth view components. In IVMP mode, motion information including mb_type, sub_mb_type, reference indices, and motion vectors of MB co-located in the texture view component are reused by the depth view component in the same view. A flag indicating whether to use the IVMP mode may be signaled in each MB. As shown in FIG. 5, the flag may be true for the identified MB in the fourth picture of the depth view, and the motion vector of the co-located MB in the fourth picture (identified as the fourth picture) of the texture view. Is reused for the highlighted MB in the depth view component. Note that in some examples, the IVMP mode only applies to non-anchor pictures. The term “fixed picture” may be defined as any random access point (RAP) that is different from an instantaneous decoding refresh (IDR) picture.

As mentioned above, compared to conventional techniques for predicting a motion vector for one view based on the motion of another view, the techniques of this disclosure may realize additional compression. For example, some conventional scalable techniques may allow motion prediction of the enhancement view based on the motion information of the base view, and in some cases, the base view may be a text view and the enhancement view may be a depth view. In this case, however, in addition to the prediction information (or flag) indicating that the base view is used to predict the reinforcement view, the motion vector difference value (eg delta) is always coded. In contrast, the techniques of this disclosure may utilize an IVMP mode in which no delta is coded or allowed. Instead, in the IVMP mode, the motion information of the texture view is adopted as the motion information of the depth view.

Details of several additional signaling techniques for signaling compressed video data are described below. A view parameter set (VPS) may be signaled as "in-band", meaning that the parameter set is associated with the coded picture and transmitted together in one channel or session. If the VPS exists in an access unit (AU) that is a coded representation of a time instance of the bitstream, it may be necessary to precede any VCL NAL units. Multiple frames may have the same VPSs replicated to introduce resiliency.

In some examples, the techniques of this disclosure may address inver_view_flag and may extend the syntaxes of inter_view_flag. In one example, inter_view_flag such as # 0 specifies that the current view component is not used for inter-view prediction by any other view component to the current access unit with the same or different spatial resolution. In this example, inter_view_flag equal to 1 specifies that the current view component may be used for inter-view prediction by other view components to the current access unit.

The value of inter_view_flag may be the same for all VCL NAL units of the view component.

In one embodiment, the left and right views are half resolution and the center view is full resolution. In an asymmetric 3DV profile, this flag may be set to 1 for the right view, for example. However, if the MVC sub bitstream is extracted, this flag need not be 1.

Define a flag called inter _ asy _ view _ flag :

In some examples, an inter _ asy _ view _ flag , such as 0, specifies that the current view component is not used for inter-view prediction by any other view component with the current access unit having a different spatial resolution. Inter_asy_view_flag equal to 1 specifies that the current view component may be used for inter-view prediction by other view components with different spatial resolution to the current access unit.

In the example above, in the left view, NAL units may have inter_view_flag equal to 1 and inter_asy_view_flag equal to 1. From left view. NAL units may have inter_view_flag equal to 0 and inter_asy_view_flag equal to 1, and in the central view, all NAL units may have these two flags equal to zero.

This disclosure may provide a response to a call for proposal (CfP) in 3D Video Coding issued by MPEG. This proposal can be based on H.264 / MVC reference software JMVC and additions with several enhancements, which may incorporate joint coding of texture and depth of multiple views. The proposal of this disclosure may include joint coding of texture and depth, prediction of texture from depth within a view, and asymmetric coding of view components with different resolutions. In the proposal, MPEG view synthesis software may be used for view generation without any change.

Compared to the JMVC 8.3.1 anchor, the proposal of this disclosure shows that when the bitrates are total bitrates of both texture and depth of the two views, and the peak signal to noise ratio (PSNR) values of the two decoded texture views. When average PSNR values for the values, for a 2-view case, a rate reduction of up to 22.6% (average 11.7%) may be realized, and for a 3-view case, up to 15.8% (average 7.3%) Rate reduction may be realized.

For 2-view case, when the total bitrates versus PSNR values of the synthesized view are used, the BD rate reduction is up to 24.7% (and on average 13.9%), and for the 3-view case, the total bitrates If average PSNR values of the two composite views are used, the BD rate reduction is up to 19.0% (and 15.0% on average).

The present disclosure may also provide the following.

Compatibility and possible multiview high profile for both H.264 / AVC high profile and H.264 / MVC stereo high profile;

Joint coding of texture and depth for multiview sequences;

Symmetric spatial and temporal resolutions for the texture and depth view components of each view;

Asymmetric spatial resolutions for different views.

Additional codec changes on top of the H.264 / MVC codec may also include:

High level syntax to support joint coding of texture view components and depth view components;

A mode in which motion vector prediction and depth view motion between texture and depth view components are adopted from the associated texture view motion.

This disclosure also describes other tools, such as those that allow prediction between view components at different resolutions, and prediction of slice headers from a texture view component to a corresponding depth view component. The texture view component and the depth view component may form a view component that is a coded picture of a view to the access unit. Thus, the techniques may allow for the adoption of motion information or the prediction of motion information of a depth view relative to a texture view, including deltas, in accordance with the described IVMP mode. Both tools may allow for coding flexibility, but may limit the tools to some degree to achieve the best compression. For example, the IVMP mode described herein may be limited to non-anchor pictures.

Throughout the document, AVC refers to H.264 / AVC high profile. If any other H.264 / AVC profile or amendment is referred to, that amendment or profile name will be explicitly specified. For example, H.264 / MVC or MVC refers to the multiview extension of H264 / AVC. However, any correction or profile of H.264 / AVC belongs to the AVC series, so the proposed codec is also compatible to the AVC stereo high profile if it is compatible with the MVC stereo high profile.

Codec descriptions are provided below. In this section, the proposed 3DVC codec is described in two aspects, high level framework and low level coding scheme. In the case where it is desirable to define a 3DV format that can possibly have two views and three views corresponding to different applications, the technique in the three-view case may form a superset of those in the two-view case. have. Thus, in this section, the high level framework applicable for both cases is described first, followed by the codec description for techniques in the two-view case applicable to the three-view case and then the three-view Techniques used only in cases will be described.

 The high level framework may use the following definitions.

View component: A coded representation of a view in a single access unit . When the view includes both coded texture and depth representations, the view component consists of a texture view component and a depth view component.

Texture view component: A coded representation of the texture of a view in a single access unit .

Depth view component : A coded representation of the depth of a view in a single access unit .

Coded VCL NAL units in the depth view component may be assigned nal_unit_type 21, in particular as a new type of coded slice extensions for depth view components.

The bit stream sequence is described below. In each view component, any coded slice NAL unit (with nal_unit_type 21) of the depth view component may need to follow all coded slice NAL units of the texture view component. For simplicity, this disclosure names coded slice NAL units of depth view component as depth NAL units.

The depth NAL unit has the same NAL unit header structures as the NAL unit with nal_unit_type equal to 20. 4 illustrates an example bitstream order of VCL NAL units of view components within one access unit.

As shown in FIG. 4, in the exemplary 3D video codec, the access unit includes multiple view components, each view component consisting of one texture view component and one depth view component. The texture view component of the base view with a view order index (VOIdx) equal to 0 is one prefix NAL unit (with a NAL unit equal to 14) and one or more AVC VCL NAL (with a NAL unit type equal to 1 or 5). It includes units. Texture view components in other views include only MVC VCL NAL units (with a NAL unit type such as 20). In both the base view and the non-base view, the depth view components include depth NAL units with a NAL unit equal to 21. In any view component, the depth NAL units follow the NAL units of the texture view components in decoding / bitstream order.

In a two-view case, this disclosure may employ half resolution encoding for both left and right views. Characteristics of the proposed codec may include:

Half horizontal or half vertical spatial resolution;

The same resolution for texture view components and depth view components of each view;

VC high profile compatible half resolution base view (texture only);

AVC stereo high profile compatible half resolution stereoscopic views (texture only);

Inter-view prediction from the depth view component of the base view to the depth view component of the non-base view;

Texture versus depth prediction within view components.

Half space resolution MVC is referenced below and is mentioned in Table 1 below. All sequences may be coded with half spatial resolution. Compared to H.264 / AVC frame compatible coding, half space resolution MVC is more efficient and it is more convenient to meet the following requirements.

Forward compatibility: This two-view 3DVC bitstream includes an MVC subbitstream, which also includes an AVC subbitstream. Thus, the proposed codec meets the following present requirements: All compressed bitstreams conforming to this mode enable existing AVC decoders to reconstruct samples from mono and stereo views from the bitstream .

Stereo / mono compatibility: VCL NAL units may be extracted simply by checking the NAL unit type to obtain an MVC or AVC subbitstream. Therefore, the proposed codec is thus especially meet the following requirements: a delta compression format that enables a simple extraction of the bit stream with respect to the stereo and mono output Mode must be included and support high reconstruction of samples from left and right views of stereo video .

Half-space resolution sequences are MPEG 13-tap downsampling filters ([2,0, -4, -3,5,19,26,19,5, -3, -4,0) for both texture and depth sequences. , 2] / 64). In order to realize better quality, downsampling can be applied in the horizontal or vertical direction. For sequences with dominant horizontal high frequency components, half vertical resolution can be used. In some examples, only one sequence is considered to belong to this category: " Poznan _ Hall2 " . Other sequences are considered to have dominant vertical high frequency components and horizontal downsampling is applied to obtain the harp horizontal resolution sequences.

Symmetric resolution for texture and depth may be used. The depth view component may be coded as an 8-bit mono sequence with the same resolution as the texture view component of the same view. In this setting, the prediction from the texture view component to the depth view component may be performed without scaling, eg, without motion vectors or pixels in the macroblock (MB).

Inter-view prediction for depth view components may be supported. Depth view component may be predicted by other depth view components to the same access unit in the same manner as inter-view prediction in MVC. The depth view component refers to a subset sequence parameter set (SPS) with view dependencies signaled in the extension.

Typically, the prediction dependencies of the depth view components share the same view dependency of the texture view components as shown in FIG. 6. Also note that several sequences do not benefit from inter-view prediction between depth views. Thus, inter-view prediction for depth views can simply be disabled for these cases. 6 shows a prediction structure of the 3DVC codec. Depth view components (shown crosshatched) have the same prediction structure as texture view components (shown without shading).

Thus, a flag (disable_depth_inter_view_flag) may be signaled in the SPS to disable or enable inter-view prediction for depth views. More specific SPS designs for two-view and three-view cases are described in more detail below. For depth map sequences that may benefit from inter-view prediction, the depth view components have the same inter prediction and inter-view prediction structures as the texture view, as shown in FIG. 6.

7 shows that the prediction structure of the 3DVC codec does not allow inter-view prediction for depth view components. Components shown shaded in the components shown in FIG. 7 are texture views and components shown shaded with crosshatching are depth views. As shown in FIG. 7, inter-view prediction may be enabled for texture view components, but disabled globally for depth view components. In this case the depth view component may have a different slice type than the corresponding texture view component.

Hereinafter, motion prediction from texture to depth is described. Since the texture view component and its associated depth view component have similar object silhouettes, they have similar object boundaries and movements, so there is redundancy in these motion fields.

According to this disclosure, motion prediction from a texture view component to its associated depth view component may be enabled as a new mode in the proposed codec. In some examples, inside view motion prediction (IVMP) mode is enabled for intercoded MB only in depth view components. In IVMP mode, motion information including mb_type, sub_mb_type, reference index, and motion vectors of co-located MBs in the texture view component are reused by the depth view component of the same view. A flag indicating whether to use the IVMP mode may be signaled in each MB. Consistent with FIG. 5, the flag may be true for the fourth picture of the depth view and the motion vector of the co-located MB in the fourth picture (labeled fourth) of the texture view is reused for the depth view component. In some examples, IVMP mode applies only to non-anchor pictures.

Slice header prediction is described below. In each view component, there may be redundancy between slice headers between the depth view component and the texture view component. Thus, given the slice header of the texture view component, the depth view component in the same view of the same access unit has most of its slice header information already determined.

According to this disclosure, depth view components share slice text syntax elements of most of the corresponding text. The different syntax elements include pic_parameter_set_id, slice_qp_delta, and possibly syntax elements related to the reference picture list construction, which include num_ref_idx_l0_active_minus1, num_ref_idx_l1_active_minus1, and the reference picture list change syntax table.

The slice header of the depth view component may be signaled in the slice header depth extension. Note that pred_slice_header_depth_idc may be signaled in the sequence parameter set. In some examples, the encoder may always be set to one.

An example slice header depth extension syntax may follow the example in Table 1 below.

Table 1

The three-view case is described below. The techniques of this disclosure may adopt half resolution encoding for both left and right views, and full resolution for the center view. Encoding methods enabled in the two-view case may also be supported for the codec in the three-view case. The codec may include the following features for the 3-view case.

Asymmetric spatial resolution in different views;

Inter-view prediction from the low resolution view to the high resolution view;

The sub-bitstream comprising the texture view components of the low resolution views is compatible with H.264 / MVC stereo high profile.

Signaling of inter-view prediction dependency on high resolution views.

Hereinafter, inter-view prediction in asymmetric 3DVC codec is described. In both the texture view components and between the depth view components, prediction from the reconstructed low resolution view to the high resolution view may be enabled.

More specifically, in a three-view case, the left view and the right view may be coded at half resolution, and the center view may be coded at full resolution. When inter-view prediction occurs from the half resolution view component to the full resolution (texture or depth) view component, the decoded picture of the half resolution view component, when used for inter-view prediction, is determined using the AVC 6-tap filter [ Upsampled to 1, -5, 20, 20, -5, 1] / 32. In this case, the low resolution picture (needed for output) and also the upsampled picture may need to temporarily coexist in the buffer. The upsampled pictures from the left view and the right view can then be placed in the reference picture lists of the view component of the center view to the same access unit.

Asymmetric inter-view prediction is shown in FIG. 8, where both the left view (VL) and the right view (VR) have half widths. Since view dependencies allow them to be used as inter-view references to the center view (VC), these views are both upsampled to intermediate pictures.

For simplicity, regardless of whether the "MVC view" refers to both text and depth portions or just a texture portion, the raw resolution views compatible with MVC are MVC (when considering only texture). It's called views. Other views with full resolution are called additional views. In this three-view case, these are two MVC views and one additional view. Each MVC view includes both texture and depth at the same resolution, which is half the resolution of the additional view.

The sequence parameter set design is described below. In some aspects of the present disclosure, a new SPS extension may be introduced. If the profile indicated in seq_parameter_set_data () is related to 3DV, a new SPS extension is added to the subset SPS. According to the present disclosure, two possible profiles: "3DV profile" and "asymmetric 3DV profile" are considered for two different cases. That is, the 3DV profile applies to the two-view case, and the asymmetric 3DV profile applies to the three-view case.

In MVC, a new sequence level parameter set, that is, an SPS MVC extension, may be introduced and signaled in a subset SPS. If MVC is considered to be the base specification, in any of the newly added profiles, the subset SPS is further extended to signal the sequence parameter set 3DVC extension on top of the SPS MVC extension.

In one proposed codec, a new SPS extension, i.e., sequence parameter set 3DVC extension, further signals inter-view dependencies for high resolution views for asymmetric 3DV profile in addition to inter-view dependency for depth view components. Which is applicable to both 3DV profiles and asymmetric 3DV profiles.

In 3DV related applications, other syntax elements, such as those related to camera parameters and depth range and / or depth quantization, may also be signaled to the SPS. However, in one proposed codec, this information may be considered available and therefore is not transmitted in the coded bitstream.

Table 2 shows examples of subset sequence parameter set RBSP (raw byte sequence payload) syntax.

Table 2

Table 3 shows examples of sequence parameter set 3DVC extension syntax.

Table 3

In one proposed 3DVC codec, camera parameters in addition to depth ranges may not be included in the bitstream since they do not have a standard effect on the decoded views. However, they can help with view synthesis and possible coding tools that use view synthesis as a particular mode, for example. If camera parameters or depth ranges are required for a particular coding tool, they may be a set of parameters, such as a SPS, Picture Parameter Set (PPS) or even a new type of parameter when such information may vary on a per frame basis. It may be transmitted in a standard, mandatory manner within a set, ie View Parameter Set (VPS). If they are not necessary for decoding any transmitted texture or depth, they may be signaled (at sequence level or picture level) in an SEI message.

This section provides a realization of how the above information can be signaled in the bitstream. Signaling for camera parameters and depth ranges may be implemented in software but is not enabled for generation of bitstreams.

Table 4 shows examples of camera parameters and depth ranges in the SPS 3DVC extension.

Table 4

In the camera parameters syntax table, the floating point value V may be expressed as having its precision ( P ) and integer value ( I ), the number of digits before and after the decimal point, and V = I * 10 ^P. Becomes The sign of V may be the same as that of I. This proposed representation may be sufficiently accurate for camera parameters and depth ranges, and it may be relatively easy to parse and construct a floating point value.

Given the requirement that "source video data should be rectified to avoid misalignment of camera geometry and colors " as stated in CfP in this disclosure, multiple views may be subject to external parameters except for horizontal translation. It may be taken to share most and the same internal parameters.

Table 5 and the paragraphs that follow represent example camera parameters syntax and semantics.

Table 5

In Table 5, cam _ param _ present _ flag , such as 1, may indicate that camera parameters are signaled in this SPS. Cam_param_present_flag equal to 0 may indicate that camera parameters are not signaled in this SPS.

In Table 5, focal length _ _ precision is specified the precision of the value of focal_length_x and focal_length_y and focal_length_x and focal_length_y is a focal length x coordinate and y coordinate of the focal length all cameras.

In Table 5, focal length _ _x_I specifies the integer value of value of focal_length_x.

focal _ length _x = focal _ length _x_I * 10 ^{focal _ length _ precision}

In Table 5, focal length _y_I_ _ diff _x plus focal_length_x_I specifies the integer part of the value of focal_length_y.

focal _ length _y = ( focal _ length _x_I + focal _ length _y_I_ diff _x) * 10 ^{focal _ length _ precision}

In table 5, principal _ precision Specifies the precision of the values of principal_point_x and principal_point_y, where principal_point_x and principal_point_y are the x coordinate main point and y coordinate main point of all cameras.

In Table 5, principal _ point _x_I specifies the integer part of the value of principal_point_x.

principal _ point _x = principal _ point _x_I * 10 ^{principal _ precision}

In Table 5, principal _ point _y_I_ diff _x plus principal_point_x specifies the integer part of the value of principal_point_y.

principal _ point _y = ( principal _ point _x_I + principal _ point _y_I_ diff _x) * 10 ^{principal _ precision}

The rotation matrix R for each camera may be expressed as follows:

In Table 5, rotation _ kl _ half _ pi represents the diagonal components of the rotation matrix R , where kl is equal to xy, yz, or xz, where R _kl = (-1) ^{rotation _ kl _ half _ pi} . This flag equal to 0 R _kl A flag indicating that = 1 equals R _kl = -1.

In Table 5, translation _ precision specifies the precision of the value of the translation of all views. The precision of the translation values applies to all translation values of the views referenced by this SPS.

In Table 5, numViewsMinus1 is derived as num_views_minus1 + num_add_views_minus1 +1.

In Table 5, anchor _ view _ id Specifies the view_id of the view, which is used as an anchor to compute the translations of other views.

In Table 5, zero translation _ _ present _ flag equal to 1 indicates that the translation is 0, the view with the same view_id as anchor_view_id; This value equal to 0 indicates that the translation of the view with view_id equal to anchor_view_id is signaled.

translation _ anchor _ view _I specifies the integer part of the translation of the anchor view. The translation of the anchor view is denoted as translation_anchor_view. If zero_translation_present_flag is equal to 0, Translation_anchor_view is equal to 0, otherwise the translation is calculated as follows.

In table 5,

translation_anchor_view = translation _ anchor _ view _I * 10 ^{translation _ precision}

In Table 5, translation _ diff _ anchor _ view _I [ i ] plus translation_anchor_view_I specifies the integer portion of the translation of the view with view_id, such as view_id [i], and is denoted as translation_view_I [i].

Translation of a view with view_id equal to view_id [i] is denoted as translation_view [i].

translation_view [i] =

( translation _ diff _ anchor _ view _I [i] + translation _ anchor _ view _I ) * 10 ^{translation _ precision}

Table 6 and the paragraphs that follow show example depth ranges syntax and semantics.

Table 6

In Table 6, a depth _ range _ present _ flag equal to 1 Indicates that the depth range for all views is signaled in this SPS, and depth_range_present_flag equal to 0 indicates that the depth range for all views is not signaled in this SPS.

In Table 6, z_ near _ precision specifies the precision of the z_near value. The precision of z_near specified in the SPS applies to all z_near values of the views referenced in this SPS.

In Table 6, z_ far _ precision specifies the precision of the z_far value. The precision of z_far specified in the SPS applies to all z_far values of the views referenced in this SPS.

In Table 6, a different _ depth _ range _ flag , such as 0, indicates that the depth ranges of all views are the same and are in the range of z_near to z_far (including these values). Different_depth_range_flag equal to 1 indicates that the ranges of all views may be different: z_near and z_far are depth ranges for the anchor view, and z_near [i] and z_far [i] are of the view with the same view_id as view_id [i] Depth range is further specified in this SPS.

In table 6, z_ near _ integer Specifies the integer part of the value of z_near. z_near = z_ near _ integer * 10 ^{z_ near _ precision}

In table 6, z_ far _ integer specifies the integer part of the value of z_far. z_ far = z_ far _ integer * 10 ^{z_ far _ precision}

In Table 6, z_ near _ _ diff anchor view _ _I plus z_near_integer is specific for the constant portion of the shallow depth value of the view with the same view_id as view_id [i] and is denoted as z_near_I [i].

z_near of view having a view_id such as view_id [i] is denoted by the z_near [i].

z_near [i] = ( z_ near _ diff _ anchor _ view _I [i] + z_ near _ integer ) * 10 ^{z_ near _ precision}

In Table 6, z_ far _ _ diff anchor view _ _I plus z_far_Integer shall specify the integer portion of the deepest depth value of the view with view_id such as view_id [i], is expressed as z_far_I [i].

z_far [i] = ( z_ far _ diff _ anchor _ view _I [i] + z_ far _ integer ) * 10 ^{z_ far _ precision}

Table 7 shows an example view parameter set RBSP syntax.]

Table 7

A NAL unit including the view parameter set RBSP may be assigned a new NAL unit type, for example 16.

Table 8 and the paragraphs that follow show exemplary view parameter set syntax and semantics.

Table 8

The depth range and translation of the camera may vary on a picture basis. The updated depth range or camera parameters may be applicable to the view components of the current access unit and the following view components in the bitstream until the new VPS, following the current VPS, updates these values of related views.

For simplicity, no semantics of syntax elements are given. For the translation and depth range of each view, the integer portion of the difference between the value signaled in the SPS (having an identifier such as seq_para_set_id) and the new value may be signaled in this VPS. The updated values for the translation and depth range can be calculated as follows:

translation_view [i] = ( translation _ view _ integer [i] + translation _ update _ view _I [i] ) * 10 ^{translation _ precision}

z_near [i] = ( z_ near _ integer [i] + z_ near _ update _ view _I [i] ) * 10 ^{z_ near _ precision}

z_far [i] = ( z_ far _ integer [i] + z_ far _ update _ view _I [i] ) * 10 ^{z_ far _ precision}

Where translation_view_integer [i], z_near_integer [i] and z_far_integer [i] are integer parts of the values of translation_view [i], z_near [i], and z_far [i], and can be calculated based on signaling in the SPS have.

One or more of the techniques of this disclosure may be used to provide coding enhancements in terms of compression and / or quality. Encoding time and complexity may be improved using one or more of the techniques of this disclosure. Decoding time and complexity may also be improved. In addition, memory usage at the encoder and decoder may be improved or reduced over other techniques.

In some examples, both the encoder and the decoder may have the same memory consumption level as the JMVC encoder and decoder. Thus, memory usage may be considered to be proportional to the number of view components, eg, access unit. The depth view component is always stored as 4: 0: 0, and in the same number of views, the proposed solution may consume approximately 5/3 (about 67% increase) of the memory used by JMVC for the encoder or decoder. have. For simplicity of operations, eg, viewing depth maps and using these depth maps for view synthesis, the encoder and decoder may still retrieve and output depth files in a 4: 2: 0 chroma sampling format. Please note.

In the following, the complexity characteristics of the decoder are described. In some examples, both the encoder and the decoder according to the techniques of this disclosure may have the same complexity level as the JMVC encoder and decoder. Compared with JMVC, the computational complexity of the codec according to this disclosure may relate to the spatial resolution of each view and the number of views. That is, a codec according to this disclosure may require the same amount of computation as the JMVC codec, as long as both take the same video with the same number of pixels.

At the decoder side, standard picture level upsampling may be required for an asymmetric 3DV profile. However, this decoding process may be considered less complex than other decoding processes for decoding of the high resolution view component, so that the complexity characteristic may still be indicated by, for example, how many MBs should be processed per second.

An encoder according to the techniques described herein may follow the current JMVC encoder scheme, and the views are encoded one by one. Inside each view, the texture sequence is encoded first, and the depth sequence is then encoded.

When the IVMP mode is enabled, during texture view component encoding, the filed motion of each texture view component can be recorded in a motion file and its name can be specified in the configuration file. If the relevant depth sequences of the same view are encoded, the motion file is read as a reference.

The encoder has the following additional items and may use the same configuration as JMVC.

MotionFile

String , default : " Motion "

Specifies the file name (without date) of the motion sequence to be created. This sequence is provided in IVMP mode. motion _0. dat , motion _1. dat etc. will be generated automatically by the encoder.

Halfsizedimension

Unsigned Int , default : 0

It indicates whether asymmetric spatial resolution is used and the subsampling dimensions when it is used. The following values are supported:

0-all views are encoded with the same spatial resolution.

1-asymmetric spatial resolution is used and half resolution views have half the width of other views.

2-asymmetric spatial resolution is used and the half resolution views have half the height of other views.

BasisQP _ texture

Double , default : 26

Specifies a basic quantization parameter of the texture view component with half space resolution.

BasisQP _ depth

Double , default : 26

Specifies a basic quantization parameter of the depth view component with half spatial resolution.

BasisQP _ texture _ delta

Unsigned Int , default : 0

Specifies a base quantization parameter offset for the base quantization parameter of the texture view component with full spatial resolution, compared to the base quantization parameter of the texture view component with half space resolution. The default quantization parameter for texture view components with full spatial resolution is BasisQP _ texture ( full spatial resolution ) = Calculated by BasisQP_texture + BasisQP _ texture _ delta .

BasisQP _ depth _ delta

Unsigned Int , default :0

Specifies a base quantization parameter offset for the base quantization parameter of the depth view component with full spatial resolution, compared to the base quantization parameter of the depth view component with half spatial resolution. The default quantization parameter for the depth view component with full spatial resolution is BasisQP _ depth ( full spatial resolution ) = calculated by BasisQP _ depth + BasisQP_depth_delta .

NoDepthInterViewFlag

Flag  (0 or  One), default :0

Specifies whether inter-view prediction is enabled for any depth view component. If NoDepthInterViewFlag is equal to 0, inter-view prediction is enabled. If NoDepthInterViewFlag is equal to 1, inter-view prediction is disabled.

Halfres

Flag  (0 or  One), default : 0

This value is a portion of each of the reference characteristic view of the signaled in the view dependency portion is associated with a View _ ID values.

A view to identifying View _ ID specifies whether the half spatial resolution. If HalfRes is 0, this is a full spatial resolution view. If HalfRes is 1, this is a half space resolution view.

The encoder can be used to generate bitstreams. An example encoder is shown in the following example.

Where mcfg represents the filename of the configuration files. The configuration file may be specified for each encoder call. The element view _ id indicates the view to be encoded. Element component _ idx Indicates whether or not the current sequence of the texture are encoded for a particular view that the (component _ idx the case of 1) or field (if the component_idx is zero). The encoder may be executed for each view component of each view to be encoded.

The decoder may be similar to the JMVC decoder, and there are major changes that also decode and output the depth sequence for each view. In an asymmetric 3DV profile, upsampling is required to convert the MVC view (left or right) to high resolution for prediction of additional view (center).

The assembler can have very minor changes to discard the duplicated parameter set NAL units, the complexity of which is the same as the JMVC assembly.

For views, synthesizer changes to JMVC may not be required.

Several features of the H.264 / MVC based 3DVC codec have been described, which can meet all the “shall” requirements of the present proposal and may provide good coding performance for a relatively small amount of additional coding methods. The methods include a high level framework for joint coding of texture and depth, prediction from texture inside the view component to depth, and inter-view prediction between texture or depth view components with asymmetric spatial resolutions.

The MVC-based 3DV codec may be standardized for short term market requirements, and the proposed features of this disclosure may be the basis of the reference software and working draft of this 3DV codec.

9 is a flowchart illustrating a technique that may be performed by a video encoder in accordance with this disclosure. 9 is described in terms of video encoder 20 of FIG. 2, other video encoders may also be used. As shown in FIG. 9, prediction module 41 receives 3D video, eg, video blocks that represent 3D rendition (901). The 3D video includes a texture view video block and an associated depth view video block (901). Prediction module 41 encodes the texture view video block (902). In addition, prediction module 41 encodes the depth view video block (903).

According to this disclosure, prediction module 41 supports IVMP mode. In particular, prediction module 41 generates a syntax element that indicates whether motion information for the depth view is adopted from the texture view (903). In this way, if the IMVP mode is enabled, the depth view component may not contain any additional delta values for its motion information; instead, the motion information of the texture view component is used as the motion information of the depth view component. May be adopted. In particular, in IMVP mode, the depth view component may not include any motion vector difference values and may fully adopt the motion vector of the corresponding texture view component. By defining a mode that fully adopts the motion information of the texture view as the motion information of the depth view, improved compression may be realized without signaling any motion vector delta values for this motion information.

The texture view video block and the depth view video block may be coded together in a network abstraction layer (NAL) unit, and the syntax element indicates whether motion information associated with the texture view video block is adopted as motion information associated with the depth view video block. The flag indicating may be included in the NAL unit. In this case, the composite element indicates that the texture view video block is adopted as motion information associated with the depth view video block, and the depth view video block does not contain any additional deltas to the motion information associated with the depth view video block. NAL units are one particular type of access unit used to code video data, and the techniques may also be used for other types of video units.

More specifically, the syntax element may include one or more bits indicating whether the IVMP mode is enabled. When the IVMP mode is disabled, motion information associated with the texture view video block is included in the NAL unit, and motion information associated with the depth view video block is separately included in the NAL unit. Alternatively, if the IVMP mode is enabled, motion information associated with the texture view video block is included in the NAL unit, and motion information associated with the texture view video block is adopted as motion information associated with the depth view video block. Thus, if the IVMP mode is enabled, the depth view video block does not include any additional deltas for the motion information associated with the depth view video block. In some examples, IVMP mode applies only to non-anchor pictures and not to anchor pictures.

10 is a flowchart illustrating a technique that may be performed by a video decoder in accordance with this disclosure. Although FIG. 10 has been described from the perspective of video decoder 30 of FIG. 3, other video decoders may also be used. As shown in FIG. 10, prediction module 81 receives a 3D video, eg, a video block representing 3D video data (1001). The 3D video includes a texture view video block and an associated depth view video block (1001). Prediction module 41 decodes the texture view video block (1002). In addition, prediction module 41 decodes the depth view video block (1003).

According to this disclosure, prediction module 81 supports IVMP mode. In particular, prediction module 81 decodes 1003 a syntax element that indicates whether motion information for the depth view is adopted from the texture view. The syntax element may be interpreted by the decoder as indicating whether motion information of the depth view is adopted from the texture view. If the IMVP mode is enabled, the depth view component may not include any additional delta values for that motion information and may instead adopt the motion information of the texture view component as the motion information of the depth view component. In addition, by defining a mode that fully adopts the motion information of the texture view as the motion information of the depth view, it is possible to realize improved compression without any delta of this motion information.

In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the functions may be stored or transmitted on a computer-readable medium as one or more instructions or codes, and executed by a hardware-based processing unit. The computer-readable medium may comprise computer-readable storage media, such as a data storage medium, or a computer readable medium, such as, for example, a computer from one place to another, And any medium that facilitates transmission of the program. In this way, computer-readable media may generally correspond to (1) non-transitory type computer-readable storage media or (2) communication media such as signals or carrier waves. The data storage medium may be one or more computers or any available media that can be accessed by one or more processors to retrieve instructions, code, and / or data structures for implementation of the techniques described in this disclosure . The computer program product may comprise a computer-readable medium.

By way of example, and not limitation, such computer-readable storage media may be embodied in a computer-readable medium such as RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, Instructions, or any other medium that can be used to store data in the form of data structures and which can be accessed by a computer. Also, any connection is properly referred to as a computer-readable medium. For example, when commands are transmitted from a web site, server, or other remote source using coaxial cable, fiber optic cable, dual winding, digital subscriber line (DSL), or wireless technologies such as infrared, , Coaxial cable, fiber optic cable, dual winding, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of the medium. However, it should be understood that the computer-readable storage mediums and data storage media do not include connections, carrier waves, signals, or other transient mediums, but instead relate to non-transitory type storage media. A disk and a disc as used herein include a compact disk (CD), a laser disk, an optical disk, a digital versatile disk (DVD), a floppy disk and a Blu-ray disk, ) Usually reproduce data magnetically, while discs reproduce data optically with a laser. Combinations of the foregoing should also be included within the scope of computer-readable media.

The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits Lt; / RTI > Thus, the term "processor" when used herein may refer to any of the structures described above or any other structure suitable for implementation of the techniques described herein. In addition, in some aspects, it may be provided in software modules that are configured for functional dedicated hardware and / or encoding and decoding described herein, or may be included in a combined codec. In addition, the techniques may be implemented entirely with one or more circuits or logic elements.

The techniques of the present disclosure may be implemented in a wide variety of devices or devices, including a wireless handset, an integrated circuit (IC) or a set of ICs (e.g., a chipset). Various components, modules, or units are described in this disclosure to emphasize the functional aspects of the devices configured to perform the disclosed techniques, but do not necessarily require realization by different hardware units. More precisely, as described above, several units may be coupled to a codec hardware unit or provided with a collection of interactive hardware units, including one or more processors as described above, along with suitable software and / or firmware .

In still other examples, this disclosure may be taught with respect to computer readable storage media having stored data structures. Data structures may include 3D video data that is compressed in the manner described herein, such as by using an IVMP mode to code a depth view for a texture view.

Several examples of the disclosure have been described. These and other examples are within the scope of the following claims.

Claims (34)

A method of coding three-dimensional (3D) video data,
Coding the texture view video block; And
Coding a depth view video block, wherein the depth view video block comprises coding the depth view video block associated with the texture view video block,
Coding the depth view video block includes coding a syntax element indicating whether motion information associated with the texture view video block is employed as motion information associated with the depth view video block, A method of coding three-dimensional (3D) video data. The method according to claim 1,
The texture view video block and the depth view video block are coded together into an access unit,
The syntax element includes a flag defined at a video block level indicating whether the motion information associated with the texture view video block is employed as the motion information associated with the depth view video block; How to code your data. 3. The method of claim 2,
If the syntax element indicates that the motion information associated with the texture view video block is adopted as the motion information associated with the depth view video block, the depth view video block is any delta to the motion information associated with the depth view video block. Also not included, a method of coding three-dimensional (3D) video data. 3. The method of claim 2,
And the syntax element defines whether or not inside view motion prediction (IVMP) mode is enabled. 5. The method of claim 4,
If the IVMP mode is disabled, the motion information associated with the texture view video block is included in the access unit, motion information associated with the depth view video block is separately included in the access unit,
When the IVMP mode is enabled, the motion information associated with the texture view video block is included in the access unit, and the motion information associated with the texture view video block is adopted as motion information associated with the depth view video block, A method of coding three-dimensional (3D) video data. 6. The method of claim 5,
If the IVMP mode is enabled, the depth view video block does not include any delta for the motion information associated with the depth view video block. The method according to claim 1,
The coding comprises encoding, and coding the syntax element comprises generating the syntax element. The method according to claim 1,
The coding comprises decoding, and the coding of the syntax element comprises decoding the syntax element from an encoded bitstream, wherein the syntax element is included in the encoded bitstream. How to code video data. A device for coding three-dimensional (3D) video data,
The device includes one or more processors,
The processor comprising:
Code the texture view video block,
Is configured to code a depth view video block,
The depth view video block is associated with the texture view video block,
Coding the depth view video block includes coding a syntax element indicating whether motion information associated with the texture view video block is adopted as motion information associated with the depth view video block. Device for coding data. The method of claim 9,
The texture view video block and the depth view video block are coded together into an access unit,
The syntax element includes a flag defined at a video block level indicating whether the motion information associated with the texture view video block is employed as the motion information associated with the depth view video block; Device for coding data. 11. The method of claim 10,
If the syntax element indicates that the motion information associated with the texture view video block is adopted as the motion information associated with the depth view video block, the depth view video block is any delta to the motion information associated with the depth view video block. A device for coding three-dimensional (3D) video data, which also does not include. 11. The method of claim 10,
Wherein the syntax element defines whether an inside view motion prediction (IVMP) mode is enabled. 13. The method of claim 12,
If the IVMP mode is disabled, the motion information associated with the texture view video block is included in the access unit, motion information associated with the depth view video block is separately included in the access unit,
When the IVMP mode is enabled, the motion information associated with the texture view video block is included in the access unit, and the motion information associated with the texture view video block is adopted as motion information associated with the depth view video block, Device for coding three-dimensional (3D) video data. 14. The method of claim 13,
And if the IVMP mode is enabled, the depth view video block does not include any delta for the motion information associated with the depth view video block. The method of claim 9,
Wherein said coding comprises encoding, and coding said syntax element comprises generating said syntax element. The method of claim 9,
The coding comprises decoding, and coding the syntax element comprises decoding the syntax element from an encoded bitstream, wherein the syntax element is included in the encoded bitstream. A device for coding the. The method of claim 9,
The device is a device for coding three-dimensional (3D) video data comprising a wireless handset. The method of claim 9,
The device comprising:
Digital tv,
Devices in digital direct broadcast systems,
Devices in wireless broadcast systems,
Personal digital assistants (PDAs),
Laptop Computer,
Desktop Computer,
Tablet computers,
e-book reader,
digital camera,
Digital recording devices,
Digital media player,
Video gaming devices,
Video game console,
Cell cordless phone,
Satellite cordless phone,
Smartphone,
Video teleconferencing devices, and
A device for coding three-dimensional (3D) video data, comprising one or more of video streaming devices. A computer-readable storage medium having stored thereon instructions,
The instructions cause the one or more processors to execute,
Code the texture view video block;
To code a depth view video block,
The depth view video block is associated with the texture view video block,
Coding the depth view video block includes coding a syntax element indicating whether motion information associated with the texture view video block is adopted as motion information associated with the depth view video block. Storage media. 20. The method of claim 19,
The texture view video block and the depth view video block are coded together into an access unit,
The syntax element includes a flag defined at a video block level indicating whether the motion information associated with the texture view video block is employed as the motion information associated with the depth view video block; Storage media. 21. The method of claim 20,
If the syntax element indicates that the motion information associated with the texture view video block is adopted as the motion information associated with the depth view video block, the depth view video block is any delta to the motion information associated with the depth view video block. A computer readable storage medium having instructions stored thereon that is not included. 21. The method of claim 20,
And the syntax element defines whether or not inside view motion prediction (IVMP) mode is enabled. 23. The method of claim 22,
If the IVMP mode is disabled, the motion information associated with the texture view video block is included in the access unit, motion information associated with the depth view video block is separately included in the access unit,
When the IVMP mode is enabled, the motion information associated with the texture view video block is included in the access unit, and the motion information associated with the texture view video block is adopted as motion information associated with the depth view video block, A computer readable storage medium having stored thereon instructions. 24. The method of claim 23,
And when the IVMP mode is enabled, the depth view video block does not include any delta to the motion information associated with the depth view video block. 20. The method of claim 19,
Wherein the coding comprises an encoding, and wherein coding the syntax element comprises generating the syntax element. 20. The method of claim 19,
The coding comprises decoding, and coding the syntax element comprises decoding the syntax element from an encoded bitstream, the syntax element being included in the encoded bitstream. media. A device configured to code three-dimensional (3D) video data, the device comprising:
Means for coding a texture view video block; And
Means for coding a depth view video block, the depth view video block comprising means for coding the depth view video block associated with the texture view video block,
The means for coding the depth view video block includes means for coding a syntax element indicating whether motion information associated with the texture view video block is adopted as motion information associated with the depth view video block. 3D) a device configured to code the video data. 28. The method of claim 27,
The texture view video block and the depth view video block are coded together into an access unit,
The syntax element includes a flag defined at a video block level indicating whether the motion information associated with the texture view video block is employed as the motion information associated with the depth view video block; A device configured to code data. 29. The method of claim 28,
If the syntax element indicates that the motion information associated with the texture view video block is adopted as the motion information associated with the depth view video block, the depth view video block is any delta to the motion information associated with the depth view video block. A device configured to code three-dimensional (3D) video data, which also does not include. 29. The method of claim 28,
Wherein the syntax element defines whether three-dimensional (3D) video data defines whether an inside view motion prediction (IVMP) mode is enabled. 31. The method of claim 30,
If the IVMP mode is disabled, the motion information associated with the texture view video block is included in the access unit, motion information associated with the depth view video block is separately included in the access unit,
When the IVMP mode is enabled, the motion information associated with the texture view video block is included in the access unit, and the motion information associated with the texture view video block is adopted as motion information associated with the depth view video block, A device configured to code three-dimensional (3D) video data. 32. The method of claim 31,
And if the IVMP mode is enabled, the depth view video block does not include any delta for the motion information associated with the depth view video block. 28. The method of claim 27,
The means for coding comprises means for encoding and the means for coding the syntax element comprises means for generating the syntax element. 28. The method of claim 27,
The means for coding comprises means for decoding and the means for coding the syntax element comprises means for decoding the syntax element from an encoded bitstream, wherein the syntax element is included in the encoded bitstream. A device configured to code dimensional (3D) video data.

Images