WO2014166338A1 - Method and apparatus for prediction value derivation in intra coding - Google Patents
Method and apparatus for prediction value derivation in intra coding Download PDFInfo
- Publication number
- WO2014166338A1 WO2014166338A1 PCT/CN2014/074130 CN2014074130W WO2014166338A1 WO 2014166338 A1 WO2014166338 A1 WO 2014166338A1 CN 2014074130 W CN2014074130 W CN 2014074130W WO 2014166338 A1 WO2014166338 A1 WO 2014166338A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- mode
- depth
- value
- segment
- depth block
- Prior art date
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/103—Selection of coding mode or of prediction mode
- H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
Definitions
- the current depth block (410) is divided into two segments.
- the DC prediction values (Predoci and Predoc 2 ) for the two segments of the current depth block (410) are determined based on respective neighboring reconstructed depth values.
- the original depth values are shown in the current depth block (410).
- the residual values (residual i and residual 2 ) are obtained by applying inverse lookup on the residual indexes received.
- the reconstructed depth values (Recoci and Recoci) for the two segments of the current depth block are obtained respectively by adding residuah to Predoci and adding residual 2 to Predoc 2 -
- the reconstructed depth values (Recoci and Recoci) are then used for all depth samples in the two respective segments of current reconstructed depth block (420).
- the DC prediction value (Predoc) for the current depth block is determined based on the mean of the predicted depth values for the current depth block.
- the predicted depth values for the current depth block are derived based on neighboring reconstructed depth values using linear interpolation (right column and bottom row) and bilinear interpolation (other depth samples).
- the original depth values are shown in the current depth block (510).
- the residual value is obtained by applying inverse lookup on the residual index received.
- the reconstructed depth value (Recoc) for the current depth block is obtained by adding residual to Predoc-
- the reconstructed depth value (Recoc) is then used for all depth samples in the current reconstructed depth block (520).
- VSP View synthesis prediction
- the reconstructed depth block In the conventional SDC for depth block coding, a same predicted value is used for the whole depth block. Therefore, the reconstructed depth block always has a uniform value. Accordingly, the reconstructed depth block is very coarse and lack of details. It is desirable to develop a technique to improve the quality of the reconstructed depth data.
- a method and apparatus for sample-based Simplified Depth Coding (SDC), which is also termed as Segment-wise DC Coding are disclosed.
- Embodiments according to the present invention encode or decode a residual value for a segment of the current depth block, determine prediction samples for the segment of the current depth block based on reconstructed neighboring depth samples according to a selected Intra mode, and derive an offset value from a residual value for the segment of the current depth block.
- the final reconstructed samples are reconstructed by adding the offset value to each of the prediction samples of the segment.
- the offset value may correspond to the difference between the reconstructed depth value and the predicted depth value for the segment of the current depth block.
- the offset value may be derived from the residual value, wherein the residual value is derived implicitly at a decoder side or the residual value is transmitted in a bitstream.
- the offset value can be derived from a residual index according to an inverse Lookup Table.
- Fig. 3 illustrates an exemplary reconstruction process for Simplified Depth Coding (SDC) using the DC mode.
- Fig. 7 illustrates an exemplary reconstruction process for sample-based Simplified Depth Coding (SDC) using the Planar mode according to an embodiment of the present invention.
- the input signal to be coded is the mean of the original depth value (d orig ) of the depth block and the output is the predicted depth value (d pred ) of the depth block which is derived from mean of the predicted depth values for the depth block.
- the predicted depth value is also referred to as DC prediction value, or simply predicted value.
- the predicted depth samples for the Planar mode block are generated using linear interpolation or bilinear interpolation based on neighboring reconstructed depth values at the top row and the left column directly adjacent to the current depth block.
- the neighboring reconstructed depth values at the top row and the left column directly adjacent to the current depth block are also available at the decoder side. Therefore the predicted depth samples can be derived at the decoder side. Accordingly, the mean of the predicted depth values can also be derived at the decoder side.
- the residual index i res i to be coded into the bitstream is derived according to:
- the derived residual index i res i is then coded using a significance flag and a sign flag.
- the magnitude of the residual index is coded using [log 2 dyaiidl bits, where [xl is a ceiling function corresponding to the smallest integer not less than x.
- the reconstructed depth value, d rec is derived according to
- the reconstructed depth value is used as all depth samples of the reconstructed block/PU. In other words, the whole depth block will have a same reconstructed value for DC mode and Planar mode. There are two reconstructed values for the DMM Mode 1 for the two segments respectively.
- the reconstruction process is also performed in the reconstruction loop.
- embodiments of the present invention disclose sample-based SDC to improve the performance of depth coding.
- New reconstructed sample of the current block/PU according to an embodiment of the present invention is then derived by adding the reconstructed residual to each predicted sample, P(x,y).
- the reconstructed sample according to the present invention may vary from sample to sample as indicated by the sample location (x,y).
- An example of the reconstructed sample according to an embodiment of the present invention is shown as follows:
- the reconstructed samples, P'(x, y) for the Planar mode is derived according to the prediction samples of the Planar mode plus an offset value (i.e., the reconstructed residual, R rec ) as shown in Fig. 6, where the offset value is derived from the residual index.
- Fig. 6A illustrates that the reconstructed samples for the right column and the bottom row of the current depth block are formed by adding the predictors (210) of the Planar mode to an offset value (610).
- Fig. 6B illustrates that the reconstructed samples for other sample positions of the current depth block are formed by adding the respective predictors (220) of the Planar mode to the offset value (610).
- Fig. 7 illustrates an exemplary reconstruction process for sample-based Simplified Depth Coding (SDC) using the Planar mode according to an embodiment of the present invention. As illustrated in Fig. 7, the reconstructed depth block (710) according to the present invention will be able to reproduce shading within the depth block.
- SDC sample-based Simplified Depth Coding
- the offset value is directly derived from the residual value.
- the offset value R rec is given by
- Rrec I _1 (iresi), (6) where may be the inverse Index Lookup Table or other mapping table.
- Each prediction sample of the current depth block/PU is then updated with the reconstructed residual, i.e., the reconstructed residual is added to each prediction sample as the reconstructed sample.
- the third embodiment is based on the first embodiment or the second embodiment, where the types of prediction may be changed from ⁇ DC mode, DMM mode 1, Planar mode ⁇ to other sets of prediction types.
- the prediction types may be changed to:
- the sample-based SDC can achieve 0.2% BD-rate saving for video over total bit-rate in both common test conditions and all-intra test conditions, and 0.2% and 0.1% BD-rate savings for the synthesized view in common test conditions and all-intra test conditions, respectively.
- Poznan_Hall2 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.1% 102.0% 95.9% 102.1%
- Fig. 8 illustrates an exemplary flowchart of sample-based Simplified Depth Coding (SDC) for depth data using Intra modes according to an embodiment of the present invention.
- the system receives input data associated with a current depth block as shown in step 810.
- the input data associated with the depth block corresponds to the depth samples to be coded.
- the input data associated with the current depth block corresponds to the coded depth data to be decoded.
- the input data associated with the current depth block may be retrieved from memory (e.g., computer memory, buffer (RAM or DRAM) or other media) or from a processor.
- Prediction samples for the current depth block are then determined based on reconstructed neighboring depth samples according to a selected Intra mode as shown in step 820.
- a residual value (of each segment) of the current depth block is encoded or decoded, and an offset value (of each segment) is then derived from the residual value (using Eqn. 4 as an example) as shown in step 830.
- the reconstructed samples are derived by adding the offset value to the prediction samples (for each segment) as shown in step 840.
- Embodiment of the present invention as described above may be implemented in various hardware, software codes, or a combination of both.
- an embodiment of the present invention can be a circuit integrated into a video compression chip or program code integrated into video compression software to perform the processing described herein.
- An embodiment of the present invention may also be program code to be executed on a Digital Signal Processor (DSP) to perform the processing described herein.
- DSP Digital Signal Processor
- the invention may also involve a number of functions to be performed by a computer processor, a digital signal processor, a microprocessor, or field programmable gate array (FPGA). These processors can be configured to perform particular tasks according to the invention, by executing machine- readable software code or firmware code that defines the particular methods embodied by the invention.
- the software code or firmware code may be developed in different programming languages and different formats or styles.
- the software code may also be compiled for different target platforms.
- different code formats, styles and languages of software codes and other means of configuring code to perform the tasks in accordance with the invention will not depart from the spirit and scope of the invention.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
A method and apparatus for sample-based Simplified Depth Coding (SDC) are disclosed. The system determines prediction samples for the current depth block based on reconstructed neighboring depth samples according to a selected Intra mode and determines an offset value for the current depth block. The final reconstructed samples are derived by adding the offset value to each of the prediction samples. The offset value corresponds to a difference between a reconstructed depth value and a predicted depth value for the current depth block. The offset value can be derived from the residual value, and the residual value can be derived implicitly at a decoder side or transmitted in the bitstream. The selected Intra mode may correspond to Planar mode, the prediction samples are derived according to the Planar mode.
Description
METHOD AND APPARATUS FOR PREDICTION VALUE
DERIVATION IN INTRA CODING
CROSS REFERENCE TO RELATED APPLICATIONS
The present invention claims priority to U.S. Provisional Patent Application, Serial No.
61/810,797, filed on April 11, 2013, entitled "Methods of Deriving the Predicting Value in Intra Coding". The U.S. Provisional Patent Application is hereby incorporated by reference in its entirety. FIELD OF INVENTION
The present invention relates to three-dimensional and multi-view video coding. In particular, the present invention relates to depth coding using Simplified Depth Coding.
BACKGROUND OF THE INVENTION
Three-dimensional (3D) television has been a technology trend in recent years that is targeted to bring viewers sensational viewing experience. Multi-view video is a technique to capture and render 3D video. The multi-view video is typically created by capturing a scene using multiple cameras simultaneously, where the multiple cameras are properly located so that each camera captures the scene from one viewpoint. The multi-view video with a large number of video sequences associated with the views represents a massive amount data. Accordingly, the multi-view video will require a large storage space to store and/or a high bandwidth to transmit. Therefore, multi-view video coding techniques have been developed in the field to reduce the required storage space and the transmission bandwidth. In three-dimensional and multi-view coding systems, the texture data as well as depth data are coded.
For depth map, the Simplified Depth Coding (SDC), which is also termed as Segment- wise DC Coding, is an alternative Intra coding mode. Whether SDC is used is signalled by a SDC flag at coding unit (CU) level. For SDC, the depth block is Intra predicted by a conventional Intra mode or depth modelling mode 1. The partition size of SDC-coded CU is always 2Nx2N and therefore there is no need for signaling in the bitstream regarding the block size of SDC-coded CU. Furthermore, instead of coded as quantized transform coefficients, the SDC-coded residuals are represented by one or two constant residual values depending on whether the depth block is divided into one or two segments.
According to existing three-dimensional video coding based on HEVC (3D-HEVC),
certain information is signalled for SDC-coded blocks. The information signalled includes:
1. type of segmentation/prediction of the current block. Possible values are
i. DC (Direct Current; 1 segment);
ii. DMM (Depth Modelling Modes) Mode 1 - Explicit Wedgelets (2 segments);
iii. Planar (1 segment);
2. For the DMM, additional prediction information is coded.
3. For each resulting segment, a residual value (in the pixel domain) is signalled in the bitstream.
In the depth coding process, the depth residuals are mapped to limited depth values, which are present in the original depth map. The limited depth values are represented by a Depth Lookup Table (DLT). Consequently, residuals can be coded by signalling indexes pointing to entries of this lookup table. The depth values present in a depth map are usually limited to a number smaller than the total number that can be represented by a depth capture device. Therefore, the use of DLT can reduces the bit depth required for residual magnitudes. This mapping table is transmitted to the decoder so that the inverse lookup from an index to a valid depth value can be performed at the decoder.
At the encoder side, the residual index iresi to be coded into the bitstream, is determined according to:
iresi = I(dorig)— I(dpred); (1) where dorig denotes an original depth value determined for the depth block, dpred denotes the predicting depth value, and I(. ) denotes the Index Lookup Table. The computed residual index iresi is then coded with a significance flag, a sign flag and with [log2 dvaiidl bits for the magnitude of the residual index, where dvalid denotes the number of valid depth values and [xl is a ceiling function corresponding to the smallest integer not less than x.
The Depth Lookup Table takes advantage of the sparse property of the depth map, where only a small number of depth values out of a full available depth range (e.g., 28) will typically be present in the depth map. In the encoder, a dynamic depth lookup-table is constructed by analyzing a number of frames (e.g. one Intra period) of the input sequence. This depth lookup- table is used during the coding process to reduce the effective signal bit-depth of the residual signal.
In order to reconstruct the lookup table, the encoder reads a pre-defined number of frames from the input video sequence to be coded and scans all samples for presence of the depth values. During this process a mapping table is generated that maps depth values to existing depth values based on the original uncompressed depth map.
The Depth Lookup Table D(. ), the index Lookup Table I(. ), the Depth Mapping Table M(. ) and the number of valid depth values dvalid are derived by the following process that analyses the depth map Dt:
1. Initialization
boolean vector B(d) = FALSE for all depth values d,
index counter i = 0.
2. Process each pixel position p in Dt for multiple time instances t:
Set B(Dt(p)) = TRUE to mark valid depth values.
3. Count the number of TRUE values in B(d). The result is set to the value for dvalid . 4. For each d with B(d) == TRUE:
Set D(i) = d,
Set M(d) = d,
Set 1(d) = i, and
1 = 1 + 1.
5. For each d with B(d) == FALSE:
Find d = arg min|d— d| and B(d) == TRUE
Set M(d) = d.
6. Set 1(d) = 1(d).
As mentioned above, there are three types of segmentation and prediction in the existing SDC. The respective processes for the three types of segmentation and prediction are described as follows.
DC:
The DC prediction value (Predicting depth value ( dpred )) is predicted from neighboring blocks using a the mean of all directly adjacent samples of the top and the left blocks.
DMM Mode:
Edge information is defined by start/end side and corresponding index.
The DC prediction values (Predicting depth value (dpred)) for each segment are predicted by neighboring depth values as shown in Fig. 1. Two depth blocks (110 and 120) are shown in Fig. 1, where each block is divided into two segments as shown by the dashed line. The reconstructed neighboring depth samples for block 110 are indicated by references 112 and 114 and the reconstructed neighboring depth samples for block 120 are indicated by references 122 and 124.
Planar :
Generate the predictors of the Planar mode as shown in Fig. 2. Linear interpolation is used to generate predictors for the right column and the bottom row as shown in Fig. 2A. For the right column, the linear interpolation is based on depth values at A and Z. For the bottom row, the linear interpolation is based on depth values at B and Z. After the right column and the bottom row are interpolated, the predictors for the rest of depth positions are bilinear interpolated using four respective depth samples from four sides as shown in Fig. 2B.
The DC prediction value (Predicting depth value ( dpred )) is the mean of the predictors of the Planar mode.
In the above derivation processes, prediction sample refers to the predicted value generated by the Intra coding mode, which may be the DC mode, DMM Mode 1 or the Planar mode in the existing 3D-HEVC. The reconstruction process for the DC mode at the decoder side is illustrated in Fig. 3. The DC prediction value (Predoc) for the current depth block (310) is determined based on neighboring reconstructed depth values. In Fig. 3, the original depth values are shown in the current depth block (310). The residual value is obtained by applying inverse lookup on the residual index received. The reconstructed depth value (Recoc) for the current depth block is obtained by adding residual to Predoc- The reconstructed depth value (Recoc) is then used for all depth samples in the current reconstructed depth block (320).
The reconstruction process for the DMM Mode 1 at the decoder side is illustrated in Fig. 4.
The current depth block (410) is divided into two segments. The DC prediction values (Predoci and Predoc2) for the two segments of the current depth block (410) are determined based on respective neighboring reconstructed depth values. In Fig. 4, the original depth values are shown in the current depth block (410). The residual values (residual i and residual2) are obtained by applying inverse lookup on the residual indexes received. The reconstructed depth values (Recoci and Recoci) for the two segments of the current depth block are obtained respectively by adding residuah to Predoci and adding residual2to Predoc2- The reconstructed depth values (Recoci and Recoci) are then used for all depth samples in the two respective segments of current reconstructed depth block (420).
The reconstruction process for the Planar mode at the decoder side is illustrated in Fig. 5.
The DC prediction value (Predoc) for the current depth block (510) is determined based on the mean of the predicted depth values for the current depth block. The predicted depth values for the current depth block are derived based on neighboring reconstructed depth values using linear interpolation (right column and bottom row) and bilinear interpolation (other depth
samples). In Fig. 5, the original depth values are shown in the current depth block (510). The residual value is obtained by applying inverse lookup on the residual index received. The reconstructed depth value (Recoc) for the current depth block is obtained by adding residual to Predoc- The reconstructed depth value (Recoc) is then used for all depth samples in the current reconstructed depth block (520).
View synthesis prediction (VSP) is a technique to remove interview redundancies among video signal from different viewpoints, in which a synthetic signal is used as references to predict a current picture.
In 3D-HEVC Test Model, HTM-6.0, there exists a process to derive a disparity vector predictor, known as Do BDV (Depth oriented Neighboring Block Disparity Vector). The disparity vector identified from DoNBDV is used to fetch a depth block in the depth image of the reference view. The fetched depth block has the same size as the current prediction unit (PU), and the fetched depth block is then used for backward warping for the current PU.
In addition, the warping operation may be performed at a sub-PU level precision, such as 2x2 or 4x4 blocks. A maximum depth value is selected for a sub-PU block and used for warping all the pixels in the sub-PU block. The VSP based on backward warping (BVSP) is applied in both texture and depth component coding.
In existing HTM-6.0, BVSP prediction is added as a new merging candidate to signal the use of BVSP prediction. When the BVSP candidate is selected, the current block may be a Skip block if there is no residual to transmit or a Merge block if there is residual information to be coded.
In the conventional SDC for depth block coding, a same predicted value is used for the whole depth block. Therefore, the reconstructed depth block always has a uniform value. Accordingly, the reconstructed depth block is very coarse and lack of details. It is desirable to develop a technique to improve the quality of the reconstructed depth data.
SUMMARY OF THE INVENTION
A method and apparatus for sample-based Simplified Depth Coding (SDC), which is also termed as Segment-wise DC Coding, are disclosed. Embodiments according to the present invention encode or decode a residual value for a segment of the current depth block, determine prediction samples for the segment of the current depth block based on reconstructed neighboring depth samples according to a selected Intra mode, and derive an offset value from a residual value for the segment of the current depth block. The final reconstructed samples are reconstructed by adding the offset value to each of the prediction samples of the segment.
The offset value may correspond to the difference between the reconstructed depth value and the predicted depth value for the segment of the current depth block. The offset value may be derived from the residual value, wherein the residual value is derived implicitly at a decoder side or the residual value is transmitted in a bitstream. The offset value can be derived from a residual index according to an inverse Lookup Table.
The selected Intra mode may correspond to the Planar mode where the current depth block only includes one segment, the prediction samples are derived using linear interpolation and bilinear interpolation from the reconstructed neighboring depth samples of the current depth block according to the Planar mode, and the offset value is derived from the residual value or a residual index. The selected Intra mode can be selected from a set of Intra modes and the selection of the selected Intra mode from the set of Intra modes can be signalled in a bitstream. The set of Intra modes may correspond to {DC mode, DMM Mode 1, Planar mode} or {DC mode, DMM Mode 1, VSP}. The ordering of the Intra modes within the set can be changed. A truncated unary code can be used to indicate the selected Intra mode from the set of Intra modes.
BRIEF DESCRIPTION OF THE DRAWINGS
Fig. 1 illustrates two examples of Depth Modelling Mode (DMM) for depth coding based on Simplified Depth Coding (SDC), where the depth block is divided into two segments and each segment is modelled as a uniform area.
Fig. 2 illustrates the linear interpolation and bilinear interpolation used to generate prediction samples for the depth block based on reconstructed neighboring depth samples according to the Planar mode in SDC.
Fig. 3 illustrates an exemplary reconstruction process for Simplified Depth Coding (SDC) using the DC mode.
Fig. 4 illustrates an exemplary reconstruction process for Simplified Depth Coding (SDC) using the Depth Modelling Mode (DMM) Mode 1.
Fig. 5 illustrates an exemplary reconstruction process for Simplified Depth Coding (SDC) using the Planar mode.
Fig. 6 illustrates an example of sample-based Simplified Depth Coding (SDC) for the Planar mode.
Fig. 7 illustrates an exemplary reconstruction process for sample-based Simplified Depth Coding (SDC) using the Planar mode according to an embodiment of the present invention.
Fig. 8 illustrates an exemplary flowchart for a system incorporating sample-based Simplified Depth Coding (SDC) using the Planar mode according to an embodiment of the
present invention.
DETAILED DESCRIPTION
In the existing Simplified Depth Coding (SDC) which is later termed as Segment-wise DC coding (SDC), the input signal to be coded is the mean of the original depth value (dorig) of the depth block and the output is the predicted depth value (dpred) of the depth block which is derived from mean of the predicted depth values for the depth block. There is only one predicted depth value for each segment according to the existing SDC. The predicted depth value is also referred to as DC prediction value, or simply predicted value. The predicted depth samples for the Planar mode block are generated using linear interpolation or bilinear interpolation based on neighboring reconstructed depth values at the top row and the left column directly adjacent to the current depth block. The neighboring reconstructed depth values at the top row and the left column directly adjacent to the current depth block are also available at the decoder side. Therefore the predicted depth samples can be derived at the decoder side. Accordingly, the mean of the predicted depth values can also be derived at the decoder side. At the encoder side the residual index iresi to be coded into the bitstream is derived according to:
iresi l(d0rig) I(dprecj), (2) where I(. ) denoting the Index Lookup Table. The derived residual index iresi is then coded using a significance flag and a sign flag. The magnitude of the residual index is coded using [log2 dyaiidl bits, where [xl is a ceiling function corresponding to the smallest integer not less than x.
At the decoder side, the reconstructed depth value, drec is derived according to
dree = I^ Oresi + l(dpred)), (3) where I_1(. ) denotes the inverse Index Lookup Table. After the reconstructed depth value is obtained, the reconstructed depth value is used as all depth samples of the reconstructed block/PU. In other words, the whole depth block will have a same reconstructed value for DC mode and Planar mode. There are two reconstructed values for the DMM Mode 1 for the two segments respectively. At the encoder side, the reconstruction process is also performed in the reconstruction loop.
As illustrated above, a same reconstructed value is used for all depth samples in a segment according to the existing SDC. While the method of using a single reconstruction value for each segment is simple, it may cause noticeable distortion in the reconstructed depth blocks. Accordingly, embodiments of the present invention disclose sample-based SDC to improve the
performance of depth coding.
First Embodiment. In the first embodiment of the present invention, pixel-based (or sample-based) Simplified Depth Coding (SDC) is disclosed. At the decoder side, the reconstructed residual Rrec is derived according to,
Rrec ^rec ^pred - (4)
The reconstructed depth values, drec may correspond to the reconstructed mean of the depth block as in the conventional SDC. Nevertheless, in the present invention, drec may correspond to other reconstructed depth value that is used by the encoder. For example, drec may correspond to a reconstructed median or majority of an original depth block.
New reconstructed sample of the current block/PU according to an embodiment of the present invention is then derived by adding the reconstructed residual to each predicted sample, P(x,y). In other words, the reconstructed sample according to the present invention may vary from sample to sample as indicated by the sample location (x,y). An example of the reconstructed sample according to an embodiment of the present invention is shown as follows:
P'(x, y) = Rrec + P(x. y). (5)
According to the above embodiment, the reconstructed samples, P'(x, y) for the Planar mode is derived according to the prediction samples of the Planar mode plus an offset value (i.e., the reconstructed residual, Rrec) as shown in Fig. 6, where the offset value is derived from the residual index. Fig. 6A illustrates that the reconstructed samples for the right column and the bottom row of the current depth block are formed by adding the predictors (210) of the Planar mode to an offset value (610). Fig. 6B illustrates that the reconstructed samples for other sample positions of the current depth block are formed by adding the respective predictors (220) of the Planar mode to the offset value (610). While the Planar mode is used as an example to illustrate the sample-based SDC, the present invention is not limited to the Planar mode. For other Intra modes, the sample-based SDC can also be applied to improve the performance. Fig. 7 illustrates an exemplary reconstruction process for sample-based Simplified Depth Coding (SDC) using the Planar mode according to an embodiment of the present invention. As illustrated in Fig. 7, the reconstructed depth block (710) according to the present invention will be able to reproduce shading within the depth block.
Second Embodiment. According to the second embodiment of the present invention, the offset value is directly derived from the residual value. For example, the offset value Rrec is given by
Rrec = I_1 (iresi), (6) where may be the inverse Index Lookup Table or other mapping table. Each
prediction sample of the current depth block/PU is then updated with the reconstructed residual, i.e., the reconstructed residual is added to each prediction sample as the reconstructed sample.
Third Embodiment. The third embodiment is based on the first embodiment or the second embodiment, where the types of prediction may be changed from {DC mode, DMM mode 1, Planar mode} to other sets of prediction types. For example, the prediction types may be changed to:
{DC mode, DMM Mode 1, VSP}, or
{Planar mode, DMM Mode 1, VSP}.
Fourth Embodiment. The fourth embodiment is based on the first embodiment or the third embodiment, where the order of the type of prediction might also be changed. Based on this order, a truncated unary code can be used to signal the type selected. For example, the order {Planar mode, DC mode, DMM Mode 1 } or {Planar mode, DMM Mode 1, DC mode} can be used.
The performance of a 3D/multi-view video coding system incorporating sample-based Simplified Depth Coding (SDC) according to an embodiment of the present invention is compared to that of a conventional system based on HTM-6.0. The types of prediction include DC mode, DMM Mode 1 and Planar mode. The embodiment according to the present invention uses sample-based SDC, where the reconstructed samples for the Planar mode are derived according to eqn. (5). The performance comparison is based on different sets of test data listed in the first column. The test results of the system incorporating an embodiment of the present invention under the common test conditions and under the all-Intra test conditions are shown in Table 1 and Table 2, respectively. As shown in the tables, the sample-based SDC can achieve 0.2% BD-rate saving for video over total bit-rate in both common test conditions and all-intra test conditions, and 0.2% and 0.1% BD-rate savings for the synthesized view in common test conditions and all-intra test conditions, respectively.
Table 1
Poznan Street 0.0% 0.0% 0.1% 0.0% -0.3% -0.3% 99.0% 99.1% 96.6%
Undo Dancer 0.0% -0.2% 0.2% -0.1% -0.1% -0.4% 99.4% 102.9% 98.1%
1024x768 0.0% -0.1% 0.0% 0.0% -0.3% -0.1% 99.9% 101.8% 98.0%
1920x1088 0.0% -0.3% 0.0% 0.0% -0.2% -0.2% 99.1% 100.2% 98.2% average 0.0% -0.2% 0.0% 0.0% -0.2% -0.2% 99.4% 100.8% 98.1%
Table 2
video video synth
video video video PSNR PSNR PSNR ren
enc time dec time
0 1 2 / video / total / total time bitrate bitrate bitrate
Balloons 0.0% 0.0% 0.0% 0.0% -0.3% 0.0% 101.9% 99.8% 96.7%
Kendo 0.0% 0.0% 0.0% 0.0% -0.3% 0.0% 101.7% 103.1% 101.5%
Newspaper CC 0.0% 0.0% 0.0% 0.0% -0.1% 0.1% 101.3% 102.3% 103.9%
GT Fly 0.0% 0.0% 0.0% 0.0% -0.2% -0.2% 97.8% 100.6% 97.1%
Poznan_Hall2 0.0% 0.0% 0.0% 0.0% 0.0% 0.1% 102.0% 95.9% 102.1%
Poznan Street 0.0% 0.0% 0.0% 0.0% -0.2% -0.2% 100.2% 97.2% 97.0%
Undo Dancer 0.0% 0.0% 0.0% 0.0% 0.0% -0.1% 99.5% 99.3% 96.8%
1024x768 0.0% 0.0% 0.0% 0.0% -0.2% 0.0% 101.6% 101.7% 100.7%
1920x1088 0.0% 0.0% 0.0% 0.0% -0.1% -0.1% 99.9% 98.3% 98.2% average 0.0% 0.0% 0.0% 0.0% -0.2% -0.1% 100.6% 99.7% 99.3%
Fig. 8 illustrates an exemplary flowchart of sample-based Simplified Depth Coding (SDC) for depth data using Intra modes according to an embodiment of the present invention. The system receives input data associated with a current depth block as shown in step 810. For encoding, the input data associated with the depth block corresponds to the depth samples to be coded. For decoding, the input data associated with the current depth block corresponds to the coded depth data to be decoded. The input data associated with the current depth block may be retrieved from memory (e.g., computer memory, buffer (RAM or DRAM) or other media) or from a processor. Prediction samples for the current depth block are then determined based on reconstructed neighboring depth samples according to a selected Intra mode as shown in step 820. A residual value (of each segment) of the current depth block is encoded or decoded, and an offset value (of each segment) is then derived from the residual value (using Eqn. 4 as an example) as shown in step 830. The reconstructed samples are derived by adding the offset
value to the prediction samples (for each segment) as shown in step 840.
The flowchart shown above is intended to illustrate an example of sample-based Simplified Depth Coding (SDC). A person skilled in the art may modify each step, re-arranges the steps, split a step, or combine steps to practice the present invention without departing from the spirit of the present invention.
The above description is presented to enable a person of ordinary skill in the art to practice the present invention as provided in the context of a particular application and its requirement. Various modifications to the described embodiments will be apparent to those with skill in the art, and the general principles defined herein may be applied to other embodiments. Therefore, the present invention is not intended to be limited to the particular embodiments shown and described, but is to be accorded the widest scope consistent with the principles and novel features herein disclosed. In the above detailed description, various specific details are illustrated in order to provide a thorough understanding of the present invention. Nevertheless, it will be understood by those skilled in the art that the present invention may be practiced.
Embodiment of the present invention as described above may be implemented in various hardware, software codes, or a combination of both. For example, an embodiment of the present invention can be a circuit integrated into a video compression chip or program code integrated into video compression software to perform the processing described herein. An embodiment of the present invention may also be program code to be executed on a Digital Signal Processor (DSP) to perform the processing described herein. The invention may also involve a number of functions to be performed by a computer processor, a digital signal processor, a microprocessor, or field programmable gate array (FPGA). These processors can be configured to perform particular tasks according to the invention, by executing machine- readable software code or firmware code that defines the particular methods embodied by the invention. The software code or firmware code may be developed in different programming languages and different formats or styles. The software code may also be compiled for different target platforms. However, different code formats, styles and languages of software codes and other means of configuring code to perform the tasks in accordance with the invention will not depart from the spirit and scope of the invention.
The invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is therefore, indicated by the appended claims rather than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.
Claims
1. A method of Intra coding for a depth block in a three-dimensional coding system, the method comprising:
receiving input data associated with a current depth block;
determining prediction samples in a segment of the current depth block based on reconstructed neighboring depth samples according to a selected Intra mode;
encoding or decoding a residual value for the segment of the current depth block;
deriving an offset value from the residual value for the segment of the current depth block; and
reconstructing final reconstructed samples by adding the offset value to each of the prediction samples of the segment.
2. The method of Claim 1, wherein the offset value corresponds to a difference between a reconstructed depth value and a predicted depth value for the segment of the current depth block.
3. The method of Claim 1, wherein the offset value is derived from the residual value, wherein the residual value is derived implicitly at a decoder side or the residual value is transmitted in a bitstream.
4. The method of Claim 1, wherein deriving the offset value from the residual value comprises determining a residual index according to an inverse Lookup Table.
5. The method of Claim 1, wherein the selected Intra mode corresponds to Planar mode, the prediction samples are determined using linear interpolation and bilinear interpolation from the reconstructed neighboring depth samples of the current depth block according to the Planar mode, and the offset value is derived from the residual value or a residual index.
6. The method of Claim 1, wherein the selected Intra mode is selected from a set of Intra modes.
7. The method of Claim 6, wherein selection of the selected Intra mode from the set of Intra modes is signalled in a bitstream.
8. The method of Claim 7, wherein the set of Intra modes consists of DC (Direct Current) mode, DMM (Depth Modelling Modes) Mode 1 and Planar mode.
9. The method of Claim 7, wherein the set of Intra modes consists of DC mode, DMM Mode 1 and VSP mode.
10. The method of Claim 7, wherein the set of Intra modes consists of Planar mode, DMM Mode 1 and VSP mode.
11. The method of Claim 7, wherein a truncated unary code is used to indicate the selected Intra mode from the set of Intra modes.
12. The method of Claim 1, wherein the current depth block comprises only one segment when the selected Intra mode is DC mode or Planer mode.
13. The method of Claim 1, wherein the current depth block comprises only one segment when the selected Intra mode is selected from the Intra modes in High Efficient Video Coding (HEVC).
14. An apparatus for Intra coding of a depth block in a three-dimensional coding system, the apparatus comprising one or more electronic circuits, wherein said one or more electronic circuits are configured to:
receive input data associated with a current depth block;
determine prediction samples for a segment of the current depth block based on reconstructed neighboring depth samples according to a selected Intra mode;
encoding or decoding a residual value for the segment of the current depth block;
deriving an offset value from the residual value for the segment of the current depth block; and
reconstructing final reconstructed samples by adding the offset value to each of the prediction samples of the segment.
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/762,498 US20150365698A1 (en) | 2013-04-11 | 2014-03-26 | Method and Apparatus for Prediction Value Derivation in Intra Coding |
EP14782399.1A EP2920970A4 (en) | 2013-04-11 | 2014-03-26 | Method and apparatus for prediction value derivation in intra coding |
CN201480020741.4A CN105122809A (en) | 2013-04-11 | 2014-03-26 | Method and apparatus for prediction value derivation in intra coding |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201361810797P | 2013-04-11 | 2013-04-11 | |
US61/810,797 | 2013-04-11 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2014166338A1 true WO2014166338A1 (en) | 2014-10-16 |
Family
ID=51688934
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2014/074130 WO2014166338A1 (en) | 2013-04-11 | 2014-03-26 | Method and apparatus for prediction value derivation in intra coding |
Country Status (4)
Country | Link |
---|---|
US (1) | US20150365698A1 (en) |
EP (1) | EP2920970A4 (en) |
CN (1) | CN105122809A (en) |
WO (1) | WO2014166338A1 (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2016070363A1 (en) * | 2014-11-05 | 2016-05-12 | Mediatek Singapore Pte. Ltd. | Merge with inter prediction offset |
WO2016200235A1 (en) * | 2015-06-11 | 2016-12-15 | 엘지전자(주) | Intra-prediction mode-based image processing method and apparatus therefor |
WO2019009752A1 (en) * | 2017-07-05 | 2019-01-10 | Huawei Technologies Co., Ltd. | Devices and methods for video coding |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20120140181A (en) * | 2011-06-20 | 2012-12-28 | 한국전자통신연구원 | Method and apparatus for encoding and decoding using filtering for prediction block boundary |
WO2014166116A1 (en) * | 2013-04-12 | 2014-10-16 | Mediatek Inc. | Direct simplified depth coding |
KR20150010660A (en) * | 2013-07-18 | 2015-01-28 | 삼성전자주식회사 | Method and apparatus for video encoding for Intra prediction of depth map, Method and apparatus for video Decoding for Intra prediction of depth map |
KR20150043227A (en) * | 2013-10-14 | 2015-04-22 | 삼성전자주식회사 | Method and apparatus for depth inter encoding and method and apparatus for depth inter decoding |
US9756359B2 (en) * | 2013-12-16 | 2017-09-05 | Qualcomm Incorporated | Large blocks and depth modeling modes (DMM'S) in 3D video coding |
US11166048B2 (en) * | 2018-10-02 | 2021-11-02 | Tencent America LLC | Method and apparatus for video coding |
US12081742B2 (en) * | 2021-06-11 | 2024-09-03 | Tencent America LLC | Intra mode coding |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20120183057A1 (en) * | 2011-01-14 | 2012-07-19 | Samsung Electronics Co., Ltd. | System, apparatus, and method for encoding and decoding depth image |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2127395B1 (en) * | 2007-01-10 | 2016-08-17 | Thomson Licensing | Video encoding method and video decoding method for enabling bit depth scalability |
KR101716636B1 (en) * | 2009-07-27 | 2017-03-15 | 코닌클리케 필립스 엔.브이. | Combining 3d video and auxiliary data |
EP2942954B1 (en) * | 2011-10-24 | 2020-06-03 | Innotive Ltd | Image decoding apparatus |
KR102216585B1 (en) * | 2013-01-04 | 2021-02-17 | 삼성전자주식회사 | Encoding apparatus and decoding apparatus for depth map, and encoding method and decoding method |
US10271034B2 (en) * | 2013-03-05 | 2019-04-23 | Qualcomm Incorporated | Simplified depth coding |
-
2014
- 2014-03-26 US US14/762,498 patent/US20150365698A1/en not_active Abandoned
- 2014-03-26 CN CN201480020741.4A patent/CN105122809A/en active Pending
- 2014-03-26 WO PCT/CN2014/074130 patent/WO2014166338A1/en active Application Filing
- 2014-03-26 EP EP14782399.1A patent/EP2920970A4/en not_active Withdrawn
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20120183057A1 (en) * | 2011-01-14 | 2012-07-19 | Samsung Electronics Co., Ltd. | System, apparatus, and method for encoding and decoding depth image |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2016070363A1 (en) * | 2014-11-05 | 2016-05-12 | Mediatek Singapore Pte. Ltd. | Merge with inter prediction offset |
WO2016200235A1 (en) * | 2015-06-11 | 2016-12-15 | 엘지전자(주) | Intra-prediction mode-based image processing method and apparatus therefor |
WO2019009752A1 (en) * | 2017-07-05 | 2019-01-10 | Huawei Technologies Co., Ltd. | Devices and methods for video coding |
US11277621B2 (en) | 2017-07-05 | 2022-03-15 | Huawei Technologies Co., Ltd. | Devices and methods for video coding |
Also Published As
Publication number | Publication date |
---|---|
EP2920970A1 (en) | 2015-09-23 |
EP2920970A4 (en) | 2016-04-20 |
US20150365698A1 (en) | 2015-12-17 |
CN105122809A (en) | 2015-12-02 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111819852B (en) | Method and apparatus for residual symbol prediction in the transform domain | |
KR102171788B1 (en) | Adaptive partition coding | |
US20150365698A1 (en) | Method and Apparatus for Prediction Value Derivation in Intra Coding | |
CN107257485B (en) | Decoder, encoder, decoding method, and encoding method | |
CN113840143A (en) | Encoder, decoder and corresponding method using IBC-specific buffers | |
CN112868232B (en) | Method and apparatus for intra prediction using interpolation filter | |
CN111837397A (en) | Bitstream indication for error concealment in view-dependent video coding based on sub-picture bitstream | |
US9503751B2 (en) | Method and apparatus for simplified depth coding with extended prediction modes | |
CN113508592A (en) | Encoder, decoder and corresponding inter-frame prediction method | |
JP2022179505A (en) | Video decoding method and video decoder | |
CN111837389A (en) | Block detection method and device suitable for multi-sign bit hiding | |
CN112673626A (en) | Relationships between segmentation constraint elements | |
KR102464520B1 (en) | Method and apparatus for image filtering using adaptive multiplication coefficients | |
CN114424531A (en) | In-loop filtering based video or image coding | |
CN114424567A (en) | Method and apparatus for combined inter-intra prediction using matrix-based intra prediction | |
CN114679583B (en) | Video encoder, video decoder and corresponding methods | |
CN114128273A (en) | Video or image coding based on luminance mapping | |
CN113875251A (en) | Adaptive filter strength indication for geometric partitioning modes | |
WO2020063687A1 (en) | Video decoding method and video decoder | |
CN114175651A (en) | Video or image coding based on luma mapping and chroma scaling | |
CN114424554A (en) | Method and apparatus for chroma QP offset Table indication and derivation | |
CN114930851A (en) | Image coding method and device based on transformation | |
CN114270823A (en) | Video or image coding based on luma mapping and chroma scaling | |
CN114270851A (en) | Video or image coding based on luminance mapping | |
CN114586354A (en) | Matrix-based intra prediction apparatus and method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 14782399 Country of ref document: EP Kind code of ref document: A1 |
|
REEP | Request for entry into the european phase |
Ref document number: 2014782399 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2014782399 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 14762498 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |