KR102471528B1 - Adaptive partition coding - Google Patents

Abstract

While Wedgelet-based segmentation seems to represent a good trade-off between the secondary information rate on the one hand and the achievable diversity of splitting possibilities on the other hand compared to contour segmentation, it relaxes the constraints of segmentation to the extent that segmentation must be Wedgelet segmentation. The ability allows to apply relatively uncomplicated statistical analysis onto overlapping spatially sampled texture information to derive good predictors for double segmentation in depth/disparity maps. Thus, according to the first aspect, it exactly increases the freedom to mitigate the signaling overhead in the presence of co-located texture information in the form of a picture. Another aspect relates to the possibility of storing the auxiliary information rate associated with signaling each coding mode supporting irregular partitioning.

Description

Adaptive Partition Coding {ADAPTIVE PARTITION CODING}

The present invention relates to sample array coding using contour block partitioning or block partitioning that enables a high degree of freedom.

Most coding schemes compress sample array data using subdivisions of the sample array into blocks. A sample array may define a texture, i.e., a spatial sampling of a picture, but of course other sample arrays may be compressed using similar coding techniques such as depth maps and the like. Due to the different nature of the information spatially sampled by each sample array, different coding concepts are best suited to different kinds of sample arrays. However, regardless of this kind of sample array, most of these coding concepts use block-subdivisioning to assign individual coding options to blocks of the sample array, on the one hand Find a good tradeoff between the side information rate for coding the coding parameters and the residual coding rate for coding the residual prediction due to the prediction failure of each block, with or without residual coding Find a good compromise in rate/distortion sense.

Most of the time, blocks are rectangular or quadratic in shape. Clearly, it is desirable to be able to adapt the shape of a coding unit (block) to the content of the sample array to be coded. Unfortunately, however, adapting the shape of a block or coding unit to the contents of an array of samples involves using additional auxiliary information for signaling of block divisions. A wedgelet-type partitioning of blocks has been found to be an appropriate compromise between possible block partitioning shapes and the auxiliary information overhead involved. Wedgelet-like partitioning leads to partitioning of blocks, for example with Wedgelet partitioning in which specific coding parameters can be used.

However, even the limitation on Wedgelet partitioning leads to a significant amount of additional overhead for signaling of partitioning of blocks, thus enabling a higher degree of freedom in partitioning of blocks when coding sample arrays in a more efficient manner. It is desirable to have effective coding concepts at hand.

This object is achieved by the subject matter of the pending independent claims.

The main idea underlying the present invention is that although wedgelet-based segmentation appears to represent a good trade-off between the auxiliary information rate on the one hand and the achievable diversity of the splitting possibilities on the other hand, compared to contour segmentation, the segmentation should be a wedgelet partition. The ability to relax the constraints of segmentation to the extent that it does so is a relatively uncomplicated statistical analysis over the overlapped spatially sampled texture information to derive a good predictor for bi-segmentation in the depth/disparity map. make it possible to apply Thus, according to the first aspect, it exactly increases the freedom to mitigate the signaling overhead in the presence of co-located texture information in picture form.

Another concept on which a further aspect of the present invention is based is the just described idea that the derivation of double segmentation is based on a co-located reference block in a picture with subsequent propagation of the double segmentation onto the current block of the depth/disparity map. It is only valid if the probability of achieving a good approximation of the contents of the current block of the depth/disparity map is sufficiently high to justify the reservation of each predetermined value of the corresponding coding option identifier to trigger this dual segmentation propagation mode. that it is In other words, it avoids the need to take each predetermined value of the coding option identifier for the current block of the depth/disparity map when entropy coding these coding option identifiers in the case where it is highly likely that the respective double segmentation delivery will not be chosen anyway. By doing so, the auxiliary information rate can be saved.

Further auxiliary aspects are the subject of the dependent claims.

A preferred embodiment of the present invention is described in more detail below with reference to the drawings.

1 shows a block diagram of a multi-view encoder on which an embodiment of the present invention may be built according to an example.
Figure 2 shows a schematic diagram of a portion of a multiview signal to illustrate information reuse across view and video depth/displacement boundaries.
Fig. 3 shows a block diagram of a decoder consistent with Fig. 1;
Figure 4 shows Wedgelet partitioning of quadratic blocks in continuous (left) and discrete signal space (right).
Figure 5 shows a schematic diagram of six different directions of Wedgelet block division.
Figure 6 shows an example of a Wedgelet partitioning pattern for block sizes 4x4 (left), 8x8 (middle), and 16x16 (right).
Fig. 7 shows an approximation of a depth signal with a Wedgelet model by combining segmentation information and CPV (average value of depth signals of segmented areas).
8 shows the creation of a Wedgelet split pattern.
Figure 9 shows contour segmentation of a quadratic block in continuous (left) and discrete signal space (right).
Fig. 10 shows an example of a contour segmentation pattern for a block size of 8x8.
Fig. 11 shows an approximation of a depth signal with a contour model by combining segmentation information and CPV (average value of depth signals of segmented areas).
Figure 12 shows intra prediction of a Wedgelet partition (blue) for a scenario where the reference block above is either a wedgelet partition (left) or in a regular intra direction (right).
Figure 13 shows the prediction of wedgelet (blue) and contour (green) segmentation information from a texture luma reference.
Figure 14 shows the CPV of a block partition showing the relationship between different CPV types: CPV prediction from adjacent samples of neighboring blocks (left) and cross-sections of blocks (right).
15 shows mode pre-selection based on texture luma variance.
16 shows a block diagram of a decoder according to an embodiment.
Fig. 17 shows a block diagram of an encoder consistent with Fig. 16;
18 shows a block diagram of a decoder according to an embodiment.
Fig. 19 shows a block diagram of an encoder consistent with Fig. 18;
20 shows a block diagram of a decoder according to an embodiment.
Fig. 21 shows a block diagram of an encoder consistent with Fig. 20;
22 shows a block diagram of a decoder according to an embodiment.
Fig. 23 shows a block diagram of an encoder consistent with Fig. 22;
24 shows a block diagram of a decoder according to an embodiment.
Fig. 25 shows a block diagram of an encoder consistent with Fig. 24;

The following description of preferred embodiments of the present invention begins with possible circumstances in which embodiments of the present invention may be advantageously employed. In particular, the multiview codec according to the embodiment is described with respect to FIGS. 1 to 3 . However, it should be emphasized that the embodiments described below are not limited to multiview coding. Nevertheless, some aspects described below can be better understood and have a particular synergistic effect when used with multiview coding, or, to be more precise, especially with coding of depth maps. Thus, after Figures 1-3, the description proceeds to the introduction to irregular block partitioning, and the problems involved therewith. This description is referred to FIGS. 4-11 and forms the basis for the description of embodiments of the present invention described hereafter.

As noted above, the embodiments described further below take advantage of non-rectangular or irregular block partitioning and modeling capabilities in image and video coding applications, particularly of scenes, although such embodiments are also applicable to conventional image and video coding. Applicable for coding of depth maps, such as for representing geometric shapes. Embodiments described further below further provide concepts for using non-rectangular block segmentation and modeling functionality in image and video coding applications. The embodiment is particularly applicable to the coding of depth maps (to represent the geometry of a scene), but is also applicable to conventional image and video coding.

In multiview video coding, two or more views of a video scene (captured simultaneously by multiple cameras) are coded into a single bitstream. The main purpose of multi-view video coding is to provide end users with an enhanced multimedia experience by providing a three-dimensional viewing impression. Once the two views are coded, the two reconstructed video sequences can be displayed on a conventional stereo display (with glasses). However, requiring the use of glasses for conventional stereo displays is often cumbersome for users. Enabling a high-quality stereo viewing experience without glasses is an important topic of current research and development. A promising technology for such autostereoscopic displays is based on lenticular lens systems. In principle, an array of cylindrical lenses is mounted on a conventional display in such a way that multiple views of a video scene are displayed simultaneously. Each view is displayed on a small cone so that each eye of the user sees a different image, an effect that creates a stereo feel without the need for special glasses. However, such autostereoscopic displays typically require 10-30 views of the same video scene (more views may be required as technology improves further). Three or more views can also be used to provide the user with the possibility to interactively select a viewpoint for a video scene. However, coding of multiple views of a video scene greatly increases the required bit rate compared to conventional single-view (two-dimensional) video. Typically, the required bit rate increases approximately linearly with the number of coded views. The concept of reducing the amount of transmitted data for an autostereoscopic display is to transmit only a small number of views (perhaps 2-5 views), but additionally the depth of image samples for one or more views (real world relative to the camera). distance of the object) is supposed to transmit a so-called depth map. Given a small number of coded views in the corresponding depth map, high-quality intermediate views (virtual views between the coded views) and, to some extent, also additional views on one or both ends of the camera array are suitable rendering techniques. It can be generated on the receiver side by

픽처는 또한 예를 들어 투명도 정보 또는 깊이 맵을 할당할 수 있는 부가적인 보조 샘플 어레이와 관련될 수 있다. 각각의 픽처 또는 샘플 어레이는 일반적으로 블록으로 분해된다. 블록(또는 샘플 어레이의 대응하는 블록)은 인터픽처 예측 또는 인트라픽처 예측 중 하나에 의해 예측된다. 블록은 서로 다른 크기를 가질 수 있고, 이차 또는 직사각형일 수 있다. 블록으로의 픽처의 분할은 구문(syntax)에 의해 고정될 수 있거나, (적어도 부분적으로) 비트스트림 내에 시그널링될 수 있다. 종종 미리 정의된 크기의 블록에 대한 세분을 시그널링하는 구문 요소가 전송된다. 이러한 구문 요소는 블록이 더욱 작은 블록으로 세분되고, 예를 들어 예측을 위해 관련된 코딩 파라미터인지의 여부 및 방법을 할당할 수 있다. 블록(또는 샘플 어레이의 대응하는 블록)의 모든 샘플의 경우, 관련된 코딩 파라미터의 디코딩은 어떤 방식으로 할당된다. 예에서, 블록의 모든 샘플은 (이미 코딩된 픽처의 세트의 참조 픽

픽처를 식별하는) 참조 인덱스, (참조 픽처와 현재 픽처 사이에서 블록의 움직임에 대한 측정을 할당하는) 움직임 파라미터, 보간 필터를 할당하는 파라미터, 인트라 예측 모드 등과 같은 예측 파라미터의 동일한 세트를 이용하여 예측된다. 움직임 파라미터는 수평 및 수직 구성 요소을 가진 변위 벡터에 의해 나타내거나 6개의 구성 요소로 이루어지는 아핀(affine) 움직임 파라미터와 같은 고차 움직임 파라미터에 의해 나타낼 수 있다. 또한, (참조 인덱스 및 움직임 파라미터와 같은) 특정 예측 파라미터의 둘 이상의 세트는 단일의 블록과 관련되는 것이 가능하다. 그 경우에, 이러한 특정 예측 파라미터의 각 세트에 대해, 블록(또는 샘플 어레이의 대응하는 블록)에 대한 단일의 중간 예측 신호가 생성되고, 최종 예측 신호는 중간 예측 신호를 중첩하는 것을 포함하는 조합에 의해 구축된다. 대응하는 가중 파라미터 및 잠재적으로 또한 (가중된 합에 부가되는) 일정한 오프셋은 픽처 또는 참조 픽처 또는 참조 픽처의 세트에 고정될 수 있거나, 이들은 대응하는 블록에 대한 예측 파라미터의 세트에 포함될 수 있다. 또한, 잔여 신호로 지칭되는 원래의 블록(또는 샘플 어레이의 대응하는 블록)과 이의 예측 신호 사이의 차이는 일반적으로 변환되어 양자화된다. 종종, 2차원 변환은 잔여 신호(또는 잔여 블록에 대한 대응하는 샘플 어레이)에 적용된다. 변환 코딩의 경우, 예측 파라미터의 특정 세트가 이용된 블록(또는 샘플 어레이의 대응하는 블록)은 변환을 적용하기 전에 추가로 분할될 수 있다. 변환 블록은 예측에 이용되는 블록과 동일하거나 작을 수 있다. 또한, 변환 블록은 예측에 이용되는 둘 이상의 블록을 포함하는 것이 가능하다. 서로 다른 변환 블록은 서로 다른 크기를 가질 수 있고, 변환 블록은 이차 또는 직사각형 블록을 나타낼 수 있다. 변환 후에, 생성된 변환 계수는 양자화되고, 소위 변환 계수 레벨이 획득된다. 변환 계수 레벨 뿐만 아니라 예측 파라미터 및, 제공된다면 세분 정보가 엔트로피 코딩된다.In current state of the art image and video coding, a picture or a specific set of sample arrays for a picture is usually decomposed into blocks associated with specific coding parameters. A picture is usually composed of multiple arrays of samples (luminance and chrominance). Besides, pick

A picture can also be associated with an additional auxiliary sample array that can be assigned, for example, transparency information or a depth map. Each picture or array of samples is usually decomposed into blocks. A block (or a corresponding block in an array of samples) is predicted by either inter-picture prediction or intra-picture prediction. Blocks can have different sizes and can be quadratic or rectangular. The division of a picture into blocks may be fixed by syntax, or may be signaled (at least partially) in the bitstream. Often a syntax element signaling the subdivision into blocks of a predefined size is transmitted. These syntax elements can assign whether and how blocks are subdivided into smaller blocks and related coding parameters, for example for prediction. For every sample of a block (or the corresponding block of the sample array), the decoding of the associated coding parameter is assigned in some way. In an example, all samples of a block (reference picture of a set of already coded pictures

Prediction using the same set of prediction parameters, such as reference index (which identifies the picture), motion parameters (which assigns a measure of the block's motion between the reference picture and the current picture), parameters which assign interpolation filters, intra-prediction modes, etc. do. A motion parameter can be represented by a displacement vector with horizontal and vertical components or by a higher order motion parameter such as an affine motion parameter with six components. It is also possible that more than one set of specific prediction parameters (such as reference index and motion parameters) are associated with a single block. In that case, for each set of these particular prediction parameters, a single intermediate prediction signal for the block (or corresponding block in the sample array) is generated, and the final prediction signal is generated in a combination comprising overlapping the intermediate prediction signals. is built by Corresponding weighting parameters and potentially also constant offsets (added to the weighted sum) may be fixed to a picture or reference picture or set of reference pictures, or they may be included in a set of prediction parameters for the corresponding block. Also, the difference between the original block (or the corresponding block of the sample array), referred to as the residual signal, and its prediction signal is generally transformed and quantized. Often, a two-dimensional transform is applied to the residual signal (or the corresponding array of samples for the residual block). In the case of transform coding, the block for which a particular set of prediction parameters was used (or the corresponding block in the sample array) may be further partitioned before applying the transform. The transform block may be the same as or smaller than the block used for prediction. Also, a transform block may include two or more blocks used for prediction. Different transform blocks can have different sizes, and transform blocks can represent quadratic or rectangular blocks. After transformation, the generated transform coefficients are quantized, and so-called transform coefficient levels are obtained. The transform coefficient levels as well as prediction parameters and, if provided, granularity information are entropy coded.

Also, ITU-T Rec. H.264 | Current state-of-the-art coding techniques such as ISO/IEC JTC 1 14496-10 or working models for HEVC are also applicable to depth maps, and the coding tools are specifically designed for coding of natural video. Depth maps have different characteristics from pictures in natural video sequences. For example, depth maps contain little spatial detail. They are mainly characterized by sharp edges (representing object boundaries) and large regions of nearly constant or slowly varying sample values (representing object regions). The overall coding efficiency of multiview video coding with a depth map can be improved if the depth map is coded more efficiently by applying a coding tool specifically designed to exploit the properties of the depth map.

In order to serve as a basis for possible coding environments in which the embodiments of the present invention described in the following may be used to advantage, possible multiview coding concepts are further described below with respect to FIGS. 1 to 3 .

1 shows an encoder for encoding a multiview signal according to an embodiment. Although the embodiment of FIG. 1 can also be realized with multiple views, the multiview signal of FIG. 1 is illustratively shown at 10 comprising two views 12 ₁ and 12 ₂ . Moreover, according to the embodiment of Fig. 1, although many of the advantageous principles of the embodiments described further below are also beneficial when used in conjunction with multiview signals with views that do not contain any depth/disparity map data, Each view 12 ₁ and 12 ₂ includes video 14 and depth/disparity map data 16 .

Video 14 of each view 12 ₁ and 12 ₂ represents a spatio-temporal sampling of a projection of a typical scene along different projection/viewing directions. Preferably, the temporal sampling rates of the videos 14 of views 12 ₁ and 12 ₂ are equal to each other, although this constraint does not necessarily have to be met. As shown in Figure 1, each video 14 preferably comprises a sequence of frames, each frame being associated with a respective timestamp t, t-1, t-2,... have. In FIG. 1, a video frame is indicated by a V _{view number and a time stamp number} . Each frame V _i,t represents a spatial sampling of scene i along each view direction at each time stamp t, thus e.g. one sample array with luma samples and two sample arrays with chroma samples , or simply one or more sample arrays, such as luminance samples or sample arrays for other color components, such as color components in an RGB color space. The spatial resolution of one or more sample arrays can be different both within one video 14 and within video 14 of different views 12 ₁ and 12 ₂ .

Similarly, depth/disparity map data 16 represents a spatio-temporal sampling of the depths of scene objects in a typical scene as measured along respective viewing directions of views 12 ₁ and 12 ₂ . The temporal sampling rate of the depth/disparity map data may be the same as or different from the temporal sampling rate of the related video of the view as shown in FIG. 1 . In the case of FIG. 1 , each video frame v is associated with a respective depth/disparity map d of the depth/disparity map data 16 of each view 12 ₁ and 12 ₂ . In other words, in the example of FIG. 1, each video frame v _i,t of view i and timestamp t has a depth/disparity map d _i,t associated with it. Regarding the spatial resolution of the depth/disparity map d, the same applies as described above for video frames. That is, the spatial resolution may be different between depth/disparity maps of different views.

To effectively compress the multiview signal 10, the encoder of FIG. 1 encodes views 12 ₁ and 12 ₂ into data stream 18 in parallel. However, the coding parameters used to encode the first view 12 ₁ are reused to adopt or predict the same as the second coding parameters to be used to encode the second view 12 ₂ . By this measure, the encoder of FIG. 1 takes advantage of the fact that parallel encoding of views 12 ₁ and 12 ₂ causes the encoder to similarly determine the coding parameters for these views, so that the overlap between these coding parameters is the compression ratio or rate /to be used efficiently to increase the distortion rate (distortion is measured as the average distortion of all views, for example, and rate is measured as the coding rate of the entire data stream 18).

In particular, the encoder of FIG. 1 is indicated generally by reference numeral 20 and includes an input for receiving a multiview signal 10 and an output for outputting a data stream 18 . As can be seen in FIG. 2 , the encoder 20 of FIG. 1 includes two coding branches per view 12 ₁ and 12 ₂ , one for video data and the other for depth/disparity map data. . Accordingly, the encoder 20 has a coding branch 22 _v,1 for the video data of view 1, a coding branch 22 d,1 for the depth/disparity map data of view 1, and a coding branch 22 _d,1 for the video data of the second view. coding branch 22 _v,2 , and coding branch 22 _d,2 for the depth/disparity map data of the second view. Each of these coding branches 22 is similarly configured. To describe the construction and function of the encoder 20, the following description begins with the construction and function of the coding branch 22 _v,1 . This function is common to all branches 22 . After that, the individual characteristics of the branches 22 are discussed.

Coding branch 22 _v,1 is for encoding video 14 ₁ of first view 12 ₁ of multiview signal 12, thus branch 22 _v,1 corresponds to video 14 ₁ . It has input to receive. In addition to this, the branches 22 _{v, 1} are serially connected to each other in the above-described order, the subtractor 24, the quantization/transformation module 26, the requantization/inverse transformation module 28, the adder 30, the addition of the processing module 32, the decoded picture buffer 34, consequently the two prediction modules 36 and 38 connected in parallel with each other, and the outputs of the prediction modules 36 and 38 on the one hand and the subtractor on the other hand ( and a combiner or selector 40 connected between the inverting inputs of 24). The output of combiner 40 is also connected to a further input of adder 30. The non-inverting input of subtractor 24 receives video 14 ₁ .

Elements 24 to 40 of coding branch 22 _v,1 cooperate to encode video 14 ₁ . Encoding encodes the video 14 ₁ in units of certain parts. For example, when encoding video 14 ₁ , frame v _1,k is divided into segments such as blocks or other groups of samples. Segmentation can be constant over time or can change over time. Moreover, the segmentation may be known to the encoder and decoder by default or may be signaled within the data stream 18 . Segmentation can be a regular segmentation of a frame into blocks, such as a non-overlapping arrangement of blocks into rows and columns, or it can be a quad-tree based segmentation into blocks of various sizes. The currently encoded segment of video 14 ₁ entering the non-inverting input of subtractor 24 is referred to as the current block of video 14 ₁ in the following discussion of FIGS. 1 to 3 .

Prediction modules 36 and 38 are for predicting the current block, for which purpose prediction modules 36 and 38 have their inputs connected to decoded picture buffer 34 . In effect, both prediction modules 36 and 38 use previously reconstructed portions of video 14 ₁ present in decoded picture buffer 34 to predict the current block entering the non-inverting input of subtractor 24. use In this regard, prediction module 36 acts as an intra predictor that spatially predicts the current portion of video 14 ₁ from a spatially neighboring already reconstructed portion of the same frame of video 14 ₁ , while Prediction module 38 acts as an inter predictor that temporally predicts the current portion from a previously reconstructed frame of video 14 ₁ . Modules 36 and 38 both perform predictions according to or described by certain prediction parameters. More precisely, the latter parameter is determined to be the encoder 20 in some optimization frameworks for optimizing some optimization goal, such as optimizing rate/distortion ratio with or without some constraint, such as maximum bitrate.

For example, intra-prediction module 36 can determine spatial prediction parameters for the current portion, such as the intra-prediction direction, so that any content in a neighboring already reconstructed portion of the same frame of video 14 ₁ can be compared to the latter. is extended/copied into the current part to predict .

Inter prediction module 38 may use motion compensation to predict the current portion from previously reconstructed frames, inter prediction parameters included therewith include motion vector, reference frame index, motion prediction refinement information about the current portion, number of hypotheses or any combination thereof.

Combiner 40 may combine one or more of the predictions provided by modules 36 and 38 or select only one of them. The combiner or selector 40 forwards the generated prediction of the current portion to the insert input of subtractor 24 and the additional input of adder 30, respectively.

At the output of the subtractor 24, the residual prediction of the current part is output, and the quantization/transform module 36 is configured to transform this residual signal by quantizing the transform coefficients. The transform can be any spectral decomposition transform, such as DCT. Due to quantization, the processing result of quantization/transform module 26 is irreversible. That is, coding loss results. The output of module 26 is the residual signal 42 ₁ transmitted within the data stream. Not all blocks can be residual coding. Rather, some coding modes can suppress residual coding.

Residual signal 42 ₁ is inversely quantized and inverse transformed in module 28 to reconstruct a possible residual signal, ie to correspond to the residual signal as it is output by subtractor 24 despite quantization noise. The adder 30 sums these reconstructed residual signals and combines them with the prediction of the current part. Other combinations are also possible. For example, subtracter 24 can act as a divider to measure the ratio of the residuals, and adders can alternatively be implemented as multipliers to reconstruct the current part. Thus, the output of adder 30 represents a preliminary reconstruction of the current part. However, additional processing in module 32 may optionally be used to augment the reconstruction. Such additional processing may include, for example, deblocking, adaptive filtering, and the like. All reconstructions available so far are buffered in the decoded picture buffer 34. Thus, decoded picture buffer 34 buffers the previously reconstructed frame of video 14 ₁ and the previously reconstructed portion of the current frame to which the current portion belongs.

To enable the decoder to reconstruct the multiview signal from the data stream 18, the quantization/transform module 26 sends the residual signal 42 ₁ to the multiplexer 44 of the encoder 20. At the same time, prediction module 36 sends intra prediction parameters 46 ₁ to multiplexer 44, inter prediction module 38 sends inter prediction parameters 48 ₁ to multiplexer 44, and further processing Module 32 sends additional processing parameters 50 ₁ to multiplexer 44, which in turn multiplexes all this information and inserts it into data stream 18.

As is evident from the above discussion according to the embodiment of FIG. 1 , the encoding of video 14 ₁ by coding branch 22 _v,1 , the encoding consists of depth/disparity map data 16 ₁ and other views 12 ₂ ) is self-contained in that it is independent of any data in From a more general point of view, the coding branch 22 _v,1 determines the coding parameters, according to which the video 14 ₁ is encoded by the encoder 20 into the data stream 18 before the encoding of the current portion according to the first coding parameters. Predict the current portion of the video 14 ₁ from the previously encoded portion of the video 14 ₁ by determining the prediction error of the prediction of the current portion to obtain the correction data, namely the residual signal 42 ₁ described above. can be considered as encoding into the data stream 18. Coding parameters and correction data are inserted into the data stream 18.

The just-mentioned coding parameters inserted into the data stream 18 by the coding branch 22 _v,1 may include one, a combination, or all of the following:

- Firstly, the coding parameters for video 14 ₁ may define/signal the segmentation of frames of video 14 ₁ as briefly discussed previously.

- Furthermore, the coding parameter may include coding mode information indicating each segment or current portion, and the coding mode should be used to predict each segment, such as intra prediction, inter prediction or a combination thereof.

- Coding parameters may also include just mentioned prediction parameters, such as intra prediction parameters for parts/segments predicted by intra prediction, and inter prediction parameters for inter predicted parts/segments.

However, the coding parameters are additionally processed further signaling to the decoding side how to further process the already reconstructed part of the video 14 ₁ before using the same for predicting the current or next part of the video 14 ₁ . It can contain parameters (50 ₁ ). These additional processing parameters 50 ₁ may include an index indexing each filter, filter coefficient, and the like.

- Prediction parameters (46 ₁ , 48 ₁ ) and further processing parameters (50 ₁ ) additionally define the granularity of the mode selection, or for devices of different adaptive filters for different parts of the frame within further processing. Subsegmentation data may be included to define an additional subsegmentation to the above-described segmentation that defines a completely independent segmentation, such as

- Coding parameters may also influence the determination of the residual signal and thus may be part of the residual signal 42 ₁ . For example, while the spectral transform coefficient levels output by the quantization/transform module 26 may be considered correction data, the quantization step size may also be signaled within the data stream 18, a quantization step size parameter. can be regarded as a coding parameter.

- Coding parameters may further define prediction parameters defining the two-step prediction of the prediction residual of the first prediction step discussed above. Intra/inter prediction may be used in this regard.

In order to increase the coding efficiency, the encoder 20 includes a coding information exchange module 52 that receives all coding parameters and additional information, such additional information being sent to the coding information exchange module 52 from each module, for example. ) affects or is affected by processing within modules 36, 38 and 32, as exemplarily indicated by the vertically expanding arrows pointing to . Coding information exchange module 52 is responsible for sharing coding parameters and optionally additional coding information between coding branches 22 so that the branches can predict or adopt coding parameters from each other. In the embodiment of FIG. 1 , an order is defined for this among the data entities of the views 12 ₁ and 12 ₂ of the multiview signal 10 , namely video and depth/disparity map data. In particular, the video 14 ₁ of the first view 12 ₁ precedes the depth/disparity map data 16 ₁ of the first view, followed by the video 14 ₂ , then the second view 12 ₂ ) of depth/disparity map data (16 ₂ ), etc. Here, it should be noted that this strict order among the data entities of the multi-view signal 10 need not be strictly applied for the encoding of the entire multi-view signal 10, but for easier discussion, in the following, this order is constant. it is estimated The order between the data entities naturally also defines the order between the branches 22 associated with them.

As already mentioned above, further coding branches 22 such as coding branches 22 _d,1 , 22 _v,2 and 22 _d,2 respectively encode respective inputs 16 ₁ , 14 ₂ and 16 ₂ . It acts similarly to the coding branch (22 _v,1 ) to However, due to the order just mentioned between the video and depth/disparity map data of views 12 ₁ and 12 ₂ , respectively, and the corresponding order defined between coding branch 22 , coding branch 22 _d,1 eg has additional freedom in predicting the coding parameters used to encode the current portion of the depth/disparity map data 16 ₁ of the first view 12 ₁ . This is due to the aforementioned ordering between the video and depth/disparity map data of the different views: for example, each of these entities not only uses its own reconstructed part that precedes the specific ordering between these data entities, It is allowed to be encoded using its entity. Thus, when encoding the depth/disparity map data 16 ₁ , the coding branch 22 _d,1 is allowed to use known information from the previously reconstructed portion of the corresponding video 14 ₁ . Branch 22 _d,1 of video 14 ₁ is used to predict certain properties of depth/disparity map data 16 ₁ that enable a better compression rate of compression of depth/disparity map data 16 ₁ . The method of using the reconstructed part is theoretically not limited. In order to obtain coding parameters for encoding the depth/disparity map data 16 ₁ , the coding branch 22 _d,1 , for example, uses coding parameters included when encoding the video 14 ₁ as described above. Can predict/recruit. In case of adoption, the signaling of certain coding parameters relating to the depth/disparity map data 16 ₁ within the data stream 18 may be suppressed. In the case of prediction, only prediction residual/correction data relating to these coding parameters may need to be signaled within the data stream 18 . Examples of such prediction/adoption of coding parameters are described further below.

Notably, in addition to the modes described above for modules 36 and 38, coding branch 22 _d,1 may have additional coding modes available for the code blocks of depth/disparity map data 16 ₁ . These additional coding modes are described further below and relate to irregular block partitioning modes. In an alternative view, an irregular division as described below can be viewed as a series of subdivisions of the depth/disparity map into blocks/divisions.

In any case, additional predictive capabilities are provided for subsequent data entities, namely the video 14 ₂ of the second view 12 ₂ and the depth/disparity map data 16 ₂ . For this coding branch, its inter-prediction module may not only perform temporal prediction, but also inter-view prediction. Corresponding inter-prediction parameters are temporal predictions, i.e. interview-predicted per-segment predictions, disparity vectors, view indices, reference frame indices and/or indications of multiple hypotheses, i.e., summation, to form interview inter-prediction. It contains similar information compared to the indication of the number of participating inter predictions. These interview predictions are available to the inter prediction module 38 of branch 22 _v,2 for video 14 ₂ , as well as branch 22 _d,2 for depth/disparity map data 16 ₂ . . Of course, these interview prediction parameters also represent coding parameters that can serve as a basis for adoption/prediction for subsequent view data of a possible third view not shown in FIG. 1 .

Due to the above measurements, the amount of data inserted into data stream 18 by multiplexer 44 is even smaller. In particular, the amount of coding parameters of the coding branches 22 _d,1 , 22 _v,2 and 22 _d,2 adopt the coding parameters of the previous coding branch, or just predict residuals for them as data via the multiplexer 44. It can be greatly reduced by inserting into stream 28. Due to the ability to choose between temporal and inter-view predictions, the amount of residual data 42 ₃ and 42 ₄ in the coding branches 22 _v,2 and 22 _d,2 is small. The reduction in the amount of residual data overcompensates for the additional coding effort in distinguishing between temporal and inter-view prediction modes.

To explain the principle of coding parameter adoption/prediction in more detail, reference is made to FIG. 2 . 2 shows an exemplary portion of the multiview signal 10 . 2 shows a video frame v _1,t divided into segments or portions 60a, 60b and 60c. For simplification reasons, only three parts of the frame (v _1,t ) are shown, although segmentation can evenly and seamlessly divide a frame into segments/parts. As mentioned above, the segmentation of the video frame (v _1,t ) can be fixed or temporally variable, and the segmentation can be signaled or not signaled within the data stream. FIG. 2 shows temporal analysis using motion vectors 62a and 62b from a reconstructed version of a reference frame of video 14 ₁ , which is an example frame v _1,t-1 when portions 60a and 60b view it. shows what is predicted by As is known in the art, the coding order between frames of video 14 ₁ may not match the presentation order between such frames, so the reference frame is in presentation temporal order 64. It can follow the current frame (v _1,t ). Portion 60c is an intra-predicted portion where, for example, intra-prediction parameters are inserted into the data stream 18 .

When encoding the depth/disparity map d _1,t , the coding branch 22 _d,1 may utilize the above-described possibilities in one or more of the ways below illustrated next with respect to FIG. 2 .

- For example, when encoding the depth/disparity map (d _1,t ), the coding branch (22 _d,1 ) is the video frame (v _{1 ,} as used by coding branch (22 _v,1 ). _t ) can be employed. Therefore, if there is a segmentation parameter in the coding parameter for the video frame (v _1,t ), its retransmission for the depth/disparity map data (d _1,t ) can be avoided. Alternatively, coding branch 22 _d,1 signals the segmentation deviation for video frame v _1,t via data stream 18 to determine the segmentation to be used for depth/disparity map d _1,t . Segmentation of the video frame (v _1,t ) can be used as a basis/prediction for 2 shows the case where the coding branch 22 _d,1 uses the segmentation of the video frame v _1,t as the pre-segmentation of the depth/disparity map d _1,t . That is, the coding branch 22 _d,1 adopts or predicts a pre-segmentation from the segmentation of the video frame v _1,t .

- Furthermore, the coding branch 22 _d,1 is a portion of the depth/disparity map d _{1,t (} The coding modes of 66a, 66b and 66c) can be adopted or predicted. In the case of different segmentation between video frame (v _1,t ) and depth/disparity map (d _1,t ), the adoption/prediction of the coding mode from video frame (v _1,t ) is It can be controlled to be obtained from the co-located part of the segmentation of (v _1,t ). A suitable definition of co-location may be: The co-location portion of the video frame (v _1,t ) to the current portion of the depth/disparity map (d _1,t ) is, for example, the upper left corner of the current frame of the depth/disparity map (d _1,t ). It may include co-located locations. In the case of coding mode prediction, the coding branch 22 _d,1 is the depth/disparity map d _1,t for the coding mode within the video frame v _1,t explicitly signaled in the data stream 18. The coding mode deviation of parts 66a to 66c of ) may be signaled.

- as far as prediction parameters are concerned, the coding branch 22 _d,1 spatially employs or predicts, or predicts, the prediction parameters used to encode neighboring portions within the same depth/disparity map d _1,t ; , has the freedom to adopt/predict from the prediction parameters used to encode the co-located portions 60a to 60c of the video frame v _1,t . For example, in FIG. 2 , the portion 66a of the depth/disparity map d _1,t is an inter-predicted portion, and the corresponding motion vector 68a is the co-located portion of the video frame v _1,t ( 60a) can be employed or predicted from the motion vector 62a. In the case of prediction, only the motion vector difference has to be inserted into the data stream 18 as part of the inter prediction parameters 48 ₂ .

- From the point of view of coding efficiency, it may be advantageous for the coding branch 22 _d,1 to have the ability to subdivide the segments of the pre-segmentation of the depth/disparity map d _1,t using non-uniform block partitioning. Some non-uniform block segmentation modes, referring to embodiments described further below, derive segmentation information such as the wedgelet separation line 70 from the reconstructed picture (v _1,t ) of the same view. By these measures, the blocks of the pre-segmentation of the depth/disparity map (d _1,t ) are subdivided. For example, block 66c of the depth/disparity map d _1,t is subdivided into two wedgelet-shaped partitions 72a and 72b. Coding branch 22 _d,1 may be configured to encode these subsegments 72a and 72b. In the case of Fig. 2, subsegments 72a and 72b are both illustratively shown to be inter predicted using respective motion vectors 68c and 68d. According to sections 3 and 4, the coding branch 22 _d,1 may have the freedom to choose between a number of coding options for irregular block partitioning and to signal the choice as side information to the decoder within the data stream 18. have.

Upon encoding video 14 ₂ , coding branch 22 _v,2 has the option of interview prediction in addition to the coding mode options available for coding branch 22 _v,1 .

FIG. 2 shows, for example, that the segmentation 64b of the video frame v _2,t uses the disparity vector 76 to temporally correspond to the video frame v _1,t of the first-view video 14 ₁ . ) shows what the interview predicts from.

Despite this difference, the coding branch 22 _v,2 additionally includes all information available from the encoding of the video frame v _1,t and the depth/disparity map d _1,t , in particular the coding used for this encoding. Information such as parameters can be used. Thus, the coding branch 22 _v,2 corresponds to the motion vectors 62a and 62a of the co-located parts 60a and 66a of the temporally assigned video frame v _1,t and the depth/disparity map d _1,t , respectively. 68a) may adopt or predict motion parameters including motion vectors 78 for the temporally inter-predicted portion 74a of the video frame v _2,t from any one or combination thereof. In any case, the prediction residual may be signaled for the inter prediction parameters for portion 74a. At this point, it should be recalled that motion vector 68a has already been predicted/adopted from motion vector 62a itself.

Another possibility to employ/predict coding parameters for encoding the video frame (v _2,t ) as described above in relation to the encoding of the depth/disparity map (d _1,t ) is also the coding branch (22 _v,2 ) Applicable to the encoding of a video frame (v _2,t ) by V , but the available common data varied by module 52 is a video frame (v _1,t ) and a corresponding depth/disparity map (d _1,t ). ) is increased because both coding parameters of ) are available.

Coding branch 22 _d,2 then encodes the depth/disparity map d _2,t similar to the encoding of depth/disparity map d _1,t by coding branch 22 _d,1 . This is true for all cases of employing/predicting coding parameters from video frame v _2,t eg of the same view 12 ₂ . However, additionally, the coding branch 22 _d,2 has the opportunity to also adopt/predict coding parameters from the coding parameters used to encode the depth/disparity map d _1,t of the previous view 12 ₁ . have Additionally, coding branch 22 _d,2 may use interview prediction as described for coding branch 22 _v,2 .

After describing the encoder 20 of FIG. 1, it should be noted that it may be implemented in software, hardware or firmware, ie programmable hardware. Although the block diagram of FIG. 1 suggests that encoder 20 structurally includes parallel coding branches, i.e., one coding branch per video and depth/disparity data of multiview signal 10, this need not be the case. . For example, a software routine, portion of circuitry configured to perform the task of elements 24-40, or portion of programmable logic may be used to sequentially perform the task for each of the respective coding branches. In parallel processing, the processes of parallel coding branches may be performed in parallel processor cores or parallel execution circuits.

FIG. 3 shows an example of a decoder capable of decoding a data stream 18 to reconstruct one or more view video corresponding to a scene represented by a multiview signal from the data stream 18 . In general, the structure and function of the decoder of FIG. 3 is similar to the encoder of FIG. 2 such that the reference numerals in FIG. 1 are reused as far as possible to indicate that the functional description provided above for FIG. 1 also applies to FIG. 3 .

The decoder of FIG. 3 is indicated generally by reference numeral 100 and includes an input to a data stream 18 and an output outputting a reconstruction of one or more views 102 described above. The decoder 100 includes a demultiplexer 104 and a pair of decoding branches 106 for each of the data entities of the multiview signal 10 (FIG. 1) represented by the data stream 18 as well as a view extractor 108. and a coding parameter exchanger (110). As with the encoder of FIG. 1 , the decoding branch 106 includes the same decoding elements of the same interconnection, and thus is typically responsible for decoding the video 14 ₁ of the first view 12 ₁ . The decoding branch 106 _v,1 is described. In particular, _each coding branch 106 has _a multiplexer ( 104) and an output connected to each input of view extractor 108. In the middle, each coding branch 106 includes an inverse quantization/inverse transform module 28, an adder 30, a further processing module 32, and a decoding connected in series between the multiplexer 104 and the view extractor 108. and a picture buffer 34. The adder 30, the further processing module 32 and the decoded picture buffer 34 form a loop together with the parallel connection of the prediction modules 36 and 38 and then, in the order mentioned, the decoded picture buffer 34 and a further input of the adder 30 forms a combiner/selector 40. As indicated using the same reference numerals as in the case of FIG. 1 , the structure and function of the elements 28 to 40 of the decoding branch 106 indicate that the elements of the decoding branch 106 are information conveyed within the data stream 18. It is similar to the corresponding element of the coding branch of FIG. 1 in that it emulates the processing of the coding process using . Of course, while decoding branch 106 only inverts the coding procedure for the coding parameters finally selected by encoder 20, encoder 20 of FIG. /need to find an optimal set of coding parameters in some optimization sense, such as coding parameters that optimize the distortion cost function.

The demultiplexer 104 is for varying the data stream 18 to the various decoding branches 106. For example, demultiplexer 104 provides residual data 42 ₁ to inverse quantization/inverse transform module 28, additional processing parameters 50 ₁ to further processing module 32, and intra prediction. Module 36 is provided with intra prediction parameters 46 ₁ , and inter prediction module 38 is provided with inter prediction parameters 48 ₁ . The coding parameter exchanger 110 acts like the corresponding module 52 of FIG. 1 to vary common coding parameters and other common data between the various decoding branches 106.

The view extractor 108 receives the multi-view signal reconstructed by the parallel decoding branch 106 and from it receives one or more view angles or view directions defined by the externally provided intermediate view extraction control data 112. Extract the view (102).

Due to the similar configuration of the decoder 100 to the corresponding part of the encoder 20, the functions or interfaces for the view extractor 108 are easily described similarly to those described above.

In fact, the coding branches 106 _v,1 and 106 _d,1 are the scaling parameters in (42 ₁ , parameters 46 ₁ , 48 ₁ , 50 ₁ ), and the coding parameters of the second branch 16 _d,1 . First view 12 ₁ , according to a first coding parameter included in the data stream 18 , such as the corresponding unadopted and prediction residual, i.e. 42 ₂ , parameters 46 ₂ , 48 ₂ , 50 ₂ . predicts the current part of the first view 12 ₁ from the previously reconstructed part of the multiview signal 10 reconstructed from the data stream 18 before reconstruction of the current part of the The first view 12 1 of the multiview signal 10 from the data stream 18 by correcting the prediction error of the prediction of the current portion of the first view 12 ₁ using the _first correction data in 42 ₁ and 42 ₂ ) work together to reconstruct the While coding branch 106 _v,1 is responsible for decoding video 14 ₁ , coding branch 106 _d,1 assumes responsibility for reconstructing depth/disparity map data 16 ₁ . See Fig. 2 for example: the coding branch 106 _v,1 is the corresponding coding parameter read from the data stream 18, namely the scaling parameter in 42 ₁ , parameters 46 ₁ , 48 ₁ , 50 ₁ predicting the current portion of the video 14 ₁ , such as 60a, 60b or 60c, from a previously reconstructed portion of the multiview signal 10 according to , obtained from the transform coefficient levels in the data stream 18, i.e. 42 ₁ The video 14 ₁ of the first view 12 ₁ is reconstructed from the data stream 18 by correcting the prediction error of this prediction using the corresponding correction data. For example, the coding branch 106 _v,1 can use the coding order among video frames and, to code the segments within a frame, the coding order among the segments of those frames as the corresponding coding branch of the encoder does. The video 14 ₁ is processed in units of . Thus, all previously reconstructed parts of the video 14 ₁ are available for prediction on the current part. Coding parameters for the current portion may include one or more of an intra prediction parameter 50 ₁ , an inter prediction parameter 48 ₁ , a filter parameter for the further processing module 32 , and the like. Correction data for correcting the prediction error may be represented by a specific conversion coefficient level in the residual data 42 ₁ . All of these coding parameters need not be transmitted in their entirety. Some of these could have been spatially predicted from coding parameters of neighboring segments of video 14 ₁ . A motion vector for video 14 ₁ may be transmitted in the bitstream as a motion vector difference between motion vectors of neighboring parts/segments of video 14 ₁ , for example.

As far as the second decoding branch 106 _d,1 is concerned, it is signaled in the residual data 42 ₂ , and in the data stream 18, to each decoding branch 106 _d,1 by the demultiplexer 104. Provided to decoding branch 106 _v,1 via demultiplexer 104 when variance via coding information exchange module 110, as well as corresponding prediction and filter parameters, i.e., coding parameters that are not predicted across interview boundaries. Indirectly access coding parameters and correction data or any information deduced from them. The decoding branch 106 _d,1 is a depth/disparity from the portion of the coding parameters transmitted through the demultiplexer 104 as a pair of decoding branches 106 _v,1 and 106 _d,1 for the first view 12 ₁ . Determines the coding parameters for reconstructing the map data 16 ₁ , which partly overlaps the part of these coding parameters transmitted and dedicated specifically to the decoding branch 106 _v,1 . For example, the decoding branch 106 _d,1 can be defined as, for example, the motion vector difference to another neighboring part of the frame v _1,t on the one hand and the motion vector difference transmitted explicitly within 48 ₂ on the other hand. Determine motion vector 68a from motion vector 62a transmitted explicitly within 48 ₁ . Additionally, or alternatively, decoding branch 106 _d,1 is described above for prediction of wedgelet separation lines to derive irregular block partitioning as briefly described above with respect to decoding depth/disparity map data 16 ₁ . As done, we can use the reconstructed portion of video 14 ₁ , which will be explained in more detail below.

More precisely, the decoding branch 106 _d,1 is predicted at least in part from (or is employed from) the coding parameters used by the decoding branch 106 _v,1 , and/or the decoding branch 106 _v, Reconstruct the depth/disparity map data 16 ₁ of the first view 12 ₁ from the data stream using coding parameters predicted from the reconstructed portion of the video 14 ₁ in the decoded picture buffer 34 of ₁ ) do. Prediction residuals of coding parameters may be obtained via demultiplexer 104 from data stream 18 . Other coding parameters for the decoding branch 106 _d,1 are all or on another basis, ie with reference to coding parameters used to code some previously reconstructed portion of the depth/disparity map data 16 ₁ itself. can be transmitted in the data stream 18. Based on these coding parameters, decoding branch 106 _d,1 is the depth that is reconstructed from data stream 18 by decoding branch 106 _d,1 before reconstruction of the current portion of depth/disparity map data 16 ₁ . / Predict the current portion of the depth/disparity map data (16 ₁ ) from the previously reconstructed portion of the disparity map data (16 ₁ ), and use the respective correction data (42 ₂ ) to estimate the depth/disparity map data (16 ₁ ). ) corrects the prediction error of the prediction of the current part.

The functionality of the pair of decoding branches 106 _v,2 and 106 _d,2 for the second view 12 ₂ is similar to that for the first view 12 ₁ as already described above for encoding. Both branches cooperate to reconstruct the second view 12 ₂ of the multiview signal 10 from the data stream 18 using their coding parameters. Only a portion of these coding parameters needs to be transmitted through the demultiplexer 104 to either of these two decoding branches 106 _v,2 and 106 _d,2 , and to be varied, either of which is the view 12 ₁ and 12 ₂ ), and not employed/predicted across the remainder of the interview prediction portion. The current part of the second view 12 ₂ is the previous part of the multiview signal 10 reconstructed from the data stream 18 by some decoding branch 106 before reconstruction of the respective current part of the second view 12 ₂ . Prediction error is calculated using the correction data, i.e., 42 ₃ and 42 ₄ , which is predicted from the reconstructed part in , and thus transmitted by the demultiplexer 104 as a pair of these decoding branches 106 _v,2 and 106 _d,2 . correct

Decoding branch 106 _{d,2 is} derived from decoding branch 106 _{v,1 ,} 106 _{d , 1} and 106 _v,2 ) can finally determine the coding parameters by adoption/prediction from the coding parameters used by either. For example, data stream 18 may include coding parameters for current portion 80b and any portion thereof of video 14 ₁ , depth/disparity map data 16 ₁ , and video 14 ₂ , or a suitable subset thereof. It is possible to signal about the current portion 80b of the depth/disparity map data 16 ₂ which one of the co-located portions is or should be adopted. The portion of interest of these coding parameters may include, for example, a motion vector, such as 84, or a disparity vector, such as disparity vector 82. Moreover, other coding parameters, such as for irregularly partitioned blocks, can be derived by the decoding branch 106 _d,2 .

In any case, the reconstructed portion of the multiview data 10 arrives at the view extractor 108, the view contained within the view extractor 108 extracts a new view, i.e. a view of the video associated with this new view, for example. becomes the basis for This view extraction may include reprojection of videos 14 ₁ and 14 ₂ using the depth/disparity map data associated therewith. Frankly speaking, when reprojecting the video into another intermediate view, the portion of the video corresponding to the portion of the scene located closer to the viewer has a transition beyond the portion of the video corresponding to the portion of the scene located further away from the viewer's position. It is shifted along the direction, that is, the direction of the view direction difference vector.

It should be mentioned that the decoder does not necessarily include the view extractor 108 . Rather, the view extractor 108 may not be provided. In this case, the decoder 100 is only for reconstructing any one of the views 12 ₁ and 12 ₂ , such as one, several or all of the views 12 ₁ and 12 ₂ . Even if depth/disparity data does not exist for individual views 12 ₁ and 12 ₂ , view extractor 108 performs intermediate view extraction by using disparity vectors related to corresponding parts of neighboring views relative to each other. can do. Using this disparity vector when supporting the disparity vector of the disparity vector field associated with the video of the neighboring view, view extractor 108 applies this disparity vector field to extract the disparity vector from this video of neighboring views 12 ₁ and 12 ₂ . You can build a mid-view video. For example, imagine that video frame (v _2,t ) has 50% of the predicted partial/segment interviews. That is, for 50% of parts/segments, disparity vectors exist. For the remaining parts, the disparity vector may be determined by the view extractor 108 through interpolation/extrapolation in a spatial sense. Temporal interpolation using disparity vectors for portions/segments of previously reconstructed frames of video 14 ₂ may also be used. The video frame (v _2,t ) and/or the reference video frame (v _1,t ) may then be distorted according to these disparity vectors to create an intermediate view. To this end, the disparity vector is scaled according to the intermediate view position of the intermediate view between the view positions of the first view 12 ₁ and the second view 12 ₂ . Details regarding these procedures are described in more detail below.

However, the embodiment described below is advantageously shown in FIGS. 1 to 3 when considering coding of only one view comprising video and corresponding depth/disparity map data, such as the first view 12 ₁ of the above-described embodiment. can be used in the framework of In that case, the information of the transmitted signal, that is, the single view 12 ₁ , may be referred to as a view synthesis compliant signal, that is, a signal enabling view synthesis. Accompanying video 14 ₁ with depth/disparity map data 16 ₁ causes view extractor 108 to use depth/disparity map data 16 ₁ to move view 12 ₁ to a new neighboring view. Reprojection allows you to perform a kind of view synthesis. In other words, coding efficiency gains are obtained using irregular block partitioning. Thus, the embodiment of irregular block partitioning described further below can be used within a single view coding concept independent of the above-described interview coding information exchange aspect. More precisely, the above embodiment of FIGS. 1 to 3 can be varied to such an extent that the branch 22 , 100 _v/d,2 and the associated view 12 ₂ disappear.

Thus, Figures 1 to 3 illustrate an example of a multiview coding concept in which the successively described irregular block partitioning can be advantageously used. However, it is emphasized again that the coding modes described below may also be used in conjunction with other types of sample array coding, whether or not the sample array is a depth/disparity map. Some of the coding modes described below do not require the coexistence of depth/disparity maps with corresponding texture maps.

In particular, the embodiments described below include several coding modes in which a signal in a block is represented by a model that separates samples of the signal into two sets of samples and represents each set of samples with a constant sample value. Some of the coding modes described below can be used to directly represent the signal of a block, or to generate a prediction signal for a block that is further refined by coding additional residual information (e.g., transform coefficient levels). can be used Advantages may result from the fact that, in addition to other preferred aspects, the depth signal is mainly characterized by slowly changing regions and corners between slowly changing regions, when one of the coding modes described in succession is applied to the depth signal. have. Although slowly changing regions can be represented efficiently by a transform coding approach (based on the DCT), the representation of an edge between two nearly constant regions requires a large number of transform coefficients to be coded. Such blocks containing corners can be better represented using a model that divides the block into two regions, each of which has a constant sample value, as described for some of the embodiments described below.

In the following, several embodiments of the present invention are described in more detail. In Sections 1 and 2, the basic concept for dividing a block into two regions of constant sample values is explained. Section 3 describes several embodiments for assigning how a block can be divided into different regions and what parameters need to be transmitted to indicate the division as well as the sample values for the regions.

The embodiment signals splitting information independently of any other block, signals splitting information based on data transmitted for a spatially neighboring block, and already transmitted texture pictures related to the depth map to be coded (conventional It includes a concept for signaling segmentation information based on a video picture). Accordingly, Section 4 describes embodiments of the present invention in terms of coding of mode information, segmentation information, and constant sample values related to some embodiments for processing irregularly located blocks.

Although the following description primarily targets coding of depth maps (particularly in the context of multiview video coding), and the following description is based on a given depth block, some embodiments of the present invention may also be applied to conventional video coding. have. Thus, if the term "depth block" is replaced with the generic term "signal block", the description may be applied to other signal types. Moreover, although the following description sometimes focuses on quadratic blocks, the present invention can also be applied to rectangular blocks or other concatenated or simply concatenated sets of samples.

1. Wedgelet

For example, in block-based hybrid video coding as shown in FIGS. 1-3, a frame is subdivided into rectangular blocks. Often these blocks are secondary, and the processing of each block follows the same functional structure. Although most of the examples in this section use quadratic blocks, the Wedgelet block partitioning and all related methods are not limited to quadratic blocks, but rather are capable of any rectangular block size.

1.1 Wedgelet block division

The basic principle of Wedgelet block division is to divide the region of block 200 into two regions 202a and 202b separated by line 201, as shown in Fig. 4, where the two regions are P ₁ and P . ₂ is labeled. The separation line is determined by the starting point S and the ending point E, both located at block boundaries. Sometimes, in the following, region P ₁ is referred to as Wedgelet partition 202a, while region P ₂ is referred to as Wedgelet partition 202b.

이고, 종료점 위치는

이며, 둘 다 블록 크기

및

로 제한된다(여기서 좌표 중 하나는 최소 값(0) 또는 최대 값(

또는

)과 동일해야 한다). 이러한 정의에 따르면, 분리 라인의 방정식은 다음과 같다:For continuous signal space (see left in Fig. 4), the starting point location is

, and the endpoint position is

, both block size

and

(where one of the coordinates is the minimum value (0) or maximum value (

or

) should be the same as). According to this definition, the equation of the separation line is:

(1)

(One)

에 대해서만 유효하다는 것을 주목한다. 그리고 나서, 두 영역(P₁ 및 P₂)은 제각기 라인의 왼쪽 영역 및 오른쪽 영역으로 정의된다.These equations are

Note that it is only valid for Then, the two regions P ₁ and P ₂ are defined as the left region and the right region of the line, respectively.

및

와의 블록(200)의 경계 샘플에 대응하며, 둘 다 블록 크기

및

로 제한된다. 이산 경우에, 분리 라인 방정식은 (1)에 따라 공식화될 수 있다. 그러나, 완전한 샘플만이 도 4의 오른쪽에 도시된 두 영역 중 하나의 부분으로서 할당될 수 있을 때에 영역(P₁ 및 P₂)의 정의는 여기서 서로 다르다. 이러한 할당 문제는 섹션 1.4.1에 설명된 바와 같이 알고리즘식으로 해결될 수 있다.In digital image processing, a discrete signal space (see right side of Fig. 4) is usually used, where a block consists of an integer number of samples 203 depicted as a grid square. where the start and end points (S and E) are the positions

and

and correspond to the boundary samples of block 200, both of which block size

and

is limited to In the discrete case, the separation line equation can be formulated according to (1). However, the definitions of regions P ₁ and P ₂ are different here as only complete samples can be assigned as part of one of the two regions shown on the right side of FIG. 4 . This allocation problem can be solved algorithmically as described in Section 1.4.1.

Wedgelet block divisions 202a and 202b require start and end points 204 to be located on different edges of block 200 . As a result, six different directions of wedgelet block divisions 202a and 202b can be distinguished for a rectangular or quadratic block 200 as shown in FIG.

1.2 Wedgelet Split Pattern

의 어레이로 구성되며, 각 요소는 부합하는(according) 샘플이 영역(P₁ 또는 P₂)에 속하는지를 판단하는 이진 정보를 포함한다. 도 6은 서로 다른 블록 크기에 대한 예시적인 웨지렛 분할 패턴을 도시한다. 여기서, 이진 영역 정보, 즉 이중 세그먼테이션은 흑색 또는 흰색 샘플(203)로 나타낸다.When Wedgelet block division is used in the coding process, division information may be stored in the form of a division pattern. These patterns are

It consists of an array of, and each element contains binary information for determining whether the matching sample belongs to the region (P ₁ or P ₂ ). Figure 6 shows an exemplary Wedgelet partitioning pattern for different block sizes. Here, binary region information, i.e. double segmentation, is represented by black or white samples 203.

1.3 Wedgelet Modeling and Approximation

In order to model the depth signal of a block with a wedgelet, the necessary information conceptually consists of two elements. One is segmentation information, for example in the form of a segmentation pattern (see Section 1.1), which assigns each sample 203 to one of two regions (see Section 1.2). Another required information element is the value assigned to the samples of the region. Each value of the two Wedgelet regions can be defined as being a constant. This is the case according to some of the embodiments described below. Accordingly, this value will be referred to as the constant division value (CPV). In that case, the second information element consists of two representative sample values for the assigned region.

As shown in Fig. 7, when approximating the signal of a depth block by Wedgelet, the CPV of a given division can be calculated as the average value of the original depth signal of the corresponding region. On the left of Fig. 7, a gray-scaled portion of an exemplary depth map is shown. Block 200, which is currently the subject of Wedgelet-based partitioning, is shown as an example. In particular, an example location within the original depth signal 205 is shown, as well as an expanded grayscale version. First, segmentation information in terms of regions P ₁ and P ₂ , ie possible double segmentation, is overlapped with block 200 . Then, the CPV of one region is calculated as an average value of all samples covered by each region. When the segmentation information in the example of FIG. 5 matches the depth signal 205 very well, the generated Wedgelet model is the lower CPV (dark gray) for region P ₁ and the upper CPV for region P ₂ . The prediction of block 200 based on the Wedgelet partitioning mode described by CPV (light gray) represents a good approximation of the depth block.

1.4 Wedgelet Processing

1.4.1 List of Wedgelet Patterns

For efficient processing and signaling of Wedgelet block partitioning, partitioning patterns can be constructed in a lookup list. This Wedgelet pattern list contains patterns for all possible combinations of start and end positions for region separation lines, or contains a suitable subset of all possible combinations. Thus, one lookup list can be created for each prediction block size. The same list can be made available at the encoder and decoder to enable signaling between the encoder and decoder that depends on the position or index of a particular pattern within the list of some block size (see Section 3 for details). This can be implemented by including a predefined set of patterns or by running the same generation algorithm as part of encoder and decoder initialization.

어레이)과 시작점 S 및 종료점 E 좌표(도 8 왼쪽)가 주어지면, 제 1 단계는 분리 라인을 그리는 것이다. 이를 위해, Bresenham 라인 알고리즘이 적용될 수 있다. 일반적으로, 알고리즘은 주어진 두 지점 사이의 직선에 대한 근사치를 형성하기 위해 샘플(203)이 도시되어야 하는지를 판단한다. 웨지렛 분할 패턴의 경우에, 시작점 S 과 종료점 E 사이의 선의 근사치를 내는 모든 요소(203)가 표시된다( 도 8의 흑색 박스, 중간 왼쪽). 마지막 단계는 표시된 샘플로 분리되는 생성된 두 영역 중 하나를 필링(filling)하는 것이다. 여기서, 상술한 할당 문제가 논의될 필요가 있다. 패턴의 요소가 이진수일 때, Bresenham 알고리즘에 의해 표시되는 분리 라인은 하나의 영역의 일부가 된다. 직관적으로, 이것은 라인 샘플이 이론적으로 두 도메인의 일부일 때 불균형이 될 것으로 보인다. 그러나, 일반성의 손실없이 하나의 영역에 분리 라인 샘플을 할당하는 것이 가능하다. 이것은 라인 표시 및 영역 필링 알고리즘이 둘 다 부합하는 방향의 루트 코너(root corner)에 대해 방향을 인식한다는 사실에 의해 보장된다. 코너 영역이 분리 라인에 의해 완전히 구분된다는 사실에 기초하여, 이러한 영역을 필링하는 것은 비교적 간단하다. 필링 알고리즘은 루트 코너 요소(206)로 시작하고, 그것이 이미 표시된 요소 및 라인(207)의 부분에 도달할 때까지(도 8 중간 오른쪽 참조) 연속으로 모든 패턴 요소 칼럼 및 라인 방향으로 표시한다. 결과적으로, 주어진 시작점 및 종료점 위치에 대한 웨지렛 분할 패턴은 이진 값으로 나타낸다(도 8 오른쪽).As shown in FIG. 8, a key function for generating a Wedgelet partitioning pattern lookup list is generating one list element. This may be realized as described below (or by a similar algorithm). An empty pattern (of binary elements)

array) and the start point S and end point E coordinates (Fig. 8 left), the first step is to draw the separation line. For this, the Bresenham line algorithm can be applied. In general, the algorithm determines whether a sample 203 should be drawn to form an approximation to a straight line between two given points. In the case of the Wedgelet division pattern, all elements 203 approximating the line between the starting point S and the ending point E are indicated (black box in Fig. 8, middle left). The final step is to fill one of the two resulting regions separated by the marked sample. Here, the aforementioned allocation problem needs to be discussed. When the elements of a pattern are binary, the separation lines represented by Bresenham's algorithm become part of one region. Intuitively, this seems to be imbalanced when the line samples are theoretically part of both domains. However, it is possible to assign separate line samples to one region without loss of generality. This is ensured by the fact that both the line marking and area filling algorithms are orientation aware with respect to the root corner of the matching orientation. Based on the fact that the corner area is completely demarcated by the separation line, filling this area is relatively straightforward. The filling algorithm starts with the root corner element 206 and marks all the pattern elements column and line directions in succession until it reaches the portion of already marked elements and lines 207 (see Fig. 8 middle right). As a result, the Wedgelet split pattern for a given start and end point location is represented by a binary value (Fig. 8 right).

The generation process for a Wedgelet partitioning pattern lookup list of any block size creates list elements for possible line start and end positions. This is realized by iterating through the six directions shown in FIG. 5 . For each direction, the starting location is located at one edge of the block, the ending location is located at the other edge of the block, and the list generation process is the same as the Wedgelet pattern generation method introduced above for each possible combination of starting and ending locations. run For efficient processing and signaling, the Wedgelet pattern list should contain only unique patterns. So, before a new pattern is added to the list, it is checked for being identical or inverse identical to any pattern already in the list. In this case, the pattern is redundant and is therefore discarded. Besides that, flat patterns, i.e. all samples, are assigned to one region and are also dropped from the list when they do not represent valid Wedgelet block divisions.

As an extension to the described Wedgelet pattern list, the resolution of the line start and end positions used to create the pattern may be adaptively increased or decreased depending on the block size, for example. The purpose of this extension is to find a good tradeoff between coding efficiency and complexity. Compared to normal resolution, increasing resolution leads to lists with more patterns, while decreasing resolution results in shorter lists. As a result, resolution is typically increased for small block sizes and decreased for large block sizes. It is important to note that regardless of the resolution for the start and end positions, the Wedgelet partitioning pattern stored in the list must always have the normal resolution, i.e. the original block size. Reducing the resolution can be realized simply by generating a pattern as described above, but only for a subset of start and end positions. For example, half the resolution means limiting pattern generation to every second start and end position. In contrast, increasing the resolution is more difficult. To cover all start and end positions, a temporary pattern with increased resolution is first created using the algorithm described above. In a second step, the generated pattern is downsampled to normal resolution. Note that for binary data, downsampling does not support interpolated values that produce many identical patterns for the case of increased resolution.

As an end result of the Wedgelet pattern generation described above, an ordered list of Wedgelet patterns is derived on both the encoder and decoder sides. In actual implementation, these patterns may also be predefined by the coding algorithm/coding standard used. Moreover, it is not necessary to generate patterns by the actual algorithms described above, and modifications of these algorithms can also be used. It only matters that both the encoder and decoder generated the same list of Wedgelet patterns for the encoding and decoding processes (used later).

1.4.2 Minimum Distortion Wedgelet Search

Based on the above lookup list, the best approximation of the signal of the block by Wedgelet division can be found by the search algorithm. For Wedgelet-based coding algorithms, the best approximation can be understood as the Wedgelet model that induces the least distortion. In other words, the search tries to find the Wedgelet split pattern that best matches the given block. The search uses a derived pattern list containing all possible Wedgelet splitting patterns for a given block size (see Section 1.4.1 for details). This list helps limit the processing time of the search when the pattern does not need to be regenerated every time the least distortion Wedgelet search is run. Each search step may consist of the following steps:

주어진 분할 패턴 및 원래의 블록 신호로부터의 CPV 값의 계산.

Calculation of CPV values from given splitting pattern and original block signal.

원래의 블록 신호와 웨지렛 모델 사이의 왜곡

의 계산.

Distortion between the original block signal and the Wedgelet model

calculation of.

의 평가: 참(true)일 경우,

를 설정하고, 현재 분할 패턴의 리스트 인덱스를 저장함으로써 최소 왜곡 웨지렛 정보를 업데이트한다.

evaluation of: if true,

, and update the minimum distortion Wedgelet information by storing the list index of the current split pattern.

를 제공한 관련된 파라미터를 전송하기 위해 필요한 레이트 R을 가진 특정 웨지렛 패턴에 의해 얻어진 왜곡 D에 가중치를 주는 가중 합

이다.Instead of distortion, a Lagrangian cost measure can be used to find the Wedgelet pattern used. The Lagrangian const measure is the Lagrangian multiplier

A weighted sum weighting the distortion D obtained by a particular Wedgelet pattern with a rate R required to transmit the relevant parameters that provided

to be.

Several strategies are possible for the search algorithm, from exhaustive to fast search strategies. An exhaustive search means that all elements of the Wedgelet pattern list are successively tested for minimal distortion. This strategy will definitely find a global minimum if not for the price of being slow (especially important for encoders). Fast retrieval refers to an improved strategy that reduces the number of retrieval steps required. A fast search strategy can be, for example, successive refinement. In a first step, a minimum distortion wedgelet is searched for a subset of segmentation patterns that results from a limited number of line start and end positions, for example only every fourth boundary sample. In the second step, the start and end positions must be refined, for example by allowing every second boundary sample, but limiting the range of tested start and end positions to the range around the best result of the first step. By refining the step size per cycle, the minimum distortion wedgelet is finally found. In contrast to full search, only this fast search strategy allows finding local minima, but significantly reduces the number of Wedgelet patterns to be tested, resulting in a fast search. Note that the step size of the first step need not be a fixed value, but can be set adaptively, for example as a function of the block size.

The just-discussed index indexing the process of a Wedgelet line or Wedgelet pattern can be referred to as wedge_full_tab_idx.

2 outline

Note that although most of the examples in this section use quadratic blocks, the contour block segmentation and all related embodiments are not limited to quadratic blocks, but rather are possible for any rectangular block size.

2.1 Split Contour Block

The basic principle of contour block segmentation is to divide the area of block 200 into two areas 202a and 202b. Unlike for Wedgelet block divisions, the separation line 201 between regions cannot be described by a geometric formulation. As shown by the two regions labeled P ₁ and P ₂ in FIG. 9 , the regions of the outline may be of any shape, and they do not even need to be connected.

Figure 9 also shows the difference between continuous and discrete signal space for contour block segmentation. In other words, only complete samples can be allocated as part of one of the two regions for the discrete signal space (Fig. 9 right). If the contour segmentation information is derived from a discrete reference signal rather than from a geometric formulation (see Section 3.2.2 for details), the same allocation problem as for Wedgelet block segmentation need not be considered here.

2.2 Contour Segmentation Pattern

의 어레이로 구성되고, 각 요소는 부합하는 샘플이 영역(P₁ 또는 P₂)에 속하는지를 판단하는 이진 정보를 포함한다. 도 10은 흑색 또는 흰색 샘플 색상으로 이진 영역 정보를 나타내는 예시적인 윤곽선 분할 패턴을 도시한다.Conforming to the Wedgelet splitting pattern (see Section 1.2), the contour block splitting information can be stored in the form of a splitting pattern. These patterns are

It consists of an array of , each element containing binary information for determining whether a matching sample belongs to the region P ₁ or P ₂ . 10 shows an exemplary contour segmentation pattern representing binary region information with a black or white sample color.

2.3 Contour Modeling and Approximation

The principle of approximating a block's depth signal with a contour is the same as the Wedgelet concept described in Section 1.3. In other words, the required information may consist of two-element splitting information and a split-filling command, which in turn may include one constant splitting value (CPV) for each of the two regions, which values correspond to It can be calculated as the average value of the original depth signal of the region.

A contour approximation is shown in Fig. 11, where the original depth signal of the prediction block 200 is shown magnified and exaggerated to show its surroundings. In other words, the partitioning information at the side of the regions P ₁ and P ₂ is first overlapped with the block, and then the CPV is calculated as an average value of all samples covered by the region. When the segmentation information in the example of FIG. 11 matches the depth signal very well, the generated contours have a low CPV (dark gray) for region P ₁ and a high CPV (light gray) for region P ₂ . The model represents a good approximation of the depth block.

3. Block segmentation coding

When using the methods and algorithms described in the previous section within a coding framework for multiview video plus depth (MVD), such as the coding environment of FIGS. 1 to 3, a new coding routine or mode must be defined, and the necessary tools are Must be implemented in the encoder and decoder.

In the case of hybrid video coders such as the encoder of Fig. 1 or coding branch pair 22 _v/d,1 , these tools can be classified as part of estimation, prediction or signaling. Estimation summarizes a tool that is only part of the encoding process when the tool relies on the original input information (eg uncompressed picture). In contrast to this, prediction encapsulates a tool that is part of the encoding and decoding process when the tool only relies on transmitted and/or reconstructed information (eg a decoded picture). Signaling encapsulates the tools for coding the information transmitted from the encoder to the decoder in a bitstream. Consequently, they are required to use the same syntax and the same entropy coding state.

For estimation tools, the difference between the block's distortion and the original depth signal is measured, for example, as the mean squared error (MSE) or mean absolute difference (MAD), or the block's distorted depth signal causes it to be displaced. Distortion can be derived as is known in conventional video coding approaches that measure the difference in synthesized views produced by the samples being synthesized as the distortion of the synthesized views.

The concept of irregular block division coding according to an embodiment described below can be divided into a concept for processing partition information (see Sections 3.1 and 3.2) and a concept for processing CPV (see Section 3.3).

3.1 Wedgelet-based Intra Coding

This section presents two intra coding modes based on Wedgelet block partitioning (see Section 1). Both modes can be combined with the delta CPV method (see Section 3.3.2).

3.1.1 Intra-modeling of wedgelet block division

The basic principle of this mode is to find the best matching Wedgelet split in the encoder and send the split information explicitly in the bitstream. At the decoder, the block's signal is reconstructed using the explicitly transmitted segmentation information. As a result, the main tool for this mode is part of estimation and signaling.

Wedgelet split information for these modes is not predicted, but retrieved within the estimation process at the encoder. To this end, a minimum distortion Wedgelet search as described in section 1.4.2 is performed using the original depth signal of the current block as a reference. The search produces the best matching Wedgelet split for the distortion method used.

When reconstructing a block at the decoder, the Wedgelet partition information must be signaled in the bitstream. This is achieved by explicitly sending the index or position of the matching pattern in the list (see section 1.4.1). This list index is signaled in a fixed number of bins. Given N elements in a Wedgelet pattern list, the index of the pattern used is coded using a fixed-length code, or a variable-length code, or arithmetic coding (including context-adaptive binary arithmetic coding), or some other entropy coding method. do. An advanced method for signaling wedgelet splitting information either sorts the list based on the probability of each splitting pattern, or alternative representations of the splitting information, e.g. line start and end position or line start position and gradient It may include the step of using.

3.1.2 Intra prediction of wedgelet block division

The basic principle of this mode is to predict the Wedgelet partition from the information available for the same picture, i.e. the previously coded block in intra prediction. For a good approximation, the predicted split is refined in the encoder, for example by changing the line end position. A single transmission of the offset for the end-of-line position in the bitstream may suffice, and the signal of the block at the decoder is the segmentation information generated from combining the predicted segmentation with the transmitted refinement information as the offset. can be reconstructed using Consequently, the main tools for this mode are parts of prediction, estimation, and signaling.

The prediction of Wedgelet splitting information for these modes works internally with a Wedgelet representation consisting of the starting position and the gradient of the separating line. For further processing, ie to adapt the line end position offset and to reconstruct the signal of the block, the prediction result is converted into a representation consisting of line start and end positions. This mode of prediction process derives line start positions and gradients from information in previously coded blocks, such as to the left of and above neighboring blocks of the current block. In Fig. 12, only the current block 210 and the above neighboring block 212 are shown. Note that for some blocks, one or both of the neighboring blocks are unavailable. In this case, processing for this mode continues to set the skipped or lost information to a meaningful default value.

As shown in Fig. 12, two main concepts are distinguished for predicting Wedgelet partitioning information according to the presently presented embodiment. The first concept covers the case where one of the two neighboring reference blocks is a Wedgelet of the type shown in the example on the left of Fig. 12, where block 212 is exemplarily Wedgelet partitioned. The second concept covers the case when the two neighboring reference blocks are not of type Wedgelet, but of type intra direction, which may be the default intra coding type shown in the example on the right of Fig. 12, where block 212 ) is illustratively intra-coded.

If the reference block 212 is of type Wedgelet, the prediction process can operate as follows: According to Fig. 12 left, the gradient m _ref of the reference wedgelet is the starting position S _ref and the ending position E in the first step. derived from _ref . The principle of this concept is that the reference wedgelet within the current block 210 is possible only if the continuation of the separation line 201′ of the reference wedgelet 212 actually intersects the current block 210, i.e. the wedgelet separation line ( 201'). So, the next step is to check if we can continue the reference wedgelet. The example on the left of Fig. 12 is possible, but shows a scenario in which the continuation of the line does not intersect under the block if the starting and ending positions of the reference wedgelet are located at the left and right edges of the block. In case the check is positive, the start position S _p and the end position E _p are predicted in the final step. When the gradient m _p is equal to m _ref by definition, the position is simply computed as the intersection of a block boundary sample and a continuous line.

If the reference block 212 is of type intra direction, the prediction process can work as follows: According to Fig. 12 right side, the gradient m _ref of the reference block 212 is the intra prediction direction 214 in the first step. is derived from In the case of only intra prediction directions 214 provided in the form of abstract indices, a mapping or transform function may be needed to achieve the gradient m _ref . Contrary to the concept of predicting from the reference block 212 of the type wedgelet, no separation line information is provided by the reference block 212 of the type intra direction. Consequently, the starting position S _p is also derived from the information available in the decoder, ie the adjacent samples of the left and above neighboring blocks. These are shown hatching on the right side of FIG. 12 . The density of hatching represents the values of neighboring samples. As shown on the right of Fig. 12, the adjacent pair of neighboring samples with the maximum slope from this adjacent sample is selected as the starting position S _p . Here, the slope is understood as the absolute difference between the values of two successive samples. For Wedgelet division, the line start point S _p separates the two regions 202a and 202b at different values at one edge 216 of block 210. Therefore, the point of maximum slope between adjacent samples of neighboring blocks is the best predictor of S _p . Regarding the gradient, m _p is again equal to m _ref by definition, and the end position E _p together with _Sp can be calculated as a final step.

The two presented concepts are complementary. Prediction from a reference block of type Wedgelet has better matching split information, but is not always possible, and prediction from a reference block of type intra direction is always possible, but the split information is not suitable. Therefore, it is advantageous to combine both concepts into one prediction mode. To realize this without further signaling, the following processing layer can be defined: If the above reference block is of type Wedgelet, try to predict the split. Otherwise, if the left reference block is of type Wedgelet, we try to predict the split. Otherwise, the split is predicted from the top and left reference information. In the latter case, different decision criteria for deciding between the up and left orientations are possible, ranging from simply priorities above to advanced approaches to jointly evaluate the orientation and tilt of adjacent samples. These advanced criteria can also be applied if the above and left reference blocks are of type Wedgelet.

End-of-line position offsets for refining the Wedgelet division are not predicted, but can be retrieved within the estimation process at the encoder. As shown in Fig. 12, for the search, a candidate partition is generated from the predicted Wedgelet partition and the offset value for the end-of-line position E _off . By iterating over a range of offset values and comparing the distortions of different generated Wedgelet partitions, the offset value of the best matching Wedgelet partition is determined for the distortion method used.

To reconstruct a block at the decoder, the end-of-line position offset value must be signaled in the bitstream. It consists of three syntax elements: the first signaling element whether any offset E _off exists, i.e. whether this offset is zero, and the sign of the offset, if the offset is not zero, i.e. clockwise or clockwise. It can be signaled using a second element meaning the sign of the deviation in the opposite direction and a third element indicating absolute offset value minus 1: dmm_delta_end_flag, dmm_delta_end_sign_flag, dmm_delta_end_abs_minus1. In pseudocode, these syntactic elements could be included as:

dmm_delta_end_flag ( dmm_delta_end_flag ) if { dmm_delta_end_abs_minus1 dmm_delta_end_sign_flag}

dmm_delta_end_abs_minus1 and dmm_delta_end_sign_flag can be used to derive DmmDeltaEnd, i.e. E _off as follows: DmmDeltaEnd [ x0 ] [ y0 ] = ( 1 - 2 * dmm_delta_end_sign_flag[ x0 ] [ y0 ] ) * ( dmm_delta_end_abs_minus1 [ x0 ] [ y0 ] + 1) The most probable case is that the offset value is zero. For efficient signaling, a first bin as a function of a flag indicating whether the offset is zero is sent. If the offset is non-zero, k+1 additional bins follow to signal the offset value in the range ±2 ^k , where the first bin represents the sign and the remaining k bins represent the absolute value of the offset. For efficient coding, k is usually a small number and can be set adaptively according to the block size, for example. End-of-line position offsets may also be transmitted by some other entropy coding technique, including fixed-length codes, variable-length codes, or arithmetic coding (including context-adaptive binary arithmetic coding).

3.2 Intercomponent Prediction for Block Division Coding

This section presents two coding modes based on predicting segmentation information from texture. Both modes can be combined with the delta CPV method (see Section 3.3.2). It is assumed that the texture information (i.e. the conventional video picture) is transmitted before the associated depth map.

The basic principle of this mode can be described as predicting segmentation information from a texture reference block as a wedgelet or contour block segmentation. This type of prediction may be referred to as inter-component prediction. Unlike temporal or inter-view prediction, no motion or disparity compensation is needed here as the texture reference pictures simultaneously show the scene from the same perspective. When segmentation information is not transmitted during this mode, inter-component prediction uses the reconstructed texture picture as a reference. Depending on the color space used for texture coding, one or more components of the texture signal are considered for inter-component prediction. For video coding, the YUV color space is commonly used. Here, the luma component includes the signal of the depth block, that is, the most important information for predicting an edge between objects. Thus, the simple inter-component prediction approach only uses the information of the luma component, but the advanced approach additionally uses the chroma component for joint prediction or to refine the luma prediction result.

3.2.1 Texture-based prediction of wedgelet block division

The basic principle of this mode is to predict the Wedgelet division of the depth block 210 in the depth map 213 from the texture reference block 216. This is realized by searching for the best matching Wedgelet partition on the reconstructed texture picture as shown in FIG. 13 . To this end, a minimum distortion wedgelet search as described in section 1.4.2 is performed using the reconstructed texture signal 215, specifically the luma block 216 having the same location and size as the depth block 210 as a reference. do. The generated wedgelet split pattern 218 is used for prediction 220 of the depth block. In Fig. 13, this is highlighted by the upper box, and for the example shown, the predicted Wedgelet split (middle) approximates the depth block 210 very well. Signaling of partitioning information is not needed during this mode, as the described Wedgelet prediction can be equally performed at the encoder and decoder.

3.2.2 Texture-based prediction of contour block segmentation

픽처(215)에 대한 윤곽선 분할(218')을 도출함으로써 실현된다. 이를 위해, 윤곽선 근사치는 재구성된 텍스처 신호(215), 특히 참조로서 깊이 블록(210)과 같은 위치 및 크기를 가진 루마 블록(216)을 이용하여 실행된다. 이러한 윤곽선 예측이 인코더 및 디코더에서 동일하게 수행될 수 있을 때, 분할 정보의 시그널링은 이러한 모드 동안에 필요로 되지 않는다.The basic principle of this mode is to predict the contour segmentation of the depth block from the texture reference block. This is a reconstructed texture pick as shown in Figure 10.

This is achieved by deriving the contour segmentation 218' for the picture 215. To this end, contour approximation is performed using the reconstructed texture signal 215, in particular the luma block 216 having the same location and size as the depth block 210 as a reference. Signaling of segmentation information is not needed during this mode as this contour prediction can be performed equally at the encoder and decoder.

The contour segmentation pattern can be generated by calculating the average value of the reference block 216 and setting it as a threshold value. Depending on whether the sample value in reference block 216 is below or above the threshold, the matching location is indicated as part of region P ₁ or P ₂ in segmentation pattern 218'. The generated contour segmentation pattern 218' is used for prediction 220 of the depth block 210. In Fig. 13, this is highlighted by the lower box, and for the example shown, the predicted Wedgelet split (middle) 218' approximates the depth block 210 very well. However, the threshold approach potentially leads to a frayed pattern with many isolated small parts that do not closely approximate the depth signal. To improve the consistency of the contour pattern, the deviation process can be extended, for example by filtering or segmentation approaches.

A binary segmentation pattern that defines a contour segmentation pattern, dmmWedgeletPattern[x, y], and represents the location of the sample within the block to which (x, y) with x, y =0..nT-1 is segmented is: x, y = 0..nT-1 may be derived from the luma samples of the co-located texture video block videoLumaSamples[x,y].

The threshold value tH is derived as follows:

tH = sumDC/(nT*nT), sumDC += videoLumaSamples[x, y], x, y = 0..nT-1

The pattern value is set as follows:

- If videoLumaSamples[x, y] is greater than tH, the following applies:

dmmWedgeletPattern[x, y] = 1

- Otherwise, the following applies:

dmmWedgeletPattern[x, y] = 0

3.3 CPV Coding

Concepts for CPV coding are presented in these sections. These can equally be applied to all four modes for predicting or estimating block partitioning information when both partitioning types, i.e., wedgelets and contours, have two partitions with constant values by definition. As a result, CPV processing does not need to differentiate between splitting types or coding modes, but rather assumes that a splitting pattern is provided for the current depth block.

3.3.1 Predicted CPV

For a good understanding of CPV prediction, three types of CPV are distinguished: original CPV, predicted CPV, and delta CPV. The relationship between them is shown schematically on the right side of FIG. 14 for a cross-section of the block (dotted line 230 on the left side of FIG. 14 ). Here, line 232 represents the original signal of block 200 along line 230 . According to the discussion of Sections 1.3 and 2.3, the original CPV (lines 234 and 236 on the right side of FIG. 14 ) is computed as the average value of the signals covered by the corresponding regions P ₁ and P ₂ , respectively.

및

는 주어진 블록 분할에 대한 원래의 신호의 최상의 근사치(도 14 왼쪽, 또는 라인(232))에 이르지만, 원래의 신호가 디코더에서 이용할 수 없을 때, 비트 스트림으로 값을 전송할 필요가 있다. 이것은 비트 레이트의 관점에서 상당히 비싸고, CPV에 대한 예측의 원리를 채용함으로써 회피될 수 있다. 원래의 CPV와는 대조적으로, 도 14 왼쪽에서 해칭된 샘플(203) 상에 예시된 바와 같이, 예측된 CPV는 또한 디코더, 즉 왼쪽 위의 이웃한 블록의 인접한 샘플에서 이용할 수 있는 정보로부터 도출된다. 여기서, 인접한 샘플은 회색으로 표시되고, 주어진 분할 패턴의 각 영역에 대한 예측된 CPV는 대응하는 영역(도 14 왼쪽의 라인(238 및 240))에 인접하는 이들 샘플의 평균값을 계산하여 생성된다. 왼쪽 또는 위의 이웃한 블록이 항상 이용 가능한 것은 아니다는 것을 주목한다. 이러한 경우에, 각각의 인접하는 샘플은 디폴트 값으로 설정될 수 있다.original CPV

and

arrives at the best approximation of the original signal for a given block partition (Fig. 14 left, or line 232), but when the original signal is not available at the decoder, it needs to transmit the value in the bit stream. This is quite expensive in terms of bit rate and can be avoided by employing the principle of prediction for CPV. In contrast to the original CPV, as illustrated on the hatched sample 203 on the left of FIG. 14 , the predicted CPV is also derived from information available in the decoder, i.e., adjacent samples of the upper left neighboring block. Here, adjacent samples are displayed in gray, and the predicted CPV for each region of a given split pattern is generated by calculating the average value of these samples adjacent to the corresponding region (lines 238 and 240 on the left side of Fig. 14). Note that the neighboring block to the left or above is not always available. In this case, each neighboring sample may be set to a default value.

및

는 라인(238 및 240)으로 나타내고, 예시는 원래의 CPV 및 예측된 CPV가 상당히 서로 다를 수 있다는 것을 강조한다. 사실상, 원래의 값과 예측된 값 사이의 차이

및

는 현재 블록(200)의 원래의 신호(232)와 해칭된 샘플(203) 상에 표시된 재구성된 이웃한 블록의 경계 신호 사이의 유사성에 의존한다. 이러한 차이는 대응하는 영역의 델타 CPV로 정의된다. 이것은 델타 CPV

및

가 인코더에서 추정되고, 비트 스트림으로 전송되는 경우에 델타 CPV를 예측된 CPV에 추가함으로써 디코더에서 원래의 CPV를 재구성할 수 있다는 것을 의미한다. 원래의 값 대신에 델타를 전송하는 것만이 필요한 비트 레이트를 상당히 감소시킨다.On the right of Figure 14, the predicted CPV

and

is represented by lines 238 and 240, and the example highlights that the original CPV and the predicted CPV can differ significantly from each other. In effect, the difference between the original value and the predicted value

and

depends on the similarity between the original signal 232 of the current block 200 and the boundary signal of the reconstructed neighboring block displayed on the hatched sample 203. This difference is defined as the delta CPV of the corresponding area. This is Delta CPV

and

is estimated at the encoder and can reconstruct the original CPV at the decoder by adding the delta CPV to the predicted CPV when transmitted as a bit stream. Only sending the delta instead of the original value significantly reduces the bit rate needed.

The predicted constant partition value CPVs are denoted by dmmPredPartitionDC1 and dmmPredPartitionDC2 and can be derived from neighboring samples p[x, y] as follows. In the following, dmmWedgeletPattern represents the division of the current block surrounding sample (x,y), exemplarily x, y = 0..nT-1. That is, the sample position adjacent to the upper edge is located at (x, -1), x = 0..nT-1, the sample position adjacent to the left edge is located at (-1, y), and y = 0 ..nT-1. The already reconstructed neighboring sample values are denoted by p[x, y]. sumPredDC2, sumPredDC1, numSamplesPredDC2 and numSamplesPredDC1 are initially set to 0:

For x = 0..nT-1, the above neighboring samples are summarized as:

- If dmmWedgeletPattern [x, 0] is equal to 1 (eg division P ₁ ), the following applies:

sumPredDC2 += p[x, -1] and numSamplesPredDC2 += 1

- Otherwise (eg partition P ₂ ), the following applies:

sumPredDC1 += p[x, -1] and numSamplesPredDC1 += 1

For y = 0..nT-1, the left neighboring samples are summarized as:

- If dmmWedgeletPattern [0, y] is equal to 1, then the following applies:

sumPredDC2 += p[-1, y] and numSamplesPredDC2 += 1

- Otherwise, the following applies:

sumPredDC1 += p [-1 , y ] and numSamplesPredDC1 += 1

The predicted constant split value is derived as follows.

- If numSamplesPredDC1 is equal to 0, the following applies:

dmmPredPartitionDC1 = 1 << ( BitDepth _Y - 1)

- Otherwise, the following applies:

dmmPredPartitionDC1 = sumPredDC1 / numSamplesPredDC1

- If numSamplesPredDC2 is equal to 0, the following applies:

dmmPredPartitionDC2 = 1 << ( BitDepth _Y - 1)

- Otherwise, the following applies:

dmmPredPartitionDC2 = sumPredDC2 / numSamplesPredDC2

3.3.2 Quantization and Adaptation of Delta CPV

을 가진

로서 델타 CPV에 대한 양자화 스텝 크기를 설정한다.Based on the principles of CPV prediction, concepts for efficient processing of delta CPV are introduced in this section. Transmitting the delta CPV in the bitstream is to reduce distortion of the reconstructed signal for block divisional coding. However, the bit rate required to signal the delta CPV value limits the benefits of this approach when the difference between the original and predicted signals is also covered by transform coding of the residual. Thus, the quantization of the delta CPV can be introduced as follows: the values are linearly quantized after estimation at the encoder and inverse quantized before reconstruction at the decoder. Transmitting the quantized delta CPV has the advantage of reduced bit rate, but only the signal reconstructed from the inverse quantized value differs slightly from the best possible approximation. Consequently, this results in a lower rate distortion cost compared to the case without quantization. For the step size of linear quantization, performance can be further improved by applying well-known principles from transform coding, i.e., by defining the quantization step size as a function of QP rather than a fixed value. to be efficient and powerful

with

Set the quantization step size for the delta CPV as

Possible signaling of the delta CPV in the bitstream of the two regions of the divided block can be understood as:

dmm_dc_1_abs[x0 + i][y0 + i] dmm_dc_1_sign_flag[x0 + i][y0 + i] dmm_dc_2_abs[x0 + i][y0 + i] dmm_dc_2_sign_flag[x0 + i][y0 + i]

Transmission in the bitstream for a block can be done in dependence on the syntax element DmmDeltaFlag, which is either explicitly transmitted or derived from some coding mode syntax element. can be used to: DmmQuantOffsetDC1[ x0 ][ y0 ] = ( 1 - 2 *dmm_dc_1_sign_flag[ x0 ][ y0 ] )*dmm_dc_1_abs[ x0 ][ y0 ]

DmmQuantOffsetDC2[ x0 ][ y0 ] = ( 1 - 2 *dmm_dc_2_sign_flag [ x0 ][ y0 ] )*dmm_dc_2_abs[ x0 ][ y0 ]

The inverse quantized offsets dmmOffsetDC1 and dmmOffsetDC2 may be derived from DmmQuantOffsetDC1 and DmmQuantOffsetDC2 as follows.

BitDepth _Y may be the bit depth at which DmmQuantOffsetDC1 and DmmQuantOffsetDC2 are internally expressed in the encoder and decoder, and QP' may be, for example, the above-mentioned quantization parameter QP included in the transform coefficient level of the coding prediction residual of the current slice.

Then, a constant split value CPV can be obtained by adding the inverse quantized offset to the predicted CPV:

For the first partition: dmmPredPartitionDC1 + dmmOffsetDC1

For the 2nd partition: dmmPredPartitionDC2 + dmmOffsetDC2

As already mentioned at the beginning of Section 3, distortion for an estimation tool can be measured in two different ways. Regarding delta CPV, this distortion method significantly affects the estimation process. If distortion is measured as the difference between the block's distorted depth signal and the original depth signal, the estimation process simply calculates and quantizes the delta CPV as described above to find a close approximation of the original CPV. In cases where distortion is measured for a synthesized view, the estimation process can be extended to better adapt the delta CPV to the quality of the synthesized view. This is based on the fact that such a delta CPV leading to the best approximation of the original CPV does not necessarily lead to the best synthesized view quality. When finding the delta CPV that leads to the best synthesized view quality, the estimation process is extended by an iterative least distortion search (see section 1.4.2) for all possible delta CPV combinations for the two splits. For efficient processing and signaling, the range of tested values can be limited. The search combines the delta CPVs that cause the least distortion in the synthesized view, and these values are finally quantized for transmission.

Note that the delta CPV method can potentially skip transform/quantization and transmission of (remaining) residuals. Due to the close approximation of the original or optimal depth signal, respectively, the impact of omitting residuals when evaluated against the quality of the rendered view is limited.

4. Mode Coding

4.1 Mode signaling

In the encoding process, one mode is selected per block through rate-distortion optimization, and the mode information is signaled in the bitstream as before the segmentation and CPV information, for example. According to Section 3, the following four block partitioning modes can be defined (e.g., in addition to non-irregular partitioning mode):

Wedgelet_ModelIntra: 웨지렛 블록 분할의 인트라 모델링 (섹션 3.1.1 참조)

Wedgelet_ModelIntra: Intra modeling of wedgelet block divisions (see section 3.1.1)

Wedgelet_PredIntra: 웨지렛 블록 분할의 인트라 예측 (섹션 3.1.2 참조)

Wedgelet_PredIntra: intra prediction of wedgelet block split (see section 3.1.2)

Wedgelet_PredTexture: 웨지렛 블록 분할의 텍스처 기반의 예측 (섹션 3.2.1 참조)

Wedgelet_PredTexture: Texture-based prediction of wedgelet block division (see section 3.2.1)

Contour_PredTexture: 윤곽선 블록 분할의 텍스처 기반의 예측 (섹션 3.2.2 참조)

Contour_PredTexture: Texture-based prediction of contour block segmentation (see section 3.2.2)

Each of the four modes can be applied with or without a delta CPV processing method that generates eight different mode_IDs for signaling the decoder (see section 3.3.2), some type of processing for prediction and reconstruction of blocks. should be applied

If the block partitioning modes introduced above are implemented as an additional set of block coding modes to existing coding frameworks such as those of FIGS. It can signal whether the mode is used or not. If this flag is not set, normal block coding mode signaling follows. Otherwise, the mode_ID that assigns the actual block partitioning mode and whether the delta CPV is also transmitted is signaled. In the bitstream, mode_ID is represented through 3 bins.

4.2 Mode pre-selection

The idea behind mode preselection is to reduce the processing and signaling effort for block partitioned coding by implementing the concept of excluding modes that are highly unlikely to be selected for the current block.

The concept of first mode pre-selection makes very low-probability modes unavailable for small block sizes. This means that in most cases the distortion is high compared to the rate needed to signal the mode information. Of the four modes defined in section 4.1, this applies to Wedgelet_PredIntra and Contour_PredTexture. Based on statistical analysis, these two modes become unusable for block sizes 4x4 and smaller.

로서 설정된다.The concept of second mode pre-selection is applied to two modes based on inter-component prediction, namely Wedgelet_PredTexture and Contour_PredTexture. The idea behind this concept is to appropriately exclude these modes in cases where the likelihood that a meaningful block division pattern can be derived from a texture reference block is very low. These blocks are characterized as being relatively planar without significant edges and contours. To identify these blocks, the variance of the texture reference block is analyzed. The criterion for disabling the two mentioned modes is that the amount of change is below a certain threshold. This mode pre-selection method is implemented as follows: The amount of variation is measured as the average absolute error (MAE) between the average value of the luma sample and the reference block (see 216 in FIG. 13). Instead of a fixed value, the threshold is set as a function of the quantization parameter (QP). Based on the results of the statistical analysis of the MAE values, the threshold value for high QP and vice versa is based on the fact that these two modes are excluded for more blocks.

is set as

15 shows a visualization of this mode preselection method with details for the two texture luma blocks 250 ₁ and 250 ₂ , and the absolute difference versus average value to the right at 252 ₁ and 252 ₂ , respectively. Block 250 ₁ has a very planar spatial sample value appearance with little structure as reflected by the very low amount of variance. When no meaningful segmentation information is predicted from this block 250 ₁ , the modes Wedgelet_PredTexture and Contour_PredTexture are not considered. In contrast, block 250 ₂ has a high amount of variance resulting from significant edges and contours. Thus, two modes are considered when the splitting information derived from block 250 ₂ is likely to be a good predictor for splitting of a matching depth block.

Limit: block size
Limit: Texture Reference Variation Amount off
off On
off off
On On
On

x
x
x
x
x
x
x
x x
x


x
x x
x
x
x x
x

8
3 4
2 4
2 2
One

Table 1: Modes by preselection decision. Table 1 summarizes the effect of the two-mode preselection concept on the available modes. By excluding certain modes, the number of mode_IDs that need to be signaled in the bitstream is reduced. The table shows that the two methods can be combined in an efficient manner when each method reduces the number of required bins signaling mode_ID by 1, and the combination of the two modes reduces the number of bins by 2. generalization

A number of possible irregular partitioning modes, on the one hand the conceptual subdivision into double segmentation decisions (see 3.1 and 3.2), on the other hand the coding parameters for coding for the two generated partitions (see 3.3) as well as possible adoption in the coding framework and After describing possible coding environments in which these modes may additionally be provided, the generated embodiments for each decoder and encoder should be partially described in a more general view. In particular, the following sections highlight certain advantageous details described above and explain how these details can be used in decoders and encoders in a more general sense than described above. In particular, as described below, some of the beneficial aspects used in the above modes may be used individually.

5.1 Wedgelet Separation Line Continuation Across Block Boundaries

As is evident from the discussion above, the use of Wedgelet partitioning forms a possible compromise between the signaling overhead for signaling the partitioning on the one hand and the amount of variety achievable by irregular partitioning on the other hand. Nonetheless, a significant amount of ancillary information data can be obtained by dividing information, i.e., by using the indexing of the position of the wedgelet separation line, for example according to the concept described above for section 3.1.1. This is necessary to explicitly transmit the position of the line.

Thus, Wedgelet separation line continuation across block boundaries forms one possible way to solve the problem just described. The above description in Section 3.1.2 described a specific example for using a solution to this problem. More generally, however, when using the idea of a wedgelet-separating line continuation across block boundaries, the decoder may be configured as described below with respect to FIG. 16, in accordance with an embodiment of the present invention. Nevertheless, all details set forth in Sections 3.1.2 and any other of Sections 3 to 4 should be understood as possible implementation details that may be individually combined with the description set forth below.

The decoder of FIG. 16 is indicated generally by reference numeral 300 and is configured to reconstruct a sample array 302 from a data stream 304 . The decoder is configured to perform reconstruction by block-based decoding. For example, the sample array 302 can be part of a sequence of sample arrays, and the decoder 300 can be implemented as a block-based hybrid decoder supporting a different coding mode for each block 304. A sample array can be any spatially sampled information, for example a texture or depth map. For example, the decoder 300 of FIG. 16 can be implemented to reconstruct a view comprising a texture/video representing the sample array 302 and a depth/disparity map. The decoder 300 may be implemented as a pair of decoding branches 106 _d,1 plus 106 _v,1 , or may be implemented separately according to the decoding branches 106 _d,1 . That is, the decoder 300 may be configured to reconstruct the sample array 302 using coding modes such as intra prediction, temporal (motion compensated) prediction, and/or interview (variation compensated) prediction, with or without residual coding of prediction residuals. can Coding modes may also include, for example, explicit Wedgelet coding modes, whereby for each block the location of the Wedgelet separation line is specified within the data stream 304, such as the mode described in Section 3.1.1. is transmitted negatively.

In any case, this decoder 300 is configured to perform a predetermined coding mode option on a current block 210, such as the block signaled within the data stream 304, steps are now described. Functionality related to these steps may be incorporated within the intra prediction module 36 or the intra prediction module and exchange module 52 .

The steps performed by the decoder (300) for each mode of block are the wedgelet separation line position prediction step (306) followed by a position refinement step (308) and a decoding step (310). In particular, the decoder 16 of FIG. 16 allows the wedgelet separation line at the predicted position 312 to form an extension or continuation of the wedgelet separation line 201' of the neighboring block 212 into the current block 210. Predict in step 306 the position 312 of the wedgelet separation line in block 210 of sample array 302 according to the wedgelet separation line 201' of the neighboring block 212 of block 210 to is configured to In other words, the decoder 300 may perform each explicit signaling for a block 212 from the data stream 304, or by some other coding option, such as detecting an edge within the texture sample array to which the sample array 302 belongs. The location of the wedgelet separation line 201' of block 212 can be derived. Other possibilities have been described above and will be further explained below.

As noted above, the wedgelet separation line of block 210 where position 312 is predicted at 306 may be a straight line, as in the case described above in Section 3.1.2. Alternatively, however, a line may be defined more generally, for example using a sequence of sample location hops, i.e., a sequence of symbols each of which defines the next pixel within the line belonging to a separate line. The line may also have an analytically determined predetermined curvature that may be predicted from line 201' or may be derived from some other previously processed portion of data stream 304.

In particular, prediction 306 is later determined for locations where the wedgelet separation line of block 210 is not only predetermined with respect to the general direction of extension, but also laterally to the general direction of extension of the wedgelet separation line. In the case of curves, curve fitting, for example using a polynomial function, can be used to infer the separation line of block 212 and find the separation line of block 210 respectively. In the case of a straight line, the slope and position in the direction lateral to the wedgelet separation line are determined.

It should also be mentioned that for the prediction 306, the neighboring part and the extended part need not necessarily be defined in terms of space. Rather, blocks 210 and 212 may also be adjacent in time. For example, block 212 may be a co-located block of a sample array of a sequence of sample arrays temporally adjacent to sample array 302 . In that case, the extension of wedgelet separation line 201 to block 210 must be "temporal continuous".

The explicit possibility that prediction 306 can be performed is detailed in Section 3.1.2, the description of which is referred to herein. Position refinement 308 is to refine the predicted position 312 . That is, the decoder 300, in the position refinement 308, uses the refinement information signaled within the data stream 304 to determine the predicted position 312 of the wedgelet separation line 301 of block 210. It is configured to purify. Thereby, refined wedgelet separation line 201 divides block 210 into first and second wedgelet divisions 202a and 202b.

As described above, the decoder 300 causes the wedgelet separation line 201 at the predicted location 312 to form a spatially collinear extension of the wedgelet separation line 201' of the neighboring block 212. can be configured, and the refinement maintains the start position 314 of the wedgelet separation line of the predetermined block 210 adjacent to the neighboring block 212 relative to the predicted position 312 regardless of the refinement information. can be limited as much as possible. That is, in the case of a straight wedgelet separation line, only its slope can be refined, but the starting point of the wedgelet separation line 201 at the edge 316 of block 210 separating blocks 210 and 212 is It doesn't change. For example, the offset of the opposite end 318 of the wedgelet separation line 201, i.e. the offset of the end position of the wedgelet separation line 201, depends on the location 312 of the predicted wedgelet separation line, the end position ( The offset of the opposite end 318 of wedgelet separation line 201 along the perimeter of block 210 from 320, i. can be signaled within the data stream 304 as

In Section 3.1.2, the offset is denoted as E _off . As described in this section, the decoder 300 can be configured to extract refinement information from the data stream using entropy decoding, where the number of blocks 210 measured in units of sample position pitch along the periphery direction. The different possible offsets from the direct extension sample position 320 along the periphery have associated with it a probability estimate that increases monotonically from large to small offsets, such that small offsets have a higher probability than large offsets associated with it. For example, the VLC codeword length may decrease monotonically.

Also, as described above, three syntax elements, first signaling whether any offset E _off exists, i.e. whether this offset is zero, and if the offset is not zero, the sign of the offset, i.e. clockwise or counterclockwise. The second signaling meaning the sign of the deviation, and the third signaling indicating the absolute offset value minus 1: dmm_delta_end_flag, dmm_delta_end_sign_flag, dmm_delta_end_abs_minus1 can be used to transmit E _off . In pseudocode, these syntactic elements can be included as:

dmm_delta_end_flag if( dmm_delta_end_flag ) { dmm_delta_end_abs_minus1 dmm_delta_end_sign_flag}

을 제 1 웨지렛 분할(202a) 내의 샘플 어레이 위치의 샘플에 할당하고, 제 2 일정한 분할 값

을 제 2 웨지렛 분할(202b) 내의 샘플 어레이 위치의 샘플에 할당함으로써 현재 블록(210)의 예측을 포함한다. 디코딩 절차(310)의 구현의 이점은 보조 정보의 량이 낮게 유지될 수 있다는 것이다. 특히, 이러한 가능한 구현은 특히 깊이 맵의 경우와 같이 급격한 에지를 가진 비교적 평평한 값 플래토(value plateau)로 구성되는 상술한 특성을 갖는 샘플 어레이에 의해 반송되는 정보의 종류의 경우에 특히 유리하다. 그러나, 심지어 디코더는 개별적으로 다른 코딩 파라미터를 웨지렛 분할(202a 및 202b)에 할당하는 것이 가능하다. 예를 들면, 움직임 및/또는 변이 보상된 예측은 디코딩(310)에서 분할(202a 및 202b)에 개별적으로 적용되어, 개별 벡터와 같이 분할(202a 및 202b)에 대해 개별적으로 각각의 움직임 및/또는 변이 파라미터를 획득할 수 있다. 대안적으로, 분할(202a 및 202b)은 예컨대 각각의 인트라 코딩 방향을 개별적으로 동일한 방향에 적용함으로써 디코딩(306)에서 개별적으로 인트라 코딩될 수 있다.dmm_delta_end_abs_minus1 and dmm_delta_end_sign_flag can be used to derive DmmDeltaEnd, i.e., E _off as follows: DmmDeltaEnd[x0][y0] = ( 1 - 2 * dmm_delta_end_sign_flag[x0][y0]) * ( dmm_delta_end_abs_minus1[x0][y0 ] + 1) Then, in decoding 310, the decoder 300 is configured to decode the predetermined block 210 in units of first and second Wedgelet partitions 202a and 202b. In Sections 3 and 4, and particularly in the preceding description in Section 4 above, decoding 310 performs a first constant division value

is assigned to the sample at the sample array position in the first Wedgelet division 202a, and the second constant division value

includes the prediction of the current block 210 by assigning to the sample of the sample array position in the second Wedgelet partition 202b. An advantage of implementing the decoding procedure 310 is that the amount of auxiliary information can be kept low. In particular, this possible implementation is particularly advantageous for the kind of information carried by the sample array having the above-described characteristics consisting of a relatively flat value plateau with sharp edges, such as in the case of a depth map. However, it is even possible for the decoder to assign different coding parameters to Wedgelet partitions 202a and 202b individually. For example, the motion and/or disparity compensated prediction is applied separately to partitions 202a and 202b in decoding 310, so that each motion and/or disparity compensated prediction separately for partitions 202a and 202b as separate vectors. Variation parameters can be obtained. Alternatively, partitions 202a and 202b may be individually intra-coded in decoding 306, for example by applying each intra-coding direction separately to the same direction.

According to Figure 16, the following information: 1) a coding option identifier with each predefined condition triggering steps 306-310, 2) refinement information such as an end position offset, and 3) optionally, a neighboring sample. /Coding parameters such as CPV or CPV residuals, optionally for one or both partitions 202a, 202b, 4) Optionally, coding parameter residuals such as DeltaCPV block ( 210) can be provided in the data stream.

Moreover, the decoder 300 of FIG. 16 has two coding modes in which the coding modes realized by procedures 306-310 are only triggered by respective common predetermined values of respective coding option identifiers within the data stream 304. It can be configured to be a default coding mode option among options. For example, the decoder 300 of FIG. 16 retrieves a coding option identifier from the data stream 304, and if the coding option identifier has a predetermined value, a set of candidate blocks neighboring the predetermined block is a block ( 210) to check if it has a wedgelet separation line. For example, a candidate block is a spatially neighboring neighbor of the sample array 302 that precedes the current block 210 applied by the decoder 300 in the decoding block 304 of the sample array 302 in coding order or decoding order. Block 304 may be included. For example, the coding order may scan block 304 row-wise from left to right and top to bottom, in which case the candidate block includes the block immediately adjacent to the left of current block 210 and , blocks immediately adjacent to the top of the current block 210, such as block 212. If the check reveals that such a Wedgelet block is among the set of candidate blocks, the decoder 300 may perform prediction 306, refinement 308, and decoding 310 in an unmodified manner. However, otherwise, decoder 300 may perform prediction 306 differently. As detailed in Section 3.1.2, and described in more detail in the next section, the decoder (300) performs intra prediction of one or more intra-predicted blocks of candidate blocks or reconstructed neighboring samples neighboring the current block (210). It can be configured to predict the position of the wedgelet separation line 201 within the current block 210 by setting the extension direction of the wedgelet separation line 201 within the current block 210 according to the direction. Reference is made to the description above and below as far as possible implementations for prediction of constant split values in decoding 310 are concerned.

Moreover, it is noted that the continuation of Wedgelet separation lines across block boundaries yields certain advantages when combined with coding modes that are more liberal with double segmentation of the current block, such as contour mode, as described above and further below. It should be. To be more precise, the decoder 300 can be configured to support the modes realized by blocks 306 to 310 as well as the contour segmentation mode, allowing appropriate adaptation of the coding overhead to the needs of the blocks. have.

In some cases, the block decoded/reconstructed by procedures 306-310 can serve as a reference for the prediction loop of decoder 300. That is, a prediction result in the case of using bi-valued prediction may serve as a reference for, for example, motion and/or disparity compensated prediction. Moreover, the reconstructed values obtained by decoding 310 may serve as spatial neighboring samples in intra-predicting any one of the blocks 304 of the sample array 302 according to the decoding order.

Fig. 17 shows a possible encoder that fits the decoder of Fig. 16; In particular, FIG. 17 shows an array of samples along the Wedgelet separation line of a neighboring block of a predetermined block such that the Wedgelet separation line at the predicted location forms an extension of the Wedgelet separation line of the neighboring block into the predetermined block. It depicts an encoder 330 for encoding an array of samples into a data stream configured to predict the position of a wedgelet separation line within a predetermined block. This function is shown at 332. Furthermore, the decoder 330 has a function 334 to refine the predicted position of the wedgelet separation line using the refinement information, wherein the wedgelet separation line of the predetermined block divides the predetermined block into first and second wedgelets. Divide by division. Encoder 330 also has an insert function 336 where refinement information is inserted into the data stream, and an encoding function where encoder 330 encodes a predetermined block in units of first and second Wedgelet partitions.

5.2 Wedgelet Separation Line Extension Direction Prediction from Intra Prediction Direction of Neighboring Blocks

As mentioned above, even Wedgelet-based block division requires a significant amount of auxiliary information to inform the decoding side about the location of the Wedgelet separation line.

The idea on which the embodiments described below are based is that the intra-prediction direction of a neighboring intra-prediction block predicts the direction of extension of the Wedgelet separation line of the current block, so that it can be used to reduce the auxiliary information rate required to convey partition information. that there is

In the description above, Section 3.1.2 has shown possible implementations of the embodiments described below, which are described in more general terms in order not to be limited to the glomeration of irregular partitioning modes described above in Sections 3 and 4. . Rather, the ideas just mentioned may advantageously be used independently of the other details discussed in Section 3.1.2, as described in more detail below. Nevertheless, all details set forth in Section 3.1.2 and other Sections should be understood as details of possible realizations that may be combined with the descriptions set out individually below.

18 illustrates an embodiment of a decoder 400 that can be implemented as described above with respect to Section 3.1.2 and/or FIGS. show That is, the decoder 400 of FIG. 18 is configured to reconstruct the sample array 302 from the data stream 304. In general, the decoder 400 of FIG. 18 is defined by functions 306 to 310 and, except for the optional coding mode for the decoder of FIG. 18 , as specified above in Section 5 for the decoder 300 of FIG. can be implemented as That is, the decoder 400 of FIG. 18 is operable to reconstruct the sample array 302 of FIG. 18 by block-based decoding, such as block-based hybrid decoding. Among the coding modes available for block 303 of sample array 302 is an intra prediction mode, further described with respect to functional module 402 of decoder 400 . As in the case of the decoder 300 of FIG. 18 , the subdivision of the sample array 302 into blocks 303 may be essentially fixed, or may be signaled within the data stream 304 by respective subdivision information. . In particular, the decoder 400 of FIG. 18 may be configured as described above with respect to FIGS. 1 to 3 , i.e., the decoder of FIG. 3 or any view decoder, such as the pair of coding branches 106 _v/d,1 , or , can simply be constructed as a depth decoder such as (106 _d,1 ).

In particular, the decoder 400 of FIG. 18 has a function 402 according to which the first block 212 of the sample array 302 is predicted using intra prediction. For example, intra prediction mode is signaled within data stream 304 for block 212 . The decoder 400 copies the reconstructed value of the sample 404 of the sample array 302 adjacent to the first block 212 along the intra prediction direction 214 to the first block 212 to obtain the first block ( 212) to perform intra prediction 402. The intra prediction direction 214 may also be signaled within the data stream 304 for block 212, such as by indexing one of a number of possible directions. Alternatively, the intra prediction direction 214 of block 212 itself may be predicted. For example, see the description of FIGS. 1-3 in which the intra predictor 36 of the decoding branch 106 _d,1 can be configured to perform step 402 . Specifically, the neighboring sample 404 is a sample array 302 already passed in decoding order by the decoder 400 such that a reconstruction containing the reconstructed value of the neighboring sample 404 in block 212 is already available. may belong to block 303 of ). As discussed above, various coding modes may have been used by decoder 400 to reconstruct the previous block, which precedes in decoding order.

Moreover, the decoder 400 of FIG. 18 sets the extension direction of the Wedgelet separation line 201 in the second block 210 according to the intra prediction direction 214 of the neighboring block 212, so that the first block 212 ) is configured to predict the position 312 of the wedgelet separation line 201 within the second block 210 neighboring the second block 212, and the wedgelet separation line 201 divides the second block 212 into first and second Divide into wedgelet partitions 202a and 202b. For example, the decoder 400 of FIG. 18 sets the extension direction of the wedgelet separation line 201 to be the same as the intra prediction direction 214 as much as possible for the quantization of the representation of the extension direction of the wedgelet separation line 201. can be configured to set In the case of a straight separation line, the extension direction simply corresponds to the slope of the line. In the case of a curved separation line, the intra prediction direction may be employed as the local slope of the current block's separation line at the boundary to the neighboring block, for example. For example, the decoder 400 of FIG. 18 selects an extension direction from among a set of possible extension directions that form the best approximation of the intra prediction direction 214 .

Thus, in prediction 404, the decoder 400 predicts the location 312 of the wedgelet separation line 201 of the current block 210, at least as far as the expansion direction is concerned. The derivation of the position of the wedgelet separation line 201 of the second block 210 may be completed without modifying the extension direction 408 . For example, in section 3.1.2 it is shown that the prediction of the starting point 314 of the wedgelet separation line 201 can be performed by the decoder 400 in deriving the wedgelet separation line position in step 406. Although described, the decoder 400 may alternatively be configured to derive this starting point 314 by explicit signaling within the data stream 304 . Furthermore, temporally predicting the distance of a transverse position from a Wedgelet block of the same position in a previously decoded sample array, or spatially predicting a transverse position from another sample array belonging to a different view relative to the sample array 302. 18 decodes the wedgelet separation line 201 on derivation 406 transverse to the extension direction 316 with the extension direction 316 parallel to the extension direction 316 by predicting It can be configured to be spatially arranged.

However, upon deriving the location of the Wedgelet separation line 201 within the second block 210 of the sample array 302, the decoder 400 uses a second block 210 along a portion around the second block 210. It is preferred to place the starting point 314 of the wedgelet separation line 201 at the location of the maximum change between successive ones of the sequence of reconstructed values of the samples of the line of extended samples adjacent to block 210. The line of samples is indicated by reference numeral 410 in FIG. 18 where samples are symbolized by small crosses. A line of samples 410 may be limited to samples of available spatially neighboring blocks. In any case, it should be emphasized that the line 410 of samples for which the maximum change is determined may extend around one of the corners of the rectangular block 210 as shown in FIG. 18 . Thus, according to this procedure, the decoder 400 wedges in derivation 406 to start between neighboring samples and sample lines 410 where there is a maximum difference in the reconstructed value parallel to the direction of extension 408. It may be configured to place a let separation line.

Thus, the auxiliary information rate is saved since a good prediction has been found to derive the location of the wedgelet separation line 201 by means other than explicit signaling within the data stream 304.

Decoding 412 is then performed by the decoder 400, whereby the decoder 400 divides the second block in units of the first and second Wedgelet partitions 202a and 202b as described with respect to FIG. decode

Naturally, the decoder 400 of FIG. 18 can also be modified to include the refinement function 308 of FIG. 16 . Therefore, the end position 318 of the wedgelet separation line 201 of the current block 210 relative to the end position 320 of the position of the wedgelet separation line, which may or may not be constrained to be a straight line as described above. The offset may be signaled within data stream 304 as derived in step 406 .

Also as mentioned above, three syntax elements: the first signaling element for which offset E _off exists, i.e. whether this offset is zero, the sign of the offset if the offset is not zero, i.e. clockwise or A second element meaning the sign of the counterclockwise deviation, and a third element indicating the absolute offset value minus 1: dmm_delta_end_flag, dmm_delta_end_sign_flag, dmm_delta_end_abs_minus1 can be used to transmit this end position offset E _off . In pseudocode, these syntactic elements could be included as:

dmm_delta_end_flag ( dmm_delta_end_flag ) if { dmm_delta_end_abs_minus1 dmm_delta_end_sign_flag}

dmm_delta_end_abs_minus1 and dmm_delta_end_sign_flag can be used to derive DmmDeltaEnd, i.e. E _off as follows: DmmDeltaEnd [x0][y0] = ( 1 - 2 * dmm_delta_end_sign_flag[x0][y0]) * ( dmm_delta_end_abs_minus1 [x0][y0] + 1) However, alternative procedures are also feasible. For example, instead of signaling an end position offset, a directional or angular offset for an extension direction set according to intra prediction direction 214 may be signaled within data stream 304 for block 202 .

According to Fig. 18, the following information: 1) a coding option identifier with a respective predetermined condition triggering steps 406-412, 2) optionally refinement information such as an end position offset, 3) optionally, Optionally a coding parameter such as CPV or CPV residual for one or both partitions 202a, 202b, 4) optionally a coding parameter residual such as DeltaCPV, since it may be allowed to be spatially or temporally predicted from neighboring samples/blocks may be provided in the data stream for block 210.

For possible modifications of the decoding step 412 to the description of Section 3.3, reference is made to the above description of step 310 of FIG. 16 .

The decoder 400 of FIG. 18 can be configured to process steps 406 and 412 as coding mode options activated by coding option identifiers within the data stream 304, and the wedgelet separation line position derivation 406 If none of the set of candidate blocks in the neighborhood of the current block 210 have a Wedgelet separation line within such a block then form a secondary measure to derive the Wedgelet separation line position, the extension of the Wedgelet separation line being Continue to current block 210.

FIG. 19 shows an embodiment of an encoder suitable for the decoder of FIG. 18 . The encoder of FIG. 19 is indicated generally by reference numeral 430 and is configured to encode an array of samples into a data stream 304 . Internally, the encoder 430 is configured to predict the first block of the sample array using intra prediction in block 432 and, in block 434, line derivation according to the description of block 406 of FIG. 18 . Then, the encoder 430 encodes the second block that is the target of line derivation at 434 in units of the first and second divisions at encoding block 436 .

Naturally, beyond the functionality shown in FIG. 19 , the encoder 430 is configured to operate to mirror the functionality of the decoder of FIG. 18 . That is, the encoder 430 may operate on a block basis using, for example, block-based hybrid encoding. Although not explicitly described above, it also applies to the encoder of FIG. 17 compared to the decoder of FIG. 16 .

5.3 Derivation of the wedgelet separation line with starting points placed according to the reconstructed values of neighboring samples

An additional method for reducing the auxiliary information needed to convey information about the position of the wedgelet separation line of the wedgelet block forms based on the embodiments described further below. In particular, the idea is that the reconstructed values of the previously reconstructed samples, i.e. the blocks preceding the current block according to the coding/decoding order, i.e. the reconstructed values of the samples of the lines of extended samples adjacent to the current block along the periphery of the current block. Placing the starting point of the wedgelet separation line at the position of maximum change between successive ones of the sequence allows at least a prediction of the exact placement of the starting point of the wedgelet separation line. Thus, similar to the possibilities described above for Sections 5.1 and 5.2, the auxiliary information rate required to allow the decoder to correctly locate the Wedgelet separation line can be reduced. The idea based on the embodiment described below is also used in the above description in section 3.1.2, whereby possible implementations of the embodiment described below are described.

Accordingly, FIG. 20 illustrates an embodiment of a decoder 500 that may be implemented as described above with respect to Section 3.1.2 and/or FIGS. show That is, the decoder 500 of FIG. 20 is configured to reconstruct the sample array 302 from the data stream 304. In general, the decoder 500 of FIG. 20 is defined, for example, by functions 306 to 310, and with respect to the decoder 300 of FIG. 16, except for the optional coding modes for the decoder of FIG. 18, section The decoder of FIG. 18 (except for the coding mode, which can be implemented as specified above in 5.1 or 5,2 and is defined, for example, by functions 402, 406 and 412, and which is optional for the decoder of FIG. 20) 400) may be implemented as specified above in section 5.1 or 5,2. That is, the decoder 500 of FIG. 20 may operate to reconstruct the sample array 302 of FIG. 20 by block-based decoding such as block-based hybrid decoding. As in the case of decoder 300 of FIG. 18 , the subdivision of sample array 302 into blocks 303 may be essentially fixed, or may be signaled within data stream 304 by respective subdivision information. . In particular, the decoder 500 of FIG. 20 may be configured as described above with respect to FIGS. 1 to 3 , i.e., the decoder of FIG. 3 or any view decoder, such as the pair of coding branches 106 _v/d,1 , or , can simply be constructed as a depth decoder such as (106 _d,1 ).

To be frank, the decoder of FIG. 20 indicated at reference numeral 500 mainly corresponds to the decoder of FIG. 18 . However, the functions described with respect to FIG. 18 in blocks 402 and 406 only represent optional steps with respect to FIG. Rather, the decoder 500 of FIG. 20 moves along a portion of the periphery of the predefined block 210 between successive ones of a sequence of reconstructed values of a line 410 of extended samples adjacent to the predefined block 210 . Position of the wedgelet separation line 201 within the predetermined block 210 of the sample array 302 at step 406' by placing the starting point 314 of the wedgelet separation line 201 at the location of the maximum change in and the wedgelet separation line 201 divides the predetermined block 210 into first and second wedgelet divisions 202a and 202b. Then, at step 412', the decoder 500 performs decoding of the partitions 202a and 202b generated in the equation described above with respect to FIG.

More precisely, in derivation 406', the decoder 500 traverses the samples counter-clockwise or clockwise, according to the order of their occurrence, the reconstructed values of the samples of the already decoded neighboring blocks of block 210. order The resulting sequence of reconstructed values is shown in FIG. 20 at 502 . As can be seen, the maximum difference between successive reconstructed values occurs between the nth and (n + 1)th neighboring samples, so the decoder of FIG. 20 results in a pair of neighboring samples directly adjacent to each other. Between the samples in block 210, pairs of neighboring samples place wedgelet separation lines at adjacent edges 316. As described above, decoder 500 may use a row-wise block scanning direction, so that neighboring samples of sample line 410 may extend along the left edge and top edge of block 210 . This can be achieved by using a mix of row-wise scans of tree root blocks that are scanned row-wise in decoding/coding order, and for each currently visited tree-root block, a quad-tree Subdivision is performed, and its leaf root blocks are scanned in depth-first traversal order. When using this ordering, the probability of having the maximum number of already reconstructed neighboring samples is increased compared to using a breadth first traversal order.

In derivation 406 ′, the decoder 500 of FIG. 20 may use the derivation of the wedgelet separation line extension direction 408 as described in an optional manner in Section 3.1.2 with respect to FIG. 18 . Alternatively, the Wedgelet separation line extension direction in which the decoder 500 locates the Wedgelet separation line 201 of the current block 210 is a previously decoded sequence of sample arrays including, for example, the sample array 302. can be predicted differently in time from co-located Wedgelet blocks of the sample array. Alternatively, explicit signaling of the endpoint 318 of the wedgelet separation line may be used. The explicit signaling indicates the offset of the end point 318 from the sample position opposite to the start position 314 across the midpoint of block 210 . Other solutions are of course also feasible.

In this regard, it should be noted that starting point 314 may be defined by decoder 500 at step 406' to correspond to the nth sample position, the (n+1)th sample position, or a subpixel position therebetween. It should be noted.

Most of the combination possibilities mentioned above in Sections 5.1 and 5.2 can also transfer to the examples in this section. For example, the coding mode of the decoder 500, realized by blocks 406' and 412', whenever one of the set of candidate neighboring blocks has a Wedgelet separation line continuing to the current block 210, instead A common predetermined value of a common coding option identifier with the Wedgelet Separation Line Continuation concept of Section 5.1 indicating the basic coding mode to be performed and a subsidiary fallback function to be triggered. Other generalizations and modifications are also feasible. For example, the decoder 500 may also support contour segmentation mode and the like.

According to Fig. 20, the following information: 1) a coding option identifier with each predetermined condition triggering steps 406'-412', 2) optionally, refinement information such as an end position offset, and 3) optionally , optionally a coding parameter such as CPV or CPV residual for one or both partitions 202a, 202b, 4) optionally coding such as DeltaCPV, since it can be allowed to be spatially or temporally predicted from neighboring samples/blocks. Parameter residuals may be provided in the data stream for block 210 .

FIG. 21 shows an encoder suitable for the decoder of FIG. 20 . This is indicated by reference numeral 530 and is configured to perform line derivation 434' in accordance with step 406 and to perform encoding 436' for block 436 as described with respect to FIG.

5.4 Tile (pixel) based double segmentation of the depth/disparity map by thresholding the co-located parts of the picture

As should be apparent from the discussion above, wedgelet-based partitioning represents a kind of tradeoff between the auxiliary information rate on the one hand and the achievable diversity in the splitting potential on the other. In comparison, contour segmentation seems more complicated in terms of auxiliary information rate.

The idea, based on an embodiment described further below, is that the ability to relax the constraints of a split to the extent that it must be a wedgelet split is relatively uncomplicated to derive good predictions for double segmentation in depth/disparity maps. Statistical analysis can be applied to overlapping spatially sampled texture information. Thus, according to this idea, it increases the freedom to mitigate the signaling overhead precisely in the case where co-located texture information in the form of a picture is provided, and a meaningful texture change is visible here. Possible implementations of the present idea using this idea were detailed in Section 3.2.2, but are described in more detail below in a more general perspective. In other words, all details set forth in Section 3.2.2 and in other Sections are to be understood as possible implementation details that may be combined with the descriptions presented individually below.

In particular, Fig. 22 shows a decoder 600 according to this embodiment of the present invention. The decoder (600) is for reconstructing a predetermined block of the depth/disparity map (213) associated with the picture (215) from the data stream (304). The decoder includes a segmenter 602, a spatial transporter 604 and a decoder 606. Decoder 600 may be configured as described above with respect to any one of decoding branches 106 _{d, 1/2} . That is, the decoder 600 may operate based on blocks. Moreover, it can be implemented as a hybrid video decoder. The subdivision of the depth/disparity map 213 into blocks may derive entirely from, or may deviate from, the subdivision of the pictures 215 into blocks, the subdivision of the depth/disparity map signaling within the data stream 304. may be, or may otherwise be known to the decoder (600).

The segmenter 602 divides the reference block 216 of the picture 215 by thresholding the picture 215 within the reference block 216 to obtain a double segmentation of the reference block 216 into first and second splits. and configured to be co-located in a predetermined block 210 of the depth/disparity map 213.

The spatial transporter 604 then references the picture into a predetermined block 210 of the depth/disparity map 213 to obtain the first and second divisions 202a and 202b of the predetermined block 210. Carry out the double segmentation of block 216.

The decoder 606 is configured to decode the predetermined block 210 in units of first and second partitions 202a and 202b. The function of decoder 606 corresponds to the function described above with respect to boxes 310, 412 and 412'.

1-3, the segmenter and transporter functions may be included within exchange modules 52 and 110, respectively, but the functions of decoder 606 may be implemented in an intra prediction module, for example.

픽처(215)를 재구성하도록 구성될 수 있다.First and second partitions 202a and 202b of reference block 216 of picture 215, as described above with respect to section 3.2.2, where the description may individually indicate possible implementation details for the elements of FIG. ) is a set of tiles 608 that completely cover the reference block 216 of the picture 215 and are complementary to each other, so that, when thresholding, each value has a respective pre- It may be configured to individually check the values of the pictures 215 in the reference block 216 in the two-dimensional subdivision 608 of the reference block 216 for greater or less than the determined value. That is, thresholding may be performed at sample resolution, in which case tiles 608 correspond to individual samples 610 of picture 215 . The decoder 600 may also be responsible for reconstructing a picture 215 whose values are subject to individual checks in thresholding are the reconstructed values of the reconstructed picture 215 . In particular, as described with respect to FIGS. 1-3, the decoder 600 picks prior to its associated depth/disparity map 213.

may be configured to reconstruct the picture 215 .

As already mentioned above, to obtain a double segmentation of the reference block 216 into first and second divisions, the segmenter 602, when segmenting, performs morphological hole filling and/or low pass It can be configured to apply pass-through filtering to the results of thresholding. This avoids the occurrence of many isolated two segments of the segmentation of the double segmentation obtained from the reference block 216 that is spatially transferred by the spatial transferor 604, but such rapid depth changes are much less likely to occur visually. Naturally, the encoder does this.

Moreover, the decoder 600 and the segmenter 602 determine, in thresholding, a measure of the central tendency of the reconstructed sample values of the reference block 216 of the picture 215, and the picture ( 215) to perform thresholding by comparing each reconstructed sample value of the reference block 216 with a respective threshold value dependent on the determined measurement. As discussed above, thresholds may be defined between samples 610 within reference block 216 as a whole. As a central tendency, some mean value such as an arithmetic mean or a median value may be used.

As described above in section 4.2, the decoder 600 uses block 602 only if the a-priori determined variance of the values of the samples in reference block 216 of picture 215 exceeds a predetermined threshold. to 606) may be configured to support the availability of coding modes indicated by. Otherwise, the double segmentation found by thresholding will most likely not form a good predictor for the appearance of block 210 of the depth/disparity map, and thus this coding mode will not allow such a block. can By suppressing the modal possibilities, an unprofitable and unnecessary increase in the number of symbol values of each coding option identifier for which the entropy probability estimate has to be considered is avoided.

According to Fig. 22, the following information: 1) coding option identifiers with respective pre-determined states triggering steps 602-604, 2) optionally, dual segmentation such as threshold post filtering/hole filling directivities, etc. 3) Optionally, coding such as CPV or CPV residuals for one or both partitions 202a, 202b, since they may be allowed to be spatially or temporally predicted from neighboring samples/blocks. Parameter, 4) Optionally, a coding parameter residual, such as DeltaCPV, may be provided in the data stream for block 210.

All further variations mentioned above with respect to the embodiment of FIGS. 16 , 18 and 20 are also applicable to the embodiment of FIG. 22 . 22 as part of the reference in the predictive loop of the decoder 600, and the possibility of combining the coding mode of FIG. This is especially true for the use of prediction results.

FIG. 23 shows a possible configuration for an encoder that fits the decoder of FIG. 22 . The encoder of FIG. 23 is indicated generally by reference numeral 630 and is configured to encode a predetermined block of a depth/disparity map associated with a picture into a data stream 304 . The encoder includes a segmenter 632 and a spatial transferor 634 that operate like components 602 and 604 of FIG. 22 operating on an internally reconstructed version of a previously encoded portion of the data stream 304. . Encoder 636 encodes a predetermined block in units of generated divisions.

5.5 Dependence of availability of double segmentation transfer from picture to depth/disparity map on distribution of sample values within a picture's reference block

The idea underlying the embodiment described below, that is, the derivation of double segmentation based on co-located reference blocks in the picture with subsequent transfer of double segmentation into the current block of the depth/disparity map, is the current block of the depth/disparity map. The idea that it is valid only if the probability of achieving a good approximation of the contents of s is high enough to justify the reservation of each predetermined value of the corresponding coding option identifier to trigger this dual segmentation transfer mode was already specified in section 4.2. . In other words, avoiding the need to consider each predetermined value of the coding option identifier for the current block of the depth/disparity map when entropy coding these coding option identifiers in the case where each double segmentation transport is highly unlikely to be selected anyway. By doing so, the auxiliary information rate can be saved.

픽처(215)의 텍스처 특징에 따라 픽처(215)의 참조 블록(216)을 세그먼트하도록 단순히 구성된다. 예를 들면, 이러한 수정된 실시예에 따라, 세그먼터(602)는 웨지렛 분리 라인의 가능한 확장 방향을 검출하기 위해 에지 검출을 이용하여, 이와 같이 위치된 라인을 공간 이송기(604)에 의해 블록(216)에서 공간적으로 깊이/변이 블록(210)으로 이송할 수 있다. 세그먼터(602)에 의한 다른 가능한 이중 세그먼테이션이 또한 실현 가능하다.Accordingly, according to a modified embodiment of the decoder 600 of FIG. 22, since the decoder is generally constructed similarly to the decoder of FIG. Let the reference be done. However, the segmenter 602 is not limited to be configured to segment the reference block 216 with a contour segment rather than a Wedgelet segment. Rather, segmenter 602 selects a pick in reference block 216 to obtain a double segmentation of the reference block into first and second splits.

It is simply configured to segment the reference block 216 of the picture 215 according to the texture characteristics of the picture 215 . For example, according to this modified embodiment, the segmenter 602 uses edge detection to detect the possible direction of extension of the wedgelet separation line, and places the line thus positioned by the spatial transferor 604. From block 216 we can transfer spatially to depth/disparity block 210. Other possible double segmentation by segmenter 602 is also feasible.

However, in addition to that, the decoder 600, according to the present embodiment, segmentation by the segmenter 602, spatial transfer by the spatial transferor 604, and decoding the second set of coding options of the decoder 600 and form one of a first set of coding options of a decoder (600) that is not part of , the decoder determines the variance of the values of the samples in the reference block (216) of the picture (215), and the data stream (304). ), and if the index in the boxes 602 to 606 points to one coding option, the variance is determined by performing segmentation, spatial transfer, and decoding on the predetermined block 210 to a predetermined threshold. and use the coding option identifier as an index into a first set of coding options if the value exceeds the value, and an index into a second set of coding options if the variance inherits the predetermined threshold. Accordingly, signaling overhead for signaling a coding option identifier can be saved. As the variance, mean absolute difference, standard deviation or variance can be used.

For a further modification of the just-mentioned modified embodiment of Fig. 22, reference is made to the descriptions of Sections 5.4 and 4.2.

The corresponding encoder can be derived from the encoder in FIG. 23 .

5.6. Effective prediction by double splitting using prediction of one or two constant split values from neighboring samples

As already described above in connection with the various embodiments described so far, the method of predicting the current block by assigning a constant split value to a split of a double split of a block is very useful, especially in the case of coding sample arrays such as depth/disparity maps. Effectively, the content of these sample arrays consists mostly of plateaus or simply connected regions of similar value separated from each other by sharp edges. Nonetheless, even the transmission of these constant partition values requires a significant amount of ancillary information that must be avoided.

The idea underlying the embodiment described further below is that this auxiliary information rate can be reduced if the average value of the values of the samples associated with each split or adjacent neighboring samples is used as a predictor for a given split value. The inventors have found that this coding mode for the blocks of the sample array can even leave signaling of the refinement of each constant division value.

24 shows a decoder 700 for reconstructing a sample array 302 from a data stream 304. The decoder 700 can be configured to reconstruct the sample array 302 using block-based decoding and to use hybrid decoding. In general, all possible implementations described above in Sections 5.1 to 5.5 also apply to the decoder 700 of FIG. 24 . Naturally, all possible implementations for splitting the current block 210 into two partitions represent alternative alternatives to the decoder of FIG. 700 that can only be implemented differently in this respect as well.

In particular, decoder 700 is configured to perform different tasks or functions to derive a prediction of current block 210 . In particular, the decoder 700 divides a predetermined block 210 of the sample array 302 into a dual division into a first division exemplified by hatched samples, and a second division exemplified by unhatched samples. is configured to perform derivation 702 of Moreover, the decoder 700 determines each of the first and second partitions such that each neighboring sample is adjacent to the partition to which it relates, and each association of neighboring samples of the sample array 302 adjacent to the predetermined block 210 ( 704). In FIG. 24 , neighboring samples that are the subject of association 704 are illustrated with two different kinds of hatching: dotted hatching and cross hatching. Dot hatching indicates samples adjacent to samples of block 210 belonging to one block of block 210, while cross hatching indicates samples adjacent to samples of block 210 belonging to another block. As described above with respect to Sections 5.1 to 5.4, in order to achieve a high probability of availability of a neighboring sample of block 303 of sample array 302 that has already been reconstructed by decoder 700, decoder 700 uses sample Any suitable coding/decoding order among the blocks 303 of the array 302 may be used.

Of course, it may happen that an available neighboring sample, i.e., a neighboring sample of block 210 located within already reconstructed block 303 of sample array 302, simply connects to one of the partitions of block 210. . In that case, data stream 304 may explicitly transmit a constant partition value for each other partition in which none of the neighboring samples are contiguous. Alternatively, some other fallback procedure may be performed by the decoder 700 in that case. For example, decoder 700 may, in that case, convert this missing constant split value to a predetermined value, or to a previously reconstructed sample array of sample array 302 and/or some other previously reconstructed sample array. Among the values, it can be set to a value determined from a long-term mean.

Finally, in prediction 706, the decoder 700 assigns the average value of the values of neighboring samples associated with the first partition to the samples in the sample array located within the first partition and/or the neighboring samples associated with the second partition. A predetermined block 210 is predicted by assigning the average value of the values of the samples to the samples of the sample array located in the second partition.

The decoder 700 uses the refinement information in the data stream, i.e., applies a first refinement value in the refinement information to an average value of values of neighboring samples associated with the first partition, and/or It may be configured to refine the prediction of the predetermined block 210 by applying a second refinement value to an average value of values of neighboring samples associated with the second partition. In this regard, the decoder 700, when applying the first and/or second refinement values, respectively, averages the values of the neighboring samples associated with the first partition and/or the neighboring samples associated with the second partition. It may be further configured to combine linearly, such as by adding the average value of the values of the samples and the first and/or second refinement values. The decoder (700), upon applying the first and/or second refinement values, retrieves the first and/or second refinement values from the data stream, and a predetermined spatially sampled configuration associated with the sample array. It may be configured to scale the first and/or second refinement values when the elements, ie texture and/or depth/map, are retrieved using the quantization step size according to the reference quantization step size transmitted in the data stream. The sample array can be, for example, a depth map, but the reference quantization step size can be used by the decoder 700 to reconstruct a texture sample array from the bitstream to which the depth map is associated. Moreover, further reference is made to the respective parts in section 3.3.2 for further details.

The decoder, when deriving the double division of the predetermined block of the sample array into the first and second divisions, the Wedgelet separation line at the predicted position is an extension of the Wedgelet separation line of the neighboring block into the predetermined block and predict a position of a Wedgelet separation line in a predetermined block of the sample array according to a Wedgelet separation line of a neighboring block of the predetermined block to form . The decoder is further configured to use the refinement information in the data stream to refine a predicted position of the wedgelet separation line, wherein the wedgelet separation line of the predetermined block divides the predetermined block into first and second divisions.

As mentioned above, decoder 700 can do double segmentation using some of the ideas described in Sections 5.1-5.5. By filling the reference block by copying the reconstructed values of samples of the sample array adjacent to the first block along the intra-prediction direction to the reference block, the decoder 700 obtains samples adjacent to the predetermined block 210 using intra prediction. It can be configured to predict the reference block of array 302 . When deriving the dual division of the predetermined block of the sample array into the first and second divisions, the decoder 700 sets the extension direction of the wedgelet separation line within the predetermined block according to the intra prediction direction to obtain the predetermined block. The position of the wedgelet separation line within 210 can be predicted, which divides the predetermined block into first and second divisions.

Alternatively, obtain a double segmentation of the reference block into a first predetermined division, and spatially perform the double segmentation of the reference block of the picture onto a predetermined block of the depth/disparity map to obtain the first and second divisions. By thresholding the pictures within the reference block to transfer, the decoder 700 can be configured to segment the reference block of the co-located picture into a predetermined block 210 if the sample array 302 is a depth/disparity map associated with the picture. have.

The decoder may be further configured to use the predefined block as a reference for the decoder's prediction loop.

FIG. 25 shows a possible embodiment for an encoder 730 that fits the decoder 700 of FIG. 24 . An encoder (730) for encoding the sample array into a data stream derives (732) a dual partition of a predetermined block of the sample array into first and second partitions, each such that each neighboring sample is adjacent to the associated partition. and associate 734 each of the neighboring samples of the array of samples adjacent to the predetermined block with the first and second divisions of . The encoder assigns the average value of the values of neighboring samples associated with the first partition to the samples in the sample array located within the first partition, and assigns the average value of the values of neighboring samples associated with the second partition to the sample array located within the second partition. It is further configured to predict 736 the predetermined block by assigning to samples of .

Although some aspects have been described in relation to apparatus, it is self-evident that these aspects also represent a description of a corresponding method, wherein a block or apparatus corresponds to a method step or feature of a method step. Similarly, an aspect described in connection with a method step also represents a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the method steps may be executed by (or using) a hardware device, such as, for example, a microprocessor, programmable computer, or electronic circuitry. In some embodiments, some one or more of the most important method steps may be performed by such an apparatus.

Depending on certain implementation requirements, embodiments of the present invention may be implemented in hardware or software. Such an implementation may be performed using a digital storage medium, for example a floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM or FLASH memory, which medium has electronically readable control signals stored therein. and these signals cooperate (or may cooperate) with a programmable computer system to perform each method. Accordingly, a digital storage medium may be computer readable.

Some embodiments according to the present invention include a data carrier having electronically readable control signals capable of cooperating with a programmable computer system to perform one of the methods described herein.

Generally, an embodiment of the invention may be implemented as a computer program product having program code, which program code operates to perform one of the methods when the computer program product is executed on a computer. The program code may be stored, for example, on a machine readable carrier.

Another embodiment includes a computer program for performing one of the methods described herein and stored on a machine readable carrier.

So, in other words, an embodiment of the method of the present invention is a computer program having program code to perform one of the methods described herein when the computer program runs on a computer.

Thus, a further embodiment of the method of the present invention is a data carrier (or digital storage medium, or computer readable medium) recorded on the data carrier and containing a computer program for performing one of the methods described herein. Data carriers, digital storage media or recorded media are typically tangible and/or non-transitionary.

Thus, a further embodiment of the present invention is a data stream, or sequence of signals representing a computer program for performing one of the methods described herein. A data stream or sequence of signals may be configured to be transported over a data communication connection, for example via the Internet, for example.

Additional embodiments include processing means, for example a computer configured or adapted to perform one of the methods described herein, or a programmable logic device.

A further embodiment includes a computer having installed a computer program for performing one of the methods described herein.

A further embodiment according to the present invention includes an apparatus or system configured to transfer (e.g., electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, mobile device, memory device, or the like. The device or system may include, for example, a file server for transferring computer programs to receivers.

In some embodiments, a programmable logic device (eg, a field programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. Generally, the method is preferably performed by some hardware device.

The foregoing embodiments are merely illustrative of the principles of the present invention. It is understood that modifications and variations of the arrangements and details described herein will be readily apparent to those skilled in the art. Thus, it is intended not to be limited by the specific details presented by the description of the embodiments herein, but only by the appended claims.

Claims (24)

A decoder for reconstructing a predetermined block of a depth / disparity map associated with a texture picture from a data stream,
determining a texture threshold based on a sample value of a reference block of the texture picture, wherein the reference block is co-located with the predetermined block;
Segmenting the reference block of the texture picture by thresholding the texture picture in the reference block using the texture threshold to obtain a double segmentation of the reference block into first and second divisions;
spatially transferring the double segmentation of the reference block of the texture picture onto the predetermined block of the depth/disparity map to obtain first and second divisions of the predetermined block;
A decoder configured to decode the predetermined block in units of the first and second divisions. According to claim 1,
When thresholding such that each of the first and second partitions of the reference block of the texture picture is a set of tiles that completely cover the reference block of the texture picture from each other and are complementary to each other, each value is set in advance. A decoder configured to individually check the value of the texture picture in the reference block in the tile of the two-dimensional subdivision of the reference block for greater or less than a predetermined value. According to claim 2,
When thresholding, decoder configured to individually check the value of the texture picture in the reference block at a sample resolution such that each tile corresponds to a sample position of the reference block. According to claim 1,
and apply morphological hole filling and/or low pass filtering to a result of the thresholding when segmenting to obtain the double segmentation of the reference block into the first and second divisions. According to claim 1,
In thresholding, determining a measure of the centroid tendency of reconstructed sample values of the reference block of the texture picture, and assigning each reconstructed sample value of the reference block of the texture picture to each threshold depending on the determined measure A decoder configured to perform the thresholding by comparing with a value. According to claim 1,
wherein the segmentation, spatial transfer and prediction form one of a first set of coding options that is not part of a second set of coding options, and the decoder is further configured to determine a variance of values of samples in the reference block of the texture picture. configured-,
Receive a coding option identifier from the data stream, and
The index into the first set of coding options if the variance exceeds a predetermined threshold by performing the segmentation, spatial transfer and prediction onto the predetermined block if the index points to one coding option. , and use the coding option identifier as the index into the second set of coding options if the variance inherits the predetermined threshold. A decoder for reconstructing a predetermined block of a depth / disparity map associated with a texture picture from a data stream,
determining a texture threshold based on a sample value of a reference block of the texture picture, wherein the reference block is co-located with the predetermined block;
according to a texture feature of the texture picture in the reference block using edge detection, or by thresholding the texture picture using the texture threshold to obtain a double segmentation of the reference block into first and second divisions. , segment the reference block of the texture picture,
spatially transferring the double segmentation of the reference block of the texture picture onto the predetermined block of the depth/disparity map to obtain first and second divisions of the predetermined block;
Decode the predetermined block in units of the first and second partitions, and - the decoder determines a first set of coding options of the decoder for which the segmentation, spatial transfer and decoding are not part of a second set of coding options of the decoder. 1 set, wherein the decoder is further configured to determine a variance of values of samples in the reference block of the texture picture-
Retrieve a coding option identifier from the data stream;
The index into the first set of coding options if the variance exceeds a predetermined threshold by performing the segmentation, spatial transfer and decoding onto the predetermined block if the index points to one coding option. , and use the coding option identifier as an index into the second set of coding options if the variance inherits the predetermined threshold. According to claim 7,
wherein the decoder is configured to retrieve the coding option identifier by entropy decoding. According to claim 1,
The decoder is further configured to use the predetermined block as a reference for a prediction loop of the decoder. According to claim 7,
The decoder is further configured to use the predetermined block as a reference for a prediction loop of the decoder. According to claim 1,
The decoder, when decoding the predetermined block,
A decoder configured to predict the predetermined block by assigning a first constant division value to samples located within the first division and a second constant division value to samples located within the second division. According to claim 7,
The decoder, when decoding the predetermined block,
A decoder configured to predict the predetermined block by assigning a first constant division value to samples located within the first division and a second constant division value to samples located within the second division. According to claim 12,
The decoder, in predicting the predetermined block, associates each of the neighboring samples adjacent to the predetermined block with each one of the first and second partitions such that each neighboring sample is adjacent to the partition to which it relates. assigning an average value of the values of the neighboring samples associated with the first partition to samples located in the first partition, and assigning an average value of values of the neighboring samples associated with the second partition to a sample located in the second partition A decoder configured to predict the predetermined block by assigning to the selected sample. According to claim 11,
The decoder applies a first refinement value in the refinement information to an average value of values of neighboring samples associated with the first division, and/or applies a second refinement value in the refinement information to an average value of values of neighboring samples associated with the second division. A decoder configured to refine the prediction of the predetermined block by applying an average value of values of neighboring samples. 15. The method of claim 14,
When applying the first and/or second refinement values, the average value of the values of the neighboring samples associated with the first division, respectively, and/or the values of the neighboring samples associated with the second division, respectively. A decoder configured to linearly combine the average value with the first and/or second refinement value. 15. The method of claim 14,
The depth/disparity map is a depth map, and the decoder,
When applying the first and/or second refinement values,
scaling the first and/or second refinement values using a quantization step size according to a reference quantization step size to which a predetermined spatially sampled component associated with the depth map is transmitted; A decoder configured to use the reference quantization step size to reconstruct an array of texture samples from a data stream. An encoder for encoding a predetermined block of a depth/disparity map associated with a texture picture into a data stream,
determining a texture threshold based on a sample value of a reference block of the texture picture, wherein the reference block is co-located with the predetermined block;
Segmenting the reference block of the texture picture by thresholding the texture picture in the reference block using the texture threshold to obtain a double segmentation of the reference block into first and second divisions;
spatially transferring the double segmentation of the reference block of the texture picture onto the predetermined block of the depth/disparity map to obtain first and second divisions of the predetermined block;
An encoder configured to encode the predetermined block in units of the first and second divisions. An encoder for encoding a predetermined block of a depth/disparity map associated with a texture picture into a data stream,
determining a texture threshold based on a sample value of a reference block of the texture picture, wherein the reference block is co-located with the predetermined block;
according to a texture feature of the texture picture in the reference block using edge detection, or by thresholding the texture picture using the texture threshold to obtain a double segmentation of the reference block into first and second divisions. , segment the reference block of the texture picture,
spatially transferring the double segmentation of the reference block of the texture picture onto the predetermined block of the depth/disparity map to obtain first and second divisions of the predetermined block;
encodes the predetermined block in units of the first and second partitions, and the encoder determines a first set of coding options of the encoder in which the segmentation, spatial transfer and encoding are not part of a second set of coding options of the encoder. configured to form one of a set, wherein the encoder is further configured to determine a variance of values of samples in the reference block of the texture picture;
encodes a coding option identifier into a data stream;
The index into the first set of coding options if the variance exceeds a predetermined threshold by performing the segmentation, spatial transfer and encoding onto the predetermined block if the index points to one coding option. , and use the coding option identifier as the index into the second set of coding options if the variance inherits the predetermined threshold. A method for reconstructing a predetermined block of a depth/disparity map associated with a texture picture from a data stream, comprising:
determining a texture threshold based on a sample value of a reference block of the texture picture, wherein the reference block is co-located with the predetermined block;
thresholding the texture picture within the reference block using the texture threshold to obtain a double segmentation of the reference block into first and second divisions, thereby segmenting the reference block of the texture picture;
spatially transferring the double segmentation of the reference block of the texture picture onto the predetermined block of the depth/disparity map to obtain first and second divisions of the predetermined block; and
and decoding the predetermined block in units of the first and second divisions. A method for reconstructing a predetermined block of a depth/disparity map associated with a texture picture from a data stream, comprising:
determining a texture threshold based on a sample value of a reference block of the texture picture, wherein the reference block is co-located with the predetermined block;
according to a texture feature of the texture picture in the reference block using edge detection, or by thresholding the texture picture using the texture threshold to obtain a double segmentation of the reference block into first and second divisions. , segmenting the reference block of the texture picture;
spatially transferring the double segmentation of the reference block of the texture picture onto the predetermined block of the depth/disparity map to obtain first and second divisions of the predetermined block;
decoding the predetermined block in units of the first and second partitions, the method comprising: one of a first set of coding options wherein the segmentation, spatial transfer and decoding are not part of a second set of coding options of the method; one, the method further comprising determining a variance of values of samples in the reference block of the texture picture;
Retrieving a coding option identifier from the data stream;
The index into the first set of coding options if the variance exceeds a predetermined threshold by performing the segmentation, spatial transfer and decoding onto the predetermined block when the index points to one coding option; and using the coding option identifier as the index into the second set of coding options if the variance inherits the predetermined threshold. A method for encoding a predetermined block of a depth/disparity map associated with a texture picture into a data stream,
determining a texture threshold value based on a sample value of a reference block of the texture picture, wherein the reference block is co-located with the predetermined block;
Segmenting the reference block of the texture picture by thresholding the texture picture within the reference block using the texture threshold to obtain a double segmentation of the reference block into first and second divisions;
spatially transferring the double segmentation of the reference block of the texture picture onto the predetermined block of the depth/disparity map to obtain first and second divisions of the predetermined block; and
and encoding the predetermined block in units of the first and second divisions. A method for encoding a predetermined block of a depth/disparity map associated with a texture picture into a data stream,
determining a texture threshold based on a sample value of a reference block of the texture picture, wherein the reference block is co-located with the predetermined block;
according to a texture feature of the texture picture in the reference block using edge detection, or by thresholding the texture picture using the texture threshold to obtain a double segmentation of the reference block into first and second divisions. , segmenting the reference block of the texture picture;
spatially transferring the double segmentation of the reference block of the texture picture onto the predetermined block of the depth/disparity map to obtain first and second divisions of the predetermined block;
encoding the predetermined block in units of the first and second partitions - the method comprising the first of the coding options of the method wherein the segmentation, spatial transfer and encoding are not part of a second set of coding options of the method; 1 set, wherein the method further comprises determining a variance of values of samples in the reference block of the texture picture;
encoding a coding option identifier into the data stream; and
The index into the first set of coding options if the variance exceeds a predetermined threshold by performing the segmentation, spatial transfer and encoding onto the predetermined block if the index points to one coding option. , and using the coding option identifier as the index into the second set of coding options if the variance inherits the predetermined threshold. A non-transitory computer-readable storage medium comprising a computer program having program code for performing the method according to claim 19 when run on a computer. A non-transitory computer-readable storage medium comprising a computer program having program code for performing the method according to claim 22 when run on a computer.

Images